Fork me on GitHub
#tools-deps
<
2018-10-10
>
dominicm05:10:30

@kenny the issue is because of ecdsa I believe.

henryw37408:10:02

It's a bit frustrating to have to wait so long for Windows support. https://dev.clojure.org/jira/browse/TDEPS-67 has been around for a while... Please add your vote! (not sure if it'll make a difference but it can't hurt). Thanks

dominicm15:10:23

We're developing a situation where a few aliases have been split up in the name of DRY. This means that to start the project in "dev mode" requires a difficult number of aliases. Has anyone tried tackling this yet? I'm currently thinking of writing something that merges together aliases so you can create a "super-alias" that's easier to maintain.

Andreas Liljeqvist09:10:56

We use makefiles to handle starting tricky combos

dominicm09:10:00

That doesn't go very well with cider-jack-in though, or any other editor integration?

Andreas Liljeqvist09:10:04

You are right, we use it more for building releases and such

Andreas Liljeqvist09:10:15

Don't have a good solution for launching dev

dominicm15:10:31

This is really just a workaround for the lack of ability for aliases to depend on one another.

seancorfield15:10:14

@dominicm Yeah, we're running into that a bit as we convert our Boot-powered monorepo to clj/`deps.edn`...

zane16:10:37

I'd love to hear more about this if you'd like to share! Why you're doing it, etc.

seancorfield17:10:48

@U050CT4HR We've had our dependencies in external EDN files for ages (years, maybe) and if fact called them deps.edn until clj/`tools.deps` appeared 🙂 I'm running into problems with Boot due to the fileset and pod stuff (I think because our project is large -- 83K lines across 400+ files -- but we haven't managed to tie down a repro case yet). Several of our Boot tasks take a long time to fire up and run, whereas running similar tasks via clj is much faster (since there's a lot less machinery involved).

seancorfield17:10:30

I'm also a big fan of keeping close to the bleeding edge with Clojure releases and "core" tooling so having fewer dependencies in our system would make me happy.

zane20:10:12

That's really helpful Sean. Thanks so much!

dominicm15:10:48

@seancorfield have you had any particular ideas on how to tackle it? I'm quite happy to open source my solution. It will likely be a key piece in edge.

Alex Miller (Clojure team)15:10:14

have you considered just repeating yourself? :)

dominicm15:10:37

@alexmiller In some cases, yep! For a single dependency it's usually not so bad. When it's 2 or 3 aliases with 2 or 3 changes, the maintenance burden of bumping in multiple places is failing quite often. I'll admit that I'm trying to add in more "easy" onto this design than usual, as I can't impose pull request standards onto all projects.

seancorfield15:10:20

We've had to repeat a chunk of our testing context in two separate aliases, which is a bit annoying. The alternative is a longer string of aliases. Right now we have a dummy project containing a deps.edn with mostly aliases. That has a :default aliases which has all our :override-deps where we "pin" versions of libraries. Then we have a deps.edn in each project that specifies the deps for that project (mostly as some/lib {} since the actual version comes from the :default alias) and references to other subprojects as worldsingles/something {:local/root "../something"}. This works pretty well for working within any given subproject and then we'll likely have a couple of shell scripts for running tests across the whole suite of projects.

seancorfield15:10:43

Doing this has actually highlighted some problematic cross-project dependencies that Boot sort of hid for us, so that's been nice. And of course being able to use a much more lightweight process is always nice 🙂

dominicm15:10:56

heh, I actually just caught myself this second failing to update one of two places. I only caught it because I realized that a :main-opts looked out of place in the alias I had updated.

seancorfield15:10:15

One thing I do find is that I'd desperately love to be able to parameterize deps.edn sometimes. For example, we can have an alias that starts an nREPL server to work with a project in an editor, but we can't affect which port it starts on -- that has to be hardcoded in the alias. So I'm thinking we may end up with one or more aliases that bring in (and run) certain small Clojure scripts that exist purely to support dev/test work. That way we could do clj -A:some:stuff:here <port-number> and run a -main in a dev namespace that starts nREPL on that port.

seancorfield15:10:42

I'd be interested to hear more about how Cognitect folks deal with this in large projects. hint, hint @alexmiller 🙂

Alex Miller (Clojure team)15:10:55

don’t make large projects?

seancorfield15:10:43

Is 63K lines of production Clojure and 20K lines of tests "large" tho'?

Alex Miller (Clojure team)15:10:13

deps.edn being “just data” (and not having parameters, references, etc) has vast benefits for “simple” and is not something I will give up easily

Alex Miller (Clojure team)15:10:33

at some point of complexity, you are really doing programming here

seancorfield15:10:46

Oh, true. I totally get the benefits. I just haven't (yet) found the ideal balance yet, with the new tooling.

Alex Miller (Clojure team)15:10:56

you can use clj to run programs - so pushing it into a script (which can take args) is one path

dominicm15:10:59

@seancorfield I'm in favor of a main for the nREPL situation.

Alex Miller (Clojure team)15:10:14

you can also wrap clj in another program (probably a shell script)

Alex Miller (Clojure team)15:10:22

but could also be, say, make

Alex Miller (Clojure team)15:10:37

make has the nice capability to define dependencies on smaller pieces which I think often is useful in build situations where you are making dependent subtasks

seancorfield15:10:46

Being able to do something like this to run all subprojects tests is convenient enough:

for d in * ; do (cd $d; CLJ_CONFIG=../versions clj -A:defaults:test:runner); done

seancorfield15:10:12

We're trying hard to get away from Ant. All that XML shudder

Alex Miller (Clojure team)15:10:46

I don’t think running a standard “program” per project is any different though

seancorfield16:10:51

Yeah, hence why I'm coming around to having a "dev" project with short Clojure scripts with -main entry points. And, to be honest, that's not hugely different from having tasks in a build.boot file which are all just functions.

dominicm16:10:38

@seancorfield would the dev project be a dependency of all projects?

seancorfield16:10:59

Only under an alias for dev/test tooling -- it wouldn't be a regular dependency.

dominicm16:10:58

sorry, yeah.

dominicm18:10:50

@seancorfield once add-lib is stable, it might be a good idea to dynamically fetch dependencies based on the task(s) being run. e.g. nrepl will grab cider/tools.nrepl on demand. That would probably reduce the number of aliases way down. Basically making most of the project static (deps.edn) but providing a dynamism for the non-simple stuff.

sparkofreason15:10:53

What's add-lib? Sounds intriguing.

dominicm19:10:23

It's a function in a branch of tools.deps

seancorfield18:10:56

I dream of a world where tooling bootstraps itself on top of prepl instead of needing server-side dependencies 🙂

seancorfield18:10:06

(I love unravel for that reason)

dominicm18:10:14

socket repl already does that, but I agree 🙂. But I think the same applies (for me) for loading in cljs to do a one-time build.

seancorfield18:10:52

Well, specifically the tooling. I'd like to see a version of CIDER that could bootstrap itself over a socket REPL or prepl. So you didn't need all the CIDER deps on the project side of the equation. Same with ProtoREPL for Atom.

🙏 8
âž• 4
seancorfield18:10:27

What I like about unravel is that I can spin up a bare Socket REPL (e.g., on a production JAR, via JVM options) and then connect via unravel and when I (require 'compliment.core) it sends it over the wire from the REPL client so I don't need it in my project. Then I have a nice interactive REPL with history, documentation hinting, completion...

seancorfield19:10:22

Your project then becomes tooling neutral. You don't need any dependencies in it to support tooling. And then you can use any tooling with it. I know CIDER can inject itself into a newly-started REPL process, but that's a heavyweight server containing all of those dependencies. I basically want CIDER-like functionality over a bare Socket REPL started from a JAR that doesn't have those dependencies 🙂

dominicm19:10:04

I just discovered I can use find-deps as a filter! Nice. This means I can do !aE in vim, followed by my find-deps command to auto-update the buffer I'm working on. Cool!

dominicm19:10:03

@alexmiller have you had any thoughts about comments being obliterated by -Sresolve-tags?

dominicm19:10:18

I should open a jira if there isn't one

Alex Miller (Clojure team)19:10:58

reader doesn’t preserve comments

Alex Miller (Clojure team)19:10:28

I’m not planning on doing any more work on resolve-tags. most likely will actually extract it all out into a separate project and remove it entirely

Alex Miller (Clojure team)19:10:42

as an external program you could invoke it from clj if needed

dominicm19:10:21

If it's an external program that shifts expectations I think, cool. I would be more inclined to reimplement it with rewrite-clj so I could preserve comments/formatting.

dominicm19:10:31

I've been thinking about whether a wrapper over rewrite-clj that implements IAssoc is feasible.

Alex Miller (Clojure team)20:10:19

Well feel free to rewrite it and make it do whatever :)

arrdem20:10:45

@seancorfield have you seen https://github.com/Unrepl/unrepl? It does pretty much that self-bootstrapping dance.

seancorfield22:10:20

@arrdem yes I use that with socket server REPLs a lot.

seancorfield22:10:28

It can bootstrap compliment into a running REPL to give you auto-completion, which I like.

andy.fingerhut22:10:35

I guess bootstrap here just means "send the necessary text across the socket to define functions/etc. to create the necessary code on the server?"

seancorfield22:10:17

Yeah. In this case, "send the Compliment library across the socket..."