Fork me on GitHub
#tools-deps
<
2021-08-10
>
Alex Miller (Clojure team)00:08:35

there is a new Clojure CLI prerelease out that you could try if you like: • TDEPS-199 Use default http-client in S3 transporter • Cache S3 transporter for a repo • Fixed session cache to work properly across threads / binding stacks for better perf • Replace specific maven version range requests with non-range request to reduce repo metadata lookups

🎉 5
Alex Miller (Clojure team)00:08:22

Mainly, it should be faster to make classpaths. It may also improve some of the Maven concurrency issues as a side effect. I wasn’t working on that explicitly, but may be improved. So if you are using -Sthreads 1 on a ci server to avoid that, I’d love to hear if it helps

Alex Miller (Clojure team)02:08:59

I guess I forgot the version 1.10.3.939

Alex Miller (Clojure team)20:08:34

new Clojure CLI prerelease with some additional performance enhancements: 1.10.3.943 • Load and cache Maven settings once for perf • Cache version range resolution results for perf

Alex Miller (Clojure team)20:08:34

in the original case I've been working on the total download time has gone from ~58 sec to 7 sec - this is just one use case and is pretty heavily impacted by what transitive deps you have, whether version ranges are in use, whether you're using an s3 repo, etc. and of course, if you have a cached classpath, nothing changing here is being run, so caveat emptor

seancorfield20:08:07

Sounds like it might benefit CI setups quite a bit @alexmiller?

Alex Miller (Clojure team)20:08:58

would appreciate anyone kicking tires (and turning off -Sthreads 1 if you're using that - I believe I have probably mitigated a lot of the maven concurrency issues we were seeing so this should not be needed and will of course be faster). have not worked on the git download concurrency issues, so you may still need it for that.

Alex Miller (Clojure team)20:08:47

if you do find a reproducible (-ish) case for maven concurrency issues, would love to have a deps.edn that demonstrates that

seancorfield20:08:15

I opened a ticket at work to vendor in the .943 prerelease. Should get to that tomorrow and run some local tests then see how it does on our CI setup.

seancorfield16:08:28

@alexmiller So far, so good. Definitely seems faster to download everything.

👍 2
deactivateduser21:08:48

Sorry if this has been answered (my search-fu is weak), but if I want to use t.d.a. programmatically (as a library), specifically with add-libs support, is the add-lib3 branch on GitHub still my best source? I notice it’s 5 months / 113 commits behind master, hence the question.

Alex Miller (Clojure team)21:08:41

yeah, it hasn't been merged recently. not sure it will work with Clojure CLI as is

Alex Miller (Clojure team)21:08:10

I just merged master into it in case that helps you

💯 2
deactivateduser21:08:34

Thanks! I’ll give it a spin and see what happens.

Alex Miller (Clojure team)21:08:32

note also that master is actually ahead of the stable release, so you're effectively getting the changes mentioned above too

Alex Miller (Clojure team)21:08:26

just out of curiosity, what do you use it for?

deactivateduser21:08:18

I’m messing about with a little “jig” project to make it easier to find, introspect, and prototype with 3rd party Clojure or Java libraries from the REPL, without having to mess with deps (either deps.edn or -Sdeps) and restart the REPL. Obviously it will have the same limitations as add-libs itself, but that’s still better than my current workflow (look for a library in my browser, express a dep to it, (re)start my REPL to pick it up, start exploring the library’s API, realise it doesn’t meet my needs, pick an alternative, …repeat…).

deactivateduser21:08:46

It’s open source (ofc), and happy to share a link once I finish battling with git identities on this new(ish) laptop…

deactivateduser21:08:50

Be warned that it is pretty hacky - it’s intended to be a cheap & cheerful tool to support my actual development, not something I’m super focused on in and of itself.

deactivateduser05:08:12

Just reporting back in to say that as you predicted the add-libs3 branch doesn’t work. When invoked with clj -Sdeps 'org.clojure/tools.deps.alpha {:git/url "" :sha "02809dbd41d4fe6e269ee67316d054a04bc70678"}' -r, an IllegalArgumentException is thrown and the REPL fails to start. When invoked with that coordinate in a deps.edn file and started with clj, the REPL starts, and the clojure.tools.deps.alpha.repl namespace can be required. But calling add-libs always returns nil, albeit with a tremendous amount of Apache HttpComponents logging output if the dependency doesn’t yet exist in ~/.m2 (and the JAR file and associated metadata files appear to be downloaded correctly - they just don’t get added to the active classloader).