Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2019-12-13

Channels

#tools-deps
<
2019-12-13
>
mattyulrich04:12:25

Hi all - I’m sure this has been asked before but I just can’t find the answer. Is there a way to signal in my deps.edn that one of my deps is a bom and not a jar?

mattyulrich05:12:15

Or, actually more specifically, I seem to have problem with a bom as transitive deps.

dominicm08:12:52

What's a bom?

dominicm08:12:53

Oops. I was totally expecting it to be a typo of pom.

andy.fingerhut09:12:45

You may be right, but bom does have a relevant meaning here.

plexus09:12:43

not to be confused with a byte order mark

mattyulrich14:12:46

Hey all - actually noticed some updated support for this. Switched to the latest tools.deps and the transitive dep was resolved.. yeah - I was talking about a “bill of materials” dep..

codeasone11:12:58

What route should I take to get the equivalent of lein uberjar with :aot :all using the Clojure CLI tools?

codeasone11:12:21

I'm presently using depstar but there doesn't appear to be an equivalent option

seancorfield17:12:27

We use depstar at work to build uberjars for production deployment and we do not AOT anything.

codeasone00:12:39

Thanks for replying @U04V70XH6. Along with deploying our uberjar to ECS within a docker image, we also invoke it multiple times within our CI pipeline. My principle motivation was to shave some time off the CI runs, and additionally plain curiosity around why the facility wasn't being supported yet.

seancorfield00:12:55

Fair enough. If you (manually) create a classes folder in your project and add it as an :extra-paths entry when building your uberjar, you can run clojure -e "(compile,'my.entry.point)" as a way to compile the entry namespace (`my.entry.point` here) and its transitive deps.

codeasone00:12:02

Great tip: thanks!

seancorfield00:12:57

depstar tries hard to avoid messing with your project or your classpath: it just packages up whatever you tell it to, using the classpath as constructed by clojure using whatever options you've provided. Because compile requires that classes exists -- or else you can use binding to tell the compiler to put class files somewhere else, that must still exist -- and that the output path of the compiler needs to be on the classpath for depstar to include those files, it just doesn't seem like something depstar itself should try to do.

seancorfield00:12:37

Note that the depstar README does address this case https://github.com/seancorfield/depstar in terms of Main-Class in the manifest but it stops short of explaining specifically how to compile code and get the classes onto your classpath.

codeasone18:12:07

Thanks for clarifying.

kszabo11:12:58

AOTing is not recommended anymore unless you deeply care about startup perf

kszabo11:12:44

but if you really want to do it, I prefer https://github.com/EwenG/badigeon

👍 4
borkdude12:12:41

if you have an :aliases alias in ~/.clojure/deps.edn and one in your project deps.edn, it seems they are not merged?

borkdude12:12:25

e.g. putting: 

:babashka {:classpath-overrides {org.clojure/clojure nil
                                            org.clojure/spec.alpha nil
                                            org.clojure/core.specs.alpha nil}}
in ~/.clojure/deps.edn and in the project.deps:: 
:babashka {:classpath-overrides {my_gist_script nil}}
does not seem to merge

Alex Miller (Clojure team)12:12:32

They merge-with merge at the aliases level iirc

mattyulrich14:12:46

I’m sure this has been asked before, but can’t seem to find it. When within a running clj repl, is it possible to resolve new deps; kind of like boot-clj set-env! ?

Alex Miller (Clojure team)15:12:02

there's a speculative feature called add-lib in a branch for this

Alex Miller (Clojure team)15:12:25

(sha in that article is out of date)

vinnyataide19:12:19

where do I find the correct sha?

Alex Miller (Clojure team)19:12:56

it was pretty far out of date so I just merged it in 19d197ab221d37db750423eb970880cb87a91100

mattyulrich15:12:30

Wow - this is great! Thanks Alex!

Alex Miller (Clojure team)15:12:57

I think we will probably add this, but some of the details are likely to change

seancorfield17:12:27

We use depstar at work to build uberjars for production deployment and we do not AOT anything.

seancorfield17:12:35

@codeasone why do you want to AOT?

Alex Miller (Clojure team)17:12:29

it is perfectly reasonable to AOT a final uberjar for deployment

hlolli18:12:14

I asked this question few months ago, but since that time, did anyone write a lein-ancient equivalent for tools.deps?

seancorfield18:12:24

We've been using Depot very happily (although I've recently developed a bash script that does crazy stuff with clojure -Stree and a synthesized deps.edn file to get outdated deps in a way that suits us better)

hlolli19:12:49

wow nice, essential tool in any toolkit. I knew someone had made it by now 🙂

borkdude19:12:13

❤️ depot

borkdude19:12:10

@U04V70XH6 care to share the bash script? I'd be curious if I can port it to babashka and make a nice example out of it 🙂

seancorfield19:12:27

Trust me when I say it's "crazy stuff" -- and it relies very heavily on our custom build shell script that drives our use of clojure...

seancorfield19:12:34

The TL;DR is that it runs clojure -Stree to a file, runs that through sed etc to produce a temporary deps.edn with "RELEASE" for all of the dependency versions (that were :mvn/version explicit versions), then runs clojure -Stree on that, and then it compares the output of both trees, with some fudging to account for only the top two levels of dependencies...

borkdude19:12:21

sounds like a good hack to me 🙂

seancorfield19:12:01

Running Depot reliably on our huge monorepo and getting all the outdated deps that matter and avoiding the noise from transitive deps that I don't care about was very tricky, especially since we use CLJ_CONFIG to setup a baseline deps.edn with :override-deps for a lot of stuff...

seancorfield19:12:37

...any t.d.a. tooling that tries to read and process deps.edn files in isolation can run into problems with our setup -- tooling that uses the same process to read/merge the environment as clojure itself does works better for us.

seancorfield20:12:35

Wow, hahaha!! Important to note that your code won't work with any deps.edn that relies on ~/.clojure/deps.edn (or CLJ_CONFIG) to override/default any deps... 🙂

borkdude20:12:12

yeah, this is just a quick sketch which should be tuned to the needs of the scripter

seancorfield20:12:39

It just goes to show what can easily be done -- and it would work for "most people" I suspect.

borkdude20:12:47

probably one could come up with a way to use git to inspect newer shas as well

borkdude20:12:26

but that's an exercise for the reader 🙂

borkdude20:12:24

One could even use this for project.clj by just parsing the deps from it, generate a deps.edn and then run this script

scottlowe21:12:04

@ghadi I’m reporting back on yesterday’s tools.gitlibs OOME issue, which was seen when downloading large repo dependencies: bumping JVM Xms from 256m to 512m seems to have worked. I’ve not seen a failure yet, anyway. Thanks again for your instantaneous help.

ghadi21:12:35

No problem. What jvm version and vendor do you use btw?

scottlowe21:12:57

For the CI environment it’s OpenJDK 1.8.0_222

Alex Miller (Clojure team)21:12:28

seems pretty weird that jvm wouldn't grow to 512m

Alex Miller (Clojure team)21:12:51

I wonder if you're getting client jvm by default instead of server

scottlowe21:12:47

Indeed - I had wondered myself why this would be so; I shouldn’t need to babysit the jvm (?); but it does seem to be the server jvm:

scottlowe21:12:48

$ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)

Alex Miller (Clojure team)21:12:44

using same version locally, I get an initial of 256m and max of 4g with no settings

Alex Miller (Clojure team)21:12:04

with -Xms256m as set in the script, I see the same

Alex Miller (Clojure team)21:12:21

however, the max is dependent on the available ram - I think it's 1/4 total ram by default

scottlowe21:12:24

I agree that it is odd. This is on a Travis CI instance: Ubuntu 16.04.6 LTS (xenial). We hope to move to more recent versions of the JVM and Ubuntu soon. We’ve never seen this issue on developer machines (macos). The CI environment is more memory constrained than our dev machines. 7.5 GB, I think; but some of that memory is taken up by docker processes, as well as a JVM database process.

andy.fingerhut21:12:46

Maybe the Travis CI instance has 1G RAM default?

Alex Miller (Clojure team)21:12:56

certainly could be something like that

scottlowe21:12:15

Have checked the Travis CI instance: System Memory: 7680MiB, 64 bits