Fork me on GitHub
#tools-deps
<
2019-04-09
>
borkdude16:04:15

how can I add compiled .class files to the classpath with tools.deps?

dominicm16:04:20

Put them in a folder, add it to your paths.

borkdude16:04:42

so I put it in target/classes and then :paths ["src" "target/classes"]?

borkdude16:04:37

I tried this from the command line: -J-cp target/classes but that didn’t work

Alex Miller (Clojure team)16:04:51

no, that won't work as clj is building the classpath

Alex Miller (Clojure team)16:04:00

probably should actually make that an error

borkdude16:04:12

and if target/classes doesn’t exist, it won’t bail out?

Alex Miller (Clojure team)16:04:14

that's the one java property you can't set :)

Alex Miller (Clojure team)16:04:51

(but it also won't pick up stuff that gets added later, I don't think)

borkdude16:04:57

that’s fine

borkdude16:04:17

I only need to compile a class if I’m using GraalVM compilation, for normal JVM usage it’s unnecessary

borkdude16:04:30

so this dir is going to be in my gitignore

rschmukler17:04:26

Is it possible to set environment variables via the deps.edn file, similar to the :env section of a project.clj? If not, what's the recommended way to handle that when using deps.edn?

hiredman17:04:55

:env as a way to set env variables is something some lein plugin provides

hiredman17:04:57

I don't think that plugin actually sets environment variables, it provides its own kind of shadow system that you get access to as long as you use that plugin's functions for reading env variables

rschmukler17:04:39

Ah! I misunderstood. There was a github issue that made it seem like it did something better than that -namely they were overwriting LD_PRELOAD and it was working it seemed

rschmukler17:04:33

(I'm hooking into some C FFI with an old relic of a library)

hiredman17:04:24

it looks like environ dumps the environment stuff to .lein-env, which is maybe a standard thing? I am not sure

hiredman17:04:47

but like, if you have environment variables, why not just set them?

hiredman17:04:08

LD_PRELOAD=... clj ...

rschmukler17:04:22

I don't want to overwrite them outside of the project, and I typically invoke clojure via cider

rschmukler17:04:34

so, adding project specific environment variables to my emacs config feels quite dirty

rschmukler17:04:58

(and requires all engineers on the project solve it on their own)

rschmukler17:04:12

I was able to solve it via dir-locals w/ emacs, for anyone interested...

rschmukler17:04:14

((nil . ((eval . (setenv "LD_PRELOAD" "libcurl.so.3")))))

Alex Miller (Clojure team)17:04:10

on clj command line, you can use prefix -J to pass env vars - like -J-Dmy.prop=val

Alex Miller (Clojure team)17:04:15

or you can put those in an alias in deps.edn under :jvm-opts

Alex Miller (Clojure team)17:04:52

{:aliases
 {:foo {:jvm-opts ["-Dmy.prop=val"]}}}

rschmukler17:04:14

Is there a default equivalent?

Alex Miller (Clojure team)17:04:33

no, but that has been requested multiple times and I expect will be added when I next get around to it

Alex Miller (Clojure team)17:04:37

there's a ticket for it

rschmukler17:04:03

Cool cool. It's completely understandable that there wouldn't be one (yet, or even ever). Thanks for the response!

Alex Miller (Clojure team)17:04:54

oh, you're asking about env vars

Alex Miller (Clojure team)17:04:00

I'm talking about jvm opts

Alex Miller (Clojure team)17:04:05

those are different things

Alex Miller (Clojure team)17:04:21

sorry for the confusion

Alex Miller (Clojure team)17:04:33

I consider that out of scope for clj right now

rschmukler18:04:08

Makes complete sense. Frankly, I think it probably introduces more nasty things than it's worth (eg. people using it as a config store for default environment variables, etc)... This is definitely an edge-case

Alex Miller (Clojure team)18:04:27

yeah, I'll resist adding that as long as I can :)

rschmukler18:04:36

:thumbsup: I think that's a good call

seancorfield18:04:09

That begs an interesting question: when folks are providing "environment" to processes, do they a) use only environment variables b) use only JVM properties or c) use a combination with one overriding the other? And, for c) which overrides which?

rschmukler18:04:02

@seancorfield As someone who came to Clojure (and the JVM) after heavily integrating docker + kubernetes, I can tell you I use environment variables for almost everything, and, if possible, don't intend to ever use or learn JVM properties

rschmukler18:04:09

I think it's mostly that containers (and golang especially) harbored in a lot of workflow / tooling around the idea of using environment variables, and it's something that can apply to outside any one run-time. ie. I can write a service and guarantee environment vars as a config option... If I switch away from JVM for any reason, I now need to provide a config migration story too

rschmukler18:04:19

regarding (c) - I would expect environment variables to be the last authority... but I think that's likely because of the esteem that I hold them in above

Alex Miller (Clojure team)18:04:33

usually in an app I load config from an edn file, but allow override via env, and don't use jvm props at all

👍 4
Alex Miller (Clojure team)18:04:28

I find that gives you a lot of flexibility and options for local dev, CI, prod

Alex Miller (Clojure team)18:04:44

but then I don't work on many apps :)

rschmukler18:04:16

@alexmiller that's exactly what I've done as well. I will sometimes have multiple configs corresponding to an environment (`prod.edn`, dev.edn, test.edn) which might overwrite certain keys in config.edn. I then use ENV to determine which env-specific config to use, while still allowing overwrites for specific vars via environment variables

rschmukler18:04:04

Elixir also does something similar to the above

Alex Miller (Clojure team)18:04:45

it's not well-known, but I like many things about https://github.com/levand/immuconf

Alex Miller (Clojure team)18:04:12

if you're willing to ignore the ENV stuff above and use only a stack of appropriately created config files

rschmukler18:04:07

Brilliant! Similarly, duct does EDN-based config quite well. The docs for it are a bit lacking... But they let you do coercion and environment variables directly via EDN reader macros

Alex Miller (Clojure team)18:04:11

yeah, I like duct too from what I recall

Alex Miller (Clojure team)18:04:21

I've used environ quite a bit, but can't say I love it

rschmukler18:04:06

Duct + Integrant have quickly become my favorite way to build / prototype applications. I feel like I'm building an arsenal of components that I can use to build almost any system that I have seen in my career. They integrate the config map w/ spec so you get good error messages, while still having a single map that shows the entire system. It's basically stuartsierra's component extended to a map / graph, rather than relying on manual instantiation from the programmer

Alex Miller (Clojure team)18:04:59

yeah, weavejester is great at maximizing the Clojure Way ™️

jcf20:04:34

Hi all. I'm looking into deploying a small toy project to Heroku inside a Docker container and I'm running into an interesting problem with my use of tools.deps. When I build the Docker container locally I can see all the dependencies being downloaded. When I push the container to Heroku and run the thing the same dependencies are installed all over again, which takes so long Heroku kill my container and the app fails to start. Is there anything container-specific to look out for with tools.deps? My first guess was there was a missing Maven repo somewhere but when I run the container locally all the dependencies are there. Naturally I'm now wondering if Heroku do something different on their platform that might prevent tools.deps from noticing the Maven repo.

jcf20:04:08

I'm running clojure -Srepro -e '(println "Dependencies installed.")' to install things in my Dockerfile, and then I run the app with exec clojure -Srepro -A:measure -m foo.bar.main. The measure alias only adds a single JVM opt.

jcf20:04:45

:measure {:jvm-opts ["-Dio.pedestal.log.defaultMetricsRecorder=foo.bar.report/reporter"]}

jcf20:04:03

I can reproduce the artefact-download behaviour if I run the container on Docker and invoke my CMD by hand.

dominicm20:04:47

Is it the same user?

jcf20:04:29

And I think I might have cracked it. It looks like ~/.m2 is /app/.m2/repository when I run the container but the Docker build step will install things in /root/.m2.

jcf20:04:41

@dominicm I think you've cracked it too!

jcf20:04:39

I'm guessing Heroku run things as some other user, which messes with $HOME. I wonder if I can tell tools.deps where to put the Maven repo with the Docker invocations only. Don't really want to mess with that in the project deps.edn as it's kinda up to the dev where they want to keep Maven artefacts.

jcf20:04:08

Does :maven/local-repo not work in an alias? 🙂

dominicm20:04:26

@jcf try -Sdeps perhaps? But remember permissions.

jcf20:04:30

I'll give that a stab.

jcf20:04:56

I'm giving -Dmaven.local.repo=/app/.m2 a try. Let's see if that does the trick. #docker 🙂

jcf20:04:00

That didn't work. Because it's maven.repo.local. 😕

Alex Miller (Clojure team)20:04:14

you can set it in deps.edn

👀 4
Alex Miller (Clojure team)20:04:34

:mvn/local-repo "/app/m2" as a root level entry

jcf20:04:07

@alexmiller I was hoping to only specify that for the Docker side of things to enable the default behaviour for local development.

Alex Miller (Clojure team)20:04:07

that's not something you can set in an alias

Alex Miller (Clojure team)20:04:33

but you could append by using clj -Sdeps '{:mvn/local-repo "..."}'

👍 4
dominicm20:04:05

That's what I was trying to say ^^ but with the brevity that mobile prefers :)

jcf20:04:35

I tried that originally and the running container still installed everything. I'll bring back the -Sdeps ... approach and poke around inside the container to see if the user in the running container maybe can't access the :mvn/local-repo.

jcf20:04:47

The -Sdeps {:mvn/local-repo "/app/.m2"} did the trick! Just had to blow away the container on Heroku and push it up again. Now I need to make sure we stick within allocated memory and start up fast enough despite not installing deps. Might have to uberjar yet. Thanks for the help, @dominicm @alexmiller!