Fork me on GitHub
#tools-deps
<
2019-07-12
>
kenny16:07:35

@ghadi We talked on Tuesday (https://clojurians.slack.com/archives/C6QH853H8/p1562697498052500) about a tool that would "localize" (i.e. git/url -> local/root) certain dependencies before starting a REPL (or some other task). This would solve the problem of having a single project that depends on lots of internal libraries that often need to be developed in parallel with the project you are working on. You said "monorepo is a big hammer" and to take a "`read` => update => spit" approach. The general approach makes sense but there are a few issues I immediately ran into while trying to work on this tool. I'm curious how you solved these for your company (it sounded like you guys have a tool that does this too). When you "localize" dep libraries, how do you ensure that the library at a give local/root path is up to date with VCS? Often my local libraries are not up to date with the version on master. You could run a git check before localizing and warn the user that the code at a given local/root is not up to date. Similarly, should you ensure your localized dep libraries are all on the same branch? I could easily see a situation where the local/root of a lib is on a different branch (maybe you started some work there and didn't switch back to master) than the other libs or the service itself that you are working on. Both of those issues could easily cause strange behavior to occur which could be costly when debugging because you didn't have know the lib was not up to date or on the wrong branch. Having a monorepo would solve both of these problems (it introduces another set of problems though).

dnolen17:07:20

clojure -J-Xms4g -J-Xmx8g ...

dnolen17:07:31

this should set the JVM memory settings right?

dnolen17:07:22

I keep seeing OOMs from JGIT - is there a different way to set the memory for the tools deps itself?

dnolen17:07:16

does clojure have it's own memory settings that have to be set independently?

andy.fingerhut17:07:20

One way is to create an alias in a deps.edn file e.g. deep merge this into deps.edn map: {:aliases {:bigmem {:jvm-opts ["-Xms4g" "-Xmx8g"]}}}. Then run clojure -A:bigmem

dnolen17:07:32

@andy.fingerhut so are you saying what I'm trying above won't work?

dnolen17:07:03

note the OOMs are from clojure tool trying to fetch git deps

dnolen17:07:13

no from launching any stuff I'm trying to run

andy.fingerhut17:07:21

Oh, no, I believe it will work. I was responding more to the question I thought you were asking about a way to have settings for this.

dnolen17:07:51

I guess I'm skeptical that 8g wouldn't be enough for JGIT

andy.fingerhut17:07:59

And I probably misunderstood which JVM you wanted those options to go to, now that I read your question again.

dnolen17:07:30

yes not my own process

dnolen17:07:41

I want clojure to not fail fetching git deps

dnolen17:07:10

@alexmiller it looks like clojure has a hardcoded memory setting of -Xmx256m??

dnolen17:07:36

I'm using clojure in a CI context, and it consistently fails

andy.fingerhut17:07:01

Yeah, that is what I see in my /usr/local/bin/clojure shell script, too.

Alex Miller (Clojure team)17:07:43

It’s hard coded for the process that builds the classpath

Alex Miller (Clojure team)17:07:51

This is the first report I’ve seen of an issue with that but happy to mod

Alex Miller (Clojure team)17:07:41

Curious what it’s doing when it fails. Maybe a git version compare?

dnolen18:07:05

I thought I could set this myself with JAVA_CMD

dnolen18:07:18

but no good because of the hard-coding at the moment

dnolen18:07:00

I hacked it by seding clojure shell script on the CI instance