Fork me on GitHub
#tools-deps
<
2018-12-12
>
potetm01:12:43

I write a clj script and wrap it w/ a bash script: https://github.com/potetm/advent-of-code/tree/master/bin

potetm01:12:13

You can put those on your path, add namespaced aliases to your ~/.clojure/deps.edn, and you can run nrepl from any dir containing a deps.edn.

timgilbert16:12:09

Say, what directories do I need to cache in between invocations to take advantage of tools.dep's caching? I'm running builds on circleci and I'm caching ~/.m2, ~/.gitlibs, and .cpcache after running clojure -A:foo -Spath, but even after the cached directories are restored, when it invokes clojure -A:foo I see it downloading a bunch of jars from clojars.

timgilbert16:12:40

Oh, also caching ~/.clojure which has its own .cpcache

dominicm16:12:41

M2 & gitlibs should be enough

Alex Miller (Clojure team)16:12:23

the ~/.clojure/.cpcache is only used if you are running clj without a deps.edn, so probably doesn’t apply

Alex Miller (Clojure team)16:12:55

and you would need the local .cpcache if you want to avoid the second pre-jvm hit of building the classpath

Alex Miller (Clojure team)16:12:21

but all of those file checks are based on timestamps, so when you say “restored” I wonder if that changes the timestamps

Alex Miller (Clojure team)16:12:33

particularly in relation to the deps.edn file

timgilbert16:12:30

Hmm, yeah, not certain how it works. Maybe I can "negative touch" the deps.edn file to decrease its last-modified time

Alex Miller (Clojure team)17:12:24

easier to touch the cpcache before you try to use clj?

Alex Miller (Clojure team)17:12:56

I guess we could have a flag that was the opposite of -Sforce

Alex Miller (Clojure team)17:12:14

that would use the cached cp regardless of the timestamps

Alex Miller (Clojure team)17:12:02

although then you’d have to make sure you did something else to force any deps changes to be picked up

timgilbert17:12:09

Just confirmed on circleci that my deps.edn winds up being newer than the .cpcache and .m2 directories. I presume that circleci saves the mtimes on cache and restores them on restore. I'll try recursively touching those directories as an initial workaround

hiredman18:12:41

I dunno, the touch thing will likely work, but I am suspicious of deps.edn always being newer and you always download jars

hiredman18:12:13

even if those files are older, it shouldn't cause you to re-download jars into your m2

hiredman18:12:35

yeah, if you are re-downloading jars over and over in to m2 that seems like the up stream issue and fixing it will take care of down stream

hiredman18:12:12

I dunno what facilities circleci provides, but you might stick a ls -R ~/.m2 at the start of your build before running clojure

timgilbert19:12:12

Hmm, the touch doesn't seem to have any effect:

ls -lRt --time-style=full-iso deps.edn .cpcache ~/.m2 || true

-rwxr-xr-x 1 circleci circleci 1660 2018-12-12 19:25:36.984493358 +0000 deps.edn

.cpcache:
total 36
-rw-r--r-- 1 circleci circleci  7989 2018-12-12 19:25:38.420533085 +0000 2750907078.cp
-rw-r--r-- 1 circleci circleci 24551 2018-12-12 19:25:38.420533085 +0000 2750907078.libs
-rw-r--r-- 1 circleci circleci    64 2018-12-12 19:25:38.420533085 +0000 2750907078.main

/home/circleci/.m2:
total 4
drwxr-xr-x 15 circleci circleci 4096 2018-12-12 19:25:38.380531978 +0000 repository

/home/circleci/.m2/repository:
total 52
drwxr-xr-x  3 circleci circleci 4096 2018-12-12 19:25:38.388532199 +0000 aopalliance
drwxr-xr-x  3 circleci circleci 4096 2018-12-12 19:25:38.388532199 +0000 commons-io

timgilbert19:12:13

Same results with ls -lRtc to get the ctime

timgilbert19:12:15

...hmm, might be a PEBKAC problem

timgilbert19:12:35

Argh, turns out it was a problem in my config where I changed the cache key things were being saved into but not the one where they were being restored from. Sorry for the noise! Once I fixed that up everything worked as expected and the deps are not being redownloaded in the second step.