tools-deps 2020-04-23 | Slack Archive

ikitommi06:04:12

when adding dependency using :git/url, using clojure.tools.namespace.repl/refresh seems to refresh also namespaces from those dependencies. Is there a way not to do that?

ikitommi06:04:55

added dependency to malli and it pulled sci, refreshing a lot of non-project ns’es:

:reloading (sci.impl.namespaces sci.impl.utils sci.impl.destructure sci.impl.fns sci.impl.interop sci.impl.doseq-macro sci.impl.exceptions sci.impl.max-or-throw sci.impl.for-macro sci.impl.analyzer sci.impl.readers sci.impl.parser sci.impl.interpreter sci.core malli.core malli.provider malli.json-schema malli.transform malli.edn malli.error malli.mermaid malli.swagger user sci.impl.main malli.util malli.generator)

Alex Miller (Clojure team)06:04:27

I don't know enough about refresh to say

Alex Miller (Clojure team)06:04:58

I think refresh looks at directories on the Java classpath, and git deps will cause dep paths to be there

Alex Miller (Clojure team)06:04:09

refresh doc string says "The directories to be scanned are controlled by 'set-refresh-dirs'; defaults to all directories on the Java classpath."

Alex Miller (Clojure team)06:04:24

so you could use that to narrow the scope

Alex Miller (Clojure team)06:04:11

(clojure.tools.namespace.repl/set-refresh-dirs "src")

Alex Miller (Clojure team)06:04:14

might be sufficient

ikitommi06:04:42

when using :mvn dependency, the code is in jars and because of that, not in scope of refreshing?

Alex Miller (Clojure team)06:04:14

right

ikitommi06:04:14

#Also sent to the channel

set-refresh-dirs did the trick, thanks!

Alex Miller (Clojure team)06:04:32

when in doubt, read the docs :)

☝️ 4

ikitommi06:04:46

would be great if this worked out-of-the-box, e.g. convention from DEPS to which folder the pulled deps are in, that is automatically excluded in TNS.

Alex Miller (Clojure team)07:04:36

by "box" I assume you mean tools.namespace?

Alex Miller (Clojure team)07:04:25

I think I would invert it - only include src dirs that are relative to the current directory and exclude others

👍 4

➕ 4

Alex Miller (Clojure team)07:04:41

tns doesn't know anything about deps or git deps or whatever

ikitommi06:04:14

replied to a thread:when adding dependency using `:git/url`, using `clojure.tools.namespace.repl/refresh` seems to refresh also namespaces from those dependencies. Is there a way not to do that?

set-refresh-dirs did the trick, thanks!

Jakub Holý (HolyJak)08:04:01

Hi! Is there support for adding an optional dependency, without specifying its version? clj-http has an optional dependency on cheshire and I would like to include it, using whatever version this version of clj-http wants. So instead of cheshire/cheshire {:mvn/version "5.8.1"} I want just cheshire/cheshire . Is this possible? Thank you!!!

borkdude11:04:20

I don't think it's possible since clj-http doesn't prescribe the cheshire version as a dependency, it's only available at development time for clj-http I think?

Alex Miller (Clojure team)12:04:37

No, this is not possible, you’ll need to pick a version

👍 4

Laverne Schrock14:04:06

I feel like I've heard mention of others running into this:

Cloning: <actual repo name snipped>
Error building classpath. Destination path "ops-cli" already exists and is not an empty directory
org.eclipse.jgit.api.errors.JGitInternalException: Destination path "ops-cli" already exists and is not an empty directory
	at org.eclipse.jgit.api.CloneCommand.verifyDirectories(CloneCommand.java:257)
	at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:189)
	at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
	at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:49)
	at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
	at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
	at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
	at clojure.tools.gitlibs.impl$ensure_git_dir$fn__955.invoke(impl.clj:109)
	at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:105)
	at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
	at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
	at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
	at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
	at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
	at clojure.tools.deps.alpha.extensions.git$eval999$fn__1001.invoke(git.clj:41)
	at clojure.lang.MultiFn.invoke(MultiFn.java:239)
	at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
	at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:169)
	at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:237)
	at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:217)
	at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:55)
	at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:47)
	at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:81)
	at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:72)
	at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:101)
	at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:95)
	at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:146)
	at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:118)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.core$apply.invokeStatic(core.clj:665)
	at clojure.main$main_opt.invokeStatic(main.clj:514)
	at clojure.main$main_opt.invoke(main.clj:510)
	at clojure.main$main.invokeStatic(main.clj:664)
	at clojure.main$main.doInvoke(main.clj:616)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:40)

Unfortunately, when I was trying to figure out what the reproduction steps were, the problem suddenly went away. I can however say, that before it started working I was able to confirm that ~/.gitlibs/libs/.../ops-cli existed with a few SHA directories, but not the one corresponding to the one I was using in my deps.edn. This was running version 1.10.1.492 of clj/clojure on Linux. Would it be possible to add some logging here: https://github.com/clojure/tools.gitlibs/blob/master/src/main/clojure/clojure/tools/gitlibs/impl.clj#L109 so that in the future we can see what the original Throwable was?

Alex Miller (Clojure team)15:04:52

I have some good guesses at this point what the problem is

Alex Miller (Clojure team)15:04:39

I think it's related to having two threads/processes cloning the same lib in parallel

Alex Miller (Clojure team)15:04:03

because deps expansion is now parallel this is much easier to encounter now

Alex Miller (Clojure team)15:04:36

and/or it could even just be bugs in the much older version of jgit we're using

Alex Miller (Clojure team)15:04:28

I have a long paused set of work in a branch to update to latest jgit etc and I wanted to do that first

👌 4

Alex Miller (Clojure team)15:04:32

if it is the parallel stuff, you can set -Sthreads 1 to turn it down

Laverne Schrock15:04:46

I'll give that a try if I run into it again

Alex Miller (Clojure team)15:04:36

probably longer term to actually "fix" this, needs more thought

2020-04-23

Channels