Fork me on GitHub
#tools-deps
<
2023-01-18
>
Jakub Holý (HolyJak)00:01:16

Hello! When I start repl with clojure -A:some-alias , can I somehow tell you to (pre)load a namespace or possibly switch into a ns? Something like #C0AB48493’s :repl-options {:init-ns user} ? Perhaps my only solution is to name the ns user and put it on a dev-only path, then it will be loaded automatically?

seancorfield00:01:51

You can pass -i and/or -e -- clojure.main options to load a file or eval an expression -- and then -r to force a REPL to start.

❤️ 2
seancorfield00:01:22

(~/clojure)-(!2003)-> cat deps.edn
{:aliases
 {:dev {:extra-paths ["dev"]
        :main-opts ["-e" "(do (require 'startup) (in-ns 'startup))" "-r"]}}}

Tue Jan 17 16:05:23
(~/clojure)-(!2004)-> cat dev/startup.clj
(ns startup)
(println "Starting up dev...")

Tue Jan 17 16:05:43
(~/clojure)-(!2005)-> clj -A:dev
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
Starting up dev...
#object[clojure.lang.Namespace 0x1827a871 "startup"]
startup=>
(should use -M instead of -A there I guess)

seancorfield00:01:13

Or:

clj -M:build -i build.clj -e "(in-ns 'build)" -r
which is how we start a REPL with our build.clj file in the build ns.

dpsutton00:01:24

I’ve seen this request come up a bunch. I’ve never understood the importance of it.

seancorfield00:01:37

People seem to want to avoid doing a require and in-ns manually? 🙂

dpsutton00:01:39

Yeah. I don’t mean to say it’s wrong. Just would love to hear the benefit

skylize00:01:00

I think the common reason is to do all your work out of a single namespace, as you would when typing directly in to a CLI; except with a bunch of useful development tools preloaded along with access to your state management system. I use Calva, where there is definitely friction against that idea, because it automatically switches the repl to the namespace of whichever file you eval code from. Which namespace would be appropriate to load all these tools into?

seancorfield00:01:13

I use Calva but I have it's REPL window hidden and don't ever type into (so I don't care what ns it thinks it is in 🙂 ) but with the build.clj command above, I type build/test commands into it from time to time while working (e.g., resetting the dev DB, run Polylith tests since the last stable tag & update the tag if successful)

skylize02:01:27

> so I don't care what ns it thinks it is in > Well I think you do "care", just not attentively. The vars available to you (or not) are determined by the namespace being evaluated in, right?

seancorfield02:01:10

@U90R0EPHA If you're eval'ing code from your editor window, Calva/nREPL takes care of switching the REPL to the right ns. It only matters if you're typing directly into the REPL windows.

skylize02:01:47

But that was my point to begin with. 🤤 Calva (and presumably other nREPL workflows too?) conflicts with the idea of working out of a dev namespace with dev tools already required, because it switches namespaces out from under you.

seancorfield02:01:05

Ah, right. Then we're in violent agreement, as my partner likes to say.

😹 2
seancorfield02:01:45

Part of what I liked about Chlorine/Clover was that the REPL window was readonly -- because Mauricio wanted to discourage "bad" behavior 🙂

seancorfield02:01:33

I like that I can have Calva setup so the REPL window doesn't open by default. Then I use Portal as my "readonly REPL" window.

skylize02:01:48

Yeah. Putting your repl-bound code into a savable file definitely seems superior in most cases.

dominicm19:01:33

What are people doing to generate deps.edn in ~/.clojure with relative paths? I must admit I'm disappointed that my config will be generated rather than symlinked. As it breaks my current "setup" of using stow to manage my dotfiles.

seancorfield19:01:50

Can you elaborate on "relative paths"? What is the use case?

dominicm19:01:38

I need to use the deps.edn on a Mac and a Linux machine, and they'll refer to different paths when I want to, e.g. include a library from a directory.

seancorfield19:01:27

You have a library in source form on multiple machines? Is that library in git?

dominicm19:01:07

It is, but I want it in source form because I'll do local development on it. Although that's not a huge issue I suppose, I do that much less on the mac.

seancorfield19:01:57

Seems a bit dodgy to just assume you have the "same" library code in the "same" place on both machines -- I assume you're doing it this way to have a common alias that brings in that dependency on each machine? (rather than have the dependency in the projects that actually depend on it -- so I guess this is a tooling library rather than an actual dependency?)

dominicm19:01:14

I organize my code in the same way. I use ghq so my repos are always ~/src/github.com/juxt/clip (for example). Then if I'm working on a project and I'm also developing clip, I would launch the project with -A:user/clip so I could develop them in tandem.

seancorfield19:01:26

Hmm, yeah, fair enough... I have "often" felt the need for env vars in deps.edn for situations like this 😐

dominicm19:01:13

Yeah. I understand why it's the way it is, too. But, it would be a solution to this kind of problem 🙂

dharrigan21:01:55

The way I do it, and not ideal, is that I use ansible to push out to my linux boxen and mac boxen a templated deps.edn, replacing values depending upon the architecture.

dharrigan21:01:33

If only deps.edn supported variable substitution, even if just from $env variables, would make that all go away.

dominicm21:01:38

I'd be reasonably happy with the ability to include other deps.edn files too

dharrigan21:01:24

A bit like aero, eh? 🙂

seancorfield21:01:37

(which we just switched all our config EDN files over to at work!) ❤️ Aero!

dharrigan21:01:18

Indeed, I use aero for my config edn files too! 🙂

dominicm21:01:45

Env vars are probably problematic for caching. But if you could load aliases from files then you could write a few vars into a platform alias and join paths on that.

dvingo14:01:39

I have a small babashka script that will write the fully expanded pathnames (using fs/expand-home) into ~/.clojure/deps.edn - so my deps.edn file lives in a bb script and I put that in a git repo. (Also a happy aero user after reading https://www.pixelated-noise.com/blog/2022/04/28/integrant-and-aero/ !)

dominicm06:01:54

Having pondered this a little, I think the main thing I'd want is a portable alias/aliases I could use on all machines. That won't use any absolute paths. But I'd like to be able to add aliases that are specific to the machine alongside those aliases which do.

dominicm06:01:54

Phrased like that, I'm wondering if I could abuse the tools aliases for it. Except it wouldn't be a tool, just a normal -A alias

jeroenvandijk19:01:27

I’m seeing

Error building classpath. Malformed \uxxxx encoding.
with a specific dep:
clj -Sdeps '{:deps {prismatic/plumbing {:mvn/version "0.6.0"}}}'
But with lein there doesn’t seem to be an issue. Any idea?

jeroenvandijk19:01:10

With the following no issue

lein try prismatic/plumbing 0.6.0

seancorfield19:01:03

Just tried it on WSL2/Ubuntu and it works for me. What platform are you on? (and what version of JDK?)

2
jeroenvandijk19:01:28

> Just tried it on WSL2/Ubuntu and it works for me. What platform are you on? (and what version of JDK?) On mac osx, and I tried with Java 8 and 11.0.7 and Clojure CLI version 1.11.1.1182

jeroenvandijk19:01:51

removing the maven artifact didn’t help (at .m2/repository/prismatic/plumbing/0.6.0)

dpsutton19:01:48

❯ cd /tmp

/tmp on ☁️  metabase-query
❯ mkdir deps

/tmp on ☁️  metabase-query
❯ cd deps

/tmp/deps on ☁️  metabase-query
❯ clj -Sdeps '{:deps {prismatic/plumbing {:mvn/version "0.6.0"}}}'
Downloading: prismatic/plumbing/0.6.0/plumbing-0.6.0.pom from clojars
Downloading: de/kotka/lazymap/3.1.0/lazymap-3.1.0.pom from clojars
Downloading: de/kotka/lazymap/3.1.0/lazymap-3.1.0.jar from clojars
Downloading: prismatic/plumbing/0.6.0/plumbing-0.6.0.jar from clojars
Clojure 1.11.1
user=>

/tmp/deps on ☁️  metabase-query took 23s
❯ clojure -version
Clojure CLI version 1.11.1.1189

jeroenvandijk19:01:49

Ok weird something specific to my setup maybe. So far I’ve only seen it with this specific dep. I’ll try to isolate it more. I have found a workaround so no rush

dpsutton19:01:05

i’m on osx as well. very strange

seancorfield19:01:41

I'm on CLI 1.11.1208 but tried with JDK 8 and 19 on Ubuntu -- no problems. Very weird, indeed. Please let us know if/when you figure it out!?!? Curious minds need to know now...

☝️ 2
👍 2
jeroenvandijk19:01:05

The other issue I found in slack sounds as strange as this one. Maybe just unlucky 😬

borkdude20:01:00

@U0FT7SRLP What does:

(System/getProperty "file.encoding")
and
(java.nio.charset.Charset/defaultCharset)
return for you

jeroenvandijk21:01:08

@U04V15CAJ

➜  ~ clj
Clojure 1.11.1
user=>        (System/getProperty "file.encoding")
"UTF-8"
user=>        (java.nio.charset.Charset/defaultCharset)
#object[sun.nio.cs.UTF_8 0xd02f8d "UTF-8"]

lread00:01:07

Maybe something wackadoodled in your classpath? I expect you can't even get a result for?:

clj -Sdeps '{:deps {prismatic/plumbing {:mvn/version "0.6.0"}}}' -Spath

lread00:01:44

Did you rm -rf .cpcache?

jeroenvandijk11:01:20

yeah even -Spath doesn’t work here and I’m doing a lot of rm -rf .cpcache lately for another project when things don’t resolve. Maybe it’s related. I’m also on an older OS X. I have been postponing the updates, but maybe it’s time

lread13:01:28

If you are super curious, I suppose you could figure out exactly what is going wrong by debugging tools deps. The error message seems to be generated https://github.com/clojure/tools.deps/blob/6ae2b6f71773de7549d7f22759e8b09fec27f0d9/src/main/clojure/clojure/tools/deps/script/make_classpath2.clj#L224-L234.

jeroenvandijk13:01:53

Thanks for the pointer! Yeah i think I’ll do that after I get the work done that initiated this error 😅. Curious to see where this rabbit hole ends

jeroenvandijk15:01:50

hmm interesting

grep -rnw ~/.m2 -e '\u0000'

Binary file /Users/jeroen/.m2/repository/org/clojure/clojure/resolver-status.properties matches
Maybe that’s not good

borkdude15:01:42

you can try -Sdeps '{:mvn/local-repo "/tmp/mvn"}'

👌 2
jeroenvandijk15:01:45

clj -Sdeps '{:deps {prismatic/plumbing {:mvn/version "0.6.0"}} :mvn/local-repo "/tmp/mvn"  }' -Spath
src:/tmp/mvn/org/clojure/clojure/1.11.1/clojure-1.11.1.jar:/tmp/mvn/prismatic/plumbing/0.6.0/plumbing-0.6.0.jar:/tmp/mvn/org/clojure/core.specs.alpha/0.2.62/core.specs.alpha-0.2.62.jar:/tmp/mvn/org/clojure/spec.alpha/0.3.218/spec.alpha-0.3.218.jar:/tmp/mvn/de/kotka/lazymap/3.1.0/lazymap-3.1.0.jar:/tmp/mvn/prismatic/schema/1.2.0/schema-1.2.0.jar

jeroenvandijk15:01:13

So something wrong in my ~/.m2 then

borkdude15:01:28

corrupt files like the stackoverflow issue also indicated

✔️ 2
jeroenvandijk15:01:29

I removed the line containing \u0000 in the .m2/repository/org/clojure/clojure/resolver-status.properties. It was looking like this

#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Fri Aug 26 16:26:40 CEST 2022
maven-=
maven-metadata-central.xml.error=Could not transfer metadata org.clojure\:clojure/maven-metadata.xml from/to central (https\://repo1.maven.org/maven2/)\: \: nodename nor servname provided, or not known
maven-metadata-central.xml/@default-central-https\://repo1.maven.org/maven2/.lastUpdated=1661524000327
maven-metadata-clojars.xml.error=Could not transfer metadata org.clojure\:clojure/maven-metadata.xml from/to clojars (https\://repo.clojars.org/)\: \: nodename nor servname provided, or not known
maven-metadata-clojars.xml/@default-clojars-https\://repo.clojars.org/.lastUpdated=1661524000346
maven-metadata-my.datomic.com.xml.error=
maven-metadata-my.datomic.com.xml.lastUpdated=1634040492738
maven-metadata-my.datomic.com.xml/@default-my.datomic.com-https\://\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0

jeroenvandijk15:01:58

Not sure what this file does, but this was the issue!

jeroenvandijk15:01:31

Thanks for your help everyone!

jeroenvandijk15:01:29

Still curious why this only happened with prismatic/plumbing so far :thinking_face:

seancorfield16:01:50

Glad you tracked it down! Weird, indeed. Sometimes just blowing away the whole .m2/repository tree and letting it download "the world" again is a worthy exercise:grin:

2