Fork me on GitHub
#tools-deps
<
2021-07-14
>
seancorfield05:07:33

clojure -Ttools install com.github.seancorfield/depstar '{:git/tag "v2.1.267"}' :as depstar
Major documentation overhaul, aligns with tools.build.

šŸ™Œ 8
dominicm10:07:47

For -X is there yet a strategy for substitution (i.e. where I can't manually do edn escaping) so that I can do something like clj -X:foo :bar-key "$(wl-paste)" ? I'm only ever expecting strings when doing this.

borkdude11:07:07

@dominicm

$ clj -X clojure.core/prn :foo \"$(echo "dude")\"
{:foo "dude"}

dominicm11:07:49

@borkdude that isn't general purpose though, what if my command outputs a "? e.g. clj -X clojure.core/prn :foo \"$(echo '"dude')\"

borkdude11:07:25

you want an escaped Java string?

borkdude12:07:00

@dominicm

$ escaped=$(echo '"dude' | bb -e '(prn (slurp *in*))')
$ clj -X clojure.core/prn :foo $escaped
{:foo "\"dude\n"}

pbrown18:07:23

I came across a new warning after updating TDA recently: WARNING: Use of :paths external to the project has been deprecated, please remove: ... What's the rationale behind disallowing such paths? Some of my builds include paths that symlink into /tmp, which look 'internal' e.g. :extra-paths ["target/cljs-resources"] but become 'external' when canonicalized in clojure.tools.deps.alpha/validate-paths. (I use /tmp to include build inputs from previous steps, and because I can mount it as tmpfs for speed and to avoid wearing out my SSD.)

seancorfield18:07:07

@alexmiller ā˜ļø:skin-tone-2:

Alex Miller (Clojure team)19:07:33

> What's the rationale behind disallowing such paths? Projects that reach outside their project do not play well when moved to other people's machines, particularly as gitlibs

Alex Miller (Clojure team)19:07:36

I don't understand "to avoid wearing out my SSD" but I do understand "for speed" :)

Alex Miller (Clojure team)19:07:29

this may be a place where some careful path handling could keep the path to the in-project link. if you like, file a question on https://ask.clojure.org and I can add it to the queue of things to look at

pbrown19:07:23

Ah makes sense, thanks! Sounds good, I'll file it there.

dominicm19:07:21

@borkdude right, but ideally without needing an extra tool in there. Not everyone has bb installed (yet)

hansbugge19:07:57

$ echo -n '"bla' | clj -M -e '(prn (slurp *in*))'
"\"bla"
šŸ™‚

borkdude19:07:00

bb runs normal Clojure. You can use normal Clojure is bb is not there, so there's always an alternative

dominicm19:07:13

Yeah, true. Not particularly fast though. 2.46s overhead cost for it. And not great to type at the terminal either.

Alex Miller (Clojure team)19:07:45

can you use something like this?

clj -X:foo :bar-key '"'$(wl-paste)'"'

dominicm19:07:38

@alexmiller that's going to fail if there's a space in the paste šŸ™‚

Alex Miller (Clojure team)19:07:42

% echo $FOO
a b c
% clj -X clojure.core/prn :a '"'$FOO'"'
{:a "a b c"}

dominicm20:07:16

I get a different result:

$ VAR='foo bar baz'           
$ clj -X clojure.core/prn :bar-key '"'$VAR'"'
Unreadable arg: "\"foo"

dominicm20:07:26

Is this perhaps a bashism?

Alex Miller (Clojure team)20:07:07

you could surround $VAR with "" there again if you need to, not sure why mine is different

Alex Miller (Clojure team)20:07:14

maybe bash version - Mac uses old bash

dominicm20:07:18

Hmm, no. Shellcheck says that's still not right, even with bash.

dominicm20:07:13

For me, it works on zsh but not with bash or dash.

borkdude20:07:56

Maybe just don't bother with -X and use -M in this case?

dominicm20:07:55

The world seems to be moving to -X though.

Alex Miller (Clojure team)20:07:30

clj -X clojure.core/prn :bar-key '"'"$VAR"'"' seems like it works on both zsh and bash

dominicm20:07:26

@alexmiller That doesn't work for anything containing a quote.

āÆ VAR='b"'                                     
āÆ clj -X clojure.core/prn :bar-key '"'"$VAR"'"'
{:bar-key "b"}

dominicm20:07:44

Should be {:bar-key "b\""}

Alex Miller (Clojure team)20:07:03

have you heard the good word about putting your data in files? :)

šŸ”„ 2
borkdude20:07:21

Moving to -X causes shell-related pains like this and the solution is different for each shell/OS, which is a pity for a platform like the JVM which is supposed to be write-once, run anywhere, but yes, it seems the world is moving towards that.

dominicm20:07:21

I'm thinking about things like -X some.fn/bar '[:db :password]' "$(pass show -p password-for-db)" or other things where users want to use the clojure stuff as a swiss army knife of dynamic parts.

Alex Miller (Clojure team)20:07:38

env vars is probably a better choice for that particular case (where you don't want your passwords in your command lines, visible in process commands)

dominicm20:07:06

Well there're other things too, like script clj-new (which I've done for testing my templates before). You get into the game of escaping your temporary names. Rewriting clj-new to take env vars doesn't make sense. I'm looking for parameterization a la carte, rather than hard-coded into functions.

Alex Miller (Clojure team)20:07:42

well you're making these calls from some shell, so there's no way to avoid the eccentricities of that API

Alex Miller (Clojure team)20:07:13

if you want more control, write a clojure program into a file and run it

2
Alex Miller (Clojure team)20:07:53

or pipe it over stdin with clojure.main :)

dominicm20:07:00

Clojure is worse than a shell here. Process substitution is much harder than $() for example. Shells are optimized for combining things in ways like this on the fly.

dominicm20:07:17

Maybe we need a type system. clj -X clojure.core/prn ^String :foo-bar "$(echo 'b"zt')" and then -X can just treat it as a literal string. Of course, that breaks if someone accidentally echoes something like ^String as their key. So we'll need an escaping system, just in case.

ā›” 2
dominicm20:07:29

In case not obvious, I'm trolling.

Joshua Suskalo20:07:22

Sounds like the job of a tiny library for use with #babashka

borkdude20:07:14

I think most of the pain with shells + -X comes from having to quote strings. If that was not needed, it would probably a bit smoother.

Alex Miller (Clojure team)20:07:52

Agreed, still an open topic

dominicm20:07:07

Something that would work without sub-shells or crazy escaping, although is eh compared to normal shell usage, would be FOO="$(echo 'b"zt')" clojure -X clojure.core/prn :foo-bar '#env FOO' where env is doing (System/getenv).

dominicm20:07:28

https://github.com/clojure/clojure/blob/b1b88dd25373a86e41310a525a21b497799dbbf2/src/clj/clojure/core_print.clj#L200-L221 This looks reasonably easy to port to a C/zig program that would be portable & tiny enough to include in a clojure distribution for making this escaping reasonably easy. It's like how xargs ports stdin to args, this ports shell args to clojure args so cargs? šŸ˜„

dominicm08:07:07

#!/bin/sh

printf '"'

sed 's/\\/\\\\/g' |
    sed "s/\r/\\\\r/g" |
    sed 's/$/\\n/g' | tr -d '\n' |
    sed "s/\t/\\\\t/g" |
    sed 's/"/\\"/g' |
    sed "s/\f/\\\\f/g" |
    # \b, but sed uses \b for word boundary
    sed 's/\x08/\\b/g'

printf '"'
That seems to work as an edn escaper. I could see this going into the clj tool so you can do printf '"' | clj arg

dominicm08:07:04

Compared with the native zig version it's much slower, but also portable so probably worth it šŸ™‚ Overall, both are fast.

didibus04:07:26

Or a GraalVM version like https://github.com/borkdude/deps.clj which can properly work on Windows as well (which I wish the official cli followed this model instead of using bash)

souenzzo20:07:10

@dominicm why not -X app.start-script/prod where (defn prod [] (sh/sh "pass" "show" "-p" ....)) ?

dominicm09:07:19

The use of pass is local to myself, not to the whole team.

dominicm09:07:38

When I'm working from the shell I'll often use parameter subsitution in various ways, change it, etc.

seancorfield20:07:27

That would have to be (defn prod [_] ...) to be invoked with -X but at this point I agree with Alex: write a (Clojure) script for something that is too complex to do easily from the command-line.

Cam Saul23:07:44

sorry if this is a dumb question, but how do I clojure -X:deps prep a library that's only included under :extra-deps for a specific alias? At first I assumed either clojure -X:my-alias:deps prep or clojure -X:deps:my-alias prep would work, but they both fail with the libs must be prepared before use message. I tried clojure -X:deps prep :basis '{:aliases #{:my-alias}}' after reading the https://github.com/clojure/tools.deps.alpha/blob/655ea0a037f041e365cc56db0dc8181268c29998/src/main/clojure/clojure/tools/cli/api.clj#L68 but that doesn't work either, since {:aliases #{:my-alias}} isn't actually a basis (although it could be used to create one, e.g. with tools.build create-basis) -- I'm not sure how to get an actual basis from the CLI. I tried clojure -X:deps basis '{:alias #{:my-alias}}' thinking maybe that would return a basis that I could then pass to -X:deps prep (e.g. something like clojure -X:deps prep :basis "$(clojure -X:deps basis '{:alias #{:my-alias}}')") but basis doesn't write any results to the CLI. How do I prep libs in an alias?

Alex Miller (Clojure team)23:07:27

At this moment, you can't but come back next week :)

šŸ‘ 2
dominicm20:07:28

https://github.com/clojure/clojure/blob/b1b88dd25373a86e41310a525a21b497799dbbf2/src/clj/clojure/core_print.clj#L200-L221 This looks reasonably easy to port to a C/zig program that would be portable & tiny enough to include in a clojure distribution for making this escaping reasonably easy. It's like how xargs ports stdin to args, this ports shell args to clojure args so cargs? šŸ˜„