Fork me on GitHub
#tools-deps
<
2021-06-10
>
Alex Miller (Clojure team)20:06:25

New prerelease of Clojure CLI 1.10.3.875: • New: support for specifying multiple functions with -X with -> semantics • https://clojure.atlassian.net/browse/TDEPS-182 - Improve deprecation message to be more accurate • https://clojure.atlassian.net/browse/TDEPS-183 - Fix -Sdescribe output to be valid edn on Windows • https://clojure.atlassian.net/browse/TDEPS-179 - Fix incorrect classpath when :classpath-overrides removes path

sheepy 6
🚀 11
catjam 3
party-corgi 2
seancorfield03:06:56

We’ve simplified our CI build by using this to run multiple steps at once so it’s already a win for us — thank you!

🎉 8
seancorfield20:06:36

Awesome! Can you point me to an example of how that new -X stuff works? Is it literally just -X:myalias foo.bar/fn1 foo.bar/fn2 some.other/thing and each should take a hash map and return a hash map?

seancorfield20:06:19

What happens if any of them return nil? Does exec just pass the original input hash map into the next function, or does it end up passing nil?

Alex Miller (Clojure team)20:06:26

well importantly, the first function gets the kvs from the exec-args/ cli

Alex Miller (Clojure team)20:06:50

it's like ->, so if one function returns nil, the next gets nil

seancorfield20:06:27

Ah, so that also means that the functions in the 2nd pos on don’t need to take hash maps, right? It could be any data produced from the earlier fns?

Alex Miller (Clojure team)20:06:42

I will not claim that this is fully baked yet, so I reserve the right to change my answer before it becomes a stable release :)

seancorfield20:06:43

(I’m already imagining how folks like @borkdude might think of abusing this 🙂 )

catjam 2
👀 2
seancorfield20:06:58

evil chuckle

😈 6
dpsutton20:06:11

that seems really interesting

dpsutton20:06:44

with low startup time that makes bash very close to a useable repl

Alex Miller (Clojure team)20:06:05

repls are also very usable repls :)

dpsutton20:06:28

haha very true 🙂

vlaaad20:06:00

> -X:myalias foo.bar/fn1 foo.bar/fn2 some.other/thing how does it know that it's (-> {} foo.bar/fn1 foo.bar/fn2 some.other/thing) and not (foo.bar/fn1 {'foo.bar/fn2 'some.other/thing}) ?

borkdude20:06:42

@seancorfield could you help me understand how I could abuse this? ;)

Alex Miller (Clojure team)20:06:45

symbols at the front are functions

borkdude20:06:04

and @dpsutton about startup time: how does this solve startup time?

dpsutton20:06:39

i was thinking about sci/bb but at that point you could just pass in forms to be evaled

vlaaad20:06:54

ah, so clj-exec first consumes all symbol args as fn chain, and then rest args as map arg?

Alex Miller (Clojure team)20:06:15

well we now have trailing map support ala 1.11 too

vlaaad20:06:53

> fn* [kpath v]* map? this is great explanation 😄

Alex Miller (Clojure team)20:06:56

parsing is done with that spec

vlaaad20:06:00

Just tried it out on pwsh, it works

vlaaad20:06:34

not sure how I feel about the -X threading

vlaaad20:06:18

I tried this:

clj -X clojure.core/pr-str clojure.core/read-string clojure.pprint/pprint :a 1
but when I was playing with it I felt the order is somewhat hard to follow, because arg-map is in the end, unlike in real ->

rickmoynihan16:06:28

(defmacro <- [,,,],,,) :grinning_face_with_one_large_and_one_small_eye:

Alex Miller (Clojure team)20:06:50

interested in feedback, so do more , and come back

vlaaad20:06:11

intuitively I wanted to write this:

clj -X clojure.pprint/pprint clojure.core/read-string clojure.core/pr-str :a 1
because arg-map is closer to fn that will receive it. e.g.
(clojure.pprint/pprint (clojure.core/read-string (clojure.core/pr-str {:a 1})))

Alex Miller (Clojure team)20:06:13

I don't know that this is something you're going to want to use all the time, but it is another dimension of composition available

Alex Miller (Clojure team)20:06:36

the goal is not to replicate the repl. if you want that, use a repl

vlaaad20:06:54

mm yeah...

seancorfield21:06:58

I guess my big test is going to be breaking depstar up into multiple -X-compatible functions so you can run parts of it or all of it in a pipeline 🙂

Alex Miller (Clojure team)21:06:11

if you do do that, I would be very interested in hearing your questions/feedback. if I may be so bold, that might be a better thing to do in design space before actually cutting the code :)

seancorfield21:06:18

Oh, definitely. I need to tease apart the various useful processes that depstar performs and figure out exactly what inputs and outputs each of them have.

seancorfield21:06:34

I have smaller things to test it on before that 🙂

Alex Miller (Clojure team)21:06:13

the key thing is that inputs/outputs need to align, which harkens to namespaced keys ... but those fight with conciseness of what you want at the CLI

vlaaad21:06:22

Okay, another argument why (fn1 (fn2 argmap)) might make sense:

PS C:\Users\Vlaaad\Projects\deps-test\lib> Get-Content .\deps.edn
{:aliases {:deploy {:ns-default deploy}}}
PS C:\Users\Vlaaad\Projects\deps-test\lib> Get-Content .\src\deploy.clj
(ns deploy)

(defn dev [args]
  (merge
   {:env :dev
    :url "dev-cluster"}
   args))

(defn prod [args]
  (merge
   {:env :prod
    :url "prod-cluster"}
   args))

(defn deploy [env]
  (prn :deploy env))
Suppose I want to deploy my app. Currently I have to use this order: dev deploy
PS C:\Users\Vlaaad\Projects\deps-test\lib> clj -X:deploy dev deploy :a 1
:deploy {:env :dev, :url "dev-cluster", :a 1}
But this is unintuitive, as I would expect the at the command line first specify an "action" and then "env"
PS C:\Users\Vlaaad\Projects\deps-test\lib> clj -X:deploy deploy dev :a 1
:deploy {:a 1}
but with -> style the main action goes last instead of being first

seancorfield21:06:09

I don’t understand this at all. -> says “do things in this order” and that’s exactly what I would expect: setup the env, run the command.

Alex Miller (Clojure team)21:06:37

agreed, the order is the most important thing here

vlaaad21:06:36

so it's more like clj -X:build clean jar than clj -X:build deploy dev ?

seancorfield21:06:28

clojure -X:depstar sync-pom compile jar for example 🙂

💡 2
vlaaad21:06:26

mm yeah, that makes a lot of sense

phronmophobic00:06:38

> clojure -X:depstar sync-pom compile jar If I wanted to add a prepare-assets step? How would that fit in? Is it like writing any other clojure function, or would it be like writing a depstar plugin? How likely is it that an idiomatic clojure function would also function as a cli target? One of the reasons automating git is a pain is that there 1) a low level programmatic library and 2) a high level command line interface and not much in between. Maybe there's no way around it, but it seems like it's easy to fall into the trap where the cli is the primary interface and the programmatic interface becomes less idiomatic. It's a tough problem to crack since the clojure cli wants to tame the bashisms, clojurisms, and javaisms with just the right isms on top.

seancorfield01:06:25

I’ll try to choose the hash map contents/keys carefully but it would be just writing a function that took a hash map and return a hash map and then making sure it’s on the classpath. Some of it would depend on what inputs your prepare-assets function needed?

seancorfield01:06:09

I think having a task that computes the project basis (possible based on aliases provided as :exec-args) and then passes that through the pipeline would be helpful.

seancorfield01:06:59

I think we may all get a better sense of how this should work once we see tools.build — I suspect it will have a few “utility” tasks that standardize some of this setup. @alexmiller has indicate that tools.build will have a number of “standard” build steps built-in so it’ll be a case of other tools following that lead and building on the basic pipeline.

👍 3
seancorfield01:06:18

depstar does a lot of “smart” stuff when building uberjar files (in terms of merging things) so once I have tools.build to play with, I’ll figure out how to make the “smart” part of depstar replace the (presumably, much simpler) uber/`jar` tasks.

Alex Miller (Clojure team)02:06:31

There will be a basis building function. Well, there are multiple atm, I’m trying to sort all that out :)

😁 3
3
seancorfield22:06:50

@alexmiller What’s the thinking around -X functions that would require (shutdown-agents) calls at the end to avoid the 60 second “hang”? That’s been a long-standing issue with -main functions. I didn’t check the latest exec source but does it handle that? (since you clearly cannot chain exec functions that call shutdown-agents)

Alex Miller (Clojure team)22:06:40

Haven’t thought about it

seancorfield22:06:30

I just looked at the exec.clj code and it does not call (*exit* 0) or (shutdown-agents) in the “happy” case…

seancorfield22:06:46

Yeah, my first test of chaining tasks leads to a 60 second hang at the end because of that.

seancorfield22:06:59

(I used to run the tasks individually and they had calls to (shutdown-agents) at the end — which I removed so I could chain them — and I can’t just call clojure.core/shutdown-agents via -X because it takes no arguments 😞

Alex Miller (Clojure team)02:06:17

Yeah, we should probably do that for you

seancorfield03:06:15

Thank you, yeah, that would be super-helpful!