Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2021-05-13

Channels

#babashka
<
2021-05-13
>
pithyless07:05:32

What is the idiomatic way to know what was the original task that bb was called with?

borkdude07:05:19

@pithyless this is why I wanted to add these :before and :after things, but I decided not to add those yet because it wasn't clear what the added value was. Right now I believe there is no way to know this. What is your use case?

pithyless07:05:43

Probably some self-inflicted pain. ;] I used the :enter hooks to log some info about each running task, but I was thinking in CI I want to spew a lot more info about each task (since it's harder to recover the context), but e.g. on my dev machine I may just want to log that the primary task is running (without having all the dependencies logging their info).

pithyless07:05:32

I thought I could maybe just use :init for this, but not sure if there is a reference to a task I can grab at that point?

borkdude07:05:21

not really no

borkdude07:05:30

Maybe you can make some DEV environment variable which turns off the logging?

pithyless07:05:30

also FYI, I was surprised this morning about the way :continue works in shell. I expected it to ignore the error if passed true and otherwise call the proc if one was passed as :continue Turns out if you pass true it works as expected, but if you pass a proc then you deal with all exit-codes (including zero). It makes sense, when you actually read the code for handle-non-zero but it was surprising nonetheless. Maybe that part of the docs could be revised? I thought of :continue as "how to handle errors", but it seems the name comes more from "override the default handling logic, ala continuation-passing style code"

borkdude07:05:26

the :continue fn should return either true or false (or some other truthy value), what is surprising there?

pithyless07:05:34

facepalm re-reading the code now; OK, so I totally misread that part of the codebase this morning

pithyless07:05:31

so actually, what I need to do was override what the error-handling code does (the part that throws the ex-info); and I wanted to avoid rewriting a lot of the exit-checking code, but handle-non-zero is an internal detail of shell right now without much of a way to override it

borkdude07:05:10

should we "rename" / "change" :continue to :non-zero ?

pithyless08:05:48

so, let me take a step back; if :continue is either true or a fn that returns a truthy value, I think the name is fine; it was a misunderstanding on my part

pithyless08:05:38

for me the bigger issue, is if :continue is falsy, I can't change the behavior of the error condition (aside from just catching the ex-info exception)

pithyless08:05:21

ideally, I'd like to re-use all the logic of shell and handle-non-zero but pass in an alternative operation for the (throw (ex-info ..)) part of handle-non-zero

pithyless08:05:03

something like :error-handler (fn [proc opts] ...)

pithyless08:05:11

^ a good example of this is replacing the existing exception handler with something like (line-status/die exit-code "...") that will both exit the process and print a nice visible warning

pithyless08:05:49

this, BTW, is also what lread seems to want to do here - https://github.com/lread/rewrite-clj/blob/main/script/helper/shell.clj - but since he's just wrapping babashka.process/process he's missing out on the other goodies like automatic tokenize , etc.

borkdude08:05:00

@pithyless gotcha. So maybe we can have an additional :error-handler function then, which will be triggered on a non-zero exit code or if :continue returns false?

pithyless08:05:20

@U04V15CAJ Yep, that'd be great!

borkdude08:05:41

alrighty

🙏 3
borkdude08:05:17

and about the "main" task, what should we do there? we can add back what we had with :dependents perhaps? if :dependent is empty, then you know you are the primary task. although with explicit run this will probably not behave like you think it will?

pithyless08:05:30

My original hack was going to be calling tasks->dependees but I noticed it's commented out for now. You're probably right, that it may not always work as expected, so maybe just #hammock it for now? I don't want to be partially responsible for some not well thought-out features that are going be a maintenance burden :D

borkdude08:05:06

Babashka adds babashka.file to the System properties when you call bb with bb foo.clj. We could do something similar for run: babashka.task or so

borkdude08:05:01

or we could add another function babashka.tasks/plan which returns a map with some information about the execution plan

pithyless09:05:25

Both of those ideas sound sensible. The former would be a small overhead to the existing surface area (and would also solve my immediate ask). The latter could potentially have more use-cases, but also more edge-cases. IIRC, the last discussion about an exec plan was halted due to expectations vs non-deterministic ordering in parallel execution.

borkdude09:05:41

I guess adding another system property doesn't hurt

borkdude09:05:51

As for bike-shedding the name for the error handler: 

(shell {:error-handler (constantly nil)} "ls foo")
(shell {:on-error (constantly nil)} "ls foo")
(shell {:error (constantly nil)} "ls foo")
(shell {:error-fn (constantly nil)} "ls foo")

borkdude09:05:14

perhaps :error is nice and succinct?

borkdude09:05:50

You could then write this instead of :continue: 

(shell {:error (constantly nil)} "ls foo")

borkdude09:05:36

Another question: should the :error function determine the result value of shell or should it just cause side effects (like printing or throwing)

borkdude09:05:21

e.g.: 

(def x (shell {:error (fn [_] :dude)} "ls foo"))
(= x :dude)

pithyless09:05:45

:error-handler
:on-error
:error-fn
:error
^ I think you've just listed all legitimate and/or expected forms for the callback. :) I'm not usually the fan of the standalone version :error, but in this case I think it works fine (and symmetric to :continue)

pithyless09:05:17

if :error is just for side-effects, what would shell be returning?

borkdude09:05:50

the process

borkdude09:05:04

like it does now

pithyless09:05:22

I think it's safe to assume, that :error is the new return value; the caller can easily return proc if that's the intent

borkdude09:05:36

it surely is the more flexible solution

pithyless09:05:37

so (fn [proc] proc)

pithyless09:05:55

also, should it be (fn [proc opts]) or just [proc]?

borkdude09:05:23

it's now just one map arg: {:proc .. :task .. :babashka/exit ...}

borkdude09:05:42

if you just decide to re-throw this entire map in an ex-info, then it will behave the same as the original error-handler

pithyless09:05:27

handle-non-zero takes additional opts from shell; so if we don't pass that on to the handler, it can't use those options

borkdude09:05:13

those opts are part of the :proc, e.g. :in and :out

borkdude09:05:21

or am I missing something?

pithyless09:05:02

you tell me ;)

pithyless09:05:42

so this could be rewritten as (get-in (deref proc) [:opts :continue]) ?

borkdude09:05:57

why would you like to have access to :continue in the :error handler?

pithyless09:05:58

so, not necessarily to continue, but if we call (shell {:foo :bar :error (fn []..) } "ls") I guess it doesn't necessarily make sense that we would have access to :foo ; and besides you can always wrap that error handler in a closure if you need additional context

borkdude09:05:36

I guess so yeah

borkdude09:05:16

Let's go with :error-fn. :continue accepts a naked boolean, whereas :error-fn only accepts a function 

$ clojure -M:babashka/dev -e '(babashka.tasks/shell {:error-fn (constantly 1337)} "ls foo")'
ls: foo: No such file or directory
1337

👍 3
borkdude09:05:55

Applied both things on master now. Please test :) Binaries should appear soon in #babashka-circleci-builds

borkdude10:05:54

:babashka/exit is safe to depend on: you can throw an ex-info anywhere in a script and set this value, then bb will exit with this value, if no other code handles the exception

pithyless10:05:57

ah cool, TIL!

borkdude10:05:46

this is only supported since 0.4.0

pithyless10:05:39

@U04V15CAJ confirming that "babashka.task" and "error-fn" works on my machine. Kudos for the frenzied and selfless work, as always! metal

🎉 3
pithyless07:05:03

> Maybe you can make some `DEV` environment variable which turns off the logging? Actually, I'm passing around a :task/log-level in my context so no issue there; what I wanted to solve was can I have an :enter that logs stuff dependent on the log-level AND if we're now in the primary original task

borkdude07:05:50

I guess we can add a :primary flag to current-task but what if you have tasks like: 

{a :task-a
 b {:depends [a] :task :task-b}
 b:clean (do (clean) (run 'b)}
When you run b:clean then b isn't the primary task anymore and I think you still want to treat it as such in some cases?

borkdude07:05:56

btw, perhaps the b:clean approach is also a way to deal with your problem: since you can execute something before the task (set some state?) and then run the "primary" task?

pithyless08:05:15

I would consider whatever bb XX was originally called with as the primary task; there just doesn't seem to be a way to hook in and know what bb was originally called with right now, correct?

pithyless08:05:27

But not sure how important of a use-case this is for everyone

borkdude08:05:13

@pithyless yes, I tried to explain what the problem with this is here: https://clojurians.slack.com/archives/CLX41ASCS/p1620892550258200

pithyless08:05:11

for me the run 'b doesn't really change anything, since it's no longer what the user initiated from the shell; but I can see how the reverse may also be considered true and obvious; so I guess I don't have a really good case to argue here :]

Akiz08:05:48

Hi, do you know why i am getting “WARNING: this project requires babashka 1.0.0 or newer, but you have: 0.4.0”?

borkdude08:05:39

@zikajk This is printed when someone has in bb.edn: :min-bb-version "1.0.0"

3
borkdude08:05:52

but version 0.4.0 is currently the most recent :)

pithyless08:05:24

Someone's already living the dream of bb 1.0 🙂

👍 3
borkdude08:05:46

I had this in the docs somewhere as an example, perhaps someone copied it ;)

borkdude08:05:31

I had to use future version as an example since it didn't work with other versions in the first version that this feature was added ;)

Akiz08:05:32

The joke is one me 😄

Akiz08:05:54

{:paths          ["src"]
 :deps           {seancorfield/honeysql                                {:mvn/version "2.0.0-beta2"}
                  douglass/clj-psql                                      {:mvn/version "0.1.2"}
                  clojure-term-colors/clojure-term-colors                {:mvn/version "0.1.0"}
                  borkdude/spartan.spec                                  {:git/url ""
                                                                          :sha "12947185b4f8b8ff8ee3bc0f19c98dbde54d4c90"}}
 :min-bb-version "1.0.0"}

borkdude11:05:20

babashka 0.4.1 🎉 https://github.com/babashka/babashka/blob/master/CHANGELOG.md#041 Among other tiny updates, babashka linux static is now based on musl and should work on pretty much any amd64 linux version! 🎉 thanks @rahul080327 and @thiagokokada

babashka 21
bananadance 9
grazfather13:05:28

So has anyone yet started using BB as their installer/dotfile manager?

👀 3
borkdude15:05:50

@U08ALHZ2N what do you do when one of your cloned repos needs updating?

cldwalker15:05:20

Just manual for now as the diff b/n computers is usually only a repo or two. Could be a useful thing to add down the road

cldwalker13:05:38

Added a repo update for all repos. Thanks for the nudge on this. Ended up making the tasks data driven

🔥 6
grazfather14:05:07

Very nice, I was planning something similar, but for all my dotfiles

grazfather13:05:06

a bb.edn with dependencies (e.g. install homebrew before installing things using homebrew) sounds like the perfect use

🤯 3
grazfather13:05:59

there’s obviously a bootstrapping problem but with staticallly compiled bb and everything having bash/curl/wget that’s not too much of a barrier 🙂

borkdude13:05:52

most systems have curl now, except those Alpines

borkdude13:05:58

even Windows has it

grazfather13:05:28

yeah, I am saying basically: curl babashkainstall && ./bb is no problem

grazfather15:05:54

hah! nice. I actually didn’t know about brewfile, so that solves that part

borkdude16:05:25

it does

👍 3
ericdallo16:05:51

How can I call a bb task from outside a folder with bb.edn?

borkdude16:05:45

(cd the-folder; bb the-task) ?

Darin Douglass16:05:47

(shell {:dir "the-path"} "bb my-task")
?

ericdallo16:05:16

yeah, I thought there was a option for that, but the cd one looks good and easy

ericdallo16:05:17

Hum not sure cd one would work, since the CI uses on clojure JVM 

(apply sh (concat command [:dir dir])
and my command would be something like 
["cd" "../my-dir" "&&" "bb" "-m my-project.main"]

ericdallo16:05:25

not sure that would work with clojure sh

borkdude16:05:24

no, that won't work as shelling out from the JVM is not the same as executing bash, but you provide :dir so why not set that to the bb.edn dir?

ericdallo16:05:42

the dir is dynamic for multiple shell tasks, and this one I want to make using babashka

ericdallo16:05:51

I can't change that :dir 😕

borkdude16:05:13

so shell out another time? (sh "bb" "-m" "my-project.main" :dir "../my-dir")

ericdallo16:05:09

oh, I see, would be: 

["bb" "-m my-project.main" ":dir" "../my-dir"]
right?

ericdallo16:05:18

that's the option I want so

borkdude16:05:21

or (sh "bb" "-m" "my-project.main" :dir (str dir "/../my-dir")) (but then using proper file system stuff)

borkdude16:05:38

which sh are you using here?

ericdallo16:05:54

clojure.java.shell

ericdallo16:05:22

it's a common file for multiple tasks, I can't change that code, that's why I can only change the command one that is dynamic

borkdude16:05:15

it's a bit yucky to execute a bunch of shell commands like this, as you can't really see what's going on, unless you print a giant string blob at the end

borkdude16:05:12

does it execute a bash script? 🧵

borkdude16:05:37

your "&&" suggests so

ericdallo16:05:06

yeah, all other places have something like: ["../other-dir/other-script.sh"]

ericdallo16:05:13

but this one, I want to use bb instead of a bash one

borkdude16:05:15

and you don't see any output?

borkdude16:05:48

just fire and forget?

ericdallo16:05:21

yes, I think

borkdude16:05:13

well, if it's bash, then you can append (cd the-folder && bb the-task) , the parens start a subshell

ericdallo16:05:30

hum.. sounds good

ericdallo16:05:34

I'll try, thanks!

ericdallo16:05:16

but does clojure.java.shell understand the () ? like (apply sh ["(cd ../my-dir && bb -m main)"])

borkdude16:05:12

no. I asked if this was bash

borkdude16:05:42

if this is a standalone sh invocation, then why don't you set the :dir yourself

ericdallo16:05:43

yeah, sorry, it is not, it's a 

(apply sh (concat command [:dir dir])

ericdallo16:05:07

because the dir is set dynamicly for all other commands

borkdude16:05:50

what do the other commands look like?

borkdude16:05:05

clojure.java.shell/sh can only execute one file at a time

borkdude16:05:39

so unless the first thing is "bash" "-c" I have no clue how that setup works

ericdallo16:05:50

It's a Nubank migration tool, that allows you create multiple migrations and we have something like a migration.edn: 

{:migrations [{...
               :command ["../other-dir/migration.sh"]}
              {...
               :command ["../my-dir/migration.sh"]}]}

ericdallo16:05:12

and I'd like to add a new command, but that uses bb with a bb.edn with custom deps like rewrite-edn

borkdude16:05:23

so each :command is executed by a separated sh invocation?

borkdude16:05:04

can you have {:dir ... :command ["..."]} ?

ericdallo16:05:16

The CI/other service kind of parse the edn and call (apply sh (concat command [:dir dir]))

ericdallo16:05:37

there dir is the dir with all migrations

ericdallo16:05:46

so I can't change that easily

borkdude16:05:51

interesting, so let's see how clojure.java.shell/merges :dir, maybe you can override it

ericdallo16:05:03

yeah, if that works would be nice

borkdude16:05:10

you know, you can just make a bash script which then cds into the right dir and hook that up to your command thing

ericdallo16:05:16

> can you have `{:dir ... :command ["..."]}` ? How this works?

ericdallo16:05:24

yeah I thought about that, but...

ericdallo16:05:34

calling directly bb would look way better 😛

ericdallo16:05:52

and if that works, next tasks would follow the same standard

borkdude16:05:11

then you should make your migration tool more flexible using an overridable :dir argument, I think

borkdude16:05:32

I can't control your migration tools, sorry :)

ericdallo16:05:36

Yeah, it seems would need that change

ericdallo16:05:48

haha no problem, is not that easy to change the migration tool

ericdallo16:05:57

I'll probably end with a .sh to cd and call bb

👍 3
ericdallo16:05:15

is just that maybe in the future could make sense bb has a --from-dir or something

ericdallo16:05:23

if that is possible somehow

borkdude16:05:16

to work around the limitations of your migration tool?

ericdallo16:05:15

hahaha no, maybe someone can need that as well

borkdude16:05:00

I'm not even sure if that is possible to implement

ericdallo16:05:06

the .sh that cd and call bb worked, thanks for the help!