Fork me on GitHub
#babashka
<
2021-04-27
>
mike_ananev07:04:33

@thiagokokada you can do it like this

(shell (format "env VAR1=%s VAR2=%s %s" "var1" "var2" "ls -la")

kokada13:04:43

Like I said in the above thread, I want to change the current environment variables for the program, not for a subprocess

mike_ananev17:05:48

@thiagokokada https://gist.github.com/mikeananev/c87c1596aed50b7bb095a816f6921737 you can read env vars for the current process (see line 14) and then modify them as usual map.

kokada18:05:11

I may have understand wrong, but in this case I really need to export the environment variables again

kokada18:05:53

I can of course inject then in every shell call and every usage in my Clojure program, but this get old really quickly

borkdude07:04:46

@mike1452 the above can be done better by passing a map with :extra-env as the first arg:

(shell {:extra-env {"FOO" "BAR"}} "ls -la")

borkdude09:04:41

There is also :env to overwrite all env vars. These are the same options as in babashka.process

borkdude09:04:10

You can also use :out "foo" :err "log.txt" to write to a file

mike_ananev10:04:02

what is fqdn of shell ? babashka.process ? can't find it there

borkdude10:04:17

@mike1452 shell is a variant of babashka.process tweaked more towards usage in bb.edn. It is automatically imported from babashka.tasks

mike_ananev10:04:05

Thanks! Now it works

bb -e '(babashka.tasks/shell  {:extra-env {"FOO" "BAR"}} "env")'

mike_ananev10:04:25

bb -e '(babashka.tasks/shell  {:env {"FOO" "BAR"}} "env")'

mike_ananev10:04:02

I wanted to check env settings from one line script.

kokada14:04:25

Maybe this will interest someone, it is a modification of test_runner.clj that automatically discovers test files: https://gist.github.com/thiagokokada/fee513a7b8578c87c05469267c48e612

🚀 9
borkdude14:04:13

awesome!

🙌 3
jeroenvandijk14:04:31

FYI, i’ve been playing with autocompletion on the command line. I’ve created a small POC that could maybe grow into something bigger. It’s inspired by how the aws cli tool is structured and can do autocompletion in the same way. The responsiveness is much better though 😎 https://github.com/jeroenvandijk/matryoshka

😎 9
👍 6
jeroenvandijk14:04:32

@UGNFWPPGF I started with your work as inspiration, but i couldn’t figure out how to do nested completions. I’m not sure if the AWS approach is better or not, but it works

metal 3
borkdude14:04:25

https://github.com/jeroenvandijk/matryoshka/blob/9f70e63418b7b9ee7a58255d83a392da4b7f35de/matryoshka#L83 This is not a "doesn't work yet" problem, but a problem that aliases are only valid in the next top level s-expression. You should use resolve to resolve a fully qualified symbol, if you are in the same expression.

borkdude14:04:04

Clojure 1.10.1
user=> (defn foo [] (require '[clojure.set :as set]) (set/union #{1 2 3} #{4 5 6}))
Syntax error compiling at (REPL:1:47).
No such namespace: set

jeroenvandijk14:04:39

Yeah indeed. I guess I have to have a better look at what you did in bb tasks. This was a quick “addon” for now (i should have left it out)

borkdude14:04:20

bb tasks pushes all the :requires to the top level

borkdude14:04:51

@U0FT7SRLP would your solution work for both bash / zsh? it would be very nice :)

jeroenvandijk15:04:30

I’m using zsh, not sure about bash, but I’m guessing if we try we could even make it work for Windows 🙂 It uses the same approach as the AWS tool https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-completion.html#cli-command-completion-linux

jeroenvandijk15:04:39

The output is slightly different from what @UGNFWPPGF did. Maybe the underlying framework is different

jeroenvandijk15:04:47

I don’t know if this is the way for bash, but in my (basic) bash shell it doesn’t work:

bash-3.2$ autoload bashcompinit && bashcompinit
bash: autoload: command not found
bash-3.2$ autoload -Uz compinit && compinit
bash: autoload: command not found
bash-3.2$ compinit
bash: compinit: command not found
bash-3.2$ export PATH=".:$PATH"
bash-3.2$

jeroenvandijk15:04:34

not out of the box at least 😅

russmatney15:04:28

i'll have to dig into the implementation later today

russmatney15:04:48

the other thing that keeps coming to mind is wanting to support all of bb's cli opts, not just tasks

👍 6
jeroenvandijk15:04:07

@U04V15CAJ I’ve fixed the :deps and :requires “feature”. This works now https://github.com/jeroenvandijk/matryoshka/blob/main/tasks/custom.edn

borkdude15:04:16

nice! is this about completion or also about building an alternative task runner?

jeroenvandijk15:04:30

I don’t know yet. I am looking for something that I can use or all my projects. The autocompletion makes this actually feasible. You start with a category and the suggestions help you navigate

jeroenvandijk15:04:13

I was thinking of wrapping these commands https://github.com/practicalli/clojure-deps-edn so that I can use them everywhere without having to do constant lookups

borkdude15:04:45

what are you currently missing from bb.edn's task runner?

jeroenvandijk15:04:09

I’m not sure if I’m missing something other than the global part

borkdude15:04:47

how does your task runner implement "global" ness?

jeroenvandijk15:04:00

good question. Not in some particular way, but the difference with babashka tasks is that all the tasks are isolated and it has a tree structure so it should stay fast

jeroenvandijk15:04:21

I’m not saying this could not be part of babashka tasks and that this is better (i just started working on it today so it’s nothing really). I don’t know yet where this will end. When I have a structure I like I’ll try to see what the main differences are

borkdude15:04:28

fast in what sense?

jeroenvandijk15:04:37

If I have a global library of tasks this could easily become really big. So code loading would be an issue if you are not careful. If you try the aws cli you feel the difference 🙂

jeroenvandijk15:04:06

But i think it’s a bit premature to talk about my “benefits”. I’m not actually using it for something

borkdude15:04:37

well, it's good to take future use case into account without locking down the design to exclude that possibility right now, that's why I am asking

jeroenvandijk15:04:46

Especially the autocompletion needs to stay fast for a nice experience

jeroenvandijk16:04:30

So you don’t get hit by a huge startup penalty when you add many libraries. It’s only about the nesting. So a nesting of 4 would mean having to load 4 of these files

jeroenvandijk16:04:53

instead of something to the power of 4

borkdude16:04:02

I think these things can also be implemented using normal keywords instead of reader tags btw

borkdude16:04:27

right now a pretty large bb.edn still loads in 1ms

borkdude16:04:45

it's first parsed as EDN only and only the task that is executed is actually re-parsed as code

jeroenvandijk16:04:00

ah yeah so i’m guessing it’s a bit premature

jeroenvandijk16:04:14

i was also thinking what if you could add a module

jeroenvandijk16:04:34

i was thinking you could have an aws module and it would look like this file

jeroenvandijk16:04:32

that was my fantasy at least 😅 Not sure what parts are actually required for this to work

jeroenvandijk16:04:27

I’m curious how you would use keywords instead of the reader tags btw

jeroenvandijk16:04:53

(I stole the #include idea from https://github.com/juxt/aero btw. That one is not lazy though)

borkdude16:04:54

{:cmds {aws {:cmds "tasks/aws.edn"}}}
perhaps? :)

jeroenvandijk16:04:20

haha hmmm no that would be too simple 😛

jeroenvandijk16:04:34

fair enough, that would be better

borkdude16:04:35

yeah, I know that lib. personally I would like to avoid reader tags a little bit, but maybe my feeling isn't justified on good reasons

jeroenvandijk16:04:13

no i think you are right, probably better to keep it simple

jeroenvandijk14:04:13

(forgot the link)

borkdude15:04:52

Hmm, crap, if you have :deps in bb.edn but if you also have deps.edn, those deps.edn deps are actually also downloaded and added to the classpath ;)

😅 3
2
jeroenvandijk16:04:41

Note sure if it is related, but I noticed that babashka.deps/add-deps doesn’t complain when I give it rubbish

borkdude17:04:26

what kind of rubbish

borkdude17:04:36

it just passes stuff through to tools.deps basically

borkdude17:04:03

and tools.deps picks up on the local deps.edn

jeroenvandijk18:04:46

I didn't add the '{:deps ...}' wrapping and I was wondering why my require wasn't working. So not really rubbish, but it was invalid config

jeroenvandijk09:04:49

+1 here. I ran into the above described situation as well. I can confirm it can give unexpected errors and slower load time

borkdude09:04:29

in which case?

borkdude09:04:21

slower as in what, can you be more specific? btw, I am working on a fix as we speak

borkdude10:04:52

it should be fixed now. try one of the binaries from #babashka-circleci-builds

jeroenvandijk12:04:51

The case where it was giving me unexpected results was when I had my custom bb tasks runner script and giving it empty {:deps {} } . So I wasn’t expecting anything, however the deps.edn in that directory had an error, so the whole call failed. But the script I was trying to run was a hello world (without deps), so it was confusing at first

jeroenvandijk12:04:28

Normally this hello world would be quick, but due to the (failing) deps it was slower

jeroenvandijk12:04:45

i’ll try the new binary!

borkdude12:04:47

I'll add another check for an empty deps map, since it doesn't have to call out to tools.deps in that case

jeroenvandijk12:04:09

so far everything works. Calling tools.deps with an empty map adds 10ms. Not major

emilaasa16:04:20

I've been in a few programming language chats and one popular topic / channel was #naming-things

emilaasa16:04:27

Maybe we need that here as well? 🙂

6
dharrigan16:04:01

ha! what a great idea

madstap20:04:02

I'm trying to make a babashka project with a structure that mirrors a clojure one, and I want a test task that comes with the test dir added to the classpath. First I tried this, but it looks like there's no :extra-paths like in deps.edn

{:paths ["src"]
 :tasks
 {test {:extra-paths ["test"]
        :requires ([foo.test-runner :refer [run]])
        :task (run)}}}
Then I tried to work around it, but it looks like the require needs to be in a separate expression from where it's used, so this doesn't work either.
{:paths ["src"]
 :tasks
 {test {:requires ([babashka.classpath :as cp])
        :task (do (cp/add-classpath "test")
                  (require '[foo.test-runner :refer [run]])
                  (run))}}}
I also tried something similar with requiring-resolve , but no luck. Any suggestions?

borkdude20:04:15

@madstap tasks do not (yet) support :extra-paths. you can take a look at how @thiagokokada did it here: https://clojurians.slack.com/archives/CLX41ASCS/p1619532085157000

borkdude20:04:11

you could just add the test dir to the classpath in the script and load the script using (load-file ...) in the task

madstap20:04:35

The test-runner script would have to be on the classpath then, for me to be able to run it as a task, or I'd have to run it as a standalone script

borkdude20:04:37

you can run standalone scripts with load-file without adding them to the classpath

madstap20:04:05

Oh, right, sorry, didn't catch that. Yeah, that works

borkdude20:04:05

you could first use load-file and then call a function from that script, that would work too

madstap20:04:51

Maybe I'm missing something, but this doesn't work

{:paths ["src"]
 :tasks
 {test {:task (do (load-file "test/foo/test_runner.clj")
                  (require '[foo.test-runner])
                  (foo.test-runner/run))}}}

borkdude20:04:17

what exactly doesn't work here?

madstap20:04:29

The file does get loaded, but I'm not able to call the function

borkdude20:04:33

you probably don't need the require since you already loaded the file

borkdude20:04:57

but it should not hurt

madstap20:04:18

I get a ^--- Could not resolve symbol: foo.test-runner/run

borkdude20:04:59

hmm, right. this is the gilardi scenario again.

borkdude20:04:26

a bit ugly but ((requiring-resolve 'foo.test-runner/run)) should work

madstap20:04:33

That sounds ominous

borkdude20:04:39

or simply resolve

borkdude20:04:23

The gilardi scenario is basically this:

user=> (defn foo [] (require '[clojure.set :as set]) (set/union #{1 2 3} #{4 5 6}))
Syntax error compiling at (REPL:1:47).
No such namespace: set

borkdude20:04:40

The reader has to know about the alias (or namespace) set already before the requires has happened

madstap20:04:26

requiring-resolve works! thanks

madstap20:04:45

I see, that makes sense

borkdude20:04:12

I am considering an :enter step which is a hook before the task gets executed

borkdude20:04:21

this is for logging the task name for example

borkdude20:04:30

but possibly it would also solve this, not sure...

borkdude20:04:07

Maybe introducing aliases is better for this

madstap20:04:12

I guess that depends on whether the enter step is ran before reading the task code

madstap20:04:38

or analyzing, maybe

borkdude20:04:30

:aliases {:test {:extra-paths ["test"]}}
:tasks {test {:aliases [:test] :task foo.test-runner/run}}

madstap20:04:25

Yeah, that would solve it nicely

madstap20:04:35

Are aliases preferable to an :extra-paths key in the task?

:tasks {test {:extra-paths ["test"] :task foo.test-runner/run}}

borkdude20:04:54

I guess it's not too hard to support

borkdude20:04:33

I think one could also do it like this, perhaps:

{:tasks {-add-test-cp (babashka.classpath/add-classpath "test"),
           test {:depends [-add-test-cp]  :task foo.test-runner/run}
could you try that, just for fun...?

borkdude20:04:57

(indenting in slack is hard)

madstap20:04:41

Easier on the eyes than the requiring-resolve dance, I think

madstap20:04:19

This doesn't work though.

{:paths ["src"]
 :tasks {-add-test-cp (babashka.classpath/add-classpath "test"),
         test {:depends [-add-test-cp]
               :requires ([foo.test-runner :refer [run]])
               :task (run)}}}

borkdude20:04:13

that doesn't work because the :requires are processed on the top level before anything else

borkdude20:04:43

it should work when you do it like this: :task (do (require ...) (run)

borkdude20:04:45

But maybe this is fixable. I think it is. I'll make an issue for this.

borkdude21:04:14

Maybe just adding :extra-paths is nicer though. I wonder how common this is

borkdude21:04:54

@madstap I pushed support for :extra-paths and :extra-deps in tasks to master now. Are you willing to take it for a spin?

borkdude21:04:34

Maybe we won't ever need aliases if we support it like this

borkdude21:04:14

since you can depend on a task that adds extra-deps and extra-paths as a sort of alias

madstap21:04:17

wow, that was fast! 🔥

madstap21:04:16

Hmm, yeah, then tasks kinda serve as aliases as well

madstap21:04:38

I'll check it out

borkdude21:04:17

are you using macos or linux?

madstap21:04:22

Looks like it works

borkdude21:04:48

cool so :tasks {test {:extra-paths ["test"] :task foo.test-runner/run}} it is then

metal 3