This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-02-14
Channels
- # announcements (11)
- # babashka (82)
- # beginners (51)
- # calva (11)
- # cider (3)
- # clj-kondo (62)
- # cljdoc (10)
- # cljs-dev (22)
- # clojure (75)
- # clojure-boston (1)
- # clojure-brasil (3)
- # clojure-czech (4)
- # clojure-europe (49)
- # clojure-france (10)
- # clojure-italy (16)
- # clojure-nl (5)
- # clojure-uk (9)
- # clojurescript (69)
- # community-development (33)
- # conjure (12)
- # core-async (6)
- # cursive (2)
- # datalevin (7)
- # datomic (6)
- # graalvm (13)
- # gratitude (2)
- # honeysql (3)
- # introduce-yourself (1)
- # lsp (37)
- # nextjournal (62)
- # off-topic (29)
- # pathom (1)
- # quil (2)
- # reitit (4)
- # releases (2)
- # sci (1)
- # shadow-cljs (28)
- # spacemacs (10)
- # sql (1)
- # tools-build (3)
- # vim (3)
I'm thinking about a tool that queries stuff from a database and with the info gained queries some logs (aws cloudwatch). Let's say I make those processes bb tasks. I can depend the second task on the first. How would I have the output of the first task as input to the next task? Guess just writing to a file does it
@benjamin.schwerdtner If you write:
{:tasks {a 1, b {:depends [a] :task (+ 1 a)}}}
a is bound to the return value of task a in b
Just published a new version of babashka to master which has a massive performance improvement for loops (2-4x). Please try it out locally and report any regressions. I hope enough people will test drive it to find any problems before the next release!
$ bash <(curl ) --version 0.7.5-SNAPSHOT --dir .
$ ./bb "(time (loop [val 0 cnt 10000000] (if (pos? cnt) (recur (inc val) (dec cnt)) val)))"
Just try on Linux and no regressions...it works great! 👍 1408 ms against 3435 ms 🔥
fine for me, twice as slow as nbb which is surprising to me
[email protected] ~ % ./bb "(time (loop [val 0 cnt 10000000] (if (pos? cnt) (recur (inc val) (dec cnt)) val)))"
"Elapsed time: 1235.124292 msecs"
10000000
[email protected] ~ % nbb -e '(time (loop [val 0 cnt 10000000] (if (pos? cnt) (recur (inc val) (dec cnt)) val)))'
"Elapsed time: 643.817084 msecs"
10000000
nbb runs on a JIT-ed environment which is probably the reason it works so well. Are you using nbb 0.1.6?
Yep just updated both, so is it just specific examples like this loop that nbb would do better at or do you think it will be faster in all cases? trying to choose between bb and nbb for a project at the moment, its probably not going to make much of a difference but still interested 😄
Hard to say. Parsing is way slower in nbb (subject to optimization improvements) so it really depends on the workload
interesting, bb has always seemed like it had slower startup time in my tests
[email protected] ~ % time ./bb "(prn \"hello\")"
"hello"
./bb "(prn \"hello\")" 0.11s user 0.03s system 57% cpu 0.239 total
[email protected] ~ % time nbb -e "(prn \"hello\")"
"hello"
nbb -e "(prn \"hello\")" 0.11s user 0.01s system 107% cpu 0.112 total
maybe my tests are flawed thoughhuh, that's weird! try to run it multiple times. the first time may be a little slower due to file system stuff.
$ time ./bb "(prn \"hello\")"
"hello"
./bb "(prn \"hello\")" 0.01s user 0.01s system 82% cpu 0.025 total
on my system, I get for nbb:
$ time nbb -e "(prn \"hello\")"
"hello"
nbb -e "(prn \"hello\")" 0.19s user 0.02s system 108% cpu 0.192 total
Perhaps you are on m1 running bb with rosetta?looks like it could be close though, you can build a version that works on m1 yourself now https://github.com/oracle/graal/issues/2666
I think trying the arm binary in docker on m1 in a linux container should also resemble what performance you could get. But it'd be cool if you could try the real m1 version too :)
It seems bb is now as fast as python3 in loops?
import time
start = time.time()
val = 0
cnt = 10000000
while (cnt != 0):
cnt = cnt - 1
val = val + 1
end = time.time()
print("Took:", end - start)
print(val)
Took: 1.1271159648895264
10000000
node does it in 8ms, about the same time as jvm clojure
console.time("nbb")
var val = 0;
var ctn = 10000000;
while(ctn !== 0) {
ctn = ctn - 1;
val = val + 1;
}
console.timeEnd("nbb");
yeah I know, just surprising node is so fast, I always thought of it as slower than the jvm
for comparison, ruby:
$ time ruby /tmp/test.rb
10000000ruby /tmp/test.rb 0.27s user 0.02s system 98% cpu 0.288 total
With GraalVM ruby:
$ time $GRAALVM_HOME/bin/ruby /tmp/test.rb
10000000$GRAALVM_HOME/bin/ruby /tmp/test.rb 0.15s user 0.06s system 145% cpu 0.147 total
Graal Python:
$ time $GRAALVM_HOME/bin/graalpython /tmp/test.py
Took: 0.21900010108947754
10000000
$GRAALVM_HOME/bin/graalpython /tmp/test.py 1.16s user 0.19s system 278% cpu 0.483 total
bash 😆 (I mean it is what babashka is meant to replace )
time ./loop.sh
./loop.sh 42.95s user 2.40s system 99% cpu 45.354 total
Btw in the browser (chrome) I get about the same time as with bb: https://babashka.org/xterm-sci/
Safari seems to be faster!
user=> (time (loop [val 0 cnt 10000000] (if (pos? cnt) (recur (inc val) (dec cnt)) val)))
"Elapsed time: 735.000000 msecs"
10000000
Is there any bb (debug/verbose) switch that makes (shell... & (clojure...
calls made from tasks print the command they are executing?
@U08BJGV6E babashka does have a --debug
setting but the output is purely for debugging, not for daily usage. I think we could have a babashka.tasks global switch for this, like :command-print-fn
or so in which you can do your own thing and also provide a default
Makes sense, thank you. It would be a welcome feature for sure. Again, mostly for debugging:
; bb --debug run test
;; deps
(ns user-4b3db898-5cee-4604-8d36-839d2be6f97d )
(require '[babashka.tasks])
(when-not (resolve 'clojure)
;; we don't use refer so users can override this
(intern *ns* 'clojure babashka.tasks/clojure))
(when-not (resolve 'shell)
(intern *ns* 'shell babashka.tasks/shell))
(when-not (resolve 'current-task)
(intern *ns* 'current-task babashka.tasks/current-task))
(when-not (resolve 'run)
(intern *ns* 'run babashka.tasks/run))
nil
(def test (binding [
babashka.tasks/*task* '{:name test, :doc "Run project tests via Kaocha. Args will be forwarded to Kaocha.", :task (apply shell "bin/kaocha" *command-line-args*)}]
nil
(apply shell "bin/kaocha" *command-line-args*))) test
Loading namespaces: (e
This prints (apply shell "bin/kaocha" *command-line-args*)
which is what I'm invoking but what I'm interested in is whether I managed to invoke bin/kaocha
as I had intended
yeah make sense. What about doing this in :init
:
:init (do (defn shell [& args] (println (str/join " " args)) (apply babashks.tasks/shell args))
Yep, that should work for now although it still only tells me what args are being passed to shell
and not whether that results in the process invocation I intended to do. I.e. it won't tell me if I'm misusing the shell
api
I'm not sure if I get that. What else other than the above would you expect a built-in function to do?
I looked at examples https://book.babashka.org/#parallel and https://book.babashka.org/#tasks:clojure and I see that multiple args to the cli are sometimes given in one string and other times one arg per string. And I wasn't sure how quoting etc would work
which works similar to how shell strings work: foo bar
becomes ["foo" "bar"]
but 'foo bar' becomes
["foo bar"]`, etc
so (into (process/tokenize (first args)) (rest args))
should be the accurate representation
but for usability I think it'd be nice to have an optional function which gets to see the arguments before invocation
Thank you. Yeah, really not a strong point from my end, just that it's probably more accurate if done closer to actual invocation
$ bb -e '(babashka.process/process ["ls" "-la"] {:escape (fn [arg] (prn arg) arg)})'
"ls"
"-la"
This :escape
hook is really there for Windows which has some odd corner cases with process arguments and defaults to some sane behavior. It's applied to every argument
what is the best way to fail all tasks in the chain of dependent tasks, but to keep going with the ones that do not depend?
My tasks would all return an :anomaly object if they fail I guess I wrap everthing with a check
You can control failure with :continue true
which will never throw but you will have to handle the result yourself
can I do (pods/load-pod 'org.babashka/aws "0.1.2")
before the requires somehow?
:tasks...
:requires
([babashka.pods :as pods]
[cos.iap.info.impl.dynamo :as dynamo])
:init
do
(pods/load-pod 'org.babashka/aws "0.1.2")
(require '[pod.babashka.aws :as aws])
doesn't work because I require aws in the dynamo namespace before I loadThe best way to do this currently is probably to move your code to a file and then do load-pod + require in that file.
Hey everyone! Does babashka.fs have access to the Watch Service API?
@U012RPAAR0E It hasn't
Ah ok
within bb, I'm trying to see how to watch for file changes
Preferably with something like inotify
We have two file watcher for bb, one written in Rust and one written in Go: https://github.com/babashka/pod-registry Both should work fine.
Oh excellent! Thank you
I have an example here where I watch sources from the babashka book and rebuild on change, and then refresh a browser to show the new renderings https://github.com/babashka/book/blob/master/script/watch.clj
Can I have a pod dependency in bb.edn?
I’ve made a https://asciinema.org/a/ZnJ0kaaMr2FuRyDgpTqSbgLwA. Now, I am investigating the mechanisms around making a pod. My pod would run some js against a js library by shelling out to node, and thus has a dependency on a js library. Would it make sense to install the dep in the pod’s node_modules (via npm install … ), and just assume node is installed? To me it seems to be asking a lot of the pod user. Another option would be skipping pods entirely — I guess I can just distribute this feature as a directory (js file, and clj file), and let users install node and run npm install. But I feel like there must be a better way than that!
Maybe you could try writing it in #nbb too? that can load arbitrary js dependencies 😄