Fork me on GitHub
#babashka
<
2021-04-21
>
Karol W贸jcik10:04:10

We're talking with @viesti about holy-lambda support for babashka runtime. I would love to provide the same interface for babashka as I do for both Java and native runtime. My questions are: 1. Is it possible to pack some macros in babashka pod? If it's possible then on weekend I would release holy-lambda with babashka support 馃槃 2. Can babashka artifacts be packed in zip?

馃憤 2
borkdude10:04:01

@karol.wojcik Yes, it is possible to define macros in pods. This is done using :code which defines "client side" code.

borkdude10:04:52

> Can聽babashka聽artifacts be packed in zip? Not sure what you're asking here. I will go for a run but will be back later.

Karol W贸jcik10:04:43

I want to provide holy-lambda.pod which exposes all the functions and macros to babashka. As I understand it babashka might be packed in the zip, but may I pack the dependencies in zip as well? If yes then in what format should I keep those dependencies?

borkdude11:04:51

@karol.wojcik Let's back up a bit. What does unholy lambda do that it needs to be a pod instead of, let's say, a script?

viesti11:04:24

might be that @karol.wojcik realised that on #holy-lambda lambda, but I'll let him explain (explaining out leaves mental tracks :))

Karol W贸jcik11:04:52

I don't know much about babashka. I've used very basic functionality of it to speedup docker builds in my company. @viesti said that babashka can read namespaces from .jar file as well, so maybe it would work that way. Still though don't know how to provide .jar dependencies to babashka.

borkdude11:04:57

@karol.wojcik It depends what kinds of dependencies. Not all Clojure dependencies are compatible with bb. So that's why I asked the above question.

borkdude11:04:20

You can run jar files with bb -cp foo.jar -m foo for example

Karol W贸jcik11:04:29

Thanks. I can see that clojure.lang.RT is not supported.

Karol W贸jcik11:04:44

I will create a following patch:

#?(:bb
   (defn- process-interceptors
     [mixin payload type]
     (if-let [interceptors (seq (:interceptors mixin))]
       (loop [interceptors interceptors
              result payload]
         (if-not (seq interceptors)
           result
           (recur (rest interceptors) (if-let [interceptor (type (first interceptors))] (interceptor result) result))))
       payload))
   :clj
   (defn- process-interceptors
     [?mixin ?payload ?type]
     (if-let [it (some-> ?mixin :interceptors clojure.lang.RT/iter)]
       (loop [result ?payload]
         (if (.hasNext it)
           (recur
            (if-let [interceptor (some-> (.next it) ?type)]
              (interceptor result)
              result))
           result))
       ?payload)))

borkdude11:04:47

yeah that makes sense

borkdude11:04:42

You can suggest that you need RT for this iter method in that issue, it's pretty much the nr 1 reason why people depend on RT...

Karol W贸jcik11:04:52

Ok. Babasha fails on including jsonista 馃槥

----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  Unable to resolve classname: jsonista.jackson.DateSerializer
Location: jsonista/core.clj:50:3

----- Context ------------------------------------------------------------------
46:   jsonista uses Jackson Databind while Cheshire uses Jackson Core. In our
47:   benchmarks, jsonista performs better than Cheshire (take look at
48:   json_perf_test.clj). On the other hand, Cheshire has a wider set of features
49:   and has been used in production much more."
50:   (:import
      ^--- Unable to resolve classname: jsonista.jackson.DateSerializer
51:     (jsonista.jackson
52:       DateSerializer
53:       FunctionalKeyDeserializer
54:       FunctionalSerializer
55:       KeywordSerializer

borkdude11:04:09

babashka has cheshire.core, not jsonista

Karol W贸jcik11:04:42

Will it change in future?

Karol W贸jcik11:04:03

Ok I will just use reader conditionals to use cheshire instead

馃憤 2
borkdude11:04:09

Well, I probably won't remove it since that will be breaking, but perhaps add more options later, not sure.

borkdude11:04:38

To save you some time, what other deps are you using?

borkdude11:04:49

Then I can tell you if that will work or not

Karol W贸jcik11:04:43

It's all I'm using.

Karol W贸jcik11:04:04

Probably in 30 minutes I will have babashka example working.

Karol W贸jcik12:04:50

@borkdude May i ship babashka with holy-lambda?

Karol W贸jcik12:04:59

@borkdude What is some babashka internal thing that I can prove it's running on babashka?

Karol W贸jcik13:04:48

Ok will use babashka.main.version 馃槃

Karol W贸jcik13:04:17

Huh. babashka.main is not available in scripts? 馃槷

borkdude13:04:50

You can do (System/getProperty babashka.version) (sorry typing on phone)

Karol W贸jcik15:04:43

May I please announce the new version here as well @borkdude? Just once to let people know about babashka runtime :)

Karol W贸jcik15:04:43

Thank you so much 馃檪

Karol W贸jcik13:04:09

Not officially announcing yet, but holy-lambda with some minor tweaks can run on either java, native or babashka 馃檪 Thanks @borkdude @viesti

(ns babashka-example.core
  (:gen-class)
  (:require
   [fierycod.holy-lambda.retriever :as r]
   [fierycod.holy-lambda.interceptor :as i]
   [fierycod.holy-lambda.agent :as agent]
   [fierycod.holy-lambda.native-runtime :as native]
   [fierycod.holy-lambda.response :as hr]
   [fierycod.holy-lambda.core :as h]))

(i/definterceptor LogIncommingRequest
  {:enter (fn [request]
            (println "Log incomming request:" request)
            request)})

(h/deflambda HelloBabashka <
  {:interceptors [LogIncommingRequest]}
  [{:keys [ctx]}]
  #?(:bb
     (hr/text (str "Babashka is sweet friend of mine! Babashka version: " (System/getProperty "babashka.version")))
     :clj (hr/text "Why you didn't use babashka? ;/")))


(native/entrypoint [#'HelloBabashka])

(agent/in-context
 (println "In agent context"))
https://bdsa1uru44.execute-api.eu-central-1.amazonaws.com/Prod/

馃憤 3
Karol W贸jcik13:04:28

Babashka is well implemented I must say. It was just easy to provide such runtime. You're doing great work @borkdude

Dig14:04:45

Ah yes that is the one I was thinking about, reading tutorial.

Karol W贸jcik14:04:59

There is not much yet in tutorial. Would love to work on it more, but my time is very limited 馃槥

Karol W贸jcik15:04:02

Just checked babashka tasks. @borkdude I would love to have them in the project. Is it possibile to provide tasks in jar then somehow automatically merge user and jar tasks?

borkdude15:04:05

Right now bb tasks is only supports in bb.edn

Karol W贸jcik16:04:52

Ok got it. Will manualy copy bb.edn from one repository to another.

Karol W贸jcik15:04:00

As per @borkdude permission, I'm cross-posting this announcement here as well. If you interested in babashka layer then please put babashka in official announcement. Hope that #holy-lambda will make some new friends in #babashka community 馃檪

babashka 3
borkdude16:04:24

Did some more automation, this time around the clj-kondo LSP server: https://github.com/clj-kondo/clj-kondo.lsp/blob/master/bb.edn I can't believe I did many of these steps manually, it's all very tedious. And almost every step depends on a previous step so a perfect fit for tasks. Also used fs/modified-since again to not rebuild the jar if nothing changed.

馃憤 4
madstap16:04:05

There's a babashka asdf plugin, but the 0.3.* releases don't work. Is this something that should be fixed in the asdf plugin? The code is here: https://github.com/fredZen/asdf-babashka/blob/master/lib/utils.bash#L34

$ asdf install babashka latest
* Downloading babashka release 0.3.5...
curl: (22) The requested URL returned error: 404
asdf-babashka: Could not download 
asdf-babashka: An error ocurred while installing babashka 0.3.5.

$ asdf install babashka 0.2.1
* Downloading babashka release 0.2.1...
Archive:  /home/aleks/.asdf/installs/babashka/0.2.1/babashka-0.2.1.zip
  inflating: /home/aleks/.asdf/installs/babashka/0.2.1/bin/bb  
babashka 0.2.1 installation was successful!

borkdude17:04:14

@madstap Might be related to a migration from .zip to .tar.gz

madstap17:04:50

False alarm, someone fixed it (just an hour ago)

borkdude17:04:39

Nice co-incidence

madstap17:04:17

Might be worth mentioning asdf in the install section of the readme?

borkdude17:04:11

Feel free to PR anything that's worth mentioning there

馃憤 2
russmatney21:04:03

Is it possible to exclude files from being included in a jar via the bb uberjar command?

russmatney21:04:18

I'm moving from an uberscript to an uberjar to make use of pods in a bb-based tool. Building with:

bb uberjar clawe.jar -cp $(clojure -A:remove-clojure -Spath) -m clawe.core
The jar seems to include everything in the build directory, including .git history. I'm hopeful there won't be a performance hit, so want to reduce the jar size as much as possible, but this feels more like a bug. I did find this https://github.com/babashka/babashka/issues/780 which might be relevant.

russmatney21:04:05

I can take a shot at a reproduction or a PR if it's helpful - not sure what the expected behavior is

borkdude21:04:39

@UGNFWPPGF you should set the classpath before the uberjar command probably

borkdude21:04:05

bb -cp $(clojure -A:remove-clojure -Spath) uberjar clawe.jar  -m clawe.core
can you try that?

russmatney21:04:31

----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  File does not exist: uberjar

russmatney21:04:41

I tried that first to match the docs, but i get this error

borkdude21:04:46

ok, try this:

bb --classpath server/src --uberjar foo.jar

borkdude21:04:51

this is the old style

borkdude21:04:11

clearly something is wrong, please add to the issue

馃憤 1
borkdude21:04:16

and PR is welcome to fix it

russmatney21:04:27

that seems to build properly, and more reasonably sized - i'm seeing a number of warnings for clashing jar items, which is interesting

{:warning "clashing jar item", :path ".cpcache/1246900521.cp", :strategy :noop}
{:warning "clashing jar item", :path ".cpcache/0DF01609C7D30D4D8B3579B5256F3E83.libs", :strategy :noop}
{:warning "clashing jar item", :path ".cpcache/1246900521.basis", :strategy :noop}
...

borkdude21:04:46

try removing the jar file before uberjaring again

馃憤 1
russmatney21:04:49

It could also be a strange path being built on my part - when i run

bb -cp src/clawe --uberjar clawe.jar -m clawe.core
it builds almost instantly, and the result seems to work fine (and is fast!)

russmatney21:04:44

this is great - the uberjar is at least as fast as the uberscript

bb clawe.jar hello  0.11s user 0.03s system 102% cpu 0.139 total
vs
bb clawe-script.clj hello  0.12s user 0.05s system 103% cpu 0.160 total
very happy to learn that this will work

russmatney21:04:52

thanks for the debugging help @borkdude!

borkdude21:04:24

ok. I see the command line parsing in the tests is different from the help

russmatney21:04:36

I figured out why it was adding everything - my clojure -Spath had a few empty entries in it :: (`src::/home/russ/russmatney/ralphie/src:/etc` ), which seemed to add the entire current directory

borkdude21:04:38

in the tests it accepts --classpath as an argument to the uberjar task

borkdude21:04:05

oh that's another bug then probably

borkdude21:04:22

please comment all your problems to the issue

russmatney22:04:41

Issue updated - the new info is that it looks like the 'blanks' in the classpath are coming from the classpath-overrides

{:aliases
 {:remove-clojure
  {:classpath-overrides
   {org.clojure/clojure          nil
    org.clojure/spec.alpha       nil
    org.clojure/core.specs.alpha nil}}}
==> resulting path:
src::/home/russ/russmatney/ralphie/src:::/home/russ/.m2/repository/cheshire/cheshire/5.10.0/cheshire-5.10.0.jar:/home/russ/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.10.2/jackson-core-2.10.2.jar:/home/russ/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.10.2/jackson-dataformat-cbor-2.10.2.jar:/home/russ/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.10.2/jackson-dataformat-smile-2.10.2.jar:/home/russ/.m2/repository/tigris/tigris/0.1.2/tigris-0.1.2.jar

borkdude22:04:53

@UGNFWPPGF Note that you can now also set the classpath in bb.edn under :paths

borkdude22:04:07

but we should still fix this regardless

russmatney22:04:16

gotcha - i'll move over to use that