Fork me on GitHub
#babashka
<
2022-11-07
>
Matthew Twomey00:11:59

Hi all - I’m having trouble getting this script to run in babashka, when in a future:

#!/usr/bin/env bb

(ns test2)

(require '[com.grzm.awyeah.client.api :as aws]
         '[com.grzm.awyeah.credentials :as credentials])

(defn get-tasks
  [& {:keys [profile], :or {profile "default"}}]
  (aws/invoke (aws/client
                {:api 'dms,
                 :credentials-provider (credentials/profile-credentials-provider "default")})
              {:op :DescribeReplicationTasks}))

(defn main []
  (print @(future (get-tasks :profile "default"))))
  ;; (print (get-tasks :profile "default")))

(main)
This runs fine outside a future and it also runs fine in a future - in Clojure without babashka. I’m not sure if there is another step I need to take here, or how to interpret the error. The response I get is:
----- Error --------------------------------------------------------------------
Type:     java.util.concurrent.ExecutionException
Message:  java.lang.IllegalStateException: Can't set!: clojure.core/*warn-on-reflection* from non-binding thread
Location: /private/tmp/t/src/./test2.bb:16:10

----- Context ------------------------------------------------------------------
12:                  :credentials-provider (credentials/profile-credentials-provider "default")})
13:               {:op :DescribeReplicationTasks}))
14:
15: (defn main []
16:   (print @(future (get-tasks :profile "default"))))
             ^--- java.lang.IllegalStateException: Can't set!: clojure.core/*warn-on-reflection* from non-binding thread
17:   ;; (print (get-tasks :profile "default")))
18:
19: (main)

Matthew Twomey00:11:20

Lastly, and strangest of all (to me): If I simply add an additional declaration into the file, it runs in a future, in babashka just fine:

(def not-used (aws/client
                {:api 'dms,
                 :credentials-provider (credentials/profile-credentials-provider "default")}))

borkdude09:11:01

@U0250GGJGAE Aha! I might have an idea

borkdude09:11:03

This is a repro:

bb -e '@(future (set! *warn-on-reflection* true))'

borkdude09:11:21

With Clojure you get the same thing:

$ clj -M -e '@(future (set! *warn-on-reflection* true))'
Execution error (IllegalStateException) at user/eval1$fn (REPL:1).
Can't set!: *warn-on-reflection* from non-binding thread

borkdude09:11:00

My suspicion is that there is a dynamic require which is then happening inside the future

borkdude09:11:34

but if you do the dynamic require outside the future once, you won't have the problem inside the future anymore since the namespace is already loaded

borkdude11:11:58

I narrowed it down a bit:

$ clojure -M -e "(prn @(future (load-string \"(set! *warn-on-reflection* true)\"))) (shutdown-agents)"
true
This works in Clojure But this doesn't:
$ clojure -M -e "(prn @(future (set! *warn-on-reflection* true))) (shutdown-agents)"
Now I need to find out why load-string does accept this while literally executing the expression doesn't

borkdude11:11:25

it's because of this:

borkdude11:11:38

You can work around this with:

(require '[com.grzm.awyeah.client.api :as aws]
         '[com.grzm.awyeah.credentials :as credentials]
         '[clojure.spec.alpha]
         '[com.grzm.awyeah.http-client]
         '[com.grzm.awyeah.http.awyeah]
         '[com.grzm.awyeah.protocols.json])

Matthew Twomey16:11:05

Interesting - thanks so much @U04V15CAJ. Yes, just adding protocols.json seems to fix it for me. I really appreciate you taking a look sir!

borkdude16:11:07

Another workaround:

(defn get-tasks
  [& {:keys [profile], :or {profile "default"}}]
  (aws/invoke (aws/client
               {:api 'dms,
                :credentials-provider (credentials/profile-credentials-provider "default")})
              {:op :DescribeReplicationTasks}))

(defn main []
  (print @(future
            (binding [*warn-on-reflection* *warn-on-reflection*]
              (get-tasks :profile "default")))))

borkdude16:11:19

So establish the root binding of the dynamic var inside of the future first

Matthew Twomey16:11:03

Nice - I have not yet dealt with the binding function (relatively new to Clojure). Any pros & cons between the two workarounds, or really just pick one?

borkdude16:11:11

just pick one

Matthew Twomey16:11:25

👍 thanks again

borkdude16:11:29

why are you using a future to call this function?

Matthew Twomey16:11:33

The example I gave was a distilled down version of a monitoring utility that calls a whole bunch of remote APIs and endpoints. So they run in parallel then report results.

Matthew Twomey16:11:18

(I tore everything out to try to narrow down the problem)

borkdude16:11:23

makes sense

Matthew Twomey16:11:27

I’m really loving Clojure - we also use nodejs a lot where I work. So it’s been a fun challenge to decide which patterns make sense to carry into Clojure and which should be rethought. For example, I really like the “pretty much never blocking” nature of nodejs. So I’m tending to develop for Clojure in a similar sense when it comes to API calls. I’m still figuring out what does and doesn’t work well. I like that with futures + immutability I can develop something non-blocking and multi-threaded in Clojure - without too much hassle. That’s pretty awesome.

borkdude16:11:40

@U0250GGJGAE Cool. There is also #C029PTWD3HR for scripting on Node.js

Matthew Twomey16:11:57

Oh cool - I hadn’t run across that!

borkdude16:11:53

If you run npx nbb you're dropped into an nbb (Clojure, SCI) REPL

borkdude16:11:09

I'm just mentioning this because you mentioned you're familiar with Node.js

Matthew Twomey16:11:10

We’re running our own APIs in node, I haven’t yet had a moment to think on consider cljs - but it is definitely on my list to explore!

Matthew Twomey16:11:39

We have a lot of SPAs that could possibly take advantage.