Fork me on GitHub
#nbb
<
2022-10-19
>
Daniel Gerson13:10:41

Question: What's the easiest way to get node and babashka on an alpine docker image? I know there's a babashka alpine image, but I was using the official node image (and probably should stick with that)... however I need bb in order to get deps from nbb.edn. Anyone have this issue?

borkdude13:10:53

@danielmgerson There are multiple ways. E.g. use a staged docker build and copy the binary from one layer to another, or simply curl-install the bb alpine binary

borkdude13:10:37

I think I would already fetch the deps in the image build, so you won't have to fetch them on execution

Daniel Gerson13:10:48

The following installs but doesn't run. So you're probably saying I should fetch the binaries from the alpine image (which I'll have to look up how to do)

RUN apk update && apk add bash curl
RUN bash < <(curl -s )

borkdude13:10:16

@danielmgerson You need to provide the --static flag to the installer script.

borkdude14:10:05

You can also just download a binary from Github releases, hard-coded

borkdude14:10:13

Use the linux amd64 static one

Daniel Gerson14:10:01

Thanks. Replying in 🧵 . I had to google to figure out how to pass --static but the following seemed to work during image creation. 😅

RUN curl -s  | bash -s -- --static
Alas, I still have the errors connecting via cli and running bb. Do you think downloading it manually will be different? That's next for me to investigate.

Daniel Gerson14:10:18

@U04V15CAJ Presumably the aarch64 is the right one chosen via the --staticcommand below? I wasn't able to see during image creation, but cli'ing in results in the following:

/app # curl -s  | bash -s -- --static
Downloading  to /tmp/tmp.NdhjNE
Moving /usr/local/bin/bb to /usr/local/bin/bb.old
Successfully installed bb in /usr/local/bin
/app # cd /usr/local/bin
/usr/local/bin # ls
bb                    docker-entrypoint.sh  npx
bb.old                node                  yarn
corepack              npm                   yarnpkg
/usr/local/bin # bb
/bin/sh: bb: not found
/usr/local/bin # sh bb
bb: line 1: ELF����@p�@8: not found
bb: line 2: syntax error: unexpected ")"
/usr/local/bin #

borkdude14:10:32

are you sure you have aarch64 architecture?

borkdude14:10:37

and are you on 64bit?

borkdude14:10:49

So, the aarch64 binaries are not fully static and dynamically linked against libc. You can find more info here: https://stackoverflow.com/a/47602147/6264 Or use amd64

Daniel Gerson14:10:12

I'm on a Macbook pro M1, but I thought that docker obscured that? I'm not familiar enough with these bits and pieces, will look at your links shortly.

borkdude14:10:16

@U7ERLH6JX might know something here too

🙏 1
borkdude14:10:35

@danielmgerson If it's an option, using an ubuntu based image might be easier too

Daniel Gerson14:10:36

@U04V15CAJ Yeah, I think you're right. Alpine seems too minimal. Will try. Thanks for the help 👍

lispyclouds09:10:36

@danielmgerson re: but I thought that docker obscured that? on M1/2 macs, docker uses mac's virtualization tech to create the linux vm and hence it advertises itself as linux/arm64. so if you do a docker pull its gonna be pulling for that os/arch combo. since your base image is alpine your probably getting affected by a particular version of glibc not being there. in general the pattern i follow is that if i need to install anything else in an alpine image which isnt there in apk, i'll assume theres a good chance it won run and I'll use a ubuntu base image which is only slightly fatter

Daniel Gerson11:10:13

@U7ERLH6JX Thanks. Yesterday I just removed -alpine to FROM node:18.1.0 and switching to apt-get and installing release-jre it worked.

borkdude13:10:27

as alpine doesn't support dynamically linked binaries

Brice Ruth13:10:55

Question (sorta re-post from a question in #C03S1L9DN, after getting some feedback there) - I’m trying to put together a sample project that would help teams in my org that are working on JS projects (backend, to start) have a path to incorporating CLJS incrementally in their work - I have a similar project in Java land using Gradle/clojurephant/java-interop, where everything can live under the same source tree (you can just start working in src/main/clojure versus src/main/java, for instance). My goal, I guess, was to achieve something similar for CLJS - that is, assuming I have an existing Node.JS-based project, how would I incorporate CLJS incrementally … but, based on what I’m seeing and the feedback I got on my initial question, pointing at #C6N245JGG and #C029PTWD3HR - I’m wondering if a more workable approach is to isolate the CLJS code into it’s own node module and depend on that from the original project, versus trying to make the original project a hybrid of some sort … am I on the right track?

borkdude13:10:14

Is this a Node.js project?

borkdude14:10:46

It can work with #nbb. The way to do this is similar to this: https://github.com/babashka/nbb/blob/main/doc/aws_lambda.md So loadFile is the thing you need to load a .cljs file. There is no compilation there, the .cljs is interpreted. If performance isn't that import for that code, it's ok. There is also a lighter-weight compiler being worked on, named #cherry which is available from npm:

npm install cherry-cljs
npx cherry compile foo.cljs
This emits a foo.mjs file which you can then import from the rest of your CLJS. This is also achievable with #shadow-cljs but you have to use the JVM to produce the JS output there. Similarly there is a project called #squint for which the output doesn't depend on CLJS at all and emits "raw JS" without the immutable data structure, for smaller JS output. You still get to use CLJS syntax and and a library which imitates the core functions in JS.

Brice Ruth14:10:54

> Is this a Node.js project? yes, at least, that’s what I want to demo - doing so for a front-end project looks to be more involved (and I don’t have a simple base to demo from)

Brice Ruth14:10:37

thanks3 - looking at the nbb lambda example now - this looks promising 🙂

Brice Ruth14:10:40

got the nbb example working with loadFile - thanks3

alex17:10:42

We're currently using Playwright with nbb (thanks!) Is there a way to get a more useful stack trace when Playwright throws an exception due to failing test/assertion? Right now we get something like the below -- a bunch of nbb internals.

locator.waitFor: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for selector "audio"
============================================================
    at Xba (file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:312:60)
    at gca (file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:322:137)
    at file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:529:453
    at P (file:///Users/alex/code//ui_tests/node_modules/nbb/lib/nbb_core.js:290:469)
    at file:///Users/alex/code//ui_tests/node_modules/nbb/lib/nbb_core.js:576:419
    at P (file:///Users/alex/code//ui_tests/node_modules/nbb/lib/nbb_core.js:290:469)
    at file:///Users/alex/code//ui_tests/node_modules/nbb/lib/nbb_core.js:577:274
    at P (file:///Users/alex/code//ui_tests/node_modules/nbb/lib/nbb_core.js:290:469)
    at file:///Users/alex/code//ui_tests/node_modules/nbb/lib/nbb_core.js:383:307
    at file:///Users/alex/code//ui_tests/node_modules/nbb/lib/nbb_promesa.js:39:342 {
  name: 'TimeoutError'
}

borkdude17:10:45

@UGGU8TSMC There is a way to do that

alex17:10:59

That's what I was hoping to hear! I'd be happy to dig into it if you could provide a pointer 🙂

borkdude17:10:17

Haha, sorry, I was a bit distracted

borkdude17:10:28

Let me whip up an example

❤️ 1
borkdude18:10:08

How are you running this, as a script or from the REPL?

alex18:10:41

I'm running this as a script via bb task

borkdude18:10:25

I'm trying to reproduce this here: https://github.com/babashka/nbb/commit/98bd4a86c38931a3a87bd59d4fa3f8ed0008f9c6 but the stacktrace isn't yet showing up

borkdude18:10:50

can you maybe make a repro of this?

alex18:10:10

Sure! Give me one moment

alex18:10:14

If you add a p/catch (before p/finally ) for the TimeoutError thrown by Playwright, we can log the error/stacktrace

(p/catch (fn [err]
                    (js/console.log err)
                    (is false)))

alex18:10:44

I copied your test to my test file and added that block to generate the stacktrace

alex18:10:18

=== borkdude-test

locator.waitFor: Timeout 100ms exceeded.
=========================== logs ===========================
waiting for selector "audio" to be visible
============================================================
    at Xba (file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:312:60)
    at gca (file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:322:137)
    at file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:529:453
    at P (file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:290:469)
    at file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:576:419
    at P (file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:290:469)
    at file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:577:274
    at P (file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:290:469)
    at file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_core.js:383:307
    at file:///Users/alex/code/ui_tests/node_modules/nbb/lib/nbb_promesa.js:39:342 {
  name: 'TimeoutError'
}

borkdude18:10:21

but if you log the err, isn't that stacktrace expected?

borkdude18:10:07

I do get your wish to get a better stacktrace there. Let me see. With sync code this is possible now by async may complicate this...

alex18:10:45

> but if you log the err, isn't that stacktrace expected? I guess I'm not 100% sure what I expected here. I would hope that the TimeoutError stacktrace shows the Playwright code that resulted in the error. Are you saying the stacktrace is expected due to the p/let macro + nbb?

borkdude18:10:06

The trace you're seeing is showing what JS was executing at the moment of the error, which is nbb itself, which is interpreting your script. There is something you can do about this in user space: https://github.com/babashka/nbb/commit/c30d9315d42347cc6a69d04b1ec659ecdc62a34b

💡 1
borkdude18:10:36

It might not be so automatic as you wish it to be but at least you get reasonable locations for your error

🙌 1
borkdude18:10:49

Of course you could wrap this in another macro too

alex18:10:47

Oh cool! TIL about (meta &form)

alex18:10:14

Does this macro need to go into a separate macro namespace? I recall that's the pattern I've used for other macros in cljs space

borkdude18:10:52

you can place macros anywhere you want

alex18:10:33

Ah right, thanks. This is really helpful

alex18:10:14

Interestingly the macro's console.error doesn't log anything for me, but after switching to console.log, it works fine

borkdude18:10:28

oh weird. might be your node version

borkdude18:10:32

what version are you on?

borkdude18:10:54

I'm on 17.8.0

alex18:10:25

node 16.14.2

borkdude18:10:15

I'll change the example to log then

alex18:10:23

no worries - I can look into that separately

borkdude18:10:43

I updated this in the example on main now

🙌 1
Daniel Gerson19:10:35

There's always the Java version of Playwright from Clojure... :face_with_peeking_eye:

😄 1
💡 1
alex20:10:58

> Of course you could wrap this in another macro too Just realized that the test output prints the line number where (print-error error) is invoked, rather than the assertion/test form that generated the error. In order to get the actual form that generates the error, I'm thinking I'll need to write a macro, that among other things, attaches (p/catch print-error) to each async form?

borkdude20:10:07

Yes. If the code were synchronous this would work much better (e.g. in bb) but async makes this harder. As an aside, I'm also working on #cherry (CLJS compiler which works in JS) and #squint (same compiler basically, but outputs direct JS without CLJS, so mutable objects all the way down). Both support js/await and ^:async functions. The error reports from those would be much better (with proper source map support). They are a bit experimental though, although some people have been using them already.

borkdude20:10:58

Both are also supposed to get REPL support (but this part is immature right now)

borkdude20:10:27

What I meant earlier is: if the code is sync, e.g. babashka has a way for the user to get access to the interpreter stacktraxce

borkdude20:10:32

E.g. try this script with the newest bb:

(ns dude
  (:require [sci.core :as sci]))

(defn foo []
  (try (/ 1 0)
       (catch ^:sci/error Exception e
         (run! prn (sci/format-stacktrace (sci/stacktrace e))))))

(foo)

borkdude20:10:49

$ bb /tmp/foobar/dude.clj
"clojure.core// - <built-in>"
"dude           - /tmp/foobar/dude.clj:5:8"

borkdude20:10:15

This works because SCI controls try/catch

alex21:10:08

> As an aside, I'm also working on #C03QZH5PG6M (CLJS compiler which works in JS) and #C03U8L2NXNC (same compiler basically, but outputs direct JS without CLJS, so mutable objects all the way down). > Both support js/await and ^:async functions. The error reports from those would be much better (with proper source map support). They are a bit experimental though, although some people have been using them already. That sounds cool! Is the compilation time negligible with #C03U8L2NXNC i.e. comparable to nbb? Also would it be separate or used in conjunction with nbb? Since it compiles to JS, it would no longer need the nbb interpreter right?

borkdude21:10:05

It compiled pretty fast. Just try it with npx squint run foo.cljs

borkdude21:10:08

$ cat package.json
{
  "dependencies": {
    "squint-cljs": "^0.0.0-alpha.49"
  },
  "type": "module"
}

 $ cat foo.cljs
(println "Hello world")

$ npx squint run foo.cljs
[squint] Running foo.cljs
Hello world

alex21:10:28

I'm playing around with your plet macro to enhance it with the p/catch print-error logic, but I'm admittedly a bit out of my depth w/ macros

(defmacro plet
  [bindings & body]
  (let [binding-pairs (reverse (partition 2 bindings))
        body (cons 'do body)]
    (reduce (fn [body [sym expr]]
              (let [expr (list '.resolve 'js/Promise expr)
                    err (gensym)]
                (list '.catch
                      (list '.then expr (list 'clojure.core/fn (vector sym)
                                              body))
                      (list 'clojure.core/fn (vector err)
                            `(do
                               (print-error ~err)
                               ;; TODO rethrow the error but pass a modified
                               ;; data structure so other `catch` handlers know
                               ;; the error has already been printed?
                               (p/rejected ~err))))))
            body
            binding-pairs)))
When I replace the p/let with plet in your test example, the (meta &form) calls seem to return nil so line/col is nil. Do you know why that might be?

alex21:10:57

Also wondering if I'm sort of heading down a dead end. Once the macro expansion happens, there's no telling if the line/col #s will match up to the actual source forms, right?

borkdude21:10:27

The issue here is that line and col are only added by the forms that are read by the reader, but they aren't part of macro-expansions, unless you explicitly take them from the original forms and add them to the expanded forms

borkdude21:10:18

so body. sym and expr do have metadata

borkdude21:10:28

but your generated call to print-error doesn't

borkdude21:10:11

gotta go now, sleep time

alex21:10:19

Gotcha! So it sounds like I just need to extract from the original forms and potentially pass them along. I'll noodle on this a bit more. Appreciate the pointers 🙂

alex20:10:53

I ended up with the following, and it seems to be working well. Thanks again for your direction!

(defmacro catch-line-col-and-reject
  "Attaches a promise catch handler that enhances an error map w/the line &
  column numbers of the thrown/rejected `expr`, then rejects the error map.
  If the `error#` already has a `:line`, we reject so it can be handled
  downstream.
  "
  [p expr]
  (let [{:keys [line column]} (meta expr)]
    `(p/catch ~p
              (fn [error#]
                (if (:line error#)
                  (p/rejected error#)
                  (let [new-error {:line ~line :column ~column :error error#}]
                    (p/rejected new-error)))))))

(defmacro do'
  "Modified from `p/do!` to attach `catch-print-reject` on each of the promise forms."
  [& exprs]
  `(pt/-bind
    (pt/-promise nil)
    (fn [_#]
      ~(condp = (count exprs)
         0 `(pt/-promise nil)
         1 `(catch-line-col-and-reject (pt/-promise ~(first exprs)) ~(first exprs))
         (reduce (fn [acc e]
                   `(catch-line-col-and-reject (pt/-bind (pt/-promise ~e) (fn [_#] ~acc)) ~e))
                 `(catch-line-col-and-reject (pt/-promise ~(last exprs)) ~(last exprs))
                 (reverse (butlast exprs)))))))

(defmacro let'
  "Modified from `p/let` to attach `catch-line-col-and-reject` on each of the promise forms."
  [bindings & body]
  `(pt/-bind
    (pt/-promise nil)
    (fn [_#]
      ~(c/->> (reverse (partition 2 bindings))
              (reduce (fn [acc [l r]]
                        `(catch-line-col-and-reject (pt/-bind (pt/-promise ~r) (fn [~l] ~acc)) ~r))
                      `(do' ~@body))))))

alex20:10:51

I'm keeping these macros in a util file. If I import the macros for use in test files, is there a nice way to determine the file name of those test files from within this macro?

alex20:10:01

Or do I need to pass those in explicitly via *file*?

borkdude20:10:11

You should be able to read *file* during macroexpansion: it should have the name of your test file

borkdude20:10:25

So not in the expansion itself, but while it's expanding

borkdude20:10:53

$ nbb
Welcome to nbb v1.0.137!
#'user/foo
user=> (defmacro foo [] (str "from: " *ns*))
#'user/foo
user=> (ns bar)
#object[dr bar]
bar=> (defn bar-fn [] (user/foo))
#'bar/bar-fn
bar=> (bar-fn)
"from: bar"

borkdude20:10:54

When defining bar-fn the macro user/foo is called, so at that moment *ns* is bound to the namespace bar. Similar to *file*

🙌 1
alex20:10:00

Ah, way easier than I was imagining! Thanks 😄

Daniel Gerson20:10:32

There might have been a regression with bb for nbb.edn, or perhaps I'm doing it wrong.

cat nbb.edn
{
 :paths ["."
         ]
 :deps {
        }
 }
which yields
Downloading dependencies...
Exception in thread "main" java.lang.Exception: The uberjar task needs a classpath.
	at babashka.main$exec.invokeStatic(main.clj:1023)
	at babashka.main$main.invokeStatic(main.clj:1082)
	at babashka.main$main.doInvoke(main.clj:1052)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at babashka.main$_main.invokeStatic(main.clj:1115)
	at babashka.main$_main.doInvoke(main.clj:1107)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at babashka.main.main(Unknown Source)
An error occurred when calling (nbb.impl.deps/init)
Command failed: bb --config .nbb/.cache/bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f/deps.edn uberjar .nbb/.cache/bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f/nbb-deps.jar
Exception in thread "main" java.lang.Exception: The uberjar task needs a classpath.
	at babashka.main$exec.invokeStatic(main.clj:1023)
I suspect https://github.com/babashka/babashka/commit/020d42a94fab2a2f80abd15fa05b19b355b1b545 but unsure. Tried checking out bb but failed to get uberjar-test to run so thought I'd just post here 😇

borkdude20:10:15

It seems to happen when the :deps map is empty

Daniel Gerson20:10:43

Just realised I ran the babashka repl wrong (re my attempt to fix it), so ignore that bit. Thanks for looking into it.

Daniel Gerson20:10:25

Once I deleted .nbb and upgraded bbit happened. Just upgrading doesn't make it happen from what I can see. Moving stuff around while making a docker image made me come across this.

borkdude20:10:37

Hmm yeah. I think the classpath is empty when you have no deps:

$ bb --config .nbb/.cache/bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f/deps.edn -e '(prn (babashka.classpath/get-classpath))'
nil
and this is what trips the uberjar command

borkdude20:10:02

instead of nil we probably have to use an empty string or so

borkdude20:10:48

Thanks for reporting, I'll fix this in the next bb

borkdude20:10:16

As a workaround, remove :deps or put some deps in it

🙌 1
Daniel Gerson20:10:26

Ah thanks. I had one dep that I worked out wasn't necessary so removed. Still was using the :paths (but removed both for reporting).