babashka 2025-05-09

I have a newly-published open source utility published by my employer, which is very exciting for me! It's a babashka pod, written in Typescript, which allows querying Neptune graph databases. Here is the link: https://github.com/SouthwestAir/queryblast

🎉 8

Wow, this might be the very first pod written in JavaScript / TypeScript! :)

❤️ 3

Thanks! I'm very pleased about it! I would like to add this pod to the pod-registry, if at all possible

and I can see that I'll need to publish a release first, before I can add it to the pod-registry

so in any case, I still have a bit more work to do

but I'm very excited about today's milestone anyway!

so far the pod registry makes sense if you produce standalone binaries for several OS-es. I haven't really thought about how to combine this with JS

having manifests in the pod registry point you to where bb can download the binaries

I guess we could make something for "pod scripts" perhaps by just adding them to the classpath instead of doing it via the pod registry

not sure, haven't thought about this deeply

I've got this working locally now:

(require '[babashka.pods :as pods])

(pods/load-pod ".esbuild/.build/src/queryblast.js")

(require 'queryblast)

(prn (ns-publics 'queryblast))

I tried compiling to a self-contained executable using bun and deno but their netstring libraries aren't working properly like the one in node, I think maybe a flushing issue or so

Ooh nice progress, thanks for your help & attention

this could be another way to do it on node but looks complicated: https://nodejs.org/api/single-executable-applications.html

Wow and using the code from yamlscript!

this tool works for sure, it even works for non-node script runners like babashka: https://github.com/AppThreat/caxa

works with caxa:

npx caxa --input .esbuild/.build/src --output mypod  -- "{{caxa}}/node_modules/.bin/node" "{{caxa}}/queryblast.js"
(require '[babashka.pods :as pods])

(pods/load-pod "./mypod")
#_(pods/load-pod ".esbuild/.build/src/queryblast.js")

(require 'queryblast)

(prn (ns-publics 'queryblast))

oh this is rad, I'll try this now

this yields a 34mb binary which is ok in the world of pods

(it just packages the node binary along with the script)

this just returns nil without any error, don't know if that is expected:

(prn (queryblast/query! "MATCH (n) RETURN n LIMIT 1"))
I haven't set any env vars

this is after you loaded it in the babashka repl, is that right?

just a script:

(require '[babashka.pods :as pods])

(pods/load-pod "./mypod")
#_(pods/load-pod ".esbuild/.build/src/queryblast.js")

(require 'queryblast)

(prn (ns-publics 'queryblast))

(prn (queryblast/query! "MATCH (n) RETURN n LIMIT 1"))

Thanks for pointing that out, that nil response behavior could be improved with a usage string describing the required environment variables

I'll be making some changes, for producing the release binary and getting it ready to add to the pod registry, can I use this bb script and your other suggestions?

I think I know what's wrong when running the pod on deno. I ran into this with #nbb + bencode as well for nREPL: https://github.com/babashka/nbb/commit/5f800c95068b2ad4f02cbb13bbc01f157e4fb740 This thing here: https://github.com/isolomak/bencodec/blob/bd0ab57131bce2f2259e77f5866c9a6d663489c6/src/BencodeEncoder.ts#L70 doesn't work in deno. So perhaps if we change this we can compile to self-contained executables in deno.

Does Deno make a better binary than caxa? I'm only mildly familiar with Deno

don't know yet

I made a single JS file with esbuild and then compiled it to an exec with deno. This is just for the bencode demo.

$ ./standalone
5:hello
hello
The binary is 70mb, a bit big

anyway it's good to have options

with bun I get:

57669360

as the output size

last time I couldn't get either to work due to the bugfix above

with bun (and deno) it seems very easy to build multiplatform executables:

bun build --compile --target=bun-linux-x64 ./standalone.js

oh yeah in that case, since we'll need those executables on various platforms, this is a great step

thanks for making the MR!

I hope the maintainer takes note of it

bencode isn't that complicated though, it's just 80 or so lines in clojure. this even works in squint: https://squint-cljs.github.io/squint/?src=KG5zIG5iYi5pbXBsLmJlbmNvZGUKICAiQmVuY29kZSBzdXBwb3J0LCB0YWtlbiBmcm9tIGh0dHBzOi8vZ2l0aHViLmNvbS9kamJsdWUvbnJlcGwtY2xqcy9ibG9iL21hc3Rlci9zcmMvbnJlcGwvYmVuY29kZS5jbGpzIikKCihkZWZuLSBpbmRleC1vZiBbcyBjXQogIChsZXQgW2kgKC5pbmRleE9mIHMgYyldCiAgICAoaWYgKDwgaSAwKSAodGhyb3cgKGpzL0Vycm9yLiAib3V0IG9mIGlucHV0IikpIGkpKSkKCihkZWZuLSBzbGljZQogIChbYnVmZmVyIHN0YXJ0XQogICAoaWYgKDwgKC4tbGVuZ3RoIGJ1ZmZlcikgc3RhcnQpCiAgICAgKHRocm93IChqcy9FcnJvci4gIm91dCBvZiBpbnB1dCIpKQogICAgICguc2xpY2UgYnVmZmVyIHN0YXJ0KSkpCiAgKFtidWZmZXIgc3RhcnQgZW5kXQogICAoaWYgKD4gZW5kICguLWxlbmd0aCBidWZmZXIpKQogICAgICh0aHJvdyAoanMvRXJyb3IuICJvdXQgb2YgaW5wdXQiKSkKICAgICAoLnNsaWNlIGJ1ZmZlciBzdGFydCBlbmQpKSkpCgooZGVmbi0gZGVjb2RlLXJlY3VyIFtkYXRhIG9wdHNdCiAgKGNhc2UgKHN0ciAoc2xpY2UgZGF0YSAwIDEpKQogICAgImkiCiAgICAobGV0IFtkYXRhIChzbGljZSBkYXRhIDEpCiAgICAgICAgICBpIChpbmRleC1vZiBkYXRhICJlIildCiAgICAgIFsoanMvcGFyc2VJbnQgKHNsaWNlIGRhdGEgMCBpKSkKICAgICAgIChzbGljZSBkYXRhIChpbmMgaSkpXSkKICAgICJsIgogICAgKGxldCBbZGF0YSAoc2xpY2UgZGF0YSAxKV0KICAgICAgKGxvb3AgW2RhdGEgZGF0YSB2ICh0cmFuc2llbnQgW10pXQogICAgICAgIChpZiAoPSAoc3RyIChzbGljZSBkYXRhIDAgMSkpICJlIikKICAgICAgICAgIFsocGVyc2lzdGVudCEgdikgKHNsaWNlIGRhdGEgMSldCiAgICAgICAgICAobGV0IFtbdmFsdWUgZGF0YV0gKGRlY29kZS1yZWN1ciBkYXRhIG9wdHMpXQogICAgICAgICAgICAocmVjdXIgZGF0YSAoY29uaiEgdiB2YWx1ZSkpKSkpKQogICAgImQiCiAgICAobGV0IFtkYXRhIChzbGljZSBkYXRhIDEpCiAgICAgICAgICB7OmtleXMgW2tleXdvcmRpemUta2V5c119IG9wdHNdCiAgICAgIChsb29wIFtkYXRhIGRhdGEgbSAodHJhbnNpZW50IHt9KV0KICAgICAgICAoaWYgKD0gKHN0ciAoc2xpY2UgZGF0YSAwIDEpKSAiZSIpCiAgICAgICAgICBbKHBlcnNpc3RlbnQhIG0pIChzbGljZSBkYXRhIDEpXQogICAgICAgICAgKGxldCBbW2sgZGF0YV0gKGRlY29kZS1yZWN1ciBkYXRhIG9wdHMpCiAgICAgICAgICAgICAgICBbdiBkYXRhXSAoZGVjb2RlLXJlY3VyIGRhdGEgb3B0cykKICAgICAgICAgICAgICAgIGsgKGlmIGtleXdvcmRpemUta2V5cyAoa2V5d29yZCBrKSBrKV0KICAgICAgICAgICAgKHJlY3VyIGRhdGEgKGFzc29jISBtIGsgdikpKSkpKQogICAgKGxldCBbaSAoaW5kZXgtb2YgZGF0YSAiOiIpCiAgICAgICAgICBuIChqcy9wYXJzZUludCAoc2xpY2UgZGF0YSAwIGkpKQogICAgICAgICAgZGF0YSAoc2xpY2UgZGF0YSAoaW5jIGkpKV0KICAgICAgWyhzdHIgKHNsaWNlIGRhdGEgMCBuKSkgKHNsaWNlIGRhdGEgbildKSkpCgooZGVmbiBkZWNvZGUgW2RhdGEgJiBvcHRzXQogICh0cnkKICAgIChkZWNvZGUtcmVjdXIgZGF0YSBvcHRzKQogICAgKGNhdGNoIGpzL0Vycm9yIF9lIFtuaWwgZGF0YV0pKSkKCihkZWZuIGRlY29kZS1hbGwgW2RhdGEgJiBvcHRzXQogIChsb29wIFtpdGVtcyBbXSBkYXRhIGRhdGFdCiAgICAobGV0IFtbaXRlbSBkYXRhXSAoYXBwbHkgZGVjb2RlIGRhdGEgb3B0cyldCiAgICAgIChpZiAobmlsPyBpdGVtKQogICAgICAgIFtpdGVtcyBkYXRhXQogICAgICAgIChyZWN1ciAoY29uaiBpdGVtcyBpdGVtKSBkYXRhKSkpKSkKCihkZWZuIHJlYWQtYmVuY29kZSBbc3RyaW5nXSAoZmlyc3QgKGRlY29kZSBzdHJpbmcpKSkKCihkZWZuIHV0ZjgtYnl0ZXMgW3NdCiAgKC4tbGVuZ3RoICguZW5jb2RlIChuZXcganMvVGV4dEVuY29kZXIpIHMpKSkKCihkZWZuIGVuY29kZSBbZGF0YV0KICAoY29uZAogICAgKHN0cmluZz8gZGF0YSkKICAgIChzdHIgKHV0ZjgtYnl0ZXMgZGF0YSkgIjoiIGRhdGEpCiAgICAob3IgKGtleXdvcmQ%2FIGRhdGEpCiAgICAgICAgKHN5bWJvbD8gZGF0YSkpCiAgICAocmVjdXIgKHN0cgogICAgICAgICAgICAod2hlbi1sZXQgW24gKG5hbWVzcGFjZSBkYXRhKV0KICAgICAgICAgICAgICAoc3RyIG4gIi8iKSkKICAgICAgICAgICAgKG5hbWUgZGF0YSkpKQogICAgKG51bWJlcj8gZGF0YSkKICAgIChzdHIgImkiIGRhdGEgImUiKQogICAgKG9yIChzZXQ%2FIGRhdGEpICh2ZWN0b3I%2FIGRhdGEpIChuaWw%2FIGRhdGEpKQogICAgKHN0ciAibCIgKGFwcGx5IHN0ciAobWFwIGVuY29kZSBkYXRhKSkgImUiKQogICAgKG1hcD8gZGF0YSkKICAgIChzdHIgImQiICgtPj4gZGF0YQogICAgICAgICAgICAgICAgICAoc29ydC1ieSBmaXJzdCkKICAgICAgICAgICAgICAgICAgKG1hcCAoZm4gW1trIHZdXQogICAgICAgICAgICAgICAgICAgICAgICAgKHN0ciAoZW5jb2RlIGspIChlbmNvZGUgdikpKSkKICAgICAgICAgICAgICAgICAgKGFwcGx5IHN0cikpCiAgICAgICAgICJlIikpKQoKKGRlZm4gd3JpdGUtYmVuY29kZSBbZGF0YV0KICAoZW5jb2RlIGRhdGEpKQoKWyhlbmNvZGUgImhlbGxvIikgKGRlY29kZSAiNTpoZWxsbyIpXQ%3D%3D So you could compile this with squint and use it in your library too.

or you could copy paste the compiled output into your project (disable the REPL button for more optimized code)

Within the next month or so, I want to have the project ready for inclusion in the pod registry

I'll be diving in as time allows

I was working on this today, but found that process.stdin.on() was not working the way I expected in bun

I recreated the project using bun init and I copied in the source file and the demo script, and I added my dependencies, and now I'm doing bun build --compile --target=bun-darwin-arm64 ./src/queryblast.ts

but when I run

#!/usr/bin/env ys-0

queryblast =: pods/load-pod('./queryblast')
I'm seeing it hang forever, seemingly as if there is nothing coming from stdin

let me try the bb script that you posted above in this thread

(require '[babashka.pods :as pods])
(pods/load-pod "./queryblast")
Puzzlingly, I'm seeing this hang forever when I run it in the bb repl