babashka

2025-05-09T18:47:21.182169Z

I have a newly-published open source utility published by my employer, which is very exciting for me! It's a babashka pod, written in Typescript, which allows querying Neptune graph databases. Here is the link: https://github.com/SouthwestAir/queryblast

🎉 8
borkdude 2025-05-09T18:48:00.754769Z

Wow, this might be the very first pod written in JavaScript / TypeScript! :)

❤️ 3
2025-05-09T18:50:08.401749Z

Thanks! I'm very pleased about it! I would like to add this pod to the pod-registry, if at all possible

2025-05-09T18:51:26.434109Z

and I can see that I'll need to publish a release first, before I can add it to the pod-registry

2025-05-09T18:51:36.201429Z

so in any case, I still have a bit more work to do

2025-05-09T18:52:04.697009Z

but I'm very excited about today's milestone anyway!

borkdude 2025-05-09T18:53:45.883059Z

so far the pod registry makes sense if you produce standalone binaries for several OS-es. I haven't really thought about how to combine this with JS

borkdude 2025-05-09T18:54:03.607009Z

having manifests in the pod registry point you to where bb can download the binaries

borkdude 2025-05-09T18:56:46.826679Z

I guess we could make something for "pod scripts" perhaps by just adding them to the classpath instead of doing it via the pod registry

borkdude 2025-05-09T18:57:03.386489Z

not sure, haven't thought about this deeply

borkdude 2025-05-09T19:05:14.902419Z

I've got this working locally now:

(require '[babashka.pods :as pods])

(pods/load-pod ".esbuild/.build/src/queryblast.js")

(require 'queryblast)

(prn (ns-publics 'queryblast))

borkdude 2025-05-09T19:09:48.168349Z

I tried compiling to a self-contained executable using bun and deno but their netstring libraries aren't working properly like the one in node, I think maybe a flushing issue or so

2025-05-09T19:11:02.531789Z

Ooh nice progress, thanks for your help & attention

borkdude 2025-05-09T19:12:47.129869Z

this could be another way to do it on node but looks complicated: https://nodejs.org/api/single-executable-applications.html

2025-05-09T19:14:43.983549Z

Wow and using the code from yamlscript!

borkdude 2025-05-09T19:15:13.339089Z

this tool works for sure, it even works for non-node script runners like babashka: https://github.com/AppThreat/caxa

borkdude 2025-05-09T19:25:45.571999Z

works with caxa:

npx caxa --input .esbuild/.build/src --output mypod  -- "{{caxa}}/node_modules/.bin/node" "{{caxa}}/queryblast.js"
(require '[babashka.pods :as pods])

(pods/load-pod "./mypod")
#_(pods/load-pod ".esbuild/.build/src/queryblast.js")

(require 'queryblast)

(prn (ns-publics 'queryblast))

2025-05-09T19:25:59.113109Z

oh this is rad, I'll try this now

borkdude 2025-05-09T19:26:07.723079Z

this yields a 34mb binary which is ok in the world of pods

borkdude 2025-05-09T19:26:40.941089Z

(it just packages the node binary along with the script)

borkdude 2025-05-09T19:28:26.316789Z

this just returns nil without any error, don't know if that is expected:

(prn (queryblast/query! "MATCH (n) RETURN n LIMIT 1"))
I haven't set any env vars

2025-05-09T19:29:52.720049Z

this is after you loaded it in the babashka repl, is that right?

borkdude 2025-05-09T19:31:34.853379Z

just a script:

(require '[babashka.pods :as pods])

(pods/load-pod "./mypod")
#_(pods/load-pod ".esbuild/.build/src/queryblast.js")

(require 'queryblast)

(prn (ns-publics 'queryblast))

(prn (queryblast/query! "MATCH (n) RETURN n LIMIT 1"))

2025-05-09T19:35:48.718259Z

Thanks for pointing that out, that nil response behavior could be improved with a usage string describing the required environment variables

2025-05-09T19:37:41.459959Z

I'll be making some changes, for producing the release binary and getting it ready to add to the pod registry, can I use this bb script and your other suggestions?

borkdude 2025-05-09T19:37:53.217799Z

of course!

2025-05-09T19:37:57.298399Z

Thank you!

borkdude 2025-05-15T13:49:00.989169Z

I think I know what's wrong when running the pod on deno. I ran into this with #nbb + bencode as well for nREPL: https://github.com/babashka/nbb/commit/5f800c95068b2ad4f02cbb13bbc01f157e4fb740 This thing here: https://github.com/isolomak/bencodec/blob/bd0ab57131bce2f2259e77f5866c9a6d663489c6/src/BencodeEncoder.ts#L70 doesn't work in deno. So perhaps if we change this we can compile to self-contained executables in deno.

2025-05-15T14:27:50.642489Z

Does Deno make a better binary than caxa? I'm only mildly familiar with Deno

borkdude 2025-05-15T14:30:55.148189Z

don't know yet

borkdude 2025-05-15T14:30:56.092859Z

https://github.com/isolomak/bencodec/pull/31

borkdude 2025-05-15T14:35:56.155349Z

I made a single JS file with esbuild and then compiled it to an exec with deno. This is just for the bencode demo.

$ ./standalone
5:hello
hello
The binary is 70mb, a bit big

borkdude 2025-05-15T14:36:11.147289Z

anyway it's good to have options

borkdude 2025-05-15T14:38:13.227439Z

with bun I get:

57669360

borkdude 2025-05-15T14:38:18.824249Z

as the output size

borkdude 2025-05-15T14:40:35.829169Z

last time I couldn't get either to work due to the bugfix above

borkdude 2025-05-15T14:42:52.720259Z

with bun (and deno) it seems very easy to build multiplatform executables:

bun build --compile --target=bun-linux-x64 ./standalone.js

2025-05-15T15:24:12.126089Z

oh yeah in that case, since we'll need those executables on various platforms, this is a great step

2025-05-15T15:24:22.372579Z

thanks for making the MR!

2025-05-15T15:25:07.432949Z

I hope the maintainer takes note of it

borkdude 2025-05-15T15:27:31.765249Z

bencode isn't that complicated though, it's just 80 or so lines in clojure. this even works in squint: https://squint-cljs.github.io/squint/?src=KG5zIG5iYi5pbXBsLmJlbmNvZGUKICAiQmVuY29kZSBzdXBwb3J0LCB0YWtlbiBmcm9tIGh0dHBzOi8vZ2l0aHViLmNvbS9kamJsdWUvbnJlcGwtY2xqcy9ibG9iL21hc3Rlci9zcmMvbnJlcGwvYmVuY29kZS5jbGpzIikKCihkZWZuLSBpbmRleC1vZiBbcyBjXQogIChsZXQgW2kgKC5pbmRleE9mIHMgYyldCiAgICAoaWYgKDwgaSAwKSAodGhyb3cgKGpzL0Vycm9yLiAib3V0IG9mIGlucHV0IikpIGkpKSkKCihkZWZuLSBzbGljZQogIChbYnVmZmVyIHN0YXJ0XQogICAoaWYgKDwgKC4tbGVuZ3RoIGJ1ZmZlcikgc3RhcnQpCiAgICAgKHRocm93IChqcy9FcnJvci4gIm91dCBvZiBpbnB1dCIpKQogICAgICguc2xpY2UgYnVmZmVyIHN0YXJ0KSkpCiAgKFtidWZmZXIgc3RhcnQgZW5kXQogICAoaWYgKD4gZW5kICguLWxlbmd0aCBidWZmZXIpKQogICAgICh0aHJvdyAoanMvRXJyb3IuICJvdXQgb2YgaW5wdXQiKSkKICAgICAoLnNsaWNlIGJ1ZmZlciBzdGFydCBlbmQpKSkpCgooZGVmbi0gZGVjb2RlLXJlY3VyIFtkYXRhIG9wdHNdCiAgKGNhc2UgKHN0ciAoc2xpY2UgZGF0YSAwIDEpKQogICAgImkiCiAgICAobGV0IFtkYXRhIChzbGljZSBkYXRhIDEpCiAgICAgICAgICBpIChpbmRleC1vZiBkYXRhICJlIildCiAgICAgIFsoanMvcGFyc2VJbnQgKHNsaWNlIGRhdGEgMCBpKSkKICAgICAgIChzbGljZSBkYXRhIChpbmMgaSkpXSkKICAgICJsIgogICAgKGxldCBbZGF0YSAoc2xpY2UgZGF0YSAxKV0KICAgICAgKGxvb3AgW2RhdGEgZGF0YSB2ICh0cmFuc2llbnQgW10pXQogICAgICAgIChpZiAoPSAoc3RyIChzbGljZSBkYXRhIDAgMSkpICJlIikKICAgICAgICAgIFsocGVyc2lzdGVudCEgdikgKHNsaWNlIGRhdGEgMSldCiAgICAgICAgICAobGV0IFtbdmFsdWUgZGF0YV0gKGRlY29kZS1yZWN1ciBkYXRhIG9wdHMpXQogICAgICAgICAgICAocmVjdXIgZGF0YSAoY29uaiEgdiB2YWx1ZSkpKSkpKQogICAgImQiCiAgICAobGV0IFtkYXRhIChzbGljZSBkYXRhIDEpCiAgICAgICAgICB7OmtleXMgW2tleXdvcmRpemUta2V5c119IG9wdHNdCiAgICAgIChsb29wIFtkYXRhIGRhdGEgbSAodHJhbnNpZW50IHt9KV0KICAgICAgICAoaWYgKD0gKHN0ciAoc2xpY2UgZGF0YSAwIDEpKSAiZSIpCiAgICAgICAgICBbKHBlcnNpc3RlbnQhIG0pIChzbGljZSBkYXRhIDEpXQogICAgICAgICAgKGxldCBbW2sgZGF0YV0gKGRlY29kZS1yZWN1ciBkYXRhIG9wdHMpCiAgICAgICAgICAgICAgICBbdiBkYXRhXSAoZGVjb2RlLXJlY3VyIGRhdGEgb3B0cykKICAgICAgICAgICAgICAgIGsgKGlmIGtleXdvcmRpemUta2V5cyAoa2V5d29yZCBrKSBrKV0KICAgICAgICAgICAgKHJlY3VyIGRhdGEgKGFzc29jISBtIGsgdikpKSkpKQogICAgKGxldCBbaSAoaW5kZXgtb2YgZGF0YSAiOiIpCiAgICAgICAgICBuIChqcy9wYXJzZUludCAoc2xpY2UgZGF0YSAwIGkpKQogICAgICAgICAgZGF0YSAoc2xpY2UgZGF0YSAoaW5jIGkpKV0KICAgICAgWyhzdHIgKHNsaWNlIGRhdGEgMCBuKSkgKHNsaWNlIGRhdGEgbildKSkpCgooZGVmbiBkZWNvZGUgW2RhdGEgJiBvcHRzXQogICh0cnkKICAgIChkZWNvZGUtcmVjdXIgZGF0YSBvcHRzKQogICAgKGNhdGNoIGpzL0Vycm9yIF9lIFtuaWwgZGF0YV0pKSkKCihkZWZuIGRlY29kZS1hbGwgW2RhdGEgJiBvcHRzXQogIChsb29wIFtpdGVtcyBbXSBkYXRhIGRhdGFdCiAgICAobGV0IFtbaXRlbSBkYXRhXSAoYXBwbHkgZGVjb2RlIGRhdGEgb3B0cyldCiAgICAgIChpZiAobmlsPyBpdGVtKQogICAgICAgIFtpdGVtcyBkYXRhXQogICAgICAgIChyZWN1ciAoY29uaiBpdGVtcyBpdGVtKSBkYXRhKSkpKSkKCihkZWZuIHJlYWQtYmVuY29kZSBbc3RyaW5nXSAoZmlyc3QgKGRlY29kZSBzdHJpbmcpKSkKCihkZWZuIHV0ZjgtYnl0ZXMgW3NdCiAgKC4tbGVuZ3RoICguZW5jb2RlIChuZXcganMvVGV4dEVuY29kZXIpIHMpKSkKCihkZWZuIGVuY29kZSBbZGF0YV0KICAoY29uZAogICAgKHN0cmluZz8gZGF0YSkKICAgIChzdHIgKHV0ZjgtYnl0ZXMgZGF0YSkgIjoiIGRhdGEpCiAgICAob3IgKGtleXdvcmQ%2FIGRhdGEpCiAgICAgICAgKHN5bWJvbD8gZGF0YSkpCiAgICAocmVjdXIgKHN0cgogICAgICAgICAgICAod2hlbi1sZXQgW24gKG5hbWVzcGFjZSBkYXRhKV0KICAgICAgICAgICAgICAoc3RyIG4gIi8iKSkKICAgICAgICAgICAgKG5hbWUgZGF0YSkpKQogICAgKG51bWJlcj8gZGF0YSkKICAgIChzdHIgImkiIGRhdGEgImUiKQogICAgKG9yIChzZXQ%2FIGRhdGEpICh2ZWN0b3I%2FIGRhdGEpIChuaWw%2FIGRhdGEpKQogICAgKHN0ciAibCIgKGFwcGx5IHN0ciAobWFwIGVuY29kZSBkYXRhKSkgImUiKQogICAgKG1hcD8gZGF0YSkKICAgIChzdHIgImQiICgtPj4gZGF0YQogICAgICAgICAgICAgICAgICAoc29ydC1ieSBmaXJzdCkKICAgICAgICAgICAgICAgICAgKG1hcCAoZm4gW1trIHZdXQogICAgICAgICAgICAgICAgICAgICAgICAgKHN0ciAoZW5jb2RlIGspIChlbmNvZGUgdikpKSkKICAgICAgICAgICAgICAgICAgKGFwcGx5IHN0cikpCiAgICAgICAgICJlIikpKQoKKGRlZm4gd3JpdGUtYmVuY29kZSBbZGF0YV0KICAoZW5jb2RlIGRhdGEpKQoKWyhlbmNvZGUgImhlbGxvIikgKGRlY29kZSAiNTpoZWxsbyIpXQ%3D%3D So you could compile this with squint and use it in your library too.

borkdude 2025-05-15T15:28:32.108139Z

or you could copy paste the compiled output into your project (disable the REPL button for more optimized code)

2025-05-15T15:49:21.305909Z

good suggestion

2025-05-15T15:50:44.822769Z

Within the next month or so, I want to have the project ready for inclusion in the pod registry

2025-05-15T15:50:56.301849Z

I'll be diving in as time allows

2025-05-21T22:44:06.173169Z

I was working on this today, but found that process.stdin.on() was not working the way I expected in bun

2025-05-21T22:45:10.730669Z

I recreated the project using bun init and I copied in the source file and the demo script, and I added my dependencies, and now I'm doing bun build --compile --target=bun-darwin-arm64 ./src/queryblast.ts

2025-05-21T22:46:01.525739Z

but when I run

#!/usr/bin/env ys-0

queryblast =: pods/load-pod('./queryblast')
I'm seeing it hang forever, seemingly as if there is nothing coming from stdin

2025-05-21T22:46:19.275459Z

let me try the bb script that you posted above in this thread

2025-05-21T22:53:13.002489Z

(require '[babashka.pods :as pods])
(pods/load-pod "./queryblast")
Puzzlingly, I'm seeing this hang forever when I run it in the bb repl