Fork me on GitHub

What is a sane way to develop babashka scripts in a REPL driven way, using Emacs/CIDER? My idea was to make the script(s) as much Clojure compatible as i can (for obvious reasons) and just use normal clojure REPL. I know nREPL is now supported, but would that give the same developer ergonomics?


inf-clojure was just revamped and works very well with babashka


thanks both!


anyone tried to run bb on a raspberry pi?


would be fun to try and build some home automation scripts to control my philips hue lights etc.


ah well, only one way to find out 😉


There might be a link to a binary in there that might work. Not sure


if you have a 64 bit kernel it may work


thx for the info


so with (ie.) the Ubuntu in the last link on the page above could work on my pi4 that arrives tomorrow? Will try


yes, i think there's a chance it will work


I've got hsqldb working too with babashka. A few things I'm worried about though: - It grows the binary with almost 10MB. - Adding more libraries to babashka demands more from build environments like CircleCI and Github actions. Eventually we will run out of memory again? Maybe it makes sense to split out the sql stuff to a clj-jdbc command line that is a fork/branch of babashka that includes support for Postgres and HSQLDB? The downside of this is that it's nice to have everything in one tool, without having to download additional stuff and being able to execute your program all in the same process


(there is a hsqldb branch you can look at)


I agree with your conclusion. HSQLDB also can run standalone and you can connect to it via "hsql" proprietary protocol or "http". So there are basically two modes of operations. • having a fork with hsqldb bundled in • having hsqldb operating as standalone (basically having native-hsqldb project) + in babashka only have client part either proprietary "hsql" or/and "http"


It would be sad to see "babashka" to become a monster like docker


in what sense, can you elaborate?


"a huge binary doing everything", it's a departure from more unix-like philosophy of cooperating utilities.


my feeling is that some libraries like yaml and jdbc aren't used that often for scripting, so it might be better to split those out into different bb forks with their own names?


ok, to be fair, I just looked at docker binary and it's not that bad yet


I feel bb case is more difficult because it's general scripting language there is this split between being barebone and minimalistic vs have some batteries of course. Having yaml/xml/json support is probably a good thing as they're used heavily in the wild. But with SQL i'm not sure. It's really cool to have it, once you really need it. But until that point... it's just waste.


solution probably lies somewhere in good ecosystem, but that's so general statement I'm almost shy to tell it loud.


Just thinking the same. Maybe one way to keep babashka lean is to think "could I do this with curl?" and have a mechanism for scripting to include extra deps. Something like "bb install postgres" would be very nice if you want to build an ecosystem around bb. I like the asdf model of adding packages, its very low tech bash scripting.


I don't know how bb install postgres would work though, since the postgres support would live in its own binary, it would be another version of bb, possible called clj-jdbc or something


but splitting JDBC out in its own util feels like a good choice. supporting data formats like xml and yaml feels essential to scripting

👍 8

and we can add support for mysql etc too, and not worry about bloating babashka


So, tentative conclusion: > Splitting out the JDBC part into clj-jdbc won't put any more pressure on the binary size and memory usage in CI environments. It allows for addition of more drivers for mySQL etc. without bloating babashka.

👍 4

Could additions be downloaded and added to the class path? To a .bb subdir or something like that. Asdf is just a master repo that points to other user supported repos. I was really surprised how usable it is.


That's unfortunately not how it works. The classes / drivers for various JDBC drivers cannot be added at runtime. They have to be compiled along with the rest.


That's true, native-image is doing closed-world analysis and binary is almost entirely static.


Fair enough.


Maybe bb can support some 'protocols' to speak to other programs, but outside of that options are limited.


Well you can load dynamic libraries, but then you would have to write a JNI wrapper around a clojure library


which would be a lot of work


I guess the key thing is to keep the default size small.


v0.0.89 is 51MB on Mac


zipped it's like 12MB


btw hsqldb has some different license... so be careful there as well. I believe it's BSD derived so no violention should be happening, but still let's double-check.


if you could sort that out, that would be great for the future of clj-jdbc or whatever this awesome command line tool will be named 😉


Splitting out clj-jdbc will also allow adding more nice libraries like honeysql etc


does splitting it out mean, it becomes some sort of ‘binary library’ that bb shells out to? Or is it a ‘bb with jdbc’ ‘distribution’


that first part probably doesn’t even make sense 🙂


it's a new CLI tool called clj-jdbc or something else if I can make up a better name

👍 4
Darin Douglass12:04:13

bb-core (aka bb), bb-jdbc, et al


bdbc the first of the enterprise babashka implementations. bbdb has a nice ring to it.


bb-the-sequel 😉

Darin Douglass12:04:51

Maybe we’re thinking about it slightly backwards? IMO first-class db support feels right for bb. What about having a minimal bb build for environments that have space restrictions (pies, etc)?


One of the concerns is that stacking more and more libraries in bb will prolong build times and memory consumption in CI, eventually we'll have to make choices


Also adding hsqldb support adds 10MB, I don't think that justifies it if people will only use that seldomly

Darin Douglass13:04:43

Yeah the CI resources could be an issue, if it IS an issue I’d likely pitch in some money to help make it NOT an issue. As for the binary size, install size is far, FAR down my my personal list of concerns. I’d rather have a bigger core bb and have another minimal build for people whose use-case may also be more minimal than have support for tools (even ones that I don’t use like hsqldb) limited by “will this fit on super small machines?”. What is 10MB to the hundreds of GB likely available to the majority of users?


What about keeping postgres, but building other databases into the other CLI?

👍 4

And what are the options e.g. at Github actions to up the memory against what costs?

Darin Douglass13:04:29

To be clear: I’m not advocating precisely to keep Postgres support in. Though I do use it, going back to installing and parsing psql output wouldn’t be a problem. I’m arguing that, assuming we can build the thing, binary size (within reason) shouldn’t be a limiting factor on what gets pulled into bb. If people have a use case and it doesn’t take much to implement/maintain, why not support it?


I'm all for supporting things, even niche things. But eventually I'm afraid I will run out of memory again in CI. This happened before on CircleCI. They were kind enough to bump my memory to 8gb. But I think there will be another limit we're going to hit


This discussion reminds me that there are two builds of Hugo, the static site generator. hugo which works for most sites, and hugo-extended which does everything the other one does and adds scss support. I think a separate clj-jdbc makes sense.


Binary size doesn't matter to me much, but it makes me wonder at what point it impacts start up time.


@U0510902N Can you repeat that binary size doesn't matter to you in the issue? I might be optimizing for the wrong things, so I'd like to hear this clearly recorded over there.


So this is the idea: @UGRJKK74Y Maybe you could try your script with this


Sure thing.

Darin Douglass13:04:13

Sure I can bring stuff to the issue. I’m pretty sure it’ll work assuming curl exists as well


babashka.curl isn't yet there, but is


but the idea would be to use babashka for every day scripting and then hand off the data to clj-jdbc for database interaction


so combine multiple smaller tools to achieve the goal


one thought: postgres seems popular enough to only support that in bb?

Darin Douglass13:04:02

IMO that sounds reasonable. I’ll have to look around for CI solutions. It doesn’t feel great having that being the limiting factor here.

Darin Douglass14:04:20

There still is the option of pure-clojure kind for dbs. bb-psql could be a lib that can interact with psql.

Darin Douglass14:04:55

Could have others for the other dbs. Would save on CI resources that way.


You could solve problems by piping between bb types and also by introducing a rpc system between instances perhaps. Then a master bb will run the others...


@UGRJKK74Y You mean like babashka.curl, babashka.psql?


@U09LZR36F I agree. Having multiple cooperating binaries hints at good rpc/ipc. Bash itself has an interop format: lines of text. Babashka could do that or something more (edn/transit/nippy?).


nippy could be useful to add (in the spirit of supporting data formats). transit is already there

👍 4
Darin Douglass14:04:16

Yeah something like babashka.psql et al. I like the idea of of cooperating binaries; though I wonder how cohesive a script that uses multiple of those binaries would feel.

Darin Douglass14:04:38

Though I use bb less like a bash replacement and more like a quick starting, lightweight clojure


Name suggestions: bbdb, jdbbc, dbb

Darin Douglass14:04:58

Interesting. That’s saying during build you're maxing out at about 5GB? Is circle constantly giving 8 GB?


yes. you should subtract some 1-1.5GB from that 8GB to give to the -Xmx of graalvm. So there's about 6.5GB available. For Github actions this is only 5.5GB.


maybe the build could do with less, it only takes much longer where there's just enough memory available, I guess a lot of swapping is happening


but at a certain point, it's going to throw an exception if the -Xmx value becomes too low. This has happened on CircleCI at 4GB


Interestingly, I'm trying to build with the 20.2-dev version with -Xmx=3000m, and this works on my local machine, whereas before with 19.3.1 this started to fail on CI. So maybe GraalVM improved something there as well.


I'm now trying -Xmx=3000m with 19.3.1 and it's taking a loooong time, so maybe 20.2 has some improvements in this area


And eventually it crashes. So conclusion: 3g wasn't enough anymore for 19.3.1. but it might be for 20.2


So maybe I panicked for nothing 🙂 There's still the question: do we want to pay 10 megabytes to include an embedded SQL database called hsqldb which can persist but also run in memory?


Pity about sqlite not working, it has an impressive deployment footprint and it generally follows postgres syntax. hsqldb is more in the java space only l think.


yeah, I'd love that to work. that's why it's maybe better to wait with hsqldb?

Roman Tsopin20:04:38

My 2 cents. I think it would be great to have all nice stuff (db, curl etc) under the current name but distribute stripped versions also if someone really cares about binary size. Something like brew install bb —no-curl —no-hsqldb etc. I know it doesn’t solve ci issue, but from ergonomic perspective it’s more intuitive


Can you also mention this in the issue? Would be appreciated, @UA6TBJPN0

👍 4

I've shared some thoughts to the issue, I hope they're helpful


I have collected some thoughts of the above thread in


This would be the kind of thing that's possible with an embedded hsqldb in bb: Pretty powerful, no need to shell out to sqlite anymore.


I do really like an embedded persistence solution. Yet another ability that opens up possibilities.


Yeah... I do like the idea as well. Although sqlite seems to be more popular, I can't get it to work with graalvm


sqlite is written in C, while hsqldb is pure Java


10mb of Java

Darin Douglass16:04:57

Seems like with Postgres we have support for the most used DB. And with hsqldb we have support for a slim in memory/local db. There are obviously more use cases for dbs, but I feel like those cover a good chunk of them

👍 4