Fork me on GitHub
Ahmed Hassan07:04:43

I think babashka should keep database binary separate, in this way it will be easier to add SQL DSL libraries to it without worrying about size.


Most people do not seem to care about the binary size of babashka, as long as it has good startup time.


DSL libraries (transforming data -> data) do not add that much, it's more the drivers themselves (sqlite 20MB, hsqldb 10MB) that put extra pressure on size, but more importantly memory usage in CI.


Feature flags like @lukaszkorecki has suggested may be the way forward for very specific functionality and it can be included in the standard distribution when there is sufficient demand


I updated the goals of babashka here: I added connectivity. Being able to connect to things like @lukaszkorecki is doing with UDP seems very useful. We already had sockets and http. Now we also have JDBC as a connectivity option (albeit only for Postgres at the moment). One other goal is multi-threading so from that perspective it makes sense to add some classes around the Executor stuff.

babashka 20

Any idea how hard it would be to get running in Babashka? When I try to import it I get the error message that its Unable to resolve classname: java.nio.file.OpenOption. java.nio.file.Path is resolveable though.


@rovanion extend-protocol and extend-type are not available, so that's where it currently breaks down. The java.nio.file.OpenOption class is just not mapped, but could be added, I don't see a reason why not.


I just pushed this class to master. If you want to experiment with new binaries, take a look at #babashka_circleci_builds (takes about 5 minutes for them to appear)


Right, thank you!


I do want to look at support for defprotocol because that seems the biggest incompatibility at the moment. But that could take a while 🙂


Seems like a lot of unknowns.


what do you mean with unknowns?


Looking from the outside there are a lot of things I do not know about how protocols work. It becomes hard to estimate the scope of a piece of work with a lot of unknown things about it.


the class is there now in #babashka_circleci_builds


My 2 cents, I don’t use Postgres at the moment, but I can imagine this addition is useful if you do. I also don’t care too much about the binary size. I can imagine things like this will put an extra burden on the maintainance of Babashka. I still hope there might be a future with some kind of native graalvm libraries that can be included without recompilation


That would be the most awesome.


But also very hard to pull I think, without writing a ton of boilerplate JNI code


perhaps some day this will be usable and helpful?


@U04V15CAJ Maybe babashka could emulate the JNI code specifically for a set of babashka libraries. Let’s say babashka is split up in modules, bb-core, bb-yaml, bb-psql etc, whatever make sense. Every module would have the same clojure core and serialization functions, meaning a script run by bb-core can be called with transit or just edn and we know bb-yaml will understand it because they share this serialization capability. Since bb-yaml has a very fast startup time just like bb-core and the other bb binaries, the overhead of shelling out is minimal. bb-core would have a utility function call-module that could be used like (call-module "bb-yaml", input-stream, optional-transformation-fn) and returns a promise or lazy sequence. All these libraries would need to be updated when the core functionality is upgraded (but this should become rarer over time). Individual libraries can be added and updated without touching any of the others


Does that make sense?


You could have a bb-bare that contains the minimal functionality, but is able to call other modules. People that care about binary size or want to add custom modules could use this one. A bb binary like you have right now would be a collection of modules that are nice to get started


Maybe this could be setup similar to how yada has it with: default , lean (


Yes. I want to experiment with JNI directly as well. If that doesn't work out, it's going to be bb + bb-sql + bb-whatevers


Are you also thinking about the shared library idea that you did with Rust?


I guess you would have to not just be able to call Sci from Rust, but also Rust from Sci


Maybe you already did that, I lost track 😅


The major thing I'm struggling with is how to return an Object from a native library to the caller.


Is it possible to use Java libraries like ?


so same question would be can you return bytes?


well, I want to return just a hashmap with all the functions in it, so I can hook it to sci


Not sure if I follow. What kind of functions would be on the hashmap?


I would just return {:namespaces {'next.jdbc {'execute! ...}}} from the library and insert that into babashka's sci context


Ah I see and the actual function would be called via JNI I guess


I'm about to give up on this approach again.

user=> (clojure.lang.RT/loadLibrary "sci")
user=> (babashka.modules.PgNextJDBC/init 1 2)
{:namespaces {cheshire.core2 {generate-string #object[cheshire.core$generate_string 0x3cdf2153 "cheshire.core$generate_string@114305ce0"]}}}
Execution error (NullPointerException) at babashka.modules.PgNextJDBC/initPG (


When going down the serialization route (every argument must be serialized to and from edn, transit, or whatever): I'm afraid that quickly becomes limiting, since you can't pass any database connections or Java objects etc that way


So making another bb-sql is most flexible and easiest to maintain maybe


What if the thing that is hard to serialize stays in the module? E.g. the database connection would never exit the bb-sql module. You would only refer to it by an id? The results would be normal edn data?


I guess that would not be very different from just shelling out to another bb-sql CLI and use EDN to transfer data. And that would be a lot easier


I've created an issue about native libraries here:


I added a comment about boot pods. Maybe it’s interesting to use that as an example


good analogy!