what do you mean with unknowns?

Looking from the outside there are a lot of things I do not know about how protocols work. It becomes hard to estimate the scope of a piece of work with a lot of unknown things about it.

yes

My 2 cents, I don’t use Postgres at the moment, but I can imagine this addition is useful if you do. I also don’t care too much about the binary size. I can imagine things like this will put an extra burden on the maintainance of Babashka. I still hope there might be a future with some kind of native graalvm libraries that can be included without recompilation

That would be the most awesome.

But also very hard to pull I think, without writing a ton of boilerplate JNI code

perhaps some day this will be usable and helpful? https://openjdk.java.net/projects/panama/

@U04V15CAJ Maybe babashka could emulate the JNI code specifically for a set of babashka libraries. Let’s say babashka is split up in modules, bb-core, bb-yaml, bb-psql etc, whatever make sense. Every module would have the same clojure core and serialization functions, meaning a script run by bb-core can be called with transit or just edn and we know bb-yaml will understand it because they share this serialization capability. Since bb-yaml has a very fast startup time just like bb-core and the other bb binaries, the overhead of shelling out is minimal. bb-core would have a utility function call-module that could be used like (call-module "bb-yaml", input-stream, optional-transformation-fn) and returns a promise or lazy sequence. All these libraries would need to be updated when the core functionality is upgraded (but this should become rarer over time). Individual libraries can be added and updated without touching any of the others

Does that make sense?

You could have a bb-bare that contains the minimal functionality, but is able to call other modules. People that care about binary size or want to add custom modules could use this one. A bb binary like you have right now would be a collection of modules that are nice to get started

Maybe this could be setup similar to how yada has it with: default , lean (https://github.com/juxt/yada/tree/master/bundles/lean)

Yes. I want to experiment with JNI directly as well. If that doesn't work out, it's going to be bb + bb-sql + bb-whatevers

Ok cool 🙂

Are you also thinking about the shared library idea that you did with Rust?

I guess you would have to not just be able to call Sci from Rust, but also Rust from Sci

Maybe you already did that, I lost track 😅

I'm trying this right now. https://github.com/oracle/graal/blob/master/substratevm/ImplementingNativeMethodsInJavaWithSVM.md

nice 🙂

The major thing I'm struggling with is how to return an Object from a native library to the caller.

Is it possible to use Java libraries like https://github.com/clojure/data.fressian ?

so same question would be can you return bytes?

well, I want to return just a hashmap with all the functions in it, so I can hook it to sci

Not sure if I follow. What kind of functions would be on the hashmap?

I would just return {:namespaces {'next.jdbc {'execute! ...}}} from the library and insert that into babashka's sci context

Ah I see and the actual function would be called via JNI I guess

I'm about to give up on this approach again.

user=> (clojure.lang.RT/loadLibrary "sci")
nil
user=> (babashka.modules.PgNextJDBC/init 1 2)
debug
{:namespaces {cheshire.core2 {generate-string #object[cheshire.core$generate_string 0x3cdf2153 "[email protected]"]}}}
debug2
Execution error (NullPointerException) at babashka.modules.PgNextJDBC/initPG (PgNextJDBC.java:-2).
null

When going down the serialization route (every argument must be serialized to and from edn, transit, or whatever): I'm afraid that quickly becomes limiting, since you can't pass any database connections or Java objects etc that way

So making another bb-sql is most flexible and easiest to maintain maybe

What if the thing that is hard to serialize stays in the module? E.g. the database connection would never exit the bb-sql module. You would only refer to it by an id? The results would be normal edn data?

I guess that would not be very different from just shelling out to another bb-sql CLI and use EDN to transfer data. And that would be a lot easier

I've created an issue about native libraries here: https://github.com/borkdude/babashka/issues/397

https://github.com/borkdude/babashka/issues/397#issuecomment-621259243

I added a comment about boot pods. Maybe it’s interesting to use that as an example

good analogy!