Fork me on GitHub
#xtdb
<
2022-08-08
>
emccue00:08:28

issue with the java api (thread)

emccue00:08:33

var xtdb = IXtdb.startNode(NodeConfiguration.builder()
                .with("xtdb.jdbc/connection-pool", ModuleConfiguration.builder()
                        .with("dialect", ModuleConfiguration.builder()
                                .with("xtdb/module", "xtdb.jdbc.psql/->dialect")
                                .build()
                        )
                        .with(
                                "db-spec", ModuleConfiguration.builder()
                                        .with("jdbcUrl", JDBC_DATABASE_URL)
                                        .build()
                        ).build()
                )
                .with("xtdb/tx-log", ModuleConfiguration.builder()
                        .with("xtdb/module", "xtdb.jdbc/->tx-log")
                        .with("connection-pool", "xtdb.jdbc/connection-pool")
                        .build()
                )
                .with("xtdb/document-store", ModuleConfiguration.builder()
                        .with("xtdb/module", "xtdb.jdbc/->document-store")
                        .with("connection-pool", "xtdb.jdbc/connection-pool")
                        .build()
                ).build()
        );

emccue00:08:48

tried this way and with a map from Map.of(), still end up with this error

emccue00:08:54

Caused by: xtdb.IllegalArgumentException: Arg :db-spec = {jdbcUrl=jdbc:} invalid: {"jdbcUrl" "jdbc:"} - failed: map?

emccue00:08:06

so i think there is some machinery thats not happy with a java style map

emccue00:08:16

and no conversion internally to a clojure one

refset11:08:59

hey @U3JH98J4R my Java skills aren't great but this sounds like a bug to me, would you like to create an issue? (happy to do so myself if you'd prefer)

tatut04:08:33

thinking about adding an “audit log” entry for some changes, what would be a good way to refer to the contents of a transaction and sign them? I guess you can’t refer to “current transaction” in tx functions

refset11:08:26

Hey @U11SJ6Q0K you can call indexing-tx on the transaction function ctx to get tx-id/tx-time, as per https://github.com/xtdb/xtdb/blob/ef143dcbff25dcf1fe0808309eb76dc728e381cf/core/src/xtdb/tx.clj#L196

refset11:08:34

this should really be in the docs somewhere but I can't see that it is :thinking_face: (e.g. here somewhere https://docs.xtdb.com/clients/1.21.0/clojure/) I've made a note on the project board to review and fix sometime

tatut13:08:43

thanks, good to know

hansbugge14:08:22

Hi 👋 I noticed that attempting to read from a closed (RocksDB) database snapshot crashes the JVM with a segfault instead of throwing an exception. Basically this:

(def db (xt/open-db node))
(.close db)
(xt/entity db :foo)
does the trick for me. Would this be worth a GitHub issue with a repro?

hansbugge15:08:43

I assume that this behaviour is known, I was just wondering if there was a reason why this is not avoided by checking the status of the db connection. Efficiency could be a reason, e.g.

refset00:08:34

Hey @UNBM99ED7 this is an unfortunate sharp edge with how RocksDB's JVM integration works (exactly for performance reasons as I understand things). We could attempt to hide some of this complexity from the user in XT, potentially, but it hasn't been deeply analysed. Specifically in this case I would recommend using with-open to avoid this mistake

hansbugge09:08:57

I was experimenting with a setup where multiple concurrent handlers could use the same db snapshot (for a graphql server), and then an interceptor would clean up by .close-ing the snapshot. But I didn't get very far because my REPL crashes with a segfault when I do something wrong. I'll probably need to rethink a bit and find a way to use with-open to avoid having the snapshots in scope after they have been closed.

refset11:08:16

Thanks for the context. Bear in mind that open-db is not intended for multi-threaded usage though, and instead you should have a db per thread. I would be very interested to hear about / see what you end up with :)

hansbugge11:08:46

Aaah I see... Then I'll probably end up using a timestamp for ensuring consistency, and pushing the xt/db/`xt/open-db` calls down to the individual threads. Or something along those lines 🙂

👍 1
hansbugge11:08:32

Thanks for your time – hopefully we'll end with something worth sharing 😉

🙏 1