sql

Mark Wardle 2023-06-10T10:01:24.927709Z

Hi all. I have several functions that use next.jdbc and create their own transactions. I've realised that this is probably a mistake and much better to push transactions higher-up the call chain to avoid nesting transactions, but that means those functions really must be called in a transaction so they succeed or fail as one. Is there a way of checking that a given connection handle is in a transaction and not a connection that will lead to each execute being auto-committed, so I can throw or give a warning when not called properly?

valtteri 2023-06-10T11:40:12.404479Z

Maybe you can check if the connection satisfies Transactable ? https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/protocols.clj#L58 Not sure if this is the right way though.

Mark Wardle 2023-06-10T11:41:01.021569Z

Thanks @valtteri - that sounds like a good idea.

valtteri 2023-06-10T11:41:02.069689Z

Ummm and maybe it will only say that transactions can be done but not whether there’s actually a transaction going on ATM.

valtteri 2023-06-10T11:41:58.958989Z

In the old clojure.java.jdbc there was some implementation detail where it kept track of the “level” of transactions. I remember ab(using) that to write assert-transaction for a similar case

valtteri 2023-06-10T11:42:55.814989Z

Sean probably knows what’s the best way to achieve that. 🙂 I’m interested to know as well.

valtteri 2023-06-10T11:43:29.392999Z

(or if that use-case doesn’t make sense for some reason)

Mark Wardle 2023-06-10T11:43:41.828739Z

That may be sufficient for my use-case, as I just want to throw if I get a connectable that isn't transactable. I'll do some testing with the different types of connectables. There is a private thread local called `*active-tx*` I spotted, but it is definitely marked private!

Mark Wardle 2023-06-10T11:43:55.851329Z

Yes I agree - perhaps I'm thinking about it wrong fundamentally!

Mark Wardle 2023-06-10T11:44:36.599049Z

But I want to guard against my future self making a mistake here. Perhaps I should turn off auto-commit so everything is explicitly in some form of transaction!

valtteri 2023-06-10T11:44:51.693849Z

In some code bases I remember seeing docstrings mentioning “should only be called within a transaction” and thought there must be a better way 😉

👍 1
Mark Wardle 2023-06-10T11:45:22.081219Z

Yes! I started writing that in docstrings and wondered what I was doing and how I was definitely going to forget at some point!

valtteri 2023-06-10T11:53:25.081139Z

There’s this private thing that might be useful https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/transaction.clj#L40-L41

Mark Wardle 2023-06-10T12:06:19.352599Z

I think lots of things satisfy Transactable, so that doesn't seem to work. I spotted that thread local but it is marked private so may change in future, I guess. But one can just use Java interop to get the transaction level and commit status at any point... eg... (defn conn-status [conn] {:auto-commit (.getAutoCommit conn) :isolation (.getTransactionIsolation conn)}) (def *conn* (jdbc/get-connection {:dbtype "postgresql" :dbname "rsdb"})) (conn-status *conn*) => {:auto-commit true, :isolation 2} (jdbc/with-transaction [txn *conn*] (conn-status *conn*)) => {:auto-commit false, :isolation 2} (jdbc/with-transaction [txn *conn* {:isolation :serializable}] (conn-status *conn*)) => {:auto-commit false, :isolation 8} (jdbc/with-transaction [txn *conn* {:isolation :serializable}] (conn-status txn)) => {:auto-commit false, :isolation 8}

Mark Wardle 2023-06-10T12:08:57.736819Z

So I could guard for a particular isolation level and auto commit status depending on requirements. Interested to hear if there is a better way.

Mark Wardle 2023-06-10T12:14:11.820089Z

So I could then use Clojure spec and define different types of conn with a specific transaction level and get devtime exceptions if I call with an auto committing connection.

Mark Wardle 2023-06-10T12:38:06.391239Z

Yep. Appears to work. I get an execution error if a function gets a connection not in the required transaction level.

lukasz 2023-06-10T16:30:55.968329Z

:transactionIsolation is the way to go - I think it might be driver dependent though, but it works with HikariCP / PGJDBC

seancorfield 2023-06-10T17:07:40.233839Z

I will caution that transactions are somewhat artificial: you can take any given Connection object and set it to not auto-commit and then choose to commit or rollback at any point, with or without setting the isolation level -- as @mark354 showed above, with PG, isolation level 2 is just the default, whether you're in a "transaction" or not, so you cannot rely on isolation level. You also can't really rely on auto-commit: false meaning "you're in a transaction" as there are sometimes other reasons to set it to false.

seancorfield 2023-06-10T17:09:43.855509Z

If your code is not escaping via Java interop to raw JDBC and all your operations are handled via next.jdbc, then checking #'next.jdbc.transaction/*active-tx* will be an accurate indicator of whether next.jdbc itself thinks it has an active TX (but, again, it can only know about its with-transaction call nesting).

seancorfield 2023-06-10T17:12:06.105209Z

I could add a supported next.jdbc/active-tx? predicate to the API if that would make you more comfortable -- the reason I didn't is because of the issues above, that "transactions" don't really exist, per se, they're just a convention of how you interact with a Connection when you perform multiple operations with the same one.

seancorfield 2023-06-10T17:15:57.677799Z

https://github.com/seancorfield/next-jdbc/issues/254

lukasz 2023-06-10T17:20:09.732379Z

Because I had to write java interop before to figure this out - having a direct (and way better informed) support in next.jdbc would be amazing. (gonna leave that comment in GH too)

seancorfield 2023-06-10T17:33:31.644829Z

Again, just with the caveat that this will tell you whether next.jdbc thinks it has one of its own transactions in-flight -- but "transactions" can be constructed ad-hoc as well. And this won't tell you about save points, which is a whole other can of worms.

seancorfield 2023-06-10T18:34:56.993389Z

@mark354 @lukaszkorecki You can try that out with the latest git version on develop or the 1.3.999-SNAPSHOT if you want.

👍 2
seancorfield 2023-06-10T18:35:45.826779Z

I've added extra notes and caveats to the Transactions documentation too (you can read it on GitHub -- it won't be on http://cljdoc.org until I cut a release).

valtteri 2023-06-10T18:38:19.567919Z

I think it’s good with the notes. 👍 I think it’s reasonable to assume that next.jdbc will only know about its ‘own’ transactions

👍🏻 1
Mark Wardle 2023-06-10T18:55:32.596879Z

Thanks @seancorfield I’ll have a look. Thank you very much indeed!

Mark Wardle 2023-06-10T20:07:16.635999Z

I now understand your comment that transactions are somewhat artificial. Ultimately, I can see much of the complexity lies in mutable state in the connection.

Mark Wardle 2023-06-10T20:07:23.752969Z

Thank you again.