Fork me on GitHub
#sql
<
2022-10-31
>
Jan Winkler18:10:04

Hi. I'm wondering if implementation of p/Preparable for next.jdbc.default-options/DefaultOptions is correct. DefaultOptions wraps a connectable, i.e., most likely a DataSource, and delegates to it in the p/Preparable implementation. However there is no implementation of p/Preparable for DataSource, which results in runtime errors like "No implementation of method: :prepare of protocol: #'next.jdbc.protocols/Preparable found for class: com.zaxxer.hikari.HikariDataSource" when invoking p/prepare on DefaultOptions. I could easily provide p/Preparable implementation for DataSource, but it feels wrong. Even the p/Preparable docstring says "implementation is provided for Connection only". There must be a reason behind that, I just don't see what it is. So, it seems to me either the p/Preparable implementation should be removed, or other implementations (at least for DataSource) should be added. Or -- maybe -- the DefaultOptions implementation could call (p/get-connection connectable), but I'm not sure if that would not break something else. I'm willing to cook up a patch, but I'd like to collect some opinions first.

seancorfield19:10:50

Connection-based operations work with unwrapped options, as explained in the documentation, and you need to manually re-wrap the connection if you want to pass options further in. Think of Connection as the "bottom type" here in JDBC land: it's a raw Java type and you're operating at the most efficient layer. Preparable is deliberately designed to work with just bare Connection objects -- so that control of the connection (open/close/options) is external to Preparable in user-land. I feel that adding default-options was probably a mistake -- I should have held stronger on resisting that. I don't like it and I think it encourages people to use non-default builders globally, which I think is bad. But people complained so much about the default builder and having to explicitly provide :builder-fn in every call that I got tired of the complaints... 🙂

Jan Winkler20:10:34

Well, that's exactly what I'm doing: I want to use unqualified* kebabs everywhere. 🙂 I didn't like default-options as well, but at the same time I wanted to trust the design. In the end, I'll probably drop the default-options and pass the options explicitly. It seems to be considerably less annoying than it actually looks. However, I'd like to understand if the DefaultOptions implementation of Preparable is broken or if there actually is a proper way to use it. --- *I borderline hate qualified keywords. They look so tempting, but almost always end up a liability.

seancorfield20:10:20

Not broken. This is an intentional design as explained above.

seancorfield20:10:33

Qualified keywords are idiomatic. Get used to them 🙂

Jan Winkler20:10:11

Oh, I see! If I wrapped a DataSource in default options, get-connection won't magically carry the options over. I'd have to re-wrap the conn in DefaultOptions again. I wasn't aware of with-options and its docs.

seancorfield21:10:28

Yeah, it's an inherent tension between trying to work with raw Java types (for efficiency) vs Clojure types (for wrapping).

Jan Winkler06:11:57

Just dropping back to say thank you for your time. I had to scram yesterday.

seancorfield17:11:22

NP. Hearing other people's pain points with the library is always useful feedback, even if it can be frustrating for me at times (esp. this pain point since it is essentially caused by a decision I deeply regret -- adding a default options machinery!).

Max23:10:04

I know this issue’s been closed for quite a while, but I came across the behavior described in https://github.com/seancorfield/next-jdbc/issues/172 again today. I’m having trouble understanding the concern in the most recent message, wouldn’t it be fairly straightfoward to wrap the https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/default_options.clj#L43 with one that took conn and re-added the options? On a practical note, literally every project I have worked on that uses next.jdbc has implemented its own internal version of with-transaction that re-adds the options, so in a very real way, not having this feature is pushing the pain onto users.

seancorfield23:10:25

I wish I'd never added default options. It was a bad idea and the docs explain why you need manual rewrapping here (and in the case Jan Winkler asked about today -- above). I am not adding anything to further encourage the use of this.

seancorfield23:10:34

What part of that recent message can I clarify @U01EB0V3H39? Currently, code that uses with-transaction is guaranteed that the bound value is a native Java Connection object. Auto-rewrapping breaks that promise and would break code out there in the wild.

seancorfield23:10:53

(jdbc/with-transaction [tx some-ds]
  (.someConnectionMethod tx)) ; this would breaking if auto-wrapping were added

Max23:10:52

I guess I’m surprised that the contract is based on the Java type and not another next.jdbc protocol, ie something you can pass to execute! et al

seancorfield23:10:11

The code above is Java interop -- nothing to do with protocols.

seancorfield23:10:51

tx is a java.sql.Connection -- a plain Java object.

Max23:10:57

Ok I see, the risk is that existing code assumes that it’s always a Java object. Perhaps in retrospect it would’ve been preferable to make a different guarantee, but it’s out in the wild now.

seancorfield23:10:46

A key tenet of next.jdbc for performance is that it relies heavily on and exposes native JDBC types.

seancorfield23:10:30

The "problem" here is that I caved in to a bunch of complaining about being explicit about :builder-fn options and I added the stupid default options nonsense. It was a terrible decision.

seancorfield23:10:24

The logging stuff is a similarly bad idea but I think far fewer people actually use that...

Max23:10:24

I suppose another view of this is that next.jdbc only claims to be a low-level jdbc wrapper, but a bunch of non-low-level stuff has gotten added over time. The slightly higher level wrapper(s) seems to never have emerged.

seancorfield23:10:00

I wish people would just use the default builder and get over it... 🙂

Max23:10:32

Personal opinions aside, I think a useful tenet for low-level wrappers is to not be too opinionated. People have all kinds of reasons for doing what they do, but at least if the layers were separate then people could choose which opinionated next.jdbc-powered lib they agreed with

👍 1
seancorfield23:10:37

I think I should have left it with the explicit options and folks could have built wrappers that passed options around if they wanted.

Max23:10:49

Exactly. Do you have similar feelings about the next.jdbc.sql helpers? They probably aren’t as obtrusive I guess

seancorfield23:10:53

I have been very tempted to remove all the documentation for the default options and logging stuff (and just leave the code in for folks who are using them today) and continue on as if they never existed 🙂 Tempted several times.

seancorfield23:10:25

The helpers have a very tight scope and they exist mostly to aid migration from c.j.j which had similar API functions.

seancorfield23:10:43

I have been able to resist almost all calls to extend those 😉

Max00:11:19

That makes sense. I find the insert/update ones get used a fair amount in the wild, but mostly because the equivalent queries for those are repetitive to define yourself.

seancorfield00:11:42

There's always HoneySQL 🙂

Max00:11:10

Of course. Even then if all you want to do is insert a map into a table, insert! is significantly more concise.