Fork me on GitHub
#sql
<
2023-03-17
>
slipset12:03:34

@seancorfield to what extent have you spent time optimizing jdbc.next in terms of speed. Reason I’m asking is if the answer is “a lot” I won’t spend any time spelunking, but if the answer is “not at all” I’d suspect there could be some quick wins, and I’d be interested in looking into it.

seancorfield19:03:14

Early on, there was a lot of back and forth between myself and @U055NJ5CC about how to get the most performance out of it -- he's a real "performance nut" so that was very valuable.

seancorfield19:03:50

You would need to use a custom builder based on a record, to improve on basic hash map builder performance -- there's an example in the test suite.

seancorfield19:03:55

In general, I'm always fairly resistant to adding new options/features for some edge case if it would introduce a performance overhead for "all users". It's fine if something can be done in a way that only users of a particular feature feel the performance pain of that feature.

seancorfield19:03:42

All that said, I haven't spent much time recently looking at performance optimizations so if you want to spend time on that, feel free. Just remember: no breaking changes 😉

slipset19:03:00

Cool, thanks for the answer.

ikitommi09:03:47

• jdbc-next is a great and feature-rich library, we use it in all the projects • I believe there would be some small performance wins still available, but if one needs brutal performance, much thinner wrapper on top of java should be created • some perf numbers from 4y back in the 🐷 repo comparing java & clojure options: https://github.com/metosin/porsas • in the (silly) Techempower Bencmarks, my old clojure-entry seems to be still https://www.techempower.com/benchmarks/#section=data-r21&amp;test=db on single query. It’s running on really old version of libs, I think it would be reasonable to get into top 5 with small rewrite. no ambitions for doing that

2
slipset19:03:10

In other fun stuff today, I mada a change from (merge {.,} some-map-from-jdbc-next) to (assoc some-map-from-jdbc-next …) and suddenly tests b0rked because nippy was unable to serialize something something. Turns out that jdbc.next adds metadata to the maps it returns so they’re navable.

seancorfield19:03:46

Someone reported this as a bug on next.jdbc and I told them to report it as a bug on nippy instead 🙂

seancorfield19:03:03

Or just strip metadata from data before serializing it.

slipset19:03:22

(defn post-read*
  [entity {:keys [prop->column-type-hint transform-keys entity-type post-read-please-don't-use]
           :or {post-read-please-don't-use identity}
           :as repo-config}]
  (-> (with-meta entity {}) ;; next.jdbc returns results with metadata which makes
                            ;; nippy fritz because it doesn't know how to seralize it
      (rename-keys transform-keys)
      (postgres/coerce postgres/read-coerce prop->column-type-hint)
      post-read-please-don't-use
      (core/add-transient :entity-type entity-type)))

seancorfield19:03:49

post-read-please-don't-use -- what an awesome symbol name!

slipset19:03:24

Thank you 🙂 It’s an escape hatch that should be used sparingly.

seancorfield19:03:20

post-read-please-don't-use-unless-you-really-have-to 🙂

seancorfield23:03:28

For anyone using next.jdbc.connection/component, the develop branch (and the latest 1.3.999-SNAPSHOT) now allows for the db-spec argument to contain :init-fn, to specify a function that should be called on the newly-created datasource, when start is called on the returned Component, so you can have automatic customization of the Java object or automatic database initialization etc.