Fork me on GitHub
#xtdb
<
2021-03-25
>
seancorfield06:03:22

I’m very new to Crux: just been playing with it since @taylor.jeremydavid gave a talk to Clojure Provo about it — and it addresses several of the issues I had with Datomic (yay!). I have a question about processing query results: is there an idiomatic way to run a transform over query results as they are retrieved? In next.jdbc I would use transduce or reduce over plan to apply a transformation to each row and possibly accumulate results directly as the result set is retrieved from the database. What would be the equivalent in Crux?

jarohen09:03:37

Hey @U04V70XH6, welcome 👋 We expose an open-q function which returns a Closeable Iterator of results - idiom is as follows:

(with-open [res (crux/open-q db '{:find ..., :where ...})]
  (->> (iterator-seq res)
       ...))

seancorfield16:03:05

I was hoping there was something that exposed a reducible context, but I guess I can wrap open-q up in one (after all, that’s what clojure.java.jdbc and next.jdbc do under the hood).

refset17:03:44

Funnily enough we just opened a PR to add IReduceInit to open-q today 🙂 would that be sufficient? https://github.com/juxt/crux/pull/1477/files

jarohen17:03:34

I'd err on the side of us not merging that PR as it stands, FWIW - but certainly an idea for future API development

jarohen17:03:43

an external wrapper to open-q (as opposed to wrapping the result of open-q) would probably be a better shout, as @U04V70XH6 suggested

jarohen17:03:00

I mean, wrapping the result works, as the test suggests, but it might not guarantee everything that the next.jdbc equivalents do - particularly, the ability to reduce multiple times over the same plan

seancorfield17:03:47

If you plan to do resource management via IReduceInit, it should be in charge of opening the resource (as well as closing it) — not just close it at the end.

seancorfield17:03:32

Since open-q already does the opening of the resource, you can’t make its result directly reducible. You need a wrapper.

jarohen17:03:42

Right, yep 🙂

refset22:03:21

^ the Clojure-Provo talk I gave on Tuesday, referenced above, is now available at https://www.youtube.com/watch?v=wfaExQp9kes AMA :)

👍 12
seancorfield22:03:50

It was a great talk! And it’s why I’m here and playing with Crux 🙂

🙏 12