This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-07-25
Channels
- # babashka (9)
- # beginners (56)
- # calva (18)
- # clj-kondo (2)
- # clojars (2)
- # clojure (46)
- # clojure-boston (1)
- # clojure-europe (4)
- # clojurescript (10)
- # css (1)
- # data-science (2)
- # emacs (10)
- # girouette (1)
- # helix (10)
- # jobs-discuss (4)
- # malli (2)
- # off-topic (28)
- # polylith (5)
- # re-frame (4)
- # reitit (8)
- # releases (6)
- # rewrite-clj (1)
- # sci (44)
- # sql (10)
- # tools-deps (31)
Does it makes sense to have a flag to automatically apply datafiable-row
on the row when reducing over the result-set using plan
?
My main use case of plan
is for streaming large number of rows, but I'm fine with each row being implicitly realized during the reduction step.
I have something like this, which I will have in a db utils ns, and allow the consumers to call these instead of using plan in a bespoke way everywhere. But for this, the consumers need to deal with rs/datafiable-row
in the reducing functions that they pass, which I don't want them to do.
(extend-protocol ReducibleStream
java.sql.Connection
(reduce-stream
[this sql-params f init]
(with-autocommit-off [conn this]
(reduce f init (jdbc/plan conn sql-params (stream-options)))))
(transduce-stream
([this sql-params xf f]
(transduce-stream this sql-params xf f (f)))
([this sql-params xf f init]
(with-autocommit-off [conn this]
(transduce xf f init (jdbc/plan conn sql-params (stream-options))))))
javax.sql.DataSource
(reduce-stream
[this sql-params f init]
(with-open [conn (jdbc/get-connection this)]
(reduce-stream conn sql-params f init)))
(transduce-stream
([this sql-params xf f]
(transduce-stream this sql-params xf f (f)))
([this sql-params xf f init]
(with-open [conn (jdbc/get-connection this)]
(transduce-stream conn sql-params xf f init)))))
I know I could use the raw underscored column names directly in the reducing function, but just wondering if it makes sense to have a flag to automatically datafy the row.
So that transducing functions are reusable, where I'll be dealing with the decorated row names (qualified, kebab-cased).
Nvm, I guess I have a workaround for now
(reduce-stream
[this sql-params f init]
(with-autocommit-off [conn this]
(reduce (fn [acc row] (f acc (datafiable-row row conn)))
init
(jdbc/plan conn sql-params (stream-options)))))
(transduce-stream
([this sql-params xf f]
(transduce-stream this sql-params xf f (f)))
([this sql-params xf f init]
(with-autocommit-off [conn this]
(transduce (comp (map #(datafiable-row % conn)) xf)
f
init
(jdbc/plan conn sql-params (stream-options))))))