Fork me on GitHub
#sql
<
2019-10-29
>
gmercer11:10:46

;; this works
(defn slack-allowed []
   (let [ds (jdbc/get-datasource datasource)]
     (with-open [con (jdbc/get-connection ds)]
       (conj '()
             (jdbc/execute! con ["SELECT ?" 7])
             (jdbc/execute! con ["SELECT ?" 8])))))

;; this doesn't - execute() is called on closed connection
(defn slack-denied []
   (let [ds (jdbc/get-datasource datasource)]
     (with-open [con (jdbc/get-connection ds)]
       (map (fn [num] (jdbc/execute! con ["SELECT ?" num])) (range 7 9)))))

wotbrew11:10:17

map is lazy my friend

wotbrew11:10:38

so the function is evaluated after the with-open has returned. (probably when the result is printed at the repl)

gmercer11:10:11

I can call execute! multiple times in a with-open but not using map - any ideas?

gmercer11:10:29

Forgot to mention the above is using next.jdbc

dharrigan11:10:31

try using mapv

dharrigan11:10:33

or wrap in a (doall

gmercer11:10:02

@dharrigan thanks .. that's it - and the repl trying to print forces the execution after the con is closed

dharrigan11:10:37

you're welcome (also to danstone for further clarification!) 🙂

Joe Lane17:10:49

I'm returning 1mil records from a next.jdbc/execute! call. Is there a way to avoid realizing the entire resultset into memory? I looked into next.jdbc/plan but couldn't discern if that would help in this situation.

seancorfield17:10:37

@joe.lane plan is designed for that situation, yes.

seancorfield17:10:18

Because it lets you process (`reduce`) the entire result set without realizing it into Clojure data.

Joe Lane18:10:44

Fantastic @seancorfield! My scenario is put all results on a channel one at a time. Does it make sense to just a/put! inside a reduce call?

seancorfield18:10:15

Yeah, although bear in mind: a) you'll need to realize each row into Clojure data to put them on a channel and b) you'll still end up with all that data realized into memory depending on how much is buffered and how fast it gets processed out of the channel on the other end.

noisesmith18:10:48

with put!, only the global backpressure on put! itself will keep all results from being in memory

seancorfield18:10:07

(and I suspect folks may advise against coupling blocking I/O with core.async... and there it is)

😄 4
noisesmith18:10:12

if you use >!! instead, the realization will have backpressure based on the channel buffer and consumption this should be outside a go block NB

Joe Lane18:10:23

Thanks guys