Fork me on GitHub
#sql
<
2020-02-18
>
Ramon Rios10:02:36

Hello everyone!

Ramon Rios10:02:57

(-> (sql/query ds ["select * from customers"] {:builder-fn rs/as-unqualified-lower-maps}) lazy-seq)
Would it be a good idea to transform this return of data into a lazy-seq, considering the fact that i could get a huge amount of data and then avoid issues?

dharrigan10:02:22

I believe that is what plan is for

dharrigan10:02:17

Alternatively, you could use a cursor?

seancorfield17:02:22

Yup, what @dharrigan said @ramon.rios -- query (which is the same as execute! really) gives you a complete, realized vector of hash maps -- converting that to a lazy-seq is pointless: the data is already all in memory. What you want is plan and then reduce over that.

seancorfield17:02:43

plan also lets you take advantage of lazy/streaming result sets on the JDBC side (even tho' reducing is eager on the Clojure side), because you can set options to get the JDBC sdriver/database to read results N rows at a time and process them in a reduction, so you can deal with result sets much larger than will fit in memory. See the Tips & Tricks section of Friendly SQL Functions (in the GitHub repo -- it has doc changes that have not yet been published in a release to http://cljdoc.org).

Ramon Rios18:02:34

Ok. I got that plan is the one who i need to pick up. I'll try out on what i want to achieve, thanks 🙂