This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-06-27
Channels
- # announcements (1)
- # babashka (2)
- # beginners (64)
- # cider (1)
- # cljs-dev (49)
- # cljsrn (2)
- # clojure (49)
- # clojure-europe (3)
- # clojure-norway (1)
- # clojure-spec (7)
- # clojurescript (116)
- # conjure (3)
- # cursive (4)
- # datomic (1)
- # emacs (2)
- # fulcro (15)
- # graalvm (10)
- # kaocha (1)
- # leiningen (4)
- # meander (1)
- # music (1)
- # off-topic (7)
- # re-frame (37)
- # reagent (3)
- # releases (1)
- # rewrite-clj (6)
- # sci (4)
- # shadow-cljs (16)
- # sql (8)
- # tools-deps (16)
- # xtdb (5)
@antonmos This is your fault! 🙂
(deftest fold-rs-test
(let [ds-opts (jdbc/with-options (ds) (default-options))]
(testing "foldable result set"
(let [result
(r/fold 2 r/cat r/append!
(r/map (column :FRUIT/NAME)
(jdbc/plan ds-opts ["select * from fruit order by id"])))]
(is (= 4 (count result)))
(is (= "Apple" (first result)))
(is (= "Orange" (last result)))))))
(it could just be r/foldcat
but I needed a batch size of 2 to verify it was actually doing the work on multiple threads!)
This is on the foldable branch -- and it is only implemented for plan
on a DataSource
but by the end of the weekend, it'll be fully implemented and documented and released I expect 🙂
Implemented for plan
on every data type now, with tests, and documentation. There's definitely some caveats to using this for very large data sets since there's no back pressure here -- it's possible for a very large data set to be read in faster than the reduction of batches can keep up.
FYI, the multi-rs branch has been merged to develop now and documentation added about the :multi-rs true
option. I'll be cutting a 1.1.x release in the next few days, but would appreciate some folks trying the latest SHA for develop: {:git/url "
Changes since the 1.0.478 release currently include:
* Address #125 by making the result of plan foldable (in the clojure.core.reducers
sense).
* Address #124 by extending next.jdbc.sql.builder/for-query
to support :`top` (SQL Server), :limit
/ :offset
(MySQL/PostgreSQL), :offset
/ :fetch
(SQL Standard) -- for find-by-keys
.
* Address #116 by adding a :multi-rs
option to execute!
to retrieve multiple result sets, for example from stored procedure calls or T-SQL scripts.
* Allow :all
to be passed into find-by-keys instead of an example hash map or a where clause vector so all rows will be returned (expected to be used with :offset
etc to support simple pagination of an entire table).
* Add :columns
option to find-by-keys
(and get-by-id
) to specify a subset of columns to be returned in each row. This can also specify an alias for the column and allows for computed expressions to be selected with an alias.
It's a lot of new stuff, hence the 1.1.x release, but I would appreciate some additional community testing, if anyone can spare some time.