This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-06-03
Channels
- # announcements (6)
- # babashka (14)
- # beginners (17)
- # biff (3)
- # calva (19)
- # circleci (3)
- # clj-on-windows (1)
- # cljdoc (21)
- # cljs-dev (6)
- # clojure (119)
- # clojure-australia (2)
- # clojure-europe (28)
- # clojure-france (3)
- # clojure-norway (12)
- # clojure-survey (2)
- # clojure-uk (7)
- # clojurescript (25)
- # core-typed (1)
- # cursive (11)
- # datomic (53)
- # emacs (14)
- # events (1)
- # gratitude (1)
- # holy-lambda (21)
- # integrant (2)
- # jobs (1)
- # jobs-discuss (3)
- # juxt (3)
- # kaocha (1)
- # lsp (17)
- # nbb (14)
- # off-topic (25)
- # pathom (11)
- # re-frame (24)
- # releases (1)
- # remote-jobs (2)
- # rewrite-clj (10)
- # shadow-cljs (11)
- # sql (3)
- # tools-build (6)
- # tools-deps (83)
- # vim (26)
- # xtdb (10)
Hi. I have a scenario where i have to process records from a table having around 10 million records. I want the records to be fetched in batches and processed instead of all the records as it runs into heap space and memory issues. I tried with jdbc/fetch-lazy
and jdbc/cursor->lazyseq from funcool/clojure.jdbc. But it doesnt quite seem to do the job. Any help would be appreciated. Thank you
That library hasn't had any updates in six years. The guy who created it started by copying code from the Clojure Contrib library org.clojure/clojure.java.jdbc and removing all the copyright/license stuff until he was called out on the Clojure mailing list about it. Most people either use org.clojure/clojure.java.jdbc (which is also no longer maintained) or next.jdbc which is actively maintained and supports the scenario you want via plan.
Thanks @U04V70XH6. Let me try with plan.