This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-05-22
Channels
- # announcements (1)
- # beginners (109)
- # boot (2)
- # calva (26)
- # cider (6)
- # circleci (6)
- # cljsrn (3)
- # clojure (77)
- # clojure-dev (5)
- # clojure-europe (28)
- # clojure-finland (1)
- # clojure-hamburg (1)
- # clojure-italy (21)
- # clojure-japan (13)
- # clojure-nl (36)
- # clojure-spec (22)
- # clojure-sweden (4)
- # clojure-uk (105)
- # clojurescript (91)
- # community-development (8)
- # cursive (60)
- # datascript (3)
- # datomic (4)
- # emacs (33)
- # fulcro (19)
- # graalvm (38)
- # hoplon (4)
- # instaparse (1)
- # jobs (1)
- # leiningen (22)
- # off-topic (14)
- # pathom (2)
- # perun (4)
- # planck (5)
- # re-frame (10)
- # reagent (1)
- # reitit (11)
- # rum (11)
- # shadow-cljs (97)
- # tools-deps (82)
- # vim (53)
I found out two things last night... 1) clojure.jdbc doesn't seem to use connection pooling by default. So that is something I need to fix first. and then I can do an insert-multi!
instead of loads of insert!
s
I am trying to use with-db-connection
but it isn't clear to me what the conn
is in the example. :thinking_face:
If that’s not working out @thomas then it might be worth a look at the tuple list params in HugSQL, they work well. Depending on the source data (files, S3, or another db SELECT
) then Embulk is worth a look too as it handles failures better.
I am looking at insert-multi!
at the moment... but partition
is splitting my seq in lists...and not vectors...
@thomas https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#addBatch(java.lang.String) would be the crude-but-efficient low level approach
https://www.hugsql.org/#param-tuple says
BATCH INSERTS: It should be noted that Tuple List Parameter support is only support for SQL INSERT...VALUES (...),(...),(...) syntax. This is appropriate for small-ish multi-record inserts. However, this is different than large batch support. The underlying JDBC driver for your database has a limit to the size of the SQL and the number of allowed bind parameters. If you are doing large batch inserts, you should map or doseq over your HugSQL-generated insert function within a transaction.
do you actually need a transaction, or is it good enough to use a batch?