Fork me on GitHub
#sql
<
2018-11-21
>
seancorfield00:11:54

Feel free to create a JIRA issue against clojure.java.jdbc if you want me to dig into this -- I'm surprised the difference is quite that big.

seancorfield00:11:21

(I'm up to my neck in work stuff right now, but can take a good look at it later in the week)

seancorfield00:11:15

If you can also give an idea of how many rows you're working with in the ticket, then I can create a repro case to test it with timings.

michaellindon00:11:45

with this table there are only 1500 rows

kirill.salykin08:11:30

How can I use cursor with MySQL to avoid full table load?

kirill.salykin09:11:41

or how I can process every row w/o loading them all in one go? I tried approach from here http://clojure-doc.org/articles/ecosystem/java_jdbc/using_sql.html - but it loads all records

kirill.salykin09:11:19

@seancorfield could you please help with this?

orestis18:11:37

I remember reducible-query being a thing for that @kirill.salykin , but I’m not 100% sure...

☝️ 4
seancorfield18:11:14

@kirill.salykin You'll need to tell us what you've actually tried, exactly, in code -- that page has a lot of code on it! Also, you may need specific parameters and settings to make it work with MySQL -- PostgreSQL was where others have confirmed this works, but that required an enhancement to java.jdbc to make it work (the ability to set connections to readonly and not autocommit). MySQL may require those or similar options. Finally, it's a bit rude to @ someone who is not currently engaged in ongoing channel discussions: the @ mention can cause an alert to be sent to their phone and you did that in the middle of the night 😐

kirill.salykin21:11:06

I tried to lazy process 12000000 records with reduced-query, but all records were loaded.

seancorfield21:11:05

As I said, you'll need to show us your code. The various options you actually use are important. And you may have to look into the MySQL / JDBC documentation to see what additional options you may need.

kirill.salykin08:11:37

ok, clear, thanks

mpenet20:11:00

Fyi you can setup "do not disturb" for a range of hours, it's quite handy

seancorfield20:11:47

Yes, but it's still rude to @ people directly just because you didn't get an answer on the channel in five minutes!

mpenet20:11:56

I do that on most slacks (work included), critical stuff requires an sms

seancorfield20:11:41

If anyone here wants to try Zulip -- an open source service with unlimited search history -- there's a SQL stream here https://clojurians.zulipchat.com/#narrow/stream/152063-sql -- I'm happy to provide support in both venues.

ccann21:11:24

can anyone point me in the right direction on why jdbc/set-db-rollback-only is committing transactions instead of rolling them back? I have code that looks like this:

(defn atomic-queries
  [test-fn]
  (jdbc/with-db-transaction [tx sys/db]
    (jdbc/db-set-rollback-only! tx)
    (binding [*txn* tx]
      (test-fn))))
where *txn* is a dynamic var. I’m running a test that inserts into the db
(deftest notes-for-user
  (is (= {:id 1} (notes/create-note *txn* "cody"))
      (= {:id 2} (notes/create-note *txn* "cody"))))

ccann21:11:07

the first time I run it it succeeds, and subsequent times it fails since the returned ids are incrementing

seancorfield21:11:04

@ccann What does your create-note function look like?

ccann21:11:52

i’m using HugSQL so it’s wrapping some sql:

(defn create-note
  [db user-id ref-id svg-file]
  (insert-one db {:user-id user-id
                  :ref-id  ref-id
                  :svg     (.getBytes svg-file)}))

ccann21:11:02

-- :name insert-one :returning-execute :1
-- :doc Insert one Note
INSERT INTO notes (svg, reference_id, user_id)
VALUES (:svg, :ref-id, :user-id)
RETURNING id;

seancorfield21:11:31

And I assume you're setting atomic-queries as a fixture for clojure.test?

seancorfield21:11:00

I wonder if something insert-one is doing under the hood is messing with the transaction information?

seancorfield21:11:51

(I haven't used HugSQL)