Fork me on GitHub
#sql
<
2018-12-18
>
jumar13:12:26

We have a function similar to clob-to-string here: https://gist.github.com/simonholgate/1632764

(defn declob
  "Turn a clob into a String"
  [clob]
  (when clob
      (with-open [rdr (BufferedReader. (.getCharacterStream clob))]
;;; missing \n?
        (string/join (line-seq rdr)))))
What I can't quite understand why it's joining lines without using \n separator. I think it should (always) be :
(defn declob
  "Turn a clob into a String"
  [clob]
  (when clob
      (with-open [rdr (BufferedReader. (.getCharacterStream clob))]
       ;; Note the separator here
        (string/join "\n" (line-seq rdr)))))

jumar13:12:24

Exactly 🙂

jumar13:12:57

(it's a real issue we've found when showing description stored in the db in the browser - missing new lines)

seancorfield13:12:59

Reading that code, I think they are deliberately producing a single line string and they want the newlines removed.

seancorfield13:12:35

(I deleted my first answer because I want sure if I was reading your question correctly)

seancorfield13:12:28

If you want the newlines retained, why even use line-seq at all?

jumar13:12:21

That's a good question. I'd say just a quick fix of the function that has been around for a long time. I think they copied it from somewhere without really thinking much 🙂.

jumar13:12:39

Glad you mentioned that. I think I'm going with just (slurp rdr)

jumar13:12:13

My aforementoined solution has another disadvantage - it removes a new line at the end of the string.

jumar13:12:33

This is the new implementation:

(defn declob
  "Turn a clob into a String"
  [clob]
  (when clob
    (with-open [rdr (BufferedReader. (.getCharacterStream clob))]
      (slurp rdr))))

jumar13:12:21

And I guess that with-open shouldn't be needed at all (not sure if it closes properly)

jumar14:12:32

It does seem to close the reader properly so it's just this:

(defn declob
  "Turn a clob into a String"
  [clob]
  (when clob
    (slurp (.getCharacterStream clob))))

seancorfield16:12:17

There ya go! So nice and simple!

ccann19:12:57

has anyone ever extended the clojure.java.jdbc/Connectable protocol with, essentially, a mock? I have some code that runs in a transaction (via the with-db-transaction macro) and i’m with-redefs-ing all the database queries, but i can’t redef the macro. I want it to do nothing.

ccann19:12:28

maybe this is a silly idea. My database connection is a mount state which doesn’t get started, so I tried this:

(extend-protocol clojure.java.jdbc/Connectable
    mount.core.NotStartedState
    (add-connection [db _] db)
    (get-level [_] 1))

ccann19:12:31

and I get a stack track about trying to assoc to the notstartedstate, presumably happening somewhere deeper inside with-db-transaction

ccann20:12:11

(defn- fake-transaction
  [test-fn]
  (with-redefs [jdbc/db-transaction* (fn [_ func] (func nil))]
    (test-fn)))
good enough for my purposes

seancorfield20:12:14

@ccann That feels like a code smell to me -- I'd look at teasing apart any business logic from any database persistence there if you can. Mocking out JDBC calls isn't something you should need to do in well-structured code, IMO.

ccann20:12:34

yeah, definitely

ccann21:12:27

I’m torn because I have a series of queries and updates I need to do under 1 postgres transaction (because I’m doing row-level concurrency locking via SELECT … FOR UPDATE ) and I haven’t figured out how to factor that behind a protocol as weavejester recommends in his duct concept of Boundaries

ccann21:12:54

I have my SQL defined via HugSQL and I have shim functions in the ns that loads the SQL. So I’d have to put all the functionality in those shims but that feels wrong

seancorfield21:12:56

Would simply making your test rollback the transaction be sufficient? i.e., wrap the whole test in a transaction set to rollback?

donaldball21:12:22

I have tilted at such a windmill in the past and found the most happiness when I had a set of business components that could close over something clojure.jdbc treated as a db connection, and within the context of a db transaction, I constructed components on the transaction value yielded by with-db-transaction.

ccann21:12:49

the system under test here actually mocks out the database commands and I test the DB interface separately (for speed and separation of concerns)

seancorfield21:12:22

Hmm, sounds like the wrong level of abstraction to me but I'd probably have to see the full code and the tests to offer better suggestions...

ccann21:12:35

yeah i’m finding it really hard to explain in the abstract

seancorfield21:12:23

Testing around persistence can be tricky. Sometimes just swapping the disk-based DB for a memory-based DB is sufficient to make the full tests viable without mocks. Sometimes you can just wrap tests in a rollback transaction and that's still fast enough. But if you're aiming for "separation of concerns" and you're still mocking out low-level JDBC calls... that doesn't sound very "separate" to me 🙂

ccann21:12:46

yeah, you make a great point

ccann21:12:58

mocking them felt silly

ccann21:12:16

I was trying to avoid writing tests that go from HTTP request to database and back, preferring to test the HTTP interface with a fake database and the database functions without the noise of the web stuff

seancorfield21:12:40

If all that DB stuff is wrapped up in a function that your handler calls, you could just mock that function when testing the handler. Then write separate tests for that function (with real DB operations).

ccann21:12:02

that’s essentially what’s going on, except it’s functions not function

seancorfield21:12:13

But I'd say the solution here is to introduce a nicely mockable layer between your handler logic and your persistence logic.

ccann21:12:41

agreed, and I’ll have to find a way to push the transaction down behind that mock

seancorfield21:12:47

The transaction level operation is a single function, surely? (that calls a bunch of other stuff)

ccann21:12:43

yeah it is, and the content of that function is basically 3 function calls, 2 are db commands and 1 is business logic, and I’d like to not push that business logic behind the mock

ccann21:12:23

so I could have the mockable layer expose a function which takes my business logic function as an argument

seancorfield21:12:09

☝️:skin-tone-2: Yup, that was going to be my next suggestion -- separating persistence and business logic.

👍 8
ccann21:12:34

thanks for all the help 🙂