2025-01-10 missionary | Clojure Slack Archive

missionary 2025-01-10

Eric Dvorsak 2025-01-10T12:24:18.893719Z

I made wrappers for my db calls to run them via blk:

(defn ?execute-one
  "Warning: to run multiple statements in a transaction, use
  `?with-transaction` instead with the regular `execute-one!`.
  Otherwise this will NOT work as you expect

  Executes a SQL query that returns a single row asynchronously via Missionary task.
   
   Parameters:
     sql-map - HoneySQL query map to be formatted into SQL
     opts    - Optional map of execution options, defaults to {:builder-fn default-builder-fn}
   
   Returns:
     Missionary task containing single result map or nil if no results

   Throws:
     ExceptionInfo with PostgreSQL error details"
  ([sql-map]
   (m/via m/blk (execute-fn jdbc/execute-one! sql-map)))
  ([sql-map opts]
   (m/via m/blk (execute-fn jdbc/execute-one! sql-map opts))))

Sometimes I need to run multiple statements in a transaction, I have the following macro for that:

(defmacro ?with-transaction
  "
  same as `with-transaction` except it will return a task and it will
  run it in blocking thread pool. Use the regular `execute-one!` and
  `execute!` functions to run statements and NOT their task equivalent
  `?execute-one` and `?execute`
  
  check with-transaction for more details
  "[[dbsym & opts] & body]
  `(m/via m/blk
     (if (instance? IDeref ~dbsym)
       (next.jdbc/with-transaction [t-conn# (deref ~dbsym) ~@opts]
         (binding [~dbsym (delay t-conn#)]
           ~@body))
       (next.jdbc/with-transaction [t-conn# ~dbsym ~@opts]
         (binding [~dbsym t-conn#]
           ~@body)))))

I ended up doing things like this (for single statements, a task that executes the statement, for transactions, the transaction is a task and the statements within should not be tasks) and it works, because my original method, using the vanilla version of with-transaction without the m/via m/blk and using ?execute within was not working, as the execute tasks were not run. I don't understand why it didn't work exactly, is it because with-transaction is a macro?

leonoel 2025-01-10T14:21:58.099819Z

it may be a dynamic scope issue, m/via doesn't capture bindings

Eric Dvorsak 2025-01-10T14:31:35.823239Z

interesting I didn't know binding didnt' work or forgot about it

Eric Dvorsak 2025-01-10T14:31:38.264819Z

(def ^:dynamic *v* "original")
(def r "orignal")

(defn f [ctx]
  (println ctx
           :binding *v*
           :redef r))

(with-bindings {#'*v* "rebinding"}
  (with-redefs [r "redef"]
    (f "clojure")
    (m/? (m/via m/blk (f "via m/blk")))))

Eric Dvorsak 2025-01-10T14:31:44.100889Z

clojure :binding rebinding :redef redef
via m/blk :binding original :redef redef

Eric Dvorsak 2025-01-10T14:31:50.299049Z

but redef works

leonoel 2025-01-10T14:33:29.001579Z

yes this is expected

leonoel 2025-01-10T14:38:15.494939Z

as an aside - if you block a thread waiting on a m/via you're using two threads, you could have used just one by calling the body directly

Eric Dvorsak 2025-01-10T14:41:05.889039Z

@leonoel are you talking about using ?execute within a transaction? I'm not doing that I call the non waiting version

leonoel 2025-01-10T14:42:14.848789Z

oh, I misunderstood then

leonoel 2025-01-10T14:42:56.837819Z

can you post the full example ? If there's no thread switching the dynamic scope should be alright

Eric Dvorsak 2025-01-10T16:17:56.838199Z

@leonoel so far this is my smallest exemple

(ns brian.server-components.repo
  (:require
   [missionary.core :as m]
   [mount.core :refer [defstate] :as mount]
   [next.jdbc :as jdbc]
   [next.jdbc.connection :as jdbc.connection])
  (:import
   (clojure.lang IDeref)
   (com.zaxxer.hikari HikariDataSource))
  (:refer-clojure :exclude [format]))


(defmacro with-transaction
  [[dbsym & opts] & body]
  `(if (instance? IDeref ~dbsym)
     (next.jdbc/with-transaction [t-conn# (deref ~dbsym) ~@opts]
       (binding [~dbsym (delay t-conn#)]
         ~@body))
     (next.jdbc/with-transaction [t-conn# ~dbsym ~@opts]
       (binding [~dbsym t-conn#]
         ~@body))))

(defn- close-pool [^HikariDataSource pool] (.close pool))

(defstate ^:dynamic *testdb*
  :start
  (let [config {:minimum-idle       10
                :maximum-pool-size  10
                :username           "user"
                :password           "password"
                :driverClassName "org.postgresql.Driver"
                :jdbcUrl "jdbc:"}
        ^HikariDataSource
        conn (jdbc.connection/->pool
              HikariDataSource
              config)]
    conn)
  :stop
  (some-> *testdb* close-pool))

(defn ?task1 [x]
  (m/sp (println x)))

(mount/start [*testdb*])

(m/?
 (m/via m/blk
   (with-transaction [*testdb*]
     (let [x (m/? (?task1 "hello"))
           y (m/? (?task1 "world"))]
       (println :x x :y y)))))

Eric Dvorsak 2025-01-10T16:18:17.395999Z

it will print hello world :x nil :y nil

Eric Dvorsak 2025-01-10T16:19:33.454259Z

in my real world example that was my issue as well, expect task1 was a query and I was expecting to do something with the result in the second call but since it came as nil the whole thing is not working

leonoel 2025-01-10T17:22:18.534109Z

what is the actual implementation of ?task1 ? this one clearly returns a task completing with nil

Eric Dvorsak 2025-01-10T17:32:38.565779Z

@leonoel my bad facepalm

(ns brian.server-components.repro
  (:require
   [missionary.core :as m]
   [mount.core :refer [defstate] :as mount]
   [next.jdbc :as jdbc]
   [next.jdbc.connection :as jdbc.connection])
  (:import
   (clojure.lang IDeref)
   (com.zaxxer.hikari HikariDataSource))
  (:refer-clojure :exclude [format]))


(defmacro with-transaction
  [[dbsym & opts] & body]
  `(if (instance? IDeref ~dbsym)
     (next.jdbc/with-transaction [t-conn# (deref ~dbsym) ~@opts]
       (binding [~dbsym (delay t-conn#)]
         ~@body))
     (next.jdbc/with-transaction [t-conn# ~dbsym ~@opts]
       (binding [~dbsym t-conn#]
         ~@body))))

(defn- close-pool [^HikariDataSource pool] (.close pool))

(defstate ^:dynamic *testdb*
  :start
  (let [config {:minimum-idle       10
                :maximum-pool-size  10
                :username           "user"
                :password           "password"
                :driverClassName "org.postgresql.Driver"
                :jdbcUrl "jdbc:"}
        ^HikariDataSource
        conn (jdbc.connection/->pool
              HikariDataSource
              config)]
    conn)
  :stop
  (some-> *testdb* close-pool))

(defn ?task1 [x]
  (m/via m/blk (println x) 1))

(defn ?transaction []
  (m/via m/blk
    (with-transaction [*testdb*]
      (let [x (m/? (?task1 "hello"))
            y (m/? (?task1 "world"))]
        (println :x x :y y)))))

(mount/start [*testdb*])

(m/? (transaction))

Eric Dvorsak 2025-01-10T17:34:47.931659Z

that was the construct that caused me an issue

Eric Dvorsak 2025-01-10T17:35:02.683149Z

:x #object[missionary.impl.Sequential$Process 0x25e35d78 missionary.impl.Sequential$Process@25e35d78] :y #object[missionary.impl.Sequential$Process 0x25e35d78 missionary.impl.Sequential$Process@25e35d78]
hello
world

leonoel 2025-01-10T18:12:40.926759Z

a leaking missionary.impl.Sequential$Process object is symptomatic of an indirect call to m/? from an m/sp block

leonoel 2025-01-10T18:12:58.406229Z

but I don't see any m/sp in your snippet

Eric Dvorsak 2025-01-10T18:14:26.919479Z

what would an indirect call from an m/sp block look like?

leonoel 2025-01-10T19:02:49.217589Z

https://github.com/leonoel/missionary/issues/127

Eric Dvorsak 2025-01-10T19:14:16.835359Z

oh I see, well yeah it seems like my case is slightly different

leonoel 2025-01-10T19:58:23.620099Z

you still have a potential issue related to dynamic scope

leonoel 2025-01-16T09:19:33.093549Z

have you solved your problem @yenda1 ?

Eric Dvorsak 2025-01-16T12:48:44.067949Z

I don't have the issue when I use the wrapper I described, although it makes it easy to have one if for instance your accidently introduce a task in there

Clojurians Log v2

missionary 2025-01-10