Clojurians Log v2

there's a protocol to implement for this in next.jdbc, whose name I always forget... 👀

https://cljdoc.org/d/com.github.seancorfield/next.jdbc/1.3.955/api/next.jdbc.prepare#SettableParameter

although I don't recall having to do that for ZDT, I thought either next.jdbc or the PG JDBC driver supported those OotB... 🤔

I tried requiring [next.jdbc.date-time] that is suppose to do the setup to work with zdt but still having the same issue

And that's expected since it doesn't cover java.time.ZonedDateTime I think I need to do something like that then:

(extend-protocol rs/ReadableColumn
    java.sql.Date
    (read-column-by-label [^java.sql.Date v _]     (.toLocalDate v))
    (read-column-by-index [^java.sql.Date v _2 _3] (.toLocalDate v))
    java.sql.Timestamp
    (read-column-by-label [^java.sql.Timestamp v _]     (.toLocalDateTime v))
    (read-column-by-index [^java.sql.Timestamp v _2 _3] (.toLocalDateTime v)))

@seancorfield would you happen to have an example of such setup for next.jdbc ?

@taylor.jeremydavid @jarohen This is the output using clojure 1.12.0

20:10:20.704 [xtdb-tx-subscription-pool-1-thread-1] ERROR xtdb.util -- Uncaught exception:
java.lang.IllegalArgumentException: not all nodes, buffers and variadicBufferCounts were consumed. nodes: [ArrowFieldNode [length=1, nullCount=0], ArrowFieldNode [length=1, nullCount=0], ArrowFieldNode [length=1, nullCount=0], ArrowFieldNode [length=1, nullCount=0], ArrowFieldNode [length=1, nullCount=0], ArrowFieldNode [length=1, nullCount=1], ArrowFieldNode [length=1, nullCount=1], ArrowFieldNode [length=1, nullCount=1], ArrowFieldNode [length=1, nullCount=0]] buffers: [ArrowBuf[48], address:6308528216, capacity:4, ArrowBuf[49], address:6308528224, capacity:1, ArrowBuf[50], address:6308528232, capacity:8, ArrowBuf[51], address:6308528240, capacity:1, ArrowBuf[52], address:6308528248, capacity:1, ArrowBuf[53], address:6308528256, capacity:16, ArrowBuf[54], address:6308528272, capacity:1, ArrowBuf[55], address:6308528280, capacity:8, ArrowBuf[56], address:6308528288, capacity:8, ArrowBuf[57], address:6308528296, capacity:1, ArrowBuf[58], address:6308528304, capacity:8, ArrowBuf[59], address:6308528312, capacity:1, ArrowBuf[60], address:6308528320, capacity:8, ArrowBuf[61], address:6308528328, capacity:1, ArrowBuf[62], address:6308528336, capacity:8, ArrowBuf[63], address:6308528344, capacity:1, ArrowBuf[64], address:6308528352, capacity:8, ArrowBuf[65], address:6308528360, capacity:1] variadicBufferCounts: []
        at org.apache.arrow.vector.VectorLoader.load(VectorLoader.java:98)
        at org.apache.arrow.vector.ipc.ArrowReader.loadRecordBatch(ArrowReader.java:214)
        at org.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:162)
        at xtdb.log.watcher$watch_log_BANG_$reify__27190.accept(watcher.clj:48)
        at xtdb.log$tx_handler$fn__21916.invoke(log.clj:38)
        at clojure.core.protocols$iterator_reduce_BANG_.invokeStatic(protocols.clj:42)
        at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:52)
        at clojure.core.protocols$fn__8256.invokeStatic(protocols.clj:74)
        at clojure.core.protocols$fn__8256.invoke(protocols.clj:74)
        at clojure.core.protocols$fn__8203$G__8198__8216.invoke(protocols.clj:13)
        at clojure.core$reduce.invokeStatic(core.clj:6965)
        at clojure.core$reduce.invoke(core.clj:6947)
        at xtdb.log.NotifyingSubscriberHandler$fn__21949.invoke(log.clj:97)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.base/java.lang.Thread.run(Thread.java:1583)

I guess this is the same issue as before, but instead of hanging, an uncaught exception is thrown?

Interesting, looks to me like a bug in Arrow Java hidethepain

I've opened an issue: https://github.com/xtdb/xtdb/issues/3854

@taylor.jeremydavid Does this happen on non macos/apple silicon?

Asking since this is different than previous isue (hanging)

I can only confirm that with all the dependencies up-to-date, both of the following commands run to completion as expected, on x64 Ubuntu 20.04 (WSL2):

clojure -M:poly test :dev
clj -M:dev:test -m cognitect.test-runner -d components/persistence-xtdb/test

{:mvn/repos {"ossrh-snapshots" {:url ""}}
 :aliases   {:dev  {:extra-paths ["development/src"]

                    :extra-deps  {org.clojure/clojure       {:mvn/version "1.12.0"}
                                  com.xtdb/xtdb-api         {:mvn/version "2.0.0-SNAPSHOT"}
                                  com.xtdb/xtdb-core        {:mvn/version "2.0.0-SNAPSHOT"}
                                  org.clojure/tools.logging {:mvn/version "1.3.0"}
                                  test/xtdb                 {:local/root "components/persistence-xtdb"}}}

             :test {:jvm-opts   ["--add-opens=java.base/java.nio=ALL-UNNAMED"
                                 "-Dio.netty.tryReflectionSetAccessible=true"]
                    :extra-paths ["components/persistence-xtdb/test"]
                    :extra-deps {io.github.cognitect-labs/test-runner {:git/tag "v0.5.1" :git/sha "dfb30dd"}
                                 ch.qos.logback/logback-classic {:mvn/version "1.5.12"}
}
                    :exec-fn cognitect.test-runner.api/test}

             :poly {:jvm-opts   ["--add-opens=java.base/java.nio=ALL-UNNAMED"
                                 "-Dio.netty.tryReflectionSetAccessible=true"]
                    :main-opts  ["-m" "polylith.clj.core.poly-cli.core"]
                    :extra-deps {polylith/clj-poly {:mvn/version "0.2.21"}}}}}

XTDB api 2.0.0-20240910.193517-30 and core 2.0.0-20240910.193517-28

(if I run clj -X:dev:test :dirs '["components/persistence-xtdb/test"]' it runs the tests successfully but then hangs at the end, due to no (shutdown-agents) call and -- I assume -- XTDB having some non-daemon threads active)

Thanks alot @seancorfield

I tried running it but I get a weird error:

Caused by: java.lang.RuntimeException: Unable to find static field: new in class xtdb.types.ClojureForm

ah, that error should be resolvable by upgrading to Clojure 1.12, per https://github.com/xtdb/xtdb/issues/3801#issuecomment-2421779869

I don't know about the repro itself, but is it blocking? or just annoying?

I updated the deps -- Clojure to 1.12 and Poly to 0.2.21 -- and it passes for me on Linux:

clojure -M:poly test :dev
...
Person transacted

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.

Test results: 1 passes, 0 failures, 0 errors.

Execution time: 13 seconds

Does the readme need expanding to make it clearer what the environment and problem is?

Seems to hang when the Cognitect test-runner is used in -X mode -- is that the problem that is "expected" to show up here?

If I run this, it completes successfully:

clj -M:dev:test -m cognitect.test-runner -d components/persistence-xtdb/test

@seancorfield The problem happens on a combination of polylith test, xtdb and macos (apple silicon?)

@taylor.jeremydavid Couldn't run any unit tests using XTDB v2. So I'd say kind of blocking. I'll try out tonight with upgrading clojure to 1.12 and report back