Clojurians Log v2
datalevin 2026-07-29
Does datalevin support querying transaction data, like the transaction id of a fact or transaction times?
seems not
yes, why not?
Well I get a index out of bounds error when I try to bind it in a query
@huahaiy 👆
txn id is not stored
I am really liking datalevin and considering building a commercial product with it. One concern I have is that it seems to be primarily maintained by one person. I know that there are other contributors, but it’s predominantly Huahai Yang. Also, I know it’s being used in production at Juji, but not sure where else. It would be nice to see other companies listed, if they’re known. Just wanted to throw this out there for discussion. All opinions / feedback welcome.
I agree. I think it would be nice to have a Github issue or similar where companies can confirm they are using Datalevin. E.g. something like this https://github.com/babashka/babashka/issues/254 I saw one https://github.com/datalevin/datalevin/discussions/381#discussioncomment-17722408. It would be nice to see more
Many software are maintained by a single person. That's just the norm. It's open source, so the community can always take it up.
Lots of the important software started by one person, the most important factor is that "who is that one person?" Huahai is very capable & thoughtful person
I think I have found a bug @huahaiy
(let [db (d/db-with
(d/empty-db "/tmp/datalevin.db"
{:test/unrelated {:db/valueType :db.type/string
:db/cardinality :db.cardinality/one}
:test/item {:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many}
:test.item/inner {:db/valueType :db.type/long
:db/cardinality :db.cardinality/one}})
'[[:db/add -1 :test/unrelated "bad data"]
[:db/add -2 :test/item -3]
[:db/add -3 :test.item/inner 5]
[:db/add -2 :test/item -4]
[:db/add -4 :test.item/inner 10]])]
[(d/q '[:find ?item ?bad
:where
[(q [:find ?inner ?item
:where
[?item :test.item/inner ?inner]]
$)
?list]
[(sort (comp - compare) ?list) ?sorted-list]
[(first ?sorted-list) [_ ?item]]
[?item :test/unrelated ?bad]]
db)
(d/datoms db :eav)])
;; => [#{[4 "bad data"]}
;; [#datalevin/Datom [1 :test/unrelated "bad data"]
;; #datalevin/Datom [2 :test.item/inner 5]
;; #datalevin/Datom [3 :test/item 2]
;; #datalevin/Datom [3 :test/item 4]
;; #datalevin/Datom [4 :test.item/inner 10]]]I'll also note that the strange construction here with making a sorted list then getting the first item is a result of subqueries not supporting :order-by
Why not use an aggregate fn like max or min to avoid the whole sorting thing?
@max.r.rothman the problem is that using a max aggregation will group by the id and the id is what I actually want to find based on the aggregate
Or at least that's how I understood it, I may test some with the aggregations, maybe my intuition is off
Confirmed as bugs. Actually two bugs: variables in nested q are leaking into outer query; tuple bindings are not properly validated. Fixed in master branch.
Thanks so much!
I just got a fatal error when running Datalevin 1.0.0 in some tests on GitHub. I can't reproduce the error, but I wasn't doing anything fancy - literally just a test to check I can open and close a Datalevin connection. The logged error message is:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fbd9596a5ff, pid=2280, tid=2313
#
# JRE version: OpenJDK Runtime Environment Zulu21.52+15-CA (21.0.12+8) (build 21.0.12+8-LTS)
# Java VM: OpenJDK 64-Bit Server VM Zulu21.52+15-CA (21.0.12+8-LTS, mixed mode, emulated-client, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# C [libdtlv.so+0x16a5ff] mdb_range_count_values+0x61
#
# Core dump will be written. Default location: Determined by the following: "/usr/lib/systemd/systemd-coredump %P %u %g %s %t 9223372036854775808 %h %d" (alternatively, falling back to /home/runner/work/database.datalevin/database.datalevin/core.2280)
#
# An error report file with more information is saved as:
# /home/runner/work/database.datalevin/database.datalevin/hs_err_pid2280.log
#
# If you would like to submit a bug report, please visit:
# Azul jdk is not something we have tested. Don't know if it is relevant.
It would still be helpful to post the code
Sure. https://github.com/duct-framework/database.datalevin/blob/master/test/duct/database/datalevin_test.clj - there's not much to it, it just uses create-conn to open the connection, close to close it, and closed? to check if it's been closed. Sorry I can't provide any more info - I haven't run across this bug on my local machine, and only once on the CI server.
The likely cause is your suspend/resume test.
Because it closes then recreates the connection?
in ig/resume-key, you are using doto which returns the original value, and the newly initialized connection is discarded
Oh!!
Boy do I feel dumb. Let me fix that immediately.
then you are closing the old connection, and deleted the directory, while the new connection is still open, and its background sampler then race Datalvein's JVM shutdown hook
Yeah, I see. I was trying to be clever using doto instead of do but forgot I needed the output.
😀 It happens
I will make the background sampler coordinate with the shutdown hook, so this would not crash.
If you are using Datalevin in production, I would appreciate a comment to https://github.com/datalevin/datalevin/issues/383