Clojurians Log v2
datalevin 2025-04-15
Hi. I've been learning & using Datalevin for about a month and the development experience has been good. While developing I was using the database as an embedded library and didn't face any problems. However, when I tried using the database in server mode (as prep for deployment in prod), I'm facing random issues that are not easy to reproduce. I never had any of these issues in embedded mode. So, I just wanted to understand that those of you who are running Datalevin in production, are you running it in server mode or as an embedded library? Is the server mode much less tested than embedded mode or am I doing something very wrong?
could you describe the nature of your errors?
For example:
java.lang.IllegalArgumentException: No implementation of method: :add-doc of protocol: #'datalevin.search/ISearchEngine found for class: nil
Happens after using update-schema on existing connection & then transacting data.
Or
:cause "MDB_CORRUPTED: Located page was wrong type"
DB got corrupted while importing data from a CSV.
There's also this:
## A fatal error has been detected by the Java Runtime Environment:
## SIGBUS (0x7) at pc=0x00007dc71404f557, pid=1965092, tid=1965146
## JRE version: OpenJDK Runtime Environment (24.0.1) (build 24.0.1)
# Java VM: OpenJDK 64-Bit Server VM (24.0.1, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# C [libdtlv.so+0x15557] mdb_page_search_root+0xf7
## Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" (or dumping to /home/noel/projects/datalevin/core.1965092)
## An error report file with more information is saved as:
# /home/noel/projects/datalevin/hs_err_pid1965092.log
[762.195s][warning][os] Loading hsdis library failed
## If you would like to submit a bug report, please visit:
#
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.# 
These all look like server state bugs of different kind. You may want to file issues for them separately.
The server mode by definition needs more specific testing about these state changes. So it would be helpful if you file issues on these.
I will do that. Need to investigate a bit more before I do that... But I want to know how do you deploy Datalevin at http://Juji.io? Is it in embedded mode or in server mode? And to your best knowledge, others who are using datalevin in production, are they using it as a library? What would be your recommendation?
At juji, we deploy DL only in server mode. However, we are still on 0.8.x, and have not upgraded to the latest version yet, due to the overhead of migration of large number of DBs (hence auto migration is on the roadmap). I have not surveyed the users of DL, but I assume most users are using it as a library.
I see. Thanks for the info. I will file separate issues for any problems I find. Thanks again for creating Datalevin. It is really useful.
Thank you for reporting issues.