Clojurians Log v2
Clojure programming
Channels
Apps
datalevin
amar 2025-04-11T14:22:00.837379Z
Hi. I have a read heavy API that uses datalevin to cache data and encountered the MDB_READERS_FULL error. We're not using pmap but there can be hundreds of simultaneous requests. Based on this https://web.archive.org/web/20200112155415/https://twitter.com/armon/status/534867803426533376 it seems setting :max-readers to some high value like 4000 might be ok. Is there a heuristic that's worked out well? Given that the KV DBIs are independent of one another in my use case and this is a cache, is starting datalevin with the :nolock flag an option if I want to really press on read performance? I assume writes would still require a lock, but this would help avoid the MDB_READERS_FULL error. Also is the write lock database wide or per DBI?
amar 2025-04-13T02:13:08.334499Z
Appreciate the suggestions and thanks for making and sharing datalevin!
Huahai 2025-04-13T06:27:24.927979Z
I am glad that it is useful for you 😊
Huahai 2025-04-12T06:31:12.195299Z
It is ok to set a higher max-reader. Since this is a per db file parameter, if it is possible, I would open multiple db files instead of multiple DBIs for multiple independent caches.
Huahai 2025-04-12T06:32:30.428949Z
:nolock parameter is about the write lock. I wouldn’t touch it
Huahai 2025-04-12T06:33:00.560519Z
Write lock is per DB file
Huahai 2025-04-12T06:36:19.313789Z
Setting :temp? to true or adding :nosync flag is what increasing writing speed. Btw