datalevin 2023-02-22 | Slack Archive

I upgraded a 0.6.25 db to 0.8.5 by exporting and importing. It worked for a couple of transactions, now it throws this:

datalevin.core/transact!                core.cljc:  719
datalevin.core/transact!                core.cljc:  721
datalevin.core/-transact!                core.cljc:  629
datalevin.core/-transact!/fn                core.cljc:  629
datalevin.core/-transact!/fn/fn                core.cljc:  629
datalevin.binding.java.LMDB/close-transact-kv                 java.clj:  460
clojure.lang.ExceptionInfo: Fail to commit read/write transaction in LMDB: "Environment mapsize reached (-30792)"
java.lang.Exception: clojure.lang.ExceptionInfo: Fail to commit read/write transaction in LMDB: "Environment mapsize reached (-30792)" {}

The db I am testing this on is a couple of days behind production (still on 0.6.25) which gets fed a lot more than I played with so it is very unlikely a real limit. I tried tuple as unique/identity which works well (until now) Has anyone seen this message or has a hint where to look for this env? I use the jar from clojure, no native, no server involved. lmdb 0.9.29 on FreeBSD 13.1.

magra12:02:12

I threw everything away and rebuild from clean zfs datasets and reimportet the db. So far the error has not come back, so can not reproduce.

Huahai03:02:11

Interesting. I don’t even know this works on FreeBSD, as the native dependencies were build for Linux.

Huahai03:02:08

To officially support FreeBSD, we would need to locate a free CI for this. Do you have any ideas?

Huahai03:02:08

I think 0.8.5 still use LMDBJava compiled LMDB, from 0.9.0 onwards, we will use our own compiled LMDB in order to track LMDB’s master branch, which has important fixes that we need for 0.9.0. I would love to officially support FreeBSD. The main issue with platform support is the availability of CI runners that are free for open source. If you know such a CI that support FreeBSD, please let me know.

magra13:02:15

Sorry for the delay. Slack notifications seem not to work. Thank you for your response. I will keep my eyes open for a CI runner. I know the FreeBSD folks are working on this. I know little about java. I do know that my app will not run on FreeBSD until I install the lmdb package, so I guess it uses a system wide .so file. Afaict the FreeBSD package for LMDB should track LMDB. I do not know, whether my skills will allow me to do that.

magra13:02:25

@U0A74MRCJ As for the error. It seems, that it happens when there are fields in the transaction that are not in schema.

magra13:02:09

or bad refs or something. In any case bad data in the transaction.

magra16:03:08

@U0A74MRCJ FreeBSDs port/package fetches https://github.com/LMDB/lmdb/archive/refs/tags/LMDB_0.9.29.tar.gz is this new enough?

magra16:03:37

Cirrus mentions free Instances for open source projects here https://cirrus-ci.org/features/

magra16:03:33

@U0A74MRCJ https://wiki.freebsd.org/HostedCI I have no experience with ci systems and am not a computer scientist. But I do manage two bare metal FreeBSD machines with jails, tight, dynamic firewalls etc. from scratch. If I can be of any use I will be glad to help.

Huahai17:03:39

thx for the info, I will try to add this when I get around to it

Huahai04:03:19

cirrus ci has freebsd support, so we will use that

❤️ 2

2023-02-22

Channels