Fork me on GitHub
#datalevin
<
2022-11-04
>
respatialized20:11:54

Is there documentation about the LMDB config used by Datalevin? I'm wondering what the maximum DB/memory map size is and whether I need to adjust it (and how I'd do so).

respatialized20:11:49

Ah, I missed the docs for open-kv : > :mapsize is the initial size of the database. This will be expanded as needed > I guess for me the most likely answer is "worry about maximum size when it becomes a problem, which almost certainly isn't anytime soon."

Huahai20:11:30

default is 100mb, and will be automatically increased 10x every time it is over limit

Huahai20:11:05

100mb -> 1gb -> 10 gb -> …

1
respatialized22:11:23

Have you observed any practical limits to DB size (e.g. in the 50-100GB range)?

Huahai23:11:29

i have not

Huahai23:11:24

the file size is normal much larger than actual size due to MMAP file, so copy with compact may reduce it a bit

Huahai23:11:35

theortical limit is 12TB

Huahai23:11:14

the largest file i tested is for benchmarking search, that produces an index of 80 GB

Huahai23:11:40

search in that file has medium of 1 ms, so i don’t think it is a problem

Huahai23:11:00

the datalog query engine is slow right now (still the fastest among clojure datalog stores), but we are doing a datalog engine rewrite

👍 1