datalevin

2024-10-07T20:36:18.778579Z

I’ve just found this library, says it’s 1.5x faster than nippy and zero dependencies. If that’s true it could potentially be useful for datalevin. Not sure how much datalevin uses nippy. https://github.com/igrishaev/deed#benchmarks Also supports gziped streams which could be interesting. https://github.com/igrishaev/deed?tab=readme-ov-file#gzipped-streams

Huahai 2024-10-07T20:41:09.926999Z

Nippy is used for data that have no type specified, and also for data values that are bigger than 498 bytes. So the use is not really extensive.

Huahai 2024-10-07T20:43:13.257949Z

We can certainly experiment with it to see how much it helps.

2024-10-07T21:57:29.584819Z

Ahh, so pretty minor. Probably not worth the effort then in terms of overall impact.