Clojurians Log v2

Very cool! 🎸

there is also example TUI to visualize and navigate EDN compile it with

make clean tui EXTENDED_INTEGERS=1 RATIO=1 MAP_NAMESPACE_SYNTAX=1 EXTENDED_CHARACTERS=1 METADATA=1 TEXT_BLOCKS=1 EXTENDED_INTEGERS=1

./examples/edn_tui ./docs/showcase.edn

Completely a tangent, how does a SIMD extension look in code?

https://github.com/DotFox/edn.c/blob/main/src/simd.c here and there are more in newline_finder and text_block parser

Interesting, so very much looks like you're pretty much aware of the instruction set.

frankly, I didn't know about their existence until I ran very first benchmark 🙂 then I start looking for performance improvements in optimised json parsers and did extensive research using LLMs. But I can't say I know them all and probably there are places that can have better/faster implementation. Yesterday I was shared with an absolutely wonderfully in-depth article about the details of SIMD instructions. reading it now https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/porting-x86-vector-bitmask-optimizations-to-arm-neon

@jeaye Jank could have the fastest EDN parser with this 👀

@emma.audrey.g Could be! We can do some benchmarking next year.

FWIW when WAL mode is enabled, concurrent readers aren't blocked by writers: > Writers merely append new content to the end of the WAL file. Because writers do nothing that would interfere with the actions of readers, writers and readers can run at the same time. However, since there is only one WAL file, there can only be one writer at a time. https://www.sqlite.org/wal.html#concurrency

Great to see Max, that you are pursuing your passion for developing database tech!

could a WASM compiled sqlite be used in the browser?

very cool to see sqlite used like thsi

Yes, it could be used in a browser. There is a JS lib for the Tuple encoding: https://github.com/josephg/fdb-tuple

ATM the CLJS part is broken, but it would also be very interesting to run it in Cloudflare Workers (which can do sync Sqlite reads)

@tatut it would also work without Sqlite at all in the browser. Datascript's persistent-set could be used again and the data could be stored in IndexDB. Then you have something like Datascript but with database as a value semantics.

idk if you want to encourage people to use this (yet), but the README could use a quick start example of how to start it up

atm I primarily want to check if anyone likes to invest some time to polish it to something that could be used for real-world applications. I added a quickstart note to the readme how to start hacking on the project: https://github.com/maxweber/dbval?tab=readme-ov-file#quickstart

Very cool project! Would love to try it out sometime.

This is very cool! I tried to port datascript to sqlite a few months ago but I ran into issues with my use case. Were you able to get db-with in memory "forks" of the database working?

cljs in the browser is a pain because you have to make the whole codebase async.

Regrettably, one of the trade-offs of using a mutable set is that forking is not free (no structural sharing). I had to adapt a lot of the Datascript tests where the db value was "forked". db-with will not be possible. At least copying the Sqlite file to do what-if modifications is more straight-forward in comparison to other relational databases.

ah that is what I ran into as well. I thought maybe you could maintain a separate table for every fork and merge the tables at query time, but at least for my use case having an in memory fork is ideal so I dropped the project.

wouldn't a speculative transaction keep a separate in-memory representation of the not-really-committed datoms, and merge those at query time?

what about creating views where t<=something and using INSTEAD_OF triggers for modifying them

Probably, both possible. DataScript and therefore dbval only needs a sorted-set abstraction. Another sorted-set implementation could read from Sqlite and keep the what-if / db-with datoms in-memory.

Datascript does support other extensions for integrating other backend stores. Is this an async version of that API for working with JavaScript & Sqlite or something completely different?

It's different, since my goal is to serve the database as a value (with time-travel) like Datomic does.

would a transaction work for speculation purposes?

(next.jdbc/with-transaction [tx "jdbc:sqlite:/tmp/x.sqlite" {:rollback-only true}]
    ,,,)

or this option is not actually supported for sqlite?

@onetom using a Sqlite transaction would work, but Sqlite is a single writer system, so all other transactions would be blocked, which might be okay for some use cases.

Special shout-out to @lee for helping a lot with fs!

Special shout-out to @doppiaelle1999 for helping me with issue 1882!