Fork me on GitHub
#datahike
<
2022-10-06
>
whilo07:10:35

@pat Thanks for providing an update to cljs support in konserve https://github.com/replikativ/konserve/pull/76! 🙂

❤️ 1
🎉 1
metasoarous20:10:12

Amazing! Thanks @pat!

parrot 1
pat00:10:01

Looking at the indexeddb impl and maybe konserve model better fits to blobs at keys rather than objects? I have no evidence it would perform better, but filesystem impls already have similar io expectations. Was there a bottleneck you guys found that made you lose interest?

whilo17:10:30

I cannot remember any. I worked on Indexeddb 5 years ago to be frank. It would be super cool if you could lift it to the current konserve framework. Feel free to elaborate a bit more about the design decisions between blobs and objects, I am not super familiar with it.

whilo17:10:51

One annoying problem we have found (thanks to @U05095F2K) is that many browsers silently drop indexeddb, so we would have to treat it like a useful cache instead of actual durability in many cases.

😩 1
pat18:10:23

I think you can put() https://developer.mozilla.org/en-US/docs/Web/API/Blob as values in the objectstore. You lose indexing + human readability but impl basically identical to fs code with same storage layout

pat18:10:50

I will take a crack at it

whilo19:10:27

Do you have experience with CBOR btw.?

whilo19:10:58

We started to play around with it a bit. It would be very nice because then we could also read our indices in other languages.

pat22:10:18

No & I haven't read up the full datahike stack yet, but it has occurred to me that a schema could improve read times. Is there a runtime in particular?

whilo17:10:45

I looked into Julia for instance, https://github.com/JuliaIO/CBOR.jl. Python might also not be bad just for reach.

whilo03:11:58

@pat This is not about optimizing deserialization times (which is another interesting topic), but more about generalizing Datahike beyond Clojure. There is the whole linked web and RDF out there, but using a Clojure specific format renders things very isolated from that.