Fork me on GitHub
#datascript
<
2020-05-23
>
Oliver George02:05:20

Anyone else crying out for a persisting Datascript? I'm guessing IndexedDB is the sensible (if intimidating) API to build on and living with async queries would mean keeping a full history to provide "point in time" queries.

Oliver George02:05:55

I'd also love it to persist in React Native which doesn't seem to provide IndexedDB.

adamtait22:05:49

@olivergeorge what direction are you leaning? In my exploration - AsyncStorage seems well supported but has a size limit. FS & SQLite have less support but fewer limits.

Oliver George23:05:43

For now I’m using SQLlite.

Oliver George23:05:05

AsyncStorage limits are solvable now by the way.

👍 4
Oliver George23:05:37

It doesn’t suit me because pulling everything into memory and deserializing is slow. Also the variety of access patterns I need makes it tricky to break things up into convenient documents.

Oliver George23:05:55

SQL queries are friction but solvable

Oliver George23:05:12

It’s just a shame that I can’t use the same techniques on the web app version of the same application

👏 4
Oliver George02:05:58

(holding large datasets in memory on mobile devices seems like a design flaw but I've not tested the impact on performance)

David Pham07:05:19

Is it not what datahike is about?

lilactown15:05:42

sqlite seems popular on mobile

lilactown15:05:45

@olivergeorge I don’t see much to be done tbh. I think that you could easily serialize the datascript db and put it in indexeddb or localstorage (depending on size / etc.)

Oliver George23:05:57

Yeah it’s problematically slow to load given the data I have on mobile. Also holding everything in memory on mobile is a questionable.

lilactown02:05:09

Which is slow? Having everything in memory?

Oliver George04:05:25

Slow to parse and index a large amount of data into memory ready for use.

lilactown15:05:15

but I think the flow would be to then pull out the whole db and deserialize to do queries

metasoarous18:05:41

@olivergeorge This is exactly what Datahike is about. Only thing missing right now is a durable mode on the front end. Means rewriting the query engine to be async (since no blocking in JS), which is some work, but I believe the team is queueing that up soon.

👍 8
metasoarous18:05:40

@lilactown You can do what you are suggesting if you don't have too much data, but that way you still have to hold things in memory. What's being worked on would not be limited in this way.

lilactown18:05:48

it’s a tradeoff, but I’m struggling to think of a time when have had so much data that I would rather pay the performance cost of going to disk

lilactown18:05:46

I’m sure for some niche cases that’s true, but the majority of webapps have nowhere near those kinds of requirements, which is why I think the datomic-esque “load the db in memory and query it” makes more sense

lilactown18:05:54

you could probably do something similar to datomic where you cleverly store the EAVT values in indexeddb to allow an efficient as-of without duplicating data

lilactown18:05:25

my hunch is just that you don’t want to hit indexeddb for every query. I’m interested to know if I’m wrong!

teodorlu20:05:12

Perhaps stupid question: can I browse a datascript database with REBL?