I'd like to use Datascript for an offline-only web-app, where my plan is to generate a Datascript DB from Clojure first and then serializing it so that it's available in the CLJS part of the app

however I realized that the amount of datoms I would have is quite large, probably to the tune of 100k

so I was wondering how many datoms I should expect Datascript to handle without running into performance issues

the data is basically a list of records that I want to then display based on some filters which are constructed via the UI

it depends on: - user's device - complexity of the queries (sometimes it is much more efficient to use indexes directly, but it might not be enough)

it's hard to say - you're better off generating the data manually (e.g. using test.check) and looking at it

+ what @misha said

also your domain might allow you to split data set into several databases, which might give you some performance boost too

Thanks, the user device is desktop Chrome

I might just end up using a vector of maps, but I will benchmark Datascript first

@luxbock you can read e.g. here https://github.com/tonsky/datascript/issues/130#issuecomment-159602488

> DataScript is in different category, so expect different tradeoffs: query speed depends on the size of result set, you need to sort clasuses to have smaller joins, accessing entity properties is not free given its id, etc. As a benefit, you gain ability to query dataset for different projections, forward and reverse reference lookups, joins between different sets, etc. And direct index lookup (datascript.core/datoms) is still fast and comparable to lookup in a map (at least comparable, think binary lookup vs hashtable lookup, logarithm vs constant). Queries do much more than that.

Database itself can handle 100k datoms no problem

you’ll hit perf issues if you select many datoms at once

including queries where intermediate sets contains lots of datoms

direct index lookups are still fast no matter what (they are lazy walks over existing data structures, they don’t create any new arrays/data structures)

one thing to watch out for is initial load of DB into memory

currently, DataScript can’t work with parts of the index stored on a disk/local storage/.... It must all be in memory

so every time your app starts you’ll have to read whole DB from your persistent storage

measure that performance first and then decide if it’s suitable for you

better use datascript-transit (it’s a lib) for serialization, avoid EDN or manual transact! of datoms. Perf difference might be 10×

@tonsky thanks those are some very good points