Fork me on GitHub
#datascript
<
2018-03-09
>
carocad09:03:08

hey guys, does anybody know why doesnt datascript have :vaet index? I am facing quite some performance problems because of lack of back-reference index 😞

petterik15:03:09

Refs, unique values and :db/index true end up in the :avet. isn't that good enough?

carocad10:03:24

hey @U0CM1QURZ generally that is good enough but I am trying to get the most out of datascript. At the end I went with a solution like you describe, using :db/index true and index-range search with some transducers. The thing is that that was not obvious since the :vaet index doesnt exists, querying for back-references can take 20 times more time than the previous approach

petterik10:03:09

Would it be easy for you to share (e.g repo or gist) what you're doing? That is, what your slow query looks like and how you made it faster using :db/index true and index-range?

carocad11:03:19

I dont have anymore the query anymore since it was a quick comparison but I do have the current code and schema. Here is the code to get the back-references: https://github.com/hiposfer/kamal/blob/datascript/src/hiposfer/kamal/libs/tool.clj#L52. And here is the schema https://github.com/hiposfer/kamal/blob/datascript/src/hiposfer/kamal/services/routing/core.clj#L22

carocad11:03:29

as you can see I am indexing the :node/successors such that I can do index-range on them and get only the values that I am interested in. My guess would be that having a vaet index would avoid that since quick hack since finding those backreferences would be trivial

metasoarous10:03:56

Thanks for chiming in @bbss and @atticmaverick. IIRC, @petrus was having some issues with what is now the master branch that prompted us to suggest he try checking out some of our "work in progress", which admittedly has been quite in flux, to put it gently. However, I'm happy to announce that after chatting with @mail524 to get a sense of the progress and road ahead (he's been leading development on this project for the last several months), it's looking like we'll be able to release (and merge to master) a stable iteration of this work soon, and are specifically aiming to do so by the beginning of April. This release should greatly simplify a number of issues around client/server api symmetry, generally make things much easier to set up, and put us on firmer ground for moving ahead towards our longer term goals. I look forward to sharing the details with you all soon. Till then, stay tuned!

bbss13:03:23

Exciting! Staying tuned. suspect