datascript

imre 2022-07-26T08:28:47.208869Z

I’m trying to optimize a clojure (jvm) service that uses some datascript (1.3.8) entities on its hot path. They are returned by a query and then later we use them as we would use nested maps:

(d/q '[:find ?entity ; and other stuff
       :where
       ; other clauses
       [(datascript.core/entity $ ?foo) ?entity]]
     db)

(-> entity :foo/bar :foo.bar/baz)
(-> entity :qux/_some-reverse-lookup :foo/bar :foo.bar/baz)
(contains? entity :foo/bar)
;; etc
We found through flame graphs that a noticeable amount of time is spent in these keyword lookups (more specifically in lookup-entity) despite the built-in caching. I feel like we are missing a better pattern here and was wondering what we could do at query time to speed up later lookups within the entity?

imre 2022-07-26T09:05:32.681089Z

I guess pull syntax is going to be my friend 🙂

Niki 2022-07-26T10:06:28.463069Z

Yes pull might perform better. Also see if you can do without query (via direct index lookup), as queries are doing some extra work you might not need (depends on their complexity, of course)

imre 2022-07-26T10:07:54.324319Z

Thank you. In our case the initial query cost doesn’t matter that much as the result can be and is memoized. It’s the repeated use of data returned by the query that we see costly