Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2022-09-27

Channels

#datomic
<
2022-09-27
>
enn14:09:53

Is there a significant overhead to rule invocation (not sure if that’s the right word)? Would you expect a significant difference in performance between: 

(d/q '[:find ?result
       :in $ % [?arg ...]
       :where (my-rule ?arg ?result)]
       db
       '[[(my-rule [?arg] ?result)
         ...]
       coll]
and 
(d/q '[:find ?result
       :in $ % ?args
       :where (my-rule ?args ?result)]
       db
       '[[(my-rule [?args] ?result)
         [(identity ?args) [?arg ...]]
         ...]
       coll]
I would expect these to be roughly equivalent but in practice it seems like, when coll has more than a few items, the second style where the collection is “destructured” within the rule body is significantly faster.

enn15:10:41

Following up on this and also on the discussion of recursive rules https://clojurians.slack.com/archives/C03RZMDSH/p1664053990766329, I had been imagining that rule overhead was pretty trivial, and not worth worrying much about for the average application. But testing with a tiny toy dataset, it seems that rule-invocation costs are actually pretty significant, and that it might be worth it to avoid rule composition altogether for performance-sensitive use cases. I made a https://gist.github.com/enaeher/fab806cf0d919b094a76101093272713 with a tiny amount of data and wrote the same (trivial) query four ways: 1. inline, with no rules 2. with the matching logic extracted into a rule 3. with the matching logic extracted into two rules, one of which invokes the other 4. with the matching logic extracted into two rules, both of which are invoked directly in the :where clause of the query The inline query is fastest, and the performance seems to increase with the number of rule invocations. #1 is twice as fast as #3 and four times as fast as #4. Is there something I’m missing or doing wrong? We’ve just deployed a system that resolves GraphQL queries into Datomic queries, and it makes heavy use of rule composition. It’s been a pleasure to build and work on, but these results have me wondering if I should replace the rules with functions.

👀 2
Keith13:10:44

Hello @U060QM7AA! The main thing I see causing a difference here is the fact that your rules do not keep the predicate expression close to the clause it’s intended to be used with. Notice how the variants where [(> ?value 90)] is adjacent to [?user :user/level ?level] seem to be the fastest.

enn15:10:41

Following up on this and also on the discussion of recursive rules https://clojurians.slack.com/archives/C03RZMDSH/p1664053990766329, I had been imagining that rule overhead was pretty trivial, and not worth worrying much about for the average application. But testing with a tiny toy dataset, it seems that rule-invocation costs are actually pretty significant, and that it might be worth it to avoid rule composition altogether for performance-sensitive use cases. I made a https://gist.github.com/enaeher/fab806cf0d919b094a76101093272713 with a tiny amount of data and wrote the same (trivial) query four ways: 1. inline, with no rules 2. with the matching logic extracted into a rule 3. with the matching logic extracted into two rules, one of which invokes the other 4. with the matching logic extracted into two rules, both of which are invoked directly in the :where clause of the query The inline query is fastest, and the performance seems to increase with the number of rule invocations. #1 is twice as fast as #3 and four times as fast as #4. Is there something I’m missing or doing wrong? We’ve just deployed a system that resolves GraphQL queries into Datomic queries, and it makes heavy use of rule composition. It’s been a pleasure to build and work on, but these results have me wondering if I should replace the rules with functions.

👀 2