Clojurians Log v2
datomic
Hello, On datomic peer, i am trying to query most recent activities of a user limiting by 100 items then sorting in clojure. I am executing two queries like the below, i'd assume those two queries are fast enough to never cause a perf issue. But can i do a single query instead of two? Also I am making an assumption that if the transaction is newer, its id is always higher than previous transactions. I just feel like something is off and it can be much better.
(d/pull-many (db)
'[:db/id :db/txInstant {:data/user [:user/username] :data/action [:db/ident :en/name]}]
(ffirst
(d/q '[:find (max ?n ?tx)
:in $ ?n ?u
:where [?tx :data/user ?u]]
(db) 10 user-id)))
Howdy! My read here is you are pulling all activity entities for the user before sorting and limiting. If a user has millions of activities you can end up with a huge result set to filter down. A more scalable approach might be to ensure your user entity has an attribute that has your own separately modeled time with :db/index true set and then use d/index-range or d/datoms to walk the index.
> But can i do a single query instead of two
I understand the desire to make things smaller or "one thing", but two (even completely separate queries) is great for composing the business logic of your system. Don't feel pressured to make it all one query.
In the event you only ever want to look at transactions like "What happened between X and Y time" you can use the log as you are only filtering by time. https://docs.datomic.com/reference/log.html
Thanks Jaret, my thought process was that max would efficiently take top n eids. if n kept to a minimum, sorting and pulling would not be a problem.
I am lacking knowledge around raw index access. I'll have a read and do some repl exploration then I'll comeback to ask some questions or just pop in to say thank you.