Hi! You're welcome 🙂 ordering is covered in the Datalog engine pretty well I think via an order-by key, e.g. https://github.com/juxt/crux/blob/e5e21352b7a466d20a7b57518e129770191de352/test/crux/sparql_test.clj#L467 Pagination needs to be handled in Clojure, but this is something we would like to see as a reusable library or "decorator"

Thank you @U899JBRPF for pointing me to this test example! It's strangely hard to find out these things anywhere.. I also couldn't really find anything for Datomic or Datascript... Also, thanks for documenting this: https://juxt.pro/crux/docs/decorators.html

🙂 again, you're welcome! More docs to come soon!

Looking forward to trying my hand at building a decorator, so thanks as well for any and all docs

@U899JBRPF doesn’t pagination become a problem if I want to paginate across 1 million items at 100 items per page once I start to hit higher numbers and it has to be handled in Clojure? Does that mean I pull in 900.1k items and then drop the first 900k for the range 900,000 to 900,100?

@U0545PBND I don't have a strong sense of how this operates, but I know we have a means of spilling the sort-by onto disk, see https://github.com/juxt/crux/commit/9e22d477f4f8ea74f4872682875d152c56b3d15c I will respond again soon with more detailed thoughts

@U0545PBND the answer is that the ability to do partitioning efficiently natively in the core depends on whether you can sort on a range-queryable attribute, therefore you wouldn't use :limit / :offset, but predicates like <= and < within in the query itself

can you do that by system/valid time?

So basically you would need a :document/order attribute with an ever increasing number in order to do proper pagination?

Seems less than ideal. What if you forget that in the document or realize you need it later?

The query runs on a point-in-time view of valid time, so the temporal coordinates aren't available inside the query unless you explicitly include valid time in the document with your own attribute...which is essentially the same thing as a ":document/order attribute with an ever increasing number". If you ever need to add an attribute retrospectively you can insert new document versions into the past thanks to bitemp. Feel free to migrate this discussion to a GitHub issue and we can try to figure out if there is a better way to satisfy your requirements, along with Håkan and the rest of the team.

OK. Will do. I just want to add I really like what I see so far, so take these questions in the kindest possible way 🙂