Fork me on GitHub
#datomic
<
2019-04-23
>
vemv03:04:27

Can I create a Datomic database with an past, fixed basis rather than just (now)? Use case: I create Datomic databases once per unit test. A given unit test must be able to create an entity with a :db/txInstant of (-> 10 minutes ago). But if the database was created just now, I won't be able to add such an entity: Time conflict: Tue Apr 23 05:31:11 CEST 2019 is older than database basis

favila08:04:12

An untransacted db starts at the epoc (1970) @vemv

favila08:04:01

If you are getting this error then you transacted something after that without overriding the txInstant into the past. Maybe schema?

vemv08:04:43

right, thanks for the pointer! I think it's some intricacy of this project

enocom18:04:48

I’m writing a query to support pagination for a web client. The q API supports :offset and :limit which seems perfect, but I’m having trouble ensuring a sort order for the results. How do people handle pagination for web apps when using Datomic?

favila19:04:09

Query all, then sort+limit the results before delivering to the client

favila19:04:04

I think limit+offset is only somewhat reliable when reusing the same db (anchored to specific t) in fairly rapid succession

favila19:04:22

i.e., in a loop in an application, not in a url

favila19:04:30

if your result is a set, you should expect a repeatable order if the results are the same

favila19:04:45

if the results are not a set (e.g. using :with) I'm not sure you can

favila19:04:08

(the repeatable order is due to hashing order, so additional caveat that the result items are hashable)

enocom19:04:36

That’s about where I landed as well. Here I’ve been worried about memory usage, but that’s probably unnecessary. Thanks!

favila19:04:55

well you should worry about it

favila19:04:37

one thing I've found is pulls run before limiting

favila19:04:04

even in on-prem, going :find (pull ?x [*]( . to get only one result will pull everything, then take the first tiem

favila19:04:23

rather than take the first item then execute the :find

favila19:04:55

so try to return as little as you can manage from your "big" query and followup with additional queries or pulls later, using that result as input

favila19:04:58

along the same lines, if you have the kind of query which has a large known set based on a raw datomic index, you can use d/datoms to select a subset, then feed that as input to your query

enocom19:04:06

Yeah, good idea. We’re doing a pull of eids and txInstant, sorting, dropping and taking depending on the pagination, and then querying for the full data when we know just the page’s amount.

enocom19:04:34

I’ll take a look at the raw indices. That’s an interesting idea.

favila19:04:54

this only works for query shapes where each item in the first "clause" of the query is merely filtered out by subsequent clauses

favila19:04:14

otherwise your limit and offset will get messed up again

favila19:04:40

or the same find result may show up in multiple offsets

favila19:04:03

(the query doesn't see the entire result so it can't do the deduping normally done by the result set)

enocom19:04:53

That makes sense — the raw indices don’t seem to fit just right in our use case, but it’s a good idea.

favila19:04:12

yeah you have to use it carefully

favila19:04:35

might even not be as big a win on datomic cloud, since you have more going over the wire