@ljosa: you’re sure the comparison is otherwise apples to apples? I.e. impact from other project dependencies/processing steps in your peer app is minimal?

@casperc: you can bind to the ?tx entity and specify a datalog clause that matches attribute on ?tx entity. I.e., instead of typical [?p :person/name ?name] form, something like:

[?p :person/name ?name ?tx]
[?tx :db/tt ?tt]

@andrewboltachev: that’s correct, the most recent one will be highest in list on are downloads sites. For the free version, which is on clojars, it will be the one listed here: https://clojars.org/com.datomic/datomic-free

@bkamphaus: Pretty sure that wouldn’t work since :db/tt is an instant, and what I want is a version of the database showing datoms with :db/tt vals before the specified date

I ended up making an as-of function using filter.

My problem is that I think the filter is applied before the queries that you perform on that database, and the more efficient way would be last, but I need to verify that.

@casperc I guess I'm not following some of your logic. A Datom is part of a universal schema, e/a/v/tx/assert -- you can't add an additional attribute (eg db/tt) to datoms, only to entities (by adding a Datom with db/tt in the attr position in underlying data model) - so you must get associated tx entity and look at its attributes in the custom filter?

You can use a comparison predicate (function expression) http://docs.datomic.com/query.html#function-expressions as part of any query in place of the second tuple in the example, which may be more perform any of the preceding clauses are more selective than applying your filter to the database value as its passed in to each query, for example.

hi guys do we need another separate id beside db/id like other record we create in sql ? For example :project/id

do you need to communicate the Id of your entity to anyone outside? If yes and you don't already have a natural key already - then you'll need another identifier.

yes, like normal application, I need to use the id to query for example project/by-id, but I cannot query against the db/id. I run the query like so and it doesn't work :

(prn (d/q '[:find [(pull ?e [*]) ...]
       :in $
       :where
       [?e :db/id 1]
       ]
     (d/db conn)))

@nxqd: so two things: a) if you're exposing an id externally, use an attribute with a uuid type and use squuid to generate the values. b) entity ids are the ?e part of a datom, so if you need internally to use entity ids for some reason (e.g. you already have an entity from some other external based lookup and now you want to query against it)

Does anyone know, if I filter a database and then query against it, will the filter then be applied before or after the query?

@casperc: before

Ouch

Isn’t that the same as doing a full db scan of all datums?

ah: no

the filter applies as the query walks the indexes

ah ok, so filtering a database by some domain date, would not end up being horribly inefficient then?

This is the same question explained a bit more btw: https://groups.google.com/forum/#!topic/datomic/nmFsEFk6LDE

@tcrayford: thanks

@casperc: it is a very "it depends" thing. Ultimately for performance, the best you can do is to benchmark it 😉

so the real thing with filters to note is that they're applied as effectively a lazy clojure.core/filter call over the indexes (which are ordered sets of datoms). The query runtime uses those indexes, and then whilst it's walking them, the filter is applied. It's definitely less efficient than putting that restriction in your index, at least potentially

@casperc: here's a question: is :db/tt monotonically increasing as transactions go up?

@tcrayford: re benchmarking, I agree. My problem is that I don’t have realistic data in the amounts that we are going to see. But I am working on that

@tcrayford: And no, they are not monotonically increasing, since the data will come from different sources with different update speeds.

@casperc: the reply from Linus on group fleshes out what I mentioned. I would benchmark that or something else derived from the time-rules in day of datomic ( https://github.com/Datomic/day-of-datomic/blob/master/tutorial/time-rules.clj ) against your filter on realistically sized data.

@bkamphaus: Thanks, that is a good place to look, I’ll have a look at it

Is it possible to have pull over ordinary data like q does it?

@akiel: nope

@bkamphaus: ok I was hoping to leverage pull using om.next in a non-datomic context. But I wrote it myself. It's not that hard.

@akiel: it's a shame to me that datomic's query and a bunch of it's features don't work at all over ordinary data in certain kinds. It means e.g. you have to stand up an actual database in test instead of giving it a seq of vectors (that are datoms) etc etc