Fork me on GitHub
#datomic
<
2023-02-17
>
jjttjj16:02:24

Possibly too general of a question, but wondering if there are any best practices regarding what your query-wrapping-functions take as arguments and return. You have an app that makes queries, some of which need to take certain entities into account. Do you tend to make your query functions take entity ids or lookups as args and then return entity ids or lookups? Do you often have functions just return entities (with on prem), or do you return ids so that they can be used to lookup appropriate data elsewhere? More generally, do you find yourself passing around ids/lookups throughout your app, or do you try to hide them as an implementation detail?

caleb.macdonaldblack17:02:31

Im still experimenting with Datomic so take this with a grain of salt. What I’ve been doing is treating Datomic’s eid(:db/id) as an internal implementation detail of Datomic and associating it with a generic uuid attribute like :com.example/id. And this is mainly because of the security concerns involved around what could be learned about sequential ids. But also concerns with reproducibility when migrating data. Something I’m mildly aware of, but yet to figure out, is that completely random UUIDs as a lookup are less efficient than a lookup that can be sorted. I believe there are variations of UUIDs that implement a sortable component (time based I think). However I’m not 100% sure if I’m correct on this, so do you’re own research. I also prefer referring to entities generically, for example: :com.example/id as apposed to :com.example.customer/id. This is because I believe that the latter assumes too much about what the entity actually is, and makes abstraction limited and confusing. For example, a customer could also be an employee, so would we expect the employee payroll system to need understand that customers can also be employees and need to know about both types of ids? Do we create a new entity for employee even though they’re the same entity? Do we create aliases? Instead of dealing with all that, I just use generic ids and let my attributes describe what my entity is. So I would typically pass around lookups and not eids. Also most of my datalog queries are really just identifying the entities that match that query. They really don’t have any business fetching attributes that aren’t relevant to the query. For example, if I’m fetching employees, why would I also return their names? The purpose of the query is to find employees, not names. So I typically hydrate entities separately. All that said there are many reasons I would deviate from the above. Performance is probably one of the common reasons. But I only deviate as needed. Just because I might need to do something different 5% of the time doesn’t mean I should do everything that way.

👍 2
jjttjj18:02:37

Yeah I've been thinking along much the same lines. I like that lookups and db/ids are interchangeable, which means I don't have to decide too much on the particulars of ids; if a function takes ids it could be passed either a db/id or any of my domain lookups.

Joe R. Smith16:02:24

My query functions take a db value and any ids as “identifiers”, which can be one of: an ident, a lookup ref, or an entity id. I usually include an “opts” map where you can optionally pass in a pull selector if you’re querying for a single entity or sequence of the same entity “type”. I like to default the pull selector to [:db/id].

👍 4
Joe R. Smith16:02:04

Those 3 identifier types can more or less be used interchangeably in query and pull expressions.

onetom09:03:13

we are using Datomic Cloud, which doesn't allow the use of the {:find [(pull ?e attrs) .]}, so we often have to add a (->> (map first)) to our (-> datalog-query (d/q args...))` pipeline. we always accept entity references to our queries, which can be either 1. entity IDs 2. lookup refs 3. maybe keywords too, iirc, which we use in tests, where we add :db/idents to our example-data entities we have also settled on the argument naming terminology: 1. some-entity-eid means specifically and entity ID is required 2. some-entity-ref means any of the 3 kinds of entity references are accepted 3. some-entity means an entity map

onetom09:03:18

i have 2 intellij live-templates to generate 1. query expressions:

'{:find  [(pull $ENTITY$ attrs)]
  :in    [$ attrs $PARAM$]
  :where [[$ENTITY$ $ATTR$ $PARAM$]]}
2. query execution expressions:
(-> $1$
    (d/qseq (:val 'dc) ['*] 'param)
    not-empty
    (some->>
      (map (fn [[entity]]
             entity))))
$END$
and as u can see, we typically define the query independently of the place where it's used, so we can unit test the more complicated queries that way. we also mark our query definitions with the ^:datomic/q meta-data and experimented with 2 naming schemes, eg: 1. (def ^:datomic/q something-q ...) 2. (def ^:datomic/q q:something ...) the meta-data just helps us to look for examples of existing queries, when we are writing new ones. we also use ^:datomic/query-fn, ^:datomic/xform & ^:datomic/tx meta-data for similar purposes.