datomic 2024-05-13 | Slack Archive

Tom Fenton21:05:09

Hi all. I’m looking for help with a query I’m struggling to build, and/or guidance on making my data model more Datomic-idiomatic so I avoid the problem in the first place. Details in comments.

Tom Fenton21:05:23

I have a toy manufacturing/supply chain system, linking items with their sources of supply such that:

:source/item   [:item/id “123456”]
:source/vendor [:organisation/id “ABCXYZ”]
:source/price  123.45M

This allows parts with multiple vendors and vendors with multiple parts, with the “source record” entity storing e.g. pricing data, which isn’t inherent to the item or the vendor, but is specific to a given vendor-item combination. I’m trying to query items and their source data, such that I get multiple results for items with multiple sources, but for items with no sources, I get a record with e.g. {:source “No Source”}. The best I’ve managed so far is using not-join to return only items without a source; get-else seems like the obvious answer, but it seems like it’s only built for missing properties on an already-bound entity. For example:

:where [?item-e :item/id ?id]
       [?item-e :item/name ?name]
       [(get-else $ ?item-e :source/item “NIL”) ?source]

returns “NIL” for everything because no item entity has a :source/item property. I can’t find any rearrangement of the get-else clause that gives the desired result. The obvious solution from a techical perspective is to flip my data model so I get :item/source with cardinality many, which should make get-else straightforward, but this means I’m absorbing some vendor-domain data into my item entity, which also doesn’t seem the right way to proceed. Any constructive input is appreciated!

favila22:05:31

You can do this with or-join:

:where
[?item-e :item/id ?id]
[?item-e :item/name ?name]
(or-join [?item-e ?source]
         (and [(ground "NIL") ?source]
              (not [_ :source/item ?item-e])
              )
         [?item-e :source/item ?source])

(Edit: Realized this is a reverse-ref, so you can't use missing? or get-else )

favila22:05:30

However consider whether query is what you really want. An "outer join" is usually better served by a pull expression

favila22:05:15

In this snippet for example, is there actually anything relevant about ?id or ?name other than retrieving them? Perhaps you just want a projection of items which have an id and name?

[:find (pull ?item-e [:item/id :item/name {(:source/_item :limit -1) [:source/name]}])
       :where
       [?item-e :item/id]
       [?item-e :item/name]]

favila22:05:53

Concerning ref direction, my usual rule-of-thumb is to prefer the direction with the lowest cardinality

favila22:05:24

https://favila.github.io/2023-04-18/datomic-choosing-ref-direction/

Tom Fenton08:05:51

Thanks @U09R86PA4 - really appreciate both fixing my query and the general advice! I have it working now as intended but will experiment to see if pull is a better fit. I’d like to make sure I properly understand the solution you’ve suggested. I haven’t used or-join before; am I right in thinking that you’re providing two or more possible rules to match, and returning the first truthy value, as with Clojure itself? And that the “and” block returns a set of rules if all match, or nil if any fail to match? I noticed that your query still works if I omit one part of it:

(or-join [?item-e ?source]
         [(ground “NIL”) ?source]
         [?item-e :source/item ?source])

Edit: I see that switching the order of the or-join rules doesn’t change the output, so I guess my assumption here is wrong!

favila09:05:24

@UAH1JFQV9 all branches of the or-join are evaluated and the results are unioned. The “and” is syntax to collect clauses into a single branch. Your change has the effect of giving every item a “NIL” source even if it has one.

Tom Fenton09:05:13

So it does - the perils of testing queries with a very small dataset, I didn’t notice that I had two results against one line. I’ve reinstated the “not” and will continue experimenting with it to make sure I fully understand the behaviour. Thanks again!

2024-05-13

Channels