xtdb 2023-10-03 | Slack Archive

Chris Lester01:10:56

I'm trying to get the equivalent of this query working to return all orphans that do not themselves have children (the ids passed in from another query):

{:find [(list 'pull 'e pull-keys)]
               :in '[[id ...]]
               :where '[[e :xt/id] 
                        (not (and 
                              [e :entry/parent nil]
                              [e :xt/id id]))]}

That doesn't work (seems and can only be used in an or clause?) and the version of this without the and and promoting the parent filter out of the not clause results in including the one that has no parent but is in the id list. What's the best way to structure this to achieve the "give me all orphans that do not match the parent ids"? The simpler query:

{:find [(list 'pull 'e pull-keys)]
 :in '[[id ...]]
 :where '[[e :xt/id] 
          [e :entry/parent nil]
          (not [e :xt/id id])]}

Also does not work .. it includes an entity that should be explicitly excluded by id .. which is why I'm trying different permutations. Also, using a composite predicate [(and (not= ) (= ))], as in:

{:find [(list 'pull 'e pull-keys)]
               :in '[[id ...]]
               :where '[[e :xt/id ?eid]
                        [e :entry/parent ?parent]
                        [(and (= ?parent nil)
                              (not= ?eid id))]]}

Results in an illegal argument exception ... which turned out to be a namespace issue .. once fixed (to clojure.core/and) it also returns the same excluded entity as the query above. ... in case it helps. Changing to a not-join fixed the problem. Unsure why, as the not and not-join should have been the same.

{:find [(list 'pull 'e pull-keys)]
               :in '[[id ...]]
               :where '[[e :xt/id]
                        (not-join [id]
                                   [e :entry/parent nil] 
                                   [e :xt/id id])]}

refset09:10:55

Hey @US893PCLF not should happily take multiple legs (implicit and)

refset09:10:31

[(clojure.core/and (= ?parent nil)
>                    (not= ?eid id))]]

FYI you can't use nested Clojure expressions like this in Datalog, or at least if it 'works' it's probably not doing the thing you would expect it to do

refset09:10:16

> in case it helps. Changing to a not-join fixed the problem as in you have a working solution you're happy with? 🙂

refset10:10:21

> Unsure why, as the not and not-join should have been the same. not-join is always easier to reason about because the variables in scope are explicit. In your example the not-join would only be the same as not if the head bindings were [id e]

refset10:10:00

actually I'm not sure how your last query can work correctly if the e in the out level is not connected to the clauses within the not-join :thinking_face:

refset11:10:18

I am probably misunderstanding the intent of the query, but I would be surprised if it couldn't be simplified into something more like

{:find [(list 'pull 'e pull-keys)]
 :in '[[id ...]]
 :where '[[id :xt/id]
          (not-join [id]
           [id :entry/parent nil])]}

refset11:10:38

presumably you are storing explicit :entry/parent nil entries in your documents?

Chris Lester23:10:58

Yes, I switched to explicitly doing that since missing? wasn't available in XTDB. It seemed simpler than querying for absence of the attribute. True, I missed that .. I wound up with a more complex query and reverted to using not .. will try your simpler suggestion. The intent is I have is to get a list of (non-duplicated) entities .... and if they have children, project those children into the :entry/children attribute of the entity. To do this I have a query that selects out the set of [parent child] ids, and then I group them by parent, and project the children into the parent entity. That gives me a list of parents with children, then I need to join that with a set of orphans .. orphans that are not also in the parent results. I'm sure it's much more complex than needs to be.. but am fairly new to more complex datalog operations.

Chris Lester00:10:06

For the orphans query, this appears to work (modified from your example) .. pull all entities except the ids I've passed in.

{:find [(list 'pull 'e pull-keys)]
  :in '[[id ...]]
  :where '[[e :xt/id]
           (not-join [e id]
           [e :xt/id id])]}

refset08:10:23

> pull all entities except the ids I've passed in ah, I just understood what you really meant by this - glad you figured out a neat solution 🙂

2023-10-03

Channels