Fork me on GitHub
#asami
<
2023-09-30
>
nivekuil06:09:38

is there a way to distinguish between a nil because of (optional ...) and an explicit nil?

quoll15:09:54

By joining to a statement that has that nil

quoll15:09:16

There’s no way in a filter though

quoll15:09:59

One day. I need to filter on patterns soon because I need an “exists” operation for SPARQL

nivekuil19:09:33

i'm somehow even more confused.. an example:

(d/transact conn {:tx-triples [[1 :foo 1]
                               [1 :bar nil]
                               [2 :foo 2]]})
  
  (d/q '[:find ?foo ?bar
         :where 
         (optional [?e :foo ?foo])
         (optional [?e :bar ?bar])]
       conn)
that query returns two tuples with the same value, but I'd like to distinguish between the two entities

quoll19:09:09

ack... that query is breaking my head

quoll19:09:18

what are you trying to do with it???????

nivekuil19:09:33

a full outer join I think

quoll19:09:51

it definitely won't do that

quoll19:09:06

it'll try to match on the ?e values

quoll19:09:31

oh, sorry... I was thinking you meant outer product

quoll19:09:38

ummm, let me think

quoll19:09:51

So... the first one will probably return ^{:cols [?e ?foo]} [[1 1] [2 2]]

quoll19:09:21

Then joining this with optional will match on the ?e. So then for ?e=1 it will return ^{:cols [?e ?foo ?bar]} [[1 1 nil]] as a full binding. For ?e=2 it will return ^{:cols [?e ?foo ?bar]} [[2 2 nil]] where it extrapolates to fill in that binding because it's optional. So the result would be ^{:cols [?e ?foo ?bar]} [[1 1 nil] [2 2 nil]] . Is that what you get?

nivekuil19:09:56

what's the syntax you're using? the raw output is ([1 nil] [2 nil])

quoll19:09:19

^{:cols [?e ?foo ?bar]} is the metadata

quoll19:09:34

I can't recall if it survives the projection

nivekuil19:09:40

ah, yeah it's there

quoll19:09:43

but that's how I keep track of columns

nivekuil19:09:45

though i don't query for ?e but looks right yeah

quoll19:09:15

projection is where I take the find clause and pull out just the columns that were asked for

quoll19:09:35

If you want a full outer product you'll want to union between two optionals

nivekuil19:09:09

just the outer join. I did try

(d/q '[:find ?foo ?bar
               :where 
               (or (optional [?e :foo ?foo])
                   [(identity :undefined) ?foo])
               
               (or (optional [?e :bar ?bar])
                   [(identity :undefined) ?bar])]
             conn)
but i don't know what that ended up doing

quoll19:09:19

ideally, you'd have something that selected the object types you wanted, then you could have something like:

:find ?foo ?bar
:where [?e :type :a_type]
       (or (optional [?e :foo ?foo]) (optional [?e :bar ?bar]))

quoll19:09:46

I think that binds the variable (`?foo` or ?bar) first, then looks for a matching ?e, which it doesn't find, so returns nil for each value. Then it tries to join on the 2, and fails to find a match

quoll19:09:31

optional was designed to expand your already existing bindings. So if you've selected people, then it can be used to add their email, but return nil if they don't have one. You're using optional when no binding exits. That's going to return everything that matches the binding. At least for the first one.

quoll19:09:50

after that, it would go back to the usual behavior

quoll19:09:05

what are you actually trying to do?

nivekuil19:09:16

pretty sure I just want a full outer join 😛

nivekuil19:09:35

or you mean higher level than that? it's a bit compilcated

nivekuil19:09:41

but I use asami as a backend for odoyle rules

quoll19:09:42

OK, that's a SQL operation. I tend to think of these operations in terms of what you're actually trying to do

quoll19:09:59

so, do you know what you're trying to do?

nivekuil19:09:08

odoyle rules can be added dynamically. a rule fires when all its clauses match, but if we add a rule dynamically, we can run into a problem where the state it would match on was added before the rule was added. so we need to hydrate from somewhere

nivekuil19:09:21

if the full match can be hydrated, the default inner join works. but we actually need partial hydration. consider if the fact [1 :foo "foo"] comes in, then the subscription [:what [_ :foo foo] [_ :bar bar]] queries asami, it doesn't fully match, so it would get hydrated nothing! so now [1 :bar "bar"] would not cause the subscription to fire even though asami knows enough facts to satisfy the match

nivekuil19:09:29

so wrapping everything in optional solves partial hydration but also can give wrong hydration, since we have to distinguish between facts with nil values and facts that were never asserted

nivekuil19:09:41

like i said, bit complicated.. but you asked

nivekuil19:09:15

higher level than that, this enables reactive datalog-driven UI

nivekuil19:09:47

all that said the easy workaround is to do two queries with/without optional and diff them, was just wondering if there's a better way to do it

quoll19:09:14

2 queries was my first thought 🙂

quoll19:09:58

The only other thing that comes to mind is updating Asami to return an "unbound" value for optional. The thing about optional was that I was trying to copy SPARQL, and SPARQL doesn't allow for nil values.

nivekuil19:09:43

really? how do you affirm absence

quoll19:09:35

In RDF? Typically with a predicate that indicates that something is not present

quoll19:09:24

There's also rdf:nil

nivekuil19:09:06

would it be more correct to reify nil instead of unbound then

quoll19:09:07

though that's actually a list

quoll19:09:29

well, nil is current allowed to be stored

nivekuil19:09:35

that's true

nivekuil19:09:45

I guess the user could do it

quoll19:09:07

and you'll see it in normal results. Only unions and optionals can have unbound. And I really should split that apart

quoll19:09:29

in fact, now that I say it, I will need this when I bring in Twylyte

quoll19:09:52

(that's a SPARQL wrapper)

quoll19:09:14

(def unbound (Object.))
Then project will need to convert these to nil, and isBound can test for equality.

quoll23:09:00

Yay! Raphael 0.3.0 is done! Now I can integrate it into Asami!

❤️ 4
quoll23:09:08

That was a LOT of work

🥳 3
quoll23:09:46

The commit says: Showing 8 changed files with 994 additions and 883 deletions.

quoll23:09:57

Thank goodness I had a lot of tests

🚀 2