Clojurians Log v2
asami 2021-06-02
@quoll getting somewhere with the datalog-in-JS, finally figured out how to synchronize the result sets with the dom nodes - https://gist.github.com/alandipert/ce0fa7dcb6cd74c4f5ea6679fe29dce1
ended up going bottom-up and diffing results as you suggested. seems to work nicely
the ANY.VALUE thing is a weird discovery, it seems to be a way to do disjunction based on query param, without changing the query
...being a sentinel value understood by the join loop to match any value
was curious if you'd run across anything like that in your travels, or if it's otherwise a well know thing
I have not used JS very much, so I haven’t seen it before, no
oh, ANY.VALUE is just the name of the sentinel value. when you pass that particular value in as a query parameter, it matches anything. sort of like a user-supplied wildcard
... as opposed to a (static) wildcard in the query text itself
OK, I was busy and I’ve thought about it a little now.
I think I see what you’re saying. So you have a pattern of:
[_.e, "status", _.status]
where the _.status value is supplied as a parameter, but in this case you have a parameter that’s a wildcard.
If I’m getting that right… I haven’t seen that parameterized before
In Asami, the find and where clauses wouldn’t change, but if you didn’t want to restrict the status value, you’d have to remove the use clause (called :in in Asami)
yes i think that's right, thanks for grokking
it came up for me because i need to (re-)parameterize queries asynchronously, and it didn't seem right for the query pattern itself to change within the "extent" of the subscription established by view
how's asami going btw?
Up and down. I keep having things pull me away from it, but I keep getting back to it 🙂
This week, I’m turning it into a document store 😜
though today I’m taking the time to write wiki pages
interesting, like a document-flavored frontend layer? or a rethink of the storage etc
New element to the storage
right now, if you store objects in it, it disassembles the objects into triples, and stores those. If you ask for the object back it reassembles it
Which works correctly, and it’s why I can do round-trips on JSON documents
But it takes time to rebuild an object to return
so I’m storing the object as well
nice, with an invalidation/rebuild scheme?
yes
in case you are really trying to mess the system up by updating triples that belong to objects
To enable this, all sub-entities now have ownership attributes that connect them to their root object
If you scroll down a little from https://github.com/threatgrid/asami/wiki/Entity-Structure#structures-and-queries, you’ll see it in an example ball-and-stick diagram
All the light lines are the ownership attributes
well, I’m still writing it
it’s much easier in-memory, since the objects themselves just go into a map. The on-disk version is trickier because I have to serialize everything
hm, the example reminds me of stuff i've done in datomic in the past, converting trees to sets of tuples. the document-in interface seems broadly useful. then i see you have something like datomic pull api on the other end. i like it 👍
The facilities are already there. This will just make it faster (I hope!)
very cool. yeah i don't think datomic has an option to cache entities or anything
This is why I can load a JSON file in 2 lines of code, and start querying it on line 3
https://github.com/alandipert/intension is how i used to do it
oh… and because of misbehaving JSON, I removed some of the type checking. So now entities and attributes can be anything. This is a semantic nightmare, but turns out to be useful. 🙂
heck yeah, json compat on any level brings a lot of reach and applicability
you know, i suppose what i'm working on is very similar
updating dom nodes surgically in response to a stream of query changes. it's almost exactly document maintenance with datalog backend
well, modulo all the ways asami is different, and javascript 🙃
I was asked about one file, and ended up exploring it with things like:
(def conn (d/connect "asami:"))
(def tx @(d/transact conn (json/parse-string (slurp "cb.json"))))
(d/q '[:find (count ?a) . :where [?e ?a ?v]] db)
(d/q '[:find [?a ...] :in $ ?bad-string :where [?e ?a ?v] [(?bad-string ?a)]]
conn #(and (string? %) (re-find #" " %))) yes but we like datalog simple_smile
what does the ... in the :find do?
But then I wanted to find the entities that contain an attribute with a space in them…
(->> (d/q '[:find [?ancestor ...]
:where [?ancestor :tg/entity true]
[?ancestor ?a+ ?e]
[?e ?attr ?v]
[(bad-string? ?attr)]] db #(and (string? %) (re-find #" " %)))
(map #(d/entity db %)))Normally a query returns a sequence of bindings, where each binding is a sequence of values that align with the :find clause
If the :find clause only contains one element, then you get back a sequence of 1 element sequences. e.g.
(["foo"] ["bar"] ["baz])(map first …) but it’s an extra stepgotcha, i like that
By specifying [?ancestor …] then you get back a seq of values instead
?a+ is another interesting one, do i understand right that deals with arbitrary depth?
Not only is it arbitrary depth, it’s with arbitrary attributes!
yeah that's a really good one
this operation is typically reserved for a single attribute. Like:
[?person :has-child+ ?child]Following arbitrary attributes to arbitrary depth is fraught! If your graph has a spoke and hub structure (like DNS), then you could traverse the entire graph
… if you happened to start at the wrong spot 🙂
can you have cycles? this could help you find them
It detects cycles
also, if you have ?a in your :find, do you get the product of the lvars and every ?a in the tree?
no. I actually cheat a bit
the ?a value does not bind to a single value per binding, as you’d normally expect
instead, it binds to a vector containing a path
So you can actually select the path, if that’s what you want
I’m sure that this will break in some corner cases. I haven’t tested that all that much. But for simple queries it provides a neat way to find paths
yeah i think affordances like this are a net win, and i'm sure i don't have nearly the experience working with this stuff that you do
...because these things can be understood in terms of the underlying model, they're not arbitrary like with many other query tools
The issue isn’t calculating it. It’s figuring out how to make these calculations show up in query results in a way that makes sense, and does not require an excessive number of special cases
Also, if you’re getting back a single value, it’s annoying to get [[value]]. Even if you use the [?arg …] syntax you still get [value]. So you use .
you have some awesome extensions going on, i must say. 💯
don't mind if i cram a few of these into JS syntax
regarding :find syntax, any reason you didn't use e.g. :find ?ancestor to get the flat sequence back?
In that query, the ?ancestor node was locked (by virtue of the [?ancestor :tg/entity true] constraint. Only the ?e was floating. That ends up getting locked by the pair of constraints: [?e ?attr ?v] [(bad-string? ?attr)]
so the sequence of nodes leading from the root attribute down to the bad attribute was actually in the ?e
If that’s what you’re asking about?
i'm asking about the syntax choice of [x ...] over x
wasn't sure if ... has other uses/meanings, or if simply x is otherwise fraught
well, I find it easier to work with [:a :b :c :d] than with [[:a] [:b] [:c] [:d]]
..although i'm grateful for the words "locked" and "floating" to describe the lvar status 👍
I can get from the latter to the former with (map first) but that’s not repl friendly
oh, i agree with you, i'm asking in particular about the grammar choice of the query map
Ah!
ah yes, ty
Also… I’m going to keep saying it: I really wish it wasn’t called “Datalog”
seconded
i'm out for now, it was nice catching up 👋
good night!