Fork me on GitHub
#asami
<
2021-06-02
>
alandipert19:06:34

@quoll getting somewhere with the datalog-in-JS, finally figured out how to synchronize the result sets with the dom nodes - https://gist.github.com/alandipert/ce0fa7dcb6cd74c4f5ea6679fe29dce1

alandipert19:06:20

ended up going bottom-up and diffing results as you suggested. seems to work nicely

👍 2
alandipert19:06:12

the ANY.VALUE thing is a weird discovery, it seems to be a way to do disjunction based on query param, without changing the query

alandipert19:06:45

...being a sentinel value understood by the join loop to match any value

alandipert19:06:02

was curious if you'd run across anything like that in your travels, or if it's otherwise a well know thing

quoll19:06:55

I have not used JS very much, so I haven’t seen it before, no

alandipert19:06:49

oh, ANY.VALUE is just the name of the sentinel value. when you pass that particular value in as a query parameter, it matches anything. sort of like a user-supplied wildcard

alandipert19:06:09

... as opposed to a (static) wildcard in the query text itself

quoll20:06:52

OK, I was busy and I’ve thought about it a little now. I think I see what you’re saying. So you have a pattern of: [_.e, "status", _.status] where the _.status value is supplied as a parameter, but in this case you have a parameter that’s a wildcard.

quoll20:06:31

If I’m getting that right… I haven’t seen that parameterized before

quoll20:06:46

In Asami, the find and where clauses wouldn’t change, but if you didn’t want to restrict the status value, you’d have to remove the use clause (called :in in Asami)

alandipert20:06:07

yes i think that's right, thanks for grokking

alandipert20:06:32

it came up for me because i need to (re-)parameterize queries asynchronously, and it didn't seem right for the query pattern itself to change within the "extent" of the subscription established by view

alandipert20:06:42

how's asami going btw?

quoll20:06:07

Up and down. I keep having things pull me away from it, but I keep getting back to it 🙂

quoll20:06:23

This week, I’m turning it into a document store 😜

quoll20:06:35

though today I’m taking the time to write wiki pages

quoll20:06:31

New element to the storage

quoll20:06:10

right now, if you store objects in it, it disassembles the objects into triples, and stores those. If you ask for the object back it reassembles it

quoll20:06:32

Which works correctly, and it’s why I can do round-trips on JSON documents

quoll20:06:45

But it takes time to rebuild an object to return

quoll20:06:56

so I’m storing the object as well

alandipert20:06:11

nice, with an invalidation/rebuild scheme?

quoll20:06:49

in case you are really trying to mess the system up by updating triples that belong to objects

quoll20:06:36

To enable this, all sub-entities now have ownership attributes that connect them to their root object

quoll21:06:08

If you scroll down a little from https://github.com/threatgrid/asami/wiki/Entity-Structure#structures-and-queries, you’ll see it in an example ball-and-stick diagram

quoll21:06:24

All the light lines are the ownership attributes

quoll21:06:06

well, I’m still writing it

quoll21:06:08

it’s much easier in-memory, since the objects themselves just go into a map. The on-disk version is trickier because I have to serialize everything

alandipert21:06:50

hm, the example reminds me of stuff i've done in datomic in the past, converting trees to sets of tuples. the document-in interface seems broadly useful. then i see you have something like datomic pull api on the other end. i like it :thumbsup:

quoll21:06:41

The facilities are already there. This will just make it faster (I hope!)

alandipert21:06:11

very cool. yeah i don't think datomic has an option to cache entities or anything

quoll21:06:20

This is why I can load a JSON file in 2 lines of code, and start querying it on line 3

quoll21:06:33

oh… and because of misbehaving JSON, I removed some of the type checking. So now entities and attributes can be anything. This is a semantic nightmare, but turns out to be useful. 🙂

alandipert21:06:53

heck yeah, json compat on any level brings a lot of reach and applicability

alandipert21:06:40

you know, i suppose what i'm working on is very similar

alandipert21:06:06

updating dom nodes surgically in response to a stream of query changes. it's almost exactly document maintenance with datalog backend

alandipert21:06:48

well, modulo all the ways asami is different, and javascript 🙃

quoll21:06:14

I was asked about one file, and ended up exploring it with things like:

(def conn (d/connect "asami:"))
(def tx @(d/transact conn (json/parse-string (slurp "cb.json"))))

(d/q '[:find (count ?a) . :where [?e ?a ?v]] db)
(d/q '[:find [?a ...] :in $ ?bad-string :where [?e ?a ?v] [(?bad-string ?a)]]
  conn #(and (string? %) (re-find #" " %)))
Which is all easy in other systems as well

alandipert21:06:53

yes but we like datalog simple_smile

alandipert21:06:37

what does the ... in the :find do?

quoll21:06:41

But then I wanted to find the entities that contain an attribute with a space in them…

(->> (d/q '[:find [?ancestor ...]
            :where [?ancestor :tg/entity true]
                   [?ancestor ?a+ ?e]
                   [?e ?attr ?v]
                   [(bad-string? ?attr)]] db #(and (string? %) (re-find #" " %)))
     (map #(d/entity db %)))

quoll21:06:21

Normally a query returns a sequence of bindings, where each binding is a sequence of values that align with the :find clause

quoll21:06:53

If the :find clause only contains one element, then you get back a sequence of 1 element sequences. e.g.

(["foo"] ["bar"] ["baz])
If you want to, you can send that to a (map first …) but it’s an extra step

alandipert21:06:11

gotcha, i like that

quoll21:06:17

By specifying [?ancestor …] then you get back a seq of values instead

alandipert21:06:09

?a+ is another interesting one, do i understand right that deals with arbitrary depth?

👍 2
quoll21:06:59

Not only is it arbitrary depth, it’s with arbitrary attributes!

alandipert21:06:24

yeah that's a really good one

quoll21:06:56

this operation is typically reserved for a single attribute. Like:

[?person :has-child+ ?child]
Will return a list of all descendants

quoll21:06:30

Following arbitrary attributes to arbitrary depth is fraught! If your graph has a spoke and hub structure (like DNS), then you could traverse the entire graph

quoll21:06:55

… if you happened to start at the wrong spot 🙂

alandipert21:06:50

can you have cycles? this could help you find them

quoll21:06:16

It detects cycles

alandipert21:06:40

also, if you have ?a in your :find, do you get the product of the lvars and every ?a in the tree?

quoll21:06:44

no. I actually cheat a bit

quoll21:06:12

the ?a value does not bind to a single value per binding, as you’d normally expect

quoll21:06:27

instead, it binds to a vector containing a path

quoll21:06:48

So you can actually select the path, if that’s what you want

quoll21:06:27

I’m sure that this will break in some corner cases. I haven’t tested that all that much. But for simple queries it provides a neat way to find paths

alandipert21:06:55

yeah i think affordances like this are a net win, and i'm sure i don't have nearly the experience working with this stuff that you do

alandipert21:06:15

...because these things can be understood in terms of the underlying model, they're not arbitrary like with many other query tools

quoll21:06:21

The issue isn’t calculating it. It’s figuring out how to make these calculations show up in query results in a way that makes sense, and does not require an excessive number of special cases

alandipert21:06:44

it makes sense to me so i think you're crushing it

💖 2
quoll21:06:16

Also, if you’re getting back a single value, it’s annoying to get [[value]]. Even if you use the [?arg …] syntax you still get [value]. So you use .

alandipert21:06:44

you have some awesome extensions going on, i must say. 💯

alandipert21:06:59

don't mind if i cram a few of these into JS syntax

quoll21:06:09

It’s open source for a reason!

🍻 2
alandipert22:06:05

regarding :find syntax, any reason you didn't use e.g. :find ?ancestor to get the flat sequence back?

quoll22:06:52

In that query, the ?ancestor node was locked (by virtue of the [?ancestor :tg/entity true] constraint. Only the ?e was floating. That ends up getting locked by the pair of constraints: [?e ?attr ?v] [(bad-string? ?attr)]

quoll22:06:58

so the sequence of nodes leading from the root attribute down to the bad attribute was actually in the ?e

quoll22:06:17

If that’s what you’re asking about?

alandipert22:06:29

i'm asking about the syntax choice of [x ...] over x

alandipert22:06:00

wasn't sure if ... has other uses/meanings, or if simply x is otherwise fraught

quoll22:06:42

well, I find it easier to work with [:a :b :c :d] than with [[:a] [:b] [:c] [:d]]

alandipert22:06:45

..although i'm grateful for the words "locked" and "floating" to describe the lvar status :thumbsup:

quoll22:06:15

I can get from the latter to the former with (map first) but that’s not repl friendly

alandipert22:06:23

oh, i agree with you, i'm asking in particular about the grammar choice of the query map

quoll22:06:56

Also… I’m going to keep saying it: I really wish it wasn’t called “Datalog”

💯 3
alandipert22:06:16

i'm out for now, it was nice catching up 👋

quoll22:06:44

good night!