Clojurians Log v2

Trying to store children as array of references, I've no idea how to obtain a path between nodes. Consider:

[{:db/id :tg/node-0 :children [{:db/id :tg/node-1}]}
 {:db/id :tg/node-1 :children [{:db/id :tg/node-2}]}
 {:db/id :tg/node-2 :children []}]

(d/q '[:find ?p :where [?p :children ?a] [?a :tg/contains :tg/node-2]] (d/db conn))

Then I thought I could use :tg/owns if I stored them as array of objects, but I found out only node-0 owns everything: node-1 owns neither the array (containing node-2), nor node-2.

[{:db/id :tg/node-0
  :children [{:db/id :tg/node-1
              :children [{:db/id :tg/node-2
                          :children []}]}]}]

:tg/owns is to track objects under the top-level objects.

Try:

(d/q '[:find ?p :where [?p ?a+ :tg/node-2][?p :children] (d/db conn))

That should get every parent of node-2 and then cut it back to only the nodes with a :children attribute

(I hope)

That works. Thank you!

Makes sense. Not that it really matters unless I introduce :db.fn/retractEntity

this would tie into another change I’m considering

I don’t want to introduce schemas into Asami, but I’ve been thinking about allowing temporary schemas during transactions

something like:

(transact conn {:tx-data [{:db/ident "first" :data 1} {:db/ident "second" :data 2}]})
(transact conn {:tx-data [{:db/ident "first" :data 42}]
                :schema {:db/ident :data :db/cardinality :db/cardinality/one})

(entity conn "first")
;; {:data 42}

someone brought up the idea of temporary schemas when I was showing off datalog support for pyramid. seems like it would be welcome

I don’t know what I’ll do with schema properties I don’t care about.

Maybe use them for validity checking

(transact conn {:tx-data [{:db/ident "first" :data "forty-two"}]
                :schema {:db/ident :data
                         :db/cardinality :db/cardinality/one
                         :db/valueType :db.type/long})
;; Exception "Attribute :data cannot accept data: \"forty-two\"". Requires Long value

sounds useful!

https://github.com/threatgrid/asami/issues/223

Could :tg/own be used instead of :db/isComponent? If :tg/own was changed so that node-1 owns node-2 in example below:

[{:db/id :tg/node-0
  :children [{:db/id :tg/node-1
              :children [{:db/id :tg/node-2
                          :children []}]}]}]

It specifically goes to the root object. It's an internal method of tracking things that are components

If a sub node is not a component, then it shouldn't be inserted that way

That's a flippant reply, but it's not meant to be. Hopefully I can explain better in the morning

Not at all, and I had taken enough of your time yesterday! What I'm trying to say is that, looking at my example above naively, node-2 is in precisely the same relation to node-1 as node-1 is in relation to node-0 (and through the same attribute). Generally, if object 2 is nested within object 1, it feels natural to me to treat object 2 as a component of object 1 by default. For things that are not components, I would use references instead of nesting. But don't mind me 🙂 And thanks for your help!

Thanks for the detailed explanation!

Well, my comment was based on what :tg/owns is trying to do. It’s a system property that I use to track relationships between embedded objects. Every “top level” entity (entities that are described independently, and not exclusively inside another entity) will “own” everything underneath it that isn’t also a top level entity. One reason for this is for an unpublished index (I need to finish more work before I put it out there, but it’s being stored on disk already). Entities are serialized to bytes, and stored in this index. If you request an entity, then there will be a lookup for a triple that doesn’t yet exist (but should do… eventually): [your-entity :tg/index ?index] If there is a result for ?index then this will be looked up in the new index. This is deserialization operation, and much faster than building an object out of triples. If it’s not there, then it can be rebuilt using triples. But what if you update the object? Well, like any good immutable store, this requires a object be written to the index. It also means that anything it is embedded in will also need to be rewritten, up to the root embedding. I can find that recursively, by looking up: [?parent ?attribute+ your-entity] But this is tricky, since I don’t want to include anything that refers to your entity, but doesn’t actually own it. That means tracking the original root entity that this was created under. And that’s the :tg/owns attribute.

Incidentally, one of the reasons for the deserialization mess is because if entities look like: {:id 1, :a "a property", :b 2, :c [{:id 11, :x 1, :y 2}, {:id 12, :x 10, :y 15}]} Then there will be a pointer to the bytes that serialize all of this, but the serialization step also pulls out the offsets where the :id 1 and the :id 2 entities start, and these can be deserialized independently of the surrounding entity. As per usual, the above structure would create triples something like:

:tg/node-101 :id 1
:tg/node-101 :a "a property"
:tg/node-101 :b 2
:tg/node-101 :c :tg/node-102
:tg/node-102 :tg/first :tg/node-105
:tg/node-102 :tg/rest :tg/node-103
:tg/node-103 :tg/first :tg/node-106
:tg/node-102 :tg/contains :tg/node-105
:tg/node-102 :tg/contains :tg/node-106
:tg/node-101 :tg/owns :tg/node-102
:tg/node-101 :tg/owns :tg/node-105
:tg/node-101 :tg/owns :tg/node-106
:tg/node-105 :id 11
:tg/node-105 :x 1
:tg/node-105 :y 2
:tg/node-106 :id 12
:tg/node-106 :x 10
:tg/node-106 :y 15

:tg/node-101 :tg/index 555
:tg/node-105 :tg/index 580
:tg/node-105 :tg/index 594

Thanks for this. :id is reasonably new but it should have worked. I’ll look at this soon

Thanks!

Think I’ve found the bug. Testing now

The problem with these things is that writing tests always takes longer than fixing the bug! 😆

Example:

(d/transact conn [{:id "1" :text "text"}])  ; create
(d/transact conn [{:id "1" :text "update text"}])  ; update

;; create
(#datom [:tg/node-21976 :id "1" 1 true]
 #datom [:tg/node-21976 :text "text" 1 true]
 #datom [:tg/node-21976 :db/ident :tg/node-21976 1 true]
 #datom [:tg/node-21976 :tg/entity true 1 true])
;; update
(#datom [:tg/node-21979 :id "1" 2 true]
 #datom [:tg/node-21979 :text "id text" 2 true]
 #datom [:tg/node-21979 :db/ident :tg/node-21979 2 true]
 #datom [:tg/node-21979 :tg/entity true 2 true])

(d/transact conn [{:id "1" :text' "update text"}])  ; update

;; create
(#datom [:tg/node-22003 :id "1" 1 true]
 #datom [:tg/node-22003 :text "text" 1 true]
 #datom [:tg/node-22003 :db/ident :tg/node-22003 1 true]
 #datom [:tg/node-22003 :tg/entity true 1 true])
;; update
(#datom [:tg/node-22003 :text "text" 2 false]
 #datom [:tg/node-22006 :id "1" 2 true]
 #datom [:tg/node-22006 :text "update text" 2 true]
 #datom [:tg/node-22006 :db/ident :tg/node-22006 2 true])