Fork me on GitHub
#datalog
<
2022-05-05
>
stopa14:05:38

Hey team, curious datalog question. Imagine we are creating a Notion-like system. We have Block entities, which points to other block entities. Page points to different blocks like paragraph, link, bulleted list, which themselves point to more blocks. Some blocks only show “partial information”. For example, if you link to a different page, this in itself becomes a child “page” block, but it would only show a kind of box to link out too. Now, say I want to render this page. I would want to write a query like this: • pull from this entity id, all connection nodes, unless I hit another block with the type page I am looking at the datomic pull syntax, and it looks like something like this is not possible. Am I correct? Is there another way you’d achieve this?

zeitstein14:05:11

With pull syntax you can manually specify the joins, thus excluding joins to page links. Some care needed if using :db/isComponent (those can be automatically pulled recursively). I might be misunderstanding your data model, though.

stopa14:05:22

If I understood correctly, we would simply not get the children who are pages? I think this may not quite work though. We would want to get that node, but in essence stop the recursion right there. This for the case, where you see a “preview” box. This would require the child page entity, but just not its children.

zeitstein14:05:30

That should also be possible. I can take a crack at writing the pull pattern if you provide a simple model example.

stopa14:05:04

(Okay, one min!)

stopa14:05:38

This is how the entity can look:

block/id
block/type // can be "page", "to_do", "bulleted_list", "kanban", "toggle", "text"
block/properties // bag of json. i.e {title: "foo"}

block/child // one -> many cardinality. connects one block to a list of children
And this is some example data:
{:id :page-a, :type :page}
{:id :child-a, :type :to_do}
{:id :subchild-a, :type :text}
{:id :grandchild-a, :type :text}
{:id :page-b, :type :page}
{:id :child-b, :type text}
With the following connections:
[:page-a :block/child :child-a]
[:child-a :block/child :subchild-a]
[:subchild-a :block/child :grandchild-a]
[:child-a :block/child :page-b]
[:page-b :block/child :subchild-c]
I would want to pull:
{:id :page-a 
 :child
 [{:id child-a,
   :child [{:id :page-b},
           {:id :subchild-a
            :child [{:id :grandchild-a}]}]}]}
(idea being, that page-b did not expand, but subchild-a did expand). This is because page-b had a type page.

stopa14:05:31

(As an aside, am getting the data model from https://www.notion.so/blog/data-model-behind-notion from Notion)

zeitstein15:05:11

Ah, so recursion is done solely on the :child attribute. I had misunderstood your original post, then. So, you'd have to stop pull based on :type of the block. I don't know how you would do that, sorry.

stopa15:05:27

Gotcha! Thank you. Curious, is there a way you’d change the data model to support it?

zeitstein15:05:08

Maybe have a :type :link for a block with attribute :block/link :page-b ?

zeitstein15:05:37

I also remembered https://www.zsolt.blog/2021/01/Roam-Data-Structure-Query.html. Don't know if it could be helpful, though.

stopa15:05:56

Noting to read!

stopa15:05:34

> Maybe have a :type :link for a block with attribute :block/link :page-b I see, so then that would be a terminal node. Good idea!

zeitstein15:05:57

That would allow you to both terminate on :block/link but also continue pulling, depending on the situation.

stopa15:05:59

To make sure I understand correctly, would you mind sharing a sample pull query with what you mean? What I understand is, this pull query:

[:block/properties, {:block/child …}, {:block/link ;; here we can choose 
]]
Is this what you mean?

stopa15:05:00

Awesome. Thanks for going on this journey with me @U02E9K53C9L!

🚀 1
dvingo15:05:17

hi @U0C5DE6RK! If i'm following correctly I think recursive rules would work for this without needing to rely on changing the attribute model

stopa15:05:42

Oo. I’m definitely curious. How would you set it up?

dvingo15:05:17

ooh ok, i want to explore this, but need to grab some food first - will followup in a bit!

❤️ 1
stopa15:05:51

Woohoo, enjoy and looking forward @U051V5LLP 🙂

dvingo17:05:38

oook I think it's working!

dvingo17:05:26

(ns recursive-rules
  (:require [xtdb.api :as xt]))

(defonce node (xt/start-node {}))

(defn make-block [id type props children]
  (cond-> {:block/id id :block/type type}
    props (assoc :block/properties props)
    children (assoc :block/children children)))

(defn ident [m] (:block/id m))
(defn idents [& m] (mapv :block/id m))

(def leaf3 (make-block #uuid"2079a8b4-e939-4ebb-a88b-deb8a856d7bb" :text {:content "leaf 3"} nil))
(def leaf4 (make-block #uuid"88162056-dce8-4142-a85d-a0570eaf8e40" :text {:content "leaf 4"} nil))

(def container1 (make-block #uuid"7b85881e-40d4-4727-921e-183c9012c490" :container {} (idents leaf3 leaf4)))
(def leaf1 (make-block #uuid"fcecdd78-2a27-4e3e-9410-5ea5a45b8bda" :text {:content "leaf 1"} nil))
(def leaf2 (make-block #uuid"67b62a83-d763-4dce-9cd6-f488a149ac5a" :text {:content "leaf 2"} nil))

(def bullet-list (make-block #uuid"a2107c25-5c72-48c6-a1fa-bad341ca631b" :bullet-list nil (idents leaf1 leaf2 container1)))

(def todo1 (make-block #uuid"44ddbac9-5097-4980-a6fe-287448f98094" :todo {:content "task1 to do"} nil))
(def todo2 (make-block #uuid"fe6e9d31-3cde-4683-91ab-e558b0fc7aa3" :todo {:content "task2 to do"} nil))
(def todo3 (make-block #uuid"74b41616-c744-4054-88c4-cf2354fc7938" :todo {:content "task3 to do"} nil))

(def todo-page (make-block #uuid"a21b0958-8ee5-48d2-aeca-209d6dc47587" :page {:title "my todo list"} (idents todo1 todo2 todo3)))

(def home-page (make-block #uuid"900e63a5-0ad2-4710-aee2-84d6aeae1339" :page {:title "my home page"} (idents todo-page bullet-list)))
(defn xt-put [d] [::xt/put (assoc d :xt/id (ident d))])

(defn submit-data []
  (xt/submit-tx node (mapv xt-put [leaf3 leaf4 container1 leaf1 leaf2 bullet-list todo1 todo2 todo3 todo-page home-page]))
  (xt/sync node))

(defn only-non-pages
  [db id]
  (xt/q db
    '{:find  [(pull children [*])]
      :in    [root-id]
      :where [[block :block/id root-id]
              (bind-children block children)]
      :rules [[(bind-children parent child)
               [parent :block/children child]]
              [(bind-children parent child1)
               [parent :block/children child2]
               (not [child2 :block/type :page])
               (bind-children child2 child1)]]}
    id))

(comment
  (submit-data)
  (mapv first (only-non-pages (xt/db node) (ident home-page)))
  )

dvingo17:05:21

this returns the following:

[{:block/id #uuid"a21b0958-8ee5-48d2-aeca-209d6dc47587",
  :block/type :page,
  :block/properties {:title "my todo list"},
  :block/children [#uuid"44ddbac9-5097-4980-a6fe-287448f98094"
                   #uuid"fe6e9d31-3cde-4683-91ab-e558b0fc7aa3"
                   #uuid"74b41616-c744-4054-88c4-cf2354fc7938"],
  :xt/id #uuid"a21b0958-8ee5-48d2-aeca-209d6dc47587"}
 {:block/id #uuid"fcecdd78-2a27-4e3e-9410-5ea5a45b8bda",
  :block/type :text,
  :block/properties {:content "leaf 1"},
  :xt/id #uuid"fcecdd78-2a27-4e3e-9410-5ea5a45b8bda"}
 {:block/id #uuid"67b62a83-d763-4dce-9cd6-f488a149ac5a",
  :block/type :text,
  :block/properties {:content "leaf 2"},
  :xt/id #uuid"67b62a83-d763-4dce-9cd6-f488a149ac5a"}
 {:block/id #uuid"a2107c25-5c72-48c6-a1fa-bad341ca631b",
  :block/type :bullet-list,
  :block/children [#uuid"fcecdd78-2a27-4e3e-9410-5ea5a45b8bda"
                   #uuid"67b62a83-d763-4dce-9cd6-f488a149ac5a"
                   #uuid"7b85881e-40d4-4727-921e-183c9012c490"],
  :xt/id #uuid"a2107c25-5c72-48c6-a1fa-bad341ca631b"}
 {:block/id #uuid"88162056-dce8-4142-a85d-a0570eaf8e40",
  :block/type :text,
  :block/properties {:content "leaf 4"},
  :xt/id #uuid"88162056-dce8-4142-a85d-a0570eaf8e40"}
 {:block/id #uuid"7b85881e-40d4-4727-921e-183c9012c490",
  :block/type :container,
  :block/properties {},
  :block/children [#uuid"2079a8b4-e939-4ebb-a88b-deb8a856d7bb" #uuid"88162056-dce8-4142-a85d-a0570eaf8e40"],
  :xt/id #uuid"7b85881e-40d4-4727-921e-183c9012c490"}
 {:block/id #uuid"2079a8b4-e939-4ebb-a88b-deb8a856d7bb",
  :block/type :text,
  :block/properties {:content "leaf 3"},
  :xt/id #uuid"2079a8b4-e939-4ebb-a88b-deb8a856d7bb"}]
this is not in tree shape, but you can then use tools like fulcro's db->tree https://github.com/fulcrologic/fulcro/blob/2ad04fd57c42ffecf5aec24eaa6c8bf574814613/src/main/com/fulcrologic/fulcro/algorithms/denormalize.cljc#L212 to give you that

dvingo17:05:14

that above query gives you back the todo page but none of its children

dvingo17:05:45

maybe https://github.com/lilactown/pyramid is a better fit for then getting a tree from this

dvingo18:05:07

the other cool thing about using rules is you can add other logic like permissioning in there - like maybe some of the blocks are not visible to some users for whatever reason

stopa20:05:50

This rocks. Thank you for going so deep @U051V5LLP. Will be noodling on this.

dvingo22:05:26

no prob! this stuff is fascinating, happy to help