Fork me on GitHub
#xtdb
<
2020-03-25
>
Eric Scott13:03:18

Hi, I note that the 'official documentation' link on your README is broken: https://juxt.pro/crux/docs/index.html

Eric Scott13:03:13

Is this temporary or should I log an issue?

refset13:03:45

Hi @eric.d.scott - it's temporary, and we've been looking into it all morning as it affects the whole domain (including email!). Luckily though the docs now live here: https://www.opencrux.com/docs - so the http://juxt.pro redirect isn't too big a blocker 🙂

👍 4
dvingo14:03:30

Hey, all. I'm wondering if anyone has a helper for recursive crux/entity calls (basically "pull" a node including all children refs) that doesn't consume the stack. I have a version adopted from the entity-with-adjacent helper in the docs, but it is stack-consuming recursive:

(def tasks
    [{:crux.db/id :task-1 :children   #{:task-2 :task-3 :task-4} :name "task 1"}
     {:crux.db/id :task-2 :children #{:task-5} :name "task 2"}
     {:crux.db/id :task-3 :children #{} :name "task 3"}
     {:crux.db/id :task-4 :children #{} :name "task 4"}
     {:crux.db/id :task-5 :children #{} :name "task 5"}])

(defn pull-entity
  [entity-id ref-keys]
  (let [db     (crux/db crux-node)
        entity (crux/entity db entity-id)]
    (reduce
      (fn [e adj-k]
        (let [v (get e adj-k)]
          (assoc e adj-k
                   (cond
                     (keyword? v) (crux/entity db v)
                     (set? v) (into #{} (map #(pull-entity % ref-keys) v))
                     (vector? v) (mapv #(pull-entity % ref-keys) v)
                     :else v))))
      entity
      ref-keys)))

;; use it like so:
(pull-entity :task-1 [:children])

;; returns:
{:crux.db/id :task-1,
 :children #{{:crux.db/id :task-4, :children #{}, :name "task 4"}
             {:crux.db/id :task-2, :children #{{:crux.db/id :task-5, :children #{}, :name "task 5"}}, :name "task 2"}
             {:crux.db/id :task-3, :children #{}, :name "task 3"}},
 :name "task 1"}
I think this can work given my use case shouldn't overflow the stack, but it's not ideal.

refset15:03:39

Hey - I don't have anything handy but I'm looking at this now. I think I know how to get loop/recur working

dvingo15:03:30

awesome! thanks. I will keep pondering it see if I can get something working as well.

refset15:03:33

I think it's a matter of: lift pull-entity to pull-entities, track the collection type as a parameter, open a loop, and use loop/recur in place of the reduce. I'm pausing for 30m, will resume then

refset18:03:24

Hmm, it's tricky! I still think a loop/recur solution is possible but it will require a zipper. A much faster way to avoid worrying about the stack would be to use this: https://github.com/divs1210/xodarap (which I've used successfully before, but it brings in core.async and will be slower) I have to give up for the time being, sadly. Hopefully someone else has some good intuitions 🙂

dvingo18:03:32

No worries! thanks for taking the time to look into it. I have some ideas but won't be able to play around with it until later

👍 4
dvingo14:03:21

Thanks for the tip on using zippers. I got a working version that supports sets in value positions, but can expand to vectors and lists easily. https://gist.github.com/dvingo/44a040ba9d0de2284e151811945a0cd0

🎉 4
refset14:03:33

Awesome, I'm glad the tip helped!