Fork me on GitHub
#crux
<
2021-01-29
>
Toyam Cox07:01:34

In crux, if I include a single edge (like :git-hash "abc") in every single object, will I hit a size penalty for storing an extra edge pet object? Will that be stored as-is in the document store?

refset18:01:42

Essentially, yes there would be extra storage consumed per edge. It's possible that a future document store can implement some clever git-like compression / structural sharing, but that's not how things work today.

Toyam Cox22:01:21

Thanks for the easy answer!

Aleksander Rendtslev20:01:44

Shouldn’t this work for projections?

(def chicken (repo/q
                 *node*
                 `{:find  [(eql/project ?el [*])]
                   :where [[?el :food-item/id]]}))
I’m getting an “illegal-argument” error. It works fine if I replace it with :food-item/name So the “catch all” projection doesn’t seem to work. I’m on the latest version of crux

refset21:01:54

Hi 🙂 this looks like it should work to me. Could you share the stack trace for the error please?

refset21:01:13

Also which document store are you using?

jarohen22:01:06

it'll likely be the backtick - try replacing it with a quote '?

👌 1
Aleksander Rendtslev20:01:40

is it possible to do joins on elements in a vector? eg:

{:food-item/id        (uuid)
                            :food-item/name      "Cheese"
                            :food-item/nutrients [{:nutrient/id   NUTRIENT_TYPE_ID
                                                   :nutrient/amount 150}]}
Where calories is :
{:nutrient-type/id        (uuid)
                                :nutrient-type/nickname  "Calories"
                                :nutrient-type/unit-name "KCAL"
                                :nutrient-type/name      "Energy"}
So what I’d like to do is:
(def chicken (repo/q
                 *node*
                 `{:find  [(eql/project ?el [:food-item/name { :food-item/nutrients [:nutrient-type/id
                                                                                     :nutrient/amount]}])]
                   :where [[?el :food-item/id]]}))
Or will I have to create and look for “reverse direction” relation here?

refset22:01:42

Something approximately like that should be possible, yep, although the elements in that vector must be submitted as documents also (i.e. the projection can navigate into values) This example might help:

(let [node (crux/start-node {})]
  (crux/submit-tx node [[:crux.tx/put {:crux.db/id :a
                                       :test [{:x {:z 3}}
                                              {:y 2}]}]])
  (crux/submit-tx node [[:crux.tx/put {:crux.db/id {:x {:z 3}}
                                       :hello 123}]])
  (crux/submit-tx node [[:crux.tx/put {:crux.db/id {:y 2}
                                       :hello 456}]])
  (crux/submit-tx node [[:crux.tx/put {:crux.db/id 123
                                       :again 123123}]])
  (crux/submit-tx node [[:crux.tx/put {:crux.db/id 456
                                       :again 456456}]])
  (crux/sync node)
  (crux/q (crux/db node)
          '{:find [(eql/project e [{:test [{:hello [:again]}]}])]
            :where [[e :crux.db/id :a]]})
  )
;;=> #{[{:test [{:hello {:again 123123}} {:hello {:again 456456}}]}]}
If you can share more context on what the actual documents getting submitted look like I can probably give more specific pointers, but I'm currently unclear on what the :crux.db/id values are in your snippets

Aleksander Rendtslev19:01:59

Hi @U899JBRPF, that and changing ` to ' did most of the trick! Thank you! I’m stuck on the last step now though. (I’m using malli to validate my schemas, so this is so you can understand the structure)

(def NutrientType
  [:map
   [:nutrient-type/id {:id true} uuid?]
   [:nutrient-type/unit-name string?]
   [:nutrient-type/nickname {:optional? true} string?]
   [:nutrient-type/name string?]])

(def Nutrient
  [:map
   [:nutrient/id {:id true} [:map
                             [:food-item/id uuid?
                              :nutrient-type/id uuid?]]]
   [:nutrient/amount integer?]])

(def FoodItem
  [:map
   [:food-item/id {:id true} uuid?]
   [:food-item/name string?]
   [:food-item/nutrients
    [:vector [:map [:nutrient-type/id uuid?
                    :food-item/id uuid?]]]]])
and here’s my query:
(def bacon (repo/q
               *node*
               '{:find  [(eql/project ?el
                                      [:food-item/name
                                       {:food-item/nutrients
                                        [:nutrient/amount]}])]
                 :where [[?el :food-item/name "Bacon"]]}))
And here’s the result:
#{[#:food-item{:name "Bacon", :nutrients [#:nutrient{:amount 150}]}]}
Question: Can I get it to join :nutrient-type/unit-name somehow?

refset10:02:10

Thanks for the schema, that definitely helps 🙂 I think the issue is that the map values of :nutrient/id are essentially opaque to the Crux main query engine, and by extension eql/project. As far as I can see the options are: 1. do the join as a post-process in code 2. duplicate the information stored under :nutrient/id as extra top-level keys, e.g. with keys like :nutrient.id.foot-item/id and :nutrient.id.nutrient-type/id, and then you can do the join in the eql (there is a minor impact on storage cost...) 3. handle join(s) in Datalog and use a Clojure predicate to get the :nutrient-type/unit-name value out of that map (this is the elegant approach, and perhaps less efficient, but should also work)

refset15:02:08

Oh, Option 4 is: consider the possibility of splitting the relationship out into a second many-to-many document so that you can then join directly

Aleksander Rendtslev20:02:01

What I’m doing right now:

{:find  [(eql/project
               ?el
               [:food-item/id
                :food-item/name
                {(:nutrient/_food-item {:as   :food-item/nutrients
                                        :into []})
                 [:nutrient/amount
                  {(:nutrient/nutrient-type
                    {:as :nutrient/type}) [(:nutrient-type/unit-name {:as :unit-name})
                                           (:nutrient-type/nickname {:as :nickname})
                                           (:nutrient-type/name {:as :name})]}]}])]
      :in    [input]
      :limit 10
      :where [[(text-search :food-item/name input) [[?el]]]
              [?el :food-item/id]]}
with data model:
(def NutrientType
  [:map
   [:nutrient-type/id {:id true} uuid?]
   [:nutrient-type/unit-name string?]
   [:nutrient-type/nickname {:optional true} string?]
   [:nutrient-type/name string?]])

(def Nutrient
  [:map
   [:nutrient/id {:id true} uuid?]
   [:nutrient/food-item uuid?]
   [:nutrient/nutrient-type uuid?]
   [:nutrient/amount integer?]])

(def FoodItem
  [:map
   [:food-item/id {:id true} uuid?]
   [:food-item/name string?]
   ])
That works. But are there any significant performance implications of doing that vs what you’re suggesting? And what would that look like?

refset12:02:57

Sorry for the delayed response 😅 I think what you wrote looks good to me, assuming it really does what you want. I can't see any obvious reasons why the performance shouldn't be ~ideal, and the suggestions I wrote are not so relevant with your new model. Child->Parent relationships (i.e. reference attributes) are usually the right way to go :thumbsup:

Aleksander Rendtslev15:02:30

Sweet. I appreciate the feedback. It feels a lot more natural too. Nutrient is an edge (with data) connecting two nodes

🙂 1