Clojurians Log v2

you can't use a shared intermediate attribute, because inside an entity every attribute is realized at most once, so you can't have 2 different values for the same key (`:kv-key` in this case)

there are 2 ways we can get the batch to work across different attributes: 1. using a dynamic resolvers 2. using a single resolver for all attributes and delimiting which attributes by plan inspection - this I think can work for your case, I'll get you an example

(I may take some hours to give the example, can write it after I finish work things today :))

here is an example, I'm assuming here that you store the private data in your kv-store using the ID for the user as part of the key, following this idea this is how you can make a batch that works both across multiple attributes and entities, in the example you will see the request for 2 attributes of 2 entities (4 keys to read from kv-store), and this makes a single call to the resolver to realized all of them at once:

(ns com.wsscode.pathom3.batch-shared-read-kv
  (:require [com.wsscode.misc.coll :as coll]
            [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.connect.planner :as pcp]
            [com.wsscode.pathom3.interface.eql :as p.eql]))

(def my-kv-store
  {[:user/id 1 :user/name-private]  "Secret Name"
   [:user/id 1 :user/email-private] ""
   [:user/id 2 :user/name-private]  "I'm a secret"
   [:user/id 2 :user/email-private] ""})

;; assuming you will have some efficient way to pull multiple keys at once from your
;; kv store, this is where the impl for it lives
(defn batch-attributes-read [kv-store keys]
  (select-keys kv-store keys))

(pco/defresolver kv-items [env inputs]
  {::pco/input  [:user/id]
   ::pco/output [:user/name-private
                 :user/email-private]
   ::pco/batch? true}
  (let [ids             (map :user/id inputs)
        requested-attrs (->> env ::pcp/node ::pcp/expects keys)
        kvs-keys        (for [id   ids
                              attr requested-attrs]
                          [:user/id id attr])
        kv-data         (batch-attributes-read my-kv-store kvs-keys)]
    ; check what you read from the kv store
    (tap> kv-data)

    ;; unpack keys
    (mapv
      (fn [id]
        (into
          {}
          (map
            (fn [attr]
              (coll/make-map-entry attr (get kv-data [:user/id id attr]))))
          requested-attrs))
      ids)))

(def env
  (-> {}
      (pci/register
        [kv-items])))

(comment
  (p.eql/process env {:users [{:user/id 1}
                              {:user/id 2}]}
                 [{:users
                   [:user/name-private
                    :user/email-private]}]))

note that you should add every possible key in the ::pco/output there, but it will only fetch the keys the user request (this is that the ::pcp/expects do, it filters what the engine is expecting this resolver to return according to the plan dependencies)

Ah nice. ::pcp/expects does the trick. Thank you so much for your advice :)

hello Hendrik, I think this is a great usage for Pathom, my suggestion is just to keep things explicit, for each private attribute, make a correspondent one that reads the value from kv-store, if you have to do it a lot, make a helper function that generates the resolver, so you can end up with things like:

(private-attr :user/name)

Thanks for your response. I think, that I go with the explicit way and probably combine it with Fulcro RAD to autogenerate the resolvers. I have 2 more questions: Given these resolvers:

{::pc/input  #{:user/id}
   ::pc/output [:user/email-private}

  {::pc/input  #{:user/name-private}
   ::pc/output [:user/email}

ah, sorry, I was assuming batch only for datomic, if you like to batch the kv reading, you should use :batch true, this can batch per attribute, but if you want to batch across multiple private attributes, then we need to design things a bit different

Would this solution with an intermediate alias work?

;add a aliases to map from all private attributes to an intermediate key  
{::pc/input  #{:user/email-private}
 ::pc/output [:kv-key]}
{::pc/input  #{:user/name-private}
 ::pc/output [:kv-key]}

;look up all kv-keys
{::pc/input  #{:kv-key}
 ::batch true
 ::pc/output [:kv-value]}

;add aliases from the kv-value to all attributes
{::pc/input  #{:kv-value}
 ::pc/output [:user/name]}
{::pc/input  #{:kv-value}
 ::pc/output [:user/email]}