Fork me on GitHub
#pathom
<
2023-06-05
>
pppaul18:06:59

I searched for cache in this channel, but didn't see anything relevant to eviction. what is the general strategy to do this? should i use pathom wrappers (plugin?), should i update my smart-map env, or should i just mutate my cache (atom/voli) externally? (ei: some function changes the atom without knowing about the smart-map)

wilkerlucio18:06:28

hello, there are different ways depending on how you are using the cache, but basically its by mutating it (if its an atom), there is an example using core cache and evicting via LRU here: https://pathom3.wsscode.com/docs/cache#using-corecache

pppaul20:06:43

i read the docs. I need my cache eviction to be based on an external trigger. something in my system changed, and keys in the cache are now dirty, i want them evicted. for this specific purpose, i prefer to use a basic cache.

wilkerlucio20:06:26

sure, in this case you should be able to do it by mutating the cache, let me know if you have any issues to do it, glad to assist on it

pppaul20:06:04

i actually have tried this, and it doesn't seem to work. when i reset! my cache atom the smart-map doesn't respect the new state of the cache. i'll post a code snippet

pppaul20:06:15

(defn init [_]
  {:resource/example {}})

(pco/defresolver resource-resolver []
  {::pco/cache-store ::my-cache*}
  {::resource (init config)})

(def resource-resolver-cache (atom {}))

(def env
  (-> (pci/register resource-resolver)
      (assoc ::my-cache* resource-resolver-cache)))

(def generated-templates
  (->
    (pci/register [resource-resolver])
    (psm/with-error-mode ::psm/error-mode-loud)
    (psm/smart-map)
    (psm/sm-touch! [::resource])
    ))

(->> generated-templates ::resource :resource/example)
;; => {}

(reset! resource-resolver-cache nil)
;; => nil

(->> generated-templates ::resource :resource/example)
;; => {}

pppaul20:06:49

i expect the last line of code to return nil, or throw

pppaul20:06:20

maybe i shouldn't be using smart-maps for this? i'm very new to pathom.

pppaul20:06:21

:com.wsscode.pathom3.connect.runner/resolver-cache* #<Atom@3e0dcc3c {[scratch/resource-resolver {} {}] {:scratch/resource {:resource/example {}}}}> i see this in the smart-map env . seems like it's dowing it's own thing with the cache

wilkerlucio20:06:21

its possible to use, but its a weird place to be, because there is a bunch of mutable state involved

wilkerlucio20:06:43

in general, I suggest you prefer to use the EQL interface, SM is great for small usages, or usages where a map interface is required

wilkerlucio20:06:51

I updated your example in a way to make it work as a demo:

wilkerlucio20:06:52

(ns com.wsscode.pathom3.demo-cache-sm
  (:require [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.connect.runner :as pcr]
            [com.wsscode.pathom3.entity-tree :as p.ent]
            [com.wsscode.pathom3.interface.smart-map :as psm]))

(def config {})

(defn init [_]
  {:resource/example {}})

(pco/defresolver resource-resolver []
  {::pco/cache-store ::my-cache*}
  {::resource (init config)})

(def resource-resolver-cache (atom {}))

(def env
  (-> (pci/register resource-resolver)
      (assoc ::my-cache* resource-resolver-cache)))

(def generated-templates
  (->
    (pci/register [resource-resolver])
    (psm/with-error-mode ::psm/error-mode-loud)
    (pcr/with-resolver-cache resource-resolver-cache)
    (psm/smart-map)
    (psm/sm-touch! [::resource])))

(comment
  (->> generated-templates ::resource)
  ;; => {}

  (do
    (swap! resource-resolver-cache assoc '[com.wsscode.pathom3.demo-cache-sm/resource-resolver {} {}]
           {::resource "some other value"})
    ; note that we need this to clean up the internal entity atom that's holding the current values for
    ; the smart map
    (psm/sm-update-env generated-templates p.ent/reset-entity! {}))

  (->> generated-templates ::resource))

wilkerlucio20:06:26

the thing you are hitting is the entity cache, which is an atom the remembers the current state of the smart map

wilkerlucio20:06:50

so after you clear your cache, when you asked the key, pathom was bringing from this entity cache, and not running any process at all (this is why your cache wasn't hit)

wilkerlucio20:06:08

but even hitting it, just making it nil wont break it, since in this scenario pathom will just ignore the cache (but it will give you a warning log in this case)

wilkerlucio20:06:25

so in my example I swapped the cache to put an actual value to demonstrate it

pppaul20:06:17

i choose smart-map cus i'm updating an interface in my system that used to be a map.

wilkerlucio20:06:38

if you have a point where you can use the EQL and just forward the data down, that could work as well, what smart map gives you is the ability to make it lazy (so you process each attribute as they get requested)

wilkerlucio20:06:29

its a tradeoff, if you can know ahead of time and request it, its faster than discretly getting every attribute (which is similar to using sm-touch! after starting the smart map)

pppaul20:06:57

ok. in this case i don't care about being lazy. in other parts of my app i guess i do, but i'll get to those later. i'll try the EQL approach.

pppaul21:06:58

:com.wsscode.pathom3.entity-tree/entity-tree* #<Atom@388a0d3f {:scratch/resource {:resource/example {}}}>, i guess that this entry in the smart-map env is the entity cache

pppaul21:06:51

i was able to get your code working. how would i do a similar cache busting with the EQL style interface. would i just have to do (swap! resource-resolver-cache assoc '[com.wsscode.pathom3.demo-cache-sm/resource-resolver {} {}] {::resource "some other value"}) should i just alter-var-root!the resolver, or something else?

wilkerlucio21:06:19

just swap the cache atom and you should be fine

wilkerlucio21:06:47

because in eql processor you get fresh entities every time you run

pppaul21:06:48

ok, thanks

pppaul21:06:16

the code example you gave for smart-map cache busting seems like it works once, and then not again (i'm tracing the call to my init function, and it's printing 2 times only).

pppaul22:06:42

i was able to get EQL style working correctly, with my resolver init only running when the cache is reset. i think that the smart-map code may be missing something to achieve the same behaviour.