pathom

2023-06-05T18:12:59.583959Z

I searched for cache in this channel, but didn't see anything relevant to eviction. what is the general strategy to do this? should i use pathom wrappers (plugin?), should i update my smart-map env, or should i just mutate my cache (atom/voli) externally? (ei: some function changes the atom without knowing about the smart-map)

wilkerlucio 2023-06-05T18:52:28.701719Z

hello, there are different ways depending on how you are using the cache, but basically its by mutating it (if its an atom), there is an example using core cache and evicting via LRU here: https://pathom3.wsscode.com/docs/cache#using-corecache

2023-06-05T20:43:43.007459Z

i read the docs. I need my cache eviction to be based on an external trigger. something in my system changed, and keys in the cache are now dirty, i want them evicted. for this specific purpose, i prefer to use a basic cache.

wilkerlucio 2023-06-05T20:44:26.392939Z

sure, in this case you should be able to do it by mutating the cache, let me know if you have any issues to do it, glad to assist on it

2023-06-05T20:46:04.810509Z

i actually have tried this, and it doesn't seem to work. when i reset! my cache atom the smart-map doesn't respect the new state of the cache. i'll post a code snippet

2023-06-05T20:47:15.367639Z

(defn init [_]
  {:resource/example {}})

(pco/defresolver resource-resolver []
  {::pco/cache-store ::my-cache*}
  {::resource (init config)})

(def resource-resolver-cache (atom {}))

(def env
  (-> (pci/register resource-resolver)
      (assoc ::my-cache* resource-resolver-cache)))

(def generated-templates
  (->
    (pci/register [resource-resolver])
    (psm/with-error-mode ::psm/error-mode-loud)
    (psm/smart-map)
    (psm/sm-touch! [::resource])
    ))

(->> generated-templates ::resource :resource/example)
;; => {}

(reset! resource-resolver-cache nil)
;; => nil

(->> generated-templates ::resource :resource/example)
;; => {}

2023-06-05T20:47:49.948749Z

i expect the last line of code to return nil, or throw

2023-06-05T20:52:20.864979Z

maybe i shouldn't be using smart-maps for this? i'm very new to pathom.

2023-06-05T20:53:21.012159Z

:com.wsscode.pathom3.connect.runner/resolver-cache* #<Atom@3e0dcc3c {[scratch/resource-resolver {} {}] {:scratch/resource {:resource/example {}}}}> i see this in the smart-map env . seems like it's dowing it's own thing with the cache

wilkerlucio 2023-06-05T20:53:21.408119Z

its possible to use, but its a weird place to be, because there is a bunch of mutable state involved

wilkerlucio 2023-06-05T20:53:43.171839Z

in general, I suggest you prefer to use the EQL interface, SM is great for small usages, or usages where a map interface is required

wilkerlucio 2023-06-05T20:53:51.130579Z

I updated your example in a way to make it work as a demo:

wilkerlucio 2023-06-05T20:53:52.889519Z

(ns com.wsscode.pathom3.demo-cache-sm
  (:require [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.connect.runner :as pcr]
            [com.wsscode.pathom3.entity-tree :as p.ent]
            [com.wsscode.pathom3.interface.smart-map :as psm]))

(def config {})

(defn init [_]
  {:resource/example {}})

(pco/defresolver resource-resolver []
  {::pco/cache-store ::my-cache*}
  {::resource (init config)})

(def resource-resolver-cache (atom {}))

(def env
  (-> (pci/register resource-resolver)
      (assoc ::my-cache* resource-resolver-cache)))

(def generated-templates
  (->
    (pci/register [resource-resolver])
    (psm/with-error-mode ::psm/error-mode-loud)
    (pcr/with-resolver-cache resource-resolver-cache)
    (psm/smart-map)
    (psm/sm-touch! [::resource])))

(comment
  (->> generated-templates ::resource)
  ;; => {}

  (do
    (swap! resource-resolver-cache assoc '[com.wsscode.pathom3.demo-cache-sm/resource-resolver {} {}]
           {::resource "some other value"})
    ; note that we need this to clean up the internal entity atom that's holding the current values for
    ; the smart map
    (psm/sm-update-env generated-templates p.ent/reset-entity! {}))

  (->> generated-templates ::resource))

wilkerlucio 2023-06-05T20:54:26.273599Z

the thing you are hitting is the entity cache, which is an atom the remembers the current state of the smart map

wilkerlucio 2023-06-05T20:54:50.340759Z

so after you clear your cache, when you asked the key, pathom was bringing from this entity cache, and not running any process at all (this is why your cache wasn't hit)

wilkerlucio 2023-06-05T20:55:08.634109Z

but even hitting it, just making it nil wont break it, since in this scenario pathom will just ignore the cache (but it will give you a warning log in this case)

wilkerlucio 2023-06-05T20:55:25.738259Z

so in my example I swapped the cache to put an actual value to demonstrate it

2023-06-05T20:56:17.220019Z

i choose smart-map cus i'm updating an interface in my system that used to be a map.

wilkerlucio 2023-06-05T20:57:38.724629Z

if you have a point where you can use the EQL and just forward the data down, that could work as well, what smart map gives you is the ability to make it lazy (so you process each attribute as they get requested)

wilkerlucio 2023-06-05T20:58:29.953659Z

its a tradeoff, if you can know ahead of time and request it, its faster than discretly getting every attribute (which is similar to using sm-touch! after starting the smart map)

2023-06-05T20:59:57.881139Z

ok. in this case i don't care about being lazy. in other parts of my app i guess i do, but i'll get to those later. i'll try the EQL approach.

2023-06-05T21:00:58.020189Z

:com.wsscode.pathom3.entity-tree/entity-tree* #<Atom@388a0d3f {:scratch/resource {:resource/example {}}}>, i guess that this entry in the smart-map env is the entity cache

wilkerlucio 2023-06-05T21:01:14.045129Z

correct

2023-06-05T21:17:51.475639Z

i was able to get your code working. how would i do a similar cache busting with the EQL style interface. would i just have to do (swap! resource-resolver-cache assoc '[com.wsscode.pathom3.demo-cache-sm/resource-resolver {} {}] {::resource "some other value"}) should i just alter-var-root!the resolver, or something else?

wilkerlucio 2023-06-05T21:19:19.225759Z

just swap the cache atom and you should be fine

wilkerlucio 2023-06-05T21:21:47.449409Z

because in eql processor you get fresh entities every time you run

2023-06-05T21:33:48.846629Z

ok, thanks

2023-06-05T21:47:16.686009Z

the code example you gave for smart-map cache busting seems like it works once, and then not again (i'm tracing the call to my init function, and it's printing 2 times only).

2023-06-05T22:06:42.331169Z

i was able to get EQL style working correctly, with my resolver init only running when the cache is reset. i think that the smart-map code may be missing something to achieve the same behaviour.