pic1 from https://www.geeksforgeeks.org/bfs-vs-dfs-binary-tree/

You can use loop for graph2 by explicitly calling the same function in the loop. Also, there's nothing wrong with using an explicit stack in a loop. Conceptually it's not different from recursion.

@U2FRKM4TW thanks for your reply! do you mean something like this:

(defn func [...]
  (loop [...]
    (for [...]
      (func [...])) ;; probably bind this `for` to some symbol ...
    (recur [...])))

I'll use this opportunity to mention #C053AK3F9. ;) That's not what I meant. for is lazy - the code will not actually call func. I meant something more similar to

(defn traverse [graph root]
  (loop [children (graph root) ;; Maps are callable.
         result [root]]
    (if-let [[child & children] (seq children)]
      (recur children
             (into result (traverse graph child)))
      result)))

(defn traverse2 [graph root]
  (into [root]
        (mapcat (fn [child]
                  (traverse2 graph child)))
        (graph root)))

thanks for your help!

Somewhat related, if I want to do dfs in actual code, I usually reach for tree-seq .

(tree-seq graph graph 0) ;=> (0 1 4 5 2 6 3 7)

this is my take after reading: https://github.com/clojure/clojure/blob/clojure-1.11.1/src/clj/clojure/core.clj#L4956

(defn dfs [graph root]
    (let [children (graph root)]
      (cons root (mapcat #(dfs graph %) children))))

Very Nice. I haven't looked carefully, but doesn't that actually build stack? Look at the lazy-seq in tree-seq.

I don't think this is similar to what people usually do when using stack to dfs walk a tree using stack will involve push/pop to a vec or something similar.

Stefan meant the call stack. Unless recur or lazy-seq is used somewhere (the latter can be used by something internally) and in the right manner, recursion will be increasing the call stack size, eventually leading to a stack overflow error.

tree-seq uses lazy-seq internally and in the right way, so it won't lead to that error.

gotcha! thank for your clarification!

i think other langs like cpp or python use this style (building call stack) when doing dfs (if using recursive style instead of pushin/poping to stack data structure)

Yes I mean the call stack. Many of the sequence functions in clojure.core are learning examples. Avoid the ones that mention chunked-seq. Perhaps take-while

Clojure in most places only supports double and long primitives. loop is one of such places, loop variables cannot be ints.

https://ask.clojure.org/index.php/4612/loop-should-retain-primitive-int-or-float-without-widening

By "persists to disk" do you mean it remains if you shut down the application and re-run it would still hit a cache next time it runs?

Exactly!

I could just dump the atom to disk

Since I don't exactly know what kind of data we are dealing with, you could do that but you can also use https://github.com/clojure/core.cache to have more robust options than just memoize

Yeah

I might need to try that

You could setup a shutdown hook to write to disk before the VM is killed

Ideally I’d want something quick and hacky, since I’m just doing some ad-hoc LLM work.

Oh wow, I’ve never heard of shutdown hooks, that sounds perfect!

If it is hacky, spit and slurp will be fine and just replace the atom that would be empty upon start

Yeah, that’s probably good enough.

I’ll try replacing the memoize atom with a https://github.com/jimpil/duratom.

interesting, never used duratom before

As for the most hacky solution

user> (require '[clojure.core.cache.wrapped :as w])
nil                                                
user> (let [data-atom (w/ttl-cache-factory {})]    
        (swap! data-atom (fn [a]                   
                           (assoc a                
                                  :key             
                                  "value")))       
        (spit "foo.txt" @data-atom))     ;; Do this during the shutdown hook          
nil                                                
user> (read-string (slurp "foo.txt"))              
{:key "value"}                                     

This does not fit your requirement for replacing memoize in place unfortunately

Cool!

A slight improvement of the above - swapping and deref'ing in two separate instructions creates a race condition, but swap! returns the swapped-in value so you can (->> (swap! ...) (spit ...)).

The alternatives already mentioned seem viable, but fwiw when I had the same requirement I wrote a couple lines on top of https://github.com/jackrusher/spicerack and it was very nice. Not quite "drop-in replacement" but very very close.

Here’s what I ended up with:

(defn durmemoize
  "duratom + memoize = ❤️"
  {:added "1.0"
   :static true}
  [f & args]
  (let [mem (apply duratom args)]
    (fn [& args]
      (if-let [e (find @mem args)]
        (val e)
        (let [ret (apply f args)]
          (swap! mem assoc args ret)
          ret)))))

(defonce quick-embed
  (durmemoize
   (fn [text]
     (-> (cai/create-embedding {:model "text-embedding-3-small"
                                :input text})
         (get-in [:data 0 :embedding])))
   :local-file
   :file-path "quick-embed.duratom"))

And to extract the map from the original memoize:

@(let [field (.getDeclaredField (class quick-embed) "mem")]
          (.setAccessible field true)
          (.get field quick-embed))

It’s amazing that I can install a library, surgically extract state from a closure and switch to a durable memoization all at runtime. Really happy about not losing my memoization state :^).

You shouldn't use .getDeclaredField - that's an implementation detail that mem ends up being a field on the class. A better way would be to add metadata to the returned function.

@U2FRKM4TW Converting to a durable memoization is a one-time change so it’s fine :^) How else would I extract the memoization data from a memoized function at runtime?

(memoize fn) doesn’t expose its state so I don’t think there’s any way around it.

> How else would I extract the memoization data from a memoized function at runtime? As I said - via function metadata. Here's how core.memoize does it:

(defn- cached-function
  "..."
  [f cache-atom ckey-fn]
  (with-meta
   (fn [& args]
     ...)
   {::cache cache-atom
    ::original f}))

That’s great! In my case the problem was that there was state inside of a live memoized function that I wanted to extract. The fact that if I had memoized it differently I could’ve retrieved it via metadata, doesn’t really help me with that.

I don't understand what you mean. Here's how the code above that you ended up with can be modified to use metadata:

(defn durmemoize
  "duratom + memoize = ❤️"
  {:added  "1.0"
   :static true}
  [f & args]
  (let [mem (apply duratom args)]
    (with-meta
      (fn [& args]
        (if-let [e (find @mem args)]
          (val e)
          (let [ret (apply f args)]
            (swap! mem assoc args ret)
            ret)))
      {:mem mem})))

(def quick-embed (durmemoize unmemoized-quick-embed ...)

@(:mem (meta quick-embed))

Could you try (set! *warn-on-reflection* false) without the fully qualified name?

That set! works only at the namespace level. In this case, you cannot override it at all because it's used at the source level: https://github.com/noprompt/meander/blob/epsilon/src/meander/util/epsilon.cljc#L12 You should create an issue for Meander to get the warning fixed.

Or maybe you're using an old version of Meander: https://github.com/noprompt/meander/issues/230

If, as the second comment there states, there has been no new release since the fix, you can probably use the Git coordinate.

@U03QTHYKXK7 thank you, that works!

Hmm, I wouldn't have expected for that to work to be honest. But I've been so disenchanted by user.clj so I've probably forgotten how it works.

Just in case - have you restarted the REPL after making the change and before checking? Even if you have that set! in user.clj, the set! in meander.util.epsilon should override it.

@U2FRKM4TW thank you! have’t know about

That set! works only at the namespace level. 

actually yes:face_palm: after reload meander its spaming with warn again

*warn-on-reflection* and clojure.core/*warn-on-reflection* are exactly the same. :) The former gets resolved to the latter.

@U2FRKM4TW do you know any other way how to off reflections warn?

As I mentioned, by using the Git coordinate of Meander. That's what I would do.

Do you mean use latest from master?

Yep.

Maybe not the latest, but after that fix mentioned in the issue. Up to you, I haven't checked what the actual most recent commits are.

binding-convey-fn is not safe to use on a threadpool that might mix binding conveying work and non-conveying work

binding-conveyor-fn just blindly splats bindings in and doesn't remove them when done

Ah, good to know! Just found this too, which says binding-conveyer-fn may lead to memory leaks too. Good to know! https://clojurians.slack.com/archives/C06E3HYPR/p1584642408134800 Thanks @hiredman