Can someone help me out here? https://clojure.atlassian.net/browse/CMEMOIZE-25 says "Due to differences in locking etc, there are some subtle differences in semantics between the clojure.core/memoize function and core.memoize/memo function, and these should be explained in the README." -- which is verbatim what someone said to me about those two functions, but they did not provide any details about the "subtle differences" and I'm not sure what they meant (and, yes, I should have asked for clarification at the time but now I can't even remember who said it!).

Is this about core memoize potentially doing calculations multiple times (based on atoms, swap!), whereas with a locking based solution it would happen only once?

Could be. memoize closes over an atom but only evals the function call once, then swap!'s it into the atom as a plain value.

That is true, but multiple concurrent calls might still happen

I'm staring at the core.memoize and core.cache code -- the core.memoize/memo function uses the "basic" cache which is just a hash map, no eviction, wrapped in an atom so I'm not seeing the difference in semantics...

Both function do a check on the args collection being in the atom and then a get on the atom, else a swap! assoc to update the atom.

could it be about this locking? https://github.com/clojure/core.memoize/blob/134e463c47f40abec6f7ac4343d17d8291c62bbd/src/main/clojure/clojure/core/memoize.clj#L41

I'm having a hard time envisioning under what circumstances the semantics might be different -- using a basic cache -- since there's no eviction. That locking is actually on the anonymous function in through* which in turn calls (apply f args) for the original memoized f.

(so, yeah, whatever potential differences there might be are certainly subtle but I can't express what they are 🙂 )

Maybe search the archives for this JIRA issue number? ;)

Haha... I've tried that before and not turned anything up. Focusing on concurrent calls, I guess if two happen with memoize, f can be called twice -- after both threads check and do not find the cached value -- but for core.memoize/memo that isn't possible? Am I reading the d-lay code right in that respect?

That's what I was thinking of

I suppose this should be easy enough to verify with a little bit of test code...

user=> (defn foo* [x] (prn x) x)
#'user/foo*
user=> (def foo (memoize foo*))
#'user/foo
user=> (run! (fn [i] (future (Thread/sleep 10) (foo :bar))) (range 10))
nil
user=> :bar:bar:bar
:bar

:bar:bar

:bar
:bar
:bar
:bar

I cannot reproduce this with core.memoize

user=> (def p1 (promise))
#'user/p1
user=> (def p2 (promise))
#'user/p2
user=> (def a1 (atom 0))
#'user/a1
user=> (def a2 (atom 0))
#'user/a2
user=> (defn f1 [& _] (swap! a1 inc) _)
#'user/f1
user=> (defn f2 [& _] (swap! a2 inc) _)
#'user/f2
user=> (let [ff1 (memoize f1)] (dotimes [n 100] (future @p1 (ff1 1 2 3 4))))
nil
user=> (deliver p1 :go)
#object[clojure.core$promise$reify__8526 0x5d1e09bc {:status :ready, :val :go}]
;; wait a while
user=> @a1
3
user=> (require '[clojure.core.memoize :refer [memo]])
nil
user=> (let [ff2 (memo f2)] (dotimes [n 100] (future @p2 (ff2 1 2 3 4))))
nil
user=> (deliver p2 :go)
#object[clojure.core$promise$reify__8526 0x6c008c24 {:status :ready, :val :go}]
;; wait a while
user=> @a2
1
user=> 

So the "subtle" issue is that, unlike memoize, clojure.core.memoize/memo prevents multiple invocations of the function with a given collection of arguments.

Thanks, @borkdude