Fork me on GitHub
#clojure
<
2021-11-02
>
Shayden Martin01:11:45

Can someone help me understand why vary-meta does not throw an Unable to resolve symbol error when deftest is called? This is from the clojure.test source:

(defmacro deftest
  "[docstring omitted for brevity]"
  {:added "1.1"}
  [name & body]
  (when *load-tests*
    `(def ~(vary-meta name assoc :test `(fn [] ~@body))
          (fn [] (test-var (var ~name))))))
I would expect the symbol bound to name here to be unbound at the point vary-meta is called. But this seems to work just fine:
(deftest asdfasdf (is true true))

Shayden Martin01:11:59

I think I’ve figured it out and learnt something new about Clojure in the process. Here vary-meta is applied to the symbol bound to name directly, so there are no vars involved yet. It appears that when def is called, it checks the symbol passed as name for any metadata first and forwards it on to the var it creates. So the test is first passed as metadata to the symbol bound to name at macro expansion time, then passed on to the var at runtime when def creates it.

seancorfield04:11:35

@U02HVHE2APL Sort of. The macro expands to:

(def ^{:test (fn [] (is true true)} asdfasdf)
The ~(vary-meta ...) is evaluated at compile-time to produce a symbol with metadata attached. It's as if you ran (vary-meta 'asdfasdf ...) (because name is bound to the symbol asdfasdf).

seancorfield04:11:06

Figuring out macros can be a bit "Inception" at times but you have to remember that they are evaluated at "compile-time" -- so they run before the expanded result is evaluated (i.e., "runtime"). That means they can run any code but it runs on symbols and structures -- code -- not on values (which only exist at runtime).

seancorfield04:11:02

Hope that helps?

Shayden Martin04:11:20

Yep, thanks. I think the understanding I arrived at was more or less what you described, maybe I could have worded it better. I think the core of my confusion wasn’t the way macro expansion works, but that symbols can have metadata and that def forwards any metadata from the symbol to the new var. I was squinting really hard to try see where the var vary-meta is operating on could have come from, when the obvious answer after macro expansion was that it’s operating on the symbol.

Shayden Martin04:11:55

In fact, the way metadata reader macros work makes a lot more sense now too.

seancorfield04:11:09

Some of the interactions between syntax and code can be hard to connect around macros at first. It's kinda why the "first rule of macro club is: don't use macros" 🙂

Ben Sless15:11:30

Just realized you can extend the CollReduce protocol to CompletionStage then use transducers on them :shocked_face_with_exploding_head:

👍 4
2
Ed17:11:04

sounds like a blog post in the making 😉

😄 1
Ben Sless18:11:50

@U0P0TMEFJ if you can figure out how to correctly implement fold over Completion Stages let me know. I probably missed something there

Ben Sless20:11:03

Anyone really good with types, please chime in

Ed21:11:45

Sorry .. just making us tea ... Got code you can share and a specific problem? How far have you got?

hiredman21:11:11

the thing about doing it as a fold instead of a map is just fold expects to be combining 2 things

hiredman21:11:22

the tricky thing about using it with futures is deciding what executor to run it on

Ben Sless04:11:50

Let's put aside the question of executor for a second and focus on correctness. Does it make sense to "fold" over a functor? Even if the fold function is (fn [_ x] x)

opqdonut06:11:38

Not any functor, no. For example functions from Y to X are a functor (over X). Mapping is just composition, but to "fold", you'd have to enumerate values of Y (which might not be possible).

opqdonut06:11:45

Also, if you think about types, a map (fmap) would turn a CompletionStage<A> into a CompletionStage<B>, but a reduce (fold) would turn a CompletionStage<A> into a B. You can't do something like sum a CompletionStage<Int> and get an Int, you'd get a CompletionStage<Int>, right?

opqdonut06:11:18

(However you totally can sum a List<Int> into an Int or a HashMap<Key,Int> into an Int)

opqdonut06:11:29

of course Clojure won't care if you just

(extend-protocol CollReduce
  CompletionStage
  (coll-reduce
   ([coll f] coll)
   ([coll f val] (.thenApply coll #(f val %)))

opqdonut06:11:56

that'd be the correct coll-reduce implementation for a container of size 1, but with an added CollReduce wrapper around the result

opqdonut06:11:24

this is also the part where you kinda see how inconsistent Clojure's arity-2 reduce is: • for size 0, use (f) • for size 1, don't use f at all • for size >1, use (f x y)

opqdonut06:11:06

arguably, size 1 should use (f val (f)) to get the types right 🙂

Ben Sless12:11:03

That's about how I implemented it, too

opqdonut13:11:24

oops, I meant "an added CompletionStage wrapper around the result"

opqdonut13:11:49

so what's the cool code you can write once you have this CollReduce implementation defined?

Ben Sless13:11:23

No idea yet. It might let you smoothly transition from between box types, eg list of completion stages to list of results

Ben Sless19:11:53

@US1LTFF6D here are the experiments which prompted this question to begin with

(import 'java.util.concurrent.CompletableFuture)
  (import 'java.util.function.Function)
  (require 'clojure.core.protocols)

  (defn then
    ([^CompletableFuture cf f]
     (.thenApply cf (reify Function (apply [_ x] (f x)))))
    ([^CompletableFuture cf f v]
     (.thenApply cf (reify Function (apply [_ x] (f v x))))))

  (.get (then (CompletableFuture/completedFuture 1) inc))
  ;; => 2

  (extend-protocol clojure.core.protocols/CollReduce
    CompletableFuture
    (coll-reduce
      ([cf f val] (then cf f val))))

  (defn step
    ([] nil)
    ([^CompletableFuture x] (.get x))
    ([_ x] x))

  (transduce
   (comp
    (map inc)
    (map #(* % %)))
   step
   (CompletableFuture/completedFuture 1))
  ;; => 4

Ben Sless19:11:54

I think if step was configured to always wrap in a new CF then it would solve the executor issue, would need to use thenCompose

Ben Sless19:11:01

Check this out

(import 'java.util.concurrent.CompletableFuture)
(import 'java.util.function.Function)
(import 'java.util.concurrent.Executor)
(require 'clojure.core.protocols)

(defn then
  ([^CompletableFuture cf f]
   (.thenApplyAsync cf (reify Function (apply [_ x] (f x)))))
  ([^CompletableFuture cf f ^Executor ex]
   (.thenApplyAsync cf (reify Function (apply [_ x] (f nil x))) ex)))

(.get (then (CompletableFuture/completedFuture 1) inc))
;; => 2

(extend-protocol clojure.core.protocols/CollReduce
  CompletableFuture
  (coll-reduce
    ([cf f val] (then cf f val))))

(defn step
  ([] (java.util.concurrent.Executors/newFixedThreadPool 4))
  ([^CompletableFuture x] (.get x))
  ([_ex x] x))

(defn sleepy [x] (println (Thread/currentThread) x) (Thread/sleep 1000) x)

(transduce
 (comp
  (map sleepy)
  (map inc)
  (map sleepy)
  (map #(* % %)))
 step
 (CompletableFuture/completedFuture 1))
;; => 4

Ben Sless19:11:46

It stays in the same pool

opqdonut06:11:49

Interesting, even this works (returns nil, only prints once):

(transduce
 (comp
  (map sleepy)
  (map inc)
  (filter odd?)
  (map sleepy)
  (map #(* % %)))
 step
 (CompletableFuture/completedFuture 1))

Ben Sless06:11:49

Why wouldn't it work? Now need to figure out cat

Andrés Rodríguez17:11:52

Is there a Clojure macro equivalent to Scheme's and-let*? https://srfi.schemers.org/srfi-2/srfi-2.html I saw someone posted something similar in the http://clojuredocs.org for when-let, but according to the comment it doesn't return early on first logical false.

p-himik17:11:06

when-let cannot return early because there's no "early" - it accepts just a single binding. Without writing your own macro, I guess the alternative is just using nested when and when-let.

nbardiuk17:11:31

if the multiple bindings are just passed to each other there are some-> and some->> that shirt circuit the thread when value is absent

Mno17:11:03

I was thinking of suggesting cond-> but some-> sounds closer.

Andrés Rodríguez17:11:32

@U2FRKM4TW Sorry, by "posted something similar" I meant someone posted a macro in the when-let docs that was similar to and-let*, but didn't do early return: https://clojuredocs.org/clojure.core/when-let#example-5b8df66ce4b00ac801ed9e85 @U076FM90B @UGFL22X0Q Thanks for suggesting some-> , it gets very close but I do need previous bindings available throughout subsequent expressions. Good to know about it though!

Andrés Rodríguez17:11:47

D'oh. I skipped over an implementation in that very same clojuredocs link that does perform early return 😂

kappa 1
rutledgepaulv18:11:33

(defmacro if-let*
  "Like clojure.core/if-let except supports multiple bindings."
  ([bindings then]
   `(if-let* ~bindings ~then nil))
  ([bindings then else]
   (if (seq bindings)
     `(if-let [~@(take 2 bindings)]
        (if-let* [~@(drop 2 bindings)] ~then ~else)
        ~else)
     then)))

(defmacro if-some*
  "Like clojure.core/if-some except supports multiple bindings."
  ([bindings then]
   `(if-some* ~bindings ~then nil))
  ([bindings then else]
   (if (seq bindings)
     `(if-some [~@(take 2 bindings)]
        (if-some* [~@(drop 2 bindings)] ~then ~else)
        ~else)
     then)))

(defmacro when-let*
  "Like clojure.core/when-let except supports multiple bindings."
  [bindings & body]
  `(if-let* ~bindings (do ~@body)))

(defmacro when-some*
  "Like clojure.core/when-some except supports multiple bindings."
  [bindings & body]
  `(if-some* ~bindings (do ~@body)))

👏 1