Clojurians Log v2

the source of take is pretty authoritative i think?

and the helper function it uses:

(defn ensure-reduced
  "If x is already reduced?, returns it, else returns (reduced x)"
  {:added "1.7"}
  [x]
  (if (reduced? x) x (reduced x)))

Can you even use reduced with sequence? Seems like you'd need to use transduce explicitly here in order to have it respect reduced?

so the issue here is map's transducer arity doesn't respect reduced? it calls (rf result (f input)) instead of something like (let [ret (f input))] (if (reduced? ret) (ensure-reduced (rf result (deref ret))) (rf result ret)))?

(i've not tested that, i'm just spitballing)

sequence respects take which i take to mean it understands reduced, and looking at the code, yes it respects reduced: https://github.com/clojure/clojure/blob/a3fa897590f70207eea3573759739810f2b6ab6c/src/jvm/clojure/lang/TransformerIterator.java#L81-L83

isn’t it reducing functions that are operative on the reduced wrapper?

transducers can mark a value as such, but it’s the rf that has to stop, not the xf

ah, so i'd have to use a reduced-respecting-conj instead of conj?

let me try that out

for instance, iteration turns paginated stuff into an ireduceinit and does the looping for you. and it needs to check for reduced and stop looping. it’s a source concern, not a processing concern

changing the impl of map works, so yes the issue is the reducing function, the rf (normally conj)

cool, thanks y'all

Can you share the solution?

It’s not the reducing function that needs to respect reduced, it’s the reducible process (eg reduce, transduce, sequence, etc)

that TransformerIterator code sure looks interesting to a layperson. I'm likely completely wrong here, but when it invokes iter = xf.invoke(null, sourceIter.next()); -> doesn't that end up calling the invoke function on line 47, which then returns the acc (= null) to the loop? feels like the reduced value just gets passed through the buffer without anyone noticing it's reduced.

(sequence 
 (take-while #(not (<= 10 %)))
 (range 20))

reduced is for wrapping the final reduced state, not individual elements.

I was curious what @nbtheduke did with "changing the map impl"

Right, yes, that would be my approach. Hence my curiosity about what Noah did

here's my changed map which respects reduced:

(let [map* (fn [f]
             (fn [rf]
               (fn
                 ([] (rf))
                 ([result] (rf result))
                 ([result input]
                  (let [ret (f input)]
                    (if (reduced? ret)
                      (ensure-reduced (rf result (deref ret)))
                      (rf result ret))))
                 ([result input & inputs]
                  (let [ret (apply f input inputs)]
                    (if (reduced? ret)
                      (ensure-reduced (rf result (deref ret)))
                      (rf result ret)))))))]
  (= [0 1 2 3 4 5 6 7 8 9 10]
     (sequence (map* (fn [n] (if (<= 10 n) (reduced n) n))) (range 20))))

basically what i said above

moving the reduced from a given individual element to the returned object

this can also be done by changing conj, but due to the way sequence is written (using a buffer in the java impl), you can't get the same semantics and must drop down to (transduce (map ...) conj* [] (range 20))

i get nervous if that might call the completing arity multiple times. that gets nasty and subtle

i asked because i've implemented sequence-padded for weavejester/medley and was trying to write various tests, and was surprised that my map call in the OP didn't work. i thought it was a bug in my impl but it's actually just the way these functions are written

none of the existing transducers use the convention of returning a reduced value for an individual element.

yeah, i see that now. that's surprising to me but now i know

good point. reduced is about acc, not x right?

(let [conj* (fn conj* ([] [])
             ([coll] coll)
             ([coll x]
              (if (reduced? x)
                (reduced (conj coll (deref x)))
                (conj coll x)))
             ([coll x & xs]
              (if xs
                (recur (conj* coll x) (first xs) (next xs))
                (conj* coll x))))]
  (transduce (map (fn [n] (if (<= 10 n) (reduced n) n))) conj* [] (range 20)))

can't pass conj* to sequence tho 😢

what's wrong with using map composed with take-while?

i was curious

the point about reduced being for acc, not x is good and one i hadn't fully understood

I also like halt-when for conveying the intent - one of those functions in core that's easy to forget about:

(sequence
  (halt-when #(<= 10 %))
  (range 20))

There's some semantic mismatch which should jump out here Reduced is for reducing. Not sequencing. I wouldn't expect sequence to respect a reduced acc because what even is the acc?

"reduced" is just a name given to the concept of early exit or "break" in other languages. as noted above, you can't support functions like take without it. the acc of a sequence is the lazy sequence, constructed with conj or cons or whatever.