clojure-europe 2019-10-10

mogge mogge

morning

day 2 of getting the soffits and fascias done on our roof. Hopefully when they are done the wall at the front of the house won't leak

jasonbell07:10:41

Morning

Ben Hammond08:10:34

is your house engulfed by scaffolding?

Ben Hammond08:10:47

have they just got a ladder?

otfrom09:10:51

just ladders as we live in a bungalow

otfrom09:10:20

tho they have to go up reasonably high for the gable ends (the roof isn't pitched very steeply tho, so even then it isn't bad)

otfrom09:10:35

I've got some code where I need to take a number of sequences, do some mapping and filtering on them and then turn them into a map of results. Is dummy code example reasonable?

(transduce
 ,,, ; some more transforms
 ,,,, ; some reducing step function
 (sequence ,,, s1 s2 s3))

Ben Hammond13:10:55

transducers are really only set up to process single values so you have to figure out what a value is it might be easier to turn into a sequence of maps?

Ben Hammond13:10:25

(transduce
  (comp 
    (filter #(odd? (:seq1 %))) 
    (map :seq3)
    (map str))
  (completing string/join)
  ""
  (sequence (map #(zipmap [:seq1 :seq2 :seq3] %&))
    (range 10)
    (seq "abcdejghiklmop")
    [:one-banana :two-banana :three-banana :four :five-bananas :make-a-bunch]))

kinda thing?

Ben Hammond13:10:46

don't think sequence is buying you anything there though; might as well be

(map #(zipmap [:seq1 :seq2 :seq3] %&)
  (range 10)
  (seq "abcdejghiklmop")
  [:one-banana :two-banana :three-banana :four :five-bananas :make-a-bunch])

Ben Hammond13:10:44

that's always the problem with sequence; its unique selling point is when someone has passed you a transducer as a parameter

otfrom15:10:40

I'm sort of hoping that using sequence would get rid of some temporary collections

otfrom15:10:47

but idk yet

Ben Hammond15:10:07

I think it might avoid chunking

Ben Hammond15:10:15

if that's desirable

Ben Hammond15:10:09

if you are desperate for perfomance then its worth considering reducibles

Ben Hammond15:10:40

it sort of depends on the performance of your input data

otfrom15:10:09

@ben.hammond agreed. I think I'm trying to see if there are smaller changes I can make atm

otfrom15:10:18

and if there is anything easy

Ben Hammond15:10:13

(sotto voce) reducibles are quite easy its just a loop/recur, sugared up to make it palatable for reduce

otfrom16:10:34

@ben.hammond where would be a good place to read up on that?

Ben Hammond20:10:16

hmm, its not actually all that well described

Ben Hammond20:10:17

http://insideclojure.org/2014/12/21/reducible-repeat/

Ben Hammond20:10:24

hints at it

Ben Hammond20:10:59

maybe its worth writing a blog post over

Ben Hammond20:10:34

essentially you are reifying clojure.lang.IReduceInit

Ben Hammond20:10:00

whilst worrying about respecting 'reduced' short circuits

Ben Hammond20:10:14

so you can write something like

(defn iterator-reducible
  "expresses an iterator through the medium of IReduceInit
   if first-val is nil it will be ignored"
  [first-val ^java.util.Iterator it]
  (reify IReduceInit
    (reduce [_ f start]
      (loop [acc (if first-val
                   (f start first-val)
                   start)]
        (if (or (reduced? acc)
                (not (.hasNext it)))
          (unreduced acc)
          (recur (f acc (.next it))))))))

Ben Hammond20:10:00

(which you can use to wrap any old java.util.Iterator to make it amenable to reduce or into) (wilfully ignoring iterator-seq for now)

Ben Hammond20:10:37

so you - sit in a loop - call (f acc v) if you wish to pass the value to the outside world - just skip that value if you dont - alway check (reduced? acc); this covers you for the sort of short-circuit that is used by (take 2) - just return acc when you are done

Ben Hammond20:10:18

essentially we are just doing

(loop [acc init]
  (if (.hasNext it)
    (recur (f acc (.next it)))
    acc))

Ben Hammond20:10:50

which can run with very low overheads

Ben Hammond20:10:23

avoid consuming memory in multiple chunked sequences, etc

Ben Hammond20:10:47

you can plug it into transduce/reduce/into as though it were a collection

Ben Hammond20:10:47

(transduce (filter even?) + 0 (iterator-reducible nil (.iterator (range 10))))
=> 20

Ben Hammond20:10:23

(into [] (filter even?) (iterator-reducible nil (.iterator (range 10))))

Ben Hammond20:10:46

you do have to worry about init value though

(reduce + 0 (iterator-reducible nil (.iterator (range 10))))
=> 45
(reduce + (iterator-reducible nil (.iterator (range 10))))
Execution error (ClassCastException) at dev/eval4729 (form-init9996294491577087656.clj:1).
dev$iterator_reducible$reify__4655 cannot be cast to clojure.lang.IReduce

Ben Hammond20:10:17

you could reify clojure.lang.IReduce of course

Ben Hammond09:10:23

#Also sent to the channel

@otfrom this is the sort of thing I was talking about https://juxt.pro/blog/posts/ontheflycollections-with-reducible.html

otfrom09:10:01

thx!

Ben Hammond09:10:16

oh it's a particularly shaggy dog

Ben Hammond09:10:33

that was a subliminal angle I only just realised

otfrom09:10:01