Fork me on GitHub
#clojure-uk
<
2018-06-09
>
yogidevbear08:06:03

Anyone have anything planned for the weekend?

dominicm08:06:56

Badminton, and maybe a walk at cannock chase. Nothing else much.

dominicm08:06:05

How about yourself? 🙂

korny08:06:14

Packing to go down to Rye for the week

dominicm08:06:54

Where's rye? what's it like?

mccraigmccraig09:06:48

rye is a small south-coast seaside town, not far from hastings and romney marsh... think pleasant seaside walks, rustic pub lunches, beaches... a long time ago it was an important port iirc, but it's quite out of the way now

mccraigmccraig09:06:04

i often forget about reduce-kv and destructure instead: (reduce (fn [m [k v]] (assoc m k (do-stuff-to v)) {} m)

mccraigmccraig09:06:37

or since it's map-in map-out then maybe (into {} (for [[k v] m] [k (do-stuff-to v)]))

yogidevbear09:06:38

Rye sounds lovely. And badminton too. Haven't played it in years. We've taken the hounds for walks and now we're heading out to go strawberry picking 🍓

korny10:06:36

@mccraigmccraig yeah - it seems reduce-kv is functionally the same as destructuring - the implementation might be a lot simpler though as it probably doesn’t need to turn a map into a sequence of KV pairs? Not that that matters too much really. (and the into example is pretty well what I usually do, though I use map not for because for seems to encourage me to overly-procedural thinking 🙂 )

korny10:06:50

Re: Rye - yeah, it’s near Hastings and Camber Sands for beaches if it’s nice. We have Australian relatives to visit, and they wanted to see some of the coast - Beachy Head and the like - and Rye looked nice. (They obviously are sarcastic about English beaches though!)

korny10:06:22

(defprotocol IKVReduce
  "Protocol for concrete associative types that can reduce themselves
   via a function of key and val faster than first/next recursion over map
   entries. Called by clojure.core/reduce-kv, and has same
   semantics (just different arg order)."
  (kv-reduce [amap f init]))

thomas10:06:53

morning folks

mccraigmccraig10:06:14

@korny i often do pipelines like (->> m (map (fn [[k v]] [k (do-stuff-to v)])) (map (fn [[k v]] [k (do-other-stuff-to v)])) (into {})) too... but i find for clearer for single-step transformations

mccraigmccraig10:06:49

what dyu mean about "procedural thinking" though ?

korny11:06:33

Somehow when I type for I start writing code like I was in Java or C-ish languages. Purely psychological.

korny11:06:07

Ditto using for comprehensions in other languages

korny11:06:00

I do use for if I need to iterate over nested things, as it's much clearer.

danm14:06:58

I tend to be the same as @korny. If I start using functions that I'm more used to in non-functional languages (e.g. for loops instead of map/`reduce`) I find I drop into that mindset and start producing code that is more 'OO-like'. So I avoid using them

danm14:06:27

It's why I found Clojure so useful as a functional language. It's so different from anything else I write, being Lisp-y, that it's easy to make my brain go "Clojure = functional", whereas when I'm writing for e.g. Python even if I'm trying to do it in a functional style I find myself dropping into OO habits, and when I've (briefly) poked stuff like Scala I find the same

korny15:06:22

<heresy> I’d kind-of like to try Scala again, now, after quite a bit of clojure - I hated it when I used it last, but that was 8 years ago when my FP was largely theoretical. Maybe scala-as-if-it-were-clojure might be viable… </heresy>

korny19:06:16

So, clojure laziness question… I have some code that is sort-of like this:

(-> (reduce (fn [memo thing]
  (let [new-data (something done with thing)]
    {:log (conj (:log memo) new-data) :metadata (something irrelevant}
  {:log [] :metadata nil}
  iterator-over-stuff-from-elsewhere)
:log)

korny19:06:12

So the :metadata is used for internal state while the iterator-over-stuff-from-elsewhere is processed into the :log bit of the memo.

korny19:06:02

I throw away the :metadata and just use the :log - which I want to stream to a JSON output. But it doesn’t seem to do this lazily - I guess because I’m not returning anything from the reduce until it’s complete. Or something - I have no brain right now to think this through, which is why I’m asking you folks 🙂

korny19:06:29

Is there a simple way to make this into something streaming? I’m reducing over hundreds of thousands of rows, I’d prefer to send the output directly to an output stream, rather than accumulate it all in memory!

korny19:06:30

(of course it isn’t lazy because reduce has no way of knowing what the final :log element will be until it has finished reducing)

korny19:06:50

… or maybe I should rework this using loop/`recur` - I just like the way reduce operates 🙂

rickmoynihan20:06:24

reduce and loop/`recur` are all eager not lazy, so it won’t matter whether iterator-over-stuff-from-elsewhere is a lazy-seq or not

rickmoynihan20:06:27

also :log is initalised as a vector, which is again an eager data structure - and conj is polymorphic on that type

rickmoynihan20:06:45

so whilst (take 20 (conj (range) 10)) will work lazily, conj on a vector will return a vector too