Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2019-10-22

Channels

#clojure-uk
<
2019-10-22
>
dharrigan06:10:43

Good Morning Everyone!

💥 4
dharrigan08:10:56

Anyone in this esteemed channel using Arch linux with encrypted partitions?

otfrom08:10:19

I would have thought the people at #juxt would be doing that as they all use arch

☝️ 4
otfrom08:10:47

what would people call the various rollups, summaries, etc that are derived from any central data science work? It isn't the core bit of data science, but it is all the stuff you need to do to make it understandable by end users. I've been calling those things "data products" but I'm wondering if there is a better term I should use.

Wes Hall10:10:56

Clojure company 🙂

Wes Hall10:10:13

Well, and python. Damn 'data scientists', but lots of clojure.

tcsavage10:10:02

Sadly yes. I've had to "import" a few things from Clojure I sorely missed in Python for my current project

Wes Hall10:10:53

Well... I did try to fight the good fight. Seems like you let them take over 😛. Just kidding of course. Congrats on the news today.

yogidevbear11:10:13

Don't Signal do a lot of Scala too though?

Wes Hall11:10:30

@U0MFQNXFZ They didn't when I was there, but maybe now I don't know.

tcsavage11:10:41

You might be thinking of a different Signal 🙂. We don't have any Scala in our stack

otfrom10:10:13

very useful for me too 😄

Wes Hall10:10:55

Does feel like a little bit of basic infrastructure would be useful (though obviously you can write this yourself), but how many opportunities do you want to give yourself to forget to check for a reduced value etc? I guess if you are using this kind of thing a lot though, a little factory function would definitely be advisable.

tcsavage11:10:25

A trap I've seen a few people fall into is creating a function like this: 

(defn read-lines
  [filename]
  (with-open [r (io/reader filename)]
    (line-seq r)))
Which the raises the inevitable question of somehow automatically closing the reader when the item in the lazy seq is evaluated. I'll have to try IReduceInit to see if it's a better way to do it.

reborg11:10:55

I’ve used something like this in the past: 

(let [br (BufferedReader. r)
        lines (line-seq br)]
    (transduce
      (comp 
        (map f))
      (completing conj! #(do (.close br) (persistent! %)))
      (transient [])
      lines))

Ben Hammond12:10:23

https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/into is a good fit for that, like 

(with-open [br (BufferedReader. r)]
  (into [] 
    (map f) 
    (line-seq br)))

Ben Hammond12:10:57

does the transient stuff behind the scenes

reborg12:10:40

:thumbsup:

reborg12:10:23

Guess you need the extended transduce form when you’re not doing conj! but your custom reduce fn

Wes Hall11:10:59

@tcsavage > somehow automatically closing the reader when the item in the lazy seq is evaluated ...or even when it's not 🙂

Rachel Westmacott11:10:09

if it isn't evaluated yet... how could you know if it is still needed?

Wes Hall11:10:25

I haven't fully internalised the idea from the blog post yet, but definitely like the, "don't bother opening the file unless you know what you are going to be doing with the data", vibe. The example you post (which I have fallen for many times), reminds me of my old Java days. "Give me the data and then get off my back about it!".

tcsavage11:10:19

That's a property I hadn't considered actually. The use case I had in mind was when the file is big and you don't want the whole thing in memory at once, which lazy-seq can help a lot with. Quite like the idea of only opening the file when you're sure you need it.

folcon11:10:39

Morning =)…

folcon11:10:38

😛 No actually, lunch pangs do that for me =)…