This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # announcements (21)
- # aws (7)
- # beginners (105)
- # berlin (1)
- # calva (14)
- # cider (20)
- # clj-kondo (62)
- # cljdoc (7)
- # cljsrn (1)
- # clojure (206)
- # clojure-dev (2)
- # clojure-europe (11)
- # clojure-france (2)
- # clojure-italy (2)
- # clojure-nl (1)
- # clojure-uk (34)
- # clojured (1)
- # clojurescript (52)
- # copenhagen-clojurians (2)
- # core-async (1)
- # cryogen (3)
- # cursive (36)
- # data-science (27)
- # datomic (48)
- # emacs (1)
- # events (1)
- # fulcro (27)
- # hoplon (51)
- # jobs-discuss (1)
- # leiningen (1)
- # nrepl (2)
- # off-topic (52)
- # pathom (43)
- # quil (10)
- # re-frame (11)
- # reitit (28)
- # remote-jobs (2)
- # shadow-cljs (36)
- # sql (12)
- # tools-deps (7)
- # vim (32)
- # xtdb (17)
what would people call the various rollups, summaries, etc that are derived from any central data science work? It isn't the core bit of data science, but it is all the stuff you need to do to make it understandable by end users. I've been calling those things "data products" but I'm wondering if there is a better term I should use.
tho perhaps data products is right: https://towardsdatascience.com/designing-data-products-b6b93edf3d23
Sadly yes. I've had to "import" a few things from Clojure I sorely missed in Python for my current project
Well... I did try to fight the good fight. Seems like you let them take over 😛. Just kidding of course. Congrats on the news today.
Don't Signal do a lot of Scala too though?
@U0MFQNXFZ They didn't when I was there, but maybe now I don't know.
Nice article @ben.hammond
Does feel like a little bit of basic infrastructure would be useful (though obviously you can write this yourself), but how many opportunities do you want to give yourself to forget to check for a reduced value etc? I guess if you are using this kind of thing a lot though, a little factory function would definitely be advisable.
A trap I've seen a few people fall into is creating a function like this:
Which the raises the inevitable question of somehow automatically closing the reader when the item in the lazy seq is evaluated. I'll have to try
(defn read-lines [filename] (with-open [r (io/reader filename)] (line-seq r)))
IReduceInitto see if it's a better way to do it.
I’ve used something like this in the past:
(let [br (BufferedReader. r) lines (line-seq br)] (transduce (comp (map f)) (completing conj! #(do (.close br) (persistent! %))) (transient ) lines))
https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/into is a good fit for that, like
(with-open [br (BufferedReader. r)] (into  (map f) (line-seq br)))
does the transient stuff behind the scenes
Guess you need the extended transduce form when you’re not doing conj! but your custom reduce fn
@tcsavage > somehow automatically closing the reader when the item in the lazy seq is evaluated ...or even when it's not 🙂
if it isn't evaluated yet... how could you know if it is still needed?
I haven't fully internalised the idea from the blog post yet, but definitely like the, "don't bother opening the file unless you know what you are going to be doing with the data", vibe. The example you post (which I have fallen for many times), reminds me of my old Java days. "Give me the data and then get off my back about it!".
That's a property I hadn't considered actually. The use case I had in mind was when the file is big and you don't want the whole thing in memory at once, which lazy-seq can help a lot with. Quite like the idea of only opening the file when you're sure you need it.