This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-01-29
Channels
- # announcements (7)
- # babashka (4)
- # beginners (21)
- # calva (31)
- # cljdoc (12)
- # cljsrn (5)
- # clojure (89)
- # clojure-europe (26)
- # clojurescript (9)
- # conjure (1)
- # cursive (3)
- # data-science (20)
- # events (2)
- # fulcro (4)
- # gratitude (7)
- # introduce-yourself (1)
- # lsp (24)
- # nextjournal (3)
- # off-topic (5)
- # re-frame (22)
- # shadow-cljs (48)
- # tools-deps (11)
hi! In tablecloth, how to access the last N rows when calculating a new column? Eg how to modify the below code not to sum up the values from the current row, but sum the values from the past 2 rows?
(-> [[:a [1 2 3]] [:b [6 7 8]]]
(tablecloth.api/dataset)
(tablecloth.api/add-column :sum #(map + (:a %) (:b %)))
)
Is there some way to access the "current index" somehow and get access to the last N rows as dataset that is passed to my function?I don't really understand what you are asking, but you will have much more luck here: https://clojurians.zulipchat.com/#narrow/stream/151924-data-science/topic/tech.2Eml.2Edataset
Do you mean something like this?
(-> [[:a [1 2 3]] [:b [6 7 8]]]
tc/dataset
(tc/add-column
:sum (fn[ds]
(let [avals (->> :a ds (coll/sliding-take 2)
(mapv (fn[[x y]] [x (or y 0)])))
bvals (->> :b ds (coll/sliding-take 2)
(mapv (fn[[x y]] [x (or y 0)])))]
(mapv (fn [a b]
(->> b (concat a) vec (apply +)))
avals bvals)))))
=> _unnamed [3 3]:
| :a | :b | :sum |
|---:|---:|-----:|
| 1 | 6 | 16 |
| 2 | 7 | 20 |
| 3 | 8 | 11 |
Hmmm, can't figure out slack here - code formatting and such is also far better in Zulip than slack
@U06C63VL4 yeah I did something similar, I mean take all "a" and "b" values and do some regular sequence processing via clojure and add the column as a whole
however what I'm asking is, if there is any way of adding a column, that is more "row" oriented
so in case of this newly added column, the values are not given as a whole, but a function is taken, which gets the last N rows compared to the current row and the function calculates the current value based on the last N rows
That's what the code I posted does - for the last 2 rows
But really, you should be over on Zulip as there may be other ideas and options and that is where data science clojure hangs out - NOT on slack
Sliding take is in one of my utils libraries
Also - https://techascent.github.io/tech.ml.dataset/tech.v3.dataset.rolling.html#var-rolling
Right - this is cleaner and likely way faster
(-> [[:a [1 2 3]] [:b [6 7 8]]]
tc/dataset
(drl/rolling {:window-type :fixed
:window-size 2
:relative-window-position :right
:edge-mode :zero}
{:suma (drl/sum :a) :sumb (drl/sum :b)})
(tc/add-column
:sum (fn[ds]
(dsc/column-map
(fn[a b] (+ a b))
:long
(ds :suma) (ds :sumb)))))
@UDRJMEFSN Is there a way to pass multiple columns to a reducer? So, we would have simply {:sum (drl/sum :a :b)}
sort of thing.I haven't at this time inplemented a multi column reducer for rolling but I agree it is useful.
Has anyone used wav2vec with Clojure? I'm interested in making a simple program to transcribe audio clips but it looks like the world is Python-centric 🐍
Pretty sure your best bet is interop via https://github.com/clj-python/libpython-clj#vision