This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-02-17
Channels
- # announcements (12)
- # babashka (27)
- # beginners (65)
- # biff (8)
- # calva (22)
- # clj-kondo (1)
- # clj-otel (5)
- # clojure (65)
- # clojure-europe (127)
- # clojure-nl (1)
- # clojure-norway (11)
- # clojure-portugal (2)
- # clojure-uk (2)
- # clojurescript (18)
- # cursive (5)
- # data-science (3)
- # datahike (14)
- # datascript (3)
- # datomic (7)
- # deps-new (11)
- # emacs (31)
- # exercism (1)
- # fulcro (1)
- # honeysql (3)
- # hyperfiddle (38)
- # introduce-yourself (4)
- # leiningen (2)
- # malli (20)
- # meander (2)
- # missionary (3)
- # off-topic (4)
- # pathom (3)
- # practicalli (2)
- # reagent (5)
- # releases (1)
- # sci (1)
- # shadow-cljs (9)
- # xtdb (8)
Is it correct that http://clojuredocs.org doesn’t have the 1.11 introduced parse-x
family of functions?
I think this is community maintained, no?
Ah okay, I’m answering my own question: https://github.com/zk/clojuredocs/pull/246
The comments are community maintained but the site is still run by a single person who has sadly been absent
The official Clojure API docs are at http://clojure.github.io/clojure/
Can anyone 1) reproduce this error:
(dorun (clojure.math.combinatorics/partitions [1 1 2 3 4 5] :min 5 :max 5))
Execution error (IndexOutOfBoundsException) at clojure.math.combinatorics/m5$fn (combinatorics.cljc:859).
2) explain it?
I assume it's a bug but the code is so inscrutable it's hard to be sure what's going on.
I can't seem to reproduce it without duplicate items, or with :min
and :max
not equal, but I haven't tested too much here.
Using dorun
to avoid spamming my repl, but it does occur without.
I've reproduced it with lots of other groups of numbers, but haven't noticed a pattern yet.Seems like you're trying to realize a sequence that is too large.
You can also reproduce it like this:
(count (combo/partitions [1 1 2 3 4 5] :min 5 :max 5))
i tried to read the source code... but its way above my paygrade so I can't help you there 😄
I think a function with an args list like this:
[n m f c u v a b l r s]
is paygrade-agnostic. Looks a bit write-only to me 😄As this has been replicated, and I think I understand the conditions for throwing an error a little better, I've made a post on ask clojure: https://ask.clojure.org/index.php/12673/clojure-combinatorics-partitions-throw-error-duplicate-items
Cc @U0516PHE3 , the author
@UE1N3HAJH you definitely found a bug! it’s broken right at the start, before the error, when it generates
([1 4] [1] [2 3] [2 3] [2 3])
as its fourth optionsince that’s no longer a partition
woah, I think I’ve got it
@UE1N3HAJH phew, okay, I’d love to know if this fixes the issues that you’ve seen with other groups of numbers!
Thanks @U017QJZ9M7W - great work! All looks good to me, as commented on the ticket. I hope it gets merged soon 🎉
Maybe worth pinging @U0518RMLD instead as I think he's actually the author.
Proper fix released, my old patch wasn’t great, it generated all small partitions then filtered them so performance was poor
I have sorted pairs of index and predicate, and a seq of unknown size. I want to call the predicate on the value of the seq at the paired index and early exit on the first failure. Is nth
my best bet or is there a faster/better way of stepping through a seq to arbitrary points?
If it's performance that you're worrying about, it should be possible to write your own "caching nth
" that uses a transient vector as an indexed cache.
If you want readability, nth
along with every?
are probably the best bet here. Although, every?
helps in the above case as well.
Oh, but since the predicate collection is sorted by index, I'd probably go with loop
.
My worry was that without vectors Indexed, nth would start from 0 every time. So as the indexes get bigger, the O(n) traversal would grow too. Would it be worth using nthnext or something and calculating the distance from index to index?
nth
would indeed start from the very start every time. nthnext
is no different - it just returns the tail and not the item at that position.
You can use multiple calls to drop
in e.g. a reduce
but you'd have to keep track of the current index.
Something like this, although there are probably more clever solutions:
(def index+pred [[1 int?]
[5 pos?]
[18 #(> % 3)]
[200 zero?]])
(def coll (range))
(def result (boolean (reduce (fn [[coll curr-idx] [idx pred]]
(let [n (- idx curr-idx)
coll (drop n coll)
item (first coll)]
(if (pred item)
[coll idx]
(reduced false))))
[coll 0]
index+pred)))
What would be really neat is the ability to project a sequence over a sequence of indices (sort of like in MATLAB)
Ah yeah, indeed. Something like this:
(defn project [indices coll]
(letfn [(step [curr-idx indices coll]
(lazy-seq
(when-let [indices (seq indices)]
(when-let [coll (seq coll)]
(let [idx (first indices)
n (- idx curr-idx)
coll (drop n coll)
item (first coll)]
(cons item (step idx (next indices) coll)))))))]
(step 0 indices coll)))
Yes, although this assumes the input is sorted and unique, but it's a good fit for sequences
I am looking for an algorithm where I can use a lambda to do rolling computation, for example, rolling average would look like this:
(rolling #(/ (reduce + %) (count %)) array roll_length)
to calculate the rolling average and of course, I would be able to change the function.I wrote some code recently which takes an offset
rather than a roll-length
, so it supplies each item, along with the items n in front of, or behind it, depending on if offset
is positive or negative:
(defn partition-offset
([offset coll]
(if (neg? offset)
(let [abs-offset (abs offset)]
(concat (map #(take % coll) (range 1 abs-offset))
(partition abs-offset 1 coll)))
(partition-all offset 1 coll))))
Note that if you just care about a forward-facing window, that last line is sufficient. I also have the transducer code kicking around if that's helpful. This may be in a lib somewhere too. We can't be the first two people to need this 🙂For clarity, your code would then look like this:
(->> coll
(partition-offset -3)
(map #(/ (reduce + %) (count %))))
Just in case - averages in particular could be made much more performant if you don't use something generic for it because (count %)
is always the same and (reduce + %)
can be replaced with a single -
followed by a +
.
For this function, the count isn't always the same, because partitions "off the edge" of the collection are shorter, rather than being padded with nil
s or 0
s. Probably a good argument for allowing padding :thinking_face:
Ah, right. But that can also be taken into consideration, whether with or without a padding - depending on how you roll [the windows]. :)
partition
allows for creating windows like this
(let [average (fn [nums]
(/ (apply + nums) (count nums)))]
(->> (range 30)
(partition 3 1)
(map average)))
which will miss off the extra unpadded elements. You could also make a transducer that creates windows from a sequence, maybe something like
(defn window [n]
(fn [rf]
(let [win (volatile! [])]
(fn
([] (rf))
([r]
(rf (reduce rf r (take-while not-empty (iterate (partial into [] (drop 1)) @win)))))
([r x]
(rf r (vswap! win (fn [w]
(let [w' (conj w x)]
(if (-> w' count (> n))
(subvec w' 1)
w'))))))))))
(comment
(let [average (fn [nums]
(/ (apply + nums) (count nums)))]
(sequence (comp (window 3) (map average)) (range 10))) ;; => (0 1/2 1 2 3 4 5 6 7 8 8 17/2 9)
)
or you could reach for Christophe Grand's xforms lib (https://github.com/cgrand/xforms) which has a more flexible implementation of partition
and a window
that might do what you want (and I think can be made to do what @U2FRKM4TW was talking about with performance)The most efficient thing here would be an exponential moving average
That particular algorithm processes 1 item at a time and doesn’t need to access windows of the data
another nice one is an “exponential histogram”, which can give you an approximate rolling-window average with a guaranteed error bound with much lower memory than keeping the full window in memory
(here’s a safer average implementation than the one above, btw https://github.com/sicmutils/sicmutils/pull/454/files#diff-d25b73310b8c6a6ba432519e6a68ee73fc9a77909c007679b9f2d57b3e5af949R35-R66)
exponential histogram code in Scala, from back in the day: https://github.com/twitter/algebird/blob/develop/algebird-core/src/main/scala/com/twitter/algebird/ExpHist.scala#L7
“decayed value”: https://github.com/twitter/algebird/blob/develop/algebird-core/src/main/scala/com/twitter/algebird/DecayedValue.scala#L17
definitely nice to skip the window, and you can use reductions
to track the averaged value as you add in items, of course
Sorry, I probably missed the point of the actual question. I thought he was more asking about how to do rolling window type things than actually calculating an average.
@U0P0TMEFJ, actually, yes, you are absolutely right, I am about the rolling window type. I am using the code you recomended.
(let [average (fn [nums]
(/ (apply + nums) (count nums)))]
(->> (range 30)
(partition 3 1)
(map average)))
In this case @U012GN57FJ9 (count nums) is always 3!