This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-08-21
Channels
- # admin-announcements (1)
- # beginners (3)
- # boot (242)
- # clara (1)
- # cljsrn (8)
- # clojars (2)
- # clojure (68)
- # clojure-russia (23)
- # clojure-spec (28)
- # clojure-uk (11)
- # clojurescript (7)
- # datavis (3)
- # datomic (21)
- # emacs (2)
- # events (2)
- # hoplon (56)
- # jobs (3)
- # lambdaisland (1)
- # mount (20)
- # off-topic (4)
- # om (13)
- # onyx (17)
- # other-languages (2)
- # parinfer (7)
- # proto-repl (2)
- # proton (2)
- # protorepl (53)
- # re-frame (13)
- # reagent (3)
- # ring-swagger (22)
- # specter (5)
@lucasbradstreet: so are speed issues are definitely caused by the windowing. I turned off the window and the same data set ran in 5min vs 1hr.
@mariusz_jachimowicz: thank you! I will review it soon.
@camechis Good to know. Hmm. The next thing to figure out would be whether it’s caused by the performance journalling or whether it’s due to the windowing calculations. It’s probably the former
Yeah, we are doing a collect by key so the task that the window is on generates they key. Not much to it
What batch sizes have you tested on the windowed task?
And did perf change much as you increased the batch sizes?
How fast are the disks that back BookKeeper?
and how big are the segments, KB wise?
Hah, probably not good. They are much older servers. Sad disks. We hope to move to AWS soon. I am not really sure but might be a little large. Basically a segment with 20ish key Val pairs
that’s not too bad, though of course it depends on the key and value size