Fork me on GitHub
#clara
<
2021-10-20
>
jjttjj18:10:35

Has anyone attempted to do anything with windowing in clara? For example using moving averages, but capable of being more general, such as this transducer: https://github.com/cgrand/xforms/blob/62375212a8604daad631c9024e9dbe1db4ec276b/src/net/cgrand/xforms.cljc#L586 It should be possible in clara using accumulators right?

ethanc21:10:16

I have never played with windowing functions before, however accumulators would likely be able to handle most things. The caveat being, that it might not do them as well or as performant… transducers are usually pretty performant, where-as clara’s “reduce-fn”(accumulator) might not see benefits of what a transducer might offer. In addition to potential hotspots if retraction comes into the picture, as the default behavior of the accumulator is to “re-boil the ocean” or re-reduce all values seen sans what is being retracted…. A naive approach would be to do something like:

(acc/accum
  {:initial-value []
   :reduce-fn conj
   :convert-return-fn #(into [] (x/window-by-time <populate arguments>) (sort-by <time-field> %))})
The approach above is likely less than ideal, while the “retract” would be a linear sweep, if the accumulator had multiple executions(new facts to process after the original set of facts) it would re-call the convert-return-fn function on all accumulated facts.

jjttjj21:10:51

Thanks so much! That's a good starting point to try it out

ethanc22:10:05

If you end up needing to be more performant, a custom reduce-fn and retract-fn would likely have to be the next steps. Where the reduce-fn manipulates a data structure that can maintain previously done work, and the retract-fn that can strategically remove certain pieces of data in a more performant way than the default “brute force” manner