Fork me on GitHub
#onyx
<
2016-12-11
>
yonatanel12:12:22

Would you like a onyx-local-rt PR for not allowing to add a segment to non-existent task?

yonatanel13:12:33

Does Onyx "park" idle aggregations? I might have some aggregation window groups that are created by mistake and should disappear eventually, or at least not occupy resources after I log the occasion.

lucasbradstreet15:12:00

It doesn't really park there but you could use triggers and refinements to cull every now and again

yonatanel16:12:49

@lucasbradstreet Refinements will clean the group aggregation state but not the group aggregation itself.

lucasbradstreet17:12:33

Yes, you will still have the key in the group aggregation, though it will have nothing in it

yonatanel17:12:52

Would it make sense to have some policy to drop them?

yonatanel17:12:30

BTW, I added an ordering aggregate to my CQRS example. In case commands arrive out of order (according to an offset), they are buffered until it has a contiguous batch to process. Can anyone comment if there's a sane way to do that? https://github.com/yonatane/onyx-cqrs-example/blob/master/src/onyx_cqrs_example/aggregation/order_sensitive.cljs

lucasbradstreet18:12:12

yonatanel: I think so. Could you create an issue?

lucasbradstreet18:12:51

With the in order processing, we’ve done something similar in the past. The idea was You initialise the window with a starting point (say the known first offset). Then you create some simple aggregations to just add each message to the window state.

lucasbradstreet18:12:13

The trick is to evict all of the available in-order messages that you emit when you call the trigger

lucasbradstreet18:12:42

So the trigger grabs all of the in order messages, and emits them somewhere, then the refinement evicts them

lucasbradstreet18:12:13

Looks pretty similar to what you’re doing but I only gave it a cursory glance

yonatanel18:12:39

Mine is more complex since the segments are commands that need to be processed in order and change the aggregation internal state. The trigger evicts the already handled commands but leave the internal state as is.

yonatanel18:12:57

It's gonna get some more behavior such as timeout if we wait to long for a missing segment. I think it can be useful for anyone who also considers actors

lucasbradstreet18:12:33

Yeah, that’s basically the strategy that we took.

yonatanel18:12:38

So if there's anything in onyx that guarantees order once I found the order, it could be very nice.

lucasbradstreet18:12:06

When we land Asynchronous Barrier Snapshotting we will be able to guarantee in order processing depending on the topology