Fork me on GitHub
#onyx
<
2017-10-04
>
ben.mumford08:10:48

@lucasbradstreet i was using an out of date aeron. using this: [io.aeron/aeron-all "1.4.1"] instead of this: [uk.co.real-logic/aeron-all "0.9.4"] fixed my problem (the group had moved)

ben.mumford08:10:14

@lucasbradstreet perhaps the onyx documentation should be updated as appropriate? the code snippet in http://www.onyxplatform.org/docs/user-guide/0.11.x/#_messaging_implementations refers to the old group. i think it should be: (:import [io.aeron Aeron$Context] [io.aeron.driver MediaDriver MediaDriver$Context ThreadingMode]) not: (:import [uk.co.real_logic.aeron Aeron$Context] [uk.co.real_logic.aeron.driver MediaDriver MediaDriver$Context ThreadingMode])

michaeldrogalis15:10:31

@ben.mumford I’ll put a fix in — thanks for the report!

ben.mumford16:10:33

@michaeldrogalis drat i could have got myself down as an onyx contributer (never mind it is just documentation). keep up the good work

michaeldrogalis16:10:51

@ben.mumford Hah, next time. 🙂 Thanks.

lucasbradstreet16:10:40

We should just refer to the version in the onyx code. I might update that.

Drew Verlee22:10:40

How do people generally solve the problem of a late event arriving after the other data the window it would fall into has been evicted? I assume those events evicted events are retrievable, is it up to the user to provide a way to pull them back into onyx in order to re-process the window?

lucasbradstreet22:10:38

@drewverlee this problem is primarily solved via watermark triggers, and buffering up the event / state updates with something like the incremental state store.

lucasbradstreet22:10:13

The idea is that you track the watermarks for your stream, and once you’re sure you have all of the data, you materialize the state updates into an aggregate and trigger it via the watermark trigger

lucasbradstreet22:10:40

are you using kafka? Our automatic watermark tracking is half finished, but I have it in my todo to finish it off over the next few days.

Drew Verlee22:10:21

@lucasbradstreet thanks! I was asking more for academic reasons 🙂

lucasbradstreet22:10:42

OK sure, well the support is coming 🙂

lucasbradstreet22:10:07

Basically the idea is you buffer things up in a way that doesn’t require a lot of extra serialization, and flush when you’re sure you have all of the data