Fork me on GitHub
#onyx
<
2016-10-20
>
michaeldrogalis00:10:15

Anyone want to have a go at this? šŸ˜‰

michaeldrogalis00:10:31

This would need very little involvement from any of @lucasbradstreet, @gardnervickers, or myself, so ideal for a side project to incrementally hack on, maybe as a group.

yonatanel09:10:06

@michaeldrogalis I don't see a task there, just the link to local-rt

yonatanel09:10:45

Oh, the tutorial thing

yonatanel13:10:21

Is there a way to check after running a while whether Onyx had to handle any duplicates, network errors, crashing etc?

yonatanel13:10:16

especially the application level stuff, if RocksDB has any use, if something had to be replayed

Travis13:10:21

Metrics help a little there, checking for retries

lucasbradstreet13:10:40

Metrics definitely helps, though there may be some places that we need some additional metrics in places

lucasbradstreet13:10:54

Retries are definitely an important one

colinhicks16:10:14

@michaeldrogalis, I'll have a go at making an interactive tutorial.

michaeldrogalis16:10:49

@colinhicks Ah, brilliant. šŸ˜„ Btw, the doc tool hasn't slipped off my radar. Still searching for some time to integrate it, but Im eager.

colinhicks16:10:51

No prob. I could spend some time on docs, too, soon -- it's nice to be back from hurricane internet/power outages.

michaeldrogalis16:10:10

Oh, yikes. You were in the middle of all that?

colinhicks16:10:04

yeah. not our first rodeo, though, so we were prepared & safe. Just a bit unlucky this time with a group of trees creaming a utility pole down the street

michaeldrogalis16:10:50

Glad you're alright. That's frustrating, though. šŸ˜•

colinhicks16:10:02

Yep. Although good to get off the grid every now and then, even unwittingly. Onyx stickers rode the whole thing out in the mailbox. ... Something something great fault tolerance.

lucasbradstreet16:10:14

* Exactly one sticker not guaranteed

michaeldrogalis16:10:34

I sent a bunch, mostly because I always get an air bubble the first time

michaeldrogalis16:10:11

So for the interactive examples/tutorial, one interesting direction might be to go through the docs and lift the examples right out. We could look at bringing the interactive examples inline with the docs in the future. http://www.onyxplatform.org/docs/user-guide/0.9.11/#flow-conditions

michaeldrogalis16:10:28

But Im pretty much open to whatever anyone finds useful. This should be a really cool area to explore

colinhicks16:10:35

Yeah, that sounds great. Awesome literate programming opportunity. I'll put up an issue later today or tomorrow so we can all brainstorm.

michaeldrogalis16:10:51

That's fantastic. Thanks so much for taking the lead šŸ™‚ This'll go a long way to showing how Clojure really shines for this stuff.

Drew Verlee22:10:57

Concerning, aggregations What would be an example where the logic of the aggregation-fn would be different then that of the super-aggregation function. Im trying to write another chapter in learn-onxy on aggregations. https://github.com/onyx-platform/onyx/blob/75a48ce99f7d9cc829ea27922d987617abc7c5ac/src/onyx/windowing/aggregation.cljc

Drew Verlee22:10:52

I suppose iā€™m not sure why both of them are necessary.

michaeldrogalis23:10:17

@drewverlee super-aggregation's are necessary when two windows need to merge together.

michaeldrogalis23:10:31

The only mergable window's that Onyx offers are Session windows.

michaeldrogalis23:10:00

If you had two windows that were aggregated by count, and some data bridged those two windows into one session, you need to sum the counts to combine the windows into one.

michaeldrogalis23:10:28

Very easy to simulate the moment just before and after window merging using local-rt.