Fork me on GitHub
#onyx
<
2016-10-17
>
michaeldrogalis00:10:35

@yonatanel Thanks, fixed in b3e1882b444be75b97bfc322e31393115ea9a941 on develop.

michaeldrogalis00:10:46

And, what @camechis said is correct.

yonatanel12:10:23

How do I invalidate Onyx's state (checkpoints, aggregation state in BookKeeper etc.) if I want a clean reprocessing of all my data?

lucasbradstreet12:10:06

These are scoped to job id, so you should get a clean reprocessing without doing anything.

yonatanel12:10:45

I see that resubmitting the same job-id is not possible. Does that mean Onyx will resume on its own when started after a total crash or shutdown?

lucasbradstreet12:10:35

The only exception is if a job was killed for some reason (exception was thrown that was unhandled by lifecycle/handle-exception or flow conditions)

yonatanel12:10:31

What if I don't want it to resume? If I killed it on purpose so it would stop a faulty processing with nasty side effects.

yonatanel12:10:38

There's kill-job, but that won't be invoked if I hysterically pull the plug of the machine :)

lucasbradstreet12:10:40

You can pull the plug, then bring the machine back up, don’t start the peers, call onyx.api/kill-job which will write a message to zookeeper (note that the peers aren’t up), then start the peers up

michaeldrogalis15:10:45

Hello everyone. In preparation for something we'll be putting out in the next 24 hours, we put up a mostly complete/mostly accurate Spec for Onyx. https://github.com/onyx-platform/onyx/blob/cljc-triggers/src/onyx/spec.cljc Should be useful.

michaeldrogalis15:10:40

We stopped short of finishing the Spec in its entirety for all of the internal Onyx components because most people will not need them. This should be treated as experimental, and is not used for validation yet. We need to backport our pretty-errors that work against Schema to use Spec instead.

michaeldrogalis15:10:14

If you spot any errors, please open an issue. Much of this file was automatically generated, and the tool that does the generation still needs to be extracted from another project.

yonatanel17:10:09

I'm searching onyx source code to see how passing vector input to core-async plugin works. I've noticed test in learn-onyx knows when the task is done and triggers a window trigger even though there are less segments in the input than the tigger threshold. Any pointers?

yonatanel17:10:52

Never mind. It's the bind-inputs! function in each challenge in learn-onyx. 😬

yonatanel17:10:31

I already have in my todo list to implement a vec/seq plugin, manifold plugin and kinesis plugin. How much of a waste is it?

gardnervickers18:10:45

Manifold is also compatible with core.async channels from what I remember, a Kinesis plugin would be fantastic though!