Fork me on GitHub

@michaeldrogalis: Actually, I guess it's the same as when you start a job on every node in the cluster, so never mind.


@dave.dixon: Every subscriber receives every event in the log in the same order.


You can idempotently start a job by suppling the job ID.


@dave.dixon: See the docstring for onyx.api/submit-job.


@michaeldrogalis: I have to first do onyx.api/kill-job then onyx.api/submit-job in each subscriber, right? I assume these don't step on each other if issued at different times from different nodes.


@dave.dixon: kill-job is also idempotent, yes. Once a job has been killed, it cannot be restarted. Would recommend having your trigger write out a UUID to the log to serve as the new job ID. Then your subscriber(s) can resubmit the same job content through a different ID, which avoids the situation where two subscribers do submit, kill, submit, and thus make no progress.


Looks like Onyx was mentioned in this InfoWorld post on viable alternatives to Spark. Clojure platforms are often pretty niche compared to their Scala/Java counterparts. It’s awesome to see some recognition!


We're groooowing. 😄


@michaeldrogalis: If I'm sending a command like :restart-job via the log, do I need to implement extensions/apply-log-entry etc for :restart job? Currently getting error because this is not implemented, but maybe I'm missing something in how I should write to the log?


@dave.dixon: Yes, because all peers will consume all log messages, and need to know how to update their local replicas and state accordingly. all-log-entry, in your case, should just return the replica unaltered


fire-side-effects should be a no-op, same with replica-diff.


@michaeldrogalis: cool, that all worked, thanks for your help. Good exercise in understanding the internals of onyx.


Hi there! I’ve set up a lifecycle that injects a connection to the database for a window/trigger combination which persists the results of the workflow. I have a need now to keep track of “bad data” in the database as well but wanted to handle that in a flow condition. Can I reuse the connection from the lifecycle so that I don’t need to set up a separate connection for the flow condition?


@jholmberg: Yes. The connection object in the event map will be the same.


Thanks @michaeldrogalis, that will save me from having to set up an unneeded connection!

Chris O’Donnell22:08:28

@aaelony: I found some typos a couple weeks ago and submitted a PR; I believe that's the recommended thing to do, if you can.