@colinhicks where's the current onyx-blueprint thing running? Might try to contribute something

@martinklepsch https://github.com/colinhicks/onyx-blueprint 🙂

I remember seeing it running somewhere though? Only GH Repo?

It's not up publicly at the moment. I can put a new preview up later today if it would be helpful, @martinklepsch

Ah, yeah I was looking for the live copy a few days ago too. Thanks for the explanation @colinhicks. I’ll take a stab at doing a contribution on Sunday.

@colinhicks: ah ok, then im confusing it with this other thing I saw, sorry haha

Sorry for the confusion. I took it down in part to simplify things as other folks start to contribute

I'm AFK now, but will be back later to provide more details

Any ideas on what would roughly need to be done for a Hbase Plugin? At a high level i would assume I would need to build something to provide an “exactly once” contract between onyx and HBase.

@drewverlee For a reader or writer? Reader needs a way to partition the data set into discrete chunks that can be read in parallel. Plugins do not need to provide any exactly-once mechanism. Onyx itself takes care of that completely.

@michaeldrogalis I was semi interested in what it would take to do either. I was trying to compare it from what im reading about Apache Flink: https://ci.apache.org/projects/flink/flink-docs-master/setup/fault_tolerance.html, which discusses that the sources and sinks influence the guarantees your system has.

@drewverlee We’re getting close to having an alpha of the same fault tolerance system. Might want to wait a bit.