I got the rethink writer working last night. Will try to push up a repo tonight. It's been a fun project to work on. How would you guys compare onyx with samza? Is it attempting to solve similar problems?

@shaunxcode: the main slowdown is saving the checkpointing state updates to a log. Without that, we're at about 9000/core, but then you lose fault tolerance. As Michael says though, there's perf work to be done.