Hi folks! I'm evaluation a use case for Onyx so I have a few questions: I thinking in something akin to what Cognician made with event sourcing (congratz @robert-stuttaford and team!). My question is about deployment: how do you organize your Onyx and ZK nodes. AFAIK the minimum quórum for both clusters are 3 nodes each? I need/can co-locate them?

What is not clear to me is what is a minimal production scenario, and ways it can grow over time.

Sorry if I missed the docs if there already somewhere I can find that information.

@andrewhr you can colocate them, but you’ll hit core contention quickly. make very sure of your minimum cpu reqs first, or prepare to expand from 3 to 6 quickly. also, i highly recommend you start with 3, not 1. we’ve just used nearly 3 months dealing with the assumptions, lack of know-how, and tech debt built up around starting with 1. finally, if you can at all, just buy some consulting from the fellas. save you a LOT of hassle.

spoken as someone who’s just done it the hard way

sure, thanks for sharing!

If you’re trying to save money, going with 3 ZooKeeper nodes collocated with 3 Onyx nodes is probably the minimum recommended scenario

We recommended that Cognician use beefy enough machines that they could handle the failure of one node, and have their jobs continue fine with 2 nodes left

Colocating ZK with Onyx nodes does have downsides though, since you have a greater chance of one bringing down the other

Great @lucasbradstreet. My concern is that we are still small. I'm looking at Onyx as a programming model more than it's distributed capabilities, at least initially. I'm afraid to get something too much overkill.

Right, Cognician went with a single node collocated with ZK. This will work fine, though it’s easy to have some issues lurking that you will need to solve when you go multi-node. That’s what @robert-stuttaford is referring to

I think that’s perfectly valid, just realise that you will have technical debt that will need to be paid later as you scale up

got it. The operations side-of it will be painful but I good to know that I have this route (as the saying goes... BDSM goes hand in hand)

Just to illustrate, at my client we're planning a redesign of our backend and that's why I want to hit small first.

@andrewhr: here are the main things to watch out for while you work on it https://github.com/onyx-platform/onyx/blob/0.8.x/doc/user-guide/environment.md#multi-node--production-checklist

Thanks @lucasbradstreet and @robert-stuttaford for the help! 🍻

Good timing on that question @andrewhr.

Here's our newest tool, released a few minutes ago - Engraver. http://www.onyxplatform.org/jekyll/update/2016/03/21/Engraver-A-Tool-for-Managing-Onyx-Clusters.html

very cool

Hope it helps

cool! 👍

Hey guys, not sure if it works well with Onyx yet (should), but I just released a Zookeeper embedded server and cluster library that is actually up-to-date using the latest curator and confirmed working with Kafka 0.9 and a few zk clients I tried. Could use more testing, but works well for me in dev so far when I don't need docker, unit testing, or on the go. Onyx guys, there's maybe some stuff you can either use or steal as well if required. If anyone uses it and finds an issue or has a reasonable feature request, let me know. https://github.com/ymilky/travel-zoo

ymilky: Coffee and TV is the best Blur song ever.

@otfrom: very true! Blur isn't necessarily one of my favorite bands ever, but I do like a bunch of their songs and saw them live a bunch of times back when they were up and coming. It seemed a good mascot representation of my first name and last name shortened. ymilky is as a semi-amusing privacy helper that can be somewhat taken seriously, or one can just say "why milk, why".

Nice @ymilky thanks

ymilky: handy when debugging your code I suppose. 😛

hey @ymilky what is the fastest way to setup a Kafka + Zookeeper env nowadays?

on Ubuntu Linux if you know, or maybe docker...

I would like to try your lib