Fork me on GitHub

how does multi-node know how many peers to start per instance?


@robert-stuttaford: n-peers when you call start-peers on each node


so, we’re setting a peer-count which is the total of alll tasks on startup. this means we’re giving each instance a full peer count for all the jobs. should we not be, @lucasbradstreet ?


@robert-stuttaford: When you call start-peers with an argument of n, you're designating n slots of concurrency for that machine. There's no "right" or "wrong" number of peers - it's a decision about how much concurrency is appropriate for your workload and hardware. The only thing that would be "wrong" would be not having enough peers to start your job. So if you have 2 machines with 3 peers started on each, you have a total of 6 peers. That means you can have any combination of jobs running that does not require more than 6 peers.


To answer your question, though, Onyx knows how many peers to use and which jobs to allocate them to by using the ZooKeeper log for atomic broadcast.