Fork me on GitHub
#onyx
<
2016-09-22
>
robert-stuttaford10:09:16

got exhibitor running (in UAT for now) finally, @lucasbradstreet !

aaelony22:09:29

what's the guidance of using kafka with multiple partitions? is it simply :onyx/min-peers and :onyx/max-peers that need to match?

michaeldrogalis22:09:16

@aaelony Yes. As of now, every virtual peer will read from exactly once partition, and Onyx will automatically assign peers to partitions dynamically if you ask it to.

michaeldrogalis22:09:51

Would like to see a patch for a virtual peer to read from multiple partitions concurrently, but we haven't needed it, so it hasn't gotten implemented yet. I'm sure it'll come up in the next 1-2 months for the hosted environment we're working on though.

Travis22:09:44

I think that would be useful for us, we are assigning each customer a topic and each customer may have multiple devices. Each device writes to a partition

Travis22:09:33

So right now we have adjust the config based on the customer

michaeldrogalis22:09:38

@camechis The patch is easy enough, Franzy lets you specify which partitions to set a Consumer on.

michaeldrogalis22:09:52

Mostly just trying to stay focused on what I'm working on now.

Travis23:09:59

No worries, I know the feeling. I have work coming out my ears getting this new platform up

michaeldrogalis23:09:13

Heh, yeah definitely.

michaeldrogalis23:09:43

Probably the hardest part of managing the platform on a day-to-day basis is keeping up with patches and fixes for the plugins, because no sane person uses all the storage mediums that we support on a day-to-day basis

Travis23:09:46

Yeah, we are working on cloud platform on aws so I have technology overload

michaeldrogalis23:09:11

Makes you dizzy after a while looking at all those very similar looking icons 🙂

Travis23:09:42

Dizzy today getting all of these cloud formation template built and playing

gardnervickers23:09:55

There be dragons

michaeldrogalis23:09:42

Indeed. 😧

Travis23:09:47

I have almost got it, lol. Then I can get back to onyx