Fork me on GitHub
#xtdb
<
2022-10-24
>
nmkip17:10:30

Hi! @taylor.jeremydavid In the kafka docs it says:

Each XTDB node must be assigned a unique groupId for correct operation.
Does this mean that if we have the same topic in multiple machines, that each machine should have a different group id? cc @mail024

refset20:10:37

Hi :) assuming you're using Confluent Cloud without issues currently then you may be able to simply ignore this advice, as I'm not sure of the current defaults involved. It shouldn't be too hard to verify independently of your app though if you already have an account ready to test with (I can take a look later this week if you don't beat me to it). Have you observed issues with the transactions not being processed?

Alexis Vincent20:10:34

Hi there Jeremy, haven’t seen anything. But wondering if you can clarify if the above quote refers to different node instances for the same logical db/node or if it refers to cases where you have multiple nodes backed by different logical db/node’s on the same physical instance. I would assume the former, given that groups are scoped to topics, but not sure if theres some funky stuff going on

👋 1
Alexis Vincent20:10:26

Also, is this only for when nodes are running on the same machine, what about separate machines?

refset20:10:01

> if the above quote refers to different node instances for the same logical db This one Each XT node needs to be a full replica, regardless of how many you may run in parallel on the same machine, and therefore for each node's Kafka consumer(s) to receive a copy of all messages those nodes must not be operating within shared groups (there should be a 1:1 relationship of group and XT node). My uncertainty is simply whether explicitly setting the group.id configuration is required, or whether the (presumably) generated ID behind the scenes in the client (or broker, not sure) is sufficient. Some of the explanations and diagrams here are helpful https://stackoverflow.com/questions/34550873/difference-between-groupid-and-consumerid-in-kafka-consumer

Alexis Vincent21:10:46

Got you! thanks

👌 1