Fork me on GitHub
#jackdaw
<
2020-05-07
>
Daniel Stephens09:05:52

hope it's okay to ask general kafka questions in here, sorry in advance if not 😬 does anyone know why my consumer lag often sticks at 1?

Daniel Stephens09:05:40

that's a little bit of the output from describing the consumer group, cmd.customer is a source and goes to 0, but customer is both a sink and a source, could that be the reason? I have basically two flows in the topology:

cmd.customer -> some internal topics -> customer

customer -> aggregation -> agg.customer
So cmd.customer has commands on customer has events on those get read to produce an aggregate on agg.customer

davewm14:05:35

@ULNRSUK8C are you using exactly-once semantics? If so, the kafka console tools will report the lag as 1 even when all messages have been processed

Daniel Stephens20:05:58

ahh, yes I am. Thanks, I guess just something I need to be aware of then, cheers!

gklijs18:05:18

Not that I mind asking the question here, but you might also like the #apache-kafka channel.

👍 4