Fork me on GitHub
#jackdaw
<
2019-11-30
>
Quest01:11:59

Could anyone speak to the latency of Kafka Streams? I'd like to know the amount of milliseconds between publishing a message to a topic until it's received by a consuming Streams app (assuming negligible network latency.) Not finding much on Google

gklijs08:11:58

Depends on a lot of things. But using default settings, and having a low througput you have a latency between calling the send method in the producer and it actually be send to the broker going up to 100ms. This is because http://linger.ms is by default 100 in Kafka Streams to help throughput. Apart from that, and assuming connections already made, it should be on the order of a couple of milliseconds.

gklijs08:11:15

If latency is more important then throughput you can set it lower. It will also cause more stress for the brokers, because less messages are combined. From what I saw doing benchmarks I wouldn't set it lower then 10.

gklijs08:11:35

Some of them can be found here, when 0 the is a lot more load on the brokers, https://graphql.gklijs.tech/results/linger-ms-config/average-kb-cpu

👍 8
Quest13:11:45

Hmm, sub 40ms is probably find and achievable for my use case. Thanks for the graph and insight! 👍