hey all! Can someone tell me the different use cases for Jackdaw's test machine versus its mocks? Would anyone be able to describe their preferred approach to integration testing?

The test machine is designed so that the question of mocks vs “a real Kafka” is mostly an implementation detail that can be ignored by the test author. My typical workflow is to prepare a test using the mock driver. Once this is passing, I make sure it also works using a real local Kafka cluster (typically provided by docker compose). An optional 3rd step is to run the same test against against a remote Kafka cluster (e.g your uat environment) using the rest-proxy. There are subtle differences between the three environments that you need to be aware of but these are mostly in the setup phase. For example when using the mock driver, you don’t need to create any topics in advance but when testing against a real cluster you do. Likewise when testing against a local cluster, the test will have to arrange for the system under test to be started but when testing against uat that’s is probably handled by your deployment system (e.g Jenkins or whatever).

There’s some examples in the jackdaw repo in the jackdaw tests themselves and in the word-count example. There’s also some sample code in a few of the blog posts I’ve written about it at https://grumpyhacker.com

ah, brilliant. I had been writing tests against a Dockerized Kafka but then looking longingly at the actual Jackdaw tests with the mocks. I guess doing it at multiple levels would be great though.

Yeah the mocks give me pretty good confidence that the logic for some topology is working as intended. They even test the actual serialization of messages using whatever Serde is specified. The only limitation I’ve come across is that they won’t tell you if your topics are configured incorrectly (e.g are the inputs to a join “co-partitioned”).

The ideal setup for me would be to run with mocks locally, which you can very quickly in the repl without kafka running

But run tests against the real kafka on CI, ideally without changing any code but just with an environment variable

It is possible in theory but we haven't tried that yet, and there is probably some work to do to work around the subtle differences between the two