Hi there. I’m just beginning with Onyx, and I’m looking to cache materialized views in redis. Will redis pub/sub work with onyx? I want to avoid putting the views into kafka.

I really don’t know what I’m doing, so if the question doesn’t make sense, I wouldn’t be surprised. 🙂

Not sure I follow how you want to use onyx

basically for a cqrs and event store. I’m just a little confused about how tasks that are asynchronous work together. Kafka is the standard solution?

In that case I would think Kafka would be part of your event store and redis would be a sync

I'm not sure if there is redis support for the source of your data

ok thx. I’m digging into the the plugins to see how :input tasks work

Basically it's going to grab some number of segments from your input source and push them through the workflow

@wildermuthn we don’t currently have a supported redis input plugin, because it’s been difficult to resume from redis sources in a fault tolerant way, because we need to be able to replay from offsets, or similar. There may be ways to use it with pub / sub now. I’d have to look into how pub / sub works with redis

Ok thanks!

The main problem with the standard pub/sub methods is there is no persistence, so messages will be lost if the peers need to be reallocated. I hear redis is adding a kafka-lite type commit log though.