sql

Ben Sless 2023-05-02T19:04:40.717709Z

Anyone here has experience with setting up a CDC stream from a SQL database? Cursory reading points me in the direction of Debezium which has lots of operational overhead. Any wisdom to share?

isak 2023-05-02T20:27:33.723469Z

We did it using features that were built in to our database (SQL Server). No experience with debezium. Then we just have 1 polling thread per table across the whole cluster that publishes to a redis channel. Seems to work very quickly and efficiently. Subscribers (e.g., GraphQL websocket clients) then connect to those redis channels and do follow-up queries + responses. Only problem we have had is the CDC mechanism we selected (change tracking instead of CDC) doesn't let us know who owned records that get deleted, so we can't support these features for tables where we allow deletions for now.

2023-05-02T22:04:43.175869Z

You might be able to do it at the platform level if you're looking for less infra to manage yourself. eg on aws https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.RDSPostgreSQL

Ben Sless 2023-05-03T03:11:49.511819Z

Thanks!