This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-05-02
Channels
- # announcements (1)
- # babashka (4)
- # beginners (39)
- # calva (36)
- # cherry (11)
- # cider (23)
- # clj-on-windows (3)
- # clojure (105)
- # clojure-brasil (1)
- # clojure-chicago (3)
- # clojure-conj (8)
- # clojure-denver (4)
- # clojure-europe (18)
- # clojure-germany (5)
- # clojure-hungary (13)
- # clojure-nl (1)
- # clojure-norway (31)
- # clojure-sweden (9)
- # clojure-uk (2)
- # clojurescript (22)
- # core-async (4)
- # cursive (8)
- # data-science (25)
- # datomic (14)
- # devops (1)
- # emacs (9)
- # events (5)
- # holy-lambda (32)
- # hyperfiddle (26)
- # introduce-yourself (2)
- # kaocha (1)
- # leiningen (11)
- # lsp (17)
- # malli (8)
- # off-topic (84)
- # pedestal (4)
- # polylith (2)
- # re-frame (17)
- # reitit (1)
- # releases (1)
- # remote-jobs (1)
- # shadow-cljs (8)
- # sql (4)
- # tools-deps (8)
- # transit (5)
- # vim (1)
- # vscode (1)
- # xtdb (45)
Anyone here has experience with setting up a CDC stream from a SQL database? Cursory reading points me in the direction of Debezium which has lots of operational overhead. Any wisdom to share?
We did it using features that were built in to our database (SQL Server). No experience with debezium. Then we just have 1 polling thread per table across the whole cluster that publishes to a redis channel. Seems to work very quickly and efficiently. Subscribers (e.g., GraphQL websocket clients) then connect to those redis channels and do follow-up queries + responses. Only problem we have had is the CDC mechanism we selected (change tracking instead of CDC) doesn't let us know who owned records that get deleted, so we can't support these features for tables where we allow deletions for now.
You might be able to do it at the platform level if you're looking for less infra to manage yourself. eg on aws https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.RDSPostgreSQL