This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-10-13
Channels
- # announcements (1)
- # babashka (30)
- # beginners (43)
- # biff (24)
- # calva (72)
- # cider (12)
- # clj-commons (24)
- # clj-on-windows (10)
- # cljsrn (23)
- # clojure (123)
- # clojure-bay-area (6)
- # clojure-europe (43)
- # clojure-losangeles (2)
- # clojure-nl (2)
- # clojure-uk (9)
- # clojurescript (125)
- # core-async (9)
- # cursive (2)
- # data-science (3)
- # datomic (30)
- # devops (1)
- # emacs (13)
- # events (5)
- # fulcro (15)
- # graalvm (3)
- # gratitude (1)
- # humbleui (11)
- # lsp (5)
- # nbb (24)
- # off-topic (11)
- # pedestal (5)
- # releases (1)
- # remote-jobs (1)
- # sci (15)
- # scittle (16)
- # shadow-cljs (15)
- # sql (11)
- # tools-deps (9)
- # xtdb (5)
Hi there, we’re running into an issue where open-tx-log on a kafka backed node gives us a nil seq, but local rocks works fine. Tested on v21 and v22. Is this supported with Kafka as tx-log?
Or is there perhaps an alternative way we can get our hands on the transactions (we want to basically get a group of the documents that were written over time)
Hey @UJWLUPW13 can you share the node config map? open-tx-log
should surface the transactions properly (the ~same as using Rocks), assuming they have been indexed. You could also try creating https://docs.xtdb.com/clients/1.22.0/clojure/#_new_submit_client and use the open-tx-log
method on that, which should give you an unfiltered view of all transactions (whether they have been successfully indexed or not)
{:xtdb.kafka/kafka-config (gx/ref :xtdb/kafka-config)
:local-rocksdb
{:xtdb/module "xtdb.rocksdb/->kv-store"
:db-dir "/tmp/rocks"
:checkpointer {:xtdb/module "xtdb.checkpoint/->checkpointer"
:store {:xtdb/module "xtdb.google.cloud-storage/->checkpoint-store"
:path ""}
:approx-frequency "PT30M"}}
:xtdb/tx-log
{:xtdb/module "xtdb.kafka/->tx-log"
:kafka-config :xtdb.kafka/kafka-config
:tx-topic-opts {:topic-name "net"
:replication-factor 3}}
:xtdb/document-store
{:xtdb/module "xtdb.kafka/->document-store"
:local-document-store {:kv-store :local-rocksdb}
:kafka-config :xtdb.kafka/kafka-config
:doc-topic-opts {:topic-name "net-a"
:replication-factor 3}}
:xtdb/index-store
{:kv-store :local-rocksdb}}