xtdb 2023-06-08 | Slack Archive

Thomas Moerman12:06:05

Question: • we run xtdb in embedded mode, with rocksdb/postgres stores • we experimented with running the app with a replication factor of 2 (instead of 1), but ran in some "stale/missing read" problems when serving REST calls, which is logical because the request hitting the second node might arrive before the transaction done in the first node has propagated to the rocksdb indexes on the second one I was wondering which options are typically considered to mitigate this problem, we were thinking of either: • use (xt/sync) in each request handling chain before querying the db? I'm wondering how expensive / idiomatic this approach is? • run XTDB standalone and have both app nodes talk to XTDB as a service • others that come to mind? Thanks for your perspective 🙏

jarohen15:06:11

Hey @U052A8RUT 👋 Yep, that's most likely what's happening. Calling xt/sync before reads wouldn't be the worst thing in the world if you have relatively low write throughput; otherwise (an approach we're using in v2), you could keep track of the latest transaction you've submitted, and ask to await that before running the query Running XT as a standalone server and connecting through the remote API is also a valid option 🙂

👍 2

Thomas Moerman19:06:49

Ok great! xt/sync sounds doable as we're not that write-heavy. Cheers

kokonut15:06:25

Hi, I am using postgres for xtdb and wondering if there is any way to see the content of the entries on SQL clients (e.g. postico). For example as shown in the screenshot below, the v field only says the content is binary. But it'd be great if I can see content or any part of it there. Or should I use some other client program?

jarohen15:06:31

Hey @U022N1DU7GQ 👋 these are encoded in https://github.com/ptaoussanis/nippy format - if you have a client that'll give you back the binary representation you can fire it via nippy/thaw to get the object back

💯 2

kokonut15:06:29

@U050V1N74 I see. So I can do it on clojure repl rather than portico or psql, is this correct?

jarohen15:06:45

I probably would, yeah

👍 2

jarohen15:06:04

psql will give you the result in hex, IIRC

jarohen15:06:03

ah, you could use postgres's encode(v, 'base64'), then it can go straight to nippy's thaw-from-string

jarohen15:06:33

(if you have an XT REPL available, Nippy will be on your classpath)

kokonut15:06:05

When business people, who don't necessarily know clojure, need to analyze data, how do they see the data in xtdb usually?

jarohen15:06:02

we only really offer a Datalog query engine in 1.x I'm afraid. we have heard of a few non-Clojure folk learn Datalog rather than the whole of Clojure - understand that's quite an ask though! full SQL support is available in 2.x (currently in very early access) though, for exactly that reason 🙂

👍 2

tatut04:06:26

https://docs.xtdb.com/extensions/sql/ there is the apache calcite route if you want to connect analytics tools to your xtdb

👍 2

2023-06-08

Channels