xtdb

esp1 2024-10-10T18:25:54.281669Z

Hi - I'm trying out xtdb 2.0.0-b1, and I'm not able to get wildcard (`*`) https://docs.xtdb.com/reference/main/xtql/queries.html#_from to work. I'm starting an in-memory node and adding a doc like this:

(def node (xt.node/start-node {}))
(xt/submit-tx node [[:put-docs :my-table {:xt/id 100, :foo "hello"}]])
If I query with explicit keys, I get the key values as expected:
(xt/q node '(from :my-table [xt/id foo])) => [{:foo "hello", :xt/id 100}]
But if I try to use a wildcard query, I'm getting an empty object back:
(xt/q node '(from :my-table [*])) => [{}]
Am I doing something wrong?

refset 2024-10-10T21:40:31.235119Z

Hey @esp1 I can't think of anything to suggest which might be going wrong 🤔 that example seems to work fine https://play.xtdb.com/?version=2.0.0-b1&type=xtql&txs=W3sic3lzdGVtLXRpbWUiOm51bGwsInR4cyI6Ils6cHV0LWRvY3MgOmRvY3Mgezp4dC9pZCAxMDAgOmZvbyBcImhlbGxvXCJ9XSJ9XQ%3D%3D&query=KGZyb20gOmRvY3MgWypdKQ%3D%3D Is it possible you're accidentally running against two entirely different nodes?

esp1 2024-10-10T21:43:12.174109Z

no, i'm definitely using the same node. i can try this on the docker version of xtdb and see if it works. if so, maybe this is just an issue with the in-memory node?

esp1 2024-10-10T21:44:08.547729Z

oh i also just noticed there is a beta2 out, maybe i can try that too

refset 2024-10-10T21:51:32.372239Z

the Play environment is just spinning up in-memory nodes also, so there shouldn't be a difference there, but yes using the Docker image helps to rule out any JVM shenanigans at least

esp1 2024-10-10T21:52:13.883079Z

i just upgraded to 2.0.0-beta2 and it works now!

refset 2024-10-10T21:52:46.863279Z

aha, how random, not sure what could have changed(!)

refset 2024-10-10T21:53:07.284259Z

out of interest, have you used HoneySQL much before? are you strongly preferring the idea of XTQL?

esp1 2024-10-10T21:53:31.979539Z

thanks for looking into this with me, i appreciate you giving me the sanity check on the Play environment

🙏 1
refset 2024-10-10T21:55:24.299839Z

Play is still using beta1 though 😅 I would love to understand what happened, but I'll leave that for another day

esp1 2024-10-10T21:55:28.119119Z

i've only used honeySQL a little, so i'm probably not the best person to give a representative opinion. i like the idea of constructing queries from data structures instead of strings, but both honeySQL and XTQL will do that for you

👍 1
esp1 2024-10-10T21:57:25.486539Z

in clojure-land i've probably used datomic more than sql lately, so having a different query syntax like XTQL is ok by me, but again i may not be representative of the community

esp1 2024-10-10T22:00:29.133279Z

that said, i really do like the psql compatibility that xtdb offers. i've done way more sql than datomic back in the day.

🤘 1
esp1 2024-10-10T22:03:19.884579Z

i guess xtdb doesn't like hyphenated table names?

refset 2024-10-10T22:09:39.590039Z

ah yes, of course, the name is being 'normalized' for SQL compatibility, so the hyphen becomes an https://play.xtdb.com/?version=2.0.0-b1&type=xtql&txs=W3sic3lzdGVtLXRpbWUiOm51bGwsInR4cyI6Ils6cHV0LWRvY3MgOm15LXRhYmxlIHs6eHQvaWQgMTAwIDpmb28gXCJoZWxsb1wifV0ifV0%3D&query=KGZyb20gOm15X3RhYmxlIFsqXSk%3D that then can't be losslessly roundtripped - an unfortunate edge case 🤔

refset 2024-10-10T22:09:58.962049Z

well spotted

esp1 2024-10-10T22:15:15.990729Z

that's fine, i can use different table names. might be good to call this out in the docs tho if it can't be worked around.

👍 1
esp1 2024-10-10T22:28:07.839169Z

I had a few questions that I couldn't find answers to in the docs: • is there a way to list the databases/tables in xtdb? • is there a way to export/import data or backup/restore an xtdb database?

refset 2024-10-10T22:38:50.951189Z

> is there a way to list the databases/tables in xtdb? yep! like any SQL-spec-compliant database we offer https://play.xtdb.com/?version=2.0.0-b1&type=sql&txs=W3sic3lzdGVtLXRpbWUiOm51bGwsInR4cyI6IklOU0VSVCBJTlRPIGRvY3MgKF9pZCwgZm9vKSBWQUxVRVMgKDEsICdiYXInKSJ9XQ%3D%3D&query=U0VMRUNUICogRlJPTSBpbmZvcm1hdGlvbl9zY2hlbWEudGFibGVz tables for this kind of metadata (there's also pg_catalog) - there was a note about this in the docs a few weeks back but it seems it's been lost along the way > is there a way to export/import data or backup/restore an xtdb database? pg_dump would work ideally (we actually haven't tried it yet to see what's missing in the compatibility layer), but currently you need to write your own SQL (/XTQL) manually for this. You should also be able to rely purely on backups of the storage backends for backup/restore - just make sure that the tx-log backup is newer than the object-store backup (whilst also taking into account any configured tx-log retention window), or pause the system

refset 2024-10-10T22:40:18.092179Z

but also on the backup/restore, most importantly, please get in touch before you attempt to roll out v2 into production 🙂 (prior to GA, anyway)

esp1 2024-10-10T22:40:49.068959Z

sounds good, thank you!

🙏 1