Fork me on GitHub
#xtdb
<
2022-09-13
>
markaddleman20:09:44

I hope it’s ok to ask a question about xtdb2. First of all, this seems like a very exciting project. One of the pains that we currently have with BigQuery is that we cannot group by a complex data type. that forces us to create JSON objects and stringify causing complexity in downstream queries. I see https://github.com/xtdb/core2/projects/1#card-85266205 and wondering if this pain point will be addressed?

refset21:09:11

Hey @U2845S9KL, good to hear from you 🙂 To briefly clarify: core2 is not XTDB 2.0 (for the time being, anyway ...it needs a lot more time, thought & energy to finish maturing first) I'm very happy to answer high-level questions about Core2 here, we're just keen to not get deluged with people tyre-kicking all of a sudden as things are quite raw still, and the issue board is just about busy enough for the moment! > One of the pains that we currently have with BigQuery is that we cannot group by a complex data type. that forces us to create JSON objects and stringify causing complexity in downstream queries. I see https://github.com/xtdb/core2/projects/1#card-85266205 and wondering if this pain point will be addressed? Instinctively I would say "yes", but I can't guarantee how that might work in practice. I am aware that BigQuery is inherently fairly limited due to how they scale it (compared to e.g. Snowflake, and certainly more classic DBs), so I would be really surprised if Core2 can't beat it in the near future on the "grouping by complex data types" front, at least.

markaddleman21:09:02

> I’m very happy to answer high-level questions about Core2 here, we’re just keen to not get deluged with people tyre-kicking all of a sudden as things are quite raw still, and the issue board is just about busy enough for the moment! I can appreciate this!

☺️ 1
refset21:09:05

Core2 already has the ability to handle OBJECT and ARRAY structs, so all the machinery exists for it already. We mostly "just" need to nail down the syntax. https://partiql.org/ and https://rockset.com/ are our main inspiration to that end.

markaddleman21:09:25

Thanks for the gut-level response. That was my intuition also

🙏 1
markaddleman21:09:50

I have loads more questions and tons of tire-kicking that I’d like to do. Mostly, I’m motivated by a desire to move away from BQ since getting around its limitations are eating away at our engineering effort. The problem isn’t very bad yet but it’s definitely getting worse. For now, I’ll just sit back and hope that you guys get XTDB2 to a testable state before I’m forced to make a switch 🙂

gotta_go_fast 1
♥️ 1
refset08:09:35

> I have loads more questions and tons of tire-kicking that I’d like to do Awesome, don't let me stop you 😄 > I’ll just sit back and hope that you guys get XTDB2 to a testable state before I’m forced to make a switch We'll do our best 🙂