xtdb

avi 2024-07-07T22:55:57.550219Z

👋 in v2, is there a way in SQL to get a count of the number of values in an array, for each matched row/record/doc/etc?

jarohen 2024-07-08T08:51:01.767609Z

LENGTH should do it?

avi 2024-07-08T14:48:12.066369Z

Works great, thanks!

avi 2024-07-08T14:48:19.057919Z

Where can I look up this kind of thing?

avi 2024-07-08T14:48:38.917949Z

I tried the Calcite docs but that’s either the wrong place, or I looked in the wrong place in those docs

jarohen 2024-07-08T14:51:49.986379Z

there's a couple of docs - one is https://docs.xtdb.com/reference/main/sql/queries.html of everything you can do in XT2 SQL; the other is our https://docs.xtdb.com/reference/main/stdlib.html, with more details about each of the functions

1
jarohen 2024-07-08T14:52:26.470739Z

but more generally, we try to keep to the SQL spec as much as possible, so if there's a standard SQL function for it, chances are we either have it already, or it's something we'll strongly consider adding

➕ 2
jarohen 2024-07-08T14:54:41.751239Z

IIRC the SQL standard function is CARDINALITY(coll), which we support

refset 2024-07-08T15:23:14.421289Z

I tried the Calcite docswe're actually not using Calcite at all in XT2, instead it's a bespoke planner + optimizer written in Clojure 🙂 https://github.com/xtdb/xtdb/blob/main/core/src/main/clojure/xtdb/sql/plan.clj

💡 1
avi 2024-07-08T15:36:57.482439Z

Aha! Makes sense. Thanks y’all!

avi 2024-07-08T15:45:14.704989Z

👋 Any tips how to get this working?

select name, case cardinality(location_refs) when [] then 0 else cardinality(location_refs) end as location_count
from partners
limit 10
I’m getting results like:
[
    {
        "name": "ý񕑊ïV"
    },
    {
        "name": "\u00167µZ󳨻É\u001b{Î=",
        "location_count": 8
    },
    {
        "name": "9f"
    }
]
I would have expected that first result to include location_count with the value 0

jarohen 2024-07-08T15:46:08.715849Z

what are you trying to do, sorry?

avi 2024-07-08T15:46:37.125279Z

for each document/record/entity (😬) return the cardinality of the array location_refs or 0 if the array is empty.

jarohen 2024-07-08T15:47:07.925169Z

I'm assuming cardinality(location_refs) returns null if it's given an empty array...?

jarohen 2024-07-08T15:47:27.961379Z

hmm, seems to return 0

jarohen 2024-07-08T15:47:38.544969Z

or is the issue that location_refs may be null?

avi 2024-07-08T15:48:06.860799Z

I get this:

select name, cardinality(location_refs) as location_count
from partners
limit 3
[
    {
        "name": "ý񕑊ïV"
    },
    {
        "name": "\u00167µZ󳨻É\u001b{Î=",
        "location_count": 8
    },
    {
        "name": "9f"
    }
]
ah yes, that’s probably it, not every record has location_refs at all

avi 2024-07-08T15:48:13.006139Z

so I suppose I need to use coalesce maybe

jarohen 2024-07-08T15:48:22.053799Z

aha - in which case ... yep, coalesce is your friend here 🙂

avi 2024-07-08T15:49:23.174259Z

yup that works

avi 2024-07-08T15:49:24.723839Z

thanks!

🙏 1
avi 2024-07-08T15:50:06.150219Z

BTW those two links you shared above look useful, but neither seems to include the text length or cardinality — should I look in the source code for the planner to see what’s supported?

avi 2024-07-08T15:50:29.716899Z

I guess maybe I need to find the docs on working with collections in SQL. I tried to find them but no luck so far.

jarohen 2024-07-08T15:50:32.807179Z

aah, apologies, will ensure it's included - entirely possible we've missed some 🤦‍♂️

avi 2024-07-08T15:50:59.125629Z

ah, no worries, makes sense, it’s a WIP. Thank you!

avi 2024-07-07T22:56:09.399069Z

e.g. something like:

select name, array_count(refs) as refcount
from partners
where refs <> []
group by name