xtdb

tatut 2024-08-14T08:27:10.321769Z

I’ve been playing around with the v2 API, very simple API that is easy to build for… but I see the https://docs.xtdb.com/drivers/http/openapi/index.html#/paths/query/post the docs mention XTQL still, do I remember correctly that it is only SQL now? the body payload only has fields for sql

jarohen 2024-08-14T08:42:20.616979Z

yep, this is just supported through Transit for now, the JSON was proving very unwieldy and not something we wanted to commit to. best to access it through the XT Clojure client if you can? otherwise, let us know your use case and we can figure out how best to go forward

tatut 2024-08-14T08:43:25.111669Z

I don’t need XTQL, just curious… my “use case” for now is just playing around and using XTDB from Prolog https://x.com/tatut/status/1823375791255650318

jarohen 2024-08-14T08:43:54.834759Z

ooh, intriguing 👀

jarohen 2024-08-14T08:46:03.284719Z

that's neat ☺️ 🙇

tatut 2024-08-14T08:47:16.054039Z

the point is to provide a “candidate” record and it would make a query that fetches all rows that look like the candidate

jarohen 2024-08-14T09:03:01.842679Z

I see 🙂 any plans for how you might introduce joins, say? (it's been way too long since my brain was in Prolog mode 😅)

tatut 2024-08-14T09:24:13.210069Z

yes, this was literally just a couple of hours of hacking

tatut 2024-08-14T09:24:23.566509Z

so very young, next up is NEST_MANY/ONE at least

👌 1
tatut 2024-08-14T08:39:10.982609Z

also NEST_MANY/ONE look super neat, fixing a common relational SQL problem that you need to try to get a rectangle shaped result back into nested data, which is always a chore

jarohen 2024-08-14T08:40:40.511539Z

heh, yep - we've spent way too many hours writing those SQL queries by hand ourselves 😅

tatut 2024-08-14T08:40:56.933649Z

me too

Dave Mays 2024-08-14T12:39:59.816969Z

I know that XTDB can be used to model valid time for a record on an entry as a whole. But how would I model if multiple values for a particular item had a valid time? I'm thinking of a service based application, where a user might definite a range of time where their account is either active or paused (maybe they only need the service for weeks they're out of town). But there might be other time ranges as well, like certain dates the user has earned a free service. Maybe a referral program allowed them 1 months of free services. The user may also only live at the address for a particular amount of time, and move partway through a service window. How mould you model potentially overlapping valid date ranges for different things relating to on "user account" in XTDB v1? It's not that the entire record is valid or invalid at a specific point. (And should I be trying v2 already?) I'm new to XTDB but also to graphs in general, but I think if I can start to see the pattern of how to link info it will start to click. I'll go through the XTDB datalog tutorials.

tatut 2024-08-14T12:47:51.952889Z

my first throught would be to model it as a regular attributes, with from/to field dates, no need for time travel in that case

👍 1
refset 2024-08-14T12:49:14.161249Z

Hey @dave554 in general you should avoid trying to use valid time to represent anything other than the timeline about when a 'fact' is learned to be true (i.e. the state of an entity over time as best-known, and as defined by a particular document version). Concepts like "account status timeline" or "service window" are best represented within facts, like the response above describes. For v2 we are planning to add SQL range types that will make this kind of userspace modelling around time easier

Dave Mays 2024-08-15T06:43:05.028739Z

Thanks!

tatut 2024-08-14T14:03:03.455029Z

the v2 API docs have JSON LD types for things like local date time, but what about decimal values? are JSON numbers the only supported ones, and what is the precision

tatut 2024-08-14T14:04:47.384439Z

like a bigdecimal with a scale of 2 if I want exact numbers only (like money)

refset 2024-08-14T15:43:02.163889Z

hey @tatut you can see the currently supported scalar types here: https://docs.xtdb.com/reference/main/data-types.html#_scalar_types generally speaking, all typing reflects Arrow types first and foremost, and then considers ISO SQL semantics, and then finally Postgres semantics Arrow does actually have DECIMAL128 and DECIMAL256 as https://arrow.apache.org/docs/cpp/api/datatype.html, but v2 doesn't do anything with those currently, but it could, and then potentially map them to the equivalent of Postgres' DECIMAL type what domain type are you actually hoping to model? is it money?

tatut 2024-08-14T15:45:01.166439Z

I mean the JSON API specifiically

tatut 2024-08-14T15:45:40.399229Z

as JSON only has the 1 number type, and I didn’t see any JSON LD types for different number types

tatut 2024-08-14T15:46:11.042719Z

so if I pass in 4.20 in a JSON payload, what type is that in the database?

tatut 2024-08-14T15:47:32.861849Z

so I guess if one wanted an exact decimal number, using bigint and fixed point math is the way to go

tatut 2024-08-14T15:50:34.371819Z

for example, if I do via the API: {"txOps": [{"sql": "INSERT INTO foo (_id, n1, n2, n3) VALUES (1, $1, $2, $3)", "argRows": [[4.2, 100, 999999999999999999]]}]}

tatut 2024-08-14T15:50:56.816119Z

what will the n1, n2, n3 values be internally?

refset 2024-08-14T15:55:23.539199Z

ah, sorry I missed that bit from my answer - the JSON LD typing support has not been implemented comprehensively yet, but you can see the current subset here: https://github.com/xtdb/xtdb/blob/0c0010e9f860197230ecbd43a3de0e6aa25b4ac7/api/src/main/kotlin/xtdb/JsonSerde.kt#L60-L70 I think we can add the other types relatively easily but it will be a low priority task for us to finish until someone files a specific requests 🙂 (new issues welcome!) JSON numbers will be coerced to Longs (`FLOAT` according to the scalar types table), but if that fails then Doubles, looking at https://github.com/xtdb/xtdb/blob/0c0010e9f860197230ecbd43a3de0e6aa25b4ac7/api/src/main/kotlin/xtdb/JsonSerde.kt#L87

tatut 2024-08-14T16:09:06.392919Z

thanks

tatut 2024-08-14T16:09:22.756549Z

I don’t have a real use case yet, just “kicking the tires” out of curiosity

tatut 2024-08-14T16:10:08.063619Z

but money is something that I’ve very often had to deal with in most enterprise apps

tatut 2024-08-14T16:10:31.854909Z

so usually important to have a precise number type

💯 1
refset 2024-08-14T17:53:01.187589Z

thanks, the feedback and input is appreciate - hope you've been having fun!