xtdb

jarohen 2025-05-02T10:56:26.751169Z

Hey #xtdb - can I get a show of hands (or DM, if you don't want to publicly admit it!) of folks who'd want to be involved in/informed of any upcoming changes to XTQL? specifically, in this round, we're looking at what's involved in getting XTQL inlinable into SQL queries - both as a standalone query, but also (for bonus points) as a SQL subquery/sub-table reason we're looking to combine the two is largely to open XTQL up to normal Postgres tooling - so imagine a world where you're able to query XTQL through Metabase or similar πŸ™‚ but also, behind the scenes, because the APIs (xtdb.api and pgwire) are starting to diverge a little, which is a little concerning to us - we're at risk of needing to double-implement new functionality in the pgwire and Clojure APIs πŸ˜…

βœ‹ 1
jarohen 2025-05-02T10:56:52.229059Z

e.g. basic case - sending your XTQL queries through next.jdbc:

(require '[next.jdbc :as jdbc]
         '[xtdb.next.jdbc :as xt-jdbc])

(jdbc/execute! xt-node [(xt-jdbc/xtql '(-> (from ...) ...))])

;; before, for comparison

(require '[xtdb.api :as xt])

(xt/q xt-node '(-> (from ...) ...))

jarohen 2025-05-02T11:00:47.356729Z

(cc @taylor.jeremydavid)

jarohen 2025-05-02T11:04:32.455529Z

xt-jdbc/xtql would just be a helper - expands to the following XT SQL extension (but you're still writing data-oriented queries, so no injection attacks)

XTQL $$ (-> (from ...) ...) $$
('$$' being string delimiters in Postgres to avoid needing to escape quotes)

jarohen 2025-05-02T11:05:25.575539Z

then, because everything's 'just a relation', you can inline this within larger SQL queries:

FROM XTQL $$ (-> (from ...) (...)) $$
WHERE ...
SELECT ...
ORDER BY ...

jarohen 2025-05-02T11:09:05.811929Z

> but also, behind the scenes, because the APIs (xtdb.api and pgwire) are starting to diverge a little on this one, we'd likely either move to having the Clojure API calling next.jdbc on your behalf - or, if it's such a small layer that it's not worth a layer, we may look to deprecate it entirely (maintaining the ability to create small throwaway XT nodes for unit tests, of course, because that's awesome)

seancorfield 2025-05-02T14:05:18.893739Z

From a HoneySQL / language spec / parsing p.o.v., I'm very interested in this.

πŸ˜„ 1
jarohen 2025-05-02T14:10:01.786979Z

@seancorfield in HoneySQL, I might consider adding something like [:xtql '(-> (from ...) ...)] , implementation being (fn [xtql] (format "XTQL $$ %s $$" (pr-str xtql)))?

jarohen 2025-05-02T14:10:21.357239Z

otherwise, it should be opaque within the SQL string

seancorfield 2025-05-02T15:06:17.484379Z

Interesting suggestion...

jarohen 2025-05-02T15:07:33.125099Z

obv you're way better placed to decide what'd be idiomatic HoneySQL πŸ™‚ example above just to show that it's a simple transformation

seancorfield 2025-05-02T15:07:58.181859Z

I created a GH issue about it. I'll see what else comes up in this thread.

πŸ‘Œ 1
jarohen 2025-05-04T16:12:50.285879Z

@dpsutton yep, we've implemented just about enough of Postgres that it seems to handle XT pretty well 😊 tempted to try adding some valid-time/system-time smarts to Metabase given it's Clojure, some form of timeline sliders or history UI or something. one for a hack day πŸ™‚

jarohen 2025-05-04T16:14:59.788789Z

although tbh its graphing is already pretty good - might 'just' be the syntactic awareness of the SQL:2011 temporal additions

jarohen 2025-05-03T12:53:51.938999Z

querying XTQL through Metabase 😊 (in flight)

πŸ™Œ 4
jarohen 2025-05-03T12:55:24.044929Z

XTQL can also be a subquery in a bigger SQL query (because everything's just relations)

dpsutton 2025-05-03T20:01:29.298989Z

Woah. This is really cool

βž• 1
πŸ™ 1
☺️ 1
dpsutton 2025-05-03T20:20:03.468899Z

Is this using the Postgres driver?

πŸ’― 1
jarohen 2025-05-02T15:09:10.426779Z

how'd this appear here...? πŸ‘» πŸ˜…

seancorfield 2025-05-02T15:10:41.357019Z

I used Slack to create an issue from a message in this channel, so GH helpfully told the channel that I'd done it (and it also posted it in #honeysql)

jarohen 2025-05-02T15:10:53.344199Z

aaah, gotcha

jarohen 2025-05-02T15:11:04.550609Z

neat πŸ™‚

seancorfield 2025-05-02T15:11:14.872659Z

The Slack/GitHub integration can be a bit... enthusiastic... in its communication.

πŸ˜„ 1
jarohen 2025-05-02T15:11:17.357509Z

I've never created issues on GH from Slack

seancorfield 2025-05-02T15:13:07.934419Z

I use it a lot, since folks here often chat about good ideas for the various libraries I maintain πŸ™‚

❀️ 1
Wes Richardet 2025-05-02T00:18:08.504399Z

Hello, whats the latest on XTDB 2.x for handling geospatial data?

refset 2025-05-02T06:46:42.795969Z

Hey @wes there's no special support just yet in either the standard library or indexes. What kind of functionality do you need? Simple proximity? Polygon intersections?