xtdb

2024-11-10T02:50:33.346549Z

hey y’all, I’m seeing that if I transact a java.util.Date then xtql queries return a java.time.ZonedDateTime. Is it possible to have queries return a java.util.Date?

jarohen 2024-11-11T10:35:05.291449Z

@nonrecursive fwiw we've had the same issues with Transit - we built on the excellent https://github.com/henryw374/time-literals library with https://github.com/xtdb/xtdb/blob/main/api/src/main/clojure/xtdb/mirrors/time_literals.clj in the XT repo - feel free to copy it too, it's permissive open-source after all 🙂 regarding the return type: as @taylor.jeremydavid says we store all three of those (Date, Instant, ZDT) as an Arrow timestamp, so it's unfortunately a lossy round-trip. we did consider returning timestamps with tz=UTC as Instants (and potentially still could with a config flag or something) - but this would just have shifted the problem to users putting in ZDTs and wondering why they get Instants out in certain cases 😅

jarohen 2024-11-11T10:36:33.371749Z

Malli's got malli.experimental.time (https://cljdoc.org/d/metosin/malli/0.16.4/api/malli.experimental.time) that you can include in Muuntaja - but yeah, I share Sean's sentiments, it's been 10y now 😄

seancorfield 2024-11-10T02:56:48.577329Z

I guess I'd ask, why would you want a mostly deprecated Date type, rather than a modern Java date/time type?

2024-11-10T03:27:47.901449Z

fair question 🙂 the bigger picture is i’m trying to whip up a little prototype app and dealing with muuntaja/transit encoding for java.time is enough of a hassle that i don’t want to have to spend time on it in at this point in the app’s lifecycle like, i know it’ll be a good idea to use java.time but right now I don’t even know if this will even see the light of day and i want to get something working I can play with without having to take too many detours

2024-11-10T03:42:34.539209Z

halfway to getting transit encoding/decoding sorted 🙂

seancorfield 2024-11-10T04:40:09.670779Z

I'm a bit surprised muuntaja and transit don't already have built-in support for Java Time... it's been around since Java 8, which came out a decade ago...

2024-11-10T15:03:17.307319Z

I agree! it’s really just about transit. the muuntaja part is having to thread the transit config into where muuntaja lives in my middleware

refset 2024-11-10T19:37:17.521569Z

I believe a j.u.Date is auto-coerced and stored as a TIMESTAMP WITH TIMEZONE (an Arrow timestamp, internally) and then can only be returned as a ZDT

2024-11-11T03:36:25.157969Z

I’ve found that transacting java.util.Instant also results in a ZDT getting returned, maybe for the same reason? saw that it’s possible to cast with SQL, but not XTQL afaict

✅ 1
Panel 2024-11-10T09:38:25.399899Z

Are all the type listed here https://docs.xtdb.com/reference/main/data-types.html useable via next.jdbc ? I struggle to setup SettableParameter for dates types.

seancorfield 2024-11-10T15:02:26.469739Z

Have you required next.jdbc.date-time to enable basic PostgreSQL support?

Panel 2024-11-10T19:33:41.756259Z

Yes I did and a time/date works. But I can’t get time/zoned-date-time.

refset 2024-11-10T19:40:21.622939Z

Some types may not be fully implemented over pgwire yet (apologies!), but you should be able to work around everything, including ZDTs, using the transit type, e.g. per https://github.com/xtdb/xtdb/blob/9829b45d2599813b5847626ea66f26eefe5f9ce9/src/test/clojure/xtdb/pgwire_test.clj#L1877-L1889

👍 1
Panel 2024-11-10T19:54:56.247119Z

Cool, what’s the tradeoff of using transit ? I guess it’s like storing end as string, you can’t use it in query ?

refset 2024-11-10T20:46:16.722769Z

the transit (JSON serialization) is just a way of transferring values via the pgwire protocol, so once the data is stored everything is the same