Clojurians Log v2

Hey @sean888 I don't have a working example in https://github.com/xtdb/driver-examples/blob/main/node/index.mjs to point you at right this second, but code in this issue demonstrates what James describes https://github.com/xtdb/xtdb/issues/4469 notably this pattern:

await sql`INSERT INTO my_table RECORDS ${sql.typed(obj, 16384)}`; // transit

yep - and IIRC that then requires an incantation in the setup (which we should put in the docs)

I figured out the magic 🙂 https://github.com/xtdb/driver-examples/blob/main/node/index.mjs

Awesome, thank you!

Some additions to the PostgresJS setup can be done to make sure that nested data is conveniently passed back to client code as properly typed JS objects, JS Dates, including nested objects. (Otherwise, transit-js will just parse into transit maps and tagged values):

const transitReader = transit.reader('json', {
  mapBuilder: {
    init: () => ({}),
    finalize: (m) => m,
    add: (m, k, v) => ({ ...m, [k]: v }),
  },
  handlers: {
    'time/zoned-date-time': (rep) => {
      // We receive "2024-11-06T11:43:20.123Z[UTC]"
      // See 
      const withoutTz = rep.replace(/\[[^\]]*]$/, '');
      return new Date(withoutTz);
    },
  },
});

const transitWriter = transit.writer('json');

sql = postgres({
      ...
      fetch_types: false,
      connection: {
        fallback_output_format: 'transit',
      },
      types: {
        transit: {
          to: 16384,
          from: [16384],
          serialize: (v: unknown) => transitWriter.write(v),
          parse: (v: unknown) => transitReader.read(v as string),
        },
...

Awesome, thanks!

there's a team working with one of our design partners that are making heavy use of transit-js - I'll see if I can get them to share any details

That would be great! Thank you 🙏

Hi. You can add transit to your custom types for easy use:

import transit from 'transit-js';
...
    sql = postgres({
      ...
      types: {
        transit: {
          to: 16384,
          from: [16384],
          serialize: v => transit.writer('json').write(v),
          parse: v => transit.reader('json').read(v),
        },

Hey @sean888 I've found you can use a CASE to help with casting workarounds also, noted here: https://github.com/xtdb/xtdb/issues/3411#issuecomment-2925132548 (and this issue is our wider plan to avoid users hitting these issues so easily)

I think my first bit of advice here would be to try not to use a polymorphic column for something you're relying on in queries - it's going to make your queries more complex 🙂 We've found best results where people consider the XT type system like clojure-spec - i.e. where possible any one attribute has a consistent type, but the attributes present within nested data (or even at the top-level) can vary. i.e. Polymorphic columns are possible, but I'd probably only rely on that in the same situations when I'd have polymorphic data in my Clojure/TS app

That said - you can try casting and then ordering by the casted column - I'm not sure we support expressions directly in the order-by yet but you could do SELECT ..., col::timestamptz AS coltstz FROM ... ORDER BY coltstz

(or cast to text if some are unparseable)

I’m building a system where an AI maintains the code that maintains the database. As the AI struggled with traditional migrations as a long time clojurian I turned to XTDB for the dynamic schemas. I expect the AI to modify the schema routinely and was hoping the dynamic nature of XTDB would help here. Seems like I might be barking up the wrong tree though. None the less, thank you so much for contributing xtdb to us, I love what your building and advertise it to my network every chance I get!