xtdb 2022-01-07 | Slack Archive

javahippie11:01:54

First, thank you for building all of this! We are just getting started with XTDB, and it is possible that I just was not able to find the correct documentation for this. We are using a lot of java.time.LocalDate instances in our Clojure application and are currently trying to find a convenient way to store those in XTDB. Just simply putting them in the map results in a serialized Java Object in the DB, which is not ideal for us. Is there a way to register some kind of converter, so that we can work with java.time.LocalDate in our application, and don’t have to convert it explicitly when storing / loading from and to XTDB?

malcolmsparks11:01:44

I have a similar problem actually but I have a layer which does a postwalk on my data structures doing the mappings. I like that XTDB insists on #inst at the storage level and I'm happy to build my own layer over it. Not saying that's an ideal answer 🙂

👍 1

javahippie11:01:31

We contemplated building our own layer, but I wanted to ask first, to avoid building something that the tool already provided.

jarohen11:01:02

hey @javahippie - LocalDate is one of our supported value types, as of 1.13 (if memory serves!), so it should be being stored as a small binary representation. what are you seeing/what would you like to see from XT on this one?

javahippie15:01:55

Maybe I was mislead / confused by the default EDN representation that was printed in the REPL, now that I think about it. If it is a supported value type, does this mean I can store it and query for it, e.g. date ranges?

dvingo15:01:08

You will have to use a lib like tick https://github.com/juxt/tick this will add edn tagged literals for the java.time types and also make available comparison functions so you can use them in datalog clauses like [(tick.core/< ?a-date ?b-date)]

❤️ 1

javahippie15:01:23

Thanks for the pointer, will look into it!

refset16:01:14

> If LocalDate is a supported value type, does this mean I can store it and query for it, e.g. date ranges? That should be right, yep! Range predicates (which operate judiciously on binary sort-ordering) are usable for all first-class types (with ~undefined behaviour for everything else): https://docs.xtdb.com/language-reference/datalog-queries/#clause-range The canonical list of first-class types is here: https://github.com/xtdb/xtdb/blob/master/core/src/xtdb/codec.clj#L82-L100 ...I might just take this opportunity to add this all to the docs now 🙂

👍 1

refset16:01:48

As for using tick instead / as well, I wouldn't expect the efficiency of range predicates to come for free. Perhaps we/someone could come up with a binary-sortable encoding for various tick values though :thinking_face:

javahippie16:01:42

Thanks for the response, then everything should already work as we need it 🙂

🙏 1

dvingo19:01:38

I'd love to see an example of the range predicates working on java.time.* objects 🙂

refset21:01:48

@U051V5LLP hey! This is a lot more than you're asking for, but hopefully gives some clue(s): https://gist.github.com/refset/f737151c2956093bbe01d7b81a743328 (linked to from https://github.com/xtdb/xtdb/discussions/1514)

dvingo22:01:13

ooh thank you! @U899JBRPF will keep it in mind if I run into perf issues sorting dates

🙌 1

zeitstein21:11:52

> As for using tick instead / as well, I wouldn't expect the efficiency of range predicates to come for free. @U899JBRPF, could you clarify this, please? Since, e.g.

(type (tick/instant)) ;; => java.time.Instant

and java.time.Instant is a "first-class type" does your comment apply here?

zeitstein21:11:49

I see I don't have to use tick.core predicates with 'tick values', e.g. clojure.core/> works fine.

refset22:11:01

(type (tick/instant)) ;; => java.time.Instant

Ah, interesting! I must confess to never having used Tick in earnest...I assume it's all fine in that case, and XT's built-in > should work fine (no need to resort to clojure.core/>). Apologies for generating and spreading misinformation. You can see that Instants are first-class (indexed and sorted in a useful byte-order for range searches) here: https://github.com/xtdb/xtdb/blob/c41f4dee8af8c9673c0f04a742779665ecde4146/core/src/xtdb/codec.clj#L99

👍 1

Ash14:01:40

SPARQL… I’ve added xtdb-rdf 1.19.0 lib to xtdb-in-a-box 0.0.2 (`xtdb-core 1.19.0`)’s`deps.edn`. Then, against it, curl is successful. But curl ?o WHERE { ?s ?p ?o . } fails with an UnsupportedOperationException. (I do URLEncode that query arg when I actually try it.) Jetty’s DEBUG log entries show the corresponding incoming HTTP request but they aren’t helping me to figure out what the problem might be. Does anyone have any suggestions about how to turn on more logging, or what what I’m doing wrong?

Ash15:01:22

Mystery solved…

Does not support variables in predicate position: ?p

(I wonder how I can get that message surfaced in the log? I simply tried the sparql->datalog at the REPL to see that message.)

refset15:01:45

Ah, nice find, thanks for persevering 🙂 Surfacing the error up through the HTTP response sounds like a sensible idea though! I don't think there's currently a better way to debug than calling sparql->datalog in the REPL

refset15:01:52

Would you like to create an issue for this? I can't promise we will get to working on it soon, but I will make sure it's on the board somewhere(!)

Ash15:01:50

…done: https://github.com/xtdb/xtdb/issues/1685 Cheers.

refset15:01:02

awesome, thanks! 🙏

Steven Deobald19:01:56

Welcome, @javahippie and @ashley_mcclenaghan! We have begun to prod people toward our official community channels and I've signed myself up as the prod bot. 😉 Zulip (in this channel's description) is our official realtime chat. https://discuss.xtdb.com/ is on its way to becoming our official long-form medium. Of course, you're welcome to continue with Slack, but since Slack is web-unfriendly, we'd greatly appreciate it if you can duplicate your question / idea / thoughts into http://discuss.xtdb.com so it's both searchable on the public web and a living thread future users can contribute to. 🙏

2022-01-07

Channels