Hi, I think you could try adding a predicate in the relevant leg(s) to guard against that, i.e. [(some? abbr)]

> Changing nil values to empty string does not work I'd be curious to see what you attempted for this, as I would expect it to be another valid approach. Was it converting nil -> "" before passing into :in and running the actual query? EDIT: Ah I see, text-search accepts "" fine but Lucene itself will throw a parsing error. Hmm. Maybe that's a sensible default and should be left alone, I'm not sure :thinking_face:

Yes I check if abbr is nil and if so change it to "" before using it in the query

I tried with the predicate [(some? ?abbr)] in all legs. I still get an Exception

ah, I suppose there's no way (currently) for the query planner to know to prioritise the some? clause ahead of the text-search

something else I think you could try as a fairly hacky workaround (while I think of something better) - instead of replacing nil with "", replace it with "averyveryveryverylongstringthatdefinitelywontreturnanyresults"

This works. I'll go with the hacky solution for now. Better than writing a separate query for every combination of nil parameter values

with the hacky workaround you'll still be hitting the lucene search, you could avoid that with the dynamic query approach

I would've thought that and would short circuit as it does in clojure, but it makes sense that the planner reorders things

I wonder if some variant of andthat works as a planner fence would be possible, like (and-short-circuit clause1 ... clauseN) (with a better name perhaps)

:thinking_face: or even a guard in a predicate form like [(text-search :name ?name) [[e]] :when ?name]

sorry for the delayed response (blame the Slack DNS outage last night!)... A key idea of Datalog (and XT's Datalog included), is that it is declarative, which isn't really compatible with traditional notions of conditionals & control flow. I agree though that and is counter-intuitive when you're used to clojure.core/and

datomic doesn't have a planner that rearranges forms, so datalog doesn't preclude that imo

but yeah, nothing wrong with dynamically building queries

XT will happily store and index anything which can be serialised via https://github.com/ptaoussanis/nippy (including many funky Java object types), however you are right to ask about byte-comparable types. The list as it stands today is only defined in the codec.clj namespace, e.g. https://github.com/xtdb/xtdb/blob/a7907f35d55ce8935da0b93c7f6752e0a615010c/core/src/xtdb/codec.clj#L82-L100

We haven't documented this much mainly because the exact supported schema is still subject to roadmap changes. Also note that there are edge cases when using the range predicates (`>` etc.) where value type boundaries are not necessarily intuitive, e.g. comparing across numbers https://github.com/xtdb/xtdb/issues/1298

You can also handle your own userspace byte encodings and store byte-arrays, since https://github.com/xtdb/xtdb/pull/1168

yeah, I was mainly concerned about the comparability... because I had tried ju.Date, jt.LocalDate and Time and those seemed to work, those seem to be supported specifically

thanks for the pointers

if the comparison can't be done by just pulling from the index, will it revert to deserializing all and calling compare on them, or fail?

The byte-wise comparison will still be done, since bytes are always available, but it will be ~meaningless. You can always replace > with clojure.core/> though to get the more familiar semantics (and comparison failures!) with the cost of deserialising each value and comparing every possible entry in the index one by one

yeah, definitely don't want that if it can be avoided

so it's good to know what the byte comparable value types are

Clear datatype docs have been on the ol' todo list for quite some time. There will be a bit of hardening around datatypes with xtdb v2 so it's likely we won't publish strict formal docs until that point. Apologies for sending you to the source for now.