Clojurians Log v2
Clojure programming
Channels
Apps
xtdb 2024-07-15
flefik 2024-07-15T14:59:58.547389Z
Is there a way to limit the wildcard-text-search function to a subset of documents? https://v1-docs.xtdb.com/extensions/1.24.3/full-text-search/
I have a large database with ~100M records. I want to do a wildcard-text-search on a subset of those documents (<100) (those with attr {:type :project}.
At the moment the search is very slow, I think because it searches and indexes all documents.
{:find [p1 (pull p1 [*])]
:where [[p1 :type :project]
[(wildcard-text-search query) [[p1]]]]
:in [query]}
refset 2024-07-15T15:09:56.338009Z
hey @cfeckardt yes you can do this sort of thing by creating and registering your own Lucene-related functions, see https://github.com/xtdb/xtdb/blob/06132bc1fad63944d144d79a90053b05d78a41c8/modules/lucene/test/xtdb/lucene/extension_test.clj#L152-L168
refset 2024-07-15T15:11:37.688919Z
> At the moment the search is very slow, I think because it searches and indexes all documents
that's right, by default the built-in wildcard-text-search will process the entirely result set coming out of Lucene eagerly - i.e. there's no pushing down of limit or other affordances that might be needed at sufficient scale
Hey folks. I am wondering if there is a way to query the first and last valid times across all entities of a database? It seems that I could just query every entity and parse each time, but I'm assuming there's a more idiomatic way to do that
And as soon as I post this, I found the examples on the v1 docs about the get-vt-created and get-vt-last-updated. Those should do the trick!
🙌 1
🙂 1