xtdb 2023-12-23 | Slack Archive

Soumil Shekdar09:12:51

Running Query Plan I want to understand how queries are executed for my XT instance. @taylor.jeremydavid had previously suggested using query-plan-for so I was trying to enable on my instance of the DB. Current XT Setup for testing: 1. Local XT client on 3000 through clojure cli 2. Queries run through another clojure file This is what we are running:

(ns scripts.query-plan-for
  (:require [xtdb.api :as xt]
            [xtdb.query :as q]))

;; where our local xtdb server is running
;; define globally because passing through functions is annoying
(def xtdb-client (xt/new-api-client ""))

;; the query we want to look at
(def query '{:find [(pull ?e [*]), ?e, ?v, ?s]
   :where [
    [?e :type "concept"]
    [?vs :concept_id ?concept_id]
    [?e :xt/id ?concept_id]
    [(text-search :search_descriptions "app*") [[?e ?v ?s]]]]})

(defn -main []
  (println "query-plan-for")
  (println xtdb-client)
  ;; TODO: change this to query-plan-for
  (println (q/query-plan-for (xt/db xtdb-client) query)))

But my CLI output indicates an error:

query-plan-for
#xtdb.remote_api_client.RemoteApiClient{:url , :->jwt-token nil}
Execution error (IllegalArgumentException) at xtdb.db/eval8331$fn$G (db.clj:46).
No implementation of method: :open-index-snapshot of protocol: #'xtdb.db/IndexSnapshotFactory found for class: nil

Thoughts on what No implementation of method: :open-index-snapshot could be attributed to since it could be: 1. Clojure Script for understanding query above 2. How XT instance is being run Thoughts on how I could run query-plan-for?

refset00:12:52

Hey @U05EY098T09 apologies if I wasn't clear enough previously when I wrote (and mentioning also in case you didn't see): https://clojurians.slack.com/archives/CG3AM2F7V/p1703185649438699?thread_ts=1703143715.924169&cid=CG3AM2F7V Essentially you can't run query-plan-for remotely over HTTP. Are you able to use XT embedded temporary whilst developing your queries? That way you will have full REPL access and query-plan-for can work

👍 1

refset00:12:53

also note the reply about the workaround using a custom :rules that can be used to force the execution order

Soumil Shekdar06:01:46

@taylor.jeremydavid what would be the best source to learn how to interpret the query plan. You had particularly mentioned :vars-in-join-order , but what about the others? Also was wondering how the query plan is determined i.e. would the query plan vary based on the documents in the DB? so a query plan for a DB with 50 documents could be different than that for a DB with 10000 documents based on prevalence of field indexes?

refset10:01:27

> what would be the best source to learn how to interpret the query plan vars-in-join-order is by far the most important part, because the whole algorithm depends on picking a good ordering. You would need to read up on worst-case optimal joins (multi-way joins) to really get a feel for why this is, but there's good material out there, e.g. https://kuzudb.com/docusaurus/blog/wcoj/ for the rest I can't really help (sorry!) beyond pointing at https://github.com/xtdb/xtdb/blob/master/core/src/xtdb/query.clj - although I would be happy to talk through some more of the details and share what I can

2023-12-23

Channels