This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-12-23
Channels
- # adventofcode (4)
- # beginners (25)
- # biff (11)
- # clojure (2)
- # clojure-bay-area (1)
- # clojure-europe (5)
- # clojure-norway (2)
- # clojuredesign-podcast (4)
- # datalevin (2)
- # dev-tooling (34)
- # hyperfiddle (2)
- # missionary (5)
- # off-topic (3)
- # polylith (4)
- # rewrite-clj (13)
- # shadow-cljs (11)
- # tools-deps (1)
- # vim (1)
- # xtdb (5)
Running Query Plan
I want to understand how queries are executed for my XT instance. @taylor.jeremydavid had previously suggested using query-plan-for
so I was trying to enable on my instance of the DB.
Current XT Setup for testing:
1. Local XT client on 3000
through clojure cli
2. Queries run through another clojure file
This is what we are running:
(ns scripts.query-plan-for
(:require [xtdb.api :as xt]
[xtdb.query :as q]))
;; where our local xtdb server is running
;; define globally because passing through functions is annoying
(def xtdb-client (xt/new-api-client ""))
;; the query we want to look at
(def query '{:find [(pull ?e [*]), ?e, ?v, ?s]
:where [
[?e :type "concept"]
[?vs :concept_id ?concept_id]
[?e :xt/id ?concept_id]
[(text-search :search_descriptions "app*") [[?e ?v ?s]]]]})
(defn -main []
(println "query-plan-for")
(println xtdb-client)
;; TODO: change this to query-plan-for
(println (q/query-plan-for (xt/db xtdb-client) query)))
But my CLI output indicates an error:
query-plan-for
#xtdb.remote_api_client.RemoteApiClient{:url , :->jwt-token nil}
Execution error (IllegalArgumentException) at xtdb.db/eval8331$fn$G (db.clj:46).
No implementation of method: :open-index-snapshot of protocol: #'xtdb.db/IndexSnapshotFactory found for class: nil
Thoughts on what No implementation of method: :open-index-snapshot
could be attributed to since it could be:
1. Clojure Script for understanding query above
2. How XT instance is being run
Thoughts on how I could run query-plan-for
?Hey @U05EY098T09 apologies if I wasn't clear enough previously when I wrote (and mentioning also in case you didn't see): https://clojurians.slack.com/archives/CG3AM2F7V/p1703185649438699?thread_ts=1703143715.924169&cid=CG3AM2F7V
Essentially you can't run query-plan-for
remotely over HTTP. Are you able to use XT embedded temporary whilst developing your queries? That way you will have full REPL access and query-plan-for
can work
also note the reply about the workaround using a custom :rules
that can be used to force the execution order


@taylor.jeremydavid what would be the best source to learn how to interpret the query plan. You had particularly mentioned :vars-in-join-order
, but what about the others?
Also was wondering how the query plan is determined i.e. would the query plan vary based on the documents in the DB? so a query plan for a DB with 50 documents could be different than that for a DB with 10000 documents based on prevalence of field indexes?
> what would be the best source to learn how to interpret the query plan
vars-in-join-order
is by far the most important part, because the whole algorithm depends on picking a good ordering. You would need to read up on worst-case optimal joins (multi-way joins) to really get a feel for why this is, but there's good material out there, e.g. https://kuzudb.com/docusaurus/blog/wcoj/
for the rest I can't really help (sorry!) beyond pointing at https://github.com/xtdb/xtdb/blob/master/core/src/xtdb/query.clj - although I would be happy to talk through some more of the details and share what I can