Hey @U03DWT00WUB in short 'no' - pull (projection) is executed explicitly after the main query is finished. I think you can work around it by either: 1. doing more work inside the query (e.g. including building up the hash-map nested result) 2. running the pull in a subquery, and then filter'ing out the results of the pull in an outer query 3. performing the filtering in your app

Happy to help with examples if one or more of those seems of interest 🙂

this is actually a semi common thing, pull is so convenient, but lacks this feature

luckily you can do multiple queries from the with the same basis and use clojure code to combine results… but still I have certainly wished for a filtered pull many times

> you can do multiple queries from the with the same basis and use clojure code to combine results ah yep, of course, how did I forget that 4th option 🙂

I've never actually tried it, but I've wondered before about whether https://github.com/lilactown/pyramid 's EQL pull mechanism could play nicely "out of the box" embedded in a Datalog query (essentially a sub-option for my option (1) above). It wouldn't be able to perform joins though like the actual pull, but only work on the individual document structures

Hello @U899JBRPF I'm curious about those first two solutions you mentioned. @U11SJ6Q0K can you show me what you mean by within the same basis and combine them? I just finished the available tutorials yesterday so still trying to get my footing. Thanks for your responses.

as you call (xt/db node) you can run multiple queries on that returned db and know that they will point to the same tx (consistent results)

Oh okay, so something like pull all the container records in call to (xt/q) A, pull all the items in call to (xt/q) B, and then do some kind of merge?

yes

as you are pulling form local indexes, the “roundtrips” don’t really matter

Just had another question, if the pull happens after the query is executed, would it be possible to change it so that it’s operating against “a world” that matches all the where clauses. So in my case if I put in a where clause that says items must be priced higher than $150, then the pulls join only can only look at items that satisfy the where clause?

not in any practical way afaict

Bummer. I came up with something like this after some tinkering.

(xt/q (xt/db node) 
        '{:find [(pull e [*]) items]
           :where [[e :wishlist/id]
                   [(q {:find [(pull i [*])]
                        :where [[i :item/lot lot]]}) items]]})

I’d say benchmark first… a little formatting doesn’t sound like a problem even for big results, but memory might be, You could also stream results with open-q depending on how the data is consumed

https://github.com/xtdb/xtdb/issues/471 found this issue

Hey @U32ST9GR5! Great question, I had to double-check this one 😅 we definitely don't support CORS in the latest versions of XT and we only did so initially to try and get a cljs console off the ground (but we later changed tack) I think a proxy is the way to go (short of forking the http-server module), and this might serve for inspiration: https://github.com/wkok/re-frame-crux

Yeah, I solved it by just putting this to my backend and integrated the routes from the xtdb.http-server to my route table, along with the cors-middleware 🙂 Seems to be working nicely!

I wanted our web client to be able to make requests directly to the xtdb as well since this is an internal tool – saves a lot of effort 😄

Thanks for answering!

Awesome, always happy to help ☺️

Eg how to do this through HTTP :

(xt/q
    (xt/db node)
    '{:find [(pull ?user [*])]
      :where [[?user :core/knows ?skill]
              [(get ?skill :core/skill) ?skill-id]]
      :in [[?skill-id ...]]}
    [#uuid "e206855a-2e83-4742-a793-8b8816a61dc7"
     #uuid "62e8cb0d-8cf2-480c-8919-3d90b9fff482"
     #uuid "f8051e4e-8281-4d3f-96f3-54b80a1f8fd5"])

in-args-json implicitly can't understand actual UUID types, so you'd have to pass and store them as strings (which is lossy and inefficient). Could you use in-args-edn instead? That should work fine with #uuid literals

> Where can I find documentation about the encoding of the in-args-json suggesting to browse the source/tests is certainly not a viable suggestion for non-Clojurists, but since you're posting here I'm presuming that's not a complete turn-off 🙂 see: https://github.com/xtdb/xtdb/blob/20a2632a18a36dae2e915ab032307ce0c5b3e150/modules/http-server/src/xtdb/http_server/query.clj#L53 and https://github.com/xtdb/xtdb/blob/44303aa35ff5843b2e1130d1bb4a943bc3f8b3be/modules/http-server/test/xtdb/http_server/json_test.clj#L122 This is the main docs page for the HTTP API though https://docs.xtdb.com/clients/http/openapi/1.21.0/#/paths/~1_xtdb~1query/get

Definitely can work my way from there. Thanks! 🙂

The error handling isn't terribly helpful here, and by default obscures the underlying error, but I think you just need to add an extra [ ] around your in-args-edn value In any case, this works:

curl -G \
--data-urlencode "queryEdn={:find [e] :in [a b] :where [[e :xt/id]] :limit 10}" \
--data-urlencode "inArgsEdn=[\"9dde518b-01df-4e64-b370-13ee3ba6c248\" \"5c44fc5f-cb5c-4f19-af1b-2e99672a0053\"]" \                     
localhost:3000/_xtdb/query

For reference, I debugged my own example there by adding some try/catch printing within this existing function in http_server/query.clj:

(defn data-browser-query [options]
  (fn [req]
    (try
      (let [{query-params :query body-params :body} (get-in req [:parameters])
            {:keys [valid-time tx-time tx-id query-edn in-args-edn in-args-json]} query-params
            query (or query-edn (get body-params :query))
            in-args (or in-args-edn in-args-json (get body-params :in-args))]
        (-> (if (nil? query)
              (assoc options :no-query? true)
              (run-query (transform-req query req)
                         in-args
                         (assoc options
                                :valid-time valid-time
                                :tx-time tx-time
                                :tx-id tx-id)))
            (transform-query-resp req)))
      (catch java.lang.AssertionError e
        (prn e)))))

We could probably add some tactical log/debug logging in there (to avoid any such ad-hoc REPL patching), short of a review of more comprehensive error handling / logging changes

Done https://github.com/xtdb/xtdb/commit/a5723508b741034a2f4fa59c445e320d47fe0923 🙂

Thanks!

Just to confirm - did that [ ] do the trick?

Yes it did! 🙂