Fork me on GitHub
#xtdb
<
2023-05-02
>
tatut06:05:03

with morse being open sourced (w/ replicant) I guess that would be the best way to explore an XTDB even remotely, anyone tried it yet?

refset13:05:59

I've not, I am certainly curious though! I guess your https://github.com/tatut/xtdb-inspector module is more "operational" but are there things that morse could make simpler?

tatut13:05:11

Well, inspector really is there to get something for the ops people. Sometimes you just need a way into looking at the db and you dont have a custom UI for it. I guess in xtdb2 you can just use any postgres client?

👍 2
tatut13:05:40

I haven’t tried morse yet either.

refset13:05:13

> I guess in xtdb2 you can just use any postgres client? maybe - hopefully once we've sorted out the INFORMATION_SCHEMA story, at least

j4m3s09:05:24

Hi, I have a question about datalog / xtdb : I'm trying to understand how this could be done more efficiently. I have a "blogpost" that has multiple :post/tag that have as values some ids of tags. I'd like to query the blogpost and get the names in the result and not the ids. My insert :

(xt/submit-tx xtdb-node [[::xt/put
                             {:xt/id :another
                              :tag/name "tata"}]])
(xt/submit-tx xtdb-node [[::xt/put
                             {:xt/id :test-tag
                              :tag/name "toto"}]])

(xt/submit-tx xtdb-node [[::xt/put
                             {:xt/id :test-post
                              :post/title "title"
                              :post/content "content"
                              :post/tags [(first (first (xt/q (xt/db xtdb-node)
                                                       '{:find [id]
                                                         :where [[id :tag/name "toto"]]})))
                                          (first (first (xt/q (xt/db xtdb-node)
                                                              '{:find [id]
                                                                :where [[id :tag/name "tata"]]})))
                                          ]}]])
My query:
(xt/q (xt/db xtdb-node)
      '{:find [?title ?content ?tag-name]
        :where
        [[?post :post/title ?title]
         [?post :post/content ?content]
         [?post :post/tags ?tag-entity]
         [?tag-entity :tag/name ?tag-name]
         [(= :test-post ?post)]
         ]})
This gives the following result :
#{["title" "content" "toto"] ["title" "content" "tata"]}
This is what I want, but this would be nice if I could do like "coalescing" into a collection, I'm not sure if that's possible.

j4m3s10:05:52

Okay, seems I found what I want using pull! I replaced the tags by a set of tags for the blogposts. I can query them that way :

(xt/q (xt/db xtdb-node)
           '{:find [(pull ?post [* {:post/tags [:tag/name :xt/id]}])]
             :where
             [[?post :post/title ?title]
              [?post :post/content ?content]
              [?post :post/tags ?tag-entity]
              [?tag-entity :tag/name ?tag-name]
              [(= :test-post ?post)]
              ]})
And it gives the following
#{[{:post/title "title", :post/content "content", :post/tags [{:tag/name "toto", :xt/id :test-tag} {:tag/name "tata", :xt/id :another}], :xt/id :test-post}]}

alexdavis10:05:23

Yep pull syntax is what I would do, also very small thing but clojure has ffirst instead of (first (first ...

2
j4m3s11:05:06

Ooh nice! TIL

Xiaomin Wang18:05:59

Hi, is there a way to put a record with datetime using json (equivalent to #inst if I were to use edn)? The following will treat “1985-01-01” as string, which makes it hard to do any date comparison in a query (if there is a good way, I’m all ears). Thank you!

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"tx-ops": [["put", {"xt/id": "user1",
                              "birthday": "1985-01-01"
          }]]}' \
     "$xtdb_url/_xtdb/submit-tx"

refset20:05:33

Hey @U054KQT2081 XTDB doesn't do anything clever to interpret/coerce JSON document contents as anything other than the https://www.json.org/json-en.html JSON types - so unfortunately there's no easy answer here. However you should be able to submit documents containing well-formed UTC ISO 8601 datetime strings and rely on the lexicographical sorting

Xiaomin Wang20:05:32

got it. Thank you!

refset20:05:52

Out of interest, have you already tried using https://github.com/go-edn/edn ?

Xiaomin Wang20:05:17

yea this is where I’m leaning towards

👍 2
R.A. Porter19:05:49

I’m trying to write a query with a combo of inclusions and exclusions and having trouble trying to work out how to format the :in and how to pass the arguments to the query. 🧵

R.A. Porter19:05:40

I have a where clause that contains these two terms:

'[[?x :tags tags]
(not [?x :tags exclude-tags])]

R.A. Porter19:05:06

Then figured I should write my :in as:

'[[tags ...] [exclude-tags ...]]

R.A. Porter19:05:29

And then try to call with:

(xt/q
 (xt/db node)
 query
 [["in-tag1" "in-tag2"] ["not-tag1" "not-tag2"]])

R.A. Porter19:05:51

I’ve tried a lot of variations, but can’t figure out how to pass the two vectors successfully. If it comes to it, I can manually build the query instead, but it feels like I should be able to pull this off without that extra work.

R.A. Porter19:05:44

(I think I’m going to be embarrassed now. I think I’ve got it).

R.A. Porter19:05:39

Well, it’s not blowing up anymore, but I’m getting bad results…I’m trying to and the two disparate terms together but that’s not helping. (The fix for the initial problem, of course, was just to extract the two vectors and pass as separate args).

R.A. Porter20:05:00

Even after moving to manually adding my not clause, I’m getting results that include the tags that should be excluded. Can I not execute a “with these but not those” query at all, or is there more likely something just wrong with my structure still?

refset20:05:24

Hey @U01GXCWSRMW 🙂 I'm not sure I'm following entirely - if you can share a slightly more completely example it will probably shed light

R.A. Porter20:05:15

I’ll have to write a short example of what I want. Not sure I’ll get to it until tomorrow.

👍 2
refset20:05:24

based on your description it sounds like you want your :in bindings to be scalars (i.e. raw clojure sets) not collections/relations

refset20:05:51

then it might be:

'[[?x :tags tag]
  [(contains? tags tag)]
  (not [(contains? exclude-tags tag)])]

R.A. Porter21:05:05

Something like that, yeah.

R.A. Porter21:05:34

Except, I want it on the same field. Two sets of params against a single field. Something like… “Find me all the articles with a content tag of [‘intro’ ‘intermediate’] that do not have a content tag of [‘java’ ‘kotlin’].”

R.A. Porter21:05:11

In practice, it’d be more like: tagged-with: ‘intro’ not-tagged-with: ‘java’, ‘kotlin’, ‘golang’, ‘python’

R.A. Porter21:05:22

I can manage it manually (finally!) where I do an in clause for my vector (e.g. “intro”) and then conj a bunch of not clauses for each of my exclusions. It’s not the ugliest code, but seems like I’m solving a problem with a hammer that probably has a screwdriver that would fit. 🙂 I’ll try to get a good sample together tomorrow.

🙌 2
seancorfield20:05:23

@jonpither’s Conj talk is up!

🎉 9
Xiaomin Wang23:05:36

Hi, is https://www.xtdb.com/blog/xtdb-transaction-functions.html#_transaction_functions the recommended way to update a record in xtdb? and can transaction function be created via http client? Just copying the code didn’t work for me. Thanks!

curl -X POST \
     -H "Content-Type: application/edn" \
     -H "Accept: application/json" \
     -d "{:tx-ops [[:xtdb.api/put {:xt/id :increment-age
          :xt/fn '(fn [ctx eid]
            (let [db (xtdb.api/db ctx)
                      entity (xtdb.api/entity db eid)]
                  [[:xtdb.api/put (update entity :age inc)]]))}
                  ]]}" \
     "$xtdb_url/_xtdb/submit-tx"

refset09:05:13

Hey @U054KQT2081 transaction functions are generally the easiest way to update a record, yes, because they have the simplest consistency guarantees (whereas xtdb.api/match requires backoff & retry) and give you an opportunity to perform schema validations, integrity checks etc. > and can transaction function be created via http client? yes it should be no problem - the issue you're seeing may be because Accept: application/json needs to be application/edn also

gratitude-thank-you 2
refset09:05:22

> Just copying the code didn’t work for me what error did it give?

Xiaomin Wang14:05:55

{:error "Malformed \"application/edn\" request."}

refset15:05:32

ah, yes that would be expected 🙂 the container log might show more info about how the request was malformed (or if it doesn't currently, then it certainly could be made to do so - or indeed we could pass more info back via the HTTP response)

Xiaomin Wang15:05:42

@U899JBRPF I started the server locally with java -jar xtdb.jar -f xtdb.edn. I didn’t see any err/log in std. where else can I find logs?

refset16:05:26

you would probably need to configure a logback.xml under a resources dir like https://github.com/xtdb/xtdb/blob/master/bench/resources/logback.xml + with <logger name="xtdb" level="DEBUG" />

Xiaomin Wang16:05:20

thanks for the pointers! I see logs for other queries/tx, which will be useful for the future. no logs for the ones with malformed requests. but just removing the single quote in front of (fn [ctx eid] got me around the error! now calling the function is giving me Transaction aborted: lmk if you have a hunch of what else could be wrong. otherwise, I will play around more later. thanks again!

refset17:05:17

cool, no problem 🙂

refset17:05:10

some of the dev-time UX is definitely easier when you're working in-process with a proper JVM stack trace but Transaction aborted: is generally a little tricky due to the async nature

Xiaomin Wang17:05:03

hmm so ' is needed? but that doesn’t seem to be valid edn syntax

refset22:05:59

Ah, you probably don't need the quote unless you are inside a Clojure namespace, here you are merely inside a string