Fork me on GitHub
#xtdb
<
2021-10-29
>
pinkfrog13:10:59

Hi, What’s the query that finds all entity-ids in the database?

(xt/q (xt/db node)
        '{:find [e]
          :where [e _]})
won’t do.

pinkfrog13:10:13

(xt/q (xt/db node)
        '{:find [e]
          :where [e :xt/id _]})
This won’t either.

Piotr Roterski13:10:35

you were really close, this should get you what you want:

(xt/q (xt/db node)
        '{:find [e]
          :where [[e :xt/id _]]})
notice :where expects a vector of vectors while you supplied a single vector

🙏 1
pinkfrog13:10:39

(xt/q (xt/db node)
        '{:find [e]
          :where [[e :xt/id]]})
Turns out the missing enclosing [] in where

💯 1
pinkfrog13:10:45

right. thanks

pinkfrog13:10:59

It seems to me, when the end-time is earlier than the start-time, then the valid time would be [start-time, forever]

pinkfrog13:10:05

e.g,

(xt/submit-tx node
                [[::xt/put 
                  {:xt/id :asdf/asdf
                   :first-name :john}
                  #inst "2018-05-18"
                  #inst "2018-05-14"]])

  (xt/q (xt/db node #inst "2018-05-20")
        '{:find [e]
          :where [[e :xt/id]]})

refset15:10:29

yep, end-valid-time is simply used to put one or more nil records in the entity history, so it is essentially a noop from a query perspective to set it to a time before the start-valid-time. I expect you would still see the nil entry inserted at the appropriate place in the history if you look at the output from entity-history. Arguably XT could detect the faulty argument and reject the transaction before submitting it, but that's not how it works currently. If you're curious, see https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/core/src/xtdb/tx.clj#L102-L115

pinkfrog15:10:18

Yes. I saw a nil result.

pinkfrog15:10:12

@U899JBRPF Curious about the #xtdb-id thing, when I run:

#xtdb/id "98947c1c08ece69c6f2c19c849c54d561843a789"
in the repl, it shows:
; Syntax error (IllegalArgumentException) compiling fn* at (.calva/output-window/output.calva-repl:0:0).
; No implementation of method: :id->buffer of protocol: #'xtdb.codec/IdToBuffer found for class: clojure.lang.Symbol

refset16:10:23

I'm not certain on how reader macros work at the repl, but does it work if you eval (identity #xtdb/id "98947c1c08ece69c6f2c19c849c54d561843a789")?

pinkfrog16:10:11

Yes. That works. What’s the benefit of using #xtdb/id reader macro?

refset20:10:12

Cool. Not sure on exact history, but I think it greatly aids testing and repl work for the core team

pinkfrog02:10:37

I guess for #xtdb/id “hex-string”, for example, helps to uniform upper/lower case represenatation of things like 0xAB, and 0xab.

refset10:10:27

I think it's more like having to avoid storing a bunch of binary in the test files: https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/core/test/xtdb/codec_test.clj#L125

pinkfrog14:10:13

I wonder if there is any simple way to atomically increment a value without using transaction function. The read-and-write pattern is so common, I wonder if we have write transactoin fucntions for each cases then the process would be quite tedious.

refset15:10:57

In short, no, you can only use transaction functions in these read-then-write cases, although I imagine that a utility lib with a suite of helpful transaction functions could be valuable for the community.

pinkfrog15:10:27

This got me confused. So for all “update” operations, a dedicated function is needed. It seems it becomes impractical to use xtdb for these use cases. Any example applications that use xtdb for reference?

refset16:10:10

Not necessarily, it depends on the semantics and consistency you want (e.g. you can use match + retries instead). In case it helps to know though, you can define and use a transaction function inline easily enough https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/test/test/xtdb/tx_test.clj#L653-L664

refset16:10:56

Your transaction functions can also be super generic and reusable - you could create something like :eval-fn (though I'm not recommending that option, it only just occurred to me 🙂)

pinkfrog16:10:45

What do you mean by eval-fn?

pinkfrog16:10:18

inlining a function takes much repetitive storage for the function itself (dunno if it is a concern though, probably).

refset16:10:32

something like

[::xt/put {:xt/id :eval-fn
            :xt/fn '(fn [ctx form]
                      ((eval form) ctx)}]

😳 1
😬 1
pinkfrog16:10:57

Basically it is the same as writing an inline function as you still have to specify the form. Is storage ever a concern to you ?

refset16:10:02

since arg-docs get replaced by their evaluation, this might not actually be completely crazy as a way to efficiently (storage-wise) do things inline :thinking_face:

refset16:10:56

> Is storage ever a concern to you ? Of course(!) Although I don't like to constrain ideas & suggestions too prematurely

refset16:10:13

How big is your data set?

pinkfrog16:10:27

> since arg-docs get replaced by their evaluation, this might not actually be completely crazy as a way to efficiently (storage-wise) do things inline Are you saying, not only the function invocation argument (i.e., the [::xt/fn xxx]) part could be replaced with the actual transaction function output, but also the inline transaction function itself can be safely erased since we already get the output. This indeed is an optimization.

1
refset10:10:55

Yep, exactly that 🙂

Marc Nickert17:10:08

Hey, is it Possible to Store next to a transaction which user triggered it?

refset10:10:10

Hey @U02DYDLDZGE not as a first-class API, but you can store this information in an additional document each time (e.g. :tx-meta-entity) and correlate the transaction time(/id) when querying, or you could use a transaction function to embed the transaction time(/id) inside that that entity

refset10:10:53

There are perhaps other use-cases for a first-class API in future though, see https://github.com/xtdb/xtdb/issues/253