Fork me on GitHub
#xtdb
<
2021-04-01
>
FiVo09:04:08

Can one somehow :order-by latest :crux.tx/tx-time in crux/q?

refset12:04:03

Hi, it's possible, yep! However you may need to be cautious if you are expecting valid-time sensitive queries to also be "consistent" with the db value. See this custom predicate:

(defn get-tx-time [db e]
  (-> (crux.api/entity-history db e :desc)
      first
      :crux.tx/tx-time))

(def n (crux/start-node {}))
(->> (crux/submit-tx n [[:crux.tx/put {:crux.db/id 1 :foo :bar}]])
     (crux/await-tx n))
(->> (crux/submit-tx n [[:crux.tx/put {:crux.db/id 2 :foo :baz}]])
     (crux/await-tx n))

(crux/q (crux/db n)
 '{:find [e tx-time]
   :order-by [[tx-time :asc]]
   :where [[e :crux.db/id]
           [(my-ns/get-tx-time $ e) tx-time]]})

Jonas Svalin10:04:26

Hi there! I’m exploring Crux as an alternative for an event-sourced system. Suppose I have a user entity which looks like this:

{:crux.db/id :123
 :name "Bob"}
…and that I want to set the email of this user entity. In previous systems I’ve worked on which did event-sourcing, events were our primary source of truth, and so we would write an event roughly stating type: email-updated, user: :123, email:

Jonas Svalin10:04:30

I did some googling, and came upon a part of the bitemporal tale stating: https://nextjournal.com/crux/a-bitemporal-tale

Note that transactions in Crux will rewrite the whole entity, there are no partial updates and no intentions to put them in the core as of yet. This is because the core of Crux is intentionally slim, and features like partial updates shall be kept in the upcoming convenience projects!

Jonas Svalin10:04:43

Now my question is, in Crux as it stands now, is there a way to perform the operation I mentioned above without loading the existing entity, associng the email property into it and then submitting the transaction?

refset13:04:38

Hey! Did you already see transaction functions? https://opencrux.com/blog/crux-transaction-functions.html Under-the-hood there is a still read-assoc-submit flow of the entity happening, but it is much more convenient than trying to do it in userland with transactional match operations and retries with exponential back-off etc.

❤️ 3
refset13:04:55

Transaction functions didn't exist when the tale was written, partly because we hadn't yet solved the problem of how to implement them safely 🙂

Jonas Svalin13:04:28

That looks to be right up my alley! Will explore it in more detail, thanks @U899JBRPF!

🙏 6
seeday21:04:14

@U01SV1FBY6A if you’re still looking into this, I came up with some event sourcing/partial write transaction functions a while ago and would be willing to share/interested in what you came up with

seeday21:04:50

As it stands, the hardest part is efficiently dealing with writes with a valid-time in the past. You end up having to walk the entity history which I’m sure is slowwww

refset17:04:24

@U012QF1QVQV when you say "walk the entity history" - do mean walk and process/reduce over it? Most use-cases for history in Crux should be generally fast (i.e. as fast as dealing with the ~present), since we have a dedicated temporal indexes, if you are finding otherwise then perhaps we're doing something wrong or missing a trick 🙂

seeday17:04:11

Yeah, walk and process/reduce. Slow is entirely relative as I’m sure it’s still pretty speedy, but open-entity-history and the logic to merge the changes has to take longer than your average crux.tx/put transaction

👍 3
seeday17:04:11

Absolutely no complaints about the speed of accessing historical data

🙏 3
Jonas Svalin07:04:17

@U012QF1QVQV that’s extremely kind of you! I was able to solve my “experimental” use case using a transaction function, for now I’m still just doing some due diligence and so haven’t started looking into the more advanced use cases yet. If something like that comes up I will absolutely ping you for some examples, thanks again!

👍 3
Jonas Svalin10:04:17

And if not in Crux itself, does there now exist any convenience projects as mentioned in the bitemporal tale which would allow for something like that?

jarohen17:04:39

Evening folks - 1.16.0 is out! 🚀 * HTTP/2 support in the Crux HTTP server * Support for NIO FileSystem document- and checkpoint-stores - opening up NFS and Google Cloud Storage * eql/project has been stabilised and renamed as pull to be more consistent with other Clojure Datalog databases * The usual array of bug fixes and performance improvements - see the release notes for more details https://github.com/juxt/crux/releases/tag/21.04-1.16.0 As always, a big thanks to everyone who's contributed to this release by submitting PRs, raising issues and helping with repros! 🙏

🎉 48
crux 39