Fork me on GitHub
#xtdb
<
2022-09-14
>
Dameon02:09:34

In a piece of data with an instance {:xt/id id, :item/timestamp #inst} is there an easy way to get the data with the latest timestamp easily? I only can think of order-by with limit 1, but maybe there’s a better way to do it.

Dameon18:09:35

I read those yeah. I’ve tried everything but the inverted long technique. It seems a little hacky and I don’t want to pollute my data model if I don’t have to. It’s also tricky because there are multiple records per item and I only care about the latest record per item. So even the order-by doesn’t work because it gets every last record with the timestamp. I might need to rethink how I’m storing things.

Luis Santos21:09:29

I've been trying without success to pass the proj argument to the query. What am I doing? Are the variables only available during the query step?

(defn find-entity-by-id [node id proj]
  (xt/q (xt/db node)
        '{:find [(pull e proj)]
          :where [[e :xt/id id]]
          :in [id proj]}
        id proj))

refset21:09:05

Hey @UK00KCK6D > Are the variables only available during the query step? Specifically the :where clauses, but yes essentially that's the issue. The pull parameters can't be dynamic like that, currently.

refset21:09:47

You can do this though:

(defn find-entity-by-id [node id proj]
  (xt/q (xt/db node)
        {:find [(list 'pull 'e proj)]
         :where '[[e :xt/id id]]
         :in '[id]}
        id))

Luis Santos21:09:17

Yes. I just tried that without success. Let me try again. 😄

🙂 1
refset21:09:13

Awesome - glad to hear that !

refset21:09:25

Anytime ☺️

Luis Santos21:09:43

@taylor.jeremydavid Where can I find good resources about modeling data in XTDB? I've noticed that there many things to take into consideration such as: • Should I use namespaced keys? • In a "1 to 1 relationship" should I store everything in a nested structure or model the relationship with "FK"? ◦ Should both records reference each other or should we model the relationship in one way? ◦ If they have a 1 to 1 relationship why not store everything in the same "entitiy" • Should I use keys or strings as id? • Should I use composite keys like "user-120" and "user-120-address" or should use simple keys? • What's the impact of these decisions on performance? Most document stores have limited joined capabilities therefore some of these questions are obvious, but with XT everything feels new and unknown. I guess someone could write a book about it. 😄

refset21:09:35

Well, you're in a good place to find some answers 🙂 Here are some personal opinions for now. To tick off a few on your list a little first... > • Should I use namespaced keys? Almost certainly a good idea > • Should I use key[word]s or strings as id? Internally they're treated very similarly (so there's no performance angle to the decision) but strings probably win for "simplicity". Don't use stringified UUIDs though, as actual UUIDS will be much more efficient. Also reserving use of keywords for IDs could make it simpler to represent and store metadata about actual keys in regular entities > • What's the impact of these decisions on performance? Many-fold! The main thing to appreciate though is that more joins = worse performance. In general the tension between performance and modelling is really hard, and XT brings fewer opinions to the table than most things, which means even more opportunity be creative (both a blessing and curse) > I guess someone could write a book about it. Definitely! I don't find nearly enough time to experiment with modelling tradeoffs in practice to have hugely strong opinions, but I do want to share that I think there's something about the intersection of these things which really excites me: • Alloy (and other formal methods) https://www.hillelwayne.com/post/formally-modeling-migrations/ • Malli, particularly because it can be trivially serialised into the database https://github.com/metosin/malli (Pathom also, although I'm less familiar... @U797MAJ8M advocates for it well though) • A rejection of entity-relationship modelling altogether https://en.wikipedia.org/wiki/Object-role_modeling What I value most in XT is that it leaves us enough room to think about what "better" might look like (vs. DBs that impose their own notion of schema management)

👍 1
Panel03:09:32

I'm curious about the malli + xtdb use case. Maybe we could have a schema that map entity and generate query from that… Maybe that's what the ancient called an ORM I'm not sure…

Mikko Harju04:09:41

> Maybe that's what the ancient called an ORM I'm not sure… I had a vivid recollection I did something like this a while ago and sure enough found something like this :D https://gist.github.com/mharju/2770f61bfbeb4219ebfa3a30d2a7fe78

👍 1
Mikko Harju04:09:16

I recall that m/decode could be made into a def and be used as a function for performance gains 🙂

zeitstein08:09:58

> Don't use stringified UUIDs though, as actual UUIDS will be much more efficient. Never thought of this, thanks! I've been using stringified UUIDs which also encode the type of the document. (The thinking is this would be more performant than using a map, and more future-proof.)

🙂 1