datomic 2022-02-21 | Slack Archive

Ben Hammond09:02:01

when I use a datomic pull with a :limit https://docs.datomic.com/cloud/query/query-pull.html#limit-option do the results get returned in a stable order? Ideally I would like to limit my results to the 10 most recent entities: Can I express that just with a pull spec?

kipz10:02:16

The order does seem to be stable - I expect it reflects how the query is executing over the indexes. As far as paging and sorting results is concerned, that isn't supported as far as I know. We use index-pull to query specific indexes (in your case a date), then filter the results, then pull more until we reach our limit. It's not pretty, but it's what we have right now. In some other cases, we maintain the equivalent of "views" - such as a reference to the "entity with latest X where X.attribute = blah". We know we want to track this at transaction time, so it's no big deal. It's just a shame that we have to sully our otherwise fully normalised data model. For me, this is the most frustrating area of datomic because it's such a common use case for us. There was some talk of this here: https://forum.datomic.com/t/idiomatic-pagination-using-latest-features/1454/5 but it's gone quiet

👀 1

👍 1

Ben Hammond11:02:58

I'm hoping that I can order the entities by Entitiy Id Because Entity Id is monotonically increasing over time

Ben Hammond11:02:08

Is that true on datomic cloud?

souenzzo14:02:36

@U793EL04V maybe you can use https://docs.datomic.com/cloud/query/query-index-pull.html

👍 1

Ben Hammond14:02:13

oooh may be I can... thanks for the hint

xceno20:02:56

I'm wondering about a good way for storing the following:

{:rule/id                     1234567
 :rule/rule-name              "Some rule name"
 :rule/predicate-fn           `when
 :rule/predicate-test-fn      `entity-equals 
 :rule/predicate-test-fn-args {:entity [:some.thing/id #uuid"cfea961d-e5cf-4d38-9633-2eb22ba09e5a"]}
 :rule/action                 `add-properties! 
 :rule/action-args            {:properties {:some-kw 123}}}

I've built a little system that takes a bunch of these maps and assembles some clojure code from them. I'm expecting to have a couple hundred of these maps at max. Now, I'd like to store these maps somewhere. The first idea was to put them into datomic as a string and read them back in later, but searching through slack revealed https://clojurians.slack.com/archives/C03RZMDSH/p1613255057226100?thread_ts=1613250450.201700&cid=C03RZMDSH(?). So, would I be better off storing them as .edn files in S3? I don't want or need to pull them apart into separate datoms, nor do I need to search their content.

Joe Lane20:02:12

Hey @U012ADU90SW, I would consider this scenario to be a bit different than Gene Kim's. Do you expect all maps to be within an order of magnitude of the size you showed above? Ironically, you're extremely close to having the datomic schema for your rule entity. Is there some reason you don't want to just make them regular entities?

xceno21:02:45

> Do you expect all maps to be within an order of magnitude of the size you showed above? yeah they shouldn't grow much bigger than that > [...] Is there some reason you don't want to just make them regular entities? Well, I guess that's true! I'm mostly worrying about the :rule/....-args fields which could contain anything. So when saving them as regular entities i'd need to convert these to strings anyway, that's why I figured I could as well just serialize the whole thing. In my mind having a rule split into EAV parts doesn't make much sense because I always need all fields anyway. And I wouldn't "clutter" datomic with these attributes, but maybe that thought is misguided

2022-02-21

Channels