datomic 2021-04-14 | Slack Archive

tatut12:04:09

has there been any news on cloud disaster recovery, the discussions don’t seem active https://forum.datomic.com/t/cloud-backups-recovery/370/10 we currently have a hand rolled tx log backup/restore solution but it’s a bit painful to maintain

kenny15:04:23

Maybe you're aware already but there's a feature request on ask.datomic: https://ask.datomic.com/index.php/431/cloud-backup-and-recovery

tatut05:04:02

yes, I’ve upvoted that but it doesn’t seem to have more info

kenny22:04:19

The recommendation for enums in Datomic is to https://docs.datomic.com/cloud/schema/schema-modeling.html#enums. Using :db/ident as an enum value complicates the use with d/pull. Everywhere you pull an entity, you'll need to write code to deal with the nested enum entity. For example, say I have a :user/type enum that is defined as suggested by that doc (card one, ref attr). Anytime I pull my :user/type attribute, I need to write code to unnest the :user/type value.

(d/pull (d/db conn) '[:user/type] [:user/id "a"])
=> #:user{:type #:db{:id 87960930222153, :ident :user.type/a}}

How are folks dealing with this? Just always wrapping the d/pull return with something like (update :user/type :db/ident)? Perhaps always remembering to specify the pull pattern for all enum attributes as (:user/type :xform my-ns/get-db-ident), where my-ns/get-db-ident is just (def get-db-ident :db/ident)?

Tyler Nisonoff23:04:31

one way i was doing this was to take my nested entity thats returned from the pull and then run it through a post-walk like so:

(clojure.walk/postwalk
    #(match %
       {:db/ident ident} ident
       :else %)
    nested-entity)

if you know that all maps with db/ident are referencing enums you want to unwrap

👏 3

kenny23:04:27

Oh, nice! That's a pretty heavy penalty to pay for using enums as idents though.

thumbnail05:04:52

At work we tried both suggestions, and switched from the postwalk approach to the xforms. Mostly because of performance. We tend to def our pull expressions for re-use, so the xforms are in 1 place. Needing to remember it's a db/ident isn't much of a problem for us, it's required to pull db/ident anyway. But one could always write a linter make sure that xforms is used for idents.

tatut08:04:18

is walk so slow? I’ve used it heavily and never had it be a problem in my work loads

thumbnail09:04:53

We had a specific usecase where we had very big resultsets (~150,000 entities + nesting). That's when we replaced the postwalk. Generally it's fine for sure.

😅 6

tatut12:04:13

that is a big result, out of curiosity, how much was the difference between walk and xform with that result?

thumbnail16:04:28

Unfortuinely I don't have exact numbers. We also optimised the query at that time.

kenny16:04:11

We also have very big query results.

thumbnail21:04:58

To give a very rough idea about our postwalk approach vs a direct update fn;

(time
  (do (map (fn [x] (update x :field :db/ident))
        corpus)
    nil))
"Elapsed time: 0.093924 msecs"
=> nil
(time
  (do (walk/postwalk (fn [x]
                       (if (and (map? x)
                             (:db/ident x))
                         (:db/ident x)
                         x))
        corpus)
      nil))
"Elapsed time: 552.667018 msecs"
=> nil

(corpus is a list of 150,000 maps btw)

👍 3

tatut05:04:23

interesting, thanks

tatut05:04:43

but isn’t the first one not a good benchmark, as map returns lazy seq

😅 3

thumbnail12:04:15

You're absolutely right. I knew something was up.

thumbnail13:04:48

(time
  (do (mapv (fn [x] (update x :field :db/ident))
        corpus)
      nil))
"Elapsed time: 142.214586 msecs"
=> nil

For completionist sake:

kenny21:04:10

Thought I'd persist this question on ask.datomic so others can reference it and add their opinion: https://ask.datomic.com/index.php/606/recommended-way-to-handle-db-ident-enums-when-used-with-pull

👍 1

2021-04-14

Channels