Fork me on GitHub
#datomic
<
2023-08-23
>
kuzmin_m12:08:16

Hi! I can’t find info about keys: :apply-msec, :api-ms, :msec . What exactly do they mean?

INFO default  datomic.transaction - {:tid 119, :txid #uuid "64e5f5e8-dfe9-4b8f-9cd6-aebce505ffe7", :datom-count 3, :apply-msec 4.16, :io-stats {:api :tx-with, :io-context :db/tx, :api-ms 4.1, :reads {}}, :pid 1, :event :tx/process, :msec 22.4, :t 46982}

jaret15:08:45

@U1LR4AN75api-ms represents the time spent processing the transaction including tx-with time. • msec is the transactor time or everything that can be measured from the transacting process (i.e. everything but interprocess communication). • apply-msec in the context of a transaction is functionally the same as api-ms, I would not rely on this number outside of a support context. We have a future metric coming down the line which will show the Total apply-msec of transactions over 1 minute (process monitor log). https://docs.datomic.com/pro/api/io-stats.html#relating

👍 2
datomic 2
Black19:08:35

Hi! I am trying to write function which will return history of given entity, the problem is, that I have some refs that don't have to exists, is there any easier way than I am doing? My code is:

(defn find-history
  [db request]
  (->> (d/q '[:find ?tx ?update ?message ?state
              :in $ ?e
              :where
              [?e _ _ ?tx true]
              [?tx :db/txInstant _]
              [?e :request/updated-at ?update ?tx true]
              [?e :request/message ?message ?tx true]
              [?e :request/state ?state-eid ?tx true]
              [?state-eid :db/ident ?state]]
            (d/history db) [:request/id (:request/id request)])
       (map #(->> %
                  (map vector [:tx :update :message :state])
                  (into {})))
       (map (fn [e]
              (let [owner
                    (d/q '[:find ?owner .
                           :in $ ?e ?tx
                           :where
                           [?e _ _ ?tx true]
                           [?tx :db/txInstant _]
                           ;; :request/owner can be nil
                           [?e :request/owner ?owner ?tx true]]
                         (d/history db) [:request/id (:request/id request)]
                         (:tx e))]
                e)))
       (sort-by :tx)))

Black19:08:01

Ok I have changed the function so it takes everything from transaction and now it contains only changed attributes:

(defn find-history
  [db request]
  (->> (d/q '[:find ?tx ?attr ?value
              :in $ ?e
              :where
              [?tx :db/txInstant _]
              [?e ?attr ?value ?tx true]]
            (d/history db) [:request/id (:request/id request)])
       (map #(->> %
                  (map vector [:tx :attr :value])
                  (into {})))
       (group-by :tx)
       (map (fn [[tx xs]]
              (reduce (fn [acc {:keys [attr value]}]
                        (let [attr-ident (d/ident db attr)
                              value (if  (= :request/state attr-ident)
                                      (d/ident db value)
                                      value)]
                          (assoc acc attr-ident value)))
                      {:tx tx}
                      xs)))
       (sort-by :tx)))
Is there better way to retrieve refs then with extra d/ident call?

favila20:08:48

supply attrs as input?

favila20:08:01

This what you want?

(->> (d/q '[:find ?tx ?attr ?value
            :keys tx attr value
            :in $ ?e [?attr ...]
            :where
            [?e ?attr ?value ?tx true]]
          (d/history db) (find request :request/id) [:request/state])
     (sort-by :tx)
     (into []
           (comp
            (partition-by first)
            (map (fn [xs]
                   (into {:tx (:tx (peek xs))}
                         (map (juxt :attr :value))
                         xs))))))

favila20:08:58

You don’t really need query here either, (d/datoms history-db :aevt attr entity) gives you the same

souenzzo09:08:10

off-topic: (find request :request/id) => [:request/id 123] 🙂

Black11:08:26

@U09R86PA4 I wanted to achieve that result will be list of transactions where at least :request/state was transact and in each transaction I want to include all changed attributes

favila12:08:21

So request/state changes tells you what txs to inspect, then you want everything from that tx?

favila12:08:42

If so then the second phase should use the tx log

favila12:08:03

Or maybe not? Maybe you just want what changed on that entity?

favila12:08:30

Either way, d/ident is the fastest way to get a keyword of an attr if you have a number. The greater concern is whether your queries are fast, clear, and correct

favila12:08:03

For example, if you just want what was asserted with transactions which also asserted a change to request state, this may be better (assuming all attrs are card-one):

(->> (d/q '[:find ?tx ?attr ?value
            :keys tx attr value
            :in $ ?e
            :where
            [?e :request/state _ ?tx true]
            [?e ?attr-id ?value ?tx true]
            [(datomic.db/ident $ ?attr) ?attr]]
          (d/history db) (find request :request/id))
     (sort-by :tx)
     (into []
           (comp
            (partition-by first)
            (map (fn [xs]
                   (into {:tx (:tx (peek xs))}
                         (map (juxt :attr :value))
                         xs))))))

favila12:08:53

However if you want what the entity looked like before or after request-state transaction (i.e. including attrs that didn’t change), then you’re doing a pull with d/as-of on each tx you find.

favila12:08:39

or if you want everything that happened in the tx to any entity, you want to use tx-range to get the log entry for those txs which had request state changes

Joe R. Smith19:08:44

Getting errors back from a Datomic Ion push, but I cannot see the error due to this unmarshalling exception. I'm running Java 17.0.1 and the latest batch of Datomic Cloud libs. I have no AWS SDK dependencies in my code.

{:command-failed "{:op :push}",
 :causes
 ({:message
  "Unable to unmarshall exception response with the unmarshallers provided (Service: AWSResourceGroupsTaggingAPI; Status Code: 400; Error Code: UnrecognizedClientException; Request ID: 617bc07d-7b77-4744-a3a5-84809f3fcf5f; Proxy: null)",
  :class AmazonServiceException})}

Joe Lane21:08:06

Downgrade your jvm then try again and post the stack trace of that doesn’t answer what’s going wrong

andrewhr21:08:25

May worth to mention: Ions run on https://docs.datomic.com/cloud/ions/ions-reference.html#jvm-settings, in case you ask "which JVM I should run?" Hope that is enough to fix your issue! 🤞

2
Joe R. Smith18:08:04

I'm actually not worried about what the error is, I'm more interested in solving the unmarshalling error, in general. I was hoping it wouldn't involve downgrading my JVM.

Joe Lane19:08:38

@U087E7EJW check the bottom section of this page for flags to pass to the jvm on startup to bypass this issue if you can’t change jdk versions. https://github.com/aws/aws-sdk-java#maintenance-and-support-for-java-versions

👀 2
Joe R. Smith20:08:42

tried that previously: Unrecognized option: --add-opens java.base/java.lang=ALL-UNNAMED

Joe R. Smith20:08:58

added to my deps' :jvm-opts