datomic 2023-06-29 | Slack Archive

onetom15:06:53

On Datomic Cloud, how can I convert a basisT value (obtained from a :db-after value, with the basis-t function below), into a wall clock time? Do I have to dig :db/txInstant out of the :data attr of a d/tx-range return value?

(defn basis-t
  "Return the basis T of a db-val, regardless of its concrete implementation."
  [db-val]
  (or (-> db-val :basisT)               ;; (d/client {:server-type :dev-local})
      (-> db-val :t)                    ;; (d/client {:server-type :cloud}) or :ion
      ))

onetom15:06:11

I do have something like this at the moment, which is specific to our use-case:

(def ^:datomic/q q:latest-sync-tx-instant
  '{:find  [?tx-time]
    :in    [$ ?source-ref]
    :where [[?source-ref :source/latest-sync _ ?t]
            [?t :db/txInstant ?tx-time]]})

but it would be nice, if there would be some more obvious way to do this, since Datomic Pro does have something like this, iirc.

cch115:06:45

Here’s what I would do for Datomic Cloud:

(let [{conn :datomic/connection} @user/sysref
            {:keys [db-after] [d & tx-data] :tx-data :as result} (d/transact conn {:tx-data [{:st.legal-person/name "Chris Cyrus Hapgood"}]})]
        (d/pull db-after '[*] (:tx d)))

This relies on the fact that all transacted datoms share the same (reified) transaction entity, and that the basisT/`:db/id` of the tx are https://docs.datomic.com/client-api/datomic.client.api.html#var-datoms and that entity carries the tx-instant.

onetom16:06:09

this also works, btw, but it requires a datomic connection, instead of a database value:

(defn tx-inst [conn basis-t]
  (-> (d/tx-range conn {:start basis-t :end (inc basis-t)})
      first :data
      (->> (filter (fn [{:keys [e _a v tx]}] (and (= e tx) (inst? v)))))
      first :v))

example:

(tx-inst conn 10)
=> #inst"2023-06-29T12:29:57.921-00:00"

onetom16:06:34

it finds the 1st instant valued datom of a transaction entity from the transaction :data retrieved via d/tx-range, based on a basisT. it would be more precise to just assert, that the attribute of the datom is :db/txInstant, but i would need the entity ID of that attribute and not sure whether that's always the same in every database or not.

onetom16:06:58

i only have the db value as-of the basisT, so i don't have tx-data at hand, but i know, that the last tx was the one, which i want to see the :db/txInstant for.

cch116:06:24

I don’t think you can safely make any assumptions about schema attr ids… they might even change if the base schema ever gets updated. Why not just pull the transaction entity directly?

onetom16:06:42

given a db value as-of some time, i can't tell, what's the last transaction's entity id, without having access to the db connection too

onetom16:06:02

well... i guess i have to do this lookup earlier in my transformation pipeline, where i still have access to the conn

enn15:06:27

I had a rule with two implementations that I rewrote as an or-join, which I understand to be semantically identical. The rule implementation works as expected but the or-join implementation throws an IndexOutOfBoundsException from within datomic.datalog/join-project-coll-with. 🧵

enn15:06:11

rule implementation:

[(filter-group-roadmap-container-by-teams [?group] ?entity-id ?epic)
 [(!= :none ?group)]                      
 (or-join [?group ?entity-id]             
   [?entity-id :group-roadmap-container/group ?group]
   (and
    [?roadmap-entry-id :epic-roadmap-entry/container ?entity-id]
    [?roadmap-entry-id :epic-roadmap-entry/epic ?epic]
    [?epic :epic/group ?group]))]

[(filter-group-roadmap-container-by-teams [?group] ?entity-id ?epic)
 [(= :none ?group)]
 (or-join [?group ?entity-id]
   (not [?entity-id :group-roadmap-container/group])
   (and
    [?roadmap-entry-id :epic-roadmap-entry/container ?entity-id]
    [?roadmap-entry-id :epic-roadmap-entry/epic ?epic]))]

or-join version:

'[(or-join [?group ?entity-id ?roadmap-container-epic]
    (and
     [(!= :none ?group)]
     (or-join [?group ?entity-id ?roadmap-container-epic]
       [?entity-id :group-roadmap-container/group ?group]
       (and
        [?roadmap-entry-id :epic-roadmap-entry/container ?entity-id]
        [?roadmap-entry-id :epic-roadmap-entry/epic ?roadmap-container-epic]
        [?roadmap-container-epic :epic/group ?group])))
    (and
     [(= :none ?group)]
     (or-join [?group ?entity-id  ?roadmap-container-epic]
       (not [?entity-id :group-roadmap-container/group])
       (and
        [?roadmap-entry-id :epic-roadmap-entry/container ?entity-id]
        [?roadmap-entry-id :epic-roadmap-entry/epic ?roadmap-container-epic]
        (not [?roadmap-container-epic :epic/group])))))]

enn15:06:12

Error/stacktrace:

processing rule: (q__549760 ?entity-id), message: processing clause:
   clojure.lang.LazySeq@bdc436e2, message: processing rule: [arule__539777
   ?group:0:3 ?entity-id ?roadmap-container-epic], message: processing clause:
   clojure.lang.LazySeq@bda6aac3, message: java.lang.IndexOutOfBoundsException

               datalog.clj: 1483  datomic.datalog/eval-rule/fn
               datalog.clj: 1463  datomic.datalog/eval-rule
               datalog.clj: 1442  datomic.datalog/eval-rule
               datalog.clj: 1506  datomic.datalog/eval-query
               datalog.clj: 1489  datomic.datalog/eval-query
               datalog.clj: 1596  datomic.datalog/qsqr/fn
           query_stats.clj:   61  datomic.measure.query-stats/with-phase-stats
           query_stats.clj:   48  datomic.measure.query-stats/with-phase-stats
               datalog.clj: 1595  datomic.datalog/qsqr
               datalog.clj: 1534  datomic.datalog/qsqr
               datalog.clj: 1552  datomic.datalog/qsqr
               datalog.clj: 1534  datomic.datalog/qsqr
                 query.clj:  757  datomic.query/q*
                 query.clj:  744  datomic.query/q*
                 query.clj:  785  datomic.query/query*
                 query.clj:  778  datomic.query/query*
                 query.clj:  805  datomic.query/query/f
              io_stats.clj:   93  datomic.io-stats/with-raw-io-stats/fn
              io_stats.clj:   93  datomic.io-stats/with-raw-io-stats
              io_stats.clj:   60  datomic.io-stats/with-raw-io-stats
              io_stats.clj:  132  datomic.io-stats/with-io-stats
              io_stats.clj:  126  datomic.io-stats/with-io-stats
                 query.clj:  807  datomic.query/query/fn
                 query.clj:  812  datomic.query/query
                 query.clj:  798  datomic.query/query
                   api.clj:   48  datomic.api/query
                   api.clj:   46  datomic.api/query

enn15:06:43

What is this error trying to tell me?

favila16:06:58

IOOBE usually means a symbol reference is missing

enn16:06:55

missing in the sense of unbound?

enn16:06:18

or in some deeper sense?

favila16:06:19

in the sense of it’s not pushed down, or it attempts to unify against something that isn’t in the outer form

favila16:06:35

e.g maybe the “not” needs to be a not-join?

enn16:06:41

maybe it’s not pushing down from the outer or-join to the inner or-join again

favila16:06:10

I would prune the tree to figure out what part it doesn’t like

enn16:06:40

it’s the first branch that seems to be the issue. If I take ?roadmap-container-epic out of the first nested or-join but leave it in the second and in the top-level or-join, it runs

favila16:06:00

oh, there is one difference, not sure it matters, but the rule requires a binding but the or-join does not

enn16:06:31

right … which makes it even more odd that the or-join would throw in a situation where the rule does not. since if anything it should be more tolerant in its expectations.

favila16:06:34

you can do (or-join [[?group] ?entity-id ?roadmap-container-epic] …) to replicate

enn16:06:48

enn16:06:33

same exception with that syntax

favila16:06:41

it could also be a qslice bug: ?group:0:3 — is that rewritten correctly in the generated query? We can take offline if so

enn16:06:26

it looked correct to me but that could be part of it

2023-06-29

Channels