datomic 2021-07-30 | Slack Archive

Daniel Jomphe15:07:35

👋 Hi! Does Datomic change how we seed app-supporting data when we develop new features in an app? Normally we'd use some way of migrating schema and data to support those new features, without re-seeding the entire schema and app-level data. We'd probably track in a DB property at which version of the seed we're at. I know there are non-breaking ways of evolving the Datomic schema, but I suppose there are no such ways of evolving data seeds that can be wholly re-transacted idempotently. In other words if we run the data seed many times, we'll end up with multiple copies of the e.g. admin user, and other business-related entities, etc. So, does Datomic have any properties that help us in this area? Thanks!

tvaughan15:07:40

I wrote something to handle this.

(defn tx!
  [conn data]
  (transact conn {:tx-data data}))

(defn tx-resource!
  [conn resource]
  (tx! conn (resources/read-resource resource)))

(defn- tx-status
  [conn tx-id]
  (-> (conn->db conn)
      (q-by-ident [:tx/id tx-id] [{:tx/status [:db/ident]}])
      :tx/status
      :db/ident))

(defn- tx-apply!
  [conn {:keys [tx-id tx-data]}]
  (tx! conn (conj tx-data {:tx/id tx-id :tx/status
                           :tx.status/applied})))

(defn- tx-applied?
  [conn tx-id]
  (case (tx-status conn tx-id)
    :tx.status/applied true
    nil))

(defn tx-idempotent!
  [conn resource]
  (let [{:keys [tx-id tx-data tx-kind] :as props} (resources/read-resource resource)]
    (when-not (tx-applied? conn tx-id)
      (case tx-kind
        :fn (do
              (require (symbol (namespace tx-data)))
              (tx-apply! conn (assoc props :tx-data ((resolve tx-data) conn))))
        (tx-apply! conn props)))))

On start-up, we always transact this (not idempotent):

[{:db/ident :tx/id
  :db/cardinality :db.cardinality/one
  :db/valueType :db.type/keyword
  :db/unique :db.unique/value}
 {:db/ident :tx/status
  :db/valueType :db.type/ref
  :db/cardinality :db.cardinality/one}
 {:db/ident :tx.status/applied}]

Then our schemas and seed data get wrapped like:

{:tx-id :migration-0001
 :tx-data
 [{:db/ident :editor-session/pid
   :db/cardinality :db.cardinality/one
   :db/valueType :db.type/string
   :db/unique :db.unique/value}]}

Which are transacted on the command-line (idempotent) and can be added to a start-up script, like a systemd unit

Daniel Jomphe15:07:06

Wow, this does seem simple, solid and easy! Thanks for the answer!

👍 2

Daniel Jomphe15:07:57

So, thinking about it, I see that Datomic does help by the fact that transactions operate on data that can be easily pre-processed before being committed. This solution ends up being generic and very concise.

hden13:07:54

Hi, I’ve made a duct module for this purpose. It use ragtime to implement the same pattern. https://github.com/hden/duct.module.datomic#integrant-keys

Daniel Jomphe13:07:23

Hi, I wasn't aware there exists a migrator adapted to Datomic Cloud, thanks for sharing this!

😄 2

Drew Verlee20:07:31

I'm trying to troubleshoot why i can't connect to datomic cloud:

➜  datomic-cli git:(master) ✗ ./datomic-access client grow
download:  to ../../.ssh/datomic--grow-bastion
download:  to ../../.ssh/datomic--grow-bastion.hostkey
system grow
REGION_ARG 
PR  
s3: grow-storagef7f305e7-qyyc6pldu9b0-s3datomic-1cz3x1k0cdzby
pk: /home/drewverlee/.ssh/datomic--grow-bastion
ip: None
hk: /home/drewverlee/.ssh/datomic--grow-bastion.hostkey
ssh marker
SSH  -o IdentitiesOnly=yes
OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 22: Applying options for *
ssh: Could not resolve hostname none: Temporary failure in name resolution

The shell command that datomic-acess builds is

ssh -v -o UserKnownHOstsFile=/home/drewverlee/.ssh/datomic--grow-bastion.hostkey -o IdentitiesOnly=yes -i /home/drewverlee/.ssh/datomic--grow-bastion -CND 8182 ec2-user@None

The lat part ec2-user@None would seem to be the issue as I assume "None" should be an IP fetched by the script function gateway_ip. I looked at the gatewate_ip function and it's being given the only required arg which is the system, so i'm not sure why it would return none. Any ideas?

Joe Lane20:07:05

@U0DJ4T5U1 I'm not sure what your datomic-access script is doing but the access command should be datomic client access <system> per https://docs.datomic.com/cloud/getting-started/get-connected.html#access-gateway HOWEVER!! that only matters if you're not on the latest release. If you're on the latest release this changes. Let me know if I can help more.

Drew Verlee20:07:27

Ah, ok. I am on the latest. I'll recheck the docs. I thought I caught everything. I'll look tomorrow :)

Joe Lane21:07:30

The access gateway doesn't exist in latest 🙂 Enjoy

Daniel Jomphe22:07:42

...but you need to update your client config endpoint url with the new url.

2021-07-30

Channels