datomic 2024-01-04 | Slack Archive

Hey everyone, Let’s say i have some tree schema:

(def schema [{:db/ident :node/id
              :db/valueType :db.type/string
              :db/cardinality :db.cardinality/one
              :db/unique :db.unique/identity}

             {:db/ident :node/value
              :db/valueType :db.type/string
              :db/cardinality :db.cardinality/one}

             {:db/ident :node/children
              :db/valueType :db.type/ref
              :db/cardinality :db.cardinality/many
              :db/isComponent true}])

and the nodes:

(def nodes [{:node/id "1"
             :node/value "node 1"
             :node/children [{:node/id "2"
                              :node/value "node 2, child of node 1"}
                             {:node/id "3"
                              :node/value "node 3, child of node 1"}]}])

now, i want to change the nodes to this:

(def nodes [{:node/id "1"
             :node/value "node 1"
             :node/children [{:node/id "3"
                              :node/value "node 3 edited, child of node 1"}
                             {:node/id "4"
                              :node/value "node 4 edited, child of node 4"}]}])

what’s the best way to sync the db on the changes: • removing the children node 2 • editing the node 3 value • adding another child node 4 to node 1 Thanks 🙏

favila13:01:55

When you say “best way”, what are the different “ways” you are thinking of?

Aviv17:01:50

I meant try to avoid writing a diff algorithm, using some assumptions, such as - transacting the same data (the whole tree), won't affect its tx-time for example, changing some of the node values would only affect these attributes. The main issue is the retraction, which is harder, because the new tree won't express the retracted values. Mainly retracting refs (that aren't component)

favila17:01:08

You need to write some sort of diffing algorithm: transactions are expressed as changes.

🙏 1

favila17:01:57

Your life will probably be easier if you ditch the map syntax for transactions

favila17:01:54

Use the more primitive :db/add :db/retract :db/retractEntity operations, or your own transaction functions

👍 1

favila17:01:45

e.g. (update-tx-data db new-root-map) => [tx-op …]

Aviv17:01:32

Yes, that's exactly what I'll do 👌 I was able to write some generic code by querying the schema, and iterating the tree while checking node attributes types (ref + one/many / unique types, etc)

favila17:01:02

In the past I’ve called this the “make it look like this” approach. Be careful about the semantics of missing attributes (does that mean don’t update it, or does that mean retract it?) and ref retractions (does that mean retract the entire entity, or just stop referencing it?)

favila17:01:12

also be careful about race conditions: if you’re not doing all your work in a transaction function, a write may happen after your read, invalidating your diff

favila17:01:28

One universal approach to combat this is to do your read, record what you read, and have a transaction function reassert that it hasn’t changed. This isn’t the most commutative thing possible, but as a blunt hammer it works pretty well

favila17:01:04

e.g. a transaction function [verify-pull db pull-expr id expected-result] , where expected-result is what you got for that same pull when you were preparing the transaction

favila17:01:31

or the same thing for individual e+a: [verify-value db e a expected-vs]

Aviv18:01:27

Awesome, thank you! I've got a plan in mind and will definitely incorporate your suggestions as I start implementing it.

itaied15:01:52

I'm following datomic cloud guide (Ion tutorial) I have completed all of the example tests locally and then tried to deploy my app. The push command has succeeded, but the deploy timed out on ValidateService. Where can I dig in more to understand the issue? I'm a bit lost at this stage

silian15:01:04

I had similar issues and was never able to resolve it (!). This thread might help: https://forum.datomic.com/t/troubleshooting-deploy-status-failed-code-deploy-status-succeeded/2052

itaied15:01:28

they empty ion-config did work. what does it mean? something is broken in my super simple lambda? where can I see why?

silian15:01:55

I wish I could provide better guidance or solution. I never resolved it myself and abandoned Datomic altogether, opting instead for Datahike on top of Postgres db. @U1QJACBUM is part of Datomic Cloud team and may better point you to solution.

itaied15:01:51

I have fixed it! thank you for your replies. The problem was in my function namespace, I had a computation there that created a connection to the db and stored it in a variable. I think the validation didn't like it, because once I converted it into a function it worked

👍 1

jaret15:01:04

@U057T406Y15 You can look at your code deploy logs in your log group. You may have exceptions in your Datomic log group and the code deploy log group which can be hints. https://docs.datomic.com/cloud/operation/monitoring.html#searching-cloudwatch-logs However, if you have already confirmed you can deploy an empty ion and adding back in your lambda causes this failure I would recommend running locally.

jaret15:01:10

with the same memory settings

jaret15:01:13

Ah. You found it.

itaied15:01:09

the thing is, locally it worked. I had a working app locally that failed on the ion cloud

jaret15:01:59

Ack. There might be an error in the Cloudwatch logs i mentioned in your case given you can't connect. Just for a future person who finds this thread: https://docs.datomic.com/cloud/ions/ions-reference.html#jvm-settings And https://docs.datomic.com/cloud/ions/ions-monitoring.html specifically https://docs.datomic.com/cloud/ions/ions-monitoring.html#local-workflow might help someone run locally and troubleshoot locally.

➕ 1

2024-01-04

Channels