Fork me on GitHub
#datomic
<
2015-12-26
>
currentoor02:12:06

If I create a collection of entities using nesting like:

[{:db/id order-id
  :order/lineItems [{:lineItem/product chocolate
                     :lineItem/quantity 1}
                    {:lineItem/product whisky
                     :lineItem/quantity 2}]}]
And I want to update some of these nested lineItems, what’s the best way to do that?

currentoor02:12:26

Should I just retract the order entity and create a new one with the same entity-id?

zentrope02:12:13

I think you make those line items entities in and of themselves.

zentrope02:12:50

Then you can just assert new facts about them individually, if you want.

zentrope02:12:01

I’d make an attribute {:lineItem/id (d/squuid) } and have that be a :db.unique/identity.

zentrope02:12:31

When you make changes, something like

{:db/id [:lineItem/id “lkasjdlkas”] :lineItem/quantity 2}

zentrope02:12:25

[{:db/id order-id
  :order/id (d/squuid)
  :order/lineItems [{:lineItem/id (d/squuid)
                     :lineItem/product chocolate
                     :lineItem/quantity 1}
                    {:lineItem/id (d/squuid)
                     :lineItem/product whisky
                     :lineItem/quantity 2}]}]

zentrope02:12:30

Something like that.

raymcdermott16:12:14

@currentoor: if you update the properties in existing lineItems, Datomic will take care of it if you send the map through (assuming the entity and component entity IDs are present)

raymcdermott16:12:46

it gets funny if you want to add / delete items and its not so simple then

raymcdermott16:12:16

I wrote a DB function to cope with it (after some advice from @tcrayford here)

currentoor18:12:12

@zentrope: Thanks, but lineItems are components entities and I’d like to interact with them as part of Orders.

currentoor18:12:30

@raymcdermott: Thanks, I’ll look into that.

raymcdermott18:12:32

@currentoor: let me know if it’s useful and I will write that blog post simple_smile

currentoor18:12:10

Will do, it looks like exactly what I need!

currentoor18:12:46

@raymcdermott: so how would I use it? Like this?

[:component-crud {:db/id order-id
                  :order/lineItems [{:lineItem/product chocolate
                                     :lineItem/quantity 1}
                                    {:lineItem/product whisky
                                     :lineItem/quantity 2}]}
 :order/lineItems]

currentoor18:12:18

where the value of :order/lineItems has been updated.

raymcdermott18:12:51

very close but the updated items should have the entity IDs (that were created automatically on insert)

raymcdermott18:12:12

they come back if you do a pull query on the order-id

raymcdermott18:12:38

items that do not have an entity ID are treated as novelty

raymcdermott18:12:25

the general idea is just to send back in the updated map and let the function work it out

raymcdermott18:12:58

I have a small example based on a shopping cart...

raymcdermott18:12:26

(defn- save-new-cart [conn cart]
  "New: any embedded skus will be created as component entities"
  (let [temp-id (d/tempid :db.part/user)
        tx-data (conj [] (assoc cart :db/id temp-id))
        tx @(d/transact conn tx-data)
        {:keys [db-after tempids]} tx
        cart-id (d/resolve-tempid db-after tempids temp-id)]
    (d/pull db-after '[*] cart-id)))

(defn- save-updated-cart [conn cart]
  "Update: embedded skus will be handled by the DB CRUD function"
  (let [tx-data [[:component/crud cart :cart/sku-counts]]
        tx @(d/transact conn tx-data)
        db-after (:db-after tx)]
    (d/pull db-after '[*] (:db/id cart))))

(defn save-cart! [cart]
  (let [conn (d/connect uri)]
    (if (:db/id cart)
      (save-updated-cart conn cart)
      (save-new-cart conn cart))))

raymcdermott18:12:42

the save functions return the committed updates

currentoor18:12:55

Hmm, I was thinking about this a little bit differently. I’d like to only keep track of the outer Order entity and not worry about the entity-ids of the nested lineItems. Instead I can pass new lineItems (inside the order) in and the db-function can lookup the current lineItems and do a data-value based comparison to figure out the additions/retractions needed.

raymcdermott18:12:40

Retractions is tricky that way. How do you mark something as deleted?

raymcdermott18:12:23

additions would be easy - you would need to provide some key or composite key for the comparisons

zentrope18:12:48

@currentoor That's why I give lineItems unique IDs. When you pull-api the order out, you have all the information you need to construct retractions.

raymcdermott18:12:50

my thinking was that the data is cohesive by its nature

raymcdermott18:12:31

@zentrope: datomic gives the lineitems IDs by default if it is passed a map

zentrope18:12:43

For instance, in a web form, when the user "deletes" the line items, you can add that to a "dropped items" structure. Throw the whole thing back to the server and you can just loop through and delete.

zentrope18:12:47

Yeah, db/ids or you can make your own as well to avoid relying on those, but either way, doing set math on the data in your client turns out to be reasonably simple.

raymcdermott18:12:48

my approach is that missing data is retracted but you could keep another structure too

raymcdermott18:12:29

@zentrope: indeed my function just does some set functions

currentoor18:12:51

@zentrope: I see your point but in my use-case the nested entity is an implementation detail that should not be exposed outside.

raymcdermott18:12:57

one advantage of using a function is that the actions are atomic

zentrope18:12:19

Yeah. I guess the surprising thing is you can just assert a set of values for an attribute and thus replace what was already there. So, db function, or client-side stuff.

currentoor18:12:42

yeah I definitely want this to be an atomic function otherwise data can get into a weird state

currentoor18:12:56

@raymcdermott: I think we may be talking about two different things, I’ll go try out my ideas first. I’m probably mistaken about something here.

raymcdermott18:12:23

@currentoor: no worries, let me know how you get on

currentoor18:12:26

@raymcdermott: is there an advantage to using datomic.api/function? I’ve been writing them as regular functions in regular namespaces (requiring them into the transactor).

zentrope20:12:36

Cool. Bookmarking. ;)

curtosis20:12:57

anyone know of a good example of building datomic entities up from SQL queries?

curtosis20:12:25

(i.e., migrating from tables to Datomic)

tcrayford21:12:38

@curtosis: is the goal schema translation? Or am I confused?

curtosis21:12:20

more or less, yes. I have a bunch of data in postgres that I want to move over into Datomic. I have both schemas.

curtosis21:12:06

it's not too complex, but the source tables are relatively normalized (= lots of nested entities)

tcrayford21:12:58

yeah. Are you looking to automate the schema translation or just do it once manually?

curtosis21:12:48

automated would be nice -- I'll probably have to do it more than once simple_smile -- but that's not a major consideration.

tcrayford21:12:38

sure. In which case maybe the mbrainz example from datomic itself? Mbrainz was originally a sql database (if you're looking for an example of how to translate schema)

curtosis21:12:05

hmm... I thought mbrainz was meant to be loaded from an already-datomicized restore file. is the generating code in the repo somewhere?

tcrayford21:12:43

@curtosis: no, that's a sample of a manual translation

curtosis21:12:08

oh, wait... I don't mean translating the schema ... I already did that. I just want to move the data.

tcrayford21:12:11

I don't think doing automated translation would ever work that well, but "never say never" 😉

tcrayford21:12:24

@curtosis: oh. I don't think I have a good example of that

curtosis21:12:42

yeah... I would think that an automated schema translation would probably work but hamstring your datalog into some unnatural shapes

curtosis21:12:08

meanwhile, I'm also stuck trying to figure out what the datomic console url looks like for a sql backend, so that's going well for me too. 😛

curtosis22:12:59

"uri is a Datomic db uri with the dbname missing" doesn't make sense with datomic:sql://<DB-NAME>?jdbc:

raymcdermott22:12:50

@currentoor: the main thing is atomicity

bkamphaus22:12:06

@curtosis: The URI arg for Datomic console is exactly that, omit the <DB-NAME> — e.g., datomic:sql://?jdbc: is my local URI (w/user,password changed to datomic).

curtosis22:12:35

@bkamphaus: I'm clearly an idiot.... I tried that, which didn't work at all... until I noticed that I wasn't specifying a port.

curtosis22:12:05

but it looked like all my other failures 😛

bkamphaus22:12:50

Ah, got it, yeah full invocation for me looks like: bin/console default $(pg-uri "") -p 1121 — helper to generate the local pg-uri because I can’t be arsed to do all the typing.

bkamphaus22:12:12

Oh, right, as in it just prints the usage instead of complaining about which specific thing was missing?

curtosis22:12:21

I've at least got the console talking to it, and successfully transacted my schema and the static data (from groovysh)

curtosis22:12:11

but thanks for pointing out that it was correct, no matter how wrong it looked... that helped see that something else was broken.

curtosis22:12:22

so now I'm back to the hard part, of walking my sql results and building entity maps \o/

bkamphaus22:12:43

Gotcha. FWIW, I’ll make note of the implied feature request for a more specific error message.

curtosis22:12:03

ah, yeah. thanks for translating simple_smile

currentoor22:12:29

@raymcdermott: what's not atomic about my implementation?

currentoor22:12:12

Are you talking about d/function?