Fork me on GitHub
#datomic
<
2015-12-28
>
zentrope00:12:12

A transaction function is atomic in the sense that you can query "the value of the database" and do all kinds of lookups before committing the transaction.

zentrope00:12:25

But transactions without db-functions are also atomic.

zentrope00:12:22

If you already have all the info you need to construct an appropriate transaction, a db function isn't necessary.

Ben Kamphaus00:12:33

@zentrope: it provides ACID isolation in that it will ensure that it acts on the direct immediate proceeding database value when querying, etc. I.e. nothing happens between reading and transacting, or throwing instead of transacting, etc.

zentrope00:12:05

That's my understanding.

zentrope00:12:14

But if a user is revising ingredients to a recipe the have the whole master/detail to hand. When they want to save it, it doesn't matter what the immediately preceding value is or was. Unless, by policy, you don't want "last one to win".

zentrope00:12:56

So, if you want to figure out which "ingredients" to retract, based on the user's intention, you could just use what the user has, or you could compare it using a DB function. To me, it becomes a matter of taste at that point.

Ben Kamphaus00:12:09

@zentrope: Basic isolation example: if you’re updating a balance from $1000 to $1100 because someone deposited $100, and another deposit of $50 comes in between querying the database value and submitting the transaction, you want to (a) throw and re-attempt with latest balance (peer coordination approach), or (b) have a transaction function that guarantees isolation and adds the deposit to whatever the current (most recent prior to transaction) balance is.

Ben Kamphaus00:12:34

I.e. a cas (compare and swap) use vs. dedicated add-to-latest tx function.

zentrope00:12:52

@bkamphaus: Yes. I get that. But that's not the case with editing a master/detail thing.

Ben Kamphaus00:12:51

It’s true that there are use cases where you don’t need isolation. But if you do, transaction function or optimistic concurrency using e.g. a cas approach where peers handle the work to build the tx and retries when necesary, etc. are what to reach for.

Ben Kamphaus00:12:30

It’s true that transactions without transaction functions are atomic in ACID terms, i.e. entire transaction succeeds or fails.

zentrope00:12:44

I think the problem some folks have (I did) was when you need to adjust a bunch of isComponent style objects related to a single :ref entity.

zentrope00:12:10

You can't just assert, "Hey, these are the new ones, remove the ones no longer in this set".

zentrope00:12:08

One solution is to load all the items into client space, the user adds, deletes, alters, but you keep a reference to the original, construct asserts/retracts, then put them in a transaction.

zentrope00:12:29

Another solution is to just send down the set the user wants to keep, then use a db function to sort out the retracts.

zentrope00:12:46

What I can't see is that there's really any difference.

Ben Kamphaus00:12:08

Well you do need isolation in that case with card many refs, component or not if you need to avoid the race. I.e. if you need to remove all refs/entities pointed to by the attribute, so you want to ensure that you remove all of the latest things. That is, in case something was asserted in between you retracting all the previous values and asserting new ones. You would have a stale entry.

zentrope00:12:10

Ultimately, you're saying, "here's the new state, overlay it". The coolness of a db function doesn't protect you.

zentrope00:12:34

Yes, that's true.

Ben Kamphaus00:12:04

caveat: I haven’t really scrolled up and re-read thoroughly, so I could be missing nuance in the use case.

zentrope00:12:20

Nah, your last comment addresses it.

zentrope00:12:51

Even so, a user adds an item, and then it's suddenly gone because some other user futzed with it at the same time.

zentrope00:12:34

Datomic gives you tools to deal with whatever policy you want to implement, but it's still a bunch of tough decisions.

zentrope00:12:55

(Luckily, history lets you untangle it!)

Ben Kamphaus00:12:45

If a user choose the application equivalent of “commit” and commits something that immediately overwrites another user, or a user says “forget everything” and someone commits something just before hand, the correct action is arguably to overwrite that that user said. But yes, exactly, preservation of retractions in history means you can determine what vanished and why, and if you want expose ways to recover things.

kingoftheknoll03:12:01

Hey everyone! I have a schema question related to a FIFO stack setup. Here’s how I did it before in Django/SQL. Imagine an job scheduling tool for recursively subcontracting jobs from vendor to vendor. Start with an Event w/ 1..n unfilled jobs. Each job can be 1) filled by a worker OR 2) linked by fk to a new job , where the vendor of the new job can 1) fill with worker OR 2) subcontract with new job. So it’s a linked list using fk’s to build a stack representing a relationships of subcontracting. An event can have multiple of these job stacks.

kingoftheknoll03:12:27

It seems I can easily implement the same thing with Datomic however I’m wondering if there’s a better way to represent this structure.

kingoftheknoll03:12:25

One crazy thought is to leverage the history of a single entity over time to represent the change but seems not to fit very well.

currentoor18:12:23

@kingoftheknoll: I also came from a SQL background. The best advice I got was that datomic schema entities should represent what. When, who, and why should be addressed by annotating transactions. Keep in mind transactions in datomic are entities in their own right.

kingoftheknoll18:12:56

interesting I didn’t realize transactions could be annotated

kingoftheknoll18:12:10

My current thought after stewing on things since last night is that I should embrace doing a linked list rather than focusing on the transactions. So :event/jobs cardinality/many to :job entities and :job entities have cardinality/one to other jobs. Do isComponent for all the jobs then sort them by their links to eachother post query

kingoftheknoll18:12:58

thanks for taking the time to look!

robert-stuttaford18:12:54

i wouldn’t use transactions to model this subcontracting

robert-stuttaford18:12:31

transactions model time, which is orthogonal to the notion of delegating work

robert-stuttaford18:12:51

time can pass with no delegation, and several delegations can happen at a single time

robert-stuttaford18:12:03

the approach you’re planning sounds right

kingoftheknoll18:12:29

@robert-stuttaford: yeah that was the conclusion I came to this morning

currentoor19:12:02

@robert-stuttaford: so when do you think annotating transactions is best suited? I'm still a novice at this stuff.

currentoor20:12:08

Also how do you do pagination in Datomic. Say I have a collection of entities, I need to sort them by a particular attribute then get a particular page in this sorted collection.

currentoor20:12:38

I know sorting has to be done in the client but are there any suggestions for doing pagination otherwise?

tcrayford21:12:12

@currentoor: you have to do that with raw index walking right now. Datomic query doesn't support sorting of any kind which means no pagination inherently

currentoor21:12:08

@tcrayford: could you please elaborate on what raw index walking means?

currentoor21:12:39

Does that mean just pulling the whole collection into memory and then extracting what I need?

domkm22:12:59

@currentoor: Re: raw index walking, look at datomic.api/datoms