Fork me on GitHub
#datascript
<
2016-11-26
>
Niki05:11:49

What do you mean? You can alter the schema in Datomic

leov08:11:24

omg. I have to change the way I think after relational DBs

leov08:11:26

So if I'm periodically scanning a collection of processes, and each process has environment variables, which change over time, I have two options then. 1. It can be [<process-id> :process/LOCALE "UTF-8"] in case of datomic, though, I would have to install new attribute definitions on the go on every update, and remove old ones. In case of datascript, it just works. I guess I need a DB function to retract old keys and put it new keys on each update. Also this type of attribute storage does not seem to be affecting performance in any way - it seems I can efficiently query all the keys of all the processes or current process with the range index scan. 2. [.. :process.env/key ..] and [.. :process.env/value "UTF-8"] DB Schema in this sort of storage is completely static - key is string, value is string. In this case there is a question - do I need to add composite unique keys into schema? Quick googling seems to show that this concept is a problem even in full-version datomic. Official documentation suggests - if I understand it correctly - installing custom transaction function that will implement such 'composite unique key' constraint, retracting old values on upsert. This will not, however, autoretract older keys/values that were removed after last update. - It might be still a solution to find the latest transaction number that asserted entire fresh key/value set, and use this transaction number as a filter to get latest collection - But I guess I need to think of datomic entities during update as a diff between new and old states, in terms of which datoms should I assert and which retract, right? This is different then doing upsert into a JSON field of a Postgres column. Then I have to use/write database function that will retract all associated process.env/keys and values first. (Maybe it's one short query inside transaction, must try to do it)

leov08:11:29

In original postgres I was using JSON field in a process table - and it supports both merging and replacing, with replacing a default operation on row upsert or column update. Once you update process's configuration - you retract old configuration keys/values and assert new ones. That's what I need, thing that behave like internal JSON in dato*. Need to test components in full datomic.

leov08:11:07

(meta) Also looking at how iPad slack client receives my chat messages, with all the edits applied live in place with animations as it seemingly loads the edits from server continuously, I see it gets stuck after some 8-10 edits and didn't show me latest version until I restarted slack Ok, now it's obviously a bug - no amount of slack restarting on ios can load a message correctly that had >10 edits)

rauh08:11:08

I'd go with #2, and add a composite key that you specify as unique, keeps it simple and is effective

leov08:11:43

can you point me at a solution on how to add composite key?

rauh08:11:33

Well, that part would just be :process.env/uni-key (str proc-id "/" env-key)

leov09:11:18

it duplicates data though

rauh09:11:42

Yeah it only serves for the purpose to keep it unique

rauh09:11:26

If you are on datomic and want to do it properly you can just create a db function.

rauh09:11:44

But if you want to do a quick and dirty (and working) solution... then the above is just fine IMO.

leov09:11:36

yeah, that might work)

Niki10:11:40

@leov composite keys are pain in the ass for us too

Niki10:11:49

just spent last week working with them

Niki10:11:56

hard to query, hard to update

Niki10:11:21

one option is to completely remove old entity and add new one with new set of ENV props. That’ll save you diffing

Niki10:11:01

another option is to serialize entire env map and store it in string attr. But it’ll only work if you don’t plan to query based on values in that map

Niki10:11:17

again, that’ll make updates simple

Niki10:11:30

I wouldn’t recommend dynamically alter Datomic shema

Niki10:11:44

eps. when amount of attributes is potentially unbound

Niki10:11:23

In fact, smth like :process.env/uni-key (str proc-id "/" env-key) makes finding values easier

Niki10:11:36

we’ll probably adopt this approach :)

Niki10:11:32

about Slack update: listening to https://softwareengineeringdaily.com/2016/09/12/slacks-architecture-with-keith-adams/, they say they try the best they can, bringing server updates, but if it’s impossible, they don’t do anything about it

Niki10:11:46

“best effort” approach :)