Fork me on GitHub
#datascript
<
2017-08-10
>
thedavidmeister08:08:11

@tonsky so it looks like when i put datoms into a db that have a tx, then i do a new transaction, it doesn't force the new datoms to have a higher tx than the tx that i put in... is that right?

thedavidmeister08:08:23

i expected db-with to internally update the "next tx" value to something higher than the max value of the datoms that it puts in the db

thedavidmeister09:08:18

e.g. i have a datom in the db like [1501664966212 :project/title "fooodoodo" 536870923]

thedavidmeister09:08:27

:max-tx of the db is 536870913

Niki09:08:48

@thedavidmeister you can’t put datoms into DB with specific tx value, at least not officially

thedavidmeister09:08:18

how do i share datoms between users?

Niki09:08:33

what users? :)

Niki09:08:58

txs aren’t suppose to carry on any domain-specific value

Niki09:08:06

they are for audit purpose only

Niki09:08:15

just don’t rely on Tx numbers

thedavidmeister09:08:58

that was more of a practical question than "official" question

Niki09:08:00

how do you mean?

thedavidmeister09:08:12

if i'm trying to move datoms in and out of some dbs

Niki09:08:35

what dbs are those?

thedavidmeister09:08:11

uh, well a db in one browser and a db in another

thedavidmeister09:08:29

like, two users collaborating, so if you put datoms in one it can be shared to the other

Niki09:08:46

what do you need tx numbers for?

Niki09:08:11

I don’t think you can synchronize them

thedavidmeister09:08:32

i just thought tx would be a convenient way to deal with conflicts

thedavidmeister09:08:42

just use the newest tx, that kind of thing

Niki09:08:44

if two users make simultaneous edit to their corresponding db the tx will be the same for both the the content of it will be different

thedavidmeister09:08:01

yeah, there is a central db that would need to deal with that

thedavidmeister09:08:09

basically use timestamp as a tie breaker

Niki09:08:22

no, there’s no convenient way to deal with conflicts. Not just in datascript, but in general

thedavidmeister09:08:26

hmmm, well i'm actually not even at that point yet

thedavidmeister09:08:32

all i'm at is putting datoms into a db

thedavidmeister09:08:35

then pulling them back out

Niki10:08:21

ok, well, do not rely on tx :)

Niki10:08:32

consider it just a record-keeping

Niki10:08:42

it records the moment when you put datoms in db

Niki10:08:49

you have no control over it

Niki10:08:05

if you need something that you need to control, model it explicitly

Niki10:08:13

as attributes/entities

Niki10:08:22

the same principle applies to Datomic

thedavidmeister10:08:23

i'm not sure how to do that

thedavidmeister10:08:29

i'm putting datoms into a db on the server

thedavidmeister10:08:34

and then pulling them back out

thedavidmeister10:08:47

how can i model attributes/entities for the datoms themselves?

Niki10:08:51

that’s because you’re solving a very hard problem: distributes synchronization

thedavidmeister10:08:52

isn't that circular logic?

thedavidmeister10:08:01

actually i'm not even doing that yet

thedavidmeister10:08:08

i'm just putting datoms in the db

Niki10:08:11

well you can put them in a separate DB if that’s simpler

thedavidmeister10:08:12

and pulling them out

Niki10:08:19

ok just do that then

thedavidmeister10:08:26

but it doesn't work

Niki10:08:36

you can’t put datoms into DB?

thedavidmeister10:08:50

well ok, when i pull the datoms out, i pull out the highest tx

thedavidmeister10:08:10

so as not to send the history of all datoms over the wire every time

thedavidmeister10:08:32

so the problem is that now when it puts "new" datoms into the db

thedavidmeister10:08:39

they have lower tx than something already in the db

thedavidmeister10:08:01

so it looks like you cannot update the database, changes are "lost" when you reload the page

thedavidmeister10:08:09

even though the datoms are in the db, just with lower tx

Niki10:08:12

the problem must in somewhere else, because you can’t put datoms into DB and force any specific TX value on them. DB will assign it automatically. You have no control over it

Niki10:08:38

maybe your problem is that server changes override local changes?

thedavidmeister10:08:08

i mean yeah i thought they should

thedavidmeister10:08:26

i get e, a, v, tx and added out of the server and turn it into datoms and add it to local db

Niki10:08:54

there’s no API to add datoms to DB

Niki10:08:02

what do you use?

thedavidmeister10:08:33

uh, pretty sure i just put a list of datoms in db-with

Niki10:08:53

can’t do that

thedavidmeister10:08:29

(d/datom
  (:e r)
  (read-string.core/read-string (:a r))
  (read-string.core/read-string (:v r))
  (:tx r)
  (:added r)))

thedavidmeister10:08:14

(swap! conn #(d/db-with % [d]))

thedavidmeister10:08:24

why can't i do that?

Niki10:08:04

sorry I think I’ve added that recently, so you can do that, yes

Niki10:08:16

it’s not in Datomic so it was unofficial DS extension

thedavidmeister10:08:37

hah, i really like it, so thanks 🙂

Niki10:08:40

I guess someone found it useful

Niki10:08:09

but the normal way is to (d/db-with db [[:db/add e a v]])

Niki10:08:19

what’s wrong with that?

Niki10:08:38

also please use transact! instead of swap!

Niki10:08:36

anyways, adding datoms doesn’t have any magic properties and doesn’t magically make your system distributed

Niki10:08:36

the problem you were concerned with — getting right tx numbers — also has nothing to do with what you see in a database

Niki10:08:08

you see everything you’ve added to it. No matter what TX number is and in which order it was populated

Niki10:08:52

so you can add smth with tx 0 then smth with tx 10 then smth with tx 5 and you’ll see data from the last tx. Last in actual time, not with the highest tx number

thedavidmeister10:08:28

oh, so you see tx 5?

Niki10:08:23

yeah. These numbers do not matter. There’s no way in db to put tx 5 “before” tx 10

Niki10:08:45

you’re not even supposed to specify those numbers

thedavidmeister10:08:48

wouldn't you just drop it?

thedavidmeister10:08:01

as datascript doesn't keep history

Niki10:08:03

> you’re not even supposed to specify those numbers

Niki10:08:22

just use normal API and all will be fine

thedavidmeister10:08:36

ok but let's say i use timestamps instead of tx

thedavidmeister10:08:43

and i just leave tx be tx

thedavidmeister10:08:51

how do i get timestamps on a datom?

Niki10:08:29

I dunno. Map datomtimestamp?

thedavidmeister10:08:59

well i put timestamps on datoms in the db

thedavidmeister10:08:18

but then comparing incoming timestamps with data already in datascript

thedavidmeister10:08:30

not sure how to do that

thedavidmeister10:08:10

was thinking of putting metadata on the datom or something, but i don't know that it gets preserved through transact!

Niki10:08:25

no datoms don’t support that

Niki10:08:38

you can’t rely on datoms being the same object either

thedavidmeister10:08:57

yeah that's that i thought

thedavidmeister10:08:08

the datom is just a temporary object outside the db right?

Niki10:08:40

well they are inside DB as well but they are created and re-created in many places

Niki10:08:51

you can count on datom to contain the same information

Niki10:08:57

but it might be the new object

Niki10:08:06

no guarantees about that

thedavidmeister10:08:13

so metadata would be lost somewhere along the line

Niki10:08:53

you can’t even put metadata on datom

thedavidmeister10:08:24

if i use transact! would it update the max-tx?

Niki10:08:51

if you use :db/add it’ll update max-tx

Niki10:08:50

I actually think max-tx should update even if you use datoms

Niki10:08:56

maybe I’m wrong though

thedavidmeister10:08:21

but i can't set the tx

thedavidmeister10:08:18

also, i'm using swap! to avoid triggering listeners

thedavidmeister10:08:37

because my listeners send data

thedavidmeister10:08:42

i don't want to send what i'm receiving

Niki10:08:48

all I can say you’re working against the design, so expect it won’t be easy

thedavidmeister10:08:24

i think i need some extra accounting in between datascript and the server

misha11:08:10

@tonsky so, I should not really use any tx-meta (as in (defn transact! [conn tx-data tx-meta] ...) values I put in, during server synchronization?

misha11:08:18

to paraphrase: anything in tx-meta should be used only by the same process, which put tx-meta into db? and tx-meta should not survive transit de/serialization?

thedavidmeister15:08:11

that's a bit different to what i'm looking at i think

thedavidmeister15:08:49

you're looking at the tx data itself, not the tx id on the datoms right?

Niki15:08:37

@misha meta in general does not survive serialization