Fork me on GitHub
#datomic
<
2016-12-18
>
kwladyka05:12:34

The Datomic Free transactor is limited to 2 simultaneous peers and embedded storage and does not support Datomic Clients. <- what embedded storage exactly mean in this case? Is it mean only dev or everything expect legacy storages?

robert-stuttaford06:12:26

the free storage; literally an H2 database stored alongside the transactor binaries on disk; requiring the transactor be running to connect to the storage. all other storage types require pro starter or pro

kwladyka15:12:28

> Datomic free configuration - playtime only!! Is it mean it is impossible to use datomic free on Heroku?

tjtolton19:12:26

So, datomic has an upper limit of 10 billion datoms... is that just for one db? (i.e. once one DB has filled, can you just switch and start using another DB?)

kwladyka19:12:13

Somebody can share good article or video to understand how to use partitions. It is confuse for me.

kwladyka19:12:46

@tjtolton i am learning datomic, but i guess you can clear history in DB first to free space. If it is not enough i guess it is limit by each DB separately.

kwladyka19:12:29

as i understand :db.part/db and :db.part/tx i don’t use it manually, but i can use :db.part/user or install my own partition… ? [:db/add #db/id[:db.part/user] :db/ident :community.orgtype/community] - what #db/id doing here? When to use :db.part/user and when install new one? Never use manually :db.part/db and :db.part/tx? So :db.part/tx is created during transaction… :db.part/db is defined by schema what mean :db/ident values :foo/bar and :foo/baz are in the same partition because of namespace :foo?

kwladyka19:12:43

[[:db/add "foo" :db/ident :green] [:db/add "foo" :db/ident :red] will be in :db.part/txpartition? Also :db.part/db?

kwladyka20:12:12

or maybe i can think about that in that way… :db.part/tx is about Tx value in this table http://docs.datomic.com/entities.html and db.part/db is about.... about what? about E?

robert-stuttaford20:12:55

@tjtolton 10billion datoms is a practical limit based on ram needed by peer. no limitation in the code.

robert-stuttaford20:12:28

so using two 10bn dbs from a peer would mean needing twice the ram to hold all the roots in memory for both

robert-stuttaford20:12:24

@kwladyka @tjtolton partitions: 1. db partition. holds partitions themselves, schema, and all db primitives (try eval (seq (d/datoms db :eavt)) on a brand new in memory database!) 2. tx partition. transactions. 3. user partitions. one installed for you. holds your own data, your db functions, your enums. you can make as many as you like. why make multiple user partitions? it affect the sort order of entity ids. partitions sort together. this means that index segments hold co-partitioned data together. basically, it improves peer cache performance at scale because less index segments need be read to satisfy a given query, because the stuff is packed together in storage.

robert-stuttaford20:12:58

partitions are expressed in entity ids as the high bits of the Long. the entity’s birthday plus a counter for that transaction make up the low bits.

robert-stuttaford20:12:35

@kwladyka impossible to use transactor on heroku: no. painful: yes, because ephemerial storage. you’d need to be very good at keeping backups.

kwladyka20:12:19

do you know why heroku has this politic about datomic free?

kwladyka20:12:07

@robert-stuttaford thank you for your explanation, unfortunately still i don’t get it in 100% 😞

tjtolton20:12:33

how much memory is 10b datoms?

kwladyka20:12:58

E is.... :db.part/db ?

tjtolton20:12:00

probably not a small amount, but I want to use the figure to make a point during my demo 😉

robert-stuttaford20:12:27

@kwladyka partitions describe a numerical range for entity ids to live in

robert-stuttaford20:12:17

for schema et all, E is an integer for your entities and for transactions, E is a Long Tx is a reference to a transaction E elsewhere in the database

robert-stuttaford20:12:36

@tjtolton you could always generate a 10bn datom database 🙂

tjtolton20:12:52

I suppose I could at that!

robert-stuttaford20:12:03

but basically, it’ll be big servers with > 10 gigs of ram i would imagine

robert-stuttaford20:12:11

@kwladyka you can forget all about partitions, quite honestly. you only need to think about them if you plan to have a BIG database and you can clearly define arbitrary co-queried sections. e.g. a lot of stuff for client A and ditto for client B or what-have-you.

kwladyka20:12:14

oh so :db.part/tx is for example Tx in range 0-1000 ?

robert-stuttaford20:12:38

the latest release allows you to use simpler string temporary ids

robert-stuttaford20:12:27

the actual range values are much bigger than 1000, of course

robert-stuttaford20:12:01

looking at your question again, @kwladyka; 1. use :db.part/db when installing schema 2. use :db.part/user when adding your own data 3. use :db.part/tx when annotating the transaction you’re processing

kwladyka20:12:09

ech i don’t get it.... but maybe answer how use it will help me… so… https://www.refheap.com/124290 how should i create this :product/label “enum” values in the right way?

robert-stuttaford20:12:55

on 3, an example: (d/transact (d/connect …) [{:db/id (d/tempid :db.part/tx) :transacting-user-entity [:user/email “”]}])

kwladyka20:12:00

Why in examples about “enum” values i don’t see :db/isComponent parameter in schema?

robert-stuttaford20:12:26

this would put :transacting-user-entity on the transaction entity itself, which allows you to later see who made that change to the database

kwladyka20:12:40

ok so… 1. and 3. is done by automate yes? I don’t have to use it anywhere?

robert-stuttaford20:12:32

what is your goal, here? understand when to use them, or have an understanding of what partitions are and do?

robert-stuttaford20:12:51

if it’s merely use, you could just use the simpler string tempid system

kwladyka20:12:14

i wanted understand what partitions are and do to know how to use… but i see it is too hard for my today so i can satisfy to know how to use it today 🙂

robert-stuttaford20:12:09

ok. so if you’re using the newest version of datomic, you don’t need to ever declare a partition any more.

kwladyka20:12:39

yupi, you simplified my life 🙂

robert-stuttaford20:12:40

honestly, i think you’d be in a great place if you work through the http://docs.datomic.com/tutorial.html

robert-stuttaford20:12:49

it uses the new simpler system

kwladyka20:12:55

i read whole this doc

robert-stuttaford20:12:23

[:db/add "foo" :db/ident :green] is using db.part/user in the background

robert-stuttaford20:12:41

[:db/add "datomic.tx" :db/doc "remove incorrect assertion”] is using db.part/tx in the background

kwladyka20:12:45

but this doc use client library which is in alpha version. In Clojure probably the best is use peer?

robert-stuttaford20:12:06

{:db/ident :inv/sku
    :db/valueType :db.type/string
    :db/unique :db.unique/identity
    :db/cardinality :db.cardinality/one}
is using db.part/db in the background

kwladyka20:12:39

oh now i start understand (i hope)

robert-stuttaford20:12:46

these are transactions; both the peer and the client lib are identical in this aspect because they all just send stuff to the transactor

robert-stuttaford20:12:05

the old system had you explicitly declaring temp ids with a partition

robert-stuttaford20:12:19

[:db/add (d/tempid :db.part/user) :db/ident :green]

robert-stuttaford20:12:38

[:db/add (d/tempid :db.part/tx) :db/doc "remove incorrect assertion”]

robert-stuttaford20:12:31

{:db/ident :inv/sku
 :db/valueType :db.type/string
 :db/unique :db.unique/identity
 :db/cardinality :db.cardinality/one
  :db/id (d/tempid :db.part/db)
    :db.install/_attribute :db.part/db} ;; schema had extra ceremony!

robert-stuttaford20:12:43

so there’s the new vs the old

robert-stuttaford20:12:22

i suggest you just start doing stuff and see how it feels 🙂

robert-stuttaford20:12:07

it’s older though, and uses the older temp id syntax

kwladyka20:12:58

@robert-stuttaford thank you so much for this explanation. Probably information about Datomic from sources about old and new versions made me so confuse.

robert-stuttaford20:12:42

key to note that both tempid systems still fully applicable

robert-stuttaford20:12:56

you can learn it all and apply it all. no breaking changes in any of the APIs.

kwladyka20:12:13

so… if i have enum values for something how should i declare schema? https://www.refheap.com/124293

kwladyka21:12:38

hm… ok ok :db/isComponent doesn’t work in that way

vinnyataide21:12:48

what is :tx-data from the docs and why the <!!

(<!! (client/transact conn {:tx-data [{:db/doc "hello world"}]}))

marshall22:12:07

<!! Is blocking take from a channel. Look at the core.async docs for more info on thay

marshall22:12:46

the new client api is all asynchronous so everything returns a channel

marshall22:12:36

:tx-data is a key to a map. The transact function takes a map as an argument and the map needs to contain the transaction data as a value associated with the :tx-data key

vinnyataide22:12:34

👏 👏 👏 👏 👏

kwladyka22:12:12

https://www.refheap.com/124298 - how do you deal with transform data datomic <-> clojure spec structure?

vinnyataide22:12:12

As good as an explanation can get, thanks @marshall

tjtolton22:12:26

in this post http://martintrojer.github.io/clojure/2015/06/03/datomic-dos-and-donts about the DO's and DONT's of datomic, martin troljer says: > Do your project.clj chores >Datomic brings in lots of maven dependencies. Make sure you don’t suffer from clashes, spend time solving the ‘:exclusions puzzle’. what does he mean by "the exclusions puzzle"?

marshall22:12:05

Try running lein deps :tree

marshall22:12:05

He's referring to managing exclusions on various dependencies to avoid conflicts