I’m looking for some feedback on our datomic schema. anybody can give us some guidance? schema & questions here: https://gist.github.com/kommen/a902e4c5bfef1395e69a617d1cb427ac

@kommen I would personally favour the first approach and use :db/isComponent on the profile refs. Disclaimer: I am new to Datomic, so take my word with a grain of salt.

Lately I have been using the “document” analogy to help me decide if i should use :db/isComponent. I ask myself: “if I were to represent this entity as a document (e.g. in a document-oriented database), would I put this sub-entity in the document as well?”

This is basically asking “is this other entity (profile) a part of / a component of my entity (person)?” (quite naturally)

or “does a Profile have an existence of its own, or does it only exist as a part of some other entity?”

I am not sure these are the semantics that were intended for :db/isComponent, but that’s how I have been using it so far

@favila made a remark to me the other day which is related to what you are asking now

I was asking if it made sense to add a :db/unique constraint on attributes which have :db/isComponent. It seemed to me that these should always be unique, and so that :db/isComponent entailed the :db/unique constraint

he pointed out that it entailed an even stronger constraint: an entity that is referenced by some attribute with :db/isComponent should not be referenced by any other attribute in the database

(if I recall correctly, @favila can correct me if I don’t :<)

alright, thanks for your input!

Np 🙂 By the way on :profile/owner, I tend to avoid polymorphic references like this because they’re harder to validate

that’s purely a matter of taste though

@hmaurer @kommen that is correct, entities reachable by an isComponent attr should not be reachable (by forward references) any other way

(d/datoms db :vaet component-entity-id) should only have 1 or 0 results for all times

@favila out of curiosity, what is your opinion on polymorphic refs? (however you define “polymorphic”, since there isn’t a clear notion of “entity type” in Datomic). Is it something you try to avoid? Or do you not pay particular attention at it?

type generally depends on use

there's a distinction between entity-level validity/expectation constraints for the entities in the db, and an explicit type marker is necessary for sanity here

and what a program does with the map view of this, which seems to be more interested in attrs and sets of attrs than types

at least in my experience

but never, ever make the meaning of an attr dependent on some other attribute

@favila that’s a similar philosophy to clojure.spec I assume? The meaning of attributes should be independent of the aggregates in which they’re used

or did you mean something else?

"foo" meaning different things in different contexts I suppose?

Essentially. I just want to make sure no one misinterprets me to think that {:entity-type "earthquake" :magnitude 7} {:entity-type "decimal-number" :magnitute 2} is ok

Will the ensure-cf datomic command work without creating new AWS resources (security groups, roles etc) if existing ones are specified in the input properties files?

how do i use datomic's tempids? what are those -92233... numbers in {-9223301668109598138 17592186045422, -9223301668109598137 17592186045423} from http://docs.datomic.com/getting-started/transact-data.html ?

@cjhowe those are the IDs assigned to the 3 entities that were created by this transaction

yeah, but how do i know which is which?

you could add a :db/id attribute to each entity in that transction as a string

e.g.

{:db/id "movie-1"
 :movie/title "The Goonies"
 :movie/genre "action/adventure"
 :movie/release-year 1985}

the returned tempids should then be something like

{"movie-1" 17592186045422, -9223301668109598137 17592186045423}

@cjhowe use the d/tempid function

sorry, d/resolve-tempid

@favila taking this opportunity to ask: is there any advantage to using d/tempid and d/resolve-tempid over string tempids?

not really

you don't need an auto-increment string function?

(str (name (gensym)))

Also I assume string tempids resolve to an ID living in the :db.part/user partition?

you can control partition with tempids, not with strings

I see; thanks!

(let [tid1 (d/tempid :db.part/user)
      tid2 "string-tempid"
      {:keys [tempids db-after]} @(d/transact conn [{:db/id tid1 :db/doc "tid1"}
                                           {:db/id tid2 :db/doc "tid2"}])
      tid1-eid (d/resolve-tempid db-after tempids tid1)
      tid2-eid (d/resolve-tempid db-after tempids tid2)])

Are partitions something important to think about when developing a small to medium sized app with Datomic? As far as I understand they are “locality hints” and if used properly can increase performance; is that right?

they reduce index fragmentation and increase locality

they are not mere hints, they control sorting order

@favila how so? (regarding sorting order)

an entity id is a long composed of partition id bits and an auto-increment-id bits

since the partition bits of the entity id are higher, they have a stronger effect on magnitude than the lower bits

so entities with the same partition will group together when you sort

(d/entid-at db :db.part/db 1000)
=> 1000
(d/entid-at db :db.part/tx 1000)
=> 13194139534312
(d/entid-at db :db.part/user 1000)
=> 17592186045416

When would the grouping property when sorting by entity ID be desirable? (sorry if the question is silly)

all datoms are stored in sorted blocks

if all items are "close" to one another (same partition), reduces the likelyhood you need to fetch additional blocks or fragment blocks while indexing

e.g. suppose you have two companies using the same db for their data (not that I would recommend this)

read and write patterns are such that txes and queries only deal with one partition at a time if you have companies in separate partitions

@favila so, for example, you might want to store entities and their components in the same partitions?

or have a partition per entity type?

depending on what is often fetched together

yes

http://docs.datomic.com/indexes.html#partitions

actually components automatically get an entid in the same partition as their parent

{:db/id (d/tempid :some-part) :some-component-attr {:foo "bar"}} (for eg) was legal even before implicit tempids

the inner entity would be given a :some-part partition id automatically

http://docs.datomic.com/transactions.html#nested-maps-in-transactions