Fork me on GitHub
#datomic
<
2018-08-28
>
henrik05:08:55

With the client API (on Ions), I’ve made a basic pagination function:

(defn paginate [offset limit results]
  (take limit (drop offset (sort-by second results))))


(d/q {:query '{:find [(fully.qualified/paginate 0 10 ?tuple)]
                     :in [$]
                     :where [[?id :article/id ?uuid]
                             [?id :article/title ?title]
                             [(vector ?uuid ?title) ?tuple]]}
      :args [(get-db)]})
The above works fine, but it goes nuts when I try to parameterize the query:
(d/q {:query '{:find [(fully.qualified/paginate offset limit ?tuple)]
                     :in [$ offset limit]
                     :where [[?id :article/id ?uuid]
                             [?id :article/title ?title]
                             [(vector ?uuid ?title) ?tuple]]}
      :args [(get-db) 0 10]})
ExceptionInfo Datomic Client Exception  clojure.core/ex-info (core.clj:4739)

henrik05:08:48

What’s the correct way to pass in parameters for the pagination function?

henrik05:08:59

Or more generally, what’s the correct way to paginate sorted results?

markbastian16:08:24

As I've been learning Datomic (and Datascript) I've come across a practice that seems to make a lot of sense but I don't see it in the examples so much so I wanted to see if it was considered good or bad form or "just another way to model the data." The practice is to make heavy use of refs, sometimes to the point where the data model consists of a number of atomic values/entities and the majority of the entities are aggregations of references to those values. For example, without the practice I'm describing you might model a movie like so (let's assume each field has a schema that refers to the field and its value type - e.g. :movie/title is a string):

{:movie/title "Ben Hur"
 :movie/year 1959
 ;Cardinality many on this one
 :movie/actors ["Charlton Heston" "Stephen Boyd"]}
However, you might recognize that the movie title, year, and actor names are all other values in the model. Instead, you might do this:
{:movie/title {:title/string "Ben Hur"}
 :movie/year {:year/value 1959}
 :movie/actors [{:actor/name "Charlton Heston"}
                {:actor/name "Stephen Boyd"}]}
In this case, every field is a ref out to another entity. The movie entities are defined logically and have no actual primitive value fields themselves. These referenced values can then be used to construct other movie (or other domain) entities in which they are used. For example, you could reference other movies or books with the same title or other events that happened in that year. Is this considered good practice? Does it have any sort of negative implications on the size of your indexes?

henrik16:08:47

@markbastian I've been looking at this for some attributes, but not all. Specifically those I want to enforce as unique throughout the DB, like email and URL.

favila16:08:14

yeah, this generally makes no sense unless the value has some kind of identity

favila16:08:20

(in your data model)

favila16:08:46

e.g. actors have identity independent of anything asserted of them

favila16:08:01

but the number "1959"?

favila16:08:12

or the string "Ben Hur"?

favila16:08:29

depends on the domain but I think usually not

favila16:08:41

alternatively, if you want to use entities with value-ish semantics (so they are shared-by-value) then they should have a unique attribute or some kind of hash-derived id

favila16:08:27

we use this technique as a kind of compression and to get around datomic not having custom value types

markbastian17:08:45

As a title or year, I would think these things do have identity.

markbastian17:08:31

The number 1959 wouldn't be particularly special. There are an effectively infinite number of them. But movie release years are limited. Less than 150.

markbastian17:08:22

And as a title, there are a limited number of works related to "Ben Hur" (one book, several movies, etc.)

markbastian17:08:23

In the year example, all of the references would have schemas along the lines of

{:db/ident       :year/value
 :db/valueType   :db.type/long
 :db/cardinality :db.cardinality/one
 :db/unique      :db.unique/identity}
in which they would exist uniquely in the domain.

markbastian17:08:38

In the above case I am presenting an extreme, but the idea is that you may have a relatively finite number of values from which all other entities are built. Some things, such as movie revenue, would definitely not fall into this category as they could be effectively anything.

dustingetz18:08:11

A value like 42 is its own identity, you don't need a second layer of identity on top of it

markbastian18:08:34

BTW, I appreciate everyone's help on this. I've been trying to achieve "Datomic Enlightenment" for a while now and a few things, like establishing identity when there is no obvious primary key and a database function won't do, are still elusive for me. This was just something that I thought of that seemed to solve the problem of "weak identity". In other words, you know facts about something that, taken together, tell you exactly what you want, but the thing you want doesn't have a natural single ID.

markbastian19:08:42

Perhaps setting :db/index true on :movie/title and :movie/year would accomplish what I am going for without adding any additional concept of identity to what are otherwise primitive values.

ghadi19:08:28

I'm trying to enumerate tradeoffs on Datomic Ion placement. Our main production east1 account (A) is not the same as the Datomic Cloud east1 account (B). Assuming I need to consume a Kinesis Stream with an Ion-backed lambda, do I: Place the Stream in account A and the Lambda/Ion in acct B Place both Stream and Ion in account B, produce to the stream remotely from acct A

ghadi19:08:57

account A and B can never be the same because we have a legacy EC2 VPC