Fork me on GitHub
#datomic
<
2018-07-04
>
Drew Verlee17:07:31

It seems like I can add a datom with an attribute that isn’t described in the schema. Is that right? What does that imply? To contrast, clearly you can’t do this in a relational db (without some setup).

favila21:07:15

"I can add a datom with an attribute that isn’t described in the schema." This shouldn't be possible. Do you have an example of doing this?

Drew Verlee02:07:45

I'm using datascript, I should have asked in that channel. I'll post the example tomorrow

Drew Verlee17:07:32

I’m reading the docs and trying to connect the dots here: > The set of possible attributes a datom can specify is defined by a database’s schema. Which would imply, to me, you can’t add a attribute to the db without it being in the schema.

igrishaev19:07:40

Hello Clojurians! I’ve been googling for the whole day but without any result: how can I sort datoms/entitines by their tx date in descending order? For example, to show the last 100 messages on the dashboard.

val_waeselynck20:07:53

Don't use the tx date in your app code, prefer a custom attribute. You could use avet with either a negated timestamp, or with seek-datoms using an exponentially decreasing lower bound until you reach a count of 100

octahedrion06:07:32

@U06GS6P1N why do you advise against using :db/txInstant in your app code ?

igrishaev19:07:33

(taking into account there might be quite many of entities, so manual work with collections is not applicable)

Drew Verlee19:07:11

@igrishaev I’m not an expert, but i believe one way is to 1. make sure to return the tx date from your query 2. call sort-by :tx-date using clojures native sort function This will work, its perf implications probably have todo with how your running datomic. Remember that part of the value proposition is that <waves hands> datomic can be treated as a value in your process. </waves hands>

Drew Verlee19:07:05

Thats possible not the best way, and i might be wrong 🙂. I glanced at the docs and this might be the right place to look: https://docs.datomic.com/cloud/query/query-data-reference.html (see custom aggregates)

leblowl21:07:44

Hey, I'm a little confused on how Datomic caches entities in my application code. Reading this: https://docs.datomic.com/on-prem/caching.html#entity-caching ... I am wondering what lifetime of the Entity instance is referring to. Are we talking about until the object is garbage collected? Also is this entity cache the same as the object cache: https://docs.datomic.com/on-prem/caching.html#object-cache? Thanks... (p.s. is there a preference for asking these questions in http://forum.datomic.com ?)

leblowl21:07:06

I think I found an answer for my questions. I found this line in the Entity javadocs (https://docs.datomic.com/on-prem/javadoc/datomic/Entity.html): the values of its attributes are not obtained from the db until get or touch are called, after which they are cached in the entity. So I am pretty sure that we just talking about basic heap caching with a garbage collection lifetime. And if that's the case, I think that is different than the object cache. I'd love to get a confirmation on that.

favila21:07:02

The object cache is a cache of datoms: objects decoded out of a fressian block; the entity objects have their on cache of the map view of datoms

leblowl21:07:55

@U09R86PA4 thanks. I guess the docs @ https://docs.datomic.com/on-prem/caching.html#object-cache were confusing me because it only mentions contains segments of index or log... I think I was missing the fact that Datomic only retrieves entities through the indexes (I think that's a little different then SQL, where an index may or may not be used, right?)

favila21:07:56

All datomic's indexes (except the fulltext index, which is a derived Lucene index) are "covering" indexes: each contains all of the data for the datoms it is indexing.

favila21:07:16

you may find this helpful to understand datomic's architecture http://tonsky.me/blog/unofficial-guide-to-datomic-internals/

favila21:07:12

in datomic an "index" is just datoms in a certain order

favila21:07:01

the datoms are stored in binary blobs encoded in fressian

favila21:07:30

the blobs are arranged in a sorted tree structure

favila21:07:49

so one blob/block may have hundreds or thousands of datoms in it

favila21:07:05

the object cache is a cache of the datoms decoded out of a block

favila21:07:24

the blocks themselves are what is stored in storage, keyed by a uuid

favila21:07:33

(this is what memcache is caching)

leblowl21:07:45

Thanks, very helpful. I'll take a look at that guide

sekao23:07:17

i know this is a common problem but i’m struggling to find a clear answer anywhere…how do i mimic SQL’s order by clause? i dont want to sort the data after retrieval since that certainly wont work with large data sets