This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-01-14
Channels
- # announcements (6)
- # architecture (5)
- # aws (4)
- # beginners (79)
- # boot (3)
- # boot-dev (7)
- # calva (21)
- # cider (17)
- # cljdoc (12)
- # clojure (83)
- # clojure-art (2)
- # clojure-belgium (2)
- # clojure-brasil (1)
- # clojure-estonia (2)
- # clojure-europe (3)
- # clojure-finland (5)
- # clojure-india (2)
- # clojure-italy (49)
- # clojure-losangeles (1)
- # clojure-nl (12)
- # clojure-spec (120)
- # clojure-sweden (2)
- # clojure-switzerland (4)
- # clojure-uk (31)
- # clojurescript (80)
- # data-science (17)
- # datavis (2)
- # datomic (31)
- # emacs (31)
- # figwheel-main (28)
- # fulcro (6)
- # jobs (2)
- # liberator (7)
- # luminus (1)
- # nrepl (2)
- # off-topic (51)
- # overtone (2)
- # pathom (4)
- # re-frame (28)
- # reitit (1)
- # rum (6)
- # shadow-cljs (26)
- # specter (2)
- # tools-deps (33)
- # yada (3)
Hi. I'm trying to find some datomic benchmarks (on real cases). I am wondering how fast I can write msg into datomic (writes / s). I can only find "it is not so fast at writing", "writing is not the strongest point of datomic"...cool but this is closer to 10 writes / s or 10000 writes /s compared to other databases?
Rule of thumb is keep database size smaller than 10 billion datoms, which if you reach that in a year is 317 datoms per second. At that rate you are very likely to be read constrained, not write constrained
“datomic is not fast at writing” lacks sufficient nuance to be useful at all
If you transact 10k datoms per second, you’ll have 10 billion datoms in 10 days
@m_m_m You’re not finding the results because of a few different reasons: 1. Its against the license 2. writing of different sizes, shapes, constraints, batches of data will yield different results. 3. It’s harder to compare to other systems due to query groups allowing for reads to come for free and letting your primary groups focus exclusively on writes. My understanding with datomic cloud is that if you run into a write performance constraint you can crank up the knob on dynamodb’s write throughput and use an i3.xlarge as your primary group. When to do that and what it will produce I can’t answer, as I haven’t had to do that. Can I ask what the usecase is?
@joe.lane Sure, I would like to store items form multiple games. Those items have content and state which can be changed in time (the same with owner). Datomic can give me whole history of each of those items which is nice....but more important feature is easy access to the newest state of each of those items. Hmm is there a huge difference between performance of proverion and free version of datomic?
Well Free version of datomic is, I believe, in memory and on-prem. I was referring to datomic cloud.
My 1 and 2 are still relevant, but 3 falls off and so does the i3.xlarge/knob-cranking. I don’t have the experience with on-prem. Is the “access the whole history of each item” part of the game or just a development convenience?
development convenience. There should be possibility to see history of the item, price changes, owner changes, stats changes in time 🙂
Does anyone here have an on-prem Dockerfile, maybe with sqlite jdbc backend for CI from a known-good database?
@m_m_m you shouldn't rely on datomic's history features to model your domain, its really mostly just an audit thing, add your own time as needed
Thank you. Now I can see it more clearly. So each time when I would like to change one param in my item I have to create a new item with each of my params and change one of them ? This is a lots of unnecessary data.
@lockdown- I'm a bit confused by the author's "revision entities" example
how does it route around the fact that the tags don't exist in the previous versions?
Author here. In the example, the tag would have been retroactively added to the previous revision when the intern backfilled the data manually. Of course, that's a business decision from the blog platform, they could have chosen not to backfill the tags.
hm OK. so walk me through what this does.
- finds the version-t
. how is this different than t
? what is version-t
supposed to represent?
- finds version-eid
. Is this the eid
as-of the correct version for time t
?
and then finally, the assumption is that the intern has gone back not only to the entity, but every previous version of the entity, and added the tags?
> finds the version-t
. how is this different than t
version-t
a domain-specific timestamp, representing the creation time of an Article Revision - nothing to do with Datomic's t
.
version-eid
is the Entity Id for the Article Revision.
Keep in mind we're not using Datomic's historical features at all in this code, that's the whole point - no as-of
, no t
, etc.
so each "revision" is it's own entity; when you make a change to the latest revision, would you transact a new entity? or do you transact against the existing entity?
Normal usage would mean transacting a new Entity (after all, the whole point of a revision is to be immutable); Revisions would typically be updated in data migrations.
OK, I see. I was still stuck on using datomic's history API for tracking non-breaking modifications 😵 that explains it.
(def con! (d/connect "datomic:))
Syntax error (ClassNotFoundException) compiling at (form-init3668295776313994723.clj:1:11).
com.amazonaws.services.dynamodbv2.model.PutItemRequest
^ any ideas what the issue is?you probably need the dynamodb dependency from the AWS Java SDK, like this: https://docs.datomic.com/on-prem/storage.html#dynamodb-aws-peer-dependency