Am new to Datomic but can someone please clarify these statements: Datomic is read scalable but not write scalable. What does that mean? And why it is not best for website like twitter? And Also from Professional Clojure book: It took inspiration from multiple sources, but its basic goal is to be a drop-in replacement for the main cases where you would otherwise use a relational database as a transactional store. Stuart Halloway says that it’s targeted at the ninety-six percent use cases of relational databases, *leaving off the top four percent of high write-volume users like Netflix, Facebook, etc. * ?

@simon from the docs:

Datomic is a single-writer system. A single thread in a single process is responsible for writing transactions.

it serializes all writes

twitter needs to support massive writes

@simon I can perhaps try to clarify the second statement, having written it originally ;)

Although I was mostly paraphrasing various claims from the Datomic team. The essential idea is that it's for the majority of business systems that would traditionally be powered by a transactional SQL database. But it's not designed for extremes of scale in terms of total data volume or write throughput of the kind that top consumer-facing sites like Twitter face

Though, i doubt twitter started needing anything near the write throughput they have now, or would have gotten their if they did 🙂. Somewhat off topic but i just finished re-reading the first chapter of designing data intensive applications. Which talks about twitter and very soon after gives this advice: > Approaches for Coping with Load. Now that we have discussed the parameters for describing load and metrics for measuring performance, we can start discussing scalability in earnest: how do we maintain good performance even when our load parameters increase by some amount? > An architecture that is appropriate for one level of load is unlikely to cope with 10 times that load. If you are working on a fast-growing service, it is therefore likely that you will need to rethink your architecture on every order of magnitude load increase—or perhaps even more often than that. The question then becomes, once you outgrow (your perf start to dominate the complexity of your system) something, how easy is it to transition? I feel like the clojure community at large (though i can’t speak to datomic) approaches these issues in a very healthy way due to the focus on compostability and immutability.

Somewhat tangentially related. Does anyone have any insight into if https://github.com/denistakeda/re-posh with data-sync would play nice with datomic ions? I’m also just generally searching for more insight into data-sync (a component of one implementation of re-posh). I need to re-consult the docs but it seems like a fairly complex topic so i’m curious if anyone has any insights.

Sure, the question should be "do I have the scale of something like Twitter/Facebook/etc", not "might I at some point have that scale?"

Has anyone successfully set up a VPC endpoint as described in https://docs.datomic.com/cloud/operation/client-applications.html#create-endpoint? It fails for me with "No export named datomic-demo-VpcEndpointServiceId found. Rollback requested by user."

Hey Everyone, I’ve been going through the ions tutorial and I’m currently stuck. Granted, I don’t have a lot of familiarity with tools.deps.alpha. However I’m currently getting access denied on calling clojure or clj at the command line:

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.datomic:ion:pom:0.9.7 from/to datomic-cloud (): Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied
...

Do you have aws creds set?

End vars for keys or profile etc?

I do

Hmm, give me a sec

yep

socks proxy is running

and env vars are set in the shell I’m currently in

Just tried running aws configure. For the avoidance of doubt, the keys currently set are:

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export REGION=
export AWS_REGION=
export DATOMIC_SYSTEM=
export DATOMIC_REGION=
export DATOMIC_SOCKS_PORT=

I’m trying to run either clojure or clj, which I installed via homebrew.

Try setting AWS_REGION too

Did both REGION AND AWS_REGION and no luck

ok so the exception is an access denied.

Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied;

Bastion is working, as I can successfully call: curl -x .$DATOMIC_SYSTEM.$

I worked it out. So going through the docs you end up making two users, a datomic user account and a secondary account which is implied for development. I was under the assumption that because I was doing development, I should be using the secondary account with fewer permissions. When I swapped to the datomic account, it worked perfectly.

Thanks for the help @alexmiller!