A database per tenant isn’t realistic--databases are fairly high-cost abstractions and aren’t designed for having large numbers of them.

I suspect most people do it in the app layer entirely. However, if you are careful about assigning tenant ownership to entities in your schema in a consistent and discoverable way, and being explicit about which ref attributes can cross tenant boundaries and which cannot, you can install a lot of guardrails and affordances into your system

At one extreme, you can use a d/filter predicate into a database to filter datoms by tenant

but that has a performance cost which you may not want to bear at scale; so you could use application code and db/ensure to enforce tenancy boundaries on write, and your queries can trust that they were enforced

an offline job could double-check tenancy boundaries periodically; if you find boundary violations and put enough metadata on your transactions, you should be able to squash those bugs fairly easily, or at least have an audit trail of the write.

this is a more out-there idea, but you could encode tenancy into the entity partition (which you probably should do anyway for performance), and that would give you a blast-damage-limiting, very cheap d/filter that would at least ensure a bug can’t break out of the tenants in a partition

anyway, I think the most important thing is make sure tenancy is very clearly expressed in your schema, and then you can build on that to the degree you’re willing to trade safety for performance

or application-run code for datomic-run code

you're using some terms I'm not familiar with and I just want to make sure that I understand what you're implying -- tenancy -- would this translate to a db.type/ref for a user identity? partitions -- I thought about this. I have no idea if a partition per tenant is feasible or not, but that makes a lot of sense. metadata -- Do you mean literal metadata in the clojure sense of metadata or do you mean database metadata encoded via schema?

these are great suggestions, btw, thank you

tenancy: Roughly “ordinary scopes of read and write”--in your example each user would be a tenant. You might have smaller units or larger ones, or even hierarchical ones. I’m just using a general term for this problem.

partitions: you probably want to assign partitions via a hash function rather than explicitly. Partitions are really a performance optimization to increase locality and my suggestion is kind of an abuse of them. note also you can’t change the partition of an entity once assigned (it’s part of the entity id) so if you make a mistake this could be a real inconvenience

metadata: I mean attributes asserted on the transaction entity itself. This is just schema.

things you might assert as metadata: a reference to the authenticated user that performed the operation; the name of the operation in your code; a reference to the tenant, etc

Yeah, makes sense. You know what I think is interesting here is using Clojure in conjunction with Datomic for something like banking. I would imagine you'd want more structured like Java classes or something to put some compile-time enforcement as a backup.

but obviously the case-study says that it works. And from what I understand, non-datomic bank code is a huge mess anyway

I’m not sure how any compile-time approach would work for what is fundamentally a data reference problem

the enforcement has to live over the data at runtime; the question is just how much is lowered into the database runtime and how much perf are you willing to trade for enforcement.

My biggest concern is inappropriate egress, which could potentially occur from inappropriately transacting combined user data, which could potentially occur from merging hashmaps that should not be merged. Can this be handled without classes? Yes. But it's harder to inappropriately merge object than it is to inappropriately merge hashmaps.

Similarly, it's harder to egress "too much" data from a hashmap when marshalled into a class then serialized.

are you talking about queries that unavoidably cross tenants and aggregate somehow? otherwise I’m not sure how classes could enforce that e.g. every entity is from user A.

And yes I know you could use spec to do this, but I try to avoid anything that relies to much on discipline if that discipline can be outsourced to a different layer like the compiler or the database

if you’re just talking about what attributes on an entity are exposed, this isn’t really a tenancy problem. You control that with d/pull on the read-from-db side and explicit serialization code (i.e. just pulling certain fields, renaming fields, etc) on the write-to-wire side

And maybe this is what I need to hear -- that ultimately it is an application layer problem, no matter which way you slice it, and that I need to evaluate if the benefits of having datomic outweigh my concerns about data tenancy.

well it doesn’t have to be application layer--some databases lower quite complex authentication and row-and-column-level authorization controls into their database, but arguably the database is functioning as an application layer at this point

you gotta pay for that safety somewhere

yeah but who wants to use other databases besides datomic 😛

thanks, as always, @U09R86PA4. You probably have no idea how many times you've pulled my @$$ out of the fire regarding Datomic over the years 😅

A database per tenant isn’t realistic--databases are fairly high-cost abstractions and aren’t designed for having large numbers of them.it's perhaps worth noting that there are people trying to address this (but I can't say how successfully), e.g. with Postgres https://www.thenile.dev/blog/introducing-nile

I meant in datomic specifically

Datahike apparently is much closer to zero cost

However any system like this still needs to grapple with the fact that shared-nothing means things like schema aren’t shared, so migrations involve a lot of orchestration

I wish tenancy were a first-class concept in more databases

I use namespaced keywords per tenant. Many tenants in the same db, such as:

:tenant-1.order/total
:tenant-2.order/total 

oh interesting @U0GE6JTKK . does that imply a schema per tenant?

yes, but in my case schemas between tenants may have differences. There are some common namespaces as well. I call a tenant a workspace so you'd have: [:workspace/id :tenant-1] [:workspace/id :tenant-2] and the permissions of who can access what.

my two cents here is that this is really more of an application concern. Datomic provides a good set of building blocks to implement what you need – you’ll eventually need to write tests that prove the application boundaries anyway