Fork me on GitHub
#datomic
<
2017-07-20
>
wistb02:07:16

thank you @schmee @val_waeselynck @stuartsierra . We are thinking of microservices. personally, I am extremely concerned about the transactions, latency, consistency issues surrounding a kafka/micro-se/materialized-views approach. One thought is , use kafka as log but not as event source. instead use datomic as the database and as coordination scheme for intra-service coordination. In this model, the external commnds are logged to kafka and will be handled by some entry service. after that point, the mutations done by all services are over datomic to which all the services have visibility to. all the mutations and internal commands between services can be logged to kafka but only as a 'FYI'. In this mode, it is not a 'micro-service' per say, but, 'modular services'. if all our data needs can be supported by one datomic instance, there is no problem, but, looks like it is not the case. so .. do you think, the pain of 'distributed tx over multiple datomic' is more manageable than the pain of 'distributed tx in a event sourced design' ? any thoughts are appreciated.

val_waeselynck05:07:01

I tend to agree with you. An event sourcing system as you describe can only accept events (I.e definitive information, as opposed to commands, which call for decision) as an input, and is essentially asynchronous. As soon as your system must process commands or write synchronously, you need some sort of transactional front-end, for which Datomic is an excellent choice. If data size is a concern, you should consider making a hybrid system in which Datomic manages only a subset of the data - it may be more viable than sharding.

wistb08:07:26

thank you. The moment you have a hybrid system , you will be required to handle the distributed transaction, right ? and if you are dealing with DT, then, I could as well have another datomic instance than any other database. And the question of DT is becoming real no matter what database I use over a "kafka/Event-sourcing/micro-ser" architecture.

hmaurer20:07:14

Hi! Quick question about Datomic data modelling: it seems that relying on attributes present on entities to determine what “type” of entities they are is a bit messy. Would it make sense to tag entities with something like :node/label, defining their type?

hmaurer20:07:48

e.g. {:node/squuid ... :node/label :node.label/person ...}

favila21:07:30

that is a common technique

hmaurer21:07:00

Cool, thanks @favila 🙂