Fork me on GitHub

Hey there, I wrote down this design 50% for fun, 50% b/c it might actually be necessary at work. It doesn't seem too extravagant (in fact I view it as very aligned with RH-isms: immutable/never-broken specs, growth by accretion, no renames etc), but still, there is a gap between theory and practice 🙂 i.e. my design might actually have flaws in a real system. It would be super appreciated if someone with relevant experience could comment in the issue.

Alex Miller (Clojure team)11:06:39

Seems fine. I guess my only real question is whether apps and models really need that level of absolute separation that comes from putting the app name in the attribute qualifier

Alex Miller (Clojure team)11:06:34

Seems like you will end up basically copying the same attribute definition everywhere (and require unnecessary data transformation at each step). The whole idea of the global registry is to get reuse

Alex Miller (Clojure team)11:06:15

There is more coming on datafied specs in spec 2 btw

👏 4

> Seems like you will end up basically copying the same attribute definition everywhere Not sure I can follow. Let's say I have N repos, 1 per microservice, plus 1 special repo, the 'immutable spec registry' Over there, I (spec/def integer?) And microservices can consume that spec (and quite concisely, via require ... :as ...). There's no copying in that > and require unnecessary data transformation at each step What did you mean?

Alex Miller (Clojure team)13:06:57

I assume these services pass data to each other. It seems improbable that you won't have multiple services handling the same semantic attribute (like "age"). in your scheme, that attribute will show up in the messages for all the services -, etc

Alex Miller (Clojure team)13:06:39

if a service ever takes a map with one variant and returns a map with the other variant, you have rebuild the map and change the attribute name

Alex Miller (Clojure team)13:06:15

whereas if you had a :messaging.common.v1/age, it could flow between services and be shared

Alex Miller (Clojure team)13:06:24

which was the whole point of the global registry


This is why we tend to use more generic domain-level prefixes for our specs, rather than specific/namespace-qualified names.


Understood! I'll give it a think, however I find some value in having this design: Each microservice can retain a specific meaning and type for age. That way, services can evolve independently, without coordination. The opposite might resemble a distributed monolith, where making a change/decision needs asking everyone else (other codebases / other team members) first (btw, messaging is via Kafka so the map-in-map-out philosophy doesn't apply that much)


@U04V70XH6 could you give an example of what you mean by “domain-level”? Would you tend to favor :user/email over :myapp.auth.user/email?


For stuff that stays "inside the company", the qualifier doesn't need to be any more specific than the broadest level of uniqueness needed. So, yeah, if you want to standardize on an email representation for all your users across the whole company, :user/email is fine. It shouldn't be part of any public APIs/libraries you create -- but using commonly known domain names inside your company helps reuse across multiple systems.


A specific example is :wsbilling for us -- for a lot of things that are related to our World Singles Billing services rather than being tied to a single app, these specs are reused by all apps that support and interact with those services. We've used :wsbilling there rather than just :billing because there are other generic billing services we interact with, so this just hints that it's about our billing services rather than anything else we interact with.


Very helpful, thanks


@U45T93RA6 "making a change/decision needs asking everyone else (other codebases / other team members) first", as RH says in Spec-ulation, the way we use the word change tends to complect growth and breakage, which are completely different. As long as you're only doing growth, these is no need to ask (or even tell) anyone.


That's why it's so important to me that next.jdbc will never make breaking changes -- only accretive/fixative changes, since it moved into beta. I learned a lot of lessons from maintaining for the last eight years! 🙂

💯 4

@ jaihindh, how do you tell if a change is breaking or not? as hinted in the document, One man's non-breaking is another's breaking. My approach is pessimistic in that it assumes all changes are potentially breaking. At the same time, it encourages services to keep sending/accepting messages at old versions, so you don't actually break 3rd parties. This philosophy is partly justified by the fact that atm we don't have a tool that automatedly analyses spec compatibility. Might be unfeasible given that specs can be backed by arbitrary code

👍 4

I don't have the answer (don't even have an answer) for how to partition "change" into "growth" and "breakage". And no matter how hard we try, at some point people will rely on things that are broken, making one man's fixation another man's breakage, etc. But if we keep changing our names by incorporating versions into them and ask people to assume they are potentially breaking, we still lack a stable set of names that can act as a stable basis to reason about our data across time. That, is the real systems problem right? Sorry for the blabbering & noise 😅


You don't change the names. Just introduce new names. The old names still exist and have the old behavior. Folks can continue to rely on those old names "forever".

💯 4