Clojurians Log v2

We use separate AWS accounts (both under one top-level management account) for our production, staging and development environments. We use the same ion-config.edn for both (by using consistent naming in the CF template). For tests, we use db-local. For the deploy (to development or production or staging) the trickiest part is getting old-school AWS Java SDK v1 credentials setup (we normally use SSO credentialing, and something like https://pypi.org/project/aws2-wrap/ to bridge the gap.

To inject configuration by environment, we use SSM (using different accounts makes this really easy), SecretsManager and (for stable, non-sensitive stuff) an EDN config file that accompanies the deploy and is read at startup by aero. It leverages the aero profile feature to differentiate amongst the environments. We read the stage via the CF template using the Ion EnvironementMap option. In our case, in production, it is set to {:stage :production}. For tests (which use db-local) we just set the DATOMIC_ENV_MAP to {:stage :test}.

If I understand the crux of your question, the key is the datomic EnvironmentMap feature. With just one value set in there (and never changed) you can branch on the environment and feed that to aero for reading an environment-specific configuration.

i actually just started walking that path - figured out aero + DATOMIC_ENV_VAR and ion/get-env may be the solution thank you for sharing! i will make use of that

That is the magic ticket IMO.

in your setup, how and when do you execute the db migrations (i guess using caribou)? i want to run the migrations before the ions are deployed, but i don't want to allow access outside the compute group either (e.g ci)

We run migrations at system startup -it's not an external process.

(and yes, we use caribou for that)

do you use ions? i wonder how can i execute migrations on startup (before traffic is routed)

Yes, we do use ions. You are headed towards the subject of lifecycle hooks. Which are lacking for atomic ions. @jaret has that on his wish list, but until then there will be challenges.

The approach we use of having Caribou migrate before the system starts up ensures that migrations are done before an individual node can run new code to service requests. But coordinating that across all nodes isn’t really possible. The solution is to always use backwards compatible migrations!

adding an attribute to an endpoint before updating datomic schema will break the api datomic cannot execute a query for an attribute that doesn't exist, but adding attrs is backward compatible for example, http endpoint for "get-user" that return a new attribute

For that kind of problem, you could adopt the approach that we use: transact the schema as part of system startup. Then you know that you will never run new code against old schema. You still run the risk of running old code against the new schema, but if you follow an "accretion only" methodology then that is not a problem. We've been successful with that approach for years. You could also transact the new schema in a blocking step in the CD portion of your pipeline and get the same guarantees.

i understand, thank you i wonder what is the system startup part when using lambdas, because it just reference to fns

For that to work, I create a closure around all the handlers (lambda and http direct) that force a system startup on the inbound event.

This is where it would be really nice if ions had lifecycle hooks.

But, the closure approach works.

oh i see, so for every incoming event you run the migrations? which actually doesn't execute since there's no new migration

No. The system is started once when the first event arrives. During system startup, migrations are run to apply the deltas. What you describe could work, but it would be quite slow.

(starting the system exactly once is achieved by a closure around the event handlers and an atom)

assuming the lambdas kept alive. for example for cold starts it will execute multiple times, correct?

Clojure code does not run in the lambdas at all. The lambdas run code provided by the Datomic Cloud Ion platform. The code is simple: relay the event to the long-running EC2 instance and wait for a response. The EC2 instance is where your Clojure code runs.

I think this is a common misconception about ions: lambdas are nicely integrated into the platform, but you never have to write lambda code or worry about cold starts and their "ephemerality".

(FWIW, I have never even seen the code that runs in the lambdas. I just know its effects)

ohh right! i got confused there but i knew it thank's for bringing that up

so it's max executed by the number of machines (atom per jvm), right? a lifecycle could have been good here, but i guess we can also leverage the cloud or ci to max it at 1...

Correct. A combination of accrete-only practices and a tool (caribou, in our case) that minimizes the db interaction required to skip already-applied migrations makes it work well.

there's something nice in handling schema migrations directly in the code (as txs) without external tools

The real payoff for us has been in testing and development. But there are other benefits derived from the very strong binding of code + schema.