clojars

martinklepsch 2022-01-21T16:08:21.007900Z

I’m curious, how is the ops part of Clojars done? For cljdoc I’m running a nomad cluster but I’ve been a very hands-off maintainer of that and I think that will eventually cause some issues 😅 Kinda wondering if it could make sense to have an “ops” initiative that runs and maintains infra for projects like clojars and cljdoc (feels like a big ask but asking doesn’t hurt haha)

2022-01-21T16:11:54.008200Z

Ops for clojars is basically just me. It all runs in AWS, and we use terraform to manage it. There is one server instance and one db instance. The server instance is ephemeral, and I build and switch to a new AMI every few weeks to get the latest security updates. Happy to chat more about this/how we can make this easier for cljdoc. See for that code/docs: https://github.com/clojars/clojars-server-config

martinklepsch 2022-01-21T16:13:24.008500Z

Nice, that sounds good! cljdoc is running on sqlite which makes the ephemeral instance approach a bit hard but I guess it wouldn’t be hard to swap that for postgres or something

2022-01-21T16:14:26.008700Z

Yeah, moving to an ephemeral instance is what motivated the move from sqlite to postgres for clojars. Alternatively, you could copy the sqlite db to s3 after every write, but that feels excessive (unless the write volume is low)

martinklepsch 2022-01-21T16:15:32.008900Z

I really mostly went with the nomad thing to have 0 downtime deployments (blue/green) with 1 instance

martinklepsch 2022-01-21T16:15:58.009100Z

funnily enough write frequency is basically as often as people publish a new jar on clojars 😄

2022-01-21T16:16:15.009300Z

heh, that makes sense :)

martinklepsch 2022-01-21T16:17:53.009500Z

The cljdoc infra is also already managed via Terraform but it’s very long-lived and I’ve been dreading the day I need to recreate the instance for a while lol

martinklepsch 2022-01-21T16:18:46.009700Z

and one thing that was quite important to me was continuous deployment, e.g. if contributors merge a PR I don’t want to be blocking to get that live

martinklepsch 2022-01-21T16:19:54.009900Z

Also using Packer for some basic stuff already, all the ops docs are https://github.com/cljdoc/cljdoc/tree/master/ops

2022-01-25T14:55:22.010200Z

> funnily enough write frequency is basically as often as people publish a new jar on clojars Seems like a good example of real “RESTful” architecture that results in two related systems that logically share data “just working” together even though the two use completely different stacks and have completely independent deployment pipelines. If only most companies’ systems were so well decoupled.

2022-01-25T15:14:53.010400Z

good point @cddr!