Fork me on GitHub
#clojars
<
2022-01-21
>
martinklepsch16:01:21

I’m curious, how is the ops part of Clojars done? For cljdoc I’m running a nomad cluster but I’ve been a very hands-off maintainer of that and I think that will eventually cause some issues 😅 Kinda wondering if it could make sense to have an “ops” initiative that runs and maintains infra for projects like clojars and cljdoc (feels like a big ask but asking doesn’t hurt haha)

tcrawley16:01:54

Ops for clojars is basically just me. It all runs in AWS, and we use terraform to manage it. There is one server instance and one db instance. The server instance is ephemeral, and I build and switch to a new AMI every few weeks to get the latest security updates. Happy to chat more about this/how we can make this easier for cljdoc. See for that code/docs: https://github.com/clojars/clojars-server-config

martinklepsch16:01:24

Nice, that sounds good! cljdoc is running on sqlite which makes the ephemeral instance approach a bit hard but I guess it wouldn’t be hard to swap that for postgres or something

tcrawley16:01:26

Yeah, moving to an ephemeral instance is what motivated the move from sqlite to postgres for clojars. Alternatively, you could copy the sqlite db to s3 after every write, but that feels excessive (unless the write volume is low)

martinklepsch16:01:32

I really mostly went with the nomad thing to have 0 downtime deployments (blue/green) with 1 instance

martinklepsch16:01:58

funnily enough write frequency is basically as often as people publish a new jar on clojars 😄

tcrawley16:01:15

heh, that makes sense :)

martinklepsch16:01:53

The cljdoc infra is also already managed via Terraform but it’s very long-lived and I’ve been dreading the day I need to recreate the instance for a while lol

martinklepsch16:01:46

and one thing that was quite important to me was continuous deployment, e.g. if contributors merge a PR I don’t want to be blocking to get that live

martinklepsch16:01:54

Also using Packer for some basic stuff already, all the ops docs are https://github.com/cljdoc/cljdoc/tree/master/ops

cddr14:01:22

> funnily enough write frequency is basically as often as people publish a new jar on clojars Seems like a good example of real “RESTful” architecture that results in two related systems that logically share data “just working” together even though the two use completely different stacks and have completely independent deployment pipelines. If only most companies’ systems were so well decoupled.