Clojurians Log v2

> want to keep things managable Something that I already know, so still Debian. :D Well, Ubuntu. Package isolation is useful is the VPS is used for multiple things. Personally, I don't have that, but yeah, maybe docker/podman or LXC.

I've been running a Wordpress, personal website, several experiments, etc etc on there. I've been stuck several times with the PHP upgrades that forced me to upgrade Debian etc. It was tiring :) So maybe the docker stuff will save me that hassle

lesson i learned was do ubuntu LTS not other ones

not sure if you’re into that kind of thing, but asking claude how to make some infrastructure as code setup for your particular setup i bet would be pretty good

yeah I ran Debian LTS, is that much different?

same thing. the code repositories go away for non LTS versions. so if you don’t upgrade for a while you get bricked

i think i had ubuntu 19 or some flavor of that. and wanted to update stuff in 2021 and none of the code repos were around

yeah I'm not against using claude for this, but just trying to get some good info first. It seems kind of random what it advices. It started selling me NixOS but then later when I mentioned podman+quadlets which I randomly picked up somewhere, it first found it overkill but later it seemed quite ok, so I'm not sure if I can trust claude to decide on the global setup (which of course can be subjective)

nixos fixes problems i have not run into in the wild. I know one friend who really liked the nix layer on top of macos. not sure if he’s still enjoying it now

oh I've been trying that too and it seemed overkill, but for a fresh machine it may be good. one alternative claude mentioned was an ansible playbook (to get the fresh machine reproducible too)

now that you mention it, my friend @jakebasile really enjoyed using ansible to make all of his machines (osx and linux) ready to go very quickly

Yeah I use Ansible for this instead of something heavier like nix. It works very well, and I can set up a new desktop or server in a couple commands. It's just for me so it doesn't have a lot of wiggle room as in it will only work on Ubuntu and I usually have a hard requirement on the latest release.

I use ansible to spin up a rocky 9 machine with podman to run my containerized stuff. I prefer how it integrates to systemd, and the slower upgrade cadence. You won't get the freshest versions that way, but if I need fresh I use a container that runs on the stable server.

@michael819 exactly. ok. ansible sounds good to get the base machine right (Debian or Ubuntu LTS probably same difference). but then: docker compose or podman and what the hell are quadlets?

another thing to decide on: rootless docker or don't bother?

Take a look at LXD as well. It's also a container but runs full systems instead of docker’s “one process”

One of the reasons I prefer RHELish with podman is the easy rootless containers, and SELinux on the server and the containers.

Redhat owns podman, and I have run RH in my work since the mid 90s so I am very comfortable with it.

For instance I have one VPS that I slice up into smaller ones with LXD. Each of those can be created or destroyed (set up with my Ansible playbooks) easily. It comes with ZFS based snapshots for free. I use it for dev boxes and for game servers.

Funnily the most reliable Linux for me has been arch. You update often and it stays out of the way. You never need an upgrade like Ubuntu lts releases, just update e.g. weekly. Hell I sometimes update after 3 months and there's no hiccups

As for containers and such - the more mainstream, boring tech you choose the less you will need to fiddle with it

So I'd vote docker compose

FWIW I still use docker. Just inside of LXD system containers :) you have to turn nesting on.

and would you choose an ARM VPS or Intel? (I just read Arch is intel only?)

I would 100% go x86 but I run game servers which are approximately never compiled for arm.

heh i hadn’t thought about that. these days i might choose arm so that production is similar to dev 🙂

I am in the middle of (and will likely be for a while) of moving my home hypervisor from an old fedora install to something ucore (a variant of fedora core os I guess? does rpm-ostree stuff), my home container stuff is all via podman

Most dev stuff probably works fine on ARM. Especially anything JVM. It's usually cheaper too.

I guess I should check wordpress, postgres, mariadb

Those will probably all work on ARM with no issues. game servers and workstation stuff doesn't get ported but the server side stuff usually does.

I've been using docker compose for most of my home infra services for years and it's pretty low maintenance. Ubuntu LTS works fine as a docker host, and most things run on arm64 (I've even got a few arm32 SBC nodes running happily downstairs in a moosefs cluster, but their days are numbered). The last couple of nodes I've added to the server cabinet (or more accurately, server wardrobe) I've setup with NixOS. Still using docker on top of that for most of the services. That approach adds a little bit of work upfront, but makes it trivial to provision a new node or re-image an existing one that's run into difficulties. It also gives me a consistent shell environment across nodes -- well worth the effort IMO. And being able to roll back when it all goes wrong is a life/sanity-saver

Any recommendations for a VPS provider? Hetzner seems full right now...

Maybe NetCup is any good...?

Pay just a bit more to get a provider that makes that ? go away.

any recommendations?

I use http://hostus.us and have a VPS in dallas and one in amsterdam.

You can set reverse dns for your VPS. It's good, and not expensive.

But you are stuck with Debian truly.

ah wait, I just got a Hetzner intel box now :)

Hey you got those big bucks.

I mean one just came free, it's the cheapest one

just use boring tech: Debian/Ubuntu with rootless Podman Quadlets and Ansible. That's a good and proven approach for sysadmins. There are other ways but this works very well.

I started with nix but the box already didn't come up after claude forgot to install some driver that was important for virtualized machines. Screw this, I'm surely going with ansible now.

and debian 13

any reason not to go with docker compose but with podman+quadlets?

compose is fine as well but rootless is more secure

you can achieve rootless with docker as well

I haven't played with podman. But per @timok / boring tech -- you may be more likely to have supported / official docker images for all the services you want to stand up than podman? That's completely uninformed though - maybe podman has a large ecosystem.

@cormacc podman and docker is essentially the same thing. you can have any image because format is standardized

Ubuntu + Docker since 2010 ? Can’t remember exactly. I started on Amazon. Archiva, gitlab, email server, website,… Moving from Contabo to OVH before the end of the summer.

2010?

FreeBSD with Bastille... https://bastillebsd.org/ No need for anything else. Maybe poudriere for custom package builds if needed. And packer if one wants to build custom machine images. (This is basically where I am at personally, for hobby and for commercial work. I plan to host on vultr (no affiliation), because FreeBSD is a first-class citizen on their platform.)

I settled on debian 13 + docker compose today and I'm almost done. It's running on hetzner in Helsinki (the only available location). but man, copy paste a password in the hetzner console.. doesn't work

I'm only allowing myself to log in directly via tailscale (if I'm on the tailnet), but if tailscale fails, I need the hetzner console as an escape

brrrrr... I wanted to choose Hetzner, but they inexplicably dropped support for FreeBSD some years ago. Also they don't have an India location. Vultr has both... I've never used them, but based on what I have seen, it seems like a reasonable bet, long-term... Speaking as someone who's worked with AWS, Azure, and GCP and would rather not if he can help it.

(IAM wrangling is crazy-making, and egress costs are egregious.)

why FreeBSD? i’ve heard good things about it but what pushes you to that over linux?

Well, I have a long answer, but the short one is that I want a rock-solid solution, with as few dependencies as possible, for a whole bunch of problems/requirements that those people solved, like 20 years ago.

For anyone interested, these instructions helped me get a rootless FreeBSD install going on my Ubuntu 24.04, using QEMU and KVM: https://sethops1.net/post/run-freebsd-in-qemu-on-linux/ For Mac (or heaven help your soul, Windoze), you'll have to use Virtualbox or something like that. The documentation out there is helpful.

Oh, I'd posted a long answer elsewhere... here it is copy-pasta'd into a gist: > Preliminary thinking-aloud on Why FreeBSD + ZFS for all my servers? Especially since FreeBSD 15 dropped. https://gist.github.com/adityaathalye/889136f0fcbf2c929c2da729f290c21b I should blog it out.

I've been running my VPS on NixOS for 11 years and wouldn't have it any other way.

The migration is over and I'm left with one Ansible + Caddy + some Dockerfiles (compared to a manually accumulated setup with some notes and there). I'd try nixOS the next time, perhaps, but I already got a bad taste today with the leftovers from my previous attempt om my mac that got in the way today + the non-booting machine when the driver was forgotten. I'm just not expert enough to deal with Nix and somehow it always ends bad for me, like linux on the desktop, I'm just not made for this ;). So I'm happy with the intermediate.

Very nice. Surprised no babashka scripts :)

A couple of babashka services are running in Docker :)

like this one: https://reagami-ssr-live.michielborkent.nl/ and I have a bb script to monitor the memory/disk usage of each service.

Infrastructure as Clojure is the phrase I believe

Uncloud is worth looking into also

If you use docker conpose already uncloud is an easy next step

uncloud seems useful when you have multiple machines right?

Works from 1-N

I did consider exoscale but their prices were a bit too high compared to the low requirements I had ;)

You can dm for vouchers/sponsoring on the hosting part. We use enough of your stuff to probably provide hosting for free

I'll DM you :)

I am on vacation now, but I can relay this on Monday