I've been thinking about a SQL migration library where you describe the tables and a tool compares it to the existing database, does a diff, then runs whatever SQL is necessary. Back in the day there used to be a Google cloud computing platform where you could write python and it did something similar (it was a whole hosted framework. I think now they just offer commodity computing services, or does it still exist?). Is anyone familiar with something that already exists like that for Clojure or Java?
Django does exactly that. Horrible, horrible thing. In all my earnest, just don't do it. SQL migrations are easy and simple as they are, trivial even. There is no need to improve upon them.
Noted. I'm interested in automating and enforcing additive only changes in a structured way, rather than just allowing arbitrary SQL statements.
also having the database declaratively defined makes it a nice reference
> I'm interested in automating and enforcing additive only changes in a structured way That would be limiting yourself greatly to obtain pretty much nothing except for bugs in this hypothetical system. > having the database declaratively defined makes it a nice reference It is already declaratively defined, via SQL. If you for some reason need a different syntax, just use HoneySQL. If you want to get the current state of a DB, just dump its schema and maybe convert to HoneySQL.
This looks like what you're asking about: https://github.com/abogoyavlensky/automigrate But again, I would simply not do it, at all. 3.3 kloc for a migration library is way too much. Arguably, even a simple Migratus with its 1.1 kloc is quite "plump" for what it intends to do, but it also includes CLI and a bunch of extension points. Without them, it could be slimmed down to around 500 loc, and trivial ones at that.
Thanks for the link, but not exactly what I'm looking for. I'm well aware I can "just" dump the schema, and I often query the current schema with psql. Exactly which bugs are you expecting would occur?
Erm... each and every one of them. :) The "bugs per loc" metric is never 0 and is of a probabilistic nature. From this perspective, code is a liability, not an asset. And if two libraries do more or less the same thing but the first library has 3x loc of the second one, I would 100% prefer the second one. Even if the feature parity is not at 100% - it's much easier to extend and maintain something simple and concise.
> I often query the current schema with psql
To extend my personal perspective on this a bit - psql is not a great tool here. It's perfectly accurate and gets the job done, but it's significantly limited.
Tools that are SQL-aware and that are built into your IDE are incomparably better. E.g. DataGrip that's built into IDEA lets me autocomplete SQL accurately, highlight errors, warn about stupid things, search the whole schema for usages of e.g. a function or a table name, run the same queries against multiple DBs, write custom scripts to e.g. convert DDL to HoneySQL on click, and so on and so forth.
I pretty much never use psql for development - at this point, it feels like crawling when I'm used to running. And I'm sure there are plenty of other tools that aren't DataGrip that can do similar things.
In the past we have used https://github.com/weavejester/ragtime for similar.
But it doesn't auto-generate migrations based on some kind of schema, does it?