Fork me on GitHub
#aws
<
2022-06-04
>
Martynas M13:06:55

Hey. My friend told me that he had a problem when he was upgrading his AWS RDS DB using terraform. He said he experienced 10 minutes of downtime. I think he was upgrading between major versions. How do I write my terraform config so that this wouldn't happen?

apiology13:06:56

The output of terraform plan will tell you what it plans to change. Consult with the AWS documentation to find out what the implications are for those changes.

apiology13:06:16

Your friend was lucky if they weren’t doing that already - some seemingly simple changes require deleting and recreating resources in AWS, including databases!

Martynas M13:06:40

I think he caused prod to go down for 10 minutes. I didn't yet write a single line of terraform as I'm afraid of the options. 😄 😅

apiology13:06:40

Some things in AWS require downtime - I’m not sure what those are for RDS, and it may depend on the database, but I wouldn’t be surprised if resizing is one of them. Others can’t be done directly and require deletion and recreation - for some resources, name changes are like that. Your little PR going from kabob-case to snake-case in identifiers can get a little scary!

Martynas M13:06:20

What does it mean to recreating resources in AWS, including databases? Does it drop the instance or does it drop the storage too? So if it will recreate a DB then will it irrecoverably delete the contents and won't restore?

apiology13:06:58

Up to and including deleting data.

apiology13:06:23

It’s on you to understand the terraform plan, and cross check with AWS docs.

Martynas M13:06:03

I think that my friend said this: "well, you could use terraform with CI-CD stuff". And in my opinion the deletion may mean that you probably wouldn't want to use it with CI-CD for more than execution where you looked up through the plan beforehand :thinking_face:

apiology13:06:27

Yeah, you’d absolutely want a human in the loop for review there.

apiology13:06:17

Usually the CI/CD aspect is there to separate out the roles of writing the terraform from reviewing the plan, and to keep the high-power credentials used to apply the plans out of grubby human hands.

👍 1
orestis16:06:27

Updating RDS always has downtime, even without terraform. Major versions take 10-30 minutes, minor versions are quicker.