i hadn’t thought about that, no. are you thinking you want to resize your pool dynamically?

at least with hikari im not sure how much on-the-fly reconfiguration is even possible without stopping and then creating a new pool

Well the problem I’ve seen is that you can have a handle to a connection that you think is a primary but it gets “demoted” to replica on autoscale events. In these cases, it is sometimes necessary to basically reacquire a fresh connection. It is the reason for this existing: https://github.com/FundingCircle/pg_failover

ah i see

I saw it in two jobs now (out of two I’ve had exposure to aurora) and figured it must be something everyone who uses it needs to solve but maybe that’s jumping the gun 🙂

hm no it’s interesting and i’m glad you brought it up. i am building against an aurora pg db and i don’t think anyone has considered this/we haven’t run into autoscale issues that changed the primary

i might be showing some ignorance here but… DNS?

as in, if you point at a hostname which always refers to the primary, then maybe you could just get away with low TTL?

(just thinking out loud here)

what are you using for connection pooling?

Ah yeah that might be a better way to solve it. We’re using hikari now.

for some reason i remember in a past gig we were on aurora and we hit recovery mode, but i don’t recall if we switched DNS for the primary or if it just… was broken for a bit

could have been a full deploy from being fixed

but it was during some known maintenance scenario

https://docs.aws.amazon.com/whitepapers/latest/amazon-aurora-mysql-db-admin-handbook/dns-caching.html

ah so i wasn’t crazy!

https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-jvm-ttl.html

https://github.com/brettwooldridge/HikariCP/issues/1705

not really sure what to make of this, but maybe worth being aware of

though i do note that the second to the last post says “we have a ttl of 30sec”…“it takes about 1 minute for the connection to recover”

with a pool size of 100, depending on how they’re testing, that could be about right, but it would also depend on the default hikari opts, which i don’t know all of off the top of my head

as a for instance, I recall the hikariCP docs saying they strongly recommend setting maxLifetime , for instance, as the default is like 30min, connectionTimeout defaults to 30sec, etc.

anyway, seems like it should be easy enough to test that you’re configured correctly during a simulated failover