Fork me on GitHub
#sql
<
2020-08-14
>
benny17:08:25

i’m pretty sure i’m using with* etc the way i should which should dispose of my connections properly without manual intervention but i now hit a threshold somehow that i can’t even run my tests without getting the following error. in the past it would happen after using my repl long enough, but now it’s happening during a full test. any common pitfalls? i did look through the docs 😉 > Caused by: org.postgresql.util.PSQLException: FATAL: sorry, too many clients already

hiredman17:08:46

there should be some way to ask postgres how many connected clients it has, you should do that and see if you get an expected amount. for unix like operating systems there are commands to list open tcp connections, you should run those and see how many open sockets are connected to postgres, you should do a bisecting search by disabling half your test suite over and over to see if you can isolate a particular case/test/file etc

hiredman17:08:28

or you could just use a connection pool library and configure it to never hand out too many connections

benny17:08:40

i’m using hikari

benny17:08:46

which is why i’m especially surprised

benny17:08:21

specifically hikari-cp

hiredman17:08:31

do you know what the max client limit for your postgres install is and is hikari configured not to exceed that?

hiredman17:08:59

are you running multiple instances of your code and exceeding the limit that way?

benny17:08:00

i was trying to figure out how to line those up, i’m using the pg docker image and hikari pool defaults

benny17:08:29

max_connections in pg is 100

benny17:08:11

maximumPoolSize in hikari is 10 :thinking_face:

hiredman17:08:54

are you creating multiple hikari pools?

hiredman17:08:27

e.g. do you create a single pool somewhere and explicitly re-use it?

benny17:08:37

i’m using an atom in my test util

benny17:08:44

and i’m maybe resetting that a ton

benny17:08:56

looks like it

hiredman17:08:29

connection pools tend to create connections immediately and keep them alive, so if you create 10 pools your connections are all used up regardless of if you are using those pools or not

benny17:08:33

is there a way to re-use a pool across tests then? i believe lein runs tests in parallel so even if i check if it’s been set, it’s a race condition

hiredman17:08:29

lein does not run tests in parallel

benny18:08:31

i was checking wrong, that did it, thanks @hiredman!

benny18:08:36

you lead me down the right path 🙂