has anyone had issues where continuous requests start to take longer and longer and then eventually start timing out?

Continuous requests?

just having a client repeatedly make requests over a period of time

polling for data basically

it usually runs under 250ms but then every so often just starts to take more and more time, until its in the hundreds of seconds

eventually the healtcheck request from the instance manager times out and it restarts the machine

and the problem is gone again for a period of time

seems like some sort of resource is getting tied up, threads arent being freed or something but I don't know enough about the underlying server architecture to really know whats going on

Which adapter are you using? Are you using async or synchronous handlers? If you’re just using ring-jetty with synchronous handlers, which is the most common setup, you might have something in your handler that’s holding resources but not releasing them.

Ring-Jetty with sync handlers doesn’t keep anything open by default.

250ms is a pretty slow request in the first place. I’m curious as to what work you’re doing in such a request @mattsfrey ?

it has to essentially make multiple calls out to s3 before returning a response

so thats causing the larger times

using async I believe

Does it use async handlers? Maybe it’s not closing the response?

Or maybe you’re keeping I/O connections open

potentially

do you know of an easy way inherent to ring to view whats going on behind the scenes? I'm at the point of hooking up a jvm profiler really

It doesn’t sound like it’s Ring specifically that’s the problem, but what’s going on in your handler after Ring has passed off the request to it.

A profiler is a good place to start.

yeah haven't hooked up the profiler but checking our datadog logs it seems to be using an increasing amount of memory until it breeches the box's allocation

so something is using memory and not freeing it