holy-lambda

steveb8n 2021-11-04T02:29:39.079500Z

Q: is there any way to attach a shutdown hook fn in HL? i.e. when lambda is unloaded because it’s idle too long, I want a fn to run which disposes network connections

Karol Wójcik 2021-11-05T13:31:06.089800Z

Shutdown event is not supported via internal extension. If I create an external extension still you would not be able to access the resources, since the extension runs in a separate process.

{:headers {nil #object[java.util.Collections$UnmodifiableRandomAccessList 0x176ce308 [HTTP/1.1 403 Forbidden]], Date #object[java.util.Collections$UnmodifiableRandomAccessList 0x3ce7b033 [Fri, 05 Nov 2021 13:29:08 GMT]], Content-Length #object[java.util.Collections$UnmodifiableRandomAccessList 0x1ccbf9ac [116]], Content-Type #object[java.util.Collections$UnmodifiableRandomAccessList 0xd4b0649 [application/json]]}, :success? false, :status 403, :body {:errorMessage SHUTDOWN: ShutdownEventNotSupportedForInternalExtension, :errorType Extension.InvalidEventType}}

steveb8n 2021-11-06T00:03:26.090100Z

ok. not possible at all then?

Karol Wójcik 2021-11-04T05:23:20.081Z

You mean something like: 1. When the lambda has handled the payload. 2. And there is no more payloads, but Lambda is alive. 3. Call the hook?

Karol Wójcik 2021-11-04T05:43:56.082400Z

Or maybe something like: 1. Lambda called the user handler. 2. Lambda is waiting for the response. 3. It's close to timeout, so call the hook?

Karol Wójcik 2021-11-06T10:42:26.090300Z

Nope. Only via external extension that run in the different process.

Karol Wójcik 2021-11-06T10:42:34.090500Z

But this is not something I want to do

steveb8n 2021-11-06T22:46:31.090900Z

thanks for looking into it

steveb8n 2021-11-06T23:39:24.091100Z

my use-case for this was cleaning up Redis connections but I’ve discovered (through load testing) that they are cleaned up automatically as the lambdas are unloaded

steveb8n 2021-11-06T23:39:42.091300Z

so don’t really need to do it in a shutdown hook

Karol Wójcik 2021-11-07T06:16:03.093700Z

Cool. Also great you’re doing load testing. You prove how HL is a stable runtime

steveb8n 2021-11-04T07:16:25.082600Z

neither. when the lambda is shutting down because it’s been idle for too long. then I want to be able to call a hook so I can clean up network connections held in an atom

Karol Wójcik 2021-11-04T07:16:56.082800Z

This is not possible.

steveb8n 2021-11-04T07:17:01.083Z

would be useful for anyone wanting to see when their lambdas are shutdown since it could write to the logs

steveb8n 2021-11-04T07:17:45.083200Z

I thought I read somewhere that it is possible but only with a custom runtime

Karol Wójcik 2021-11-04T07:17:51.083400Z

Basically AWS Lambda does not provide any CPU or memory to the idle lambda. All the operations are stopped.

Karol Wójcik 2021-11-04T07:18:13.083600Z

Otherwise the runtime loop white true would run forever

Karol Wójcik 2021-11-04T07:18:18.083800Z

But it doesn't.

steveb8n 2021-11-04T07:18:24.084Z

and no events while shutting down?

steveb8n 2021-11-04T07:18:29.084200Z

yeah, that makes sense

Karol Wójcik 2021-11-04T07:19:27.084400Z

Hmm. Only the regular Linux shutdown hooks, but I don't know whether the code will be invoked. I can try.

steveb8n 2021-11-04T07:20:29.084600Z

cool. I’m sure I’m not the only person who would like to use this if it’s possible

Karol Wójcik 2021-11-04T07:21:45.084900Z

I will have some more time in few hours, so I will investigate it. 🙂

Karol Wójcik 2021-11-04T07:25:00.085100Z

I'm reading this: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-modify.html. It seems that I would need to create an extension for this.

Karol Wójcik 2021-11-04T07:34:37.085400Z

It looks like the runtime is killed immediately, while the extensions have 500ms to shutdown gracefully. ;O

Karol Wójcik 2021-11-04T07:41:10.085600Z

Ok I do understand what to do, but be aware that you will pay additional $ for this.

Karol Wójcik 2021-11-04T07:42:43.085800Z

https://docs.aws.amazon.com/lambda/latest/dg/runtimes-extensions-api.html#runtimes-extensions-registration-api The hook will be in:

(h/entrypoint ~lambdas {:shutdown-hook (fn [])})

steveb8n 2021-11-04T08:20:44.086Z

sure. it would only be for the extra cpu time of the hook fn right? these hooks should be very small/fast so that this cost is low

steveb8n 2021-11-04T08:21:43.086200Z

even just an extra log line will be useful for most people because then they can see how quickly lambdas are unloaded i.e. when cold starts will be required again

steveb8n 2021-11-04T08:22:08.086400Z

this will force me to upgrade HL 🙂

Karol Wójcik 2021-11-04T08:29:02.086600Z

I'm not sure how this gonna work. I think after the shutdown-hook is called the AWS will not wait? Or maybe I should manually exit the process.

Karol Wójcik 2021-11-04T08:29:14.086800Z

> sure. it would only be for the extra cpu time of the hook fn right? these hooks should be very small/fast so that this cost is low Yeah

Karol Wójcik 2021-11-04T08:30:20.087Z

Cold start would be a little bit higher, since I have to register the internal extension.

Karol Wójcik 2021-11-04T08:30:38.087200Z

Probably few ms, when shutdown-hook is used

steveb8n 2021-11-04T20:57:22.088100Z

It would be optional? If so, we can choose the small trade-off

Karol Wójcik 2021-11-04T21:15:50.088600Z

Yeah. It will be optional.

steveb8n 2021-11-04T21:17:50.088800Z

cool. they are small tradeoffs for this benefit