holy-lambda

Karol Wójcik 2021-11-24T08:21:50.123300Z

Hi guys. I would like to ask you whats your opinion on removing the :envs property from the ctx map. The reason I found it a necessary change is that I see to many people logging the whole event and exposing their AWS_SECRET_KEY. By removing the :envs property the runtime should also perform slightly better.

emccue 2021-11-24T15:02:48.123400Z

What was the purpose of envs? Just a shorthand for (System/getProperty)?

michaelwhitford 2021-11-24T16:03:27.123600Z

Couldn't you strip out the secrets instead of dropping all the environment variables? I'm currently using it for some other vars, though I guess I could swap to using java interop to get them.

Karol Wójcik 2021-11-24T19:29:43.136400Z

Idea behind adding envs to ctx was to support calling native-agent-payloads for generating GraalVM native-configurations.. Basically HL gots a mechanism that can gather edn files with payloads and in for loop call specified Lambda with corresponding payload. This forces GraalVM native agent to gather all possible reflections, resources, JNI usage etc. The way to go should be to use System/getenv. For generating native-configuration one can still put envs in ctx:envs and then use (or (System/getenv “XX”) (get-in ctx [:envs “XX”])) There is also another reason why I want to remove envs in ctx. To many times I’m being asked how to get envs out of handler context. Envs in ctx are just too misleading.

steveb8n 2021-11-24T20:18:14.137Z

I use it in every lambda so this is a breaking change for me. I'd prefer you redact the sensitive fields in .toString or something like that. I can patch but accretion is 'the way'

Karol Wójcik 2021-11-24T21:32:27.140700Z

The problem is that envs in general contain sensitive fields that should not be logged.

Karol Wójcik 2021-11-25T06:51:00.145500Z

Sooner or later this is something I should address. @steveb8n I’m not planning any new changes beside analitics and envs in the core runtime, so I think you can stay with the current version of HL. The next few months I will spend on tasks, ring/pedestal adapter, live environment improvements.

steveb8n 2021-11-25T06:52:08.146800Z

yeah, I’ve accidentally logged it myself in the past also

Karol Wójcik 2021-11-25T06:52:26.147200Z

I also hope that after few months I would be able to promote the new version to 1.0.0 and mark core as a complete/stable.

steveb8n 2021-11-25T06:52:48.147800Z

in cloudwatch logs for lambdas they redact the payload unless you specifically print the nested data

steveb8n 2021-11-25T06:53:17.148Z

I wonder if you could provide some kind of masking like that? it’s mostly just the aws creds right?

Karol Wójcik 2021-11-25T06:54:17.149600Z

Not only I think. One can put unmasked DB creds, some tokens, api keys etc.

Karol Wójcik 2021-11-25T06:55:34.151500Z

There is no way I can „quess” what should be hidden. Also even I would start masking those values it's already a breaking change to someone.

steveb8n 2021-11-25T07:18:00.151700Z

makes sense

steveb8n 2021-11-25T07:18:54.151900Z

I don’t really mind this breaking change. I can adapt easily. it’s just not the clojure way but then again you are not v1.0 yet either

Karol Wójcik 2021-11-25T07:37:58.152100Z

Right. There was a lot more breaking changes before, but the more we're close to v1.0 the more runtime is stable, and less breaking changes are introduced.