Fork me on GitHub
#holy-lambda
<
2022-01-13
>
azimpel11:01:00

Hi guys, I'm trying to create a native executable which is using the cognitect-aws library. Unfortunately the required service.edn file seems to be omitted during the execution of bb hl:native:executable. I've already generated the native configuration files via bb hl:native:config task and added the "\\Qcognitect/aws/dynamodb/service.edn\\E" pattern to the includes array. But still, the EDN file is missing in the final bundle. Am I missing something?

Karol Wójcik13:01:30

Yes you're missing something

Karol Wójcik13:01:52

https://fierycod.github.io/holy-lambda/#/native-backend?id=agent-in-context-calls Add agent in context call to a service you are using, so that native:config can gather the required EDN files for you.

Karol Wójcik13:01:36

For instance:

(agent/in-context 
   (aws/invoke @dynamodb-client {}))

azimpel17:01:34

Thank you @UJ1339K2B for this hint. Wrapping single function calls with the in-context macro in combination with the additional includes pattern seems to fix the described issue. It's not quite clear to me how to use the in-context macro exactly and what is the purpose of agents in this context.

Karol Wójcik18:01:14

Have you read the documentation about GraalVM tracing agent? Basically GraalVM requires a lot of statically provided information to correctly produce a working binary. Reflection and resources are sometimes not covered in the main path of analysis therefore it's required to either populate configuration by hand or to generate the configuration via the Java GraalVM tracing agent. The agent analyses the additional code paths provided in agent/in-context call and produces the required configuration for valid binary. If you cover all the reflective paths/(or paths of code where the resources are used), then you don't need to add any additional entries by hand.

Karol Wójcik18:01:57

You can try removing the native-configuration entirely. Then regenerate the folder via bb native:conf supposing agent/in-context calls are still in the code. You will see that you don't have to add anything to configuration by hand in this case.

Karol Wójcik12:01:28

@U5ABB4GQP let me know if it makes sense to you

azimpel08:01:34

Hi @UJ1339K2B, thank you for your support. I've studied the documentation you mentioned above. The point is that I've managed to build a simple function which should read an entry from a DynamoDB table. I've compiled the function using holy-lambda babashka tasks (compile, config, native-executable) without any errors and deployed it to AWS. But unfortunately the function returns nil instead of the expected result map. I'm pretty sure it has nothing to do with holy-lambda functionality.

Karol Wójcik08:01:49

How to you return the value?

azimpel08:01:04

The code looks like this (simplified): (:require [cognitect.aws.client.api :as aws] [cognitect.aws.credentials :as credentials] [fierycod.holy-lambda.agent :as agent] [fierycod.holy-lambda.core :as h] [fierycod.holy-lambda.response :as hr]) (let [client (make-client ,,,) result (agent/in-context (aws/invoke client {:op :Query :request q}))] (hr/json result))

azimpel09:01:43

Maybe the cognitect-aws library is "too dynamic" to use it with GraalVM ootb?

Karol Wójcik09:01:01

Where is an entrypoint and lambda definition?

azimpel09:01:39

sorry, the known issue with code examples on the internet:slightly_smiling_face: (:require [cognitect.aws.client.api :as aws] [cognitect.aws.credentials :as credentials] [fierycod.holy-lambda.agent :as agent] [fierycod.holy-lambda.core :as h] [fierycod.holy-lambda.response :as hr]) (defn fetch [client q] (agent/in-context (aws/invoke client q))) (defn LambdaFunction [{:keys [event ctx] :as _request}] (let [http-client (agent/in-context (aws/default-http-client)) client (agent/in-context (aws/client {:api :dynamodb :region "eu-central-1" :credentials-provider (credentials/default-credentials-provider http-client)})) q {:op :Query :request {:TableName "MyTable" :KeyConditions {"PartitionKey" {:AttributeValueList [{:S "SomeValue"}] :ComparisonOperator "EQ"} "CacheKey" {:AttributeValueList [{:S "OtherValue"}] :ComparisonOperator "EQ"}}}} result (fetch client q)] (hr/json result))) (h/entrypoint [#'LambdaFunction])

Karol Wójcik09:01:47

This is not how agent/in-context work. You have to include aws/in-context call in the top level of your code (not in function). Can you confirm if your query returns a proper result in your REPL?

azimpel09:01:23

Yes, I'm able to retrieve the expected result using the defined LambdaFunction locally. But to achieve this I have to remove the agent/in-context calls. I'll try your suggestion out, tnx.

Karol Wójcik09:01:11

Can you print the result and check what is there on Cloudwatch? Btw does your Lambda have permissions to read from DynamoDB? I do also recommend building the native executable and trying it out locally via sam local invoke.

azimpel10:01:15

I cannot confirm that the lambda gets actually executed. I've doublechecked the lambda permissions and it looks correct. Another thing which confuses me is the fact that all prints within the in-context macro gets swallowed and I do not see any output.

Karol Wójcik10:01:20

Can you execute the Lambda locally?

azimpel10:01:44

I'll try and will get back to you 👍

azimpel11:01:05

Do you have an idea why the prints gets swallowed?

azimpel19:01:15

> Can you execute the Lambda locally? Unfortunately I can't do it at the moment without further investigation. AWS SAM assumes that you use API Gateway but we don't. To keep up with my deadline I'll pause this PoC to proceed on the existing lambda implementation written in Rust. Nevertheless, here's my (interim) conclusion: This project looks very promising and could help to lower the entry barrier into the Clojure+Lambda topic. Thanks for your work! But it's very hard to proceed smoothly without deep understanding of underlying technologies (well, like almost always). The cognitect-aws library is exceptionally dynamic compared to other libs. Cheers, Andreas

Karol Wójcik20:01:06

Thanks for your insights. As far as I remember a function can be called locally even without the AWS Api Gateway. Anyway all the best! If you ever need any help don't hesitate to ask. ;)

azimpel11:01:29

Do I have to mount my local host files from resources/native-configuration into the docker container via [:holy-lambda/options :docker :volumes] in my bb.edn file?

Karol Wójcik14:01:04

All files from the directory in which the native:conf runs is auto mounted in docker context.

👍 1