This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-05-02
Channels
- # announcements (1)
- # babashka (4)
- # beginners (39)
- # calva (36)
- # cherry (11)
- # cider (23)
- # clj-on-windows (3)
- # clojure (105)
- # clojure-brasil (1)
- # clojure-chicago (3)
- # clojure-conj (8)
- # clojure-denver (4)
- # clojure-europe (18)
- # clojure-germany (5)
- # clojure-hungary (13)
- # clojure-nl (1)
- # clojure-norway (31)
- # clojure-sweden (9)
- # clojure-uk (2)
- # clojurescript (22)
- # core-async (4)
- # cursive (8)
- # data-science (25)
- # datomic (14)
- # devops (1)
- # emacs (9)
- # events (5)
- # holy-lambda (32)
- # hyperfiddle (26)
- # introduce-yourself (2)
- # kaocha (1)
- # leiningen (11)
- # lsp (17)
- # malli (8)
- # off-topic (84)
- # pedestal (4)
- # polylith (2)
- # re-frame (17)
- # reitit (1)
- # releases (1)
- # remote-jobs (1)
- # shadow-cljs (8)
- # sql (4)
- # tools-deps (8)
- # transit (5)
- # vim (1)
- # vscode (1)
- # xtdb (45)
Hello, I'm trying to use Faraday to connect to DynamoDB in a native lambda and get lots of confusing errors about log patterns. Are there packages I should exclude in deps.edn? Faraday is bringing in commons-logging.
Hey, Could you post the stacktraces? You can also try to get extensive help on #CAJN79WNT channel ;)
I think it might be something to do with the cognitect.aws classes that faraday uses, so I'm adding those to the reflect.json, but here's last night's stack trace and odd logging errors :
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2 internal initialization logging.
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:142)
at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:137)
at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:100)
at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:80)
at taoensso.faraday$client_params.invokeStatic(faraday.clj:152)
at taoensso.faraday$fn__6311.invokeStatic(faraday.clj:177)
at taoensso.faraday$fn__6311.invoke(faraday.clj:176)
at clojure.lang.AFn.applyToHelper(AFn.java:154)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$memoize$fn__6894.doInvoke(core.clj:6342)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at taoensso.faraday$db_client.invokeStatic(faraday.clj:208)
at taoensso.faraday$put_item.invokeStatic(faraday.clj:1013)
at taoensso.faraday$put_item.doInvoke(faraday.clj:994)
at clojure.lang.RestFn.invoke(RestFn.java:445)
at bmcgavin.birthday.db.dynamodb$db_put.invokeStatic(dynamodb.clj:29)
at bmcgavin.birthday.db$fn__7221.invokeStatic(db.clj:21)
at bmcgavin.birthday.db$fn__7221.invoke(db.clj:21)
at clojure.lang.MultiFn.invoke(MultiFn.java:229)
at bmcgavin.birthday.api$put.invokeStatic(api.clj:56)
at bmcgavin.birthday.api$put_handler.invokeStatic(api.clj:66)
at bmcgavin.birthday.api$put_handler.invoke(api.clj:60)
at compojure.response$fn__612.invokeStatic(response.clj:47)
at compojure.response$fn__612.invoke(response.clj:35)
at compojure.response$fn__562$G__557__569.invoke(response.clj:7)
at compojure.core$wrap_response$fn__2242.invoke(core.clj:158)
at compojure.core$wrap_route_middleware$fn__2226.invoke(core.clj:128)
at compojure.core$wrap_route_info$fn__2231.invoke(core.clj:137)
at compojure.core$wrap_route_matches$fn__2235.invoke(core.clj:146)
at compojure.core$routing$fn__2250.invoke(core.clj:185)
at clojure.core$some.invokeStatic(core.clj:2705)
at compojure.core$routing.invokeStatic(core.clj:182)
at compojure.core$routing.doInvoke(core.clj:182)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invokeStatic(core.clj:669)
at compojure.core$routes$fn__2254.invoke(core.clj:190)
at fierycod.holy_lambda_ring_adapter.core$ring_LT___GT_hl_middleware$fn__2699.invoke(core.cljc:150)
at clojure.lang.Var.invoke(Var.java:384)
at fierycod.holy_lambda.custom_runtime$next_iter.invokeStatic(custom_runtime.clj:80)
at fierycod.holy_lambda.custom_runtime$next_iter.invoke(custom_runtime.clj:60)
at clojure.lang.Var.invoke(Var.java:399)
at bmcgavin.birthday.api$_main.invokeStatic(api.clj:97)
at bmcgavin.birthday.api$_main.doInvoke(api.clj:97)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at bmcgavin.birthday.api.main(Unknown Source)
Caused by: java.lang.IllegalArgumentException
at com.amazonaws.internal.config.InternalConfig.loadfrom(InternalConfig.java:260)
at com.amazonaws.internal.config.InternalConfig.load(InternalConfig.java:274)
at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:347)
... 48 more
RequestId: 56b5ae56-95d1-4e67-a7d4-76052816fcb0 Error: Runtime exited with error: exit status 1
Runtime.ExitError
Could you take a look to this? https://clojurians-log.clojureverse.org/graalvm/2022-04-09
I've changed from faraday to the cognitect-aws package and now get :
clojure.lang.ExceptionInfo: Cannot find resource cognitect/aws/dynamodb/service.edn. {}
at cognitect.aws.service$service_description.invokeStatic(service.clj:31)
at cognitect.aws.client.api$client.invokeStatic(api.clj:69)
at bmcgavin.birthday.db.dynamodb$make_ddb.invokeStatic(dynamodb.clj:86)
at bmcgavin.birthday.db.dynamodb$db_get.invokeStatic(dynamodb.clj:89)
at bmcgavin.birthday.db$fn__13291.invokeStatic(db.clj:13)
at bmcgavin.birthday.db$fn__13291.invoke(db.clj:13)
at clojure.lang.MultiFn.invoke(MultiFn.java:229)
at bmcgavin.birthday.api$get_handler.invokeStatic(api.clj:80)
at bmcgavin.birthday.api$get_handler.invoke(api.clj:77)
at compojure.response$fn__612.invokeStatic(response.clj:47)
at compojure.response$fn__612.invoke(response.clj:35)
at compojure.response$fn__562$G__557__569.invoke(response.clj:7)
at compojure.core$wrap_response$fn__2242.invoke(core.clj:158)
at compojure.core$wrap_route_middleware$fn__2226.invoke(core.clj:128)
at compojure.core$wrap_route_info$fn__2231.invoke(core.clj:137)
at compojure.core$wrap_route_matches$fn__2235.invoke(core.clj:146)
at compojure.core$routing$fn__2250.invoke(core.clj:185)
at clojure.core$some.invokeStatic(core.clj:2705)
at compojure.core$routing.invokeStatic(core.clj:182)
at compojure.core$routing.doInvoke(core.clj:182)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invokeStatic(core.clj:669)
at compojure.core$routes$fn__2254.invoke(core.clj:190)
at clojure.lang.Var.invoke(Var.java:384)
at org.httpkit.server.HttpHandler.run(RingHandler.java:121)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.lang.Thread.run(Thread.java:829)
at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:597)
at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:194)
I added an agent call but this didn't change anything :
(agent/in-context
(let [dynamodb (delay (aws/client {:api :dynamodb
:region "eu-west-1"}))
response (aws/invoke dynamodb {:op :ListTables})]
(println response)))
Ah, now I need to regenerate the configs https://clojurians.slack.com/archives/C01UQJ4JC9Y/p1642074240005800
Exactly. You have to regenerate configs ;)
Excellent, thank you. I got a bit further. I managed to get an executable with using a dummy dynamodb query in the agent, but that gave me this error when calling the API :
"Could not locate cognitect/aws/http/cognitect__init.class, cognitect/aws/http/cognitect.clj or cognitect/aws/http/cognitect.cljc on classpath."
So I made a more true-to-life dynamodb query in the agent, and now the config generation shows this error (but completes) :
Exception in agent-context: #error {
:cause No implementation of method: :-invoke of protocol: #'cognitect.aws.client.protocol/Client found for class: nil
:via
[{:type java.lang.IllegalArgumentException
:message No implementation of method: :-invoke of protocol: #'cognitect.aws.client.protocol/Client found for class: nil
:at [clojure.core$_cache_protocol_fn invokeStatic core_deftype.clj 583]}]
:trace
[[clojure.core$_cache_protocol_fn invokeStatic core_deftype.clj 583]
[cognitect.aws.client.protocol$fn__10680$G__10636__10687 invoke protocol.clj 7]
[cognitect.aws.client.api$invoke invokeStatic api.clj 130]
[bmcgavin.birthday.db.dynamodb$fn__13282 invokeStatic dynamodb.clj 112]
[bmcgavin.birthday.db.dynamodb$fn__13282 invoke dynamodb.clj 112]
[bmcgavin.birthday.db.dynamodb__init load nil 112]
[bmcgavin.birthday.db.dynamodb__init <clinit> nil -1]
[java.lang.Class forName0 Class.java -2]
[java.lang.Class forName Class.java 398]
[clojure.lang.RT classForName RT.java 2212]
[clojure.lang.RT classForName RT.java 2221]
[clojure.lang.RT loadClassForName RT.java 2240]
[clojure.lang.RT load RT.java 449]
[clojure.lang.RT load RT.java 424]
[clojure.core$load$fn__6856 invoke core.clj 6115]
[clojure.core$load invokeStatic core.clj 6114]
[clojure.core$load doInvoke core.clj 6098]
[clojure.lang.RestFn invoke RestFn.java 408]
[clojure.core$load_one invokeStatic core.clj 5897]
[clojure.core$load_one invoke core.clj 5892]
[clojure.core$load_lib$fn__6796 invoke core.clj 5937]
[clojure.core$load_lib invokeStatic core.clj 5936]
[clojure.core$load_lib doInvoke core.clj 5917]
[clojure.lang.RestFn applyTo RestFn.java 142]
[clojure.core$apply invokeStatic core.clj 669]
[clojure.core$load_libs invokeStatic core.clj 5974]
[clojure.core$load_libs doInvoke core.clj 5958]
[clojure.lang.RestFn applyTo RestFn.java 137]
[clojure.core$apply invokeStatic core.clj 669]
[clojure.core$require invokeStatic core.clj 5996]
[bmcgavin.birthday.db$loading__6737__auto____13264 invoke db.clj 1]
[bmcgavin.birthday.db__init load nil 1]
[bmcgavin.birthday.db__init <clinit> nil -1]
[java.lang.Class forName0 Class.java -2]
[java.lang.Class forName Class.java 398]
[clojure.lang.RT classForName RT.java 2212]
[clojure.lang.RT classForName RT.java 2221]
[clojure.lang.RT loadClassForName RT.java 2240]
[clojure.lang.RT load RT.java 449]
[clojure.lang.RT load RT.java 424]
[clojure.core$load$fn__6856 invoke core.clj 6115]
[clojure.core$load invokeStatic core.clj 6114]
[clojure.core$load doInvoke core.clj 6098]
[clojure.lang.RestFn invoke RestFn.java 408]
[clojure.core$load_one invokeStatic core.clj 5897]
[clojure.core$load_one invoke core.clj 5892]
[clojure.core$load_lib$fn__6796 invoke core.clj 5937]
[clojure.core$load_lib invokeStatic core.clj 5936]
[clojure.core$load_lib doInvoke core.clj 5917]
[clojure.lang.RestFn applyTo RestFn.java 142]
[clojure.core$apply invokeStatic core.clj 669]
[clojure.core$load_libs invokeStatic core.clj 5974]
[clojure.core$load_libs doInvoke core.clj 5958]
[clojure.lang.RestFn applyTo RestFn.java 137]
[clojure.core$apply invokeStatic core.clj 669]
[clojure.core$require invokeStatic core.clj 5996]
[bmcgavin.birthday.api$loading__6737__auto____3 invoke api.clj 1]
[bmcgavin.birthday.api__init load nil 1]
[bmcgavin.birthday.api__init <clinit> nil -1]
[java.lang.Class forName0 Class.java -2]
[java.lang.Class forName Class.java 398]
[clojure.lang.RT classForName RT.java 2212]
[clojure.lang.RT classForName RT.java 2221]
[clojure.lang.RT loadClassForName RT.java 2240]
[clojure.lang.RT load RT.java 449]
[clojure.lang.RT load RT.java 424]
[clojure.core$load$fn__6856 invoke core.clj 6115]
[clojure.core$load invokeStatic core.clj 6114]
[clojure.core$load doInvoke core.clj 6098]
[clojure.lang.RestFn invoke RestFn.java 408]
[clojure.lang.Var invoke Var.java 384]
[clojure.lang.Util loadWithClass Util.java 251]
[bmcgavin.birthday.api <clinit> nil -1]]}
Hmm, I recommend to ask this question on #CAJN79WNT channel. I didn’t touch graalvm for several months.
Okey, you didn’t reference the client via @client.
Here's the agent call, is that what you mean? It's more complex than it needs to be I guess as it's using the non-agent code, let me simplify it and see if it's still broken.
(def ddb (atom nil))
(defn make-ddb [data]
(reset! ddb (aws/client {:api :dynamodb
:region (get-in data [:aws :aws-region])
:credentials-provider (credentials/basic-credentials-provider
{:access-key-id (get-in data [:aws :aws-access-key-id])
:secret-access-key (get-in data [:aws :aws-secret-access-key])})
:endpoint-override {:protocol (keyword (:db-endpoint-protocol data))
:hostname (:db-endpoint-host data)
:port (Integer/parseInt (:db-endpoint-port data))}})))
;; agent to aid with native image compilation
(agent/in-context
(let [_ (delay (make-ddb {:aws {:aws-region "us-east-1"
:aws-access-key-id "test"
:aws-secret-access-key "test"}
:db-endpoint-protocol "http"
:db-endpoint-host "localhost"
:db-endpoint-port "4566"}))
response (aws/invoke @ddb {:op :ListTables})]
(println response)))
You are returning nil from make ddb.
As far as I can see. I’m on the phone btw.
But you have it in delay.
You didn’t reference the delay yet.
Btw, you are complicating this code more than it’s necessary.
Perfect. Now the agent runs in docker context, so maybe you have to give different connection parameters for it to work.
I would first check in REPL if you can connect to dynamodb and then adjust it for agent ;)
I can, thanks! I'm connecting to localstack in a container so I'll have to make it a shared network. Thanks so much for your help.
> You are returning nil from make ddb. You nailed it with this, thanks. I've now got the agent to connect (by setting :docker :network "host" in bb.edn) and I've corrected my code to not call reset! twice. Now I get this :
localstack | [holy-lambda] Runtime error:
localstack | {:via [{:type java.io.FileNotFoundException, :message "Could not locate cognitect/aws/http/cognitect__init.class, cognitect/aws/http/cognitect.clj or cognitect/aws/http/cognitect.cljc on classpath.", :at [clojure.lang.RT load "RT.java" 462]}], :trace [[clojure.lang.RT load "RT.java" 462] [clojure.lang.RT load "RT.java" 424] [clojure.core$load$fn__6856 invoke "core.clj" 6115] [clojure.core$load invokeStatic "core.clj" 6114] [clojure.core$load doInvoke "core.clj" 6098] [clojure.lang.RestFn invoke "RestFn.java" 408] [clojure.core$load_one invokeStatic "core.clj" 5897] [clojure.core$load_one invoke "core.clj" 5892] [clojure.core$load_lib$fn__6796 invoke "core.clj" 5937] [clojure.core$load_lib invokeStatic "core.clj" 5936] [clojure.core$load_lib doInvoke "core.clj" 5917] [clojure.lang.RestFn applyTo "RestFn.java" 142] [clojure.core$apply invokeStatic "core.clj" 669] [clojure.core$load_libs invokeStatic "core.clj" 5974] [clojure.core$load_libs doInvoke "core.clj" 5958] [clojure.lang.RestFn applyTo "RestFn.java" 137] [clojure.core$apply invokeStatic "core.clj" 669] [clojure.core$require invokeStatic "core.clj" 5996] [cognitect.aws.dynaload$load_ns invokeStatic "dynaload.clj" 10] [cognitect.aws.dynaload$load_var invokeStatic "dynaload.clj" 17] [cognitect.aws.http$resolve_http_client invokeStatic "http.clj" 83] [cognitect.aws.client.shared$fn__12533 invokeStatic "shared.clj" 14] [cognitect.aws.client.shared$fn__12533 invoke "shared.clj" 14] [clojure.lang.Delay deref "Delay.java" 42] [clojure.core$deref invokeStatic "core.clj" 2324] [cognitect.aws.client.shared$http_client invokeStatic "shared.clj" 22] [cognitect.aws.client.api$client invokeStatic "api.clj" 74] [bmcgavin.birthday.db.dynamodb$make_ddb invokeStatic "dynamodb.clj" 16] [bmcgavin.birthday.db.dynamodb$db_get invokeStatic "dynamodb.clj" 19] [bmcgavin.birthday.db$fn__13291 invokeStatic "db.clj" 13] [bmcgavin.birthday.db$fn__13291 invoke "db.clj" 13] [clojure.lang.MultiFn invoke "MultiFn.java" 229] [bmcgavin.birthday.api$get_handler invokeStatic "api.clj" 84] [bmcgavin.birthday.api$get_handler invoke "api.clj" 81] [compojure.response$fn__612 invokeStatic "response.clj" 47] [compojure.response$fn__612 invoke "response.clj" 35] [compojure.response$fn__562$G__557__569 invoke "response.clj" 7] [compojure.core$wrap_response$fn__2242 invoke "core.clj" 158] [compojure.core$wrap_route_middleware$fn__2226 invoke "core.clj" 128] [compojure.core$wrap_route_info$fn__2231 invoke "core.clj" 137] [compojure.core$wrap_route_matches$fn__2235 invoke "core.clj" 146] [compojure.core$routing$fn__2250 invoke "core.clj" 185] [clojure.core$some invokeStatic "core.clj" 2705] [compojure.core$routing invokeStatic "core.clj" 182] [compojure.core$routing doInvoke "core.clj" 182] [clojure.lang.RestFn applyTo "RestFn.java" 139] [clojure.core$apply invokeStatic "core.clj" 669] [compojure.core$routes$fn__2254 invoke "core.clj" 190] [fierycod.holy_lambda_ring_adapter.core$ring_LT___GT_hl_middleware$fn__2903 invoke "core.cljc" 150] [clojure.lang.Var invoke "Var.java" 384] [fierycod.holy_lambda.custom_runtime$next_iter invokeStatic "custom_runtime.clj" 80] [fierycod.holy_lambda.custom_runtime$next_iter invoke "custom_runtime.clj" 60] [clojure.lang.Var invoke "Var.java" 399] [bmcgavin.birthday.api$_main invokeStatic "api.clj" 107] [bmcgavin.birthday.api$_main doInvoke "api.clj" 107] [clojure.lang.RestFn invoke "RestFn.java" 397] [clojure.lang.AFn applyToHelper "AFn.java" 152] [clojure.lang.RestFn applyTo "RestFn.java" 132] [bmcgavin.birthday.api main nil -1]], :cause "Could not locate cognitect/aws/http/cognitect__init.class, cognitect/aws/http/cognitect.clj or cognitect/aws/http/cognitect.cljc on classpath."}
I've just seen this from https://github.com/clj-easy/graalvm-clojure/tree/master/aws-api-s3 :
> Finally, you need to pre-create a http-client
for the same reason and pass it to all your aws clients. For this you can use Clojure's delay
, to ensure runtime initialization.
Yes, it seems you also need to pre-create http client as well.