This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-05-04
Channels
- # announcements (4)
- # aws (3)
- # babashka (58)
- # beginners (59)
- # biff (6)
- # cider (3)
- # clj-kondo (48)
- # clj-on-windows (1)
- # cljdoc (1)
- # clojure (136)
- # clojure-europe (19)
- # clojure-gamedev (7)
- # clojure-germany (2)
- # clojure-nl (7)
- # clojure-norway (1)
- # clojure-portugal (1)
- # clojure-uk (4)
- # clojurescript (41)
- # community-development (2)
- # core-async (5)
- # cursive (10)
- # data-oriented-programming (1)
- # data-science (1)
- # datahike (5)
- # datomic (60)
- # docker (2)
- # emacs (13)
- # figwheel-main (19)
- # fulcro (12)
- # graalvm (9)
- # holy-lambda (41)
- # honeysql (14)
- # introduce-yourself (3)
- # jobs (4)
- # lsp (11)
- # nrepl (1)
- # off-topic (9)
- # other-languages (2)
- # pathom (22)
- # portal (5)
- # re-frame (17)
- # remote-jobs (4)
- # reveal (14)
- # shadow-cljs (1)
- # tools-build (7)
- # tools-deps (47)
- # xtdb (8)
- # yada (2)
Heya, I want to include the chromedriver binary inside my docker image for one of my HL functions, however trying to install it I see that yum
is not available. I guess it's not included with openjdk:latest
? How can I install packages inside my HL dockerfile?
OpenJDK latest is based on Debian. See https://hub.docker.com/_/openjdk
weird, apt
wasn't working either
How does HL/lambda read the PATH at runtime? I have now have the binary at /usr/bin/chromedriver
in the docker image and I can confirm that at runtime (using sam local invoke) by printing the file with
. But for some reason etaoin
can't find the executable at that path. I also confirmed here https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html#configuration-envvars-runtime that /usr/bin
is on the PATH
by default.
This has nothing to do with HL.
HL is a custom runtime with some additional tasks to speed up the development.
Let me check etaoin source code.
Maybe before this lets start from the very begining. Could you please do which chromedriver
before running the actual jar, so that we can confirm the executable is findable in a shell?
You should be able to set a path to a binary via set-binary function. See here: https://github.com/igrishaev/etaoin/blob/master/src/etaoin/driver.clj
Thanks so much for helping with this even though it's not a HL problem!
Step 4/5 : RUN which chromedriver
---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
---> Running in e6952502ba83
/usr/bin/chromedriver
So that's not the issueAs I understand that multimethod gets called by the path flag e.g. (def driver (chrome {:path "/usr/bin/chromedriver"}))
which I tried, https://github.com/igrishaev/etaoin#additional-parameters
let me try calling it directly
(ed/set-binary (chrome-headless) "/usr/bin/chromedriver")
Does not workWhat is a little bit suprising for me is that you are on arm64 and you’re using amd64 image.
Maybe an issue is not exactly there but I found it surprising. Could you please provide a minimal repro for this issue? I'm happy to help with this.
I was able to reproduce it with clojure -Tclj-new create :template holy-lambda :name company/example-lambda :output holy-lambda-example
nick@Nicks-MacBook-Pro-2 holy-lambda-example % sam build
Your template contains a resource with logical ID "ServerlessHttpApi", which is a reserved logical ID in AWS SAM. It could result in unexpected behaviors and is not recommended.
Building codeuri: /Users/nick/code/holy-lambda-example runtime: None metadata: {'Dockerfile': 'Dockerfile', 'DockerContext': '/Users/nick/code/holy-lambda-example', 'DockerTag': 'v1'} architecture: x86_64 functions: ['ExampleLambdaFunction']
Building image for ExampleLambdaFunction function
Setting DockerBuildArgs: {} for ExampleLambdaFunction function
Step 1/4 : FROM openjdk:latest
---> 15bbbf63e09d
Step 2/4 : MAINTAINER Karol Wójcik <[email protected]>
---> Using cache
---> e1984a62bbbc
Step 3/4 : ADD .holy-lambda/build/output.jar output.jar
---> 149038560520
Step 4/4 : CMD java -jar output.jar "company.example-lambda.core.ExampleLambda"
---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
---> Running in d99c5540dc58
---> 4379543223d4
Successfully built 4379543223d4
Successfully tagged examplelambdafunction:v1
By the way, etaoin was throwing a binary not found error but when I dug deeper it was finding the binary, just not able to call it because of the older jdk I was using, https://bugs.openjdk.java.net/browse/JDK-8212828, when I upgraded the jdk it stopped being an issue
What other info about my system would be relevant for helping you debug the Warning?
I don’t need any more info.
It all depends on which architecture you do want to target.
If you’re targeting Intel and you’re on Mac M1, then you will experience this warning.
It's because docker is emulated via Rosetta.
Ah, well that's what I'm doing. Yeah what I'm targeting doesn't make much difference to me, I don't have any pressing perf requirements
Hey different question, I now have 4 different lambdas in one repo that share code but also have different dependencies. Because it's all being built together, my docker image is now 1.2GB, which has to be uploaded 4 times, even if I only want to make a change to one lambda. What are my options for separating them out?
They're currently all in the same stack but I don't think that's a requirement for me
My motivation for separating them is mainly the slow deploy times, especially when I have poor upload speeds 😉
Cannot you build the docker image one time with all the dependencies and point to different lambdas via custom CMD?
Have you also considered trimming you docker images? Maybe by using Alpine images?
Hmm, interesting, I could try that. 3 of them are using the same base image but the 4th is has a different base image, but I could potentially narrow it down to 2 Dockerfiles with multiple CMD targeting different entrypoints?
Exactly!
Or maybe even one if it's possible.
Cool, I will give that a shot 🙂 And yes, I will try using trimmed images, and maybe also see if I have any unnecessary deps at this point
As a test, I removed one of the functions with a large image from the template.yml
. Rebuilt fine, yet sam deploy
still tries to push the image (there should only be 3 layers)
nick@Nicks-MacBook-Pro-2 amc-utility % sam deploy
42f1d7875f37: Pushing [> ] 6.652MB/1.57GB
dc1b24232ab1: Layer already exists
f1ddd4093f07: Layer already exists
e57f007acf74: Layer already exists
Could it be stuck in a cache somewhere? I am not passing any cache flags.Also, when the deploy succeeds, it deletes the function from the stack. But on subsequent deploys, it still tries to push the image for the deleted function, but "underneath" each other function!
I'm not an AWS SAM expert.
Ok, sorry to bug you about this, maybe I should post in a SAM forum.
But I would say you probably have something like samconfig.yaml or packaged.yaml that causes this.