This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-21
Channels
- # announcements (1)
- # aws-lambda (62)
- # babashka (116)
- # beginners (67)
- # chlorine-clover (39)
- # cider (10)
- # cljs-dev (5)
- # clojure (30)
- # clojure-austin (2)
- # clojure-europe (2)
- # clojure-italy (6)
- # clojure-nl (24)
- # clojure-uk (28)
- # clojurescript (33)
- # data-science (6)
- # datascript (10)
- # datomic (5)
- # duct (39)
- # emacs (1)
- # events (8)
- # fulcro (9)
- # graalvm (29)
- # hoplon (7)
- # juxt (10)
- # malli (4)
- # off-topic (6)
- # pathom (10)
- # perun (1)
- # reagent (45)
- # shadow-cljs (5)
- # sql (14)
- # tools-deps (10)
- # xtdb (9)
Don't know, ask Dainius I guess 🙂 @U0FT43GKV
I’ll give it a spin once I’m back on computer! Startup-time is usually not that critical to me but I’m just curious.
he did say to me that there was one issue in bb that prevented this from working, he's going to do a PR for that
I just tried and was able to deploy it to lambda. Had to make small modofikations to makefile
on MacOS because it wasn’t happy with /tmp/...
.
Hi, yeah, the cold start is 400ms, subsequent calls are 30ms. Worth mentioning is that the cold start and the duration depends on how much RAM is dedicated to the lambda, the more the faster lambda is.
Yep I saw variance between 4ms and 120ms on hot calls with 128Mb RAM. Which I think is quite good.
@U0FT43GKV About your PR, what happens without it on AWS? Do you get an error? https://github.com/borkdude/babashka/pull/305/files#diff-b4e1e6a8c0b365c3b9fac4c30360f72d
Without the change Lambda fails in a very similar way as is described here https://github.com/oracle/graal/issues/841
Also, I've found very similar issue https://github.com/quarkusio/quarkus/issues/4262. Here it is said that
You can work around the problem by defining the environment variable DISABLE_SIGNAL_HANDLERS to any value. Obviously this means that you won't have OS signal handling though.
Maybe the environment variable for babashka should be the same?@U0FT43GKV That seems to be project specific: https://github.com/quarkusio/quarkus/blob/bb1ef1ffd952d5f3a1d75e599ef5acf107e6de55/core/runtime/src/main/java/io/quarkus/runtime/Application.java#L29
It is. I'm just not sure on how to name the environment variable. But my overall solution is just a quick and dirty hack to make the thing work. Probably more sophisticated design is needed to support Lambda use case.
So far babashka specific env variables starts with BABASHKA_
, like BABASHKA_CLASSPATH
.
What about BABASHKA_DISABLE_PIPE_HANDLER
?
ok, I'll rename the env variable in the PR
merged. I assume you don't need a new build? btw, @U6N4HSMFW - for you it did work right, without this patch?
A new build would be nice because in the lambda archive I'm packaging the bb binary. For the example lambda I've deployed a babashka docker to Docker Hub https://hub.docker.com/r/dainiusjocas/babashka and then used it to build lambda archive. So, if babashka would provide a build, I'd adjust the example. Propably, later I'd convert it into a template project.
There are pre-release builds in #babashka_circleci_builds - I'll hope to release a new one soon.
Perfect! No worries, I'm in no rush 🙂
@U04V15CAJ correct, it worked without the patch
@U6N4HSMFW Hmm, so maybe the patch isn't needed then - or is there something different in your env @U0FT43GKV
The patch is needed, I've packaged the patched version of bb to a docker, and then used the patched bb for building the lambda in the docker.
FROM dainiusjocas/babashka:latest as BABASHKA
FROM clojure:tools-deps-alpine as BUILDER
RUN apk add --no-cache zip
WORKDIR /var/task
COPY --from=BABASHKA /usr/local/bin/bb bb
ENV GITLIBS=".gitlibs/"
COPY lambda/bootstrap bootstrap
COPY deps.edn deps.edn
RUN clojure -Sdeps '{:mvn/local-repo "./.m2/repository"}' -Spath > cp
COPY src/ src/
COPY resources/ resources/
RUN zip -q -r function.zip bb cp bootstrap .gitlibs/ .m2/ src/ resources/ deps.edn
See the first line here.@U0FT43GKV I wonder if a simple try/catch would have also sufficed as a workaround to this
also, this commit ought to have fixed the problem: https://github.com/dmlloyd/graal/commit/dedcc86471b0209eeebf1df816b5f2590dc3361d which version of graalvm did you use to compile your docker image?
I haven't tried try/catch. Here is the error message from logs https://gist.github.com/dainiusjocas/feafeef5653ff2c6e8c7b2d9627a831d
I've used the same version as is specified in the Dockerfile
Looking at that log, I suspect try/catch won't do it, but you could give it a try if it's not too much to ask. I did release v0.0.78 now and renamed the variable to BABASHKA_DISABLE_PIPE_SIGNAL_HANDLER
Tomorrow I'll try the try/catch approach. I also suspect that it will not work because lambda runtime panics when lambda tries to do anything with signals. But I'll try
I think it might also be good to post an issue about this to oracle/graal, since it seems they intended to fix the issue
commented here now: https://github.com/oracle/graal/issues/841#issuecomment-602181005
I've tried with try/catch. The same failure.
It turns out that lambda runtime has curl. So no external dependencies needed. Then the issue is to parse curl response with headers. @U04V15CAJ do you plan to support parsing curl response into {:headers {} :body "" :ect :etc}
in babashka.curl?
@U0FT43GKV Cool. I think it's good to have support for this, but it's not currently implemented.
Another issue with using curl is that it is much much slower, than clj-http-lite. The echo took 200ms consistently to execute while clj-http-lite take 15ms. Here is my naive implementation of curl response parsing https://gist.github.com/dainiusjocas/d2a5ca60ae25cd260f37b7c125a0b2e6
I made an issue for parsing the headers here: https://github.com/borkdude/babashka.curl/issues/5
I think this should be optional, so maybe when passed an option like :response true
will return the full response map and when it's not set, it will return only the body?
no idea why it could be that much slower.
IMO, the current functionality should not change, so this response parsing should under some new param like :response true
. Note that response headers are retrieved with :raw-args ["-i"]
FWIW I see similar times between babashka.curl and slurp
user=> (time (do (curl/get "") nil))
"Elapsed time: 75.406528 msecs"
nil
user=> (time (do (slurp "") nil))
@U0FT43GKV Could it be that the slowness with curl was with repeated execution of requests? Maybe clj-http-lite uses connection pooling?
I doubt that the slowness is because connection pooling. As it is written in the README No persistent connection support
I have a suspicion that it might be because doing shell out on a very constrained environment, just 128MB RAM and very little CPU. I would like to test with more RAM given to the lambda and some profiling.
I have an idea to package babashka as a Lambda layer, and make a proper template that uses handler option, that would be a script file. In this way one could write a lambda function right in the cloud IDE or the packaging script could produce --uberscript
for compactness. @U04V15CAJ what do you think if I publish babashka as a public lambda layer?
@U0FT43GKV Btw, if you want to develop babashka.curl you can load it with bb from the classpath with :reload
, instead of the built-in one.