Fork me on GitHub
#aws
<
2019-10-24
>
mruzekw17:10:39

For those who use AWS, which of these is the preferable lib to use? https://github.com/cognitect-labs/aws-api https://github.com/mcohen01/amazonica

Alex Miller (Clojure team)17:10:51

they are very different approaches

Alex Miller (Clojure team)17:10:28

amazonica is a clojure wrapper for the Java SDK

Alex Miller (Clojure team)17:10:14

aws-api is a data-oriented and data-driven Clojure API directly to the AWS http api

mruzekw17:10:05

Would it be possible to adapt aws-api for cljs?

Alex Miller (Clojure team)17:10:22

that has been discussed

Alex Miller (Clojure team)17:10:35

I don't know the status of it

mruzekw17:10:38

My use case would be using CLJS for AWS lambda

mruzekw17:10:05

Nice, thanks

mruzekw17:10:38

Would it just take converting clj files to cljc?

mruzekw17:10:42

“just”

Alex Miller (Clojure team)17:10:19

you would need a cljs http library instead (maybe other things too, I don't know)

mruzekw17:10:28

Makes sense

Alex Miller (Clojure team)17:10:12

conceptually there is no reason that can't work afaik

ghadi17:10:50

we are making moves towards being able to being able to support cljs but cljs isn’t in the short term milestones

mruzekw17:10:25

Is there an existing cljs solution for the AWS api?

Alex Miller (Clojure team)17:10:36

I don't think amazonica does that for cljs

mruzekw17:10:48

Maybe interop with aws-sdk-js?

Alex Miller (Clojure team)17:10:04

seems like that would be the obvious thing to do

ghadi17:10:09

Apart from http there are also signing routines that need porting to cljs

mruzekw17:10:33

Is that done on a clj reader level or a namespace level?

ghadi17:10:59

Signing routines are not terribly complicated, but we’re trying to move forward everything compatibly

ghadi17:10:41

And that can be tricky

ghadi17:10:26

@mruzekw what makes you interested in cljs over clj?

mruzekw17:10:47

I plan on using AWS Lambda

mruzekw17:10:01

Though I’m not opposed to using CLJ with GraalVM

ghadi17:10:32

GraalVM is not possible with these dependencies AFAIK

ghadi17:10:41

Native-image, that is

ghadi17:10:48

What is your SLA for startup time?

mruzekw17:10:06

I mean, I’d like to get cold start to <= 500ms and warm <=200ms

ghadi17:10:23

What sorts of dependencies are you interested in running within lambda? Sorry for the 20 questions, doing research :)

mruzekw17:10:09

haha no worries

mruzekw17:10:03

I’m in the preliminary infra planning stages so this may change and may be incomplete, but I’m mainly looking at using the RDSDataServer API to access an Aurora Serverless SQL cluster

mruzekw17:10:46

This is just for now. I could see a use for a queue component

jsyrjala17:10:54

One thing to consider is if you need to run lambdas in VPC, which would increase the cold start time

mruzekw17:10:33

Hmm good to know, VPC restricts public access?

jsyrjala17:10:40

It allows lambda to talk to services that are inside a VPC that are not accessible from outside

mruzekw17:10:07

Okay, will have to think that through

mruzekw17:10:33

Do you know how much ms on average accessing through a VPC is?

jsyrjala17:10:54

AWS launched optimizations for cold starts for lambdas in VPC few weeks ago, so I don’t know the current situation. It used to be in order of few seconds for cold starts and no difference in warm starts, if I remember correctly.

mruzekw17:10:40

Wow a few seconds is significant

mruzekw17:10:12

933 ms is better but not great

mruzekw17:10:36

But I guess that’s just on cold

mruzekw17:10:40

Is there a good doc ref for the AWS HTTP level API?

mruzekw17:10:45

All I can find are the SDK docs

Joe Lane17:10:06

That is what aws-api solves for you.

Joe Lane17:10:51

basically there is a json based datastructure for all the api's but I'm not aware of a complete doc reference.

mruzekw17:10:13

Hmm alright

mruzekw17:10:20

> Each AWS SDK has its own copy of the data descriptions in their github repos. We use aws-sdk-js as the source for these, and release individual artifacts for each api.

Joe Lane17:10:35

I prefer the aws-api because then i'm not stuck using builders like the java sdk

mruzekw17:10:22

query.firstParam().secondParam().build() type API?

Joe Lane17:10:51

One word, RequestBuilder.

mruzekw18:10:03

Does aws-api take outside PRs?

Joe Lane18:10:13

No, but they take issues

mruzekw18:10:58

I may just use aws/aws-sdk-js in the mean time

ghadi18:10:54

Editorial: I’m all for faster startup times, but cold start is talked about way too much without surrounding use-cases

ghadi18:10:52

(That’s why I’m probing for details)

mruzekw18:10:58

Well I’m hoping to have lambdas that serve API responses

mruzekw18:10:04

Servicing a frontend SPA

mruzekw18:10:14

I’d like the user operations to be quick

mruzekw18:10:19

Thus below 1s

Alex Miller (Clojure team)18:10:33

if people are using it, then things aren't cold right?

mruzekw18:10:53

Right, within a timeout

mruzekw18:10:57

If people don’t use it for say 10 mins it destructs

Joe Lane18:10:57

Set a cloudwatch event to ping your stuff every n minutes. Then it's never cold.

mruzekw18:10:08

Sure, not a bad idea

mruzekw18:10:24

It doesn’t cost to keep a fn warm?

jsyrjala18:10:48

That keeps one lambda instance warm, you’ll get cold starts if you happen to need more than one simultaneously.

Joe Lane18:10:03

you can invoke a lambda to invoke your lambdas

mruzekw18:10:13

Dynamically?

mruzekw18:10:24

Like it knows which are warm already?

Joe Lane18:10:39

This coldstart thing is almost never an issue unless you have a thundering herd scenario.

Joe Lane18:10:22

If you make a heartbeat lambda which heartbeats all your other possible lambdas then this is solved, right?

mruzekw18:10:38

Where does the heartbeat originate?

Joe Lane18:10:47

cloudwatch event rule

mruzekw18:10:49

I think there’s a timelimit on a running lamnda

mruzekw18:10:56

Last heading: > What Is The Effect Of VPC Access?

mruzekw18:10:51

Looks like VPC only slows a cold start by about 0.1s

jsyrjala18:10:44

That is a new feature. It used to be a lot worse.

mruzekw18:10:06

I am grateful

mruzekw18:10:42

That’s on my list to watch

mruzekw18:10:50

Do you have a tldr/w?

jsyrjala18:10:59

start times depend on lot of things, like jar size. cold starts are faster in clojurescript, and warm starts are faster in clojure. Clojure(script) has some extra start time cost on top of nodejs/jvm.

mruzekw18:10:26

Sure, matches with my research so far

mruzekw18:10:56

Thanks for all your input all 🙏 🙂

jsyrjala18:10:34

also low memory lambdas have more varying performance than higher memory (more expensive) lambdas. That is you get some minimum amount of cpu but might get more if you are lucky.

Alex Miller (Clojure team)18:10:32

if cold starts are your problem, then isn't the real problem no one is using your stuff?

😂 4
ghadi18:10:46

given settings also affect how fast the code slugs are pulled down

ghadi18:10:58

and larger code slugs => slower to pull

ghadi18:10:08

that's partly what Lambda Layers help with

ghadi18:10:24

coldstart is but one aspect

ghadi18:10:32

of a much larger picture

jsyrjala18:10:09

I’ve made this leiningen template to help creating lambda projects for clojure: https://github.com/jsyrjala/aws-lambda-serverless/

mruzekw20:10:22

@jsyrjala Do you know of any other Clojure on Lambda resources?

jsyrjala20:10:18

Can’t remember anything else that I have used.

jsyrjala20:10:01

Mostly lambdas I have done are backend stuff where cold starts do not matter at all.

kenny20:10:36

@mruzekw I was pointed to this repo recently when looking for clj on AWS Lambda resources: https://github.com/StediInc/lambda. Looks useful, haven't used it.

mruzekw20:10:00

I’m trying to follow this tutorial, but they don’t provide how to correctly invoke the lambda from the AWS CLI

mruzekw20:10:27

I’m using:

aws lambda invoke --function-name clj-hello --payload '{"body": "Will"}' /dev/stdout

mruzekw20:10:00

But I get back:

{"errorMessage":"An error occurred during JSON parsing","errorType":"java.lang.RuntimeException","stackTrace":[],"cause":{"errorMessage":"com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token\n at [Source: lambdainternal.util.NativeMemoryAsInputStream@4516af24; line: 1, column: 1]"

mruzekw20:10:23

Does anyone know the expected shape of the JSON payload given:

(ns hello
  (:gen-class
   :methods [^:static [handler [String] String]]))

(defn -handler [s]
  (str "Hello " s "!"))

jsyrjala21:10:34

try returning JSON formatted string from the -handler

mruzekw21:10:04

(defn -handler [s] s)?

jsyrjala21:10:29

for example

mruzekw21:10:47

Same error

mruzekw21:10:25

I technically uploaded:

(defn -handler [s]
  (str "Hello " s "!")
  s)

jsyrjala21:10:26

I am wondering why it is complaining about JSON at all.

kenny21:10:37

I've hit that before. It has to do with the signature of your Java lambda fn. I remember having to switch to the input/output stream signature. Googling that exception gives you some answers IIRC.

mruzekw21:10:11

Huh, I thought AWS could infer what to parse given the handler signature

mruzekw21:10:14

(ns hello
  (:gen-class
   :methods [^:static [handler [String] String]]))

kenny21:10:58

The handler is expecting a pojo.

jsyrjala21:10:26

I have done handlers always like this.

mruzekw21:10:45

I’m not going through the API Gateway

mruzekw21:10:47

➜ aws lambda invoke --function-name clj-helloooo --payload 'Will' /dev/stdout

An error occurred (InvalidRequestContentException) when calling the Invoke operation: Could not parse request body into json: Unrecognized token 'Will': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"Will"; line: 1, column: 9]

kenny21:10:33

I do the same as @jsyrjala. More general that way too.

mruzekw21:10:44

Okay, I’ll try that

mruzekw21:10:54

Thanks y’all

jsyrjala21:10:28

And you can use also non-json messages like that.

mruzekw21:10:01

Quick question/poll: Which would you all choose for a lambda with an avg response time <300ms, CLJ (with or without heartbeat warmer) or CLJS?

ghadi21:10:31

don't focus on time -- it all depends on what it's doing. You will find a richer ecosystem of libraries on the JVM imho, and the warm speeds will be faster

👍 4
mruzekw21:10:38

Would a Lumo runtime have a faster startup time than a CLJS fn compiled to JS?

ghadi21:10:24

.......

😂 4
mruzekw21:10:37

I’m just curious

ghadi21:10:40

focusing on startup time is misguided

ghadi21:10:56

but super popular 😉