Fork me on GitHub
#beginners
<
2019-06-09
>
johnjelinek02:06:42

I'm having some strange behavior running my clojure package as an aws lambda function

johnjelinek02:06:08

here's the package contents:

# unzip -l lambda.zip | grep clj/

        0  06-08-2019 22:31   clj/
      436  06-08-2019 22:30   clj/core.clj
     2300  06-08-2019 22:30   clj/core$loading__6706__auto____126.class
     1313  06-08-2019 22:30   clj/core$fn__267.class
     1376  06-08-2019 22:30   clj/core$handle_event.class
        0  06-08-2019 22:31   clj/core/
     2133  06-08-2019 22:30   clj/core/Handler.class
     1753  06-08-2019 22:30   clj/core$G__270handleRequest.class
     2986  06-08-2019 22:30   clj/core__init.class

johnjelinek02:06:21

(everything else is a dep)

johnjelinek02:06:17

the first time I run the package, it works:

# sam local invoke HelloWorldFunction --event sam-app/event.json 
2019-06-09 02:31:52 Invoking clj.core.Handler (java8)
2019-06-09 02:31:53 Decompressing /root/code/clj/lambda.zip

Fetching lambci/lambda:java8 Docker container image......
2019-06-09 02:31:54 Mounting /tmp/tmpg7j08zw9 as /var/task:ro,delegated inside runtime container
Got the following event:  {"resource" "/{proxy+}", "body" "{\"message\": \"hello world\"}", "requestContext" {"resourcePath" "/{proxy+}", "protocol" "HTTP/1.1", "requestTimeEpoch" 1428582896000, "accountId" "123456789012", "resourceId" "123456", "path" "/prod/path/to/resource", "httpMethod" "POST", "requestTime" "09/Apr/2015:12:34:56 +0000", "requestId" "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", "apiId" "1234567890", "identity" {"userAgent" "Custom User Agent String", "accountId" nil, "userArn" nil, "cognitoAuthenticationProvider" nil, "cognitoAuthenticationType" nil, "cognitoIdentityId" nil, "user" nil, "sourceIp" "127.0.0.1", "accessKey" nil, "cognitoIdentityPoolId" nil, "caller" nil}, "stage" "prod"}, "path" "/path/to/resource", "httpMethod" "POST", "pathParameters" {"proxy" "/path/to/resource"}, "queryStringParameters" {"foo" "bar"}, "stageVariables" {"baz" "qux"}, "isBase64Encoded" false, "headers" {"Upgrade-Insecure-Requests" "1", "Via" "1.1  (CloudFront)", "CloudFront-Is-Tablet-Viewer" "false", "User-Agent" "Custom User Agent String", "X-Amz-Cf-Id" "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==", "Accept-Encoding" "gzip, deflate, sdch", "Cache-Control" "max-age=0", "Accept-Language" "en-US,en;q=0.8", "X-Forwarded-Port" "443", "CloudFront-Forwarded-Proto" "https", "CloudFront-Is-Mobile-Viewer" "false", "X-Forwarded-For" "127.0.0.1, 127.0.0.2", "X-Forwarded-Proto" "https", "CloudFront-Is-SmartTV-Viewer" "false", "Accept" "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", "CloudFront-Is-Desktop-Viewer" "true", "Host" "", "CloudFront-Viewer-Country" "US"}}
START RequestId: 10216fa0-76a8-4dc1-9e87-bdd420d68bd3 Version: $LATEST
END RequestId: 10216fa0-76a8-4dc1-9e87-bdd420d68bd3
REPORT RequestId: 10216fa0-76a8-4dc1-9e87-bdd420d68bd3  Duration: 134.41 ms     Billed Duration: 200 ms Memory Size: 128 MB     Max Memory Used: 8 MB

{"status":"ok"}

johnjelinek02:06:39

subsequent runs, it fails:

# sam local invoke HelloWorldFunction --event sam-app/event.json 
2019-06-09 02:34:48 Invoking clj.core.Handler (java8)
2019-06-09 02:34:49 Decompressing /root/code/clj/lambda.zip

Fetching lambci/lambda:java8 Docker container image......
2019-06-09 02:34:50 Mounting /tmp/tmpfldgw053 as /var/task:ro,delegated inside runtime container
START RequestId: e697f54a-3f19-41d4-9910-9a52520db07f Version: $LATEST
java.lang.UnsupportedOperationException: handleRequest (clj.core/G__270handleRequest not defined?)
        at clj.core.Handler.handleRequest(Unknown Source)

END RequestId: e697f54a-3f19-41d4-9910-9a52520db07f
REPORT RequestId: e697f54a-3f19-41d4-9910-9a52520db07f  Duration: 630.95 ms     Billed Duration: 700 ms Memory Size: 128 MB     Max Memory Used: 9 MB

{"errorMessage":"handleRequest (clj.core/G__270handleRequest not defined?)","errorType":"java.lang.UnsupportedOperationException","stackTrace":["clj.core.Handler.handleRequest(Unknown Source)"]}

johnjelinek02:06:55

the zip is immutable -- so, why would it fail to execute for subsequent runs?

johnjelinek02:06:16

error:

{"errorMessage":"handleRequest (clj.core/G__270handleRequest not defined?)","errorType":"java.lang.UnsupportedOperationException","stackTrace":["clj.core.Handler.handleRequest(Unknown Source)"]}
but I see it in the package:
# unzip -l lambda.zip | grep handleRequest
     1753  06-08-2019 22:37   clj/core$G__270handleRequest.class
what gives?

johnjelinek02:06:09

I'm not sure why it sometimes works though

johnjelinek02:06:20

I guess it's some shenanigans with lambada -- I'm getting consistent behavior when I strip it way down:

(ns clj.core
  (:gen-class
   :methods [^:static [handler [] String]]))

(defn -handler []
  (str "Hello World!"))

johnjelinek02:06:16

is there an easy way to string aliases together? I have the following aliases: :aot and :pack, and I want them to always execute together, but they have different :extra-deps and what-not

johnjelinek19:06:45

I tried clojure -A:aot:pack mach.pack.alpha.aws-lambda -C:aot lambda.zip, but that didn't yield the results I was looking for

seancorfield19:06:16

No idea bout that library, sorry.

phrsantos05:06:42

What is the proper way of converting Pedestal query-params into numbers, instead of strings? Is there an interceptor for that?

Ahmed Hassan06:06:58

Using Transit may be?

Tiv0w10:06:34

Hey everyone ! Absolute noob here, I was wondering what are the options for data science/machine learning in Clojure ? I've done some quick research and found mostly the name of Incanter, but everyone seems to say that it is an abandonware. Any libraries/frameworks for a beginner ? (I'm also open to a combination, kinda like the Python ecosystem: NumPy + Pandas + Matplotlib/Seaborn + TensorFlow/Keras/PyTorch)

Mno13:06:03

Don’t know any myself but I came upon a guy that blogged about it a lot, I think he’s on the slack. Anyway here’s the link to his blog about deep learning with clojure (https://dragan.rocks/)

Nate Sutton15:06:22

is there a (get map key default) where the default isn't executed until it fails to fetch the key from the map?

Nate Sutton15:06:35

maybe a (get map key fn)?

valtteri15:06:23

How about (or (get m k) :otherwise)

Nate Sutton15:06:08

that's what I'm using right now, but if nil is a valid value it's not exactly the same semantics

Nate Sutton15:06:14

looking for semantics more like (if (contains? map key) (map key) (my-fn))

codonnell15:06:01

(if-let [[_ v] (find m k)] v (my-fn))

valtteri15:06:58

Ahh ok now I think I understand. Could it make sense in your case to drop all nil values before checking?

valtteri15:06:42

I usually try to avoid nil values in maps since they’re actually meaningless.

Nate Sutton15:06:53

(defn fetch [map key default-fn] (if (contains? map key) (map key) (default-fn)))

Nate Sutton15:06:08

user=> (fetch {:a 1} :b (fn [] 5))
5
user=> (fetch {:a 1} :a (fn [] 5))
1

Nate Sutton15:06:03

the nil has meaning in this case, so it wouldn't make sense to remove them. the meaning specifically is that :a is present, not that nil is

Nate Sutton15:06:23

user=> (fetch {:a nil} :a (fn [] 5))
nil

Nate Sutton15:06:02

it's definitely a design tradeoff, though. but it allows you to also do thing like the (or (get m k) :otherwise) where it's useful

Nate Sutton15:06:16

without checking for the sentinel

Nate Sutton15:06:04

is that the right word for it? sentinel?

valtteri15:06:03

Yep, I see the point. 🙂 Personally I try to encode the ‘meaning’ into the value and not into presence of some key (regardless if there’s value or not). But there might be some cases where that’s handy.

Nate Sutton15:06:25

the meaning is in the presence of the key

Nate Sutton15:06:48

there are two groups of information represented in a map, the set of keys and values associated with those keys

codonnell15:06:54

@nate_clojurians You can avoid doing the map lookup twice by using find like (if-let [[_ v] (find m k)] v (my-fn)). Hard to say if it would make a performance difference without measuring, but it might.

Nate Sutton15:06:55

but I get what people mean

Nate Sutton15:06:29

@codonnell ah right that's a good point

Nate Sutton15:06:18

user=> (defn fetch [map key default-fn] (if-let [[_ value] (find map key)] value (default-fn)))
#'user/fetch
user=> (fetch {:a nil} :a (fn [] 5))
nil
user=> (fetch {:a 1} :a (fn [] 5))
1
user=> (fetch {:a 1} :b (fn [] 5))
5

Nate Sutton15:06:43

are single-character variable names a common thing in clojure?

Nate Sutton15:06:59

or is that just time savings typing in slack?

codonnell15:06:18

It's common to use m for a generic map, k for a generic key, and v for a generic value. I do it to avoid shadowing the core functions map, key, and val.

codonnell15:06:42

Of course, naming is very much a matter of personal opinion. 🙂

Nate Sutton15:06:54

if it's common then I'm ok with it

valtteri15:06:02

Elements of Clojure has great tips to naming. https://leanpub.com/elementsofclojure/read_sample

Nate Sutton15:06:09

I just don't want to bring over my other-language-isms into clojure

codonnell15:06:20

And I prefer more descriptive names when more is known about the value. +1 to elements of clojure chapter on naming; it's fantastic.

phrsantos20:06:15

How can I implement a 2dvector-&gt;pretty-string function? Something to take [[1 2] [3 4]] and return "1 2 \n 3 4".

john20:06:34

you seen pp/print-table?

phrsantos21:06:55

Hmm yea the thing is I don't want to print it. I want to format so it can be returned as an API response, so I thought I would format it as a string that can be displayed on the browser.

john21:06:31

You want with-out-str ala (with-out-str (println :hi)) => "hi\n"

john21:06:25

which works with pprint stuff as well

phrsantos22:06:54

Ok I'll give that a try. Thank you!

john20:06:55

If you want something slightly different, you could probably modify the implementation

seancorfield23:06:59

@nate_clojurians Highly recommended book. I bought it pretty much as soon as the first draft was available.