Fork me on GitHub
#aws
<
2018-08-29
>
n.duchenne12:08:09

Hello, is it there an easy/elegant way to sign a spandex (elasticsearch) request with an AWS signature? (I'm using aws-sig4 + clj-http at the moment)

valtteri12:08:58

Hi @n.duchenne I’m not aware of any easy way but you can look for hints here https://github.com/portkey-cloud/aws-clj-sdk/blob/master/src/portkey/awssig.clj

valtteri12:08:46

(assuming making the signature is the problem here)

valtteri12:08:35

What comes to Spandex, I think you can slip in your custom headers when making the request

n.duchenne12:08:49

Thanks @valtteri, that's useful indeed; I had been looking for such functions. I'll check the custom headers. Otherwise, maybe I could use spandex's proxy feature, and either use one of the simple command-line AWS signing proxies that are around, or write my own proxy with the signing functions you pointed out. I'd welcome recommendations/warning about ready-made signing proxies before I embark on coding mine though.

valtteri12:08:29

I have no experience on proxies, but it sounds like maybe a good idea if writing your own turns out nasty. One option is to ditch Spandex and use client that knows how to sign requests for AWS.

n.duchenne13:08:03

Yes, given that it's just REST, using such a client might be the way to go. I'm browsing the code of the aws-clj-sdk to see whether it does this. Do you know? Or any other client?

n.duchenne13:08:10

portkey.aws/-rest-json-call looks promising. Thanks @valtteri!

valtteri13:08:31

I’ve used AWS ES only with Node where there are many client libs that can talk with AWS and currently I’m using non-AWS ES with Spandex. Nice that you found something useful and let me know if you get it to work with Spandex. 🙂

n.duchenne13:08:30

Looks like there are nice things in Node indeed, including a signing proxy that could work with Spandex https://www.npmjs.com/package/aws-es-proxy. Will investigate and let you know.

valtteri13:08:46

When we were doing microservices we setup a lambda with API-GW and created custom REST-interface for ‘our search’ that hid the ES-interface and exposed only use-cases that were relevant to us. We also used lambdas to index docs into ES directly from DynamoDB streams.

valtteri13:08:01

I think this was the client lib I used in the lambdas https://www.npmjs.com/package/aws-elasticsearch-client

n.duchenne13:08:43

Cool, thanks!

valtteri13:08:45

Could have been this as well.. Can’t remember for sure 🙂 https://www.npmjs.com/package/http-aws-es

n.duchenne16:08:24

Well, @valtteri, using a third-party proxy proved straightforward. I installed node.js and the aws-es-proxy package on my EC2 instance, and then it was as simple as starting the proxy

aws-es-proxy 
and then using localhost:9200 as host with spandex
(require '[qbits.spandex :as s])
(def c (s/client {:hosts [""]}))
(s/request c {:url [:testindex] :method :put :body {:settings {"number_of_shards" 1}}})
(s/request c {:url [:testindex] :method :get})
Not full clojure solution that I wished for, but it's so simple that I'll take it for now.

valtteri17:08:44

Nice that you got it sorted out. :+1: Also good to know that there’s a proxy “escape hatch” for signing.