Fork me on GitHub
#aws
<
2020-02-06
>
defa12:02:43

Is cognitect.aws.client supposed to work with other S3 providers than amazon? I’m trying to connect to a NetApp StorageGRID S3 endpoint but don’t know how.

(aws/client {:api :s3
             :endpoint-override {:hostname ""}})
gives me an error:
Execution error (ExceptionInfo) at cognitect.aws.client.api/client (api.clj:94).
No known endpoint.

jsyrjala13:02:13

;; Mock S3 Service ;; https://github.com/adobe/S3Mock ;; https://github.com/adobe/S3Mock/blob/master/server/src/main/java/com/adobe/testing/s3mock/S3MockApplication.java (def props {S3MockApplication/PROP_HTTPS_PORT S3MockApplication/RANDOM_PORT       S3MockApplication/PROP_HTTP_PORT S3MockApplication/RANDOM_PORT       S3MockApplication/PROP_INITIAL_BUCKETS bucket       S3MockApplication/PROP_SILENT “true”       }) (def s3mock (S3MockApplication/start (HashMap. props) (into-array String []))) (def s3 (aws-api/client {:api :s3               :endpoint-override {:protocol :http                        :hostname “localhost”                        :port  (.getHttpPort s3mock)}}))

jsyrjala13:02:51

I think I got it working against locally running mock server

defa07:02:29

Haven not yet looked into what S3MockApplication really does but for me :endpoint-override seems not to work.

dchelimsky15:02:08

aws-api's interactions with aws are based on data descriptions of their apis: (https://github.com/aws/aws-sdk-js/tree/master/apis)

dchelimsky15:02:22

AWS does not make any promises about those descriptions aligning with specs that other vendors use to make s3-compatible services, so neither can we.

dchelimsky15:02:34

Actually, looking at the error you posted above, "no known endpoint", that comes from aws-api even before trying to send a request out.

dchelimsky15:02:37

OK, so the answer to your original question is that no, aws api does not promise to work with endpoints outside of aws. Please feel free to submit an issue https://github.com/cognitect-labs/aws-api/issues and we can continue the convo there.

defa16:02:22

When supplying a valid :region I manged to connect to a NetApp StorageGRID S3 endpoint.

(def s3 (aws/client {:api                  :s3
                     :region               "us-west-1"
                     :endpoint-override    {:protocol :https
                                            :hostname ""}
                     :credentials-provider (credentials/basic-credentials-provider
                                             {:access-key-id     ACCESS_KEY
                                              :secret-access-key SECRET_KEY})}))
(aws/invoke s3 {:op :ListBuckets})

defa16:02:16

withou :region the aws/client runs for a while an than failes with an error:

Execution error (ExceptionInfo) at cognitect.aws.region$chain_region_provider$reify__13062/fetch (region.clj:37).
No region found by any region provider.

defa16:02:23

with :region "foo" the error is a little misleading:

Execution error (ExceptionInfo) at cognitect.aws.client.api/client (api.clj:94).
No known endpoint. 

defa16:02:58

… but with :region "eu-central-1" the client is created and (aws/invoke s3 {:op :ListBuckets}) successfully delivers the expected results

kenny16:02:42

Can you create a presigned url with aws-api?

kenny16:02:48

It sounds like the pieces are there and just need to be assembled.

1
ghadi17:02:42

the s3 v4 signing api is really straightforward

ghadi17:02:48

(the non-streaming variant)