2024-11-04 aws | Clojure Slack Archive

aws

grounded_sage 2024-11-04T17:21:45.834479Z

Has anyone used cognitect.aws to access other services that have S3 compatibility. Like Minio and Tigris? I am trying this and it keeps saying

(def s3-client
    (aws/client {:api                          :s3
                 :region                       "us-east-1" 
                 :credentials-provider         (credentials/basic-credentials-provider
                                                {:access-key-id     tigris-access-key
                                                 :secret-access-key tigris-secret})
                 :endpoint-override            {:protocol :https
                                                :hostname "fly.storage.tigris.dev"
                                                :port     443}}))
  
  (aws/invoke s3-client {:op :ListBuckets})

I get:

{:Error
 {:Code "SignatureDoesNotMatch",
  :CodeAttrs {},
  :Message
  "The request signature we calculated does not match the signature you provided. Check your key and signing method.",
  :MessageAttrs {},
  :Resource "/",
  :ResourceAttrs {},
  :RequestId "1730740845164803807",
  :RequestIdAttrs {}},
 :ErrorAttrs {},
 :cognitect.aws.http/status 403,
 :cognitect.anomalies/category :cognitect.anomalies/forbidden,
 :cognitect.aws.error/code "SignatureDoesNotMatch"}

viesti 2024-11-06T07:26:52.076879Z

tried with Minio, and then found an issue which helped https://github.com/cognitect-labs/aws-api/issues/150#issuecomment-1193840869 > The gist of the problem here is with default-endpoint-provider, which needs valid AWS region (like us-east-1 or us-east-2 etc) to return data. So supplying any valid region here works.

$ cat deps.edn
{:deps {com.cognitect.aws/s3        {:mvn/version "869.2.1687.0"}
        com.cognitect.aws/api       {:mvn/version "0.8.692"}
        com.cognitect.aws/endpoints {:mvn/version "1.1.12.772"}}}

$ docker run --rm -d --name minio-test -p 9000:9000 -p 9001:9001  server /data --console-address ":9001"
4f0e778c3404286a590d94a36dc7843b875a6144f62a2889e077cdf115138f84

$ clj
Clojure 1.12.0
user=> (require '[cognitect.aws.client.api :as aws]
         '[cognitect.aws.credentials :as credentials])
nil
;; Make access key in Minio console: 
user=> (def access-key "yetOb0arQSRhs8Qk6lgT")
#'user/access-key
user=> (def secret-key "3IrNynpji9y1PAnRAdx76mzwAVderOEgejkw0S40")
#'user/secret-key
user=> (def s3 (aws/client {:api :s3
                           :region "us-east-1" ;; <--- region here
                           :credentials-provider (credentials/basic-credentials-provider
                                                  {:access-key-id     access-key
                                                   :secret-access-key secret-key})
                           :endpoint-override {:protocol :http
                                               :hostname "localhost"
                                               :port     9000
                                               :region "us-east-1"}})) ;; <--- region here too
#'user/s3
user=> (aws/invoke s3 {:op :ListBuckets})
{:Buckets [], :Owner {:DisplayName "minio", :ID "02d6176db174dc93cb1b899f7c6078f08654445fe8cf1b6ce98d8855f66bdbf4"}}

Jacob Emcken 2025-04-06T15:24:36.768989Z

I usually do the following when working with Minio. 1. Setup ~/.aws/config with:

[profile minio]
region = us-east-1

2. Setup ~/.aws/credentials with something like:

[minio]
aws_access_key_id = some_access_key
aws_secret_access_key = some_secret_key

3. Enable that profile: export AWS_PROFILE=minio 4. Run Clojure code:

(require '[cognitect.aws.client.api :as was])

; (System/getenv "AWS_PROFILE") ; should return "minio"

(def s3
  (aws/client {:api :s3
               :endpoint-override {:protocol :http
                                   :hostname "localhost"
                                   :port     9000}}))

(aws/invoke s3 {:op :ListBuckets})

scottbale 2024-11-04T19:24:23.864909Z

Are you able, with this same setup and credentials, to :ListBuckets directly from S3? Without overriding the endpoint, I mean, and bypassing Tigris altogether.

grounded_sage 2024-11-04T19:30:58.567289Z

I ended up getting GPT to write a wrapper around the AWS Java SDK and all works fine for me now.

👍 1

viesti 2024-11-06T08:05:25.063399Z

Hmm, seems that a custom :region-provider works too (has to be some known aws region though)

user> (def s3 (aws/client {:api :s3
                           :region-provider (reify cognitect.aws.region/RegionProvider
                                              (fetch [_this]
                                                "us-east-1"))
                           :credentials-provider (credentials/basic-credentials-provider
                                                  {:access-key-id     access-key
                                                   :secret-access-key secret-key})
                           :endpoint-override {:protocol :http
                                               :hostname "localhost"
                                               :port     9000}}))
#'user/s3
user> (aws/invoke s3 {:op :ListBuckets})
{:Buckets [], :Owner {:DisplayName "minio", :ID "02d6176db174dc93cb1b899f7c6078f08654445fe8cf1b6ce98d8855f66bdbf4"}}

Clojurians Log v2

aws