Fork me on GitHub
#babashka
<
2023-07-18
>
Stephan Renatus08:07:59

@borkdude I screwed up the atomist PR — I had assumed the script did some sort of artifact picking, but it doesn’t — and now we’ve got zip files in the manifest that don’t exist https://github.com/babashka/pod-registry/pull/81/files#diff-a5df325c189d5736bf405f48faea5eff0d3016cb757e4fb081226c38eb3d141dR10 😳

Stephan Renatus09:07:01

I can’t find an example of a non-archived (zip or tgz) artifact/url in any of the manifests. is it supported? if it’s not, let’s just revert my PR

borkdude09:07:47

I'll just revert

Stephan Renatus09:07:00

thanks. sorry for the mess 😞

borkdude09:07:10

no worries ;)

borkdude09:07:05

I reverted it now.

Stephan Renatus17:07:28

I must be holding this wrong…. I’m trying to use awyeah-api with a local minio endpoint. I can run curl and get an XML response, but I can’t get it to work with awyeah-api…

Stephan Renatus17:07:39

(ns s3-utils
    (:require [com.grzm.awyeah.client.api :as aws]
              [com.grzm.awyeah.credentials :as credentials]))

(def s3 (aws/client {:api :s3
                     :region "eu-west-1"
                     :endpoint-override {:protocol :http
                                         :host "127.0.0.1"
                                         :port 9000}
                     :credentials-provider (credentials/basic-credentials-provider
                                            {:access-key-id     "minioadmin"
                                             :secret-access-key "minioadmin"})}))
this is what I’m trying

Stephan Renatus17:07:00

and this is what I get in the REPL

s3-utils=> (aws/invoke s3 {:op :ListBuckets})
{:cognitect.anomalies/category :cognitect.anomalies/fault, :cognitect.anomalies/message "java.net.http.HttpConnectTimeoutException: HTTP connect timed out", :com.grzm.awyeah.http-client/throwable #error {
 :cause "HTTP connect timed out"
 :via
 [{:type java.util.concurrent.CompletionException
   :message "java.net.http.HttpConnectTimeoutException: HTTP connect timed out"
   :at [java.util.concurrent.CompletableFuture encodeRelay "CompletableFuture.java" 368]}
  {:type java.net.http.HttpConnectTimeoutException
   :message "HTTP connect timed out"
   :at [jdk.internal.net.http.MultiExchange toTimeoutException "MultiExchange.java" 581]}
  {:type java.net.ConnectException
   :message "HTTP connect timed out"
   :at [jdk.internal.net.http.MultiExchange toTimeoutException "MultiExchange.java" 582]}]

Stephan Renatus17:07:53

"HTTP connect timed out" sounds simple enough; but I don’t know how to approach this… I don’t know if it ends up connecting to some other endpoint, failing that

chucklehead18:07:51

Can you try :hostname instead of :host

Stephan Renatus18:07:40

💡 lovely, that’s something I can work with

(aws/invoke s3 {:op :ListBuckets})
{:Error {:HostIdAttrs {}, :Message "The request signature we calculated does not match the signature you provided. Check your key and signing method.", :CodeAttrs {}, :ResourceAttrs {}, :RequestIdAttrs {}, :HostId "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8", :MessageAttrs {}, :RequestId "17730AB06E4D9C86", :Code "SignatureDoesNotMatch", :Resource "/"}, :ErrorAttrs {}, :cognitect.anomalies/category :cognitect.anomalies/forbidden}

Stephan Renatus18:07:16

things to get sorted now: how to use it without credentials at all (its bucket can be made public); or how to fix the signature…

Stephan Renatus18:07:40

@U015879P2F8 thanks again. now if I’d only remember where I got that :host from. suddenly, all open tabs do say :hostname 🙈

chucklehead18:07:58

No problem. Not sure on the other error, but some googling seems like it may be related to the port number not being included properly in the signature. If you can start minio on port 80 you could confirm whether or not that’s the source of the issue.

chucklehead18:07:49

And probably leave the port out of your override settings if you do that test.

Stephan Renatus19:07:35

thanks I should be able to take it from here. I still feel embarrassed about being so lost in the first place — would there have been some way to have debug logging; or better yet, a debugger to use from the REPL…? :face_with_spiral_eyes:

chucklehead19:07:04

Not sure if there’s additional logging from the library that would’ve shown you where it was trying to connect. My inclination probably would’ve been to launch wireshark or configure proxying through something like fiddler or mitmproxy and then you probably would’ve seen that it was trying to connect to real S3 endpoint urls via http on port 9000 and realized your override wasn’t configured completely.

Stephan Renatus19:07:27

thanks for sharing!

chucklehead19:07:38

But since I couldn’t I poked around in the source for a couple minutes and saw what params it was destructuring

Stephan Renatus19:07:08

ah can you share that location with me? I’d love to have a look and have just started browsing them

Stephan Renatus19:07:38

thanks again 🙂

Stephan Renatus18:07:50

ℹ️ got the signatures to be accepted by changing minio to listen on 80, so the requestHost would end up as localhost instead of (before) localhost:9000. 🥳

Stephan Renatus18:07:44

@U015879P2F8 I’m afraid it took me to figure out how to look at minio’s audit logs first before I’ve tried going with this, what was your suggestion all along.

chucklehead11:07:18

Glad to hear you got it sorted 🙂

dabrazhe18:07:28

What is the hottest go-to http client library to use with bb? I'd like the generic get, post (with clojure map as payload), etc, and the ability to ignore ssl errors (like curl with the -k option).

pesterhazy18:07:26

• built into babashka, works in clojure if you want to eject • built on the rock-solid java.net.http

pesterhazy18:07:53

I don't know about ignoring ssl error, but at least with java.net.http it looks like it's possible https://stackoverflow.com/a/52995420/239678

pesterhazy18:07:43

> :insecure - if true, an insecure trust manager accepting all server certificates will be configured.

dabrazhe19:07:14

Cool, thanks. I created the SSL context, but not sure to which key in opt to bind in the request of (post url opt) ? Tried {:ssl-context sslc} but post does not accept it as is appears

dabrazhe19:07:39

Does post accept the same parameters as request ? This does not seem to work

(def client (http/client
             {::ssl-context (http/->SSLContext {:insecure true})}))
(def resp (http/post "" {:client client} ))

pesterhazy19:07:08

s/::ssl-context/:ssl-context/

2
dabrazhe19:07:17

Yep, caught that one as well )

pesterhazy19:07:37

you probably also want to merge in default-client-opts

pesterhazy19:07:56

i'm just sittin here reading the api docs, same as you 🙂

dabrazhe19:07:52

thank you for your effort) It's probably not related but post can't resolve localhost. (No name matching localhost found) curl has no issues. Is there another context i am missing

pesterhazy19:07:38

I don't know. The StackOverflow thread I linked has some options about hostname verification

pesterhazy19:07:08

Can't you give the https server a self-signed cert with the right hostname?

dabrazhe19:07:26

This is plan B, indeed. It appears the error No name matching localhost found does relate to ssl. Will need to explore

borkdude20:07:46

@U96LS78UV There is a fix on master for this, please try the --dev-build

dabrazhe11:07:20

Did you mean the other way around, fixed in dev? I have to say I don't know what to you mean by --dev-build @borkdude

borkdude11:07:45

master =d ev

borkdude11:07:59

bash <(curl ) --dev-build --dir /tmp

dabrazhe12:07:36

Yes, it been fixed indeed, thank you!

👍 2
borkdude12:07:15

cool, I'll probably release a new bb tomorrow then

borkdude09:07:16

fixed in 1.3.182

🧡 2
dabrazhe13:07:52

The installer script still installs the v 1.3.181, have checked now, by the way. https://github.com/babashka/babashka/releases/download/v1.3.181/babashka-1.3.181-macos-aarch64.tar.gz

borkdude13:07:44

whoops, should be fixed now

borkdude13:07:55

(barring github caching)

dabrazhe13:07:38

Yes, it's updating, you are a Flash: )