Fork me on GitHub
#babashka
<
2020-09-07
>
borkdude11:09:41

about http-kit client, how does one generally deal with errors? clj-http has :throw-exceptions true/false. with http-kit, should one check for :errors after deref and then throw yourself?

ep11:09:11

yes basically. iirc httpkit catches all internal exceptions and propagates them as :error

borkdude11:09:57

do you know if it generates an error on certain status codes, like 400?

ep11:09:24

i believe not. i think :error is only exceptions like SSL/Timeouts/etc... but i will double check

ep11:09:16

https://github.com/http-kit/http-kit/blob/master/src/org/httpkit/client.clj#L282 shows that :error should be a throwable. see its https://github.com/http-kit/http-kit/blob/master/src/java/org/httpkit/client/IResponseHandler.java. also see the docstring:

"Issues an async HTTP request and returns a promise object to which the value
  of `(callback {:opts _ :status _ :headers _ :body _})` or
     `(callback {:opts _ :error _})` will be delivered.
  The latter will be delivered on client errors only, not on http errors which will be
  contained in the :status of the first.

borkdude11:09:36

Right, so:

user=> (type @(c/get ""))
clojure.lang.PersistentArrayMap
Doesn't this mean every user of httpkit is going to write their own synchronous clj-http-ish wrapper?

ep11:09:52

wdym? like to do de-structuring/error-handling?

ep12:09:34

yeah, i suppose there is some forced boiler-plate there

borkdude12:09:39

maybe babashka could use a babashka.http-client namespace which removes this boilerplate, making it look like babashka.curl, but based on http-kit instead. while also adding http kit itself for async use.

borkdude12:09:58

then it could also become a drop-in replacement for babashka.curl

ep13:09:08

because you want non-2xx or 3xx to be treated as an error? i guess to be consistent you would want the same semantics for the async callback right?

borkdude13:09:26

babashka.http-client would only imitate clj-http with the synchronous interface

borkdude13:09:34

if you want async, you can use http-kit directly, but you would have to deal with more boilerplate yourself, which is a reasonably trade-off imo

ep13:09:44

ah, so they would be different namespaces in other words. you'd just be using http-kit for everything under the hood to reduce implementation complexity

borkdude13:09:11

(require '[babashka.http-client]) ;; implemented using httpkit, looks like clj-http sync, throws on a selection of status codes
(require '[org.httpkit.client]) ;; same as httpkit, SNI client already enabled by default

ep13:09:06

i personally dont mind the httpkit semantics. maybe because i got used to it, but it doesnt seem right to treat errors that happen on the level of SSL or TCP the same as HTTP errors. but for the compatibility reason with the current behavior it makes sense what you propose

borkdude13:09:02

Do you have a lot of :error checks in your httpkit code?

borkdude13:09:11

we can distinguish errors of status and errors on ssl level using an ex-info

borkdude13:09:28

or just re-throw the java exception and use an ex-info for status codes

borkdude13:09:42

I usually just want my script to exit with an error in case of a faulty request

ep14:09:24

i guess if its in the context of a script its one thing, but in the context of a long-running service at the company i work, we want to collect metrics for different response types, including per status code, but also for things like timeout/ssl errors. these can be important for back-pressure/circuit-breaking/retry considerations. but there are very different implications for 4xx vs 5xx vs timeouts vs sslerror which might be dynamic depending on the endpoint you're speaking with. also some apis will return success codes, but then return error messages/internal codes in the response body which must be parsed and considered. might point is in any case, eventually you start getting down into the granularity of status codes and exception/error types anyhow, so the boilerplate becomes unavoidable. but again, probably these considerations are not relevant in the majority of scripting cases and a binary success/error is enough

borkdude13:09:11

(require '[babashka.http-client]) ;; implemented using httpkit, looks like clj-http sync, throws on a selection of status codes
(require '[org.httpkit.client]) ;; same as httpkit, SNI client already enabled by default

hugod23:09:45

Any possibility of adding java.security,DigestInputStream to allow computation of hashes from input streams?