Fork me on GitHub
#reitit
<
2021-01-08
>
karol13:01:51

Hi sorry if question is too simple, but what would be proper middleware to decompres requests which have content encoding: gzip ? I thought it should be handled automatically but it looks like [muuntaja.format.json/decoder] is called before it gets decompressed

juhoteperi14:01:09

https://github.com/clj-commons/ring-gzip-middleware or one of others gzip middlewares is needed I think most projects handle this in nginx or such loadbalancer

juhoteperi14:01:52

Or that one looks to be only about compressing responses

manutter5113:01:04

It should happen automatically, but I think you need to tell reitit to expect gzip as a content encoding.

manutter5113:01:17

Let me see if I can find where that’s specified

manutter5113:01:01

Hmm, I may be confusing that with something else, still digging

karol13:01:54

I can't even find mention of gzip in requests in reitit repo so I wonder if maybe this should be handled by web server

karol13:01:01

on which the handler is running

manutter5113:01:11

Are you using jetty as your web server engine?

karol13:01:22

I found some article from 2015

karol13:01:29

about adding gzip handling so will try it

manutter5113:01:50

:thumbsup: It looks like that’s the bit that needs to be configured to handle gzip encoding.

karol13:01:05

just need to check that danielsz/system jetty component allows passing of configurator flag 😄

karol13:01:28

it looks like it so will test the code

karol14:01:33

unfortunately it looks like it doesn't work, switched to http kit server and the same problem persists

karol14:01:44

but the request has proper content encoding set to gzip

karol14:01:44

but doesn't this gzip only responses?

karol14:01:16

and at the moment I am dealing with gziped body in request

juhoteperi14:01:33

Yeah. No idea how it works on the requests usually, never encountered problems.

manutter5114:01:21

Maybe a tangential question, but why are your incoming requests using gzip? Are they that large? A file upload would be, but file uploads usually use a different mechanism.

karol14:01:45

yeah unfortunately the request is rather large

juhoteperi15:01:03

Looks like clients normally don't send compressed data as they can't know if server supports it. You could implment this in a middleware yourself. Just make sure it runs before Muuntaja, and wrap the req body inputstream in GZIPInputStream.

karol15:01:05

thanks for help. I will probably need to settle on writting middleware as you mentioned albeit I would like to avoid it as custom code is always a higher chance of bugs 😄

karol15:01:10

for posterity, tested such simple middleware fn and it works:

(defn gzip-wrap [handler]
  (fn [request]
    (if (= (get-in request [:headers "content-encoding"]) "gzip")
      (let [gzip-body (GZIPInputStream. (:body request))
            response (handler (assoc request :body gzip-body))]
        (.close gzip-body)
        response)
      (handler request))))

karol15:01:05

needs more work around error handling, but for simple cases seems to work

juhoteperi15:01:06

with-open saves a few lines: (with-open [gzip-body (GZIPInputStream. (:body request))] (handler (assoc request :body gzip-body)))

karol15:01:23

you're right 👍