reitit

mafcocinco 2024-11-12T05:28:35.593439Z

I think I may have found a bug with how the ring-http-response library interacts with reitit, specifically content negotiation using muuntaja. it is entirely possible that i’m doing something wrong but wanted to verify before working on a patch. Here is what I found. This handler code works:

(defn handler
  "Handler function for the `/health` end-point"
  [request]
  (-> {:status "healthy" :message "All good, Captain!"}
      (resp/ok)
      (update :headers assoc "content-type" "application/json"))

mafcocinco 2024-11-12T05:28:48.512019Z

This handler code does not work:

(defn handler
  "Handler function for the `/health` end-point"
  [request]
  (-> {:status "healthy" :message "All good, Captain!"}
      (resp/ok)
      (resp/content-type "application/json")))

mafcocinco 2024-11-12T05:29:28.524779Z

AFAICT, the only difference is in the casing. The explicit assoc has a lower case content-type while the content-type function generates Content-Type. There could be something else I’m missing but that was all I could see. My understanding is that the casing of header keys should not matter. I’m following the example code found https://github.com/metosin/reitit/blob/master/examples/openapi/src/example/server.clj with the exception that I’m using aleph as my HTTP server library.

mafcocinco 2024-11-12T13:48:36.214919Z

Do you have thoughts on fixing this? It seems like there could be code in the wild that is depending on this behavior. Technically it is a bug fix but functionally could be a breaking change.

opqdonut 2024-11-12T13:49:35.380489Z

I think fixing it in ring-http-response and marking it as a breaking change would probably be right

Casey 2024-11-12T13:51:54.063559Z

In your first code block "content-type" is being assoced directly into the response map, and not the :headers map of the response.. which seems wrong to me.

Casey 2024-11-12T13:52:44.670129Z

.. and what do you mean by "works" and "does not work"?

mafcocinco 2024-11-12T13:55:47.553929Z

good catch. Let me check it again and I will verify if the behavior I’m seeing is still happening. I am probably picking up default behavior currently with the “working” example (which will explain in a moment)

opqdonut 2024-11-12T13:56:25.827939Z

ooof I'm blind, thanks for spotting that @ramblurr

mafcocinco 2024-11-12T13:57:29.209389Z

yeah, it was picking up default but with correct code, seeing same behavior. Here are the details:

Casey 2024-11-12T13:57:41.534559Z

> I am probably picking up default behavior currently with the “working” exampl Yes, I'm still not sure what wasn't working, but I do know that muuntaja defaults to json.

mafcocinco 2024-11-12T14:00:31.499579Z

Here is the corrected function

(defn handler
  "Handler function for the `/health` end-point"
  [request]
  (-> {:status "healthy" :message "All good, Captain!"}
      (resp/ok)
      (update :headers assoc "content-type" "application/json")))
which works. “works” is defined as hitting the server with a curl request returns the expected results Here is the previous, non-working function:
(defn handler
  "Handler function for the `/health` end-point"
  [request]
  (-> {:status "healthy" :message "All good, Captain!"}
      (resp/ok)
      (resp/content-type "application/json")))
When using this implementation, the server throws the following exception: java.lang.IllegalArgumentException: Don't know how to convert class clojure.lang.PersistentArrayMap into (stream-of io.netty.buffer.ByteBuf) My understanding of what is happening here is that the uppercasing of the Content-Type key within the headers map is preventing muuntaja from recognizing the content type and applying the middleware encoder on the response. That is what I mean by “not working”.

Casey 2024-11-12T14:05:57.634609Z

Yea that is likely what is happening.

Casey 2024-11-12T14:06:44.460629Z

My guess as to why this hasn't been raised before is that I suspect most people do not use helpers like resp/content-type preferring to manipulate the response map directly.

mafcocinco 2024-11-12T14:07:09.841199Z

yes, that is my guess as well, or they are just not setting content type and getting the default, which is almost always going to be JSON.

mafcocinco 2024-11-12T14:07:33.542329Z

suffice to say, fixing it is a breaking change.

Casey 2024-11-12T14:08:01.623949Z

if you are using muuntaja in the "normal" way, you shouldnt be setting the content type yourself

👍 1
Casey 2024-11-12T14:08:42.961989Z

that's one of the whole purposes of muuntaja, to negotiate the content type and then serialize the response body into the negotiated format (whether that be json or otherwise). muuntaja will set the content-type header.

mafcocinco 2024-11-12T14:09:25.879839Z

Is it standard that the negotiated response type is the same as the request type?

Casey 2024-11-12T14:09:51.545539Z

try omitting the setting of the content type in your response entirely, then curl your endpoint with -H "Accept: application/edn" or -H "Accept: application/json" or -H "accept: application/transit+json"

Casey 2024-11-12T14:10:15.187219Z

if muuntaja is correctly configured, then you will see the response body encoded in the requested format

mafcocinco 2024-11-12T14:10:58.052569Z

Cool. I will test that out. I’m pretty confident it will work and is configured correctly as the incorrect function you pointed out above was likely picking up defaults since the content-type was not being set into the headers map.

mafcocinco 2024-11-12T14:11:02.841829Z

Thank you for your help.

opqdonut 2024-11-12T14:11:53.007869Z

the bug in ring-http-response still exists, even though it doesn't impact you

opqdonut 2024-11-12T14:12:08.361209Z

I can see that it gets fixed

👍 2
Casey 2024-11-12T14:13:20.892659Z

Is it standard that the negotiated response type is the same as the request type?I can't easily answer yes or no to that question because "request type" is ambiguous. Rather, I will say that the client should send an Accept: header that lists the content types it requests, usually the first one is the preferred one. The server then parses that header, compares it to an internal list of what it supports (with muuntaja this is json, edn, transit, etc) and then uses that type for the response. Muuntaja automates that all for you, so you just return clojure data structures in your :body and muuntaja does the rest.

Casey 2024-11-12T14:13:50.742609Z

> the bug in ring-http-response still exists, even though it doesn't impact you yea indeed. and just looking at that ns, I see other instances of Title Cased headers too, not just Content-Type

opqdonut 2024-11-12T14:14:24.512289Z

tracked here: https://github.com/metosin/ring-http-response/issues/35

🙌 2
opqdonut 2024-11-12T06:01:46.145539Z

yep... looks like muuntaja only uses lower-case https://github.com/metosin/muuntaja/blob/master/modules/muuntaja/src/muuntaja/core.clj#L79-L84

opqdonut 2024-11-12T06:02:08.478829Z

I wonder how this isn't a more widespread problem in ring. I've never seen a case-insensitive-header-lookup helper fn

opqdonut 2024-11-12T06:02:55.778449Z

ah, the ring spec says header keys should be lower-cased: https://github.com/ring-clojure/ring/blob/master/SPEC.md#headers-1

opqdonut 2024-11-12T06:03:18.390359Z

so definitely a bug in ring-http-response

Casey 2024-11-14T19:52:53.526919Z

huh interesting

Casey 2024-11-14T19:53:18.475879Z

turns out the spec was wrong! and the ring 1.x spec doesn't have anything to say about the case of the headers

Casey 2024-11-14T19:54:04.008809Z

so the bug is probably https://github.com/metosin/muuntaja/blob/master/modules/muuntaja/src/muuntaja/core.clj#L79-L84 for expecting lower case headers

mafcocinco 2024-11-14T21:39:51.837979Z

That is a more stable change. Expanding what is acceptable rather than narrowing it.

Casey 2024-11-14T22:45:43.378489Z

Indeed, though the metosin folks really care about perf and probably wouldn't like to make that change because of the extra overhead.