Clojurians Log v2

(def email-regex #"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,63}$")
(s/def ::login (s/and string? #(re-matches email-regex %)))
(s/def ::id int?)
(s/def ::success boolean?)

(s/def ::create-user-request (s/keys :req-un [::login ::id]))
(s/def ::create-user-response (s/keys :req-un [::success ::login]))

(defn create-user [req psql-client]
  (tap> {:req req})
  (-> (http-response/ok {:success true :login "anna@yobst.de"})
      (http-response/content-type "application/json")))

(defn user-routes [psql-client]
  ["/api" {:tags #{"api"}}
   ["/openapi.json" {:get {:no-doc true
                           :openapi {:info {:title "yobst portal api"
                                            :version "1.0.0"}
                                     :tags [{:name "api"}]}
                           :handler (openapi/create-openapi-handler)}}]

   ["/user" {:post {:handler #(create-user % psql-client)
                    :request {:content {"application/json" {:schema ::create-user-request}}}
                    :responses {200 {:content {"application/json" {:description "success"
                                                                   :schema ::create-user-response}}}}}}]])

sending a random request with json body consisting of a map of login and id throws

[XNIO-4 task-2] ERROR io.undertow.request - UT005071: Undertow request failed HttpServerExchange{ POST /api/user}
java.lang.UnsupportedOperationException: Body class not supported: class clojure.lang.PersistentArrayMap
        at ring.adapter.undertow.response$eval29776$fn__29777.invoke(response.clj:46) ~[?:?]
        at ring.adapter.undertow.response$eval29731$fn__29732$G__29722__29739.invoke(response.clj:11)
~[?:?]
        at ring.adapter.undertow.response$set_exchange_response.invokeStatic(response.clj:59) ~[?:?]
        at ring.adapter.undertow.response$set_exchange_response.invoke(response.clj:52) ~[?:?]
        at ring.adapter.undertow$handle_request.invokeStatic(undertow.clj:23) ~[?:?]
        at ring.adapter.undertow$handle_request.invoke(undertow.clj:19) ~[?:?]
        at ring.adapter.undertow$undertow_handler$fn$reify__29853.handleRequest(undertow.clj:41) ~[?:?]
        at io.undertow.server.session.SessionAttachmentHandler.handleRequest(SessionAttachmentHandler.java:68) ~[undertow-core-2.3.14.Final.jar:2.3.14.Final]
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393) ~[undertow-core-2.3.14.Final.jar:2.3.14.Final]
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859) ~[undertow-core-2.3.14.Final.jar:2.3.14.Final]
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) ~[jboss-threads-3.5.0.Final.jar:3.5.0.Final]
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) ~[jboss-threads-3.5.0.Final.jar:3.5.0.Final]
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538) ~[jboss-threads-3.5.0.Final.jar:3.5.0.Final]
        at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282) ~[xnio-api-3.8.16.Final.jar:3.8.16.Final]
        at java.lang.Thread.run(Thread.java:1583) [?:?]

I'd assume reitit coerces the response to json but it seems it doesn't unlike in the example. I've got all the middleware in the same order like in this example: https://github.com/metosin/reitit/blob/master/examples/ring-spec-swagger/src/example/server.clj

When I comment out the response definition for the /user endpoint and manually respond stringified json it works fine.

in addition to the middleware, you need the :data {:coercion reitit.coercion.spec/coercion

looks like muuntaja is also expecting either a Content-Type header or a :muuntaja/encode flag to activate

right but you are setting the content-type...

yeah, even without the coercion, muuntaja/format-response-middleware should be encoding the response

are you giving reitit a :data {:muuntaja instance?

the reitit format-response-middleware is a no-op without a :muuntaja key in the route data

(def instance
  (m/create
   (-> m/default-options)))

(defn json-route-data [user-config dev-mode]
  {:coercion spec-coercion/coercion
   :muuntaja instance
   :middleware [
                #(auth/wrap-token-authorization % user-config)
                #(auth/wrap-token-authentication % user-config)
                #(wrap-token-access-rules % dev-mode)
                openapi/openapi-feature
                ;; query-params & form-params
                parameters/parameters-middleware
                ;; content-negotiation
                muuntaja/format-negotiate-middleware
                muuntaja/format-response-middleware
                ;; exception handling
                coercion/coerce-exceptions-middleware
                ;; decoding request body
                muuntaja/format-request-middleware
                ;; coercing request parameters
                coercion/coerce-request-middleware
                ;; coercing response bodys
                coercion/coerce-response-middleware
                ;; exception handling
                exception/wrap-exception]})

yeah that looks about right. as long as you're passing the output of json-route-data to reitit :P

I don't really have any good ideas at this point.

Or... you could rule out your own middleware aren't returning maps and clobbering the encoded body?

can help in debugging middleware stacks like this

see https://cljdoc.org/d/metosin/reitit/0.7.0-alpha7/doc/ring/transforming-middleware-chain

I am using the print-request-diffs and it looks good to me 🤷

Thanks nonetheless @joel.kaasinen

so if you're looking at the diffs is format-response-middleware just leaving the body as-is?

if so, you can try to use a debbugger or something to figure out which if-let is failing in format-response here: https://github.com/metosin/muuntaja/blob/master/modules/muuntaja/src/muuntaja/core.clj#L484-L491

maybe the charset doesn't get resolved or something?

I'll try to debug some more tomorrow

You could just hand roll your own ring handler but why? The servers we're using conform to the ring spec, so you'll be using a ring handler anyway. Build your own or pull one in

check the source of those functions: [create-default-handler](https://github.com/metosin/reitit/blob/5589328a3cff4051fa272008d8c944ff74563597/modules/reitit-ring/src/reitit/ring.cljc#L182-L213) [ring-handler](https://github.com/metosin/reitit/blob/5589328a3cff4051fa272008d8c944ff74563597/modules/reitit-http/src/reitit/http.cljc#L115-L171) both does not directly depend on ring so it would be trivial to handcraft alternatives

Ah okay, I was curious from a dependencies point of view. As in, if I am already working with reitit's http interceptors, muuntaja for http interface, and sieppari as an executor, then what piece of the puzzle was I missing? I thought reitit.http/routing-interceptor is equivalent to reitit.pedestal/routing-interceptor.

I have been reading the sources and got a bit lost in the mix of reitit's modules and wrappers (reitit-ring, reitit-pedestal, and reitit-http etc.). I asked because I don't want to hand-roll anything ... I will create enough bugs in my app as it is 😅

@ben.sless thanks for pointing out the need to conform to the ring spec... This piece didn't occur to me at all. Now things make more sense. Thanks.

> I prefer middlewares to interceptors, especially with virtual threads @ben.sless could you please expand on that? I was doing some searching of slack for "interceptors" and "virtual threads" and your off hand comment there is about all that popped up. What about middleware and virtual threads pair better than with interceptors? We often prefer interceptors because they are easier to use async, and they work well with existing async libs. I am curious to get a different point of view.

With virtual threads you can worry less about blocking then just write everything blocking and only implement blocking middlewares.

Interceptors bring along lots of incidental complexity around dynamic queues, asynchrony and performance. Synchronous middlewares are simplest and easiest to write. Synchronous middlewares are viable for all needs (terms and conditions may apply) given virtual threads. Hence, just use synchronous everything with virtual threads

Update... ooooookay so it's taken way longer than I will confess to in polite society, but it's all falling into place. I am, as I type this, having a moment of clarity.

Took me about 3-4 years of writing servers before the significance clicked for me Every ring handler is just RingHandler RingRequest -> RingResponse Every middleware is just RingHandler -> RingHandler Everything proceeds from this. As long as you satisfy these conditions, you're good

Yeah, same... Over the years, I had become reasonably okay with the ring handler/middleware model as well as the pedestal interceptor model. For this thread, my source of confusion was my vagueness about the interface between handler <> server. I think I get it now.

So I can potentially compose a stack like this: interceptors and/or handlers to connect with the business logic <> reitit for routing (at this level, malli/schema/spec to do coercions and validations) <> sieppari as an interceptor executor <> ring's jetty adapter to run the app which is wrapped as a ring handler using reitit's functionality (or --- to be tested --- wrap the app as a servlet and run it under a servlet container like nginx unit)

Said another way, using reitit as a router forces me to mentally pull apart all the other pieces. • If I use only pedestal, I can use all of pedestal and maybe some ring functions as utilities. • If I use only ring and compojure, well that's that. • If I use reitit, now I have to choose if I use only handlers, or only interceptors, or both. I also have to decide whether I should use sieppari to execute interceptors (with the app wrapped as a ring app for use with the ring jetty adapter) OR if I use pedestal as the executor, in which case I have essentially only swapped out pedestal's routing with reitit's routing, as I can write the whole thing with interceptors.

I prefer middlewares to interceptors, especially with virtual threads

Reitit could benefit from a batteries included package to just build the thing

This is far from fully baked but something like: https://github.com/bsless/companion/blob/master/examples/reitit/ring/server.clj#L222

Then configure it with merge-with merge a config you loaded via aero

I wonder sometimes how crazy I should go with indirection

I can create a full specification of a reitit server using pure data Should I?

> I prefer middlewares to interceptors, especially with virtual threads Never thought about this, but I see how this makes sense. Reitit tries to minimise the "all-or-nothing" middleware stack (compile out non-essential interceptors/middlewares). Whereas virtual threads would make it cheap(er) to do the middleware thing, on an amortized basis --- make better use of CPU cycles to get good enough throughput even if many middleware are essentially no-ops for most requests.

> I wonder sometimes how crazy I should go with indirection I suppose there is always a problem, and one more layer of indirection is always the answer 🤪

Hah, I read companion's readme... > Why are there so many attempts to re-implement Stuart Sierra's Component library? Generally, I feel like libraries may be Clojure's version of the "Lisp curse". It's a paradox. Composable libraries are a pretty good idea per se ((v/s concrete frameworks); quite literally compositon over inheritance). Buuut... If there evolve n kinds of m libraries --- given that anyone can have a different take on solving the same type of problem --- then we have a combinatorial explosion of choices. The web stack is a particularly good example of this problem because of how many things must be solved for. The set of libraries is large, and so are the design choices.