Fork me on GitHub
#reitit
<
2021-07-07
>
Ben Sless05:07:19

How much attention has been given to middleware performance in reitit? When using the full stack as in the examples it seems like plenty of CPU is burning there, especially on ring/params doing lots of merges. I don't know if this should be solved in reitit or in the ring libraries. Weavejester is pretty conservative and might reject wacky performance enhancement PRs. Would you rather I take it with him or offer an alternative?

ikitommi06:07:50

@ben.sless most mw that are included in reitir are partially optimized, most wrapped or direct 3rd party mws are slow. Fast versions welcome!

ikitommi06:07:41

also, there is a keywordize-keys step in coercion, which might not be needed. Some perf todos around in the code

Ben Sless06:07:38

In particular, ring.middleware.params/params-request is a big offender, using both merge and a heavy regex. I'll need to recreate the flame graphs (that's how I caught that muuntaja bug previously). Wondering where to start, though. Would you rather I PRed it to the ring project and take the updated version from there, or have an in-house version?

Ben Sless06:07:28

For keywordize-keys, a compromise could be porting https://clojure.atlassian.net/browse/CLJ-1239

ikitommi07:07:18

you can pull the mw into reitit-middleware module. For perf, the current ring convention is that all params are merged into :params. To be compliant with that, it will be slow. Not adding those by default will be a big source of confusion for beginners.

Ben Sless07:07:09

Doesn't have to be, even merging the maps using reduce-kv rather than using merge-with merge will be better

ikitommi07:07:17

also, inlined lots of ring helpers, ring doesn't have a performance target, which is totally ok (and a good call looking at the history!), so most request/response helpers are not optimized

Ben Sless07:07:32

Curious regarding the historical perspective

Ben Sless07:07:22

Still, looking at this, even without breaking compatibility we can both see how it can be trivially sped up

(defn assoc-query-params
  "Parse and assoc parameters from the query string with the request."
  {:added "1.3"}
  [request encoding]
  (merge-with merge request
    (if-let [query-string (:query-string request)]
      (let [params (parse-params query-string encoding)]
        {:query-params params, :params params})
      {:query-params {}, :params {}})))

n2o07:07:46

Hi, I have a question regarding optional parameters in a request. Let’s take one of your examples and focus on it:

["/:company/users/:user-id" {:name ::user-view
                             :coercion reitit.coercion.schema/coercion
                             :parameters {:path {:company s/Str
                                                 :user-id s/Int}}}]
That’s from your readme. How can I annotate, e.g., company as optional parameter? Is this possible?

n2o07:07:03

In our case we have multiple parameters, which can optionally be set, e.g. booleans. If they are not sent, they are assumed to be false.

n2o08:07:58

Ok, I’ll just split it into separate Routes.

n2o09:07:08

Actually, I have more cases where a parameter is just optional. So, again, I am looking for a way to mark it as optional 🙂

juhoteperi12:07:19

(s/optional-key :company) s/Str

juhoteperi12:07:04

Though I empty path params probably cause problems, but you could also at least share the handler & params data even if you have multiple routes.

(let [data {:parameters ... :handler}]
  [["/users/:user-id" data]
    ["/:company/users/:user-id" data]]

juhoteperi12:07:55

Or maybe query params are better fit for params that are optional?

n2o13:07:58

Yes, in these cases that’s true. I currently share the parameters and split the routes to be more expressive. but in some cases that’s still not enough. For example I have a Feedback-Route in my webapp, and contact-name is optional. But I don’t want to have two routes for it to have a proper documentation. The spec for it would be:

(s/def ::feedback-dto (s/keys :req [:feedback/description]
                              :opt [:feedback/contact-name :feedback/contact-mail]))
But when I use this spec in the :parameters-Section in the route definition, all fields are shown as required in the resulting swagger document

n2o13:07:46

For the spec-coercion, it is fine to have the definition from above. But when we look it up in swagger, all feeds “feel” required.

n2o13:07:03

instead just description actually is required.