Fork me on GitHub
#reitit
<
2021-11-10
>
Yehonathan Sharvit11:11:35

I am asking myself what's the best way to tweak my reitit middlewares and my route Malli schema in order to achieve the following requirements: 1. post body in JSON format 2. Keys that appear in a route schema should be keywords 3. Keys that don't appear in a route schema should be strings Here is an example of a post body in JSON:

{
 "user-id": "joe",
 "num-of-purchases": 10,
 "additional-info": {
    "Address Line 1": "Baker street",
    "Custom field 2": "Cats and dogs"
 }
}
And here is the Clojure map I'd like the http handler to receive:
{:user-id "joe"
 :num-of-purchases 10
 :additional-info {
    "Address Line 1" "Baker street"
    "Custom field 2" "Cats and dogs"}}
Here is my current solution: 1. Configure muuntaja so that JSON keys are not coerced to keywords 2. Define a route Malli schema like this:
[:and
 [:map-of :keyword :any]
 [:map
   [:user-id :string]
   [:num-of-purchases :int]
   [:additional-info [:map-of :string :any]]]
It works, and here is what the middlewares do to the keys: 1. Muuntaja (with my custom config) leaves all the keys as strings 2. Malli coercion middleware transforms "user-id", "num-of-purchases" and "additional-info" into keywords, due to [:map-of :keyword :any], but leaves "Custom field 1" and "Custom field 2" as strings It works as I expect but I am not sure that it's the best way to do that.

1
juhoteperi11:11:21

That seems like a good solution.

juhoteperi11:11:58

In theory, we could have a JSON decoder which is aware of Malli schemas, so it could do this in one go, but such tool doesn't exist now.

juhoteperi11:11:33

Malli should also be able to convert those :user-id etc. keys to keywords even without :map-of schema. BUT this might be broken right now.

juhoteperi11:11:09

I guess I forgot to create issue about that.

juhoteperi11:11:34

(m/decode [:map [:user-id :string]] {"user-id" "foo"} mt/json-transformer) => {"user-id" "foo"} Yeah. This should work.

Yehonathan Sharvit11:11:00

@U061V0GG2 Do you have an opinion about what should be muuntaja default regarding the coercing of map keys from keywords into string for JSON data?

juhoteperi11:11:18

Most Clojure apps will presume all maps use keywords

juhoteperi11:11:47

It is not optimal always, but I think it is reasonable default and changing would break most existing apps.

Yehonathan Sharvit11:11:23

Makes sense. What about building an app from scratch. What would you use?

juhoteperi11:11:41

Docs (in both Muuntaja and Reitit/Malli coercion) could mention that you can switch keyword keys of and let Malli coerce keys.

Yehonathan Sharvit12:11:09

Once the bug is fixed:stuck_out_tongue_winking_eye:

juhoteperi12:11:17

I've used keywords in all recent cases, and then I let Malli to decode keyword -> string if I have [:map-of :string ...].

juhoteperi12:11:35

It is some extra work, but I don't usually need to care about perf that much.

juhoteperi12:11:22

(m/decode [:map-of :string :any] {:1 "foo" :2 "bar"} mt/json-transformer) => {"1" "foo", "2" "bar"}

juhoteperi12:11:32

I think these :map-of string key cases are so rare, it best to optimize the regular keyword keys use.

juhoteperi12:11:09

Letting Jsonista to directly use keywords for keys is faster than doing that conversion using schema in Malli decoding. Malli would have to do that for every map value, vs. [:map-of :string ...] only affects cases where :map-of schema is used.

Yehonathan Sharvit13:11:27

Let's take the simplest case: converting a JSON map to a clojure with a [:map-of keyword ...] schema, without any string keys. What is faster (and why)? 1. Using jsonista with coercion to keyword and then Malli coercion basically going over the fields and leaving them as is 2. Using jsonista without coercion to keyword and then Malli coercion basically going over the fields and coercing to keyword

juhoteperi08:11:41

The simplest and the most common case is [:map [:foo ...]]. Most of the endpoints will use some parameters like this. When using the default Jsonista keyword keys object mapper, Jsonista will build the correct Clojure maps in one go, and Malli coercion doesn't have to modify the maps keys at all. (One pass over parameters.) [:map-of :string ...] case is relatively rare, so it is not usually a problem that Malli would convert map keys back to strings. (Two passes over these maps, but doesn't happen that often.) If using string keys with Jsonista, Malli would always have to modify nearly every request body and other parameter map to convert all the keys. (Two passes over the parameters.)

vlaaad14:11:00

If I use reitit/ring/muuntaja, how do I express that for a particular endpoint :responses 200 can be ONLY of type image/svg+xml ? e.g. how do I remove default json/edn stuff?

vlaaad14:11:15

so far I have this in a route map:

:get {:responses {200 {:body string?
                       :content {"image/svg+xml" {}}
                       :description "OK"}}
it adds new content type to a list of possible types, but how do I remove the rest?

juhoteperi14:11:03

Probably :content ^:replace {"image/svg+xml" {}} I don't remember if this is just implemented as normal route-data, which is merged using meta-merge rules, but if it is, then this will tell Reitit to replace the :content value instead of merge all the values from different levels together.

vlaaad14:11:27

tried that already, no success 😞

vlaaad14:11:57

also tried putting ^:replace on maps higher in the hierarchy

juhoteperi14:11:30

Is this to remove the content types from Swagger listing?

juhoteperi14:11:17

{:swagger {:produces ["image/svg+xml"]}
 :handler ...}

juhoteperi14:11:26

Not sure if this is confusing, if, .e.g., error responses are still JSON or something

juhoteperi14:11:07

Or maybe you can set this :produces list to empty vector, to get only the content type from :responses. I don't remember how does that :responses :content works, but I think Reitit-muuntaja will add set up :swagger :produces list, so that's probably where those JSON etc. are coming from.

vlaaad16:11:01

thanks, I’ll check this out!

Michael Mackenzie21:11:13

Hey guys, quick question: I'm using clojure.spec coercion with reitit. I've managed to get it working, except I can't seem to find out how to make a parameter optional. For example, here is the route definition that I am working with:

["/pie-chart" {:get {:handler    (partial pie-chart-handler analytics-data-access)
                        :parameters {:path  {:dataset string?}
                                     :query {:startTs int?
                                             :endTs   int?}}}}]
I would like startTs and endTs to be optional query parameters (i.e. it's okay to omit them, but if present they must be integers), but can't seem to figure out how to accomplish this. Any help would be appreciated. Thank you!