This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-10-29
Channels
- # aleph (3)
- # announcements (29)
- # babashka (99)
- # beginners (30)
- # calva (46)
- # cider (9)
- # clara (1)
- # cljsrn (4)
- # clojars (10)
- # clojure (41)
- # clojure-dev (4)
- # clojure-europe (45)
- # clojure-nl (3)
- # clojure-norway (13)
- # clojure-uk (5)
- # clojurescript (61)
- # community-development (11)
- # cursive (10)
- # data-science (1)
- # events (1)
- # fulcro (17)
- # graphql (1)
- # gratitude (1)
- # holy-lambda (1)
- # jobs (4)
- # jobs-discuss (5)
- # meander (22)
- # off-topic (50)
- # pedestal (3)
- # re-frame (3)
- # reagent (3)
- # reitit (82)
- # releases (2)
- # rewrite-clj (14)
- # shadow-cljs (3)
- # spacemacs (14)
- # tools-deps (7)
- # xtdb (33)
@viebel yes it's possible. Please read the coercion docs. If not there, there should be tests to cover that. Doc PRs welcome
Hello! I’m wondering if anyone has any examples they could share of custom-coercing query params in reitit (I’m using malli coercion in a reitit-ring app, but examples using libs other than malli would be helpful too). I want to convert a query param on a single endpoint from a comma-separate string of numbers to a tuple of 4 floats and having a harder time than I expected.
At first I tried to use a middleware just on that endpoint that would transform the query param, but it appears that parameter coercion happens before middlewares run, so of course my custom middleware never executes because the coercion step fails first.
I also tried adding a :reitit.middleware/transform
key (https://cljdoc.org/d/metosin/reitit/0.5.15/doc/ring/transforming-middleware-chain) to try to re-arrange the order of the middlewares for just that endpoint, but either it doesn’t work like I thought or I did it wrong
So now I think I need to construct a custom coercer with my own transformer and looking for examples/advice 🙂
I had a requirement like that, ages ago. I felt it was just easier to accept in the comma-separated list, then have a little mapper that would look for that parameter and split it into how I want it to be.
I did something very similar a few days ago with very custom coercion, had to create a custom instance and specify transformers for everything, but it worked out
I think I’ve got something nearly working in a similar way.. but yeah very hairy. I have a custom coercer accepting a custom provider which uses a custom transformer that has my own custom string decoder for the one param. I’m mostly just guessing how malli internals work at this point, which obviously is a frustrating way to code! I was thinking it would be ideal to just figure out how to do this with the existing coercion/transformation stuff so that in the handler I just have nice params I know are valid, but certainly just doing it manually at a later stage is an option, and maybe the way to go in this case if this pile of providers/transformers/decoders/schemas/registries is really the answer.. I’m not at all confident it is, though
@kiraemclean Use decide/string
to control how value is converted from string to collection or whatever: https://github.com/metosin/malli#value-transformation.
[:map [:status [:tuple {:decode/string split-and-coerce} :double :double :double :double]]]
what is split-and-coerce
in this example? just a function that takes the incoming param?
Yeah, something like (fn [s] (map #(Double/parseDouble %) (str/split s #",")))
In fact you might not even need parseDouble
, built-in :double
decoder might work
When I do something like that I get an invalid-schema
error from malli.. it appears I can mitigate it by naming that function and adding it in the :registry
under :options
in a custom coercer (`reitit.concern.malli/create`), but it doesn’t ultimately fix the issue
(def schema
[:tuple {:decode/string (fn [s] (str/split s #","))}
:double
:double
:double
:double])
(m/decode schema "1,2,3,4" mt/string-transformer) => [1.0 2.0 3.0 4.0]
split is enough, because tuple decode/string happens before :double decode, and built-in decode fn for :double is able to parse those separate numbers
this much makes sense but I’m still lost about how to integrate into an app using malli to coerce ring query params.. I’m not calling decode
anywhere explicitly, and I’m still quite confused about how the transformers/decoders are ultimately threaded through to the router,
:coercion reitit.coercion.malli/coercion
is enough for this case
🙈 I’m definitely missing something.. so when I have a ring/router that includes :coercion reitit.coercion.malli/coercion
and a route defined something like this:
["/path/"
{:get
{:handler ,,,handler here,,,
:parameters {:query [:map
[:my-param {:optional true :decode/string my-decode-fn}]]}}}]
and try to run my app, I get the error :malli.core/invalid-schema
I think I’ve convinced myself I need to go down the path Ben was describing, but it sounds like he may have figured out a way that’s slightly less brutal than the initial one I found..
In this example, the [:my-params] map key is missing the schema
the full message is :malli.core/invalid-schema {:schema {:optional true, :decode/string my-decode-fn}}"
["/path/"
{:get
{:handler ,,,handler here,,,
:parameters {:query [:map
[:my-param {:optional true :decode/string my-decode-fn} :string]]}}}]
If a vector under :map
schema has two elements, the second one is the schema for that property. If it has three elements, second is the options map and third the schema.
ah ok so I need to add :string
.. trying… ⏳
Well what ever is the schema for the type you want to decode into
could be [:tuple :double :double]
also if that's what you want to decode into
And the :decode/string
option could be at the property schema instead of at the map key-value schema.
ohh that’s progress.. now the validation is failing, saying “should be a string”.. which I guess is fair, at this point it’s not a string, it’s my tuple of 4 numbers, but it is actually using my decoder!
["/path/"
{:get
{:handler handler-fn
:parameters {:query [:map
[:my-param {:optional true :decode/string my-decode-fn} [:tuple :double :double]]]}}}]
["/path/"
{:get
{:handler handler-fn
:parameters {:query [:map
[:my-param {:optional true} [:tuple {:decode/string my-decode-fn} :double :double]]]}}}]
Both of these work, but I think the second one makes more senseso when I run it now the validation fails with “invalid type”.. this is further than I’ve gotten yet, though.. appreciate you explaining it all for me very much!
and this is certainly simpler than what I was doing before, and gets at least as far..
oooook it’s working! I can get a hold of the parsed/coerced params in my handler now. this is amazing, and so much better than what I was doing.. thank you so much @U061V0GG2. I owe you one if we ever cross paths 😄
No reason to setup custom transformer for this, built-in string-transformer will look at this schema property so you can control it per value.
Unless you're trying to do custom decoding (which I was) and it required dragging it kicking and screaming all the way up to the coercer
cool! I’m just getting into malli for the first time now. it’s an amazing library, thanks for making it 🙂 once I figure this out it might be more realistic to be able to contribute.
@ben.sless What do you mean "custom decoding"? Like different type of input than just strings? Or need to decode same schema different way in different cases?
Okay, there are valid cases for having own transformer and own decoding property
that certainly was one of them. If you're interested in a detailed usage report I can run it by my employer and even share the ugly bits of code, otherwise I can try to sketch things in general lines and mock examples
do you mean you were decoding something other than a string, or you wanted to process the string in some weird custom way?
so I was counting on a string decoder for :int
, then added :decode/round {:leave ,,,}
right.. I kind of want to do something similar. I want to parse an incoming query param that looks like "1,2,3,4"
into [1 2 3 4]
that’s the confusing part for me since I’m using this to try to parse query params in the middle of a ring/reitit app.. I made some progress building a custom coercer with a custom string transformer provider.. but that quickly got out of hand and ultimately still didn’t work
no specifically 4, it’s coordinates for a bounding box
so you can do something like [:tuple {:decode/query-params #(str/split #"," %)} :int :int :int :int]
Then provide a transformer which works in order, i.e. (mt/transformer {:name :query-params} mt/string-transformer)
then use that transformer when instantiating the coercion in reitit for query-params
btw, @U061V0GG2 might be worth making -provider
public
We have a very similar use case, I also needed to aggressively massage query-params
where do you provide that transformer? maybe some code would be helpful.. I have tried e.g.
(mcoerce/create
{:transformers {:body {:default mcoerce/default-transformer-provider
:formats {"application/json" mcoerce/json-transformer-provider}}
:string {:default <CAN PROVIDE CUSTOM TRANSFORMER PROVIDER HERE>
}
:response {:default mcoerce/default-transformer-provider}}
:error-keys #{:type :coercion :in :schema :value :errors :humanized :transformed} ;; set of keys to include in error messages
:compile mu/closed-schema ;; schema identity function (default: close all map schemas)
:validate true ;; validate request & response
:enabled true ;; top-level short-circuit to disable request & response coercion
:strip-extra-keys true ;; strip-extra-keys (effects only predefined transformers)
:default-values true ;; add/set default values
:options nil ;; {:registry swirrl-custom-registry} ;; malli options
})
where I made a custom transformer provider to use, but it quickly unravelled all the way to some very nitty gritty details that I don’t yet fully understand (and hasn’t worked yet), so I was wondering if there was some way to do it I’m missing
so leave the :string
one as the default, but add a query-params
key with my custom one?
This touches on a question I asked previously in a channel, but if adding a transformer in query-params doesn't solve it, try adding the same transformer in string, too
did you go source diving to figure out what that should look like?
hah.. relatable
Eventually I managed to build a very minimal example and just tested out the combinations until I found what worked
don't despair, it's a solvable problem, and lucky for you I already banged my head against that wall
heh. yeah..
I’m wondering if it’s worth doing malli coercion on the params in the first place.. it’s not a hard problem to deal with manually, would just be cooler if it was all magically done by the router
cool, no worries either way, but yeah if there’s some example you’re allowed to share seems like that would be super useful. have a great night! and thanks for this much help!
Resounding YES to your question. Shoving all coercion and parsing to the edge cleans so much damn cruft out of your code, leaving only with business logic
you'll find that once you get the hang of it, you even get a positive ratio of eliminated code
yeah.. I definitely see the benefits. painful, though!
Just to pinpoint the friction point I came across, might be different from Kira's case, I have custom transformers with my schema (works great, besides the leave/compile magic). I wanted to use that transformation for query params, ended up having to use the same transformer for string, still not sure why