Clojurians Log v2

@rbarker has joined the channel

I’m getting errors using compojure-api and therefore ring-swagger with spec coercion enabled - the exception says that there is no spec defined for a keyword ‘Unable to resolve spec’ - however I am simply checking for the presence of a keyword in the response map. Any ideas?

@stathissideris we have multiple projects on reitit (and c-api), but nothing big in prod yet - the oldest projects have started last year, haven’t had any rough corners in a while. The routing core is quite stable and proven, ring-side has still some moving parts, and it’s not on par with c-api on all the features. I would say it’s alpha, but so is everything nowadays 😉

for the muuntaja: https://github.com/metosin/muuntaja/pull/77

@rbarker could you show some minimalistic code?

(GET "foo" []
        :produces ["application/json"]
        :summary "foobar"
        :return (s/get-spec ::foo)
(s/def ::foo (s/keys [::bar]))

This would cause an exception saying that spec ‘bar’ cannot be found

does :return ::foo work?

is the ::bar defined?

no

only by creating a spec (s/def ::bar some?) do the swagger docs load

my understanding is that if a spec is not defined then simply the presence of the key/value is asserted

ok, I thought it would fail too.

the code to walk over the specs is here: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/visitor.cljc

That should be changed so that if the key is not defined, return nil and handle that gracefully.

PR for that welcome.

I think I found a bug in spec-tools or in compojure-api using it... Not sure though what is causing it currently

It's a weird one. Everything was working fine until I added nil as a possible value to one of the spec definitions via (s/or ....). After that I get:

my-appl.api/fn             api.clj:   94
                               compojure.api.coercion/coerce-request!        coercion.clj:   45
              compojure.api.coercion.spec.SpecCoercion/coerce-request            spec.clj:  125
                                            clojure.spec.alpha/unform           alpha.clj:  157
                                         spec-tools.core.Spec/unform*           core.cljc:  233
                                            clojure.spec.alpha/unform           alpha.clj:  157
                     clojure.spec.alpha/regex-spec-impl/reify/unform*           alpha.clj: 1658
                                         clojure.spec.alpha/op-unform           alpha.clj: 1449
                                                                  ...                          
                                                  clojure.core/mapcat            core.clj: 2775 (repeats 2 times)
                                                   clojure.core/apply            core.clj:  652
                                                     clojure.core/seq            core.clj:  137
                                                                  ...                          
                                                  clojure.core/map/fn            core.clj: 2745
                                      clojure.spec.alpha/op-unform/fn           alpha.clj: 1449
                                         clojure.spec.alpha/op-unform           alpha.clj: 1446
                                            clojure.spec.alpha/unform           alpha.clj:  157
                       clojure.spec.alpha/map-spec-impl/reify/unform*           alpha.clj:  808
                                            clojure.spec.alpha/unform           alpha.clj:  157
                                         spec-tools.core.Spec/unform*           core.cljc:  233
                                            clojure.spec.alpha/unform           alpha.clj:  157
                          clojure.spec.alpha/every-impl/reify/unform*           alpha.clj: 1274
                          clojure.spec.alpha/tuple-impl/reify/unform*           alpha.clj:  976
                                            clojure.spec.alpha/unform           alpha.clj:  157
                                         spec-tools.core.Spec/unform*           core.cljc:  233
                                            clojure.spec.alpha/unform           alpha.clj:  157
                        clojure.spec.alpha/or-spec-impl/reify/unform*           alpha.clj: 1046
                                                                  ...                          
java.lang.UnsupportedOperationException: nth not supported on this type: Keyword

This happens when a request comes in and compojure-api tries to coerce the data

if I remove the compojure-api form the picture and try to validate the input data against the same schema definitions via (s/explain ...) it doesn't break like that but it does report bunch of problems which are due to the fact that coercions have not been ran

could you try using spec-tools.core/decode?

good idea, one moment

returns :clojure.spec.alpha/invalid but doesn't explode like that

btw, go vote up https://dev.clojure.org/jira/browse/CLJ-2251 to get the native coercion into clojure.spec. there is also CLJ-2116 but “not likely to be accepted”

yeah, would be nice

hmm.. it seems that c-api is not using the st/decode yet, but https://github.com/metosin/compojure-api/blob/master/src/compojure/api/coercion/spec.clj#L119-L125 instead.

calling conform doesn't break it either, let me replicate that logic and see if I get to duplicate the problem

the s/invalid? doesn’t work for some reason? shoudn’t be…

think you need to add s/nilable around your spec

if you copy-paste the code i highlighted and run it locally with the data, does it blow up?

and yes, s/nilable is the way to go, but still shoudn’t blow up on error.

weirdly no, it does not blow up

but I do not pass transformer in my code snippet

copy-pasting that stuff would require copying a lot of other stuff also

but let me try if s/nilable would allow me to go past this problem at this point, that being said this bug should be duplicated with a minimal reproduction case

this is way too complex as it is now to work as an example case plus I cannot put this anywhere publicly

right, changing from s/or to s/nilable bypassed the problem

which is doubly weird

anyway, I'll try to reproduce this later with a more simple example. Thanks for the ideas which allowed me to bypass this problem this time

and naturally if I do manage to reproduce it I'll post an issue to github

is there a good way to turn muuntaja off for a request body?

actually, this is the wrong question. I want to add a new format that is simply ignored by muuntaja

@bja just for somr routes? Sadly no. The middleware are applied in place, so you would need to disable it at api and re-mount the mw for some routez

basically, I want to mount an application/csv that just slurps the request body (or passes it through), but I'd still like muuntaja to handle the response negotiation via Accept

It should work out by default?

I thought (assoc-in muuntaja/default-options [:formats "application/csv"] {:decoder [slurp]} was close

ooh, maybe I did something else that is trying to cause coercion

worse, I feel like I figured this out with my old codebase (such that it was documented properly in csv), but I don't have access to said codebase anymore

I suppose, alternately, can I add per-endpoint custom swagger documentation?

I think the slurp is not right. It should be a function returning the decoder.

oh, found it here: https://github.com/metosin/compojure-api/wiki/Swagger-integration

yep, via :swagger

is there a way to encode that the body should be a CSV string in Swagger?

looking at the OpenAPI spec, it's not immediately obvious that this is supported for the body, although it seems to be an opt for individual parameters

OpenAPI has better options for body, don't think Swagger2 supports those. Setting`:consumes` might help the ui. OpenAPI3-support PRs welcome...

Spec-tools is quite close, schema-tools would require some refactoring.