Fork me on GitHub
#yada
<
2016-08-10
>
cberg07:08:57

@severed-infinity: Thanks for your replies! I’m back from sleep now. 🙂 Yes, I’m only returning the entire ctx for debugging purposes. The incoming content-type is correctly identified as text/plain, as you can see in the error message. The Content-Type: application/json is for the error message response. I tried the two values for :consumes you suggested, but both cause a NullPointerException in java.util.regex.Matcher, unfortunately.

severed-infinity10:08:53

@cberg: that's odd, documentation shows it as the any case on media-type

cberg10:08:27

Can you give me a link to the documentation about that?

severed-infinity11:08:08

@cberg ah sorry after reading over what I read (being half asleep at the time) it seems I was wrong, that the “*” wildcard applies to charset. It was here I was looking https://github.com/juxt/yada/blob/35a0be13f35a07be99b8a343d588c3e4054b6375/test/yada/representation_test.clj though reading the official docs about media—type your approach of “/” should of worked…

kurt-yagram12:08:19

Yada doesn't seem to handle path parameters easily: I have a path parameter which contains |. Having this in an url is not really a good idea, so I url-encode the path parameter. This doesn't help: I get:

error in HTTP handler: java.lang.IllegalArgumentException: cannot treat nil as HTTP response for request to ''
      at aleph.http.server$invalid_value_response.invokeStatic(server.clj:133)
      at aleph.http.server$invalid_value_response.invoke(server.clj:130)
      at aleph.http.server$handle_request$fn__17484$fn__17485.invoke(server.clj:186)
      at manifold.deferred$eval8931$chain_SINGLEQUOTE____8952.invoke(deferred.clj:752)
Can

kurt-yagram12:08:15

Is is not possible to use url encoded path parameters with yada (swagger) - I've tested this with swagger UI

malcolmsparks12:08:43

I'll need to check. There is an outstanding issue with encoding path params in bidi. Sorry for this difficulty. Can you use a string and decode it yourself?

kurt-yagram12:08:27

it did decode, but it still gives this error

kurt-yagram12:08:29

the % sign is the problem: if a path parameter contains a %, I get this cannot treat nil as HTTP response for request to ''

kurt-yagram12:08:22

The only way now is to split the one path parameter into parts, but that's a bit annoying

cberg13:08:58

@severed-infinity: Now I see that it’s not only impossible to allow any media-type (`*/*`), but that only a small set (looks like 8?) of defined media-types is allowed, which is defined here: https://github.com/juxt/yada/blob/ffd89757d86d19f68aa7eb81d6336e8e181758f6/src/yada/request_body.clj

cberg13:08:26

Any other type (e.g. ‘image/jpeg’) is denied with a 415, even if it is listed in the :consumes set.

cberg13:08:54

Is there a reason to limit it like this? Should one always use multipart/form-data to upload binary data, even if it’s not coming from a browser form?

severed-infinity13:08:11

no no those exist for default values built for auto cocerions from clojure types and common files. it excepts all mime-types, I know from personal testing

cberg13:08:03

Quote from the source:

;; We return 415 if there's a content-type which we don't
;; recognise. Using the multimethods :default method is a way of
;; returning a 415 even if the resource declares that it consumes an
;; (unsupported) media type.

cberg13:08:59

I just tried putting image/jpeg into :consumes, setting this as the Content-Type header in curl, and I’m getting a 415.

severed-infinity13:08:26

http://www.iana.org/assignments/media-types/media-types.xhtml I believe someone on there is a application/binary or something along those lines I know yada uses something like application/octel-stream as the base type which is similar to binary but check that list and try them out. I’ve personally only ran into the 415 issue when the resource doesn’t correctly match the consumed request type

cberg13:08:14

Yes, application/octet-stream is in the ‘allowed’ list, and it’s the default fall-back type for arbitrary binary data, so I guess I will use this. I just thought it would be nice to have the content-type as available as metadata, if the client sends it.

severed-infinity13:08:38

I am just quickly testing out the image/jpeg

severed-infinity13:08:37

https://github.com/juxt/yada/blob/5e2b160027d33979571b5211363a26da0b6bbe43/test/yada/media_type_test.clj you can see here that image.jpeg should work from internal testing, The only thing I can think of is the resource model is not correctly structure

severed-infinity13:08:13

Gotta go be back in a while if you still need help, malcom might be able to give you a clearer answer though if he is around.

cberg13:08:15

I think the test you linked to only concerns content negotiation for the media-type of the response. The media-types for requests seem to be much more limited.

cberg13:08:55

Thanks for going through this with me, though! I already understand much better why I’m getting the reply I’m seeing.

cberg13:08:56

Still it would be interesting to know if there is a good reason for limiting the allowed media-types for requests like this.

kurt-yagram13:08:10

how can I make a put method to consume application/json instead of application/x-www-form-urlencoded?

:consumes [{:media-type #{"application/json"}
                  :charset "UTF-8"}]
but what should I add to parameters?

kurt-yagram14:08:25

Where can I find docs on how to consume application/json in post/put-requests? I can't seem to manage that properly. (Especially: how to make the :parameters {:body ...} part)

severed-infinity15:08:03

Hey @kurt-yagram I've created a complete resource model map for the documentation but it has not been posted yet I can send it over to you if you like?

kurt-yagram15:08:13

if I set

{:body {:date java.util.Date}}
I get in the swagger data type model:
{
  "date": "2016-08-10T15:17:48.469Z",
}
which looks fine. However, when I hit 'Try this out', it seems I can't use this, since the java.util.Date is not a String (it makes sense, I just don't understand why yada seems to know that dates are represented as strings, but than it doesn't):
"error": {
    "error": "clojure.lang.ExceptionInfo: Malformed body {:status 400, :error {:date (not (instance? java.util.Date a-java.lang.String))}}",
    "data": "{:status 400, :error {:date (not (instance? java.util.Date a-java.lang.String))}}"
  }

kurt-yagram15:08:30

@severed-infinity: yes please...

severed-infinity15:08:45

In the instance you've there you are sending over a JSON string but it expects a java date object. I believe you'll have to parse the string into a date object. I recommend looking at Cheshire for that. As for the file I will do in a moment just away from the computer for a minute

kurt-yagram15:08:41

Yeah, I get that, but the problem is: the error is thrown before I can access the json object. I do

(yada/resource {:methods
  {:put
    ...
    {:parameters {:body {:date java.util.Date}}}
     :response handle-func
    ...
  })
I suppose the type is checked before 'handle-func' is called.

kurt-yagram15:08:36

yada already replies with a

"status": 400,
  "message": "Bad Request",

kurt-yagram15:08:48

which is cool, although not in this case.

severed-infinity15:08:48

I guess a best case simple solution would be '{:date #{String java.util.Date}}' that way in a java context to will use a date directly and a JSON context you parse the string

kurt-yagram15:08:30

aah, allright, that cool, I'll try

severed-infinity15:08:50

But I am sending over the resource model now

kurt-yagram15:08:24

Thanks a lot - I'm a bit struggling with the parameter types and swagger, but I'm getting closer 😛

severed-infinity15:08:13

swagger is works great for the simple types but some types you just won’t be able to deal with from my own experience

severed-infinity15:08:28

for those case I recommend using postman

kurt-yagram15:08:00

ok... I'll check postman as well, but yada is pretty cool

severed-infinity15:08:12

also here is a snippet from my own project using jason

:methods        {:post {:parameters {:form {:phone String}
                                                       :body String}
                                          :consumes   #{"application/json" "multipart/form-data;q=0.9" "application/x-www-form-urlencoded;q=0.8"}
                                          :produces   #{"application/json" "application/edn"}
                                          :response   (fn [ctx]
                                                        (let [phone (or (get-in ctx [:parameters :body])
                                                                        (get-in ctx [:parameters :form :phone]))]

kurt-yagram15:08:17

yeah, with x-www-forms-urlencoded, it works better

severed-infinity15:08:40

with swagger for example I’ve found it is always sending application/x-www-form-urlencoded so I specify it in the consumes section with a lower quality for testing and in live cases the main go to is json

severed-infinity15:08:09

I know we can customise the swagger ui to allow other means of sending data over but I could never figure it out, and yada I know uses the default configuration for swagger

kurt-yagram15:08:33

By the way, did you manage to get JWT to work with swaggered yada?

severed-infinity15:08:48

That was my main reason for looking to customise swagger, got a whole bunch of links and stuff but in the end couldn’t figure it out to save my life and for those instances I use postman

severed-infinity15:08:36

I’d much prefer to have it enabled in swagger but I guess postman will suffice for now

malcolmsparks17:08:09

Better docs are coming, here's the first release of the yada docs as a full book: https://juxt.pro/yada/manual/index.html - the delay has been converting everything from markdown to asciidoc, and using the asciidoctor stylesheets, but this will enable cross-references, make it easier to include and explain code snippets (with higher accuracy), produce indices, etc. In due course a PDF will be made available (existing buyers of the PDF from http://leanpub.com/yada will get this as a free update)

malcolmsparks17:08:45

Swagger has been somewhat of a moving target over the past year, as the UI has been steadily improved - it will take a few days of sitting down with it and going through all the case and make sure they're working (many are not right now, especially for non-trivial cases).

malcolmsparks17:08:45

I've been hols recently and moving house this week has meant I haven't been able to respond very quickly to yada issues - please bear with me, normal service will be resumed soon, and the docs are definitely going to get better

severed-infinity18:08:41

Thats not a problem, can certainly understand and best to take time with it.

shaun-mahood18:08:16

@malcolmsparks: I like the new format, definitely easier to use than the old version. Thanks for putting all that effort in.

malcolmsparks18:08:54

@shaun-mahood: thanks - it's built on https://github.com/asciidoctor so I can't take the credit.

bhagany19:08:29

very much agreed. I was surprised when I got a 500 clicking on a link in an open tab, but after discovering the new version, I'm pleased.