Fork me on GitHub
#yada
<
2016-08-08
>
bhagany05:08:16

I'm trying to create a resource that accepts datomic transaction data in the request body, but it appears that yada doesn't like the reader tags (ie, #db/id[:db.part/db]), and throws a 400. I'm unsure where to plug in to change this behavior. Any tips?

bhagany05:08:43

I think my options are to make a new content type (`application/edn+datomic` or something) and then write a process-request-body method for it, or just parse it in my response function instead of relying on yada's parameter coercion. I'm going to play around with both of those, but any input is welcome.

danielcompton06:08:27

Are reader tags part of EDN?

bhagany06:08:51

@danielcompton: they are, but loading custom tags has to be done somewhere

bhagany06:08:41

iirc, if you're using clojure.core/read-string, the runtime loads tag definitions from specially named files on the classpath at startup, and uses those

bhagany06:08:06

if you're using clojure.edn/read-string, you need to pass in a reader map as the first argument

bhagany06:08:12

yada does the latter

bhagany06:08:18

(without the reader map)

danielcompton06:08:44

So yada should also be able to accept a reader map somewhere?

bhagany06:08:22

maybe? extending process-request-body wasn't terribly hard, so I wouldn't push hard for it, I don't think

danielcompton06:08:37

Just thinking that it might be better not to make a new content type. OTOH, making a new content type might be the more semantic move, as the edn can’t be interpreted without the reader tags as well?

bhagany07:08:00

yes, that's correct, as far as I understand

bhagany07:08:42

though I feel like I remember a talk about edn where Rich (?) said something about processes not needing to know what reader tags mean

bhagany07:08:51

if that's right, then I don't understand that part of it

malcolmsparks07:08:53

I agree with @danielcompton - a mimetype isn't only about formatting but about processing expectations. An EDN reader tag is arguably the latter.

malcolmsparks07:08:02

We don't usually run into this issue with JSON because it isn't extensible in the way EDN is, and XML for that matter

bhagany07:08:36

yes, agreed 🙂

bhagany07:08:04

now I've got that reader tag passthrough thing in my head… I might have to go find that talk tomorrow

malcolmsparks07:08:44

Actually my last remark is misleading. It's quite appropriate to indicate the type of json document you are sending.

malcolmsparks07:08:12

@bhagany: yes, do share the link!

bhagany07:08:32

I've seen people do that, though I never thought very hard about when or why that might be a good thing to do. The case for edn presents itself more readily.

malcolmsparks07:08:21

I suggest your mime-type should be application/datomic+edn. That means that processes that don't understand that would revert to application/edn.

bhagany07:08:34

ah, thank you

malcolmsparks07:08:38

Also, consider using a vendor tag, application/vnd.bhagany.datomic+edn

bhagany07:08:36

hmmm, I will read, thanks

bhagany07:08:31

I think it is too late for me to understand RFC's 🙂

bhagany07:08:55

I'll pick this back up in the morning (2:15 am here). Thanks for the suggestions!

malcolmsparks07:08:49

Reading new RFCs is all part of the fun. :) It's interesting for the most part how wide the gap between RFCs and defacto implementation is today.

lmergen07:08:19

that’s also a part of the nature of RFC’s, though

lmergen07:08:36

they’re supposed to be immutable, right ?

severed-infinity18:08:45

Guys by chance are there any examples of using datomic with the resources? currently trying to add a simple user but the response always returns

Cannot JSON encode object of class: class java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: :db.error/nil-value Nil is not a legal value
it would seem the execution of a transaction is playing some role, possibly just placed in the wrong place?
(defn lookup-user [phone]
  (let [chatty-db (d/db chatty-connect)
        result (d/q '[:find ?e :where [?e :user/phone phone]] chatty-db)]
    result))

(defn register-user []
  ;;TODO tie in datomic to this resource to add new user
  ;;TODO add buddy oauth support - done: sort of
  (yada/resource {:id             :server.core/register-user
                  :summary        "register user"
                  :description    "register a new user to the system"
                  :access-control {}
                  :methods        {:post {:parameters {:form {:phone String}
                                                       :body String}
                                          :consumes   #{"application/json" "multipart/form-data;q=0.9" "application/x-www-form-urlencoded;q=0.8"}
                                          :produces   #{"application/edn" "application/json"}
                                          :response   (fn [ctx]
                                                        (let [phone (get-in ctx [:parameters :body])]
                                                          (add-user phone)
                                                          ;; TODO if true?
                                                          (let [claims {:user phone
                                                                        :iss  "severed servers"
                                                                        :sub  "user id"
                                                                        :exp  (time/plus (time/now) (time/days 3))}
                                                                token  (jwt/sign claims secret)]
                                                            (json/generate-string token))))}}}))

severed-infinity19:08:58

woops forgot to include

(defn add-user [phone]
  @(d/transact chatty-connect [[:db/add #db/id[:db.part/user] :user/phone phone]]))

severed-infinity20:08:46

figured it out

(get-in ctx [:parameters :body])
return type was not a string

bhagany21:08:52

followup on the edn discussion: I haven't found the video yet, but I did discover that in addition to putting specific reader functions into the options map that is passed to clojure.edn/read-string, you can also pass a :default reader function that is used for unknown tags.

bhagany21:08:39

So, you can pass values through a process without knowing what they are with edn, it's just that you have to explicitly call it that way.

malcolmsparks22:08:34

I think we did discuss the possibility of making the built-in application/edn algo more configurable, so specifying extra tags wouldn't involve copy-and-pasting. However, I still think that if you have non-standard tags to convey, that warrants a custom mime-type.

malcolmsparks22:08:52

FWIW, I'm delighted yada users are creating and dispatching on their own custom mime-types. That's good RESTful practise. Not that we shouldn't improve the existing types.

malcolmsparks22:08:31

@severed-infinity: printlns are your friend. :) is there anything you'd suggest to improve things? I often get issues with Chesgire not knowing how to serialize stuff. Mind you, I remember the CORBA days when serialization issues took weeks of debugging....

severed-infinity23:08:41

@malcolmsparks: Well I had assumed that the :body would of returned the input as a string because I assumed it would of noticed the json string coming in, a simple wrap of str fixed it in this instance. Cheshire for me as been easy enough to use just wrapping in generate-string, though the only problem I have for me is client side (not being Clojure) having to parse the string. I’d of though JSON would of been more straight forward especially in a JavaScript context but if you have to parse and cocerse constantly end-to-end makes a little tedious but by no means impossible. Though while I have you hear I am using tex/event-stream for a push notification system and currently I have the response as string, looking at your examples I see you use a channel as the response, just curious as to wether the response should be just a constant pulling form a channel when content is available. It’s the next step for me after I get datomic integrated

severed-infinity23:08:01

@malcolmsparks: Oh CORBA was before my time, think it was being replaced by Java around that time ha

severed-infinity23:08:33

In regards to your MIME-type discussion about edn and it’s tags, rather than aiming to cover specific tags e.g. '#db/id’ for datomic for example have a system that allows any tags but leave it up to the users themselves to validate their tags, though as I write this it does sound a bit nuts because it would mean the likes of ‘#db/id’ too would be up to users to validate. Possibly a middle ground…