Fork me on GitHub
#yada
<
2016-09-05
>
kurt-yagram11:09:42

I'm trying to implement a verify-function (`{:keys verify}`) in the verify-method (`defmethod verify`) of yada. All seems to work pretty well, except, I need some fields of a body to do some verifications. So:

(defmethod verify ::my-verification
  [ctx {:keys [verify]}]
    ...
      (partial (verify (-> (get-in ctx [:request :body])
                           io/reader
                           ...))
and well, in order to be able to read the actual body, I add line-seq. Here, the problems begin: if I use line-seq anywhere inside the verify-method as argument to the verify-function (or inside the verify-function), I get an error:
ERROR:[clojure.tools]clojure.tools.logging.invoke at logging.clj:288
Internal Error : java.lang.NullPointerException
      at manifold.stream$__GT_source.invokeStatic(stream.clj:86)
      at manifold.stream$__GT_source.invoke(stream.clj:78)
      at manifold.stream$connect.invokeStatic(stream.clj:325)
      at manifold.stream$connect.invoke(stream.clj:300)
      at manifold.stream$connect_via.invokeStatic(stream.clj:509)
      at manifold.stream$connect_via.invoke(stream.clj:494)
      at manifold.stream$map.invokeStatic(stream.clj:612)
      at manifold.stream$map.invoke(stream.clj:608)
      at yada.interceptors$process_request_body.invokeStatic(interceptors.clj:217)
      at yada.interceptors$process_request_body.invoke(interceptors.clj:181)
      at manifold.deferred$eval8982$chain___9003.invoke(deferred.clj:862)
      at manifold.deferred$eval8982$chain___9003.doInvoke(deferred.clj:886)
      at clojure.lang.RestFn.applyTo(RestFn.java:151)
      at clojure.core$apply.invokeStatic(core.clj:661)
      at clojure.core$apply.invoke(core.clj:652)
      at manifold.deferred$eval8982$chain___9003$fn__9007.invoke(deferred.clj:891)
      at manifold.deferred.Listener.onSuccess(deferred.clj:219)
      at manifold.deferred.Deferred$fn__8781.invoke(deferred.clj:379)
      at clojure.lang.AFn.run(AFn.java:22)
      at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
      at manifold.executor$thread_factory$reify__8184$f__8185.invoke(executor.clj:44)
      at clojure.lang.AFn.run(AFn.java:22)
      at java.lang.Thread.run(Thread.java:745)
ERROR:[clojure.tools]clojure.tools.logging.invoke at logging.clj:288
error in HTTP handler: com.fasterxml.jackson.core.JsonGenerationException: Cannot JSON encode object of class: class java.lang.NullPointerException: java.lang.NullPointerException
      at cheshire.generate$generate.invokeStatic(generate.clj:154)
      at cheshire.generate$generate.invoke(generate.clj:116)
      at cheshire.generate$generate.invokeStatic(generate.clj:124)
      at cheshire.generate$generate.invoke(generate.clj:116)
      at cheshire.core$generate_string.invokeStatic(core.clj:73)
      at cheshire.core$generate_string.invoke(core.clj:48)
      at yada.body$eval27830$fn__27831.invoke(body.clj:166)
      at clojure.lang.MultiFn.invoke(MultiFn.java:233)
      at yada.body$eval27810$fn__27811.invoke(body.clj:89)
      at yada.body$eval27675$fn__27676$G__27664__27683.invoke(body.clj:34)
      at yada.handler$standard_error.invokeStatic(handler.clj:73)
      at yada.handler$standard_error.invoke(handler.clj:72)
      at yada.handler$handle_request_with_maybe_subresources$fn__33188.invoke(handler.clj:150)
      at manifold.deferred$catch_SINGLEQUOTE_$fn__9029.invoke(deferred.clj:964)
      at manifold.deferred.Listener.onError(deferred.clj:220)
      at manifold.deferred.Deferred$fn__8801$fn__8802.invoke(deferred.clj:400)
      at clojure.lang.AFn.run(AFn.java:22)
      at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
      at manifold.executor$thread_factory$reify__8184$f__8185.invoke(executor.clj:44)
      at clojure.lang.AFn.run(AFn.java:22)
      at java.lang.Thread.run(Thread.java:745)

kurt-yagram11:09:27

This is so, even if I add this to my yada-resource:

{:access {:scheme ...
          :verify (fn[body arg] [true])
the vector is intended: all values need to be true (there can be multiple conditions to be checked)

kurt-yagram11:09:24

defmethod verify and line-seq don't seem to be best friends, maybe? 😛

dominicm11:09:07

@kurt-yagram This might be completely off the mark, but line-seq is lazy, so if the resource closed beneath it, it might break (NPE)?

kurt-yagram11:09:29

allright, that'll be it

kurt-yagram11:09:15

so I probably should add into '() or something?

dominicm11:09:13

doall is the idiomatic way.

dominicm11:09:36

http://paul.stadig.name/2016/08/reducible-streams.html actually, this is probably a better article contemplating your particular issue. (`doall` is an easy solution)

kurt-yagram11:09:04

ok, I'll try...

kurt-yagram11:09:58

hmmm, can't make it work so far.

kurt-yagram11:09:05

will try again after lunch 😛

dominicm11:09:17

Maybe it's not a laziness issue then 🙂. I'm not entirely sure of the issue.

kurt-yagram12:09:58

🙂 - or it may just be an api design issue. Does it make sense to authorize based on body-fields at that point on the flow? This is the actual case: the user is authorized to use a certain method (POST, PATCH, ...), but, the user is not allowed to POST, PATCH, ... everything. So, I have to check if the user is allowed to perform the actual request. I can do this in the response function as well. Which may make more sense, or not?

gphilipp12:09:00

I’m trying to configure bidi to listen on https and thought I just had to switch the scheme from http to https (like on https://github.com/juxt/edge/blob/master/src/edge/web_server.clj#L92). But it still listens on http. Do I do anything wrong ?

kurt-yagram12:09:19

you know what strikes me more about this: I do get the expected result-map from the verify-method, i.e. a map with 'user' and 'roles'. So something must be happening after the map has been constructed (and returned).

(defn- check-token ""
  [auth ctx-verify]
  (try (let [jwt (unsign auth)
             check-jwt ...
             check-verify (every? true? (ctx-verify jwt))]
         (Logger/info (str "validated:\n" jwt))
         (match [check-jwt check-verify]
                [true true] {:user ...
                             :roles #{...}}
                [false _] {:type :validation
                           :cause :issuer}
                [_ false] {:type :validation
                           :cause :context-fail}))
       (catch ExceptionInfo e
         (if-not (= (ex-data e)
                    {:type :validation
                     :cause :signature})
           (throw e)))
       ))


(defmethod verify ::jwt
  [ctx {:keys [verify]}]
  (let [bearer (get-in ctx [:request :headers "authorization"])]
    (when bearer
      (let [auth (str/replace-first bearer #"^Bearer " "")]
        (Logger/info (str "validating token: " auth))
        (when-not (nil? auth)
            (let [body ...?]
              (Logger/info (str "body:\n" body))
              (check-token auth (partial verify body))
              ))
        ))
    ))
results in :
validating token: <token>

[manifold-pool-2-1:18592]INFO:
   body:
   {  "user-id": ...,  "type": ...,  ...}

[manifold-pool-2-1:18592]INFO
   validated:
   {...
    :user_id <user-id>,
    ...
  }
   

[manifold-pool-2-1:18592]INFO
   Generate auth object:
   {:user <user-id>,
    :roles #{...}}
   

[manifold-pool-2-1:18592]ERROR
   Internal Error : java.lang.NullPointerException
      at manifold.stream$__GT_source.invokeStatic(stream.clj:86)
...

malcolmsparks13:09:45

@gphilipp - it's harder than that. bidi is only matching on the Host header in the request. Setting up https is an aleph thing - it requires at least a self-signed certificate. Setting up SSL is a bit of a blackart in Java. All my systems defer https to nginx, or similar

gphilipp13:09:29

@malcolmsparks ok, i’ll check it out

malcolmsparks13:09:04

@kurt-yagram - you're sure your call to verify isn't calling the defmulti again?

malcolmsparks13:09:22

sorry, ignore that last message

malcolmsparks13:09:49

where are you getting body from?

malcolmsparks13:09:51

is the body part of your authentication process? that would be tricky, because the body is processed by the process-request-body interceptor

kurt-yagram13:09:59

well, I only have ctx from the method verify:

(defmethod verify ::jwt
  [ctx {:keys [verify]}]
  ...

malcolmsparks13:09:02

the body isn't available at auth time

kurt-yagram13:09:23

well, that's the thing: I need the body for authorization

kurt-yagram13:09:44

well 'need' is a big word. I'd like to have the body for authorization

malcolmsparks13:09:45

@kurt-yagram - right, that's probably the issue then

kurt-yagram13:09:16

allright... so I better leave body-based authorization out of verify, and put in inside the response-function?

malcolmsparks13:09:41

currently yada does authentication and authorization BEFORE processing the body. The thinking here is that you want the person sending the request to get passed the auth checks before accepting their body

malcolmsparks13:09:46

because the body might be an attack

malcolmsparks13:09:24

there's nothing to stop you doing some authentication and authorization without the body, then processing the body (which puts it into the context) -then in your method response you complete the authorization using the body

kurt-yagram13:09:50

nope, that's how it is now. I thought it made more sense to add all authorization to verify, but that doesn't seem to be the most logic thing to do...

malcolmsparks13:09:11

@kurt-yagram yeah, sometimes you have to compromise

kurt-yagram13:09:55

I'll do that with pleasure

lmergen18:09:06

@malcolmsparks: you should add/update a link to the real manual at https://yada.juxt.pro/index.html

lmergen18:09:12

it's the #2 link in Google for "juxt yada", and caused some confusion for a coworker here today

lmergen19:09:58

no problem, figured I'd let you know