Fork me on GitHub
#reitit
<
2019-08-26
>
jussip07:08:37

Newbie question about interceptors, are there examples how to test interceptors (call interceptor from test)? I just started to use Reitit.

ikitommi07:08:53

@danielcompton so: 1) spec-tools defines transformations based on :type, here’s the list of predefined types: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/parse.cljc#L66-L86 2) reitit has custom string-transformer and json-transformer which set the transformation of type :map to spec-tools.transform/strip-extra-keys-transformer, here https://github.com/metosin/reitit/blob/master/modules/reitit-spec/src/reitit/coercion/spec.cljc#L12-L20, the code on spec-tools here: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/transform.cljc#L184-L185, e.g. the :map mapping is done on the spec-tools side (kinda hides the mapping, which is not good) 3) here’s a code to create a custom coercion: https://github.com/metosin/reitit/blob/master/modules/reitit-spec/src/reitit/coercion/spec.cljc#L137. e.g. create a new options map with modified string & json transformer. Sorry for all the boilerplate. This should be much simpler, without resolving to global settings. ideas welcome.

ikitommi07:08:44

also, as spec is not intended for the runtime transformations, the fail-on-extra-keys is kinda hack and can’t report on the extra keys.

ikitommi07:08:42

I think doing explicit closed specs would be simpler and more explicit, using spec-tools.spell, here are samples: https://github.com/metosin/spec-tools/blob/master/CHANGELOG.md#092-alpha1

ikitommi07:08:14

When Spec2 arrives, there are more official ways to close spec, currently much WIP/TBD.

jussip07:08:00

Thanks, I think it was that chain creation that was wrong.

4
ikitommi07:08:52

… about the coercion settings. I think the coercion instance could be optionally backed by a multimethod, which would dispatch on the :type and would use that as an override to default. One could change the mapping like:

(defmulti my-transformer identity)
(defmethod my-transformer :map [_] spec-tools.transform/fail-on-extra-keys)

;; all mappings from the multimethod are collected at coercion creation time
(def my-spec-coercion (reitit.coercion.spec/coercion (assoc reitit.coercion.spec/default-options :transformer my-transformer))

ikitommi12:08:25

no, the multimethod would be a bad idea. Explicit map is better in all possible ways.

👍 4
abdullahibra16:08:15

how reitit handles back button?

abdullahibra16:08:41

i would like to set back button to specific page at some pages

kenny17:08:56

What is the recommended way to parameterize your handler functions? Perhaps something like this:

(defn init-data-interceptor
  [config]
  {:enter (fn [ctx] (assoc ctx :config config))})

(defn new-handler
  [config]
  (http/ring-handler
    (http/router
      (routes)
      {:interceptors [(init-data-interceptor config)]})
    (ring/create-default-handler)
    {:executor sieppari/executor}))
Then your routes can access config in the ctx. The same could be done for DB connections, cache initialization, etc.

kenny17:08:11

If you do it like that, how do you get auto reload to work ?

ikitommi17:08:19

@kenny we use the reloaded workflow and run reset to rebuild all the things. It's few hundred millis with all the db pools, http server etc.

ikitommi17:08:09

for your code, looks good. Another way is to use compile-time injection: interceptors can access the router opts and store any data from it in a closure, so you don't need to inject things into request at request-time. All the default interceptors are written like this.

ikitommi17:08:24

hope this helps

kenny17:08:02

@ikitommi Thanks! I've only used the auto-reload in the past. Could try the reset everything approach, I suppose. How do interceptors access the router opts? e.g. What key is that under?

ikitommi17:08:19

all the keys, e.g. the whole router opts is passed as the sexond arg to the :compile function, first one being the route data for the given route: https://github.com/metosin/reitit/blob/master/modules/reitit-http/src/reitit/http/coercion.cljc#L6-L26

ikitommi17:08:12

each interceptor is compiled separately against all routes, so they can be optimized based on route data, can also be unmounted.

ikitommi17:08:39

We thought that the compiling is good just for the library interceptors/middleware, but I think all our client projects use that now. So easy to reason about, expecially when you define the route data expectations in a spec. Can fail fast at router creation on invalid data, opposed to failing at runtime when a route is called.

kenny17:08:28

Hmm, interesting. So you're suggesting to create a new key in the map passed to reitit.http/router. I would then add keys to the routes themselves to mount the necessary interceptor?

kenny17:08:37

This also implies that there's less of a reason to pass interceptors directly in a route's data.

kenny17:08:54

i.e. everything should be a "global" interceptor

ikitommi17:08:48

Of you have a :db you need to use in an interceptor, you either: 1) pass it to the function to create the interceptor (like you did) an use as a free variable in the interceptor functions (enter, leave, error) 2) put in into router options (could be just :db or :system key which has :db under it) and use a :compile hook. Can be combined with route data. 3) use request/context Injection, but it extra work at runtime

ikitommi18:08:17

for example, authorization is a good candidate for the route data: add a authorization-interceptor for all routes, make it read :roles route data at :compile: if that exists, return :enter function that verifies that the context has the roles needed, if no :roles are defined, unmount the interceptor, e.g. return nil from compile. The interceptor needs most likely the db/ldap resource, can be passed via router options to :compile or as a parameter to function that creates the interceptor.

ikitommi18:08:49

In both cases, the route data rules, and the interceptor stack adapts to routes.

ikitommi18:08:32

Add a spec to the data (and attach that to the interceptor via :spec key) and enable route data validation and you'll get expound-pretty errors at router creation time.

kenny18:08:26

I see. I think this approach makes sense. Will give it a go. Thank you!

kenny18:08:35

Is there a page that lists the keys in the ctx?

kenny18:08:24

Would it make sense to add a session-interceptor to reitit-interceptors?

ikitommi19:08:02

definetely, it would.

ikitommi19:08:37

not listed, the interceptor docs are kinda non-existing. but there is :request and :response, that’s it I recall.

ikitommi19:08:52

oh, and :error.

kenny19:08:21

I think I must be misunderstanding something. I have a session interceptor:

(def session-interceptor
  {:name    ::session
   :spec    ::session-interceptor
   :compile (fn [{:keys [session]} _]
              (prn "here")
              (when session
                (let [options (default-session-options session)]
                  {:enter (fn session-interceptor-enter
                            [ctx]
                            (update ctx :request #(session-middleware/session-request % options)))
                   :leave (fn session-interceptor-leave
                            [ctx]
                            (update ctx :response #(session-middleware/session-response
                                                     %
                                                     (:request ctx)
                                                     options)))})))})
and some routes:
(def routes2
  [["/ping" {:session {:store "asd"}
             :get     (fn [ctx]
                        (prn "ping")
                        {:status  200
                         :body    "pong2"
                         :headers {}})}]])
and I create the handler here:
(http/ring-handler
  (http/router
    routes2
    {:interceptors   [interceptors/session-interceptor]
     :validate       reitit.ring.spec/validate
     ::rs/explain    s/explain-str})
  (ring/create-default-handler)
  {:executor sieppari/executor})
The route data is invalid but I don't get a message printed out. Why not?

kenny19:08:35

Further, "here" is never printed which must mean the session-interceptor :compile function is never called.

kenny19:08:25

Ohhh, all those options are supposed to be passed to ring-handler, not http/router haha

kenny19:08:31

Well, some of them. Looks like the validate and rs/explain need to be passed to http/router.

kenny19:08:02

Ok, that rearrangement fixes the no "here" prints. Still not getting data validation errors printed though.

ikitommi19:08:33

you should put the :interceptors under :data in the router options.

kenny19:08:28

Oh wow, I was way off haha.

ikitommi19:08:06

there is no validation of router options currently, should be…

ikitommi19:08:59

also, the one and only complication of compiling each interceptor per route is that if you have something like in-memory session storage, that is instantiated in an interceptor - there is session storage per route. here’s the issue showing how to do that (and a cause to write a built-in interceptor to make this go right for all users): https://github.com/metosin/reitit/issues/205

kenny19:08:46

Ah, right - makes sense.

kenny19:08:50

Not sure I follow 3. How do things "just work"? Don't you have the same problem?

kenny19:08:40

Oh I see what you mean. There will be only one session store for all routes under /api. If there are other paths that require the session, you need to move the construction of the session middleware to contain both /api and the new route.

kenny19:08:59

Still no data validation message printed:

(http/ring-handler
  (http/router
    routes2
    {:data           {:interceptors [interceptors/session-interceptor]}
     :validate       reitit.ring.spec/validate
     ::rs/explain    s/explain-str})
  (ring/create-default-handler)
  {:executor sieppari/executor})

kenny19:08:32

The specs for the session-interceptor:

(s/def :session/store #(satisfies? session-store/SessionStore %))
(s/def :session/root string?)
(s/def :session/cookie-name string?)
(s/def :session/cookie-attrs map?)
(s/def ::session
  (s/keys :opt-un [:session/store
                   :session/root
                   :session/cookie-name
                   :session/cookie-attrs]))

(s/def ::session-interceptor
  (s/keys :opt-un [::session]))

ikitommi19:08:29

@kenny :

(ns user
  (:require [clojure.spec.alpha :as s]
            [reitit.http :as http]
            [reitit.ring :as ring]
            [reitit.http.spec :as spec]
            [reitit.interceptor.sieppari :as sieppari]
            [spec-tools.spell :as spell]
            [reitit.dev.pretty :as pretty]))

(s/def :session/store any?)
(s/def :session/root string?)
(s/def :session/cookie-name string?)
(s/def :session/cookie-attrs map?)
(s/def ::session
  (s/keys :opt-un [:session/store
                   :session/root
                   :session/cookie-name
                   :session/cookie-attrs]))

(s/def ::session-interceptor
  (s/keys :opt-un [::session]))

(def routes2
  [["/ping" {:session {:store "asd"}
             :get (fn [ctx]
                    (prn "ping")
                    {:status 200
                     :body "pong2"
                     :headers {}})}]])

(def session-interceptor
  {:name ::session
   :spec ::session-interceptor
   :compile (fn [{:keys [session]} _]
              (prn "here")
              (when session
                {:enter (fn session-interceptor-enter [ctx]
                          (println "enter")
                          ctx)
                 :leave (fn session-interceptor-leave [ctx]
                          (println "leave")
                          ctx
                          )}))})

(def app
  (http/ring-handler
    (http/router
      routes2
      {:data {:interceptors [session-interceptor]}
       :reitit.spec/wrap spell/closed
       :validate spec/validate})
    (ring/create-default-handler)
    {:executor sieppari/executor}))
; "here"
; "here"
; "here"
; "here"

(app {:request-method :get, :uri "/ping"})
; enter
; "ping"
; leave
; => {:status 200, :body "pong2", :headers {}}

ikitommi19:08:49

failing spec:

(def app
  (http/ring-handler
    (http/router
      ["/ping" {:session "kikka"}]
      {:data {:interceptors [session-interceptor]}
       :exception pretty/exception
       :reitit.spec/wrap spell/closed
       :validate spec/validate})
    (ring/create-default-handler)
    {:executor sieppari/executor}))

kenny19:08:06

What do you mean here?

ikitommi19:08:53

there is a spec error in :session, should fail with the given picture below (if the image was uploaded to slack)

kenny19:08:36

I don't follow. I pasted that example in the REPL and still didn't get that validation exception printed.

ikitommi19:08:31

did you paste the whole code above? it has all the imports & options in place. the latter is another test on top of that

kenny19:08:48

I created a new ns and pasted the user ns code into it.

ikitommi19:08:08

reitit 0.3.9?

kenny19:08:14

This is what I get:

Loading src/compute/http_api/example.clj... 
"here"
"here"
"here"
"here"
Loaded
(app {:request-method :get, :uri "/ping"})
enter
"ping"
leave
=> {:status 200, :body "pong2", :headers {}}

ikitommi19:08:27

oh, that’s correct, right?

ikitommi19:08:42

if you now paste the second one it, that should fail.

kenny19:08:01

Ok yes that happens. But why does only the second one fail?

ikitommi19:08:23

it has invalid :session route data

kenny19:08:34

The first one does too

kenny19:08:41

(def routes2
  [["/ping" {:session {:store "asd"}
             :get     (fn [ctx]
                        (prn "ping")
                        {:status  200
                         :body    "pong2"
                         :headers {}})}]])

kenny19:08:46

:session {:store "asd"}

ikitommi19:08:26

(s/valid? ::session-interceptor {:store "asd"})
; => true

kenny19:08:02

That spec must be wrong. Should be:

(s/def :session/store #(satisfies? session-store/SessionStore %))

kenny19:08:15

Yeah, it's

(s/def :session/store any?)

ikitommi19:08:19

oh, I didn’t have that class, changed the spec from your example 🙂

kenny19:08:11

If I change routes2 to match your example:

(def routes2
  [["/ping" {:session "asd"
             :get     (fn [ctx]
                        (prn "ping")
                        {:status  200
                         :body    "pong2"
                         :headers {}})}]])
it fails too. It appears nested maps aren't getting checked?

ikitommi19:08:08

but that should fail, right?

kenny19:08:25

This should fail:

(def routes2
  [["/ping" {:session {:store "asd"}
             :get     (fn [ctx]
                        (prn "ping")
                        {:status  200
                         :body    "pong2"
                         :headers {}})}]])

ikitommi19:08:19

with my data, that is ok, the other one isn’t:

(s/valid? ::session-interceptor {:store "asd"})
; => true

(s/valid? ::session-interceptor "asd")
; => false

ikitommi19:08:38

it uses the normal spec validation, so all levels are checked

ikitommi19:08:40

if you change the spec back to the original, both should fail. Spec registry might have a stale entry if you redefine the spec back to the original, you should reload the whole ns to see it change.

kenny19:08:27

Restarted the REPL and loaded that ns and got the invalid message. Must've been something like the stale spec thing. Still no messages getting printed in my actual ns though. There's gotta be one key somewhere misspelled or at the wrong level.

kenny19:08:28

Wow, ok. I used reitit.ring.spec/validate instead of reitit.http.spec/validate

ikitommi19:08:57

oh, they collect partial specs from different places. While reitit is no-defaults/all explicit/for people who have their own opinions, goal is to push a batteries-included “easy” stack built with reitit-http out at some point. Will have all the “right” settings on by default. https://github.com/metosin/talvi

kenny19:08:39

Is talvi just a place to list recommended tools? I don't see any code in the repo.

kenny21:08:00

@ikitommi Why does "here" get printed out 4 times?

kenny21:08:11

I would think the :compile function would only get called one time.

ikitommi09:08:56

@kenny not sure, reitit-http optionally auto-generates OPTIONS endpoint that might cause 2 invocations. 4 doesn’t sound good. You can inspect the expanded route-tree with: (->> app (reitit.http/get-router) (reitit.core/compiled-routes)).

kenny15:08:57

From the compile routes, it looks like there are only the two: one :get that I defined and one generated :options. Seems like it should get called 2 times, not 4.

ikitommi16:08:15

@kenny it should do it twice. Could you write an issue out of that so it get's checked & fixed?

ikitommi16:08:18

👍:skin-tone-2:

kenny21:08:27

After starting jetty and sending a request to my HTTP server, I get a 500 error and nothing logged. The response body is also empty. Any idea how to debug this? I would've expected the default to simply log errors to the console.

kenny21:08:41

Oh gosh, I see what happened. I had the min level for all non-project namespaces to ERROR. It appears errors are logged as WARN. This definitely needs to be configurable. WARN log level can be far too noisy.