meander

wilkerlucio 2021-09-21T20:07:58.084800Z

hello, trying to figure how to parte pedestal-like nested URL format using meander, for example, go from

["v1"
 ["/foo"
  ["/baz" {:post baz-handler}]]
 ["/bar"
  {:get  bar-handler
   :post post-bar}]]
to
[{:url "v1/foo/baz" :method :post :handler baz-handler}
 {:url "v1/bar" :method :get :handler bar-handler}
 {:url "v1/bar" :method :post :handler post-bar}]

wilkerlucio 2021-09-21T20:08:12.084900Z

I guess I gotta use cata, but not sure how

2021-09-21T20:31:58.085200Z

(m/rewrite '["v1"
             ["/foo"
              ["/baz" {:post baz-handler}]]
             ["/bar" {:get bar-handler :post post-bar}]]
  (m/with [%1 (m/map-of !methods !handlers)
           %2 [!paths . (m/or %2 %1) ...]]
          %2)
  [{:url !paths :method !methods :handler !handlers} ...])

2021-09-21T20:32:28.085400Z

often with is simpler than cata

2021-09-21T20:32:53.085600Z

but now I see that it is not quite so, because the paths have been concatet

wilkerlucio 2021-09-21T20:32:56.085800Z

cool, that almost solves, the missing part is accumulating the previous routes

wilkerlucio 2021-09-21T20:33:00.086Z

yeah

2021-09-21T20:35:09.086200Z

honestly, I don't know how to do this

2021-09-21T20:35:30.086400Z

but it'll keep me busy tonight xD

wilkerlucio 2021-09-21T20:37:07.086600Z

haha, would love to hear, Iโ€™m trying to figure on my end as well, if I figure Iโ€™ll post here

markaddleman 2021-09-21T20:38:59.086800Z

Give this a try

markaddleman 2021-09-21T20:40:02.087200Z

The first rewrite does the data re-organization that you're looking for but suffers from nested vectors (this is often a result of using cata).

markaddleman 2021-09-21T20:40:15.087400Z

The second rewrite "flattens" the structure

wilkerlucio 2021-09-21T20:41:02.087600Z

cool, I guess the last rewrite can be replaced with flatten, got the same result here

wilkerlucio 2021-09-21T20:41:10.087800Z

thanks @markaddleman!

๐Ÿ‘ 1
wilkerlucio 2021-09-21T20:42:42.088100Z

not sure if its a fixable issue, but when I tried with a real longer structure, it did stack overflow ๐Ÿ˜…

๐Ÿคฃ 1
markaddleman 2021-09-21T20:42:42.088300Z

It's funny - I'm so used to using meander that I reached for meander to flatten the data without even thinking of flatten ๐Ÿ™‚

markaddleman 2021-09-21T20:43:26.088600Z

cata does stack-based recursion so a sufficiently deep structure could result in an overflow, I guess

markaddleman 2021-09-21T20:43:39.088800Z

Let me see if I can rewrite it to avoid some of the recursion

wilkerlucio 2021-09-21T20:43:52.089Z

I donโ€™t think its that deep, from a quick look I guess its less than 10 depth

markaddleman 2021-09-21T20:44:02.089200Z

Can you post the structure?

2021-09-21T20:44:12.089400Z

stack overlow occurs when cata loops indefinitely

markaddleman 2021-09-21T20:44:14.089600Z

I assume you're getting the SO from the first rewrite?

wilkerlucio 2021-09-21T20:44:22.089800Z

sadly no, but there are a few other patterns that needs to be taken in account

wilkerlucio 2021-09-21T20:44:29.090Z

and I guess those are screwing it up

markaddleman 2021-09-21T20:44:56.090200Z

Unfortunately, I'm not very familiar with these routing data structures

2021-09-21T20:45:06.090400Z

(m/rewrite 1
  (m/app inc ?x)
  (m/cata ?x))

2021-09-21T20:45:46.090600Z

I recommend using cata with instructions

wilkerlucio 2021-09-21T20:45:51.090800Z

I can make a shorter version that has more of those features, let me check

markaddleman 2021-09-21T20:46:00.091Z

cool

2021-09-21T20:46:05.091200Z

is more readable and harder to error with an infinite loop

2021-09-21T20:48:28.091400Z

(m/rewrite [:start 1]
  [:start (m/app inc ?x)]
  (m/cata [:end ?x])
  [:end ?x]
  ~(str "end with ?x = " ?x))

2021-09-21T20:49:27.091600Z

you can easily set checkpoints and also check which instruction will be executed next

wilkerlucio 2021-09-21T20:52:02.091800Z

found a simple stack overflow example:

wilkerlucio 2021-09-21T20:52:03.092Z

'["v1"
         ["/foo"
          ["/baz" {:post baz-handler}]
          ["/other" []]]
         ["/bar"
          {:get  bar-handler
           :post post-bar}]]

wilkerlucio 2021-09-21T20:52:14.092200Z

sometimes instead of a map, its a vector with interceptors on the right side

markaddleman 2021-09-21T20:52:31.092400Z

ah, yeah. I can see how this would overflow

markaddleman 2021-09-21T20:52:41.092600Z

I'll take a crack at this data structure

wilkerlucio 2021-09-21T20:53:19.092800Z

feels a little ambiguous case, curious to see what you come with for that ๐Ÿ˜‰

markaddleman 2021-09-21T20:53:26.093Z

ha

markaddleman 2021-09-21T20:53:28.093200Z

you read my mind

markaddleman 2021-09-21T20:53:46.093400Z

unctions? fI'm guessing the vector of interceptors is always a vector of

markaddleman 2021-09-21T20:53:48.093600Z

functions?

markaddleman 2021-09-21T20:54:05.093800Z

What does the result for this look like?

wilkerlucio 2021-09-21T20:55:04.094Z

the real one looks more like this:

wilkerlucio 2021-09-21T20:55:06.094200Z

'["v1"
        ["/foo"
         ["/baz" {:post baz-handler}]
         ["/other" [[:interceptor
                     [{:features    {:policy-authz-enabled? false},
                       :interceptor [:scope
                                     {:scope "xxx"}]}
                      {:features    {:policy-authz-enabled? true},
                       :interceptor [:scope
                                     [{:policy "yyy"}]]}]]]]]
        ["/bar"
         {:get  bar-handler
          :post post-bar}]]

wilkerlucio 2021-09-21T20:55:29.094400Z

but Iโ€™m not sure all possible variations (like if they always start with double vector)

markaddleman 2021-09-21T20:56:02.094600Z

I've got a meeting in 5 minutes but I'll pick this up after

wilkerlucio 2021-09-21T20:56:24.094800Z

thanks, Iโ€™ll keep playing here too

wilkerlucio 2021-09-21T20:56:39.095Z

seems like a notable difference is that those never start with strings at the beginning of the vector

wilkerlucio 2021-09-21T20:56:42.095200Z

only routes do that

2021-09-21T20:58:20.095400Z

(m/rewrite '["v1"
             ["/foo"
              ["/baz" {:post baz-handler}]]
             ["/bar" {:get bar-handler :post post-bar}]]
  (m/with [%1 (m/map-of !methods !handlers)
           %2 [(m/app #(str (last !paths) %) !paths) . (m/or %2 %1) ...]]
    %2)
  [{:url !paths :method !methods :handler !handlers} ...])

2021-09-21T20:59:36.095600Z

I can't make it any shorter @wilkerlucio

2021-09-21T20:59:40.095800Z

๐Ÿ˜‰

wilkerlucio 2021-09-21T21:00:17.096Z

looks nice!

wilkerlucio 2021-09-21T21:00:44.096200Z

on the large example it didnโ€™t match though =/

wilkerlucio 2021-09-21T21:00:49.096400Z

but Iโ€™ll play with it

wilkerlucio 2021-09-21T21:01:09.096600Z

it probably because of the interceptors in the middle

2021-09-21T21:01:17.096800Z

what should be returned from the longer one?

2021-09-21T21:01:55.097Z

\other should be omitted?

wilkerlucio 2021-09-21T21:02:24.097300Z

omitting other is good

wilkerlucio 2021-09-21T21:03:04.097500Z

the longer one currently returns nil, so something is going bad in the middle, Iโ€™m trying to get a case to reproduce with your solution

2021-09-21T21:03:26.097700Z

(m/rewrite '["v1"
             ["/foo"
              ["/baz" {:post baz-handler}]
              ["/other" [[:interceptor
                          [{:features    {:policy-authz-enabled? false} ,
                            :interceptor [:scope
                                          {:scope "xxx"}]}
                           {:features    {:policy-authz-enabled? true} ,
                            :interceptor [:scope
                                          [{:policy "yyy"}]]}]]]]]
             ["/bar"
              {:get bar-handler
               :post post-bar}]]
  (m/with [%1 (m/map-of !methods !handlers)
           %2 [(m/pred string?) _]
           %3 [(m/app #(str (last !paths) %) !paths) . (m/or %3 %2 %1) ...]]
    %3)
  [{:url !paths :method !methods :handler !handlers} ...])

2021-09-21T21:05:02.097900Z

\other catches under %2, which is defined in the simplest possible way, [string? whatever]

wilkerlucio 2021-09-21T21:05:45.098300Z

here is an example thatโ€™s going off, but not the main problem yet:

wilkerlucio 2021-09-21T21:05:47.098500Z

(m/rewrite '["v1"
               ["/foo"
                [:some-interceptors]
                ["/baz" {:post baz-handler}]
                ["/other" [[:interceptor
                            [{:features    {:policy-authz-enabled? false},
                              :interceptor [:scope
                                            {:scope "xxx"}]}
                             {:features    {:policy-authz-enabled? true},
                              :interceptor [:scope
                                            [{:policy "yyy"}]]}]]]]]
               ["/bar" {:get bar-handler :post post-bar}]]
    (m/with [%1 (m/map-of !methods !handlers)
             %2 [(m/pred string?) _]
             %3 [(m/app #(str (last !paths) %) !paths) . (m/or %3 %2 %1) ...]]
      %3)
    [{:url !paths :method !methods :handler !handlers} ...])

2021-09-21T21:06:11.098700Z

probably with a more complicated example it will fall in a different place

wilkerlucio 2021-09-21T21:06:22.098900Z

(in this case the :some-interceptors should be ignored)

wilkerlucio 2021-09-21T21:07:38.099100Z

anyway you guys are meander masters, its already really helpful ๐Ÿ™‡

2021-09-21T21:09:23.099300Z

not so much I see that it is probably wrong, because it does not return a map with the path :url foo/bar ๐Ÿ˜•

2021-09-21T21:09:39.099500Z

but I'm glad I could help a little.

2021-09-21T21:11:20.099700Z

It would be nice if @noprompt offered a solution

2021-09-21T21:12:02.099900Z

I have no idea how to better concatenate urls into one, and what I've done doesn't always work

wilkerlucio 2021-09-21T21:20:05.100100Z

the examples you posted are nice as well, Iโ€™ll spend some time to understand and learn those patterns ๐Ÿ™‚

markaddleman 2021-09-21T21:44:37.100300Z

Thanks for showing with @huxley ! I was aware of it but I don't think I've ever used it before

2021-09-21T21:45:37.100500Z

โ˜บ๏ธ

wilkerlucio 2021-09-22T02:39:18.100700Z

this worked!

(m/rewrite
    (-> data :ig/system :pipo.module/web-routes :routes ffirst)
    
    (m/with [%route-map (m/map-of !methods !handlers)
             %item [(m/pred string?) _]
             %interceptors [(m/pred keyword?) . _ ...]
             %all [(m/app #(str (last !paths) %) !paths) . (m/or %all %item %interceptors %route-map) ...]]
      %all)
    [{:url !paths :method !methods :handler !handlers} ...])

wilkerlucio 2021-09-22T02:39:42.100900Z

thank you again folks!

wilkerlucio 2021-09-22T02:51:11.101100Z

I celebrated too soon, @huxley I found this way to concatenate is not doing the stack as expected, on the base example it outputs:

[{:url "v1", :method :post, :handler baz-handler}
 {:url "v1/foo", :method :get, :handler bar-handler}
 {:url "v1/foo/baz", :method :post, :handler post-bar}]

wilkerlucio 2021-09-22T02:51:34.101300Z

but it should be

({:url "v1/foo/baz", :method :post, :handler baz-handler}
 {:url "v1/bar", :method :get, :handler bar-handler}
 {:url "v1/bar", :method :post, :handler post-bar})
(which is the result given frmo the @markaddleman solution)

wilkerlucio 2021-09-22T02:52:55.101500Z

now trying to work more on the mark example to add the interceptors part

markaddleman 2021-09-22T02:53:39.101700Z

I thought you guys already had a solution so I didn't pick up the interceptor piece. I'll give it a look tomorrow morning

wilkerlucio 2021-09-22T02:57:27.101900Z

thanks, this is the example Iโ€™m working on top of now:

wilkerlucio 2021-09-22T02:57:29.102100Z

'["v1"
        ["/foo"
         [:some-interceptors]
         ["/baz" {:post baz-handler}]]
        ["/bar" {:get bar-handler :post post-bar}]]

wilkerlucio 2021-09-22T02:57:47.102300Z

you can forget the value as vectors, that wasnโ€™t a thing, just the interceptors in the middle ๐Ÿ™‚

wilkerlucio 2021-09-22T04:02:55.102600Z

got it ๐Ÿ˜„ ๐ŸŽ‰

(-> '["v1"
        ["/foo"
         [:bla :me []]
         ["/baz" {:post baz-handler}]]
        ["/bar" {:get bar-handler :post post-bar}]]
      (m/rewrite
        [?path (m/map-of !action !handler)]
        [{:url ?path :method !action :handler !handler} ...]

        [?path-prefix [(m/pred keyword?) & _] & ?rest]
        (m/cata [?path-prefix & ?rest])

        [?path-prefix [?path-suffix & ?deeper-or-handlers]]
        (m/cata [(m/app str ?path-prefix ?path-suffix) & ?deeper-or-handlers])

        [?path-prefix . !deeper ...]
        [(m/cata [?path-prefix !deeper]) ...])
      flatten)
that was fun to learn! thanks folks

๐ŸŽ‰ 1
noprompt 2021-09-23T16:42:28.109400Z

Sorry I missed this conversation. I was out with a few doctor appointments for myself and my son. ๐Ÿ˜