Fork me on GitHub
#meander
<
2021-09-10
>
ribelo00:09:36

(m/rewrites {:x {:y [:a :b :c]
                         :q [:a :b :c]}}
          {!xs {!ys (m/scan :b)}}
          [[!xs !ys] ...])
Is there any way to do this without using rewriteS but with just simple rewrite?

noprompt16:09:31

I think. Lemme try…

Richie01:09:30

Why does it work with r1 and not with r2?

Richie01:09:43

Sorry, I'm also having trouble creating a more minimal example. I feel in over my head.

ribelo01:09:42

There are a multiple keys in map r2 :active and scan can't handle it

ribelo01:09:36

Unfortunately I'm writing from my phone so I can't help you right now

Richie01:09:53

No trouble. That's already very helpful. Thank you!

ribelo07:09:26

(let [r1 [[0 {:active {:a {:ids [{:business "123"}]}}}]
          [1 {:active {:d {:ids [{:business "567"}]}}}]]
      r2 [[0 {:active {:a {:ids [{:business "123"}]}
                       :b {:ids [{:business "234"}]}}}]
          [1 {:active {:d {:ids [{:business "567"}]}
                       :c {:ids [{:business "678"}]}}}]]
      f (fn [arg]
          (m/rewrite arg
            [[(m/and !xs !ys) {:active (m/map-of _ {:ids [{:business !ids}]})}] ...]
            [[[!xs !ids] [!ys !ids] ...]]))]
  (f r1)                                ;; [[0 "123"] [1 "567"]]
  (f r2)                                ;; [[0 "123"] [0 "234"] [1 "567"] [1 "678"]]
  )

ribelo07:09:06

Surely it can be done more simply

Richie18:09:40

Hey. I figured some things out. First I got the answer I wanted from this smaller example:

(let [r2 [{:a [{:word "123"}
               {:word "abc"}]
           :b [{:word "234"}
               {:word "bcd"}]}]
      f (fn [arg]
          (m/find
            arg
            (m/gather (m/map-of _ (m/gather {:word !a})))
            !a))]
  (f r2))
  ;; => ["123" "abc" "234" "bcd"]
But it didn’t seem like I was doing this correctly since I think I should be able to search with a logic var instead of using a memory var like that. The example for scan gave me the idea to treat a map as a seq of MapEntry.
(m/search {:x 1 :y 2 :z 3}
  (m/scan [?a ?b])
  {?b ?a})
;; => ({1 :x} {2 :y} {3 :z})
So then I got this working code:
(let [r2 [{:a [{:word "123"}
               {:word "abc"}]
           :b [{:word "234"}
               {:word "bcd"}]}]
      f (fn [arg]
          (m/search
            arg
            (m/scan (m/scan [_ (m/scan {:word ?a})]))
            ?a))]
  (f r2))

  ;; => ("123" "abc" "234" "bcd")
Going back to an example of my actual problem:
(let [r2 [[0 {:active {:a {:ids [{:business "123"}
                                 {:business "aha"}]}
                       :b {:ids [{:business "234"}]}}}]
          [1 {:active {:d {:ids [{:business "567"}]}
                       :c {:ids [{:business "678"}]}}}]]
      f (fn [arg]
          (m/search
            arg
            (m/scan [?t {:active (m/scan [_ {:ids [{:business !a} ...]}])}])
            [?t !a]))]

  (f r2))
;; => ([0 ["123" "aha"]] [0 ["234"]] [1 ["567"]] [1 ["678"]])
I actually want to get a result like ([0 [“123” “aha” “234”]] [1 [“567" “678”]]) but I haven’t figured it out yet. Thank you all of you for making such a sick library and for the great support.

🎉 2
Richie18:09:07

@U0BBFDED7 Thanks for the working example. I didn’t know about map-of until I saw you use it. After that, I cloned the repo and poked around reading through some of the docstrings. Ultimately, I don’t think I want to use and since I don’t know how many things I’ll have there; I could have made a better question than to just ask why it was broken, haha. Thanks again.

Ben Sless09:09:19

Wanted to share something cool and ask if you think I made any mistakes or it could be improved - beta reduction of tools.analyzer ast node:

(m/rewrite
    node
    {:op :invoke
     :args [!args ..?n]
     :fn {:op :fn
          :methods
          [{:fixed-arity ?n
            :variadic? false
            :form _ :arglist _ :tag _ :o-tag _ ;; rudely ignore
            :params
            [{:local _ :arg-id _
              & !params} ...]
            :body ?body}
           ..1]} ;; there can be only one!
     & ?rest}
    {:op :let
     :children [:bindings :body]
     :bindings
     [{:init !args
       :local :let
       :children [:init]
       & !params} ...]
     :body ?body})

ribelo10:09:10

I don't know what it does but it looks cool

ribelo10:09:44

_ can be omitted if they are not for readability

ribelo10:09:18

whereas, if these are the required things that should be there, then it's a good idea to wrap in (m/some)

Ben Sless11:09:05

It's a poor man's dissoc but you're right, I should rewrite it

Ben Sless11:09:38

what it does: ((fn [x] B) E) -> (let [x E] B)

noprompt16:09:36

Awesome! This is definitely one of the use cases I was designing for. 🙂

Ben Sless13:09:29

Is it possible to perform a recursive search and substitution inside a bound variable?

Ben Sless13:09:39

cata still assumes I know the shape of the input (sort of). In the case where I don't want to specify all the possible patterns, is there a generic walk?

ribelo13:09:20

Try strategy namespace

ribelo13:09:23

Probably bottom-up is what you're looking for

Ben Sless14:09:31

I'd still have to specify all the patterns I want to search over 😞

noprompt16:09:55

Ben, are you able to layout in more detail what your idea/goal is? I’m not sure I totally follow.

Ben Sless16:09:35

Sure, I'm trying to implement optimization passes on tools.analyzer output. Beta reduction, copy and constant propagation

noprompt16:09:17

Cool. I follow this. I was looking for an example/sketch (could be contrived) of the problem and what you would like the solution to look like/the result, etc.

noprompt16:09:58

We do need a more general walking mechanism that isn’t strategy based. I have some ideas about that but none that I can invest time in right now.

noprompt16:09:11

One of the things that I’ve done in the past is write rules like this

(m/rewrite form
  [::special def ?tail]
  {:type :def  ,,,}

  ((m/pred special-symbol? ?symbol) & tail)
  (m/cata [::special ?symbol ?tail]))
essentially emulating a function/function call inside the system.

Ben Sless16:09:43

You could write a HOF which returns a walker

Ben Sless17:09:11

Bottom up and top down are a special case of post/pre walk derived from it

noprompt17:09:25

I wanted to mention earlier, Github discussions are available too if you want to jot down something a bit more long form that lays out the full scope of the problem.

Ben Sless18:09:10

Probably will, thank you 🙂

Richie18:09:12

Oh nice. I was upset that slack limits the conversation history since I’d like to dig through the history for more examples.

noprompt19:09:57

Please post there. And, if you do, share the link here as well.

noprompt19:09:45

Honestly, I should probably do that too.

Ben Sless19:09:41

You could probably make half a cookbook by scraping the history of this channel on the log. zulip?

Ben Sless07:09:20

OPened a discussion for both

Richie18:09:40

Hey. I figured some things out. First I got the answer I wanted from this smaller example:

(let [r2 [{:a [{:word "123"}
               {:word "abc"}]
           :b [{:word "234"}
               {:word "bcd"}]}]
      f (fn [arg]
          (m/find
            arg
            (m/gather (m/map-of _ (m/gather {:word !a})))
            !a))]
  (f r2))
  ;; => ["123" "abc" "234" "bcd"]
But it didn’t seem like I was doing this correctly since I think I should be able to search with a logic var instead of using a memory var like that. The example for scan gave me the idea to treat a map as a seq of MapEntry.
(m/search {:x 1 :y 2 :z 3}
  (m/scan [?a ?b])
  {?b ?a})
;; => ({1 :x} {2 :y} {3 :z})
So then I got this working code:
(let [r2 [{:a [{:word "123"}
               {:word "abc"}]
           :b [{:word "234"}
               {:word "bcd"}]}]
      f (fn [arg]
          (m/search
            arg
            (m/scan (m/scan [_ (m/scan {:word ?a})]))
            ?a))]
  (f r2))

  ;; => ("123" "abc" "234" "bcd")
Going back to an example of my actual problem:
(let [r2 [[0 {:active {:a {:ids [{:business "123"}
                                 {:business "aha"}]}
                       :b {:ids [{:business "234"}]}}}]
          [1 {:active {:d {:ids [{:business "567"}]}
                       :c {:ids [{:business "678"}]}}}]]
      f (fn [arg]
          (m/search
            arg
            (m/scan [?t {:active (m/scan [_ {:ids [{:business !a} ...]}])}])
            [?t !a]))]

  (f r2))
;; => ([0 ["123" "aha"]] [0 ["234"]] [1 ["567"]] [1 ["678"]])
I actually want to get a result like ([0 [“123” “aha” “234”]] [1 [“567" “678”]]) but I haven’t figured it out yet. Thank you all of you for making such a sick library and for the great support.

🎉 2
noprompt19:09:34

> Hey! I hope you’re having a great day. This sentence. 😄

noprompt19:09:24

Also the title. High marks. 100%

Richie20:09:04

Haha, thanks. I was feeling punchy after punching away at the same problem all day.

Ben Sless20:09:53

New meaning of "using X in anger"?

2
noprompt20:09:26

Ah, yeah, that can be frustrating. And I know there is a learning curve. Buuuut, hopefully, we can help. 🙂

Ben Sless20:09:06

One day I'll understand m/cata then I'll be able to just communicate telepathically or something

Richie20:09:11

It seems like the learning curve is steeper than most new things that I encounter while programming but the payoff also seems much greater than the usual new thing. So much so that the ratio of cost to benefit weighs heavily in favor of learning and using it. It’s existence and this community are also gifts so I can’t begin to be critical. I’m not actually angry; I’m just playing it up for the humor. 🙂

noprompt20:09:47

> the learning curve is steeper … the ratio of cost to benefit weighs heavily in favor of learning and using it This is a perspective I think others in the channel share. Improving the overall experience and reducing the learning curve/providing help are priorities. For me, the criticism is inspiring actually, I’ve just had a rough time the past year dealing with burn out from software, and $pandemic_related_reasons .

noprompt20:09:45

But I will say, this channel is awesome. I’m really proud of the folks in here. People are helpful and courteous. “No stupid questions” kind of attitude. 🙂

noprompt20:09:57

This channel has kept me going!

noprompt20:09:18

Also, I will try and fill in some explaining on my replies to your question.

ribelo19:09:20

regarding the learning curve, I don't think there is anything that can be done about it

ribelo19:09:37

meander is simple but not easy

ribelo19:09:02

some concepts like cata or with just have to pop in our head

Ben Sless20:09:37

Is there anything I can read which would help me understand cata? Sometimes I find original papers on subjects make it click for me. Did so for HAMTs and CSP

ribelo20:09:58

cata is simply recursion

ribelo20:09:54

of the non-obvious things that came out of using cata was discovering that you can put the processed data into a vector, where the first element is whatever tells you what to do with it

Ben Sless20:09:32

I understand all of these words, but have no way to translate them to actionable knowledge yet. Maybe 11pm is not the time

ribelo20:09:21

I guess it just requires some practice

ribelo20:09:38

I would start by looking at all the examples that are in the meander repo. Reading through all the tests also gave me a lot.

Ben Sless20:09:31

I read the cata tests several times. The trivial cases made sense but didn't click for me yet. No worries, I shall bang my head against that wall until it breaks

ribelo20:09:13

I've been through it, and after some head-banging, the tinnitus even stops

😆 2
ribelo20:09:20

in general I appreciate your work, e.g. on malli, a lot can be learned by reading the code

ribelo20:09:39

I think you have a perversion to numbers and like to speed up other people's code 🙃

Ben Sless20:09:22

And thank you 😃

ribelo20:09:22

you'll probably find something to do once you get into the meander code

Ben Sless20:09:42

Besides, I want to use meander to speed up other code. All the code, if possible 🙂

ribelo20:09:13

well, that's what I'm currently trying to do

ribelo20:09:29

and it's addictive

Ben Sless20:09:33

Speed up meander or use meander to write a compiler?

Ben Sless20:09:00

> and it's addictive Absolutely

Ben Sless20:09:07

numbers go woosh

ribelo20:09:24

dopamine over 9000!!!

Ben Sless20:09:52

have you applied meander to tools.analyzer ast?

ribelo20:09:04

code in meander can be fully declarative, imho it is often much more readable than pure clojure, moreover it is easy to come back to, because, omits some concepts like cata, it is clear and understandable

ribelo20:09:04

for now, I'm learning meander mostly by writing doxa and poking around in meander itself

Ben Sless20:09:47

it seems like a very good fit. There are some Haskell papers which are relevant

ribelo20:09:29

wait, because I don't think I quite caught on before

ribelo20:09:04

You literally want to use meander to rewrite other people's code

Ben Sless20:09:44

An alternative compiler

ribelo20:09:17

crazy idea

Ben Sless20:09:18

and the video linked in the discussion

ribelo20:09:20

it's a bit beyond my homegrown programming knowledge, but I'm happy to watch

Ben Sless20:09:52

I did not study computer science 🤷

Ben Sless20:09:12

If you can get meander, you can get Core

Ben Sless20:09:34

Besides Big Lambda type constructors don't ask me about those types are hard

ribelo20:09:12

haskel sounds like something you need a PhD to understand

Ben Sless20:09:55

Yes, which is why I'm not trying to understand Haskell, but the intermediate language and how they transform it for optimizations

👍 2
ribelo20:09:06

Meanwhile, I've got to go. Thanks for the chat.

Ben Sless20:09:28

sleep time for me 🌃

noprompt16:09:06

Core is cool. 😎

Ben Sless16:09:41

I wish I could take a sabbatical to work on this