Fork me on GitHub
#meander
<
2020-03-18
>
markaddleman02:03:37

I'm working through the meander examples at https://cljdoc.org/d/meander/epsilon/0.0.402/doc/understand-meander-s-pattern-matching-macros#rewrite I'd like to change the example a bit to return something like clojure.core's group-by. For example, I have a collection

[{:name    "entity1"
               :vals  [{:value 1} {:value 2}]}
              {:name "entity1"
               :vals [{:value 3} {:value 4} {:value 5}]}]
I'd like a meander expression to return
[{:name "entity1" :vals [1 2 3 4 5]}
The following works:
(m/rewrite [{:name    "entity1"
               :vals  [{:value 1} {:value 2}]}
              {:name "entity1"
               :vals [{:value 3} {:value 4} {:value 5}]}]
             [{:name    !name
               :vals [{:value !values} ...]} ...]
             [{:name   !name
               :value  [!values ...]} ...])
But fails to extend to multiple entity names (eg "entity2" with some vals). What's the secret sauce that I'm missing?

noprompt03:03:42

@mark340 at the moment we typically recommend group-by 🙂

noprompt03:03:48

We are working on something which will help with reduction kinds of problems like group-by but in a robust and general way.

markaddleman04:03:24

You know, I think you told me that before but I forgot. It seems such a natural thing to do in meander. Looking forward to zeta 🙂

noprompt04:03:52

Thanks. I want things to go more quickly but life has been more demanding than usual. 😅

😛 4
markaddleman04:03:23

For everyone. Stay safe; stay healthy!

noprompt03:03:51

You can do reductions with rewrite though, depending on the task, it might not be what you want.

noprompt03:03:37

(let [es [{:name "entity1"
           :vals  [{:value 1} {:value 2}]}
          {:name"entity1"
           :vals [{:value 3} {:value 4} {:value 5}]}]]
  (me/rewrite [{} es]
    [?state []]
    ?state

    [{?name [!values ...] & ?state} [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest])

    [?state [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest]))
  ;; Semantically equivalent too
  (reduce
   (fn [state e]
     (me/rewrite [state e]
       [{?name [!values ...] & ?state} {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}

       [?state {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}))
   {}
   es))
;; => (let [es [{:name "entity1"
           :vals  [{:value 1} {:value 2}]}
          {:name"entity1"
           :vals [{:value 3} {:value 4} {:value 5}]}]]
  (me/rewrite [{} es]
    [?state []]
    ?state

    [{?name [!values ...] & ?state} [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest])

    [?state [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest]))
  ;; Semantically equivalent too
  (reduce
   (fn [state e]
     (me/rewrite [state e]
       [{?name [!values ...] & ?state} {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}

       [?state {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}))
   {}
   es))
;; =>
{"entity1" [1 2 3 4 5]}

noprompt03:03:29

This, for example, is gross.

noprompt05:03:24

☝️ I’ll leave this open for the next day or so in case anyone wants to say something.

magnusdk10:03:11

Hey, I’ve started using Meander more and more and it’s great! I have started getting some weird compiler exceptions however when running tests using lein kaocha --watch that occurs when reloading namespaces containing Meander code. /t

magnusdk10:03:16

The exception is clojure.lang.Compiler$CompilerException: Syntax error macroexpanding m/rewrites at (...).

Caused by: clojure.lang.ExceptionInfo: Call to #'meander.syntax.epsilon/resolve-expander did not conform to spec:
epsilon.cljc:308
-- Spec failed --------------------
Return value
  meander.epsilon/eval13971/expander--auto--
should satisfy
  (fn
   [%]
   (or (nil? %) (sequential? %)))
-------------------------
Detected 1 error
{:clojure.spec.alpha/problems [{:path [:ret], :pred (clojure.core/fn [%] (clojure.core/or (clojure.core/nil? %) (clojure.core/sequential? %))), :val #object[meander.epsilon$eval13971$expander__9593__auto____13972 0x7abb82d6 "[email protected]"], :via [], :in []}], :clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x1a0fd147 "[email protected]"], :clojure.spec.alpha/value #object[meander.epsilon$eval13971$expander__9593__auto____13972 0x7abb82d6 "[email protected]"], :clojure.spec.alpha/ret #object[meander.epsilon$eval13971$expander__9593__auto____13972 0x7abb82d6 "[email protected]"], :clojure.spec.alpha/failure :instrument, :orchestra.spec.test/caller {:file "epsilon.cljc", :line 308, :var-scope meander.syntax.epsilon/expand-form}}
Has anyone else experienced a similar problem?

magnusdk13:03:36

This is a minimal (I think) case where CompilerException is thrown

(meander/match nil
  (meander/seqable)
  nil)
It might have something to do with use of defsyntax as scan and separated causes the same issue

jimmy16:03:46

I can definitely look into this. I'm guessing these tests are instrumenting all specs?

jimmy17:03:20

I just tried recreating this. I instrumented every thing and tried running the code above and it ran with no issues. Do you have a project that is open with the issue? Can you share your deps? Any information for recreating would be great.

magnusdk19:03:24

Thank you for checking it out :) Yes, the tests are instrumented using https://github.com/jeaye/orchestra. Sorry for leaving out details, I’ll try my best to recreate it in a fresh, open project. I’ll get back to you

jimmy19:03:28

Orchestra would explain the difference.

jimmy19:03:11

Hopefully have some time today to try that out and track down the issue.

magnusdk19:03:25

I’ve recreated the issue in a new project now. I can share it on github

jimmy19:03:27

I was able to do it with orchestra. Thanks, that was the missing piece

magnusdk19:03:12

Awesome! I also uploaded my project here incase it’s still relevant https://github.com/magnusdk/meander-orchestra-kaocha-issue

jimmy20:03:08

Found the issue. Should have a fix today. There is nothing actually broken going on just a bad spec.

🏎️ 4
magnusdk20:03:16

That is fantastic! parrot Thank you for your time 🙂 This is great

noprompt21:03:37

Just FYI, we’re probably not going to be using spec going forward. Its been a pain for a number of users and myself.

👍 4
jimmy04:03:39

Pushed a fix, will cut a release tomorrow.

jimmy20:03:07

This has now been fixed in “0.0.408”

jimmy20:03:46

Let us know if you run into any other issues using orchestra.

magnusdk22:03:06

Thank you 😊

markaddleman15:03:14

What are your thoughts around pluggable optimization strategies? I have a set of documents that, right now, are stored in plain Clojure sequences and yields plenty good enough performance. I have a somewhat complex meander pattern that encodes the business logic to find the right data within those documents. Over time, however, I expect the set of documents to grow perhaps to the point where sequence performance is not good enough. Much like adding an index in an RDBMS, I'd love to swap out my sequences of data for something else but not change the declarative search strategy encoding in Meander. In practice, the next performance step would be storing the data in Datascript. I can almost imagine providing Meander some hints to access the data using index seqs. Thoughts?

jimmy17:03:12

We don't have any plans right now for going that route. I do think having indexes and looking up by thing is a good thing to do if you have large datasets and are looking for performance. But I'd personally recommend just making those indexes and passing them to your match. In my view, that would be the same as giving us a hint. Just give us the actual index and make your pattern match on that index.

👍 4