Clojurians Log v2

Something I've been struggling to think through: a) Meander is a term rewriting library and that's very good. b) It is difficult to differentiate between a "greedy" pattern consumption, and an "all possible matches" comprehension [more accurately the former is not supported, you can only really do the latter, but this limitation is probably syntactic in the sense of how would you request greediness]. c) Instaparse has a similar tension between "greedy" and "possible interpretations of a grammar applied to an input. The way Instaparse resolves this is by having the grammar be not greedy, but including regex matchers which are greedy. This feels very natural in Instaparse because you often want to regex match on tokens. d) Instaparse benefits greatly from being able to greedily match identifiers and then fold the result into an AST. e) Can a similar strategy be used in Meander? Well the limitation here is that we don't have regex for data structures (Instaparse can use regexes for strings because that's it's fundamental building block). Or do we? I'm tempted to say that someone has probably tackled regex for data structures... certainly Clojure.spec advertises itself as regex for data structures... maybe Meader is kind of a regex for data structures already and I'm missing the point. f) Instaparse is great on strings but I don't see how to use it on data structures, Meander is great on data structures and not so much on strings which is fine they serve different purposes, but in both situations both the ability to rewrite, and the ability to create an AST are useful. Meander is excellent at AST manipulation, but I claim that it misses the capability to go from data structure to AST. Similarly Instaparse is missing the capability to go from data structures to AST. How would one define a grammar for data? Instaparse -> AST -> rewrite with Meander works really well, but could there also be a Data -> AST -> rewrite Motivation: Finding it hard to explain why I think this could be useful hahahaha g) What would it look like to provide a grammar for data? Well it's just adding labels to Meander patterns:

S = AB*
AB = A | B
A = #data[1 & _]
B = #data(2 & _)

For epsilon to have greediness all it would require is adding a committed choice operator. On the zeta branch, this is called pick. Greediness ends up looking like:

(m/with {%head <some-pattern>
         %tail ?tail
         %args (m/cons %head (m/pick %args %tail))}
  %args)

This will cause ?tail to always be empty. However, if you replace m/pick with m/some (`m/or` on epsilon) you get the behavior of ….

I haven’t been talking much about zeta because I have been working very hard on getting an initial working release of it ready hopefully by the end of this month.

It may not be be as fast as epsilon out of the box, however, it will be (is) far more flexible and powerful.

@timothypratley On that grammar thing, I think you will like the zeta approach to language extension. 🙂

zeta not only allows you to define your own operators like epsilon does but it also allows you to define your own notation (like & ?rest) and use it in a hygienic way.

And, actually, stuff like & and :as patterns are defined as notation rather than being a part of the core language.

And so is _, ?<name> , notation. 😉

Apart from the primitive parts, everything thing is done by creating new operators or new notation.

So if you wanted, say, a grammar notation like what you have above, you could define it.

zeta also has a rough draft of explanations too

(defnotation
  ^{:doc "Convert symbols that start with \"_\"? into the form (anything)."}
  anything-symbol
  (rule
   (symbol (anything) (str "_" (anything)))
   (`anything)))

The left side of the rule says “match a symbol with any namespace value, and a name value that is a string which starts with "_"“. The right side says “build the form (meander.zeta/anything)“.

So its pretty much like a macro, however, it applies to any term not just terms that look like lists.

Notation is not global like operators are. That would be crazy.

Instead, you have to tell Meander when you want to use notation:

;; Vector of length 3
(pattern [_ _ _] {:notations [anything-symbol]})

;; Vector of 3 _ symbols
(pattern [_ _ _] {:notations []})

So, if you want to build a custom dialect that doesn’t have things like _ symbols or whatever, you can do that.

This also makes the parser smaller. 🙂

Thanks for the peek into zeta !