Fork me on GitHub
#tree-sitter
<
2020-03-09
>
sogaiu02:03:35

tree-sitter gotcha: although it looks like the rules: { ... } part of the grammar looks like a js object, order matters for the very first key-value pair. getting this wrong can (will?) cause chaos.

sogaiu11:03:46

re: ignore markers and tag markers -- conceptually, i've been thinking of them as follows (hopefully correctly, but who knows?). for tags:

#my-outer-tag #my-inner-tag 'a
#my-outer-tag is "waiting" for (the function associated with) #my-inner-tag to have done something with 'a. alternatively:
#my-outer-tag (#my-inner-tag ('a))

sogaiu11:03:23

for discards, what i've currently got in the tree-sitter grammar (adapted from Tavistock's) is:

discard_form: $ =>
      seq($._discard,
          optional($.discard_form),
          $._non_discard),

sogaiu11:03:47

here $._discard is basically just #_

sogaiu11:03:29

so for:

#_ #_ 1 2
the inner #_ causes the 1 to be ignored, and the outer #_ doesn't see the #_ 1 and ends up ignoring the 2.

pez11:03:29

With the edit it makes sense. 😃

sogaiu11:03:38

he he -- sorry for the confusion 🙂

pez11:03:11

So what happens with something like:

#my-outer-tag #_ #my-inner-tag 'a 'b

sogaiu11:03:48

my guess: #my-inner-tag 'a is "swallowed" by #_ so #my-outer-tag's associated reader function(?) is passed 'b for processing

sogaiu11:03:27

here is an example:

user=> #uuid #_ #inst "1985-04-12" "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"
#uuid "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"

pez11:03:29

Cool. That's what I guessed.

sogaiu11:03:37

sadly, the current grammar i have doesn't appear to handle it though

pez11:03:40

Don't try this:

#uuid #_ #uuid "84098d2d-2ed5-4de5-bd4a-c4b967f9e468" #uuid "8ce4afa3-1599-4a98-b642-59eb5e9e83b8"

sogaiu11:03:07

i don't know what happens when #uuid is handed an actual uuid

pez11:03:09

Which is the same as this, naturally, which never returns either

#uuid #uuid "8ce4afa3-1599-4a98-b642-59eb5e9e83b8"

pez11:03:38

Nothing happens. Nothing as in things stop happening. 😃

sogaiu11:03:08

well, thanks to this discussion, i have more grammar hacking to do...and i thought i was close to finishing 🙂

pez11:03:48

Same here. To both sentiments.

👍 4
pez11:03:37

However, I have my crude, line based, scanner, and you have that sophisticated super tool. 😃

sogaiu12:03:21

actually, i'm not terribly fond of tree-sitter to be honest for certain tasks -- your scanner can be more flexible.

pez12:03:45

My scanner can be almost anything I want. Which is nice. However, the line based part of it... I wish I had lots of time so I could get rid of that!

sogaiu12:03:49

it's pretty amazing that it can do so much even being line based -- i was surprised to be sure

sogaiu12:03:10

have you considered looking at what edamame or tools.reader does?

pez12:03:23

I have looked in many different places. Often very shallow, though.

pez12:03:40

The scanner isn't of my creation, btw. I have modified it a lot so have some right to call it partly mine, but it was Matt Seddon creating it. I think it was the 100th time he created one, so he really knows these things at a level that boggles my mind. He is creating a new one as we speak, for a new LISP that he is creating at the same time. Haha, I guess he just likes to have work.

pez12:03:38

I am nudging things so that he hopefully will create a non-line-based thing this time. And DFA. Then I plan to bring that into Calva, replacing my current thing.

sogaiu12:03:31

it's good if he enjoys what he's doing -- getting this stuff working can be fun, but it can be frustrating when there are no official specs

sogaiu12:03:07

i've got some notes from digging through clojure / clojure-dev google groups archives and jira issues to try to track down what is supported by the reader

💤 8