https://github.com/mseddon/calva-fmt/tree/restartable-formatter so this is now still a bit too eager, but is incrementally lexically analysing the entire source file correctly.

need to optimize it a little bit more- since e.g. if you type " in a file, now you have inverted the ENTIRE document's state as inside and outside a string. So this needs to be lazily computed when necessary rather than rescanning everything below the line that changed.

still, once this is handled, we should be able to amortize everything really nicely, and have a very fast way to find the enclosing sexpr at any part of the file

I will be tracking the enclosing sexpr, the indentation rules for that sexpr based on the start symbol + a table, as well as the index this expr is in it's parent, which should be enough to recreate cljfmt

but then- finding the required indent will be O(1) for extremely small 1, so everything should indent instantly like it does with e.g. typescript

it will also allow us to grab exactly the lines required to pass to cljfmt and parinfer

Awesome. I'll have a look tonight.

We will need to be able to feed it custom cljfmt configs, but from what I hear that is not a problem, right?

no problem at all- we just strap an edn reader on front and we're done 🙂

that'll be how we get the original cljfmt stuff too

some more advanced settings- i am not sure how they work yet, but we can address those later 🙂

Reading EDN isn't hard for calva-lib.

nope 🙂

right, I have to walk this dog. catch you later!

I use a js edn reader for Calva today, but found that it is to flaky, will switch to doing it via calva-lib, since now Calva has that in reach.

catch ya!

hi guys I see there has been a lot of chat since I visited this channel last time. Recently I heard of Tree-Sitter API added to Atom for source code parsing. Is it going to be added to code? Any idea if Calva can use it?

I have one issue to report though

if a source file has errors in it, and if I evaluate the file, I am not seeing the stack trace

I had to evaluate the file in normal repl

by restarting it and then found it

regarding tree-sitter- I am implementing a similar thing but specifically for calva/clojure

the stack trace thing I am not sure, perhaps @pez knows?

oh okay. why not just take the Tree-Sitter bindings. The reason is we can get all the future bug fixes for free

(I am not sure how to create bindings though)

that's true, although Tree-Sitter's back end is in c, so that may be overkill

I think they were creating API bindings for other langs

as per what I saw on the youtube talk

ah, no, i mean the actual parser itself is in c

and I am unsure how we distribute the executables

https://github.com/tree-sitter/node-tree-sitter

oh, it has typescript typings too

https://github.com/tree-sitter/tree-sitter-cli

It allows for creating new parsers

huh, I will take a look at that later this week, thanks!

it is interesting, since it's fairly easy to write an incremental parser for lisp, but they are using some far more complex research to handle lalr languages

My opinion is leaning on deep pockets is better

Let them take care of the complexity of the parser

Tree-Sitter is probably not coming to VS Code. They are considering adding it temporarily until they get the parser ported from Visual Studio.

and let them pay for the maintenance and bug fixes

yeah, there is an issue here https://github.com/Microsoft/vscode/issues/50140

oh

Yes, that was the issue I read. Along with a lot of linked issues.

Doesn't mean we can't use Tree Sitter, though.

But we need only parse clojure code, which is less complex than the task Tree Sitter has to perform.

which can be done very easily since it's an LL(1) language

If anything we should be looking at rewrite-clj, me thinks.

oh okay. noted. I have not heard of rewrite-clj before. I will bing it

It's great, although it's not an incremental parser per-se

but the nice thing is I can build incremental edits to sync that up further down the line

About the stack trace, @manas.marthi, I am aware of that those are often swallowed by Calva, but I would appreciate an issue filed so that I get reminded about it, preferably with some small repro.

okay

I will reproduce the issue and file a ticket

huh. with a 25k line file even reconciling the lexer for the entire file is fast for me. 🙂

this is also tracking the parenthesis, too

Definitely we need to rebind Ctrl+W to use this datastructure instead of parinfer

I need some aggressive tests, and I realised I can be even lazier with reconciliation, because I can ask "I need everything from this position up to the next N close parens", so it can bail early there, too

and.. i need to handle parens within the line, but that's easy

basically at the moment I can instantly get the parenthesis stack and the source locations of them for the start of any line in the file. so that's nice.

(I can also immediately detect mismatched close parens, but I don't do anything about that atm)

Can you correctly find the right current enclosing form in an expression like '(foo)?

Yup

Well modulo any bugs :)

And if that close paren is a few lines down and we are deeply nested inside expressions i only have to reconcile up to the first ) in that case

Cool. Maybe it's time to replace paredit.js, at least for when we use it to figure out the current form. There are a few evaluation commands in Calva that use that.

Also, of course, the whole Calva Paredit suffers from that paredit.js is abandoned.

Yeah. It's probably not too hard to do lexical slurping and barfing with this, actually

But... small steps :)

Oh, also we can use this to add breadcrumb support quite easily 😀

what is breadcrumb support?

Oh, they are those little bits along the top of the editor. Currently you have the directory path but we can show the heads of each expression, too.

oh okay.. thanks..

Also, maybe to implement the # command?

The #?

Press CMD+P, then #. In a TS file.

Ooh, right yeah

Of course that can be done via nrepl, but seems to be a perfect match for static analyze.

Yeah.

Hi there, @shane. Welcome!

👋

I have an interesting edge-case. I have the following in my editor:

;(clojure.test/test-vars [#'get-namespace-works])
(clojure.test/run-tests)

Evaluating:
;(clojure.test/test-vars [#'get-namespace-works])

Evaluation failed for unknown reasons. Sometimes it helps evaluating the file first.

I can’t reproduce that. Maybe it is something else in the file tripping paredit.js up? What happens if you have only those two lines in the buffer?