Fork me on GitHub
#calva
<
2018-12-19
>
pez05:12:55

@mattly in the vsix I packaged there is another, similarly named command, Indent Current Form.

pez05:12:12

Obviously I need to think about the names of those commands… 😀

mseddon07:12:58

@pez, realised I can store the delta for parens in a line. For every (, +1, for every ), -1. giving me some total value.

mseddon07:12:33

this means that doesn't need to be recomputed unless the text changes, and I can scan up and down lines rapidly until I find the appropriate depth from a given point, and then look at the line itself to fine tune the position!

pez07:12:04

@mattly That was what I had in mind anyway. But it does sound strange with the behaviour you get.

pez07:12:20

Exciting stuff, @mseddon!

mseddon07:12:55

it dramatically simplifies the implementation, and should hopefully make it far more amenable to clojure porting down the line.

pez07:12:05

I noticed that vscode seems to know what the enclosing forms are for any given cursor position. (If I place the cursor at column zero and visit some different lines vscode highlights the right enclosing lines with a slightly thicker vertical indent-line. Have you seen that?

mseddon07:12:19

yeah, I have looked at how they do it, too

pez07:12:33

They don't expose it, right?

mseddon07:12:35

there's a private vscode api for it

pez07:12:53

Guessing they are using the syntax highlighting grammar?

mseddon07:12:30

yeah it comes from the tmLanguage spec

mseddon07:12:30

speaking of which, I've noticed I have subtle colouring bugs with parenthesis with the clojure highlighter

mseddon07:12:16

notice the right parenthesis changes colour

pez07:12:52

Is that Tonsky's highlighter?

mseddon07:12:22

nope, not running anything clojure related other than calva, but I suspect it's a problem with the grammar that ships with vscode.

mseddon07:12:59

so low priority it's silly though 🙂

pez07:12:03

I see. Well, I have a soon to be two months old PR on that grammar, so it is probably not something that is easily fixed.

pez07:12:16

I mean, quickly fixed.

mseddon07:12:22

can we override our own tmLanguage for clojure with the plugin?

pez07:12:30

However, I think we can ship our own grammar.

pez07:12:52

Yeah, I didn't want to do that, but I guess it's time to bite that bullet.

mseddon07:12:04

It doesn't much bother me for now. I could open an issue on it but it's so minor...

mseddon07:12:33

In fact I suspect tonsky's paren matcher would actually 'fix' it since it adds decorations to parens

pez07:12:44

Indeed. I wasn't thinking about that particular glitch, but rather in general and also the PR I filed.

mseddon07:12:34

yeah, I think especially with atom no longer supporting textmate grammars it's probably easier to just maintain our own copy looking forward

pez07:12:47

I think most of Tonsky's features should be ported to Calva Formatter once you have got your lexer/parser together.

mseddon07:12:09

yeah he has a nice bunch of stuff there, and I already have all the data I need to do it.

pez07:12:21

I also think we could maybe wipe that grammar totally and do the highlighting with your stuff.

mseddon07:12:06

I would advise against that directly, because vscode has a very, very fast highlighter https://code.visualstudio.com/blogs/2017/02/08/syntax-highlighting-optimizations

mseddon07:12:24

however we could overlay further highlighting where it makes sense, for e.g.

mseddon07:12:50

it is silly that we have to redo the work vscode does, but there it is. perhaps one day they will expose a sane api.

pez07:12:48

You have some benchmarks there to measure your tokenizer against. 😃

mseddon07:12:53

I will get hammered I am sure 😉

pez07:12:02

But, really, I was just trying to save some kittens. I guess I can live with the knowledge that we are redoing the same stuff. 😃

mseddon07:12:33

but honestly I figure 25k lines is already hitting the limits of sanity for a clojure file anyway, unless it's an edn file.

mseddon07:12:18

the main result of this upcoming PR will be to make indentation etc instant for larger files.

pez07:12:25

People who have 25k lines of Clojure code in the same file can suit themselves.

mseddon07:12:43

🙂 yes it's a silly number, but it keeps me honest

pez07:12:07

It makes sense since there is EDN.

mseddon07:12:48

Even then I don't think I'd want to read 25k lines of that. better than 10k of JSON, sure..

pez07:12:45

It is convenient to be able to do it, for when that file lands in your lap.

mseddon07:12:05

yeah. but you wouldn't expect any editor to really enjoy working with a huge file like that

mseddon07:12:35

However, if the file was lots of small forms, this would be just as fast as a 5 line file once it's done the first lex pass

mseddon07:12:31

there are tradeoffs at every turn building a datastructure to handle things at that scale

pez07:12:48

There are. We should log some telemetrics to measure how common things like this are. To get informed on what to optimize on.

mseddon07:12:20

yeah, agreed.

mseddon10:12:19

growing selection to next sexpr (or the whole document) is now robust. I can add the paren-delta stuff if needed (I realise i need 3 values, minDelta/maxDelta/totalDelta per line to avoid losing information), but I think I'll stick with what I have for now and see how it fares

pez10:12:44

Can you also shrink the selection?

mseddon11:12:38

I am about to add that now 😉

mseddon11:12:12

Just rationalizing the api a little so it is easy to write emacs-like save-excursion walks

pez11:12:11

If you can shrink it from an arbitrary selection then you also beat the current Calva Paredit command for shrinking. (it can only shrink something thatvwas previously expanded).

mseddon12:12:09

yeah. I am currently looking at the emacs operators that are used to mess around with lisp in various packages

mseddon12:12:31

I figure if we can get a small core together we can push most of the interesting stuff into calva-lib

mseddon12:12:48

possibly even this datastructure lives in there as typescript for now

pez12:12:04

I wonder if there's a way the structure could be shared amongst the users of the lib.

pez12:12:22

Thomas Heller thought that would be possible with the npm module.

mseddon12:12:06

yeah we should check that

mseddon12:12:37

so, okay the api as far as you can see it currently is called TokenCursor, which is a pointer into the document that navigates at the granularity of lexical tokens.

mseddon12:12:44

it handles 'smart' movement around the structure. I have some cheesy low-level functions in there currently, but I thought I'd steal a bunch from emacs. forward-sexp backward-sexp forward-list backward-list up-list down-list backward-up-list

mseddon12:12:38

so from there, in a perfect world we can have calva-lib handle that structure and expose those functions to other plugins

mseddon12:12:21

and then it follows similar a design that the excellent lisp handling emacs has and we save 40 years of trial and error 🙂

mseddon12:12:14

although the JS side of TokenCursor is actually mutable, the clojure wrappers can be immutable, so using it won't feel too ugly.

mseddon12:12:17

it turns out paredit.el is again, purely lexical in nature, incidentally.

mseddon13:12:37

I notice a lot of these primitives (possible all?) are also already exposed by calva-paredit

pez13:12:32

Not all.

mseddon15:12:44

oh, of course paredit can't shrink the selection if it was initially user-created

mseddon15:12:07

given the following selection (foo (bar baz) (frob)), which of the 3 expressions should it shrink to?

mseddon15:12:26

I could select all of them with a multiple selection, but that is probably not 'beating' Calva Paredit meaningfully 😉

mseddon16:12:53

parrot okay, those primitives are all implemented now and exposed in that calva-fmt branch

mseddon16:12:10

grow selection is missing again, because I want to add some 'auto-balance selection' logic and do it nicely, but given an empty, (or balanced) selection a call to backwardUpList() and upList() on the left and right positions will perform that stupidly.

mseddon16:12:38

oh and all those primitives should be completely identical to emacs (except emacs treats insides of strings slightly differently)- so if you see any discrepancies, that is a bug.

mattly17:12:09

@pez yeah this isn't working as expected either:

pez19:12:07

@mattly what happens when you try that?

mattly19:12:08

same as before, nothing really happens. it takes hitting o twice for it to even insert the line

pez20:12:36

@mattly I get this in the dev console error: ModeHandler: error handling key=o. err=Error: command 'editor.action.insertLineBelow' not found

pez20:12:31

So maybe that is what is going on. It doesn’t find that command and does not remap correctly.

mattly20:12:50

hm... that's odd

mseddon20:12:38

did you perhaps want editor.action.insertLineAfter?

mseddon20:12:51

I don't think editor.action.insertLineBelow is a thing.

mattly20:12:06

i probably got that from this

mattly20:12:06

if it's before/after instead that would certainly explain it

mseddon20:12:21

They named it stupidly it seems 🙂

mseddon20:12:35

That is the same command, but the human readable string is inconsistent

mattly20:12:33

woo, that works!

mattly20:12:42

now I just need to make it go into insert mode

pez20:12:38

Here’s another way to do it:

"vim.normalModeKeyBindings": [
        {
            "before": [
                "o"
            ],
            "after": [
                "A",
                "\n"
            ]
        }
    ],

💯 4
pez20:12:01

That won’t require the relaxed indent command to be exposed.

pez20:12:11

I think that with @mseddon’s new take on on-type-formatting we shouldn’t need any reconf of VIM key bindings. (Maybe I have already said that yesterday? Anyway, I said it again, if so. 😄 )

mseddon20:12:02

hopefully not, but once the basics are ready I will try to make that so if it isn't 🙂

mseddon20:12:53

It's been quite a slog, but it looks hopeful I will have something roughly working by the end of the week or beginning of next, so that's nice.

mseddon20:12:24

there are some further optimizations I could do, too, but if I don't draw a line somewhere this will never ship...

pez20:12:25

It is already a >10X improvement so Peter Thiel would say it is worthwhile proceeding w/o further optimizations. 😃

mseddon20:12:06

exactly. and it should hopefully address some of the things mentioned in https://github.com/BetterThanTomorrow/calva-fmt/issues/4

mseddon20:12:20

(although I know some of those are already fixed)

mseddon20:12:11

btw @pez, what are your thoughts on adding a repl buffer, akin to cider's?

mseddon20:12:03

Terminal interface seems too dumb, but perhaps it could be either a WebView or a different edit mode.

pez20:12:22

My thoughts are that it would be awesome. And I think quite possible if we use regular editors.

mseddon20:12:49

Yeah, regular editor seems the sanest way, because that way people can use their favorite extensions in it

pez20:12:07

I think that is a bit how Cider is doing it.

mseddon20:12:25

yeah, cider is based on slime, so the repl is just another major mode

mseddon20:12:19

I am slowly poking my way around how they do things, in the hope that one day we achieve feature parity with cider, many moons from now...

pez20:12:22

Not sure what that means, but in vscode it might be that we have some dirty editors that the user needs to save.

pez20:12:54

Many moons. 😃 But I like the thought.

mseddon20:12:59

ah, a 'major-mode' is just a set of keybindings and syntax highlighting rules that are active in a buffer in emacs

mseddon21:12:45

so basically it's the same as how our stuff works inside a clj(sx)? file

pez21:12:51

The work you are doing is again probably going to come in handy when we control the contents of such a repl editor.

mseddon21:12:37

yes, I hope so. Basically if we can make regions in the file 'read only' in some way, we should have an okay time.

pez21:12:57

Why would they need to be read only?

mseddon21:12:19

ah, well, you don't want to start editing previous output. it wants to sort of be a console

pez21:12:42

I am thinking that doesn’t really matter.

mseddon21:12:49

or at least, you want to know which areas are output, which may be just some println, vs the command line

mseddon21:12:56

welll.. it doesn't really. but it's a nice to have 😉

mseddon21:12:36

the other route is a hairy scary webview with exciting webgl output 🙂

mseddon21:12:54

but that would mean kinda hosting our own command line editor with all the trimmings etc, so quite a struggle

pez21:12:26

I think that would risk loosing much of the benefits from the edior being Calva powered out-of-the-box.

mseddon21:12:28

plus- not being compatible with whatever third party extensions is a bit annoying

mattly21:12:43

@pez thanks that works like a charm

pez21:12:06

@mattly cool. so for O it becomes what? "0", "k", "A", "\n"?

pez21:12:33

Doh, why the "0"

mattly21:12:00

yep, i did that without the 0

pez21:12:14

Maybe "I", "\n" works too… It was a while since I used VIM.

mattly21:12:23

now my gripes mostly align with the vim emulation 🙂

mattly21:12:57

oh that should work too

pez21:12:08

The VIM emulation seems pretty awesome given the quite limited API:s of vscode.

pez21:12:31

I think it might put the cursor in the wrong place.

mattly21:12:12

that's my understanding, most of the faults are faults of the VSCode API

mseddon21:12:21

It's only 3 years old, there's time to improve 😉

pez21:12:22

I think the VIM extension team has been granted some special API:s even.

mattly21:12:16

well, i'd settle for more public commands for things like "delete next word" or the like

pez21:12:41

How do you mean more public commands?

mattly21:12:03

something i can map a hotkey to

pez21:12:35

You mean w/i the vim extension?

mattly21:12:04

no, I mean, to do vim-like things without the vim extension

pez21:12:00

I can do alt+DEL.

pez21:12:31

I think there are command id:s for deleting a word forward.

mattly21:12:34

... oh i thought i had tried that

mattly21:12:33

del isn't common on most keyboards anymore but I have it on mine

pez21:12:44

fn+backspace on my Mac.

pez21:12:16

With Calva Paredit you can delete the next sexp, with ctrl+shift+backspace. It often makes more sense in a clojure file than deleting a word.

pez21:12:59

Haha, you can put whatever on your keyboard!

mattly21:12:09

yeah, it's been fun to play around with

mseddon21:12:55

@pez on cool lighttable/protorepl goodness, perhaps helping along https://github.com/Microsoft/vscode/issues/3220 might help

mseddon21:12:26

oh... look, you're in that thread, too, @pez. 😄

mseddon21:12:33

it's 2 years old.. so.. don't hold your breath just yet 😉

pez21:12:56

Yeah, I don’t remember what my input to the thread was, but I probably tried to bump it. 😃

pez21:12:07

I have since been thinking that a way to do something like that would be to use vscode custom tree views. Together with nav that could become pretty powerful.

pez21:12:47

Think something like “Watch Expressions” in a debugger. Whenever you evaluate something it would be added to the view, and would stay there for as long as the evaluation was active in the editor. And in the tree view you would be able to unfold whatever data structure that resulted from the evaluations..

pez21:12:50

If we can track an evaluation as the file is being edited the tree view could offer UI for reevaluating entries.

mseddon21:12:09

Yeah that might be worth persuing

pez21:12:37

Right now I have started with a thing that might be a bit too hard for me to solve. I am fed up with how I currently need to open and close sockets for every nrepl msg. But to fix that I need a much less naive way of handling bencode decoding. And neither bencode, nore sockets is really part of my skill set. I’m not really a programmer, just a product manager who caught interest in Clojure. Haha.

mattly21:12:19

coulda fooled me

pez21:12:12

We learn what we need to learn to get things done, right? 😃

mseddon21:12:28

😄 Heading out now but happy to give you a second pair of eyes tomorrow.

pez21:12:14

I might need it. Or, I probably will need it, rather. 😃