btw I am in the process of porting some cljs to orchard, stay tuned! 😄 Sooner or later we are going to have a cljs nrepl/lsp server going 😉

@richiardiandrea, I am staying tuned!

This is a very interesting project as well: https://github.com/snoe/clojure-lsp?files=1

@mseddon: Now develop is using the cond etc fixes. It's pretty nice! So nice that I published it. 😃

great! 😄

clojure-lsp, huh. yeah that does look pretty cool.

@pez ah, uhoh! couldn't find pez.cljfmt.0.0.3-SNAPSHOT, when trying to build calva-lib. Is that just a renamed version of cljfmt on master published to your local maven?

ah, no, a bit of detective work says it is the calva-fmt branch of your fork here https://github.com/PEZ/cljfmt/tree/calva-fmt/cljfmt 🙂

Did I foget to publish it?

Thinking about it… with this build process I might not need to have my own Clojars entry for it. But maybe it gets a bit too complicated if people must clone that repo out as well and lein install and stuff, even if they are not going to hack on that part (I am hoping we shall need hack on that fork as little as possible.)

There, now deployed. 😃

I need to automate much of the process…

oh, alright let me try that 🙂

yep, it found it, thanks 🙂

:the_horns:

do you want me to just replace my formatter.cljs into the one on calva-lib? there seem to be quite a few changes, but mostly relating to whether you :refer stuff, some old zprint cruft, etc. I'm trying to work out if there's anything that actually wants to be kept from calva-lib.

The most important part is probably if things are properly exported.

I tend to litter the files down a bit with (comment) block experiments. 😃

I blame Calva, making it so easy to evaluate top level forms inside comment blocks.

😉

yeah I can't see anything major- I'll move the :exports from the old calva-fmt into lib

see what breaks 😉

That :refer sounds interesting. Might be that I have required clojure.string because the new build conf uncovered that breakage.

aha, yes! I see that

I will add that one back in

About those comment blocks and various playground.clj* files you might find. I have been thinking if maybe I should add a Calva command for automatically creating a playground file for the file I am editing and open it up, split view, so then you I can play and experiment there. And these files would be meant to be commited to the repo.

yeah, there was something like that for intellij/scala... i forget what they called them, 'worksheets' or something

they had a special extension to differentiate them from standard source iirc

.playground in Swift XCode.

But maybe you are thinking about notebooks, rather?

oh, maybe! it's been ages since I used IntelliJ 🙂

I was thinking about Maria and Gorilla REPL stuff, rather.

oh yeah, gorilla repl is really cool 🙂

Did you merge the crlf-dabble branch into latest develop when you created your crlf branch?

not at the time, i'm basically doing that all now 😉

I think I have some sane merges now

although on the calva-lib side i've basically nuked formatter.cljs with the crlf one.

I'm a bit worried that I might have done something else important since I failed to fix the crlf thing.

I couldn't see much on the cljs side, but there were a bunch of tweaks with 'extraConfig' and things in the .ts side of clj-fmt, which I've tried to accomodate

Those extra configs were the eol stuff, or?

you have a js object passed in called extraConfig, which gets merged into the config passed into formatter.cljs on the ts side

and i've added in the extra :eol key from that side as well, now

extraConfig is a bad name, i have no memory about what it could be about 😃

😄

just about to play the npm link game and see if anything still works...

I think I might also have moved all parinfer stuff to inferer.cljs.

But the tests should have told you that… 😃

the compiler is bitching currently, 🙂

apparently we don't have zprint.core anymore.

I think that is commented out in shadow-cljs.edn

But what would be trying to use it, I don't know.

totally pointless dep- it's only commented out in formatter.cljs

Clean away all zprint stuff, it is just leftovers from when I thought I could use it.

yeah

ah, okay, a few bits, (I love code restructures don't you? ;)), util/current-line-empty? etc is missing in calva-lib, but I can move them over...

missing? that sounds strange.'

No, I moved those to formatter,cljs.

I moved anything that ”knows” things about the mmap from utils.cljs.

Maybe it is better to start from calva-lib fresh and apply the crlf things on it by hand.

aaaaaah, right, okay that's easy enough to put back then.

basically it's down to my merge of clrf:formatter.cljs, I just nuked out some of those utility functions 😉

I guess we will see when we diff things from what it looked like before you dropping that nuke.

it's a noisy diff 🙂

I have mostly been busy getting the new build conf to work, so not all that many things could have happened with the actual code. 😃

you're right though, it may be easier to just manually put the crlf changes back... 🙂

It's amazing how fast a branch can rot even with just one dev who has very little time with the project. 😃

😄 We all end up here.

it's just too easy to create forks 😉

huh. more interestingness https://github.com/BetterThanTomorrow/calva-lib/blob/master/package.json

no version or publisher fields, but I assume this wants to resolve to "@cospaia/calva-lib": "^0.0.17"

Haha, it's much tricker than that. 😃

uh oh 😄

The package.json for the calva.lib the npm module is in /packages

aaah

so I will want to compile and then run npm link from THERE, I guess.

The one you found is mainly the build scripts for building the module.

lol. I just published junk locally. 😉

Yes, I hope I got it right on the wiki.

I probably didn't read it right, let me go back. hurrr

The missing version field stops me from publishing junk on http://npmjs.com 😃

ah yes, there it was 😄

and good plan 🙂

Maybe we should add something in that json file about where to find the real one. 😃

Maybe even a README for the calva-lib repo! Haha

😄 early days

but yeah it probably ought to be cleaned up

but this crlf merge is going to be a nasty PR I suspect so I don't want to do a million other things as well 🙂

Zero other things seems like the perfect amount of other things.

If I can assume correctly where the calva-fmt project is checked out I can do the link as part of the watch and release tasks.

link only needs to be performed locally once

unless you trash your node_modules dir of course

well. everything works except my crlf fix... 🙂

@pez https://github.com/BetterThanTomorrow/calva-lib/pull/1 and https://github.com/BetterThanTomorrow/calva-fmt/pull/9 are now up!

it's not quite as terrible a diff as I feared once I'd cleaned it up a little

check the review notes though, for some reason shadow-cljs exports have nerfed and so we're back to a snake_case api for TS, which sucks, and I'd like to remove that 'feature'.

snake_case wasn't introduced by crlf, however.

With the old build conf you had to specify the function names manually, which wasn't too fun. Also that API sucked more since it wasn't namespaced at all.

ahh, fair enough

hold off merge for now though, I am seeing some test failures... sigh

ah, it was easy, forgot to pass :eol "\n" during the tests. Okay, that ought to work now, back to the 6 new-index-at-top-level failures.

I'd like for us to not revert the library API with this PR. The API will fatten when Calva starts to use it as well. snake_case is a price i am willing to pay.

no problem, I will remove those :exports declarations

(and look into those damn new-index-at-top-level failures)

I think those should be handled by trying to stop vscode to indent anything at all.

ah, right, are you fighting it there huh?

I have been fighting it everywhere.

🙂

can we kill it dead if we override the return key?

that may kill two birds with one stone if we reformat+insert \n then

Certainly.

But i think this one might be as simple as just configuring vscode to not indent. Just that i only thought about that option first time a few days ago.

🙂 I will have a dig around and see.

I won't be by a computer for a while now.

Hours. Not days. 😀

🙂 phew. Anyway I am in no mad rush to merge+publish that crlf fix, take your time and check through, I may still be being an idiot somewhere. It should be good as far as I can tell though.

I think i bad some out-commented test for crlf. Not sure if i kept it, though.

yick. so I've spent the day with the docs. vscode is.. rather lacking in many places in the API.

It is.

I am considering biting the bullet and building a full blown language server to deal with it

since the more I dig into the good extensions the more I see they gave up and did that 🙂

for example, solving the toplevel indent issue is pretty damn hard without performing a lot of extra work tracking the AST as the changes come in

plus also we don't really want to be passing the whole document across to the plugin for formatting long term because of fun performance issues

BUT, I can't even gain access to the tokenization information from the buffer, so that would have to be performed by us again, which is silly.

e.g. in emacs it's really quite easy to write a decent lisp formatter, because you can peek at the partial parse state across lines, we literally just have a blob of text, despite the fact somewhere in vscode, it knows the extent of folds, parenthesis, etc 😕

Vscode language servers are constrained by the same API as language extensions, aren't they?

yeah, technically we can do it inside vscode

but e.g. if I know that the first character of a line is inside a string, I can wind back one line until we're not inside one

then lex it forwards to figure out where we are. Without that information locally lexing like that is just not possible

so.. yeah. I can see why you throw the whole document about 😉

I think the main 'reason' for considering going the LSP route is that other editors also support it, like vim, sublime etc. so at least it's reusable.

that and/or backing off into jvm land for those extra cpu cycles

hmm. although sort of all that data seems to be being maintained inside paredit, right?

bleh. that is also consuming the whole document.

As is, Parinfer. While maintaining speed, even. 😉

going to try Ctrl+W on that giant file and get back to you there 😄

12k line file containing only this repeated over and over:

(defn sqr [x]
  (* x x))

Ctrl+W takes 4 seconds to expand the selection 😉

huh, it did seem to speed up to around 300ms, but that's still pretty sluggish.

Probably optimization no kicking in.

so for e.g., one stupid solution to the line terminator is to just mash Ctrl+W to get the whole range of the toplevel expr, then look at the column of the selection start...

Which will be zero.

oh, sorry i mean the selection END

basically we need to know if we're in a toplevel

There is already a command to get the top level selection. Try ctrl+alr+w space.

ah. yes. him.

okay well from that I think I can botch around the indent at end of expressions

although it's... yick.

and also then there is a hard dependency on calva-paredit in calva-fmt.

Calva-lib is using paredit.

all of these contortions are silly, of course, because somewhere inside vscode, it knows where the parens are.

ooh, right, sorry I am thinking of how everything was yesterday 😉

It was yesterday too.

So when indenting the current form, it uses paredit to figure out what the current form is.

oh, also I am confused slightly about the (let [indent-token "0"] ,,,), in add-indent-token-if-empty-current-line it looks to me that somehow it would barf on literal "0"'s in the text, but mysteriously it does not.

is it because it will ALWAYS be paired with remove-indent-token?

Yes.

It's the trick i use to figure out the indent when the cursor is on a new line.

hairy

Haha, that’s why I have asked for the community formatter to consider the cursor position.

@mseddon: you forgot about point 2 and 3 in https://github.com/BetterThanTomorrow/calva-fmt/wiki/How-to-Contribute#before-sending-pull-requests 😃

doh! s'cuse my n00bishness. 🙂

😃

It would be sweet to have a CI pipeline set up to catch things like this.

yeah we should look into that.

i get a couple of test failures btw

FAIL in (format-text-at-range) (at C:475:6)
expected: (= "(foo)\n(defn bar\n  [x]\n  baz)" (:range-text (sut/format-text-at-range {:eol "\n", :all-text "  (foo)\n(defn bar\n[x]\nbaz)", :range [2 26]})))
  actual: (not (= "(foo)\n(defn bar\n  [x]\n  baz)" "(foo)\n  (defn bar\n    [x]\n    baz)"))

FAIL in (format-text-at-idx) (at C:475:6)
expected: (= "(defn bar\n    [x]\n\n    baz)" (:range-text (sut/format-text-at-idx {:eol "\n", :all-text all-text, :idx 11})))
  actual: (not (= "(defn bar\n    [x]\n\n    baz)" "(defn bar\n    [x]\n  \n    baz)"))

I wonder if we can “advertise” somewhere for someone with experience with it CI for vscode extensions.

We should steal that formats-to? function from cljfmt. 😃

yeah, i am a bit worried about crlf's toasting those multiline strings

I only get those 6 “expected” failures..

then.. that is probably what has happened.

does the crlf mode behave as expected?

Haven’t tried it yet. Went for a release build first thing. Will try now.

the dac9531 one should fix the derpy release issue

dac9531?

ah, a new commit?

yeah, that fixed release

or at least, didn't blow up for me.

it may introduce weird test failures on those 2 cases.

I think I may have been staring at this code too long today. 🙂

CRLF mode misbehaves for me. (Haven’t pulled dac9531 yet)

ah, yeah, i think that may have been toast, hence the new test failures...

the difference was somewhere i managed to omit the call to normalize-indents, so everything wound up working as before for a bit

I’m a noob with PRs. How do I pull dac9531?

just re-pull

should get the latest version

I force-push upstream to and that should update the PR

It complains about no tracking information.

I need to run now. Will figure it out later.

probably just git branch -D it and start over 🙂

hi, is calva suppose to highlight code that wont compile?

@christopher.paul https://github.com/BetterThanTomorrow/calva/issues/89

so it’s something that is coming soon then?

Sort of. You'll need to switch linting on.

hmm

i like the plugin its bette rthan then other ones

how do i switch linting on

Glad to hear!

is that a setting in the repl?

i get linting for bad tests

but not bad code :~

😐

It's a Calva setting

oh

i see inthe docs now

sorry

that’s better

in your video you showed things like inline evaluation

but the evaluation that I get is in the terminal

They are separate commands. Check cmd+shift+p and type calva.

there are like 10 evaluate commands

and none of them mention inline

8 evaluate commands sorry

I think those not mentioning inline are inline. 😃

I mean, those not mentioning the terminal. (I’m a bit tired)

My favorite is ctrl+alt+v space for evaluating the top level form the cursor is in.

It also works inside comment forms (disregarding the comment form, so to speak).

Generally the evaluation commands with alt+ after the chord are terminal equivalents of the inline ones.

i see

well thanks for your help

evaluate current selection was the one i was looking for 🙂

@pez, okay, so I've been poking around vscode internals all day. I think I have a good solution to a lot of the irritating fiddles we have with formatting atm, but it's going to take a while to put together. All the bits I need are there. Watch this space 🙂

Cool. Please share some of your findings!

I shall! It's a bunch of upfront work but I'll let you know when it (begins) to work 😉

fortunately I have a lot of the pieces sitting around on my hd..

@christopher.paul, I most often evaluate current form instead, ctrl+alt+e. So if you have the cursor right before or right after a form, that will be evaluated. (But evaluate selection comes in handy at times too, of course.)

@mseddon, I’m a curious guy. 😃 What are the fiddles and what are the findings are two questions in my head right now.

Why do you need to force push to your crlf-fix branch, btw? Github won’t show me the diffs when that has happened.

oh, sorry that's just habit from scala.js 🙂

yea i think i prefer ctrl+alt+v space now

@pez I'll stack the updates until it passes code review, then squash the commits into a single one for merging, sound good?

that way we get a clean 1 commit per feature/fix, so we don't play untangle the commit history later

shouldn’t need to force push, that’s usually bad 🙂

it's okay for PRs, but for heaven's sake don't do it on master 🙂

I’m fine with a granular history.

alright, we'll do it that way then 🙂

But even if you squash your local history, you should not need to force push, right?

that ought to work, although i got complained to for not squash/force pushing on another project. 🙂

It's a little bit of cargo culting on my part though, I'll admit.

Yes, people really have different prefs. But this is still a very small project so I think we can manage the history.

that's fine by me. also less scope for screwups on either end

I think it is nice to be able to check the diffs on github just to orient myself on what it is I am pulling.

regarding the fiddles- the main issue is we're basically re-doing a lot of work vscode is doing incrementally in one big lump on newline... which just makes me cringe. It makes it really difficult to track where everything is from within the extension trivially

so I've figured out on paper and through some tests I can quite easily build an incremental lexer that works from editor events, that only redoes exactly the work it needs

but because it's incremental, I can build a data structure that will always let you correlate the clojure AST to a file offset

which suddenly means, for trivial format on type, we can use much simpler, incredibly quick formatting rules based on cljfmt, and drop to clj-fmt proper for 'full' formatting

you can squash commits without force pushing

I am not touching -f again 😉

good

Haha

i’ve been there and got mysef in many messes

😄

I've remained unscathed so far. let's quit while I'm ahead. 🙂

yes 🙂

do you do this stuff just in your free time, pez?

or is it your job 😄

btw, the 'force push' thing comes from here https://github.com/todotxt/todo.txt-android/wiki/Squash-All-Commits-Related-to-a-Single-Issue-into-a-Single-Commit

but it seems with a push +[branch], that is not necessary

i’m pretty sure you can squash your commits against your fork

then just do a PR

I think that what @christopher.paul and I are saying is that this is key: ” If you’ve already pushed commits to GitHub”. If you haven’t then no need to force.

you can always force to your fork i guess

but can then do a clean PR to origin

yeah, it's basically, once the PR is sent, the only way to squash without force is to close the the PR and open a new one

yeah, so I'm pushing to my fork in a PR

ah

because that way only I feel the searing pain of my own stupidity 🙂

except it bit pez. so... that didn't go well 😉

Hmmm, but I don’t think we need to squash away earlier pushed commits when adding commits to a PR.

that only really applies if you want a '1 commit per issue' policy

Personally I think it is totally OK to push all intermediate commits when preparing a PR.

But, yes, if it is a lot of commits involved and it has taken a while, maybe a rebase -i and curating the history could be good. But subsequent commits after the PR is sent will be hard to follow if they are force pushed.

No “1 commit per issue” policy applies to Calva, I would say. 😃

works for me 🙂

Awesome. Let’s hope so many people are throwing themselves at Calva contributions that we will feel the need to reconsider policies. 😃

😄 exactly. that is a good problem to have

I get those two extra test failures now, btw.

ahh, yeah. weirdly I can't reproduce them in the editor proper

but I didn't write those tests, so it's possible there was a particular situation you were thinking of

The tests look sane to me. 😃

🙂 i may just be going mad at this point

The tests are a bit hard to “read” because of newlines being entered as escapes. I like how that is handled in cljfmt.

pez, vscode doesn’t seem to recognise this setting: “calva.autoAdjustIndent”

i’m trying to disable it as i like to use parinfer

is that definatly the right key?

I think some things around these tests have moved between my attempts to fix the crlf thing and now. Getting more and more convinced it is better to apply the crlf fixes on develop HEAD:

yeah, I think I may have missed something you did in crlf-dabblings

Calva isn’t really very compatible with the parinfer extension, @christopher.paul. What error do you get with that setting?

There is some little parinfer built in to Calva, to compensate for this incompatibility. So you can get parens inferred with ctrl+alt+p for instance. And indent stuff with parens being inferred meanwhile with ctrl+i (dedent ctrl+shift+i).

yea it seems pretty funny

i really like parinfer in other editors

I think that setting, calva.autoAdjustIndent might have moved to Calva Formatter and I might have forgotten to remove it from Calva. (It is not obvious maybe, but when you install Calva you also install Calva Formatter and Calva Paredit and those have their own settings keys.

Me too! Parinfer is awesome.

But VS Code doesn’t have the API:s Parinfer needs so they can’t really make it work fully as it is working in other editors.

And the parinfer extension that is there now competes with Calva Formatter (and often wins, because it is faster).

i wondered why shaun had said he’d come back to it later

i guess it’s easier to do in ato

m

It is.

Search for “Format as you type” in settings to find the option to switch off Calva Formatter’s help with that. (Even if I think it is a better choice to settle with Calva Formatter’s on-command inferring of parens).

i’ll stick with your one for now

parinfer greys out the closing parens

what do you think of that

i think it’s really nice if it’s doing them for you

I love it. 😃

That should be added to Calva.

Maybe Tonsky’s Clojure Warrior is compatible with Calva… at least he is a Calva user and probably would have complained otherwise, I’ll try that extension now. It says it adds some coloring of the paren trail.

Yes it works i use it for the coloring 🙂

Nice! Trying it now. Will take me a while to get used to the coloring, but I’ll stick with it a while.

idk

i dont like colors

i like one colour but the end ones faded

becuase different colours to me mean different context

sometimes blue sometimes green is hard for my brain

OMG. Now I can switch off VS Code’s stupid highlighting of matching brackets. <-- @pedrorgirardi

the coloring is pretty helpful because it is based on intendation level so it gets easier to see when one form is not in the right level

maybe i just have to get used to it

the brackets are more eye catching than the code

That’s what I was addressing with that it might take me some time getting used to. Maybe a careful selection of colors will be easier on the eye: clojureWarrior.bracketColors.

@pez, huh, he's doing what I'm doing in this test 😉

Which test?

well used the to massively grey them out

so that’s kind of nice )

attempting to figure out what the hell is going on inside a vscode buffer 🙂

although I'm working incrementally on the data as you type

(and not re-scanning the whole document like he is)

@christopher.paul, this greys them out and keeps some of the nesting level info:

"clojureWarrior.bracketColors": [
        "#fff",
        "#ddd",
        "#bbb",
        "#999",
        "#888",
        "#777",
        "#666",
    ]

Who is”he”, @mseddon?

i’d set them all to 777 🙂

@pez Tonsky in that bracketcolors extension

Ah!

effectively I'm doing the same thing he does in his updateRainbowColors, but i cache the state of the lexer (inside string, etc) for each line,

so i trigger an update at the edit point, and scan down each changed line

but then i can know where all the brackets are, what symbol starts a term, and the indent level

Can you make a config of the formatter that behaves like his suggestions with that?

https://github.com/weavejester/cljfmt/issues/154?

In principle, sure!

Yeah, but I thought maybe cljfmt isn’t needed for this. But maybe it is.

oh, it's not needed for my little experiment- the idea here is to have a fast path formatOnType

which has a little better control over what's going on than we currently have

How are you planning to get the rules of cjfmt enforced?

working off a copy of his rules

https://github.com/weavejester/cljfmt/blob/master/cljfmt/resources/cljfmt/indents/clojure.clj

I see. Nice!

but of course, this is fast but imperfect, it may not be 100% cljfmt compatible

but for indenting whole blocks of code, we can still use cljfmt

Really cool.

mostly because in those cases we don't have anywhere near the horrible problems of cursors jumping around

I hope you succeed!

wish me luck 😄

I am!

you.. quite literally were 😄

You should be able to source the rules from cljfmt, right?

those indent rules basically drive cljfmt, and should be fairly easy to translate

i don't think cljfmt does any fun pretty print conditional newline stuff

I mean without copying them manually.

No, I don’t think it does.

well, i will naturally write a script to transform those rules into a bit of json 🙂

I don't want to spend the rest of my life manually keeping them in sync 🙂

haha

You’re not doing this in cljs?

i'm keeping it in ts atm, mostly because it's very mutable statey

plus it's easier to have intellisense on the vscode api for now..

yeah, i can relate.

But when you have it pinned down, I’d like to see how it can be factored so that most of the “logic” is in cljs. 😃

See, I’m already banking on that you will pin it down.

no pressure 😉

How about we divide the labour such that I apply your crlf fixes on develop HEAD and you focus on that incremental indenter?

sure thing!

I suspect I missed something you did in your work and accidentally removed it anyway 🙂

I have been the only one working with this stuff and that has “allowed” me to be a bit sloppy with things.

How about this, @christopher.paul?

"clojureWarrior.bracketColors": [
        "#666",
        "#777",
        "#888",
        "#999",
        "#bbb",
        "#ddd",
        "#fff",
    ]