Fork me on GitHub
#calva
<
2020-09-23
>
Stefan T18:09:59

Hello! Is there a keyboard shortcut to clear the repl output?

borkdude18:09:56

How does one do ctrl-c in a terminal in VSCode?

pez19:09:47

@stefan.toubia, select all, delete

😐 1
Stefan T19:09:48

Thanks! Feature request incoming…

pez19:09:11

if it's a request for a shortcut clearing the output, then I think we are pretty content with cmd+a delete 😃

Stefan T19:09:48

well it’s multistep mouser over to focus on output window, then do the deed

Stefan T19:09:17

the output window in vscode has a command for clearing output

Stefan T19:09:59

as would any interactive REPL in terminal

pez19:09:00

as long as you are interacting with it... but doing it ”remotely”,... yeah. If you make it an issue+PR, I'll merge. 😃

Stefan T19:09:47

yeah a lot of times I’m using the test runner on one side and watching the repl output in the other, so I’m not working in the repl window here

pez19:09:14

@borkdude one does not simply .... j/k, not following what you are asking actually. I just press ctrl+c ?

borkdude19:09:42

@pez ever opened a clojure repl in the terminal and pressed ctrl-c?

pez19:09:48

what is supposed to happen then?

pez19:09:26

To me the same thing happens in iTerm as in a vscode terminal.

borkdude19:09:59

what is supposed to happen? kill the REPL?

borkdude19:09:14

ctrl-c is also bound to copying things

pez19:09:43

I see. I'm on mac.

pez19:09:58

Killing the repl is ctrl+d regardless, right?

borkdude19:09:29

ctrl-d also doesn't work

pez19:09:44

You have some extension chording on ctrl+c maybe?

pez19:09:47

ctrl+c is not killing the repl for me, and I am not expecting it to. but it cancels the current prompt, as I expect it to. ctrl+d kills the repl.

borkdude19:09:56

[email protected] ~ $ clj
Clojure 1.10.1
user=>
ctrl-c in iTerm2 kills it for me

borkdude19:09:10

but whatever, ctrl-d also doesn't work

borkdude19:09:08

The same things work for me when I'm connected to the same machine over ssh in iterm2

pez19:09:10

Right so clj is killed with ctrl+c which is a bit odd, but that is what happens for me in both iterm and vscode terminal.

borkdude19:09:24

Can you view what ctrl-c is bound to in the terminal, which action?

pez19:09:06

cmd+k cmd+s shows you the key bindings. You can filter them. If you filter on ctrl+c you should see what it is bound to in different contexts, I think.

pez19:09:32

When I do that I have nothing bound to ctrl+c

chucklehead19:09:32

that status looks like Ctrl+C is set as the leading part of a chord for some other keybinding?

chucklehead19:09:46

so it's probably never getting sent to the terminal

Stefan T22:09:35

clj-kondo has been slowing my machine to a crawl, I’ve been having to turn off the extension recently. Anybody have tips for debugging this issue? I really want to keep my linting turned on.

borkdude22:09:41

@stefan.toubia About to go afk, but if you can repro this using public code, I'd be happy to take a look. Are you using configs/hooks? Maybe setting those aside makes a difference? ttyl

Stefan T22:09:03

Thanks, I’ll see what I can do. I’ve got a config in .clj-kondo, I’ll see if removing that changes anything

borkdude22:09:54

Also try linting from the command line with the binary version, see how long that takes

Stefan T22:09:13

Will do. Might be hard to compare, since it’s just happening in the background in vscode. It just ends up causing all the calva features to slow down, so paredit stops behaving, my machine tries to take off, etc

Stefan T22:09:00

Puts cpu through the roof eventually

borkdude22:09:59

Hmm, I see. I hope you can get to the bottom of this

Stefan T22:09:16

me too 🙂

Stefan T22:09:43

I also noticed that after disabling clj-kondo extension and restarting vscode this process did not terminate

borkdude22:09:06

killall java?

borkdude22:09:46

It's the job of VSCode to kill language servers. I haven't heard the problem before

borkdude22:09:59

Do you see anything in the clj-kondo output panel?

Stefan T22:09:41

so I just killed the process, wiped out the .clj-kondo config and cache and reenabled clj-kondo extension

Stefan T22:09:00

there is output but I didn’t think to check it before

Stefan T22:09:25

It’s been pretty consistent, I was on an older version of calva up until a week or two ago when I started getting this issue

Stefan T22:09:56

I’m going to keep working with it on and see if the CPU starts to rise again

borkdude22:09:03

To be sure, also lint your code with the command line version, just to see if you get any errors there

borkdude22:09:42

I mean, unexpected errors, not lint warnings

👍 1
Stefan T22:09:58

I’ll check that out too

borkdude22:09:41

There is also a VSCode plugin that can use the native version, without a java process

borkdude22:09:47

by marcomorain

borkdude22:09:57

but I'd like to get your issue solved regardless

Stefan T22:09:09

yeah worked fine running standalone in terminal

Stefan T22:09:25

linting a single file

Stefan T22:09:30

let me try it with my old config now

Stefan T22:09:03

also works fine

Stefan T22:09:47

is there any interaction between calva and clj-kondo, or are they totally separate?

borkdude22:09:26

calva does not use the clj-kondo extension, but it does use clj-kondo as a library in their JVM process

borkdude22:09:45

for different reasons, not for linting

borkdude23:09:17

afk now. keep me posted.

👍 1
Stefan T23:09:29

thanks for your help!

il-tmfv16:09:22

Having the same issue 😕 @stefan.toubia were you able to fix it?

borkdude16:09:59

the code for the plugin is here: https://github.com/borkdude/clj-kondo.lsp maybe you can run using a local version with some debug statements in it. Without any repro I can't help

il-tmfv16:09:59

I wish I knew what causes it 🙂 It works fine for hours but then suddenly some java process starts eating all CPU. And it stays even after I quit VSCode. After restart everything is fine. I’m trying to edit the same files, but it works ok.

borkdude16:09:54

using jps you can view which pid is related to clj-kondo

borkdude16:09:05

17481 clj-kondo.lsp-standalone.jar

borkdude16:09:29

and you can kill it of course, but other than that, I don't know. VSCode should be killing the server itself

borkdude16:09:36

so it might as well be a bug in VSCode

borkdude16:09:18

the source code of the language server itself hasn't been changed in over 4 months

il-tmfv16:09:32

I’ve tried to quit vscode now and kondo process was killed as well You know, maybe it is killed every time, but when it is in “100% CPU” mode it takes more time, will wait next time :thumbsup:

il-tmfv16:09:57

hmmm, I’ve just realized that I ran test via lein doo not sure how they work, I didn’t do their setup, but I see that it copies a lot of files to target/cljsbuild/public/... folder during start phase. Could it be the reason?

il-tmfv16:09:02

And their content, well, is a copy. Can it confuse kondo? “Oh there are more than 1 definition”, somthing like that

borkdude16:09:12

clj-kondo only lints files you are editing

il-tmfv16:09:57

ok, will keep an eye on it thanks:+1:

borkdude16:09:33

which OS are you on?

borkdude16:09:47

could it be some virus scanner thing maybe

il-tmfv16:09:39

macOS 10.14.6 no AV software installed It happens in the evening. Maybe OS schedules something like indexing (not sure why it affects kondo/java though), maybe something happens just b/c it works for a day and smth like memory leakage is happening

borkdude16:09:01

What we can do is put a bunch of debug statements in the plugin, make a vsix package and you watch the clj-kondo output panel

borkdude16:09:54

it should already output that it starts (and stops, but it's likely you won't see that since it only happens on workspace change )

il-tmfv16:09:27

Let’s wait, I will avoid running tests on my machine. For some reason I have a feeling that it is somehow connected. Unfortunately I don’t remember the exact day when I ran them for the first time and when the issue started to appear.

Stefan T16:09:21

I have to leave right now but I have not found anything useful yet

Stefan T16:09:46

clearing the config didn’t help, eventually it just gets back to a bad state and I have to kill the process

Stefan T16:09:57

I’m also running tests fairly often in editor, I wonder if there’s any relation

Stefan T16:09:08

I haven’t seen anything helpful in the output yet

Stefan T16:09:41

thanks for the link @borkdude that’s a good lead, I’ll read through and see if it helps

Stefan T16:09:23

actually something else I noticed is the CPU for the clj-kondo extension is shooting through the roof as I type as well. That first spike was just typing a bunch of ((((((())))))), That last block I just wrote a tiny function and the CPU stayed high for 30s after I stopped

Stefan T16:09:00

would definitely be helpful to get more verbose output on the language server, will check back in if I find out more

Stefan T16:09:14

also, jps shows two clj-kondo processes. Only one of them is misbehaving

borkdude17:09:51

I do see CPU activity as I type, which is expected, but nothing abnormal. do you use one or multiple workspaces at the same time?

borkdude17:09:28

Can you also try disabling Calva and see if you can trigger the same problem?

borkdude18:09:13

@stefan.toubia @UFU7GAE15 Added a bunch of debug statements which are visible in the clj-kondo output pane

borkdude18:09:57

Please install this VSIX

borkdude18:09:08

and keep me posted

👍 1
borkdude18:09:24

to build yourself:

script/build-server
cd vscode-extension && vsce package

Stefan T22:09:56

oh very good point I forgot I had two workspaces open

Stefan T22:09:30

thanks for the updates extension, I’ll test and see if I get any useful log info

il-tmfv18:09:21

No issues today 🙂 But I worked on another part of the app

Stefan T18:09:52

I also have had no more issues :man-shrugging: but I’ve been doing the same thing

1
Stefan T20:10:21

Ok so this issue has come up again, I’ve gotten to a state where I have high clj-kondo CPU usage and I’m getting this in the clj-kondo debug output, even without touching anything in vscode it seems caught in a loop where this is output every couple seconds

[Error - 1:12:24 PM] [email protected]
changed file, linting: file:///Users/stoubia/src/liftoff/clj/lib/user/src/liftoff/user.clj
found config dir at /Users/stoubia/src/liftoff/.clj-kondo
[Error - 1:12:32 PM] [email protected]
changed file, linting: file:///Users/stoubia/src/liftoff/clj/lib/user/src/liftoff/user.clj
found config dir at /Users/stoubia/src/liftoff/.clj-kondo
[Error - 1:12:41 PM] [email protected]

pez12:10:19

I’m running this build now. Seems I am working with things today that triggers the clj-kondo+calva combo to freak out.

pez12:10:26

Could it be that the output window gets some funny data printed to it?

borkdude12:10:01

did you install .91?

borkdude12:10:39

.90 had funny output, that should be fixed with .91

pez12:10:10

Yes. .91. I was speculating about wether the contents of the Calva output could be what triggers the condition. But maybe not, since we have disabled all linters for it…

borkdude13:10:35

you disabled the rules, but clj-kondo is still linting it

borkdude13:10:08

can you post (maybe in a gist) the problem that clj-kondo prints to its output panel?

borkdude13:10:18

it should say somethig like [Error ..] blabla in case of an exception

borkdude13:10:41

This may be the cause of this CPU spike, if calva keeps printing to the REPL and triggers clj-kondo. But please post output so we know for sure.

borkdude13:10:19

We can hard code the Calva output file name in the clj-kondo plugin so it's just always ignored

borkdude13:10:27

but first proper diagnosis

borkdude13:10:25

You can maybe replicate this scenario by printing output to the REPL every second or so

borkdude13:10:29

and then see what happens

pez13:10:39

So far no errors and my computer is cool.

borkdude13:10:22

Do you see any output in the clj-kondo output panel. Like, is it trying to lint the REPL output?

pez13:10:25

Yeah, it prints when the files are changed.

changed file, linting: file:///Users/peter/Projects/app/.calva/output-window/output.calva-repl
found config dir at /Users/peter/Projects/app/.calva/output-window/.clj-kondo
publishing diagnostics

pez13:10:12

For project files it says the same thing except no file name after the message

found config dir at 

borkdude13:10:05

that's probably a nil:

(debug "found config dir at" dir)

pez13:10:08

I’ll notice immediately when I trigger the error, because I have my computer on my lap.

1
borkdude13:10:35

nice physical feedback and also a nice way to stay warm in these colder days

borkdude13:10:51

so I think we can safely ignore output.calva-repl in the plugin right. this is just waste of clj-kondo cycles :)

borkdude13:10:47

> For project files it says the same thing except no file name after the message I think you might have no .clj-kondo dir for that project, right?

pez13:10:29

That’s correct.

pez13:10:59

Would it make sense to have a way to ignore dirs and/or files via the config?

borkdude13:10:57

so far that's not been needed, by default clj-kondo only lints .clj(s)(c) files

borkdude13:10:00

I'll just hardcode it for now, until clj-kondo gets support for it. This saves me doing another clj-kondo release for this one plugin only

borkdude13:10:22

Please install .92

👍 1
pez13:10:02

How come it lints a .calva-repl file then?

pez13:10:58

I’ll wait installing .92 I think. I want the cpu to burn my lap up so that we can try figure out what causes the error. 😃

borkdude13:10:38

You have config {:linters ^:replace {}}: this will still lint the file but not report any errors

borkdude13:10:59

I think Calva sets the output window to Clojure mode and this will make it lint with clj-kondo probably

pez13:10:02

My question was re: > by default clj-kondo only lints .clj(s)(c) files

pez08:10:15

I haven’t got a hot cpu in my lap since I started wishing for it. But I have two stack traces in the clj-kondo output that look like so:

[Error - 10:15:53 AM] java.lang.OutOfMemoryError: GC overhead limit exceeded
	at clojure.lang.PersistentHashMap$BitmapIndexedNode.ensureEditable(PersistentHashMap.java:815)
	at clojure.lang.PersistentHashMap$BitmapIndexedNode.assoc(PersistentHashMap.java:894)
	at clojure.lang.PersistentHashMap$BitmapIndexedNode.assoc(PersistentHashMap.java:876)
	at clojure.lang.PersistentHashMap$TransientHashMap.doAssoc(PersistentHashMap.java:327)
	at clojure.lang.ATransientMap.assoc(ATransientMap.java:64)
	at clojure.lang.ATransientSet.conj(ATransientSet.java:27)
	at clojure.lang.ATransientSet.conj(ATransientSet.java:15)
	at clojure.core$conj_BANG_.invokeStatic(core.clj:3366)
	at clojure.core$conj_BANG_.invoke(core.clj:3358)
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
	at clojure.core$reduce1.invokeStatic(core.clj:942)
	at clojure.core$set.invokeStatic(core.clj:4113)
	at clj_kondo.impl.linters$lint_unused_namespaces_BANG_.invokeStatic(linters.clj:364)
	at clj_kondo.core$run_BANG_.invokeStatic(core.clj:119)
	at clj_kondo.lsp_server.impl.server$lint_BANG_$fn__11528.invoke(server.clj:126)
	at clj_kondo.lsp_server.impl.server$lint_BANG_.invokeStatic(server.clj:125)
	at clj_kondo.lsp_server.impl.server.LSPTextDocumentService.didChange(server.clj:152)
	at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0(GenericEndpoint.java:65)
	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint$$Lambda$12/2115597658.apply(Unknown Source)
	at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.notify(GenericEndpoint.java:152)
	at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.handleNotification(RemoteEndpoint.java:220)
	at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.consume(RemoteEndpoint.java:187)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage(StreamMessageProducer.java:194)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:94)
	at org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run(ConcurrentMessageProcessor.java:113)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

pez08:10:16

Both are printed after the output has reported that a file has changed, and none of the files have been the output/rep file.

borkdude08:10:38

interesting

borkdude08:10:43

Maybe a memory leak

borkdude08:10:04

I think the maps containing all the stuff needs to be cleaned afterwards. Clj-kondo normally runs as a short-lived binary, so it's never been a problem there

borkdude08:10:14

I'll see if I can fix that

borkdude08:10:14

Hmm, there aren't any global things, each lint starts with a fresh context

borkdude08:10:20

so I'm not sure why this happens

borkdude08:10:35

Is there one particular file that triggers this?

borkdude08:10:41

Can you repro it?

pez08:10:39

I’ll try

borkdude09:10:10

My first guess was that the content in these atoms would get large over time: https://github.com/borkdude/clj-kondo/blob/master/src/clj_kondo/core.clj#L99-L117 but they are re-created on each lint, so the old ones would get garbage collected, I think

borkdude09:10:50

can you also check the memory usage of the corresponding java process (find the PID with jps) when it happens?

pez09:10:24

Seems like the process went to bed after the last GC error:

[Error - 10:34:21 AM] Connection to server is erroring. Shutting down server.
[Error - 10:34:21 AM] Connection to server is erroring. Shutting down server.
[Error - 10:34:21 AM] Connection to server is erroring. Shutting down server.

borkdude16:10:08

@pez I have a suspicion where the resource leak might be. I use memoized functions to check things on a config, and this might hold on to something for too long. Can you check the memory usage of the clj-kondo java process over time for a few hours during work?

borkdude18:10:00

I think I have a repro!

🎉 1
borkdude18:10:18

$ clj -J-Xmx1g
Clojure 1.10.1
user=> (require '[clj-kondo.core :as clj-kondo])
nil
user=> (dotimes [i 10] (do (clj-kondo/run! {:lint ["src"]})))
nil
user=> (dotimes [i 10] (do (clj-kondo/run! {:lint ["src"]})))
nil
user=> (dotimes [i 10] (do (clj-kondo/run! {:lint ["src"]})))
nil
user=> (dotimes [i 10] (do (clj-kondo/run! {:lint ["src"]})))
nil
user=> (dotimes [i 10] (do (clj-kondo/run! {:lint ["src"]})))
user=> (dotimes [i 10] (do (clj-kondo/run! {:lint ["src"]})))
Execution error (OutOfMemoryError) at (REPL:1).
Java heap space

borkdude18:10:27

(from clj-kondo's own repo)

borkdude18:10:23

also high CPU

borkdude18:10:26

Found it, I think

borkdude18:10:20

Fix underway

pez19:10:12

How did you go about figuring it out?

borkdude19:10:21

I already suspected it had to do with memoize since that's the only place where things are stored in a global atoms and then I saw with VisualVM that there were sci things in memory after a GC... which was weird. Then I discovered I was also using memoize in the hooks namespace and effectively holding on to all the state forever by memoizing over a ctx argument... https://github.com/borkdude/clj-kondo/issues/1036

Stefan T19:10:57

This is great news 🙂 many thanks for looking into this!

pez19:10:10

Very good find. Guess it is a valuable stresstest to have it as a long running process that makes all that work on every single keystroke. 😃

il-tmfv13:10:00

Great to hear 🎉