Fork me on GitHub
#calva
<
2022-04-30
>
jacob.maine01:04:03

As requested @pez, here’s the start of a https://github.com/BetterThanTomorrow/calva/pull/1698 for exposing clojure-lsp’s drag-forward/drag-backwards commands so they can be bound to keys. I’ve left more detailed notes in the PR about were I’m stuck on testing. Any guidance would be appreciated. /cc @brandon.ringe

metal 2
👏 2
❤️ 1
bringe01:04:51

Thanks for taking this on! I’m going to be pretty busy for the next couple weeks as I’m about to go on a vacation. I can help more when I’m back, but also other members of the Calva team can probably help you before then.

jacob.maine01:04:41

I’m all about taking vacation! I’m sure the rest of the team can help me, and besides, there’s really no rush

pez06:04:29

Thanks for your work, @U07M2C8TT 🙏. Looks like progress to me!

pez10:04:57

I've put some comments from my findings on the PR. Now a bit stuck. I don't know how, but it often seems @UKFSJSM38 understands this part of Calva better than I do. 😃 So with some pointers from him and @brandon.ringe we will probably be able to figure it out.

👀 1
jacob.maine19:04:12

Thanks for the responses on Github, @pez and everyone else. I’ve responded there, but will be online for a little so if it’s easier to chat here that’d be great. I’m going to be double-checking that the client and server exchange the correct capabilities to make showDocument work. After that I’ll see if I can figure out how to make VS code show me the wire-level LSP logs.

👍 1
bringe19:04:48

> I’m going to be double-checking that the client and server exchange the correct capabilities to make showDocument work. After that I’ll see if I can figure out how to make VS code show me the wire-level LSP logs. I recommend you turn on the logging of the messages first, and then check the initialize request to see if showDocument is passed in it.

bringe19:04:09

You may need to reload the window after enabling logging to see the initialize request

jacob.maine19:04:58

I got the message logging turned on. The client and server are exchanging showDocument requests (so capabilities must be working). Now I have to look at the actual contents of those requests and responses

💯 1
bringe20:04:14

See here (if you haven’t already) about the details of that request and response: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#window_showDocument

bringe20:04:49

Particularly, check that Calva is sending a response back, and if success is true.

jacob.maine20:04:01

So with further experimentation, sometimes the cursor is positioned correctly and sometimes it isn’t

jacob.maine20:04:11

I was slightly wrong about how the interaction works…

jacob.maine20:04:42

The client requests a drag with executeCommand. The server immediately replies with no response. Then it sends two asynchronous requests to the client. The first is applyEdit, with the new text. The second is showDocument, with the new cursor position. The client then responds to both requests, indicating that it successfully processed them

jacob.maine20:04:22

What’s interesting is that the response to showDocument sometimes appears in the logs before the response to applyEdit. I haven’t run it enough times to be certain, but it appears that in those cases the cursor isn’t positioned correctly

jacob.maine20:04:47

If the applyEdit response appears before the showDocument response the cursor may or may not be positioned correctly

ericdallo20:04:54

yeah, we should make that command sync, IIRC it's possible for some commands be sync

jacob.maine20:04:10

So, I think this is pretty clearly a problem with asynchronous processing. It’s unclear who’s to blame. We might be able to avoid the problem by having the server respond to executeCommand with an actual edit, instead of a no response, and then an async applyEdit

jacob.maine20:04:26

Is that what you’re saying @UKFSJSM38?

jacob.maine20:04:08

On the other hand, it might be a client-side problem… where the client should finish processing applyEdit before starting its processing of showDocument

ericdallo20:04:14

maybe on clojure-lsp side we should catagorize some comands as sync and async, this one is a example that should be sync

jacob.maine20:04:41

@UKFSJSM38 do you know of any commands that need to be async?

ericdallo20:04:14

yeah, I have no idea how showDocument should behave there, becase we are actually doind 2 things, a refactor and then telling to showDocument, we want both sync

jacob.maine20:04:46

Also, just for the record, I’d argue that this is a bug in the LSP spec. It should be possible to include the cursor position and the text edit in the same response

👍 1
ericdallo20:04:47

Not really @U07M2C8TT sync usually blocks editor to avoid edition, but I don't know any existing command to be that expensive, they are rpetty fast usually

jacob.maine20:04:15

Ah… there’s a problem with the reasoning that executeCommand should be sync. executeCommand is generic… it returns LSPAny, not WorkspaceEdit. That means clients can’t turn those responses into edits. So, how are edits supposed to be applied? From the spec: > The workspace/executeCommand request is sent from the client to the server to trigger command execution on the server. In most cases the server creates a WorkspaceEdit structure and applies the changes to the workspace using the request workspace/applyEdit which is sent from the server to the client. So, clojure-lsp is already doing what the spec says it should.

ericdallo20:04:29

yeah, I'm afraid it wouldn't be possible to fix that async issue without creating custom LSP requests 😔

jacob.maine20:04:24

Oh, but here’s something interesting… both applyEdit and showDocument are requests, not notifications. That means that lsp4j returns CompletableFutures. Perhaps all we need to do is deref the applyEdit (i.e. wait for the client to finish processing it) before sending the showDocument. I’ll try that.

ericdallo20:04:11

makes sense @U07M2C8TT I thought we were already doing that, it should be even the default I think for most refactors as we don't want user changes messing up with code refactors

jacob.maine20:04:01

yeah, agreed. i can’t think of a case where we’d want it to be async. i have a vague memory that we used to deref that future, but it was making tests hang and so i took it out. 😔 i might be thinking of something else, but …

jacob.maine21:04:57

Yes! That fixes it. And I can see in the VSC LSP logs that the server doesn’t send the showDocument request until the client has responded to the applyEdit request. And also yes 😞 the integration tests hang with that deref. The “client” in the integration tests is really dumb. It drops every server request on the floor, instead of responding to them. Fixing that isn’t going to be easy.

ericdallo21:04:35

Great, glad we can fix on lsp side. In the last case we branch only for integration tests, I know is not ideal but we already have to do things like that on some specific places, check for :env usage on clojure-lsp

jacob.maine21:04:57

Ah, good tip @UKFSJSM38… thanks!

jacob.maine21:04:01

Anyway, to summarize for @pez, @brandon.ringe and @U02EMBDU2JU … I have a handle on the cursor positioning and possibly on the flicker. Both need changes on the clojure-lsp side. I still don’t know why project.json data isn’t being picked up on my machine. But, if it’s working for you all, perhaps it doesn’t matter. It makes it slightly more tedious to test—I have to use the actions menu instead of the keyboard—but at least it’s possible. I’d still like to get keybindings working, but it’s not critical. I think it’s best to wait on the PR until I’ve had time to work on the clojure-lsp side, and for that to get released. Thanks for all your help so far!

👍 2
pez21:04:46

The thing with keyboard shortcuts not working for you is that you can't test what happens when you do these and other commands in quick succession. So we really should figure that out.

jacob.maine00:05:45

@pez helped me figure out the keyboard bindings… I didn’t realize that the extension host is its own editor, and that the package.json data would only be available in there. I thought the extension host was a process that needed to be running and that it would affect the behavior of the main editor. He also helped me get the Calva REPL running. 🚀 So, now I know how to test this PR… and have an outline of what I need to do on the clojure-lsp side. Thanks all!

👍 3
jacob.maine19:05:50

The clojure-lsp release this morning includes the drag fixes I made for Calva—fixed cursor-positioning and reduced flicker, by editing fewer nodes. I’d love to have a few more eyes check both of those features. Then, I think this PR is ready for another review. @pez I left a follow-up question about command naming on Github. Let me know how you’d like to proceed. And a general question about Calva development… there’s a conflict with the dev branch in the CHANGELOG. What do you prefer… Merging dev into the branch? Rebasing the branch off dev? Resolving the conflict as the branch is merged into dev?

pez20:05:04

I thought I had answered about the command naming. Maybe there are new questions? I'll go check.

pez20:05:08

I don't have your branch checked out right now, so you can rebase it on dev, solve the conflicts and force push. Usually it works with merging too, but it sometimes get a bit confusing to see what belongs in the PR or not. So in this case I prefer rebase and force push.

👍 1
jacob.maine20:05:31

@pez, yes some new questions… let me restate them here a bit more clearly:

jacob.maine20:05:08

First, in the PR, this is a command:

{
        "command": "calva.refactor.cleanNs",
        "title": "Clean NS Form",
        "category": "clojure-lsp Refactor",
        "enablement": "editorLangId == clojure && clojureLsp:active"
      }

metal 1
jacob.maine20:05:10

The category has been changed as discussed. Are you also expecting the command to change, perhaps to "clojureLsp.refactor.cleanNs"?

jacob.maine20:05:10

Second, a similar question for the drag commands…

{
        "command": "calva.refactor.dragBackward",
        "title": "Drag Sexp Backward",
        "category": "clojure-lsp",
        "enablement": "editorLangId == clojure && clojureLsp:active"
      }
As discussed, the category includes the word “clojure-lsp”, but no longer the words “Calva” or “Refactor”. The question is, do you want the command to change too? "clojureLsp.dragBackward" or something?

pez21:05:15

Thanks! I am have been threshing a bit on the ”should the commands change?” question. As you mention in the PR, it might break any existing keybindings people have made. But I now I'm down with an opinion: I think we should do it. It is easier to maintain and easier for users to find the command names if they reflect the title and category.

pez21:05:05

As for the drag commands. I'd like them to be named like the Calva commands are. So same titles, different categories. And then the same reasoning for the command names.

jacob.maine21:05:28

That all sounds good. Is "clojureLsp.dragBackward" safe, or does there need to be a second-level namespace: "clojureLsp.edit.dragBackward" ?

pez21:05:33

I think it's fine with one level. The Calva equivalent is named paredit.dragSexprBackward. So this one should then be clojureLsp.dragSexprBackward.

jacob.maine21:05:07

:thumbsup: I’ll work on getting that changed, although I think it has to be clojureLsp.dragBackward , not clojureLsp.dragSexprBackward, because the last segment of the name is auto-generated from the command that gets sent to clojure-lsp, which is drag-backward.

ericdallo21:05:19

Question: can't we support the old name but not offer as a public command? Break user's keybindings sounds a little bit bad to me :/

jacob.maine21:05:09

I’m a little hung up on breaking keybdings too… if you’d like @pez, I can look at supporting deprecated aliases. I don’t know exactly how I’d do that, but I’m sure I could figure something out—it’s all just JavaScript.

1
pez21:05:37

It's manifest stuff actually. Will mean we have several commands bound to the same function. Maintenance, and cognitive load both for us and the users.

ericdallo21:05:15

Yeah, that's bad, I thought vscode could have something special to handle those things

pez21:05:09

From The Tao of Calva: > On the Calva journey we will allow ourselves to change our minds about how things work. Calva is not a library. Its an interface between Clojure and human beings. Human beings can adapt. We'll have to remember to inform about this in the release notes.

👍 1
jacob.maine21:05:21

Wow … if only more projects had a philosophical vision they can quote from :person_in_lotus_position:

pez21:05:18

If only more than 0.1% of Calva users read this document. 😃

😂 2
pez21:05:22

That said, command id:s are a bit in between, and it does pain my maintainers heart to make changes like this. Tao or not. So that's why the threshing has been going on in my head all day long.

ericdallo21:05:04

Makes sense, given that, I'm ok with dropping and mentioning on release notes :)

jacob.maine22:05:13

@pez command renames pushed… I also updated the docs for the refactoring commands. There aren’t any docs for the new drag commands. Do they need to be added somewhere?

pez22:05:07

That would be nice. I am looking at the current docs to try help with figuring where ”somewhere” could be. I'm noting that the clojure-lsp page lacks information about that clojure-lsp contributes commands. I think a section about that should be added. This section should mainly link to the Refactor page and to the ”somewhere” place, I think.

pez22:05:21

”somewhere”... I think the Paredit page is the right place. We have https://calva.io/paredit/#drag-bindings-forwardbackward. Put one of those https://squidfunk.github.io/mkdocs-material/reference/admonitions/ there, I'd say. A ”Note: clojure-lsp also contributes commands for dragging forms backward and forward. You might prefer those, because they ... <some TL;DR about why>. See <link to issue or PR, whichever describes it best> for more on this.

jacob.maine22:05:23

Got it! Thanks

pez22:05:13

I'll be tucking in for the night here now. Tomorrow most of the day will be open source day for me (I have a sweet deal with my employer) so will be able to fit in testing your PR then. Hopefully we can have some more eyes on from other <!subteam^S03BGSAUPTQ|@calva-team> members as well. But if not, right now it looks like tomorrow is merge day tomorrow for this. 🎉 (Yeah, celebrating in advance. That is part of my life philosophy. A hard earned lesson made it so. As hard as they come, actually.)

❤️ 1
🎉 1
jacob.maine22:05:00

I hear you … I’d rather regret a premature celebration than regret never celebrating at all. ’Night. Thanks for your help!

1
pez09:05:24

I'd posted some comment, including a change request on the PR, @U07M2C8TT. Corssposting the video I attached here as well. Along with the ”captions” 😃 > As always it is hard to follow along in a video like this. Here's somekind of caption that might help: > 1. Sometimes does not drag... > 2. Does not retain cursor position. Moves the cursor to the start of the dragged text. > a. Sometimes misplaces the cursor, I can't immediately see a pattern to it. > 3. Drags the sexpr in front when adjacent behind of a form. And generally does not adhere to Calva's https://www.youtube.com/watch?v=8ygw7LLLU1wconcept at all. > 4. Does not drag words inside strings > 5. The window scrolls around in a surprising way. > 6. Significant clojure-lsp semantic tokens lag/flicker. Especially in large files. > 7. Performance drop in large files. Nothing surprising. But it is more prominent than with Calva's versions of the commands. I'm using an Apple M1 Max, haven't tested om less powerful hardware. > > 2 & 3 in combination makes for some extra weirdness. > > To me the differences in what is the dragged form is a big concern. I'd like to wish for that Calva's concept of this is adapted by clojure-lsp. It will cause less cognitive load for Calva users. And since Calva's concept here is somewhat compatible with CIDER's (Calva prioritizes the previous form when there is a competition), I think it will be less surprises also for CIDER users.

breno1232114:04:45

It's really small https://github.com/BetterThanTomorrow/calva/pull/1700 that fixes the Website url from the .md extension to the right one in the readme.md. Also first time opening a PR at Calva so if I got something wrong 🆘 hehe gratitude

❤️ 1
lspector21:04:29

Is there a way to make a new leiningen project from within VS Code/Calva?

bringe21:04:59

Well, yes, if you consider running lein new command from the integrated terminal within VS Code/Calva, and then opening the directory created in the VS Code window. simple_smile

bringe21:04:17

What kind of experience do you imagine?

lspector21:04:19

That's not too bad! Does Calva make command-line leiningen available there if the user hasn't gone through the normal process of installing it separately?

bringe22:04:15

No, it’s a responsibility of the user to install it.

bringe22:04:39

I’d recommend people use a terminal to create the project, then run code <project-dir>

bringe22:04:07

It would be quite the trouble and maintenance burden (I think) to try to have Calva include build tools or have it provide mechanisms for installing them, and might not be ideal anyway.

bringe22:04:07

But we can help users in the docs by pointing them in the right direction for installing the tools, which the docs may already do.

lspector22:04:14

Good to know. Thanks. My motivation here is for teaching, to minimize the complications for students getting set up. In my experience some in each class invariably have trouble installing leiningen, especially on Windows, and I'm not much help there. Lately I've been using IntelliJ/Cursive which does somehow bundle it so students don't have to do it separately

bringe22:04:32

It might be easier these days to have them use the Clojure CLI rather than Lein for running Clojure. Last I checked the CLI was easy to install on Windows, though I think it was in alpha/beta at the time (for Windows).

bringe22:04:33

There are tools for creating project templates with the CLI that I believe @U04V70XH6 has done some work on.

lspector22:04:13

Yeah, I've been considering switching to CLI, but haven't made the transition myself yet.

bringe22:04:13

I see. I’d recommend trying it out when you have some time. It’s progressed a good bit. I think Lein won’t be necessary in the future or maybe even used much any more. (I don’t use it unless trying to test something related to Lein for Calva.)

bringe22:04:13

Lein has been great though. I don’t have anything against it, really. I just prefer to use more official tooling.

seancorfield22:04:35

The official CLI installer on Windows is very alpha and requires some Powershell experience. Lots of people struggle with it.

lspector22:04:11

Alas, I do always have a bunch of students using Windows 😞 😞

seancorfield22:04:33

It's easier to either install @U04V15CAJ’s deps.exe or to install Scoop (package manager) and then use it to install @UBLU3FQRZ’s packaged version of the CLI tools.

seancorfield22:04:00

@U06BV1HCH Leiningen is "just" a .bat script for cmd.exe on Windows so that's "easiest" still.

seancorfield22:04:47

My personal recommendation for Windows is to use WSL2 and Ubuntu and then just use all the regular Clojure tooling (since macOS/Linux are 1st class citizens in the Clojure world).

seancorfield22:04:44

VS Code on Windows has a WSL2-Remote extension that makes the experience very smooth. And I think these days just typing bash into cmd.exe on any recent Windows 10 or Windows 11 machine will auto-install WSL2 now?

☝️ 1
seancorfield22:04:27

But that does require a bit more technical dexterity than just point'n'click stuff on Windows 😕

bringe22:04:07

I wrote this guide in the past that might be useful to you or your students @U06BV1HCH: https://gist.github.com/bpringe/4f1d07f98633a956a8b33af572e7b810

seancorfield22:04:25

Cursive is probably the "easiest" all-in-one option for Windows but it's a hella-complex UI and the whole project-based run config stuff is neither "easy" nor "simple"...

seancorfield22:04:11

Ah, I tend to forget about Chocolatey for Windows. Yeah, that's another option than Scoop.

bringe22:04:16

It’s been a while though, so something could be outdated, but it uses choco to install lein to avoid the download + run script way to install.

lspector22:04:20

A lot of this looks potentially helpful! Things have gone smoothly using Cursive lately, but for reasons like you mention @U04V70XH6, and exciting things I see happening in the Calva ecosystem, I'm looking into switching. But I will always have some students with little command line experience, no Clojure experience (but probably Java and Python), and a professor who is clueless about Windows 😱

lspector22:04:42

I've got the links posted above (thanks @U04V15CAJ and @brandon.ringe!) and will check them out soon, to see if I see a smooth path for onboarding students with this toolset.

👍 1
borkdude22:04:52

The Calva Getting Started REPL needs no command line install though

borkdude22:04:11

It also uses deps.clj (the uberjar)

borkdude22:04:31

@pez can probably tell you more about that

lspector22:04:41

Students will need to make projects, though... and there's no way to do that without lein or CLI?

borkdude22:04:44

All you need to make a project is create a src directory with a clojure file in it and then you can "jack in" from Calva. But I do think that calls out to the clojure command. @pez might be able to come up with something there based on what he did for the Getting Started REPL

borkdude22:04:00

The minimum requirement is a version of Java (1.8 or higher) which should already be installed

lspector22:04:06

The project also gives you a way to add dependencies, with project.clj or (I guess, I don't use it yet) deps.edn, right?

borkdude22:04:35

yes, you need to add a deps.edn file too, sorry I forgot that

lspector22:04:05

So if students don't have lein or CLI, they'll have to create those files, know what goes in them, etc. It's nice to have them pre-made so they can just paste in the dependencies they want.

seancorfield22:04:59

But they'd have to learn about deps.edn and the src/package/file.clj concept almost immediately anyway so why not teach them that from the get-go?

borkdude22:04:48

I agree with Sean, it's better to teach them some fundamentals

lspector22:04:59

Because they have other things to learn almost immediately. Giving them a simple way to get a working project is a big leg up.

seancorfield22:04:11

Actually, you don't need deps.edn -- I just tested:

(! 500)-> mkdir lee
(! 501)-> cd lee
(! 502)-> mkdir -p src/package
(! 503)-> echo '(ns package.file) (println "Hello!")' > src/package/file.clj
(! 504)-> clj
Clojure 1.11.1
user=> (require 'package.file)
Hello!
nil
user=> 

👍 1
borkdude22:04:14

then give them a sample project they can clone?

lspector22:04:32

The fundamentals are map, filter, reduce, etc., not project structure ceremony.

lspector22:04:05

Giving them a simple project to clone may be the solution... but nice if they could get it from the tool...

lspector22:04:30

Thanks for all of the advice!!!! I have to run for a couple of hours but then will check back in!

seancorfield22:04:19

They still need a way to get clj / clojure equivalent or lein command installed -- unless you forgo files altogether... and no one can get far with Clojure without files...

borkdude22:04:24

@U06BV1HCH if you want to have a light-weight alternative, there is also #babashka which is a standalone binary which can execute Clojure code very quickly. It does not even require Java. There's also the possibility to create a http://repl.it environment for the students to work in for Clojure.

borkdude22:04:25

Or perhaps the git pod stuff?

borkdude22:04:49

Sleep time now... I'm off.

seancorfield22:04:08

GitPod is nice -- I have GitPod links on several of my repos and you can get a REPL up and running in "VS Code on the web" and run tests etc with zero installation...

borkdude22:04:40

Btw, a nice anecdote, about a decade ago I taught Clojure to students and contributed to the lein.bat script exactly to solve the setup problem for my students. https://github.com/technomancy/leiningen/commits/master/bin/lein.bat?author=borkdude

lspector22:04:48

I absolutely love #babashka for other purposes, but students will need an IDE.

seancorfield22:04:13

2012 👀 Ancient history! 🙂

lspector22:04:40

Last I tried it, replit’s Clojure implementation was pretty broken.

lspector00:05:58

(back) I was super excited about GitPod some time ago, but IIRC I could start it in a tutorial project, but not a new empty one. Just trying it now I got pretty confused. So much on the screen. Are there instructions somewhere for getting started using GitPod to work in Clojure, starting with a new project?

lspector00:05:24

But actually, desktop VS Code/Calva may be good for my use cases anyway, since installing VS Code itself isn't the issue, if there's a way to make projects without installing lein or CLI... and maybe providing a minimal project for students always to download and start with will suffice for that, even though it's not so pretty.

lspector00:05:39

FWIW one other reservation I have about switching from Cursive to Calva is that Calva doesn't yet reformat incomplete expressions in a reasonable way (it just does nothing), and that's actually a key feature for my situation. So I guess it's that and having to install lein/CLI that's currently keeping me in Cursive, even though the complexity of IntelliJ is a bummer

seancorfield00:05:11

@U06BV1HCH Remind me what your students mainly focus on? It's not software/IT, right?

lspector00:05:22

For the classes in which I teach Clojure, the main topic is usually AI, machine learning, evolutionary computation.

seancorfield00:05:06

Interesting. But you work very hard to avoid exposing them to command-line tooling and other typical "IT" stuff, yes?

lspector00:05:01

I work very hard to avoid having IT stuff be something they have to deal with before they can work on the things for which they're taking the course. I do make them learn Clojure the language, which brings in a lot of new stuff like functional programming and immutable data structures, etc., and that's already a lot. So the less tooling stuff they need, the better. Many of them aren't even CS majors, though they'll have taken a few programming courses.

lspector00:05:54

I'm happy to expose them to the IT stuff, but I don't want it to be in their way. And since many of them don't see themselves as going into IT, the idea that it'll be useful for them later isn't so compelling

seancorfield00:05:18

I'm just poking around at Calva and it includes deps.clj.jar so it can run things without needing to install lein or clj/`clojure`, which means it could easily run a command to create a new project using deps-new as a command-line dependency without needing to both folks with any set up...

lspector00:05:22

that sounds promising... Might the stuff I've seen about Joyride help?

lspector00:05:58

And/or maybe that could be used to smuggle in formatting of unbalanced expressions?

seancorfield00:05:18

When you fire up the Getting Started REPL, it creates a temporary folder and runs:

java -jar '/Users/sean/.vscode/extensions/betterthantomorrow.calva-2.0.269/deps.clj.jar' -Sdeps '{:deps {nrepl/nrepl {:mvn/version,"0.9.0"},cider/cider-nrepl {:mvn/version,"0.27.4"}}}' -M -m nrepl.cmdline --middleware "[cider.nrepl/cider-middleware]"
so it could run this command to create a new CLI project:
(! 509)-> java -jar '/Users/sean/.vscode/extensions/betterthantomorrow.calva-2.0.269/deps.clj.jar' -Sdeps '{:deps {io.github.seancorfield/deps-new {:git/tag "v0.4.9" :git/sha "ba30a76"}}}' -X org.corfield.new/app :name myuser/myapp
Creating project from org.corfield.new/app in myapp

Sat Apr 30 17:50:09
(sean)-(jobs:0)-(~/clojure/fresh/lee)
(! 510)-> tree myapp
myapp
|____.gitignore
|____build.clj
|____CHANGELOG.md
|____deps.edn
|____doc
| |____intro.md
|____LICENSE
|____pom.xml
|____README.md
|____resources
| |____.keep
|____src
| |____myuser
| | |____myapp.clj
|____test
| |____myuser
| | |____myapp_test.clj
although we'd need to deal with the project name somehow and we'd probably want a template that is more tailored to what you want for your students (happy to help with that).

seancorfield00:05:24

(that's basically what you'd get from lein new except it's for the CLI -- and Calva can use deps.clj.jar to run anything in that project)

lspector00:05:25

Interesting. What I'm looking for is just what one gets with lein new app.

seancorfield00:05:47

Yup, the above is exactly that. Except for the CLI 🙂

lspector00:05:12

Or, I guess, the equivalent for a CLI project, though I'm not sure what that is since I haven't been using CLI yet.

lspector00:05:47

So that sounds great. But I'm not clear on what has to be done to make this an option...

seancorfield00:05:23

It requires a little bit of work from @pez or some other contributor to Calva to add a command that would run the above java -jar invocation and some work from me, collaborating with you and maybe @pez, to figure out exactly how/where the new project should be created.

seancorfield00:05:21

Somehow, you'd need to tell Calva where to create the project and what the folder should be called (and, ideally, what the top-level ns should be).

seancorfield01:05:37

Perhaps the simplest flow would be to start up Calva, add a new folder to the workspace, and then tell it to create a Clojure project there. And we could default the top-level ns to the user's name (derived from their env) and just use a stock ns name like example. So Calva/deps-new would lay down a deps.edn file, etc, and create src/<username>/example.clj and open it up in te editor?

seancorfield01:05:22

We're always looking for ways to #improve-getting-started and help #growth to bring new Clojure users into the fold 🙂

lspector01:05:20

All of that sounds good! But how about making the file src/<projname>/core.clj? Where <projname> is the name of the new folder? This would correspond to what lein new does, right? Maybe CLI doesn't require this? In any case, I'm not sure why we'd want the username in there.

seancorfield01:05:45

There's also a much simpler project template that might be even easier to work with:

(! 520)-> java -jar '/Users/sean/.vscode/extensions/betterthantomorrow.calva-2.0.269/deps.clj.jar' -Sdeps '{:deps {io.github.seancorfield/deps-new {:git/tag "v0.4.9" :git/sha "ba30a76"}}}' -X org.corfield.new/scratch :name `whoami`
Creating project from org.corfield.new/scratch in sean

Sat Apr 30 18:04:34
(sean)-(jobs:0)-(~/clojure/fresh/lee)
(! 521)-> tree sean
sean
|____deps.edn
|____src
| |____scratch.clj

Sat Apr 30 18:04:39
(sean)-(jobs:0)-(~/clojure/fresh/lee)
(! 522)-> cat sean/deps.edn 
{:paths ["src"]
 :deps  {}}

Sat Apr 30 18:04:49
(sean)-(jobs:0)-(~/clojure/fresh/lee)
(! 523)-> cat sean/src/scratch.clj 
(ns scratch
  "FIXME: my new org.corfield.new/scratch project.")

(defn exec
  "Invoke me with clojure -X scratch/exec"
  [opts]
  (println "exec with" opts))

(defn -main
  "Invoke me with clojure -M -m scratch"
  [& args]
  (println "-main with" args))

seancorfield01:05:16

core is just an artifact of Leiningen trying to avoid single-segment names -- there's no reason for core appearing anyway.

seancorfield01:05:02

The idea of using their username for the top-level ns is just so that each student would then have something personal in the code and you could likely see whose code each submission was easily.

seancorfield01:05:04

Like I say, exactly how/where the project gets created and what files are inside it, is something we can easily work out. We could create something pretty much tailored to exactly what you need/what.

lspector01:05:36

Ah, I see there's no reason to hang on to core when dropping lein. But I prefer your simpler approach, without the username. I just want them to have a generic, minimal project, in which they can add dependencies if/when they need to.

lspector01:05:08

(and to be able to create it with nothing installed other than Java and the IDE)

seancorfield01:05:43

Currently, deps-new requires a project name to create the directory as -- but I could make it create the files in the "current" folder if that's better for working with Calva. You and I would have to discuss it with the Calva team.

seancorfield01:05:11

If we can get you and your students to use Calva and make this "create a new project" experience as simple and intuitive as possible, it makes Clojure more approachable in general.

lspector01:05:43

I'm not sure I follow the issue with creating the directory, but I would be very happy to discuss this more!

seancorfield01:05:25

The scratch project above is "naughty" because it uses a single-segment namespace -- but that's fine for basic experimentation. Having two segments would be "better" (because it avoids potential problems with Java/JVM stuff later on).

lspector01:05:34

Can we just make it not single-segment? If the tool creates the template, we can just make it create a good template, as minimal as it can be without being problematic (which I gather single-segment namespaces are).

seancorfield01:05:15

Sure. We could hardcode the two segments. Or use the project name as one of the segments. Or the username 🙂

lspector01:05:06

The project name makes sense to me here. Building the user name in seems strange to me, especially if it may end up being a group project, etc.

seancorfield01:05:59

I can't really suggest anything more detailed right now because I would need to hear from the Calva folks on where Calva starts up (on Windows, when not started from the command-line) and the best way to identify where the project should be created and how to add it to the workspace etc etc.

lspector01:05:42

Couldn't we just give the user a file dialog to select the location and name the folder/project?

seancorfield01:05:04

Like I said, I have no idea about that without input from the Calva folks.

lspector01:05:15

Ah, understood.

seancorfield01:05:34

Project templates for deps-new are easy to create. Calva could even have its own templates in its installation and could point deps-new to them as :local/root deps.

seancorfield01:05:50

But the TL;DR is: we could collectively make this do whatever you wanted.

seancorfield01:05:29

Perhaps we should all get on a Zoom call next week and discuss it in more detail?

lspector01:05:34

FWIW on the big picture of #improve-getting-started and #growth, I've just been eavesdropping on the Calva world lately, while actually using Cursive, but I've seen a bunch of exciting stuff that has made me think that Calva could be the future for beginners...

lspector01:05:01

Yes, let's Zoom.

pez07:05:09

A lot to read up on here! 😃

pez07:05:56

This with creating a starter project is a missing piece in Calva's Getting Started story. I really want to fix it.

pez07:05:12

There are some low hanging fruit, like just documenting a few steps for how to do it. I think that to some extent I have always been aiming for something more than that and allowed it to stop me from just making a few obvious efforts... But, anyway... Theer are a few things I haven't figured out, like how to create a folder and make that the VS Code workspace. VS Code unloads Calva when opening a new workspace,... But if we can do everything else and then instruct the user to do the ”now open folder X”, then I think it should be perfectly doable with a reasonable amount of effort put in.

👍 1
pez07:05:05

As @U04V70XH6 mentions, deps.clj.jar makes us satisfy the requirement ”nothing but Java” needed.

pez07:05:37

And @U04V70XH6 has made tools we can use for creating a bare bones project structure, with as little distraction as possible in there.

pez08:05:09

Among our options we also have the infrastructure that Calva's Getting Started REPLs use. It is actually a set of predefined mini-projects hosted here: https://github.com/BetterThanTomorrow/dram

pez08:05:18

It consists of a few ”drams” , where a ”dram” is such a mini-project + a dram.edn file with some config that tells Calva (or any app) what to do with them. Here's the ClojureScript Quick Start Browser dram: https://github.com/BetterThanTomorrow/dram/tree/published/drams/calva_cljs_browser_quick_start

pez08:05:51

I haven't figured the API for drams out fully, so to be more generally useful, we'd need to finish some of this work.

pez08:05:37

Anyway, that is a bit of derailing. My point is that between us we have the knowledge about which experience we want to create, what we can hope to succeed with, and how to do it. We also seem to have the will. This is important stuff!

pez08:05:05

I'd be happy to discuss on Zoom.

❤️ 1
seancorfield17:05:05

drams sounds a bit like what deps-new does? (I haven't looked at your stuff but an EDN file to describe part of it and some template files to make a project)

bringe18:05:49

Good stuff here. It seems like using deps-new with deps.clj.jar, maybe behind a Calva command for initializing a project with a deps-new template is a good approach.

borkdude18:05:44

Or maybe "execute clojure CLI command in current project" with a prompt where you can fill in:

-Tinstall deps-new :as ...
and then:
-Tdeps-new ....
? which uses the deps.clj uberjar

borkdude18:05:53

(pseudo invocations)

borkdude18:05:22

and/or perhaps Calva could fall back on the deps.clj uberjar if no clojure installation is found on the current system (perhaps under a config setting)

pez18:05:10

Prompting for project name makes sense. I have no idea what I should fill in to these prompts 😃 Not at all savvy with the clojure command. Is the first one installing the alias / tool with some name?

borkdude18:05:06

Yes, but my suggestion was to just let users fill in what they normally fill in on the command line, making it more flexible

pez18:05:03

@U04V70XH6 I guess drams are a bit similar to deps-new, but it is much, much simpler than that. There is no templating going on and the drams are targeted at Getting Started material.

bringe18:05:52

Some questions: 1. Is the intention to not require lein or clojure at all? 2. Do we want the user to be able to create a new project directory + project files within it from within VS Code (via a command, maybe), and then open this directory in VS Code? A new window?

pez18:05:44

Yes and yes. @brandon.ringe. That's very much where I am going.

👍 2
borkdude18:05:04

Perhaps adding some features that are offered by neil here could also be useful: https://github.com/babashka/neil • add dependencies • find dependencies, select and then add

borkdude18:05:20

but then from within VSCode

pez18:05:30

But I don't want it to be a general tool for creating projects. That would keep us occupied with a flood of request on all the things the tool doesn't do. I mainly want to smoothen the ”my first project” baby steps. So that you can get a feel and start liking Clojure enough to think the time to install some tools and figure things out is worth spending.

👍 1
bringe18:05:34

Well, seems we could have a command like “Create new Clojure project” that prompts for a project name, creates the project using the deps-new command @U04V70XH6 posted above (using deps.clj.jar), then open the project folder in VS Code (or open it in a new VS Code window if possible/necessary), so that they’re now in the same situation they’d be in if they using deps-new in the terminal themselves and ran code <new-project-dir>.

borkdude18:05:42

although editing a deps.edn shouldn't be too hard I guess

borkdude18:05:32

All of that can also be done with joyride (but that would require them to know Clojure) :)

pez18:05:57

Much like you describe, @brandon.ringe. Just framed well to make it clear it is just a Getting Started/newbie thing.

bringe18:05:13

Makes sense.

pez18:05:48

In s astudent setting users can be instructed to clone some repository, with Joyride scripts prepared.

bringe18:05:00

Open VS Code -> Install Calva -> Create new Clojure project -> Jack-in 😎

🎉 2
borkdude18:05:45

@pez wow, that seems a good idea if you don't want to bake stuff into Calva

bringe18:05:32

Yeah, all depends on what’s desired - my thoughts above are based on the assumption that the desire was to add functionality to Calva to create a new project easily.

borkdude18:05:44

Clone repo, install Calva, install joyride, Joyride -> execute script in workspace -> create new project -> BOOM

bringe18:05:09

That’s more steps than the above, lol, but avoids adding something to Calva (although I think that something is rather simple).

pez18:05:26

Calva can also bundle Joyride as part of its extensions pack.

seancorfield18:05:28

I don't know that Lee wants to have his students work with git and clone projects.

👍 1
bringe18:05:37

It seems that @U06BV1HCH would rather his students not have to clone some repo.

seancorfield18:05:51

Focus on keeping this as non-command-line and as free of tooling as possible.

👍 2
pez18:05:12

I'm not specifically catering to @U06BV1HCH here, to be clear. I'm thinking more generally. Hopefully creating something that Lee and other tutors and their students will benefit from. Calva wouldn't care if it was git, a zip file, a shared folder on a network or whatever.

seancorfield18:05:35

As shown, Calva could easily run deps-new which keeps the "new project template" stuff outside of Calva so there would be no burden on Calva. Calva could prompt for a default project template type (defaulted to some minimal one) and for a project/folder name (presumably even new-to-Clojure students would have some idea of a name for a folder) and that's all that is needed.

pez18:05:12

If we skip ”prompt for default template” I am on. 😃

seancorfield18:05:13

So this wouldn't be specific to Lee's use case but would still be focused on "zero install, zero command-line, zero tooling" which seems in keeping with Calva's "Getting Started" focus 🙂 Sure, yeah, that was just a thought about selecting an alternative starting point. After all, some folks might want an app, some might want a lib, and others -- newer folks -- might want an equivalent to the minimal scratch illustrated above.

👍 1
bringe18:05:02

I think a prompt for project name and a prompt for template, with a rational default to something minimal (so they could just hit Enter for that prompt without typing anything), is a good general approach.

pez18:05:10

Yes, I'd like to focus on the newer folks. People should use the command line (or some tool specializing on project creation) when they want something special, I think.

bringe18:05:24

Hm fair, that’s the decision to make here. Prompt for project template or not, but one more prompt opens up the experience to everyone. I don’t see much of a downside there.

pez18:05:31

Different templates has different options. I think it is a can of, not worms, but work. My warnings sensors are tingling. 😃

bringe18:05:40

Though, maybe not many experienced people would use this.

bringe18:05:54

Good point

bringe18:05:36

We can name the command “Create minimal Clojure project” or something and just keep it to one minimal template with predefined settings (if any are needed for the template).

👍 1
bringe18:05:05

And if someone requests other templates we can say “it’s just for a minimal template” 😄

seancorfield18:05:51

Make the default template a setting in Calva so folks can change it if they want but new folks can ignore it?

pez18:05:52

Exactly. We can add some more canned stuff later as well.

pez18:05:15

”Exactly” was to @brandon.ringe 😃

pez18:05:10

We should remove the support wheels quickly, I think.

pez18:05:07

The addition of a ”Copy Jack-in command line to clipboard” was a genius move, I think. (I dare say that because it was @brandon.ringe's idea, not mine 😃 ) We can do something similar here.

pez18:05:54

I think most of this fits into Calva and shouldn't require Joyride. Then Joyride could be use to extend, if we have some well designed API to it.

seancorfield18:05:56

I would use this quite a bit TBH for creating minimal projects in Calva to help me help beginners here on Slack, instead of what I do right now (switch to command-line, run deps-new, change into that folder, fire up Calva in that folder, jack-in...

pez18:05:01

Thing is you will still need to open the folder in VS Code.

seancorfield18:05:36

But you would stay in VS Code for that, right? That's an improvement 🙂

pez18:05:42

What I do there, @U04V70XH6 is to use the integrated terminal ... deps-new ... <project> && code <project> (pseudo invokation)

seancorfield18:05:37

And cross-platform so it works in cmd.exe as well?

borkdude18:05:48

This will require code for them to work but that should be the case once you've installed VSCode on Windows I think

borkdude18:05:25

We can just try and debug that. I've got a way too expensive Windows machine over here for debugging Windows problems

🪟 1
pez19:05:06

It's just a workflow. There are equivalent commands for cmd.exe I am sure. On Windows I always use deps.clj.jar beacuse I can't get the clojure tool to work. (So does Calva, actually).

seancorfield19:05:50

Yes, code <folder> in cmd.exe works -- I just tried it. First time I've opened a cmd.exe window in ages 🙂

metal 1
pez19:05:02

We could build this stuff so that it executes the commands in a an integrated terminal. That will make it transparent what is going on and be easier to debug.

👍 1
pez19:05:37

I've seen a Joyride script for that somewhere. 😃

borkdude19:05:33

There is also a menu in VSCode to install the code command line integration if it isn't there, you might have to document that, but I expect MS to cover that already on Windows

pez19:05:23

We should check if it is exposed as a command...

bringe19:05:16

There is this.

borkdude19:05:26

That's what I meant yes

👍 1
pez19:05:38

Yes. that one. If it is exposed as a VS Code command that we can execute via vscode/commands.executeCommand then we can prompt the user if they want it to be done for them as part of the process. > Calva needs code to be installed on your system, OK if we dp it now?

pez19:05:18

Then we open the terminal and create the project and code <path/to/project>.

pez19:05:40

That's of course the next question. Where should this be created?

seancorfield19:05:50

If you open VS Code from the Start menu, it just opens whatever workspace you had open the last time... so I think the "default directory" is also wherever VS Code thinks it was opened from?

seancorfield19:05:33

If you open a New Window for VS Code from Start menu, the default dir is your home dir (for me: C:\Users\seanc)

pez19:05:40

Should the project be created wherever Calva is opened? Maybe so. Maybe in <current-vscode-folder>/.calva/<project>

pez19:05:09

The user can always move it from there to wherever.

seancorfield19:05:02

Putting it in a dot-folder would be hard to non-tech people to find/move tho'?

borkdude19:05:32

Maybe open a file picker so the user can say where they want it?

pez19:05:50

Non-tech people hacking in Clojure?

seancorfield19:05:35

That's what we want, right? 🙂

👍 1
pez19:05:50

Yeah, file picker might be the way to go. If we can use it to select a folder without opening it.

👍 1
borkdude19:05:00

You can. I tried it in joyscript :)

borkdude19:05:16

euh joyride

pez19:05:04

Can you share the code here? I wanna try.

pez19:05:07

Someone that has troubles finding something in a dot-folder,, might find it very challenging to learn to code... Especially since VS Code does nothing to hide them. Just drag it out to where you want it.

pez19:05:39

But I certainly prefer asking the user to pick a location, like @U04V15CAJ suggests.

borkdude19:05:08

I don't have it anymore

pez19:05:31

code or it didn't happen! 😃 haha, just kidding.

pez19:05:14

That opens a file. But I'll try to adapt it...

seancorfield19:05:20

Don't underestimate the ability of non-IT people to learn Clojure. ClojureBridge was a real eye-opener for me in that regard.

❤️ 1
seancorfield19:05:56

Folks can learn to program without being able to do anything at the command-line and without understanding a lot of file system weirdness that we IT pros are so used to.

pez19:05:30

But surely they can just drag a folder from VS Code to where they want to have it? I mean, most things about Calva is more complicated than that.

borkdude19:05:03

Dragging a project is a bit cumbersome

pez19:05:54

Yeah, I don't want this. And shouldn't get stuck on it because we're going to avoid it. 😃

pez19:05:29

Indeed, we can prompt for the folder:

(ns open-folder
  (:require ["vscode" :as vscode]))

(def options
  #js
   {:canSelectMany false,
    :openLabel "Select",
    :canSelectFiles false,
    :canSelectFolders true})

(.then (vscode/window.showOpenDialog options)
       (fn [fileUri]
         (when (and fileUri (nth fileUri 0))
           (.log js/console
                 (+ "Selected file: "
                    (.-fsPath (nth fileUri 0)))))))
🎉

lspector20:05:19

So great to see all of this, and my biggest :thumbsup: to the fact that "not YET tech people" can learn Clojure and that it's a win/win all around if they can do it in a way that's clear and supportive, without requiring installing multiple things, learning other new things like git, etc. Also, I can report that these days there are some very sharp students who don't yet know anything about command lines or even about file systems, since OS makers have been hiding home directories, defaulting to storing things in the cloud, etc. Calva seems very close to providing the best experience, with this "create minimal project" thing being one of the few barriers.

🙏 1
1
lspector21:05:42

FWIW I know I will be facing strong headwinds with this comment based on prior conversations, but the other things on my list to improve the beginner experience would be to default to "original" rather than "strict" editing mode, and to make reformatting work with unbalanced expressions, the way it does in some other Lisp/Clojure editors. I know well the arguments in favor of structural editing, I can see the logic in requiring people to learn it first, and I can believe that works well for some newcomers. But I've never seen students so embarrassed and alienated and discouraged as when they can't even type or delete things. If original mode is default (with reformatting working for incomplete expressions) but strict mode is mentioned prominently in the getting started docs, then those who want to learn strict mode will be delighted to see it and will have a blast with it, while those who don't won't be befuddled or worse, and they'll have a smooth path to coding without having to learn new editing skills.

seancorfield21:05:00

Ironically, the most annoying bug for me right now with Calva is that accepting auto-complete sometimes (often?) overwrites closing parens and leaves me with unbalanced expressions... for me, that's the worst thing possible 😐

😰 1
😮 1
pez21:05:49

Never happens to me, @U04V70XH6. Please file an issue with a repro.

lspector21:05:05

FYI overly aggressive autocomplete is also a problem in Cursive sometimes. It doesn't change balance for me but if I type a newline at the end of a symbol it often autocompletes the symbol to something I don't want.

pez21:05:54

VS Code is super aggressive with its autocomplete. I constantly fight with it.

seancorfield21:05:05

If I could consistent repro, I would. It happens "all the time" in my work code but if I try to repro it always works the second time.

pez21:05:37

Anyway, @U06BV1HCH strict mode will stay default. And so far no-one has found the time to spend to make incomplete code format reasonably.

pez21:05:43

Worst kind of bug, @U04V70XH6.

lspector21:05:36

I figured, @pez. My getting started instructions will always show how to turn off strict mode.

pez21:05:27

Yeah, that hurts. But it is your prerogative. 😃

lspector21:05:30

I'm only basing my view of this on 30 years of teaching lisp.

borkdude21:05:20

I just gotta share my Clojure course material I taught 10 years ago, when I was a lecturer :) https://www.michielborkent.nl/clojurecursus/ - brings back memories

❤️ 1
lspector21:05:04

It seems to me that Joyride could be the way to get formatting of incomplete code without changing Calva. Am I thinking about that right?

pez21:05:40

Yeah, you could do that with Joyride. But seems roundabout. Better send a PR on Calva, I'd say.

lspector22:05:44

I was thinking Joyride because it's Clojure. I guess what's easiest will depend on whether there's available formatting code, what that's implemented in, and what the person trying to make it work knows. That's probably more likely to be Clojure than TypeScript on my end, e.g. if I hire a student to do this.

pez22:05:43

Most of Calvas formatter is written in ClojureScript,a actually. And it mostly some glue to cljfmt which is written in Clojure. The best place to contribute this, is actially to cljfmt. Then it would benefit many editor environments, not just Calva.

lspector22:05:05

But cljfmt doesn't handle incomplete expressions? There is definitely code around that does (since many editors have done it, and for example Cursive does), although I don't know what of that is available or what it's written in

borkdude22:05:03

cljfmt is based on rewrite-clj right? then you will run into this:

bb -e '(rewrite-clj.parser/parse-string "(1 2 3")'
Type:     clojure.lang.ExceptionInfo
Message:  Unexpected EOF. [at line 1, column 7]
I'm actually the co-maintainer of rewrite-clj together with lread. in clj-kondo I tweaked the (forked) rewrite-clj to allow incomplete expressions. We could do something similar in the original rewrite-clj to address this

pez22:05:04

Cool. So you should contribute to rewrite-clj and cljfmt, @U06BV1HCH. Much better than doing it in Joyride, where it would only benefit some few, at least until we've conquered the world with Joyride.

pez22:05:51

Once cljfmt supports this, we will be eager to make whatever fixes in Calva to utilize it. Those won't be easy or small either, but at least there would be a point to it when the formatting engine is supporting it.

lspector22:05:38

I've copied those last few comments, which I don't fully understand, and share them with a student I hire for the summer, in hopes that that student will be able to understand them and make the necessary contributions.

pez22:05:44

We will be eager to help your hired man, @U06BV1HCH 😃

pez22:05:39

Even if I suspect that this formatting magic is only needed because you refuse strict mode to help you. Haha.

lspector01:05:29

Thanks @pez. Whether I refuse it or not, strict mode is a serious barrier for many.

borkdude16:05:25

Here's another Clojure course taught at a university: https://egri-nagy.github.io/popbook/

lspector16:05:50

Thanks @U04V15CAJ -- that looks like a great course with some really nice materials that I will study.

borkdude08:05:06

This is also interesting. I asked that author what his setup was like: https://twitter.com/EgriNagy/status/1521276281664737280

👀 1
lspector19:05:19

Nice. For a period of time, Gorilla REPL was my preferred platform, both for teaching and for my own work. It has a lot of what I consider the key features, including re-indenting incomplete expressions 🙂 . The document at http://gorilla-repl.org/secrets.html is something I contributed to show how it could be used as a complete solution for editing and running code, which is what I did for a while. It's a little awkward because it's not really made for that use, and one has to be careful not to pollute files you edit with unwanted markup. So a real IDE is preferable, if it does the few essential things (which include formatting incomplete expressions) and doesn't drown you in incidental complexity. If I recall correctly, maria.cloud is great for initial playing, but doesn't let you do real work with projects etc.... although maybe I'm wrong about that.