Fork me on GitHub
#beginners
<
2024-04-06
>
lspector11:04:23

Asked in #C050AN6QW recently and elsewhere periodically but asking here because this is in part for the sake of #C053AK3F9: Does anyone know of a Clojure editor with these features?: - Near-zero learning curve, acting mostly like a generic GUI-based text editor that anyone will know how to use right away. Part of this is that it should be trivial for anyone to install. - Shows what brackets match what other brackets, maybe with rainbow coloring and/or highlighting when the cursor is placed by one partner of a pair, or anything else that shows what matches what. - Properly re-indents Clojure code upon request (maybe by hitting tab or some other key, or even a GUI button), even for incomplete expressions. It's fine if this breaks or goes somewhat haywire below the point where where the structure becomes nonsensical, but it should do the right thing from the top of the file (or the top of the current expression if that's easier) to wherever things get too weird. - Allows free-form editing so that people can use their existing typing/cutting/pasting skills without dealing with strict structural editing or anything else that prevents ordinary typing from working. Aside from the insertion/deletion of spaces when re-indentation is requested, it should not insert or delete any characters that the user didn't specifically type/delete. For example, it should not automatically type closing brackets. - Can be used by Mac and Windows users, either because there are native apps or because it is browser based. Other common Clojure editor/IDE features would be lovely as long as they don't interfere with the features listed above. Sadly, many otherwise fabulous Clojure editors/IDEs either have non-trivial learning curves or don't have good support for re-indenting incomplete code. If they could be made available without compromising the above-listed features then it would be lovely also to have an integrated REPL, integrated documentation, and maybe a some other bells and whistles that fancy Clojure IDEs have. But not at the expense of the features listed above, at least when one doesn't choose to use the fancier features. There have long been tools that meet these requirements in other Lisp environments, and for brief periods there have been in the Clojure ecosystem as well, e.g. something called Clooj that I loved until it was abandoned. If you know of anything fitting this description that exists now then please let me know! I will use it for my teaching and also for my own coding. If you don't know of one but would like to build it, I will be very happy to test it and provide feedback.

Ben Sless12:04:14

Sublime text?

👍 2
Noah Bogart12:04:17

Sublime? Especially with #C030F786GAD

👍 1
sublimetext 1
dpsutton13:04:10

Vs code is like this with the exception of the free form input. Maybe you can easily disable the paredit feature? Just as an aside, I find so much utility in not ever being in invalid states. Can’t imagine coding without paredit

👍 2
Mattias14:04:35

For me, it’s been Atom+Chlorine. Now Atom is replaced by Pulsar, but either way - the default environment is simple and understandable (well, Clojure setup with REPL and stuff…), and then you are free to load up on plugins.

lspector15:04:10

Thanks @UK0810AQ2 and @UEENNMX0T: I will definitely check out Sublime. I see there are a couple of options there, and not sure which if any will do the job I'm looking for, but that is first on my list. @U11BV7MTK: I've been using VSCode/Calva and one can indeed turn off the structural editing and auto-typing of closing brackets, etc., in a few not-too-painful steps, although re-indenting of incomplete expressions doesn't work reliably. I hear and appreciate your expression of love for paredit, but I really dislike it (though I've been writing Lisp for 40 years, so it's not for lack of appreciation of Lisp structure) and it's a real barrier for the population of students I work with. @Mattias: I had seen pointers to Atom-based setups along with indications that they had been discontinued, but did not realize they had been reborn in Pulsar. I will check that out too! Thanks!

👍 1
dpsutton15:04:58

I bet the calva community would be quite open to two issues you have here: easier to remove structural guard rails and formatting partial code. They are quite attentive to features that would benefit beginners and of course attentive to issues that might benefit everyone

lspector15:04:42

@U11BV7MTK I've conversed with @U0ETXRFEW and others in the Calva universe quite a bit and they are great and have supported a lot of my requests, but apparently it is not in their plans to support re-indenting incomplete code more than it already does (which is sometimes, but not other times, and it's hard for me to tell when it will or won't)

dpsutton15:04:33

Yeah. Does feel a bit of a moving target underneath you

lspector16:04:29

Also FWIW there's a fair bit of incidental complexity in figuring out Calva and turning off features we don't want... all of which I think I'd be fine with if the re-indenting thing worked correctly, but there's some sort of block there. I think it's because so many people are so attached to structural editing... but it's a non-starter for me and a nightmare in my classes.

pez16:04:19

Calva tries to help with formatting incomplete structures, but we can only spend so much time on it. In a sense all bets are off if the code has broken structure. As for structural editing, it is super easy to disable the guardrails. Doing so will take you to the “all bets are off” square more often.

lassemaatta17:04:03

What about parinfer instead of paredit? I found it a lot simpler years ago when learning. Not sure if calva or sublime support it though

pez19:04:26

I think Parinfer wants that either the structure or the indentation is correct. If so, it won’t be able to help with indenting broken structure. Calva does not have Parinfer, but there is a Parinfer extension for VS Code.

lspector20:04:24

Thanks but FWIW Parinfer is even worse in my context. I want students to know about and type the brackets. I want to type them myself too. Also, doing freeform editing (with cutting and pasting) in Parinfer can do crazy things. I don't want any kind of smarts (as in, not what one knows how to do from normal typing and editing in every other context) at all except re-indentation.

👍 1
Noah Bogart20:04:11

Sublime or pulsar is probably best then

❤️ 1
lspector13:04:16

Alas, I've now tried to set up both Sublime and Pulsar with what look like the right Clojure packages, but in neither can I figure out how to get it to re-indent an expression. Feel like an idiot... unless they don't do this. Does anybody use either of these, know how to get it to re-indent, and care to share a pointer?

Noah Bogart14:04:04

What’s re-indent?

lspector14:04:49

Re-indent = if the indentation is messed up, because one or more lines are indented too far to the left or right, then it fixes them, so that the expression conforms to standard Clojure indentation rules.

👍 1
pez14:04:51

If I recall correctly, Atom/Pulsar is a bit Emacsy in the tradition that you need to assemble and configure your choice of packages to do exactly what you want it to do. It could be that they lack an indentation package, but could also be that you haven’t found out.

lspector14:04:02

Thanks @U0ETXRFEW. I wonder if Sublime is easier to deal with, and somebody who knows it can tell me if it supports re-indentation, and if so then how to invoke it.

pez14:04:19

I think @U5GP9FMC0 and @U050UBKAA hack Clojure support for Sublime.

🙏 2
pez15:04:48

Here seems to be the part about indenting: https://tutkain.flowthing.me/#indenting-code

pez15:04:32

Generally I think the advice here https://tutkain.flowthing.me/#editing-code is very sound. Get used to one Paredit command – Expand Selection – and never have problems with incomplete code again.

flowthing16:04:23

There are no universally accepted Clojure indentation rules, unfortunately, but Tutkain (and Clojure Sublimed, I assume) implements the indentation rules described here: https://tonsky.me/blog/clojurefmt/ And yeah, @U0ETXRFEW already linked to the relevant instructions. Tutkain will probably not do a good job (or any kind of job, really) of indenting code with unbalanced parentheses, though.

flowthing16:04:21

On a more general note, I'm sure Tutkain's getting started experience is not as refined as it could be. It's difficult for me to say — I made the thing, so I'm somewhat unaware of the most significant pain points (and no one's informed me of them so far). If you decide to try to stick with it, though, I'd be happy to help over at #C030F786GAD.

flowthing16:04:30

It's possible that Clojure Sublimed does a better job at indenting code with unbalanced parens. It's worth a try, at least. I believe the instructions for that are here: https://github.com/tonsky/Clojure-Sublimed?tab=readme-ov-file#formatterindenter

1
pez16:04:26

If I understand things correctly, the exact indentation rules used are not very important. What’s important is that the code is indented at all.

pez16:04:59

I.e. Tonsky formatting should be just fine.

👍 1
lspector17:04:35

Thanks for all of the pointers here! I will study them all. In case it helps anyone to understand what I'm looking for here (I know @U0ETXRFEW already knows from another thread), I think Calva already does exactly the right thing if you start at the top of the file and replace every newline with a fresh one manually. That works even for incomplete structures. It's just that actually doing this manually can be a pretty big pain. If a student (or me!) is confused about why some code isn't working and you can say "press this one key to re-indent" then they will, and more than half the time the re-indentation will show a structural mistake that is the root of the problem. If you have to say instead "start at the top of the file and manually replace every newline" then it's a lot less likely to happen. I don't think the expand selection idea really addresses the concern, FWIW. I guess it's a different approach to seeing problems with structure, but re-indentation does it so much more elegantly and visually, in one step (assuming it works).

lspector17:04:58

Hmmm... my scenario above about the confused student sort of assumes complete code (if it's running but does the wrong thing), so maybe it's not the best illustration of my concern 🙂. For incomplete code, I think the more relevant thing is that many of my students and I want to write code from beginning to end, in which case when we're half-way through it is not a bug that it is incomplete, and we don't want a bunch of closing brackets trailing at the end because that's not how we compose any text including code. Or in other situations we might be planning for the tail end to come from pasting in code from elsewhere, or we may want to change the idea of part of the code in a complicated non-linear way, in which case there may be mangled structure at various points, maybe again from cutting and pasting, or maybe because we're trying to figure it out as we write it. We know how to write from beginning to end and also in non-linear ways by cutting and pasting -- we know how to do this from the rest of our digital lives, and our fingers and brains are pretty much hard-wired to do this. (If you happened to be watching while I edited this comment, you'll see that it's also how I edit Slack messages!) We'd like to be able to edit code in the same potentially non-linear way, without learning new ways to type (especially if we're already learning a lot of other new things, like about immutability and functional programming and whatever our higher-level goals are, usually AI in my courses). But then we'd like some assistance in seeing where the structure is messed up, often even before it is finished, which is what re-indenting has done in many Lisp editors for many decades. And it's what Calva does if you manually replace newlines one at a time from top to bottom. If there is any solution for doing this with any reasonable Clojure indentation conventions, in a setup that is relatively easy for newbies to access, then it will solve a big problem that my students and I have had for many years.

lspector17:04:26

Alas, from @U4ZDX466T's comment it looks like Tutkain will not do the job here.

lspector18:04:43

FWIW also I suspect that my code is structurally invalid upwards of 90% of the time as I'm writing it. It's just the way I write, and think as I write. Lots of cutting and pasting and rearranging of unbalanced things until I work it all out. That means that in Calva, for example, I spend a lot of time looking at incorrectly indented stuff until it becomes too painful and I go through and replace all of the newlines one at a time manually. One might say that I should learn to think and write differently... but I've been writing Lisp for 40 years and never gotten used to structural editing, so I don't think it's going to happen. And for my students who have been cutting and pasting for years and have a lot of other things on their minds, it's a big barrier as they're getting started even if they might get used to the idea of strict structural editing later. The good news is that there's a simple solution: provide a key command or button that re-indents from top to bottom, as many Lisp editors have done for years. The bad news is that I don't think there's a beginner-friendly Clojure editor that does this now.

Ben Sless18:04:56

One of the best habits to adopt working with lisps is structural editing. You can configure your editor to disallow "illegal" editing. Cutting and pasting as well can be done in expression level, rather than textual

1
Noah Bogart18:04:48

On the other hand, i didn’t use structural editing until 4.5 years into using Clojure full time. Learning clojure and learning structural editing are two separate skills

flowthing18:04:00

@U06BV1HCH Can you perhaps give one or two concrete before-and-after examples of how you’d like the re-indentation feature to work?

flowthing18:04:33

I just gave it a brief try and Tutkain (somewhat to my surprise, because I’ve never tested this) doesn’t completely fall over itself, but it’s unclear to me where the “things get too weird” barrier is.

flowthing18:04:30

That said, I don’t know that Sublime Text is necessarily the best option for students, because it is not free. It does have an unlimited evaluation period, but I imagine the nag dialog gets pretty annoying pretty quick.

Ben Sless19:04:54

Took me some time to adopt structural editing but ironically vim motions helped, copy with yW or y%, yank with d%, it just happened to work by happy accident

lspector19:04:01

@UK0810AQ2 I believe structural editing is great for some people. For this 40-year Lisper it isn't. And for my students who have never before seen Lisp, don't yet have any plan to use a Lisp full time, and are expected to be using a Lisp to experiment with AI ideas within 2 or 3 weeks of starting, developing a structural editing habit just can't be on the agenda. They are being exposed to a fire hose of new things, and I think that breaking their ability to even type is actually pretty cruel. I won't have them work in any environment that doesn't allow free-form editing. And since bracket matching and re-indentation are what make that work well, I want an environment that provides those things. For years I worked and taught in Common Lisp and our environments did this out of the box. I've been trying to find something comparable for Clojure for a decade or so.

Ben Sless19:04:14

If they don't work with something that enforces balanced parentheses, or really helps with figuring out how to fix them, they're going to have a pretty bad time.

lspector19:04:28

@U4ZDX466T here's one that Calva doesn't handle correctly now:

(defn foo
  [x]
  (if (> x 100)
(let [blink 2]
      x)
    (map #(* x %)
Hitting tab within that changes nothing, but it should change it to:
(defn foo
  [x]
  (if (> x 100)
    (let [blink 2]
      x)
    (map #(* x %)
In the current Calva tab sometimes does the right thing, but sometimes doesn't. I haven't figured out how to tell when it will and when it won't, and although situations in which it doesn't arise in my coding frequently, it actually took some fiddling to find a relatively small example. Again, Calva does handle this correctly if you manually replace each newline, one at a time, from the top down. But there's no way to make it do that without the line by line manual process.

Ben Sless19:04:36

I believe modern plugins can help, others who are more familiar with sublime or vscode can probably give better advice. What sort of environment did you use for CL?

lspector19:04:08

@UK0810AQ2 I absolutely agree that they're going to have a pretty bad time if they don't have a way to figure out what the structure is. That's why I care so much about this. Re-indentation is a super useful tool for doing exactly that. It solves the problem beautifully if it is provided. The CL environment I used most was Macintosh Common Lisp. (Not cross platform, but those were different times and lots of students worked in our lab which was full of macs.) MCL had an editor called FRED (Fred Resembles Emacs Deliberately) which was a thing of beauty. Zero learning curve because it acted just like any other GUI text editor on a Mac, but it had key features like bracket matching, re-indentation, and sending expressions to the REPL built in. Also, installation was just drag it to your Application folder and you're done. I really pine for those days! BTW the "resembles emacs" part was that FRED was programmable like emacs, but you didn't need to learn key commands or anything else to use it, since all of the basic functionality was available through mac-standard GUI elements including menus.

lspector19:04:41

@U4ZDX466T on things getting too weird, what I mean is that if you take any properly structured file, and mangle things however you want starting at line n, but everything above line n is fine, then the re-indentation function should make all of the indentation correct at least from the top down to line n. You should be able to manually mess up indentation above line n by adding and deleting spaces and then invoke the re-indentation function to fix it all, down to line n. I don't really care what happens after line n. This is exactly what you get in Calva, I think, if you manually replace every newline with a new one. I just want it to happen all at once when invoked once.

pez19:04:51

Actually, Calva fails to format that example even when the structure is fixed…

😱 1
pez19:04:06

Seems to be cljfmt that think it is fine as it is:

❯ cljfmt check - << EOF
(defn foo
  [x]
  (if (> x 100)
    (let [blink 2]
         x)  
    (map #(* x %))))
EOF
All source files formatted correctly

lspector19:04:27

Wait but that is okay, right? In the one that's wrong the let line is all the way on the left margin.

pez19:04:21

No, it was my test project with weird settings… If I do this same thing in /tmp I get:

❯ cljfmt check - << EOF
(defn foo
  [x]
  (if (> x 100)
    (let [blink 2]
         x)
    (map #(* x %))))
EOF
STDIN has incorrect formatting
--- a/STDIN
+++ b/STDIN
@@ -2,5 +2,5 @@
   [x]
   (if (> x 100)
     (let [blink 2]
-         x)
+      x)
     (map #(* x %))))
1 file(s) formatted incorrectly
As we are expecting.

pez20:04:27

But then, if I unweird my settings, and paste the text, I get:

(defn foo
  [x]
  (if (> x 100)
    (let [blink 2]
      x)
    (map #(* x %))
So with that particular example, Calva actually handles it, I would say.

lspector20:04:26

Oh I see there's a different issue in your example, with the x) being indented too far. Can you try my example in which (let [blink 2] is all the way on the left margin?

pez20:04:36

There are an infinite amount of examples where it won’t work, of course. Calva is not doing anything very fancy.

lspector20:04:07

But Calva will do it all correctly if I replace newlines manually from the top down, so it knows what to do in every case, I think.

pez20:04:19

Not the part of Calva that needs to know it. As we have discussed.

👍 1
pez20:04:08

This text:

(defn foo
[x]
(if (> x 100)
(let [blink 2]
x)
(map #(* x %))
paste
(defn foo
  [x]
  (if (> x 100)
    (let [blink 2]
      x)
    (map #(* x %))

lspector20:04:18

Yes, we have discussed this. I don't fully understand the different parts of Calva and how hard it is to get them to talk to each other, but I know that you do.

lspector20:04:39

On your paste example, yes, that works, and also for the one I originally posted. I wonder if the answer to this long-running issue is just this: To re-indent code whether it is complete or not, select all, cut, and paste. That works with this example, and it's pretty straightforward. Do you think it would always work the same as replacing all newlines one at a time?

lspector20:04:42

If that really works then I will be super happy! I will just include it in bold in all instructions I provide.

pez20:04:57

It could work similarly. The code for healing the structure is using much of the same infra-structure as the code that indents when you press enter.

lspector20:04:36

Oh that's wonderful! I will have to try it when I'm actually coding and in one of those messy situations, to see if it does the job then and not just in the contrived example here.

pez20:04:37

But It is not the same code at work at the edges, so there are probably cases where one works and not the other.

pez20:04:05

I think it will sometimes work and sometimes not.

pez20:04:50

Broken Lisp structure, all bets are off. That’s the way it is. ¯\(ツ)

lspector20:04:33

But the manual replacement of newlines seems to always do the right thing. So bets aren't off there, right? It's just tedious, right?

pez20:04:05

I think it sometimes works and sometimes not. Just sometimes works more often, maybe.

lspector20:04:20

I've never seen that not work, FWIW.

pez20:04:15

I’ve written the code for it. And it’s not exactly an LLM. 😃 All bets are off 😃 But would be interesting to learn about cases when paste does the wrong thing and indenting stuff line by line does the right thing. That may give me clues how to improve the structure-healing.

Noah Bogart20:04:17

Interesting. I’m pretty sure Vim handles this out of the box

pez20:04:17

I’m no expert, but for me vim does nothing with it when I paste it.

lspector20:04:54

Unfortunately vim isn't really beginner friendly, so not a great option even if it does reindation

Noah Bogart20:04:47

Yeah it’s not, I’m just surprised

flowthing20:04:19

Thanks for the example. I don't think having e.g. Tutkain support that sort of thing would be unreasonably difficult (a couple of small changes seemed to be enough). Tutkain's current indentation implementation is very much per-form rather than per-line, though, so the "format all the lines above line n" scenario would be a bit more work, I think.

Noah Bogart20:04:21

Vim will paste text as given but if you reindent the “paragraph” (from last blank line to next blank line”, it should correctly indent it

flowthing20:04:41

The number of corner cases and tests, though...

pez20:04:19

I have no clue how to reindindent a paragraph in vim 😃

pez20:04:44

In Calva, out-of-the box. I can paste it and it reindents. Or I can place my cursor in it:

(defn foo|
[x]
(if (> x 100)
(let [blink 2]
x)
(map #(* x %))
Hit tab, and it reindents.

flowthing20:04:10

I suppose at the end of the day I'm not particularly interested in encouraging non-structural editing, though. I gave it a quick try, and Clojure Sublimed seems to do a better job of it, so if Sublime Text not being free does not qualify it, you could maybe give that a try. I seem to remember the author of Clojure Sublimed writes Clojure without any structural editing support, so I imagine it supports that sort of thing quite well.

pez20:04:47

I can also select the code, and do Format Selection.

pez20:04:42

It’s a bit the same here. Non-structural editing is nothing that I want to encourage. So I hardening the healer is not high on my priority list. Still, it’s there and it will quite often work.

lspector20:04:47

Thanks @flowting but now that you remind me that Sublime isn't free, which you also mentioned above, I realize that although it would work for me personally it would be a problem for some students and not the best approach here. Thanks for considering supporting this though!

👍 1
lspector20:04:34

I understand that many of you consider strict structural editing to be the only way to go, but it just doesn't work for me personally and I've seen it make students cry so I'm going to avoid it at all costs.

lspector20:04:41

@U0ETXRFEW on the example above, if you have it indented properly (because paste fixed it or whatever) and then you manually delete all of the spaces at the beginning of the line with let, then tab doesn't fix it, right?

pez20:04:22

Calva needs a bit context. Delete all the spaces, and place the cursor outside the (if and hit tab. Fixes it.

lspector20:04:27

Huh, that does work for me too. But it means that you have to know where to click to make it work, right? And if you didn't somehow know where to do that then you may think it is indented correctly but it isn't...

pez20:04:31

¯\(ツ)

lspector20:04:18

I'm really hoping that "select all, cut and paste" actually works more reliably than you think it might. That's a great solution if it works.

Noah Bogart20:04:57

What are students copy and pasting?

pez20:04:01

Well, you were the one who asked for it, so I added it. 😃

❤️ 2
lspector20:04:19

@UEENNMX0T the idea would be for students to select/cut/paste the full contents of their file if they are trying to track down a bug or if they're writing something and having a hard time seeing the structure of the code if the indentation may be messed up.

pez20:04:28

@U06BV1HCH, since Selecting the text and do Format Selection works, maybe that’s what you can instruct students to do?

pez20:04:16

Select All -> Format Selection -> BOOM.

1
lspector20:04:53

There seem to be a bunch of Calva instructions containing "format" but I don't see any that look that simple... Maybe "Calva Format: Format Current Form"?

lspector20:04:53

OH I just found it!

pez20:04:59

It’s a VS Code command. Default binding is alt+shift+f, I think.

lspector20:04:04

It doesn't have calva in the name

pez20:04:10

Though, the healer might actually like it better if only the offending code is selected. It works from the edges of the code it heals. And if the broken structure is in the middle of things, I don’t think it will work.

pez20:04:50

Not Calva in the name, because it is a VS Code command. But it’s Calva that provides it.

lspector20:04:41

Hmm... I now see it. It will be great if this works, although I see you are a bit skeptical...

pez20:04:14

Mostly about the Select All.

lspector20:04:45

I will have to give this both "select all, cut, paste" and "select all, Format Selection" a try in actual practice.

pez20:04:07

Those will work the same.

lspector20:04:51

Ah -- okay -- good to know. I will try with actual code in the coming weeks and I hope this will be reliable enough to use and to have my students use.

lspector20:04:13

Must sign off for a bit now, but really appreciate the attention all of you have given this.

pez20:04:19

What I mean is that this will probably not work:

(fine code)

(broken code

(fine code)

lspector20:04:49

It seems to handle that correctly! Remember that I don't care what happens on lines below the first broken line. If I start with:

(fine code)

(defn foo|
  [x]
  (if (> x 100)
(let [blink 2]
  x)
    (map #(* x %)

(fine code)
and do "select all, cut, paste" (which is best because everyone knows how to do it without knowing how to find or invoke a VS Code command) then it ends up looking like this, which is 100% okay by me:
(fine code)

(defn foo|
  [x]
  (if (> x 100)
    (let [blink 2]
      x)
    (map #(* x %)

         (fine code)
This is great. Everything down to the line with map is re-indented correctly.

lspector20:04:46

Maybe "select all, cut, paste" will really be the solution I've been looking for for so long. THANK YOU!! (and really must sign off for a bit now, but will check back in 10 hrs or so)

🙏 2
lspector07:04:57

After a rest I want to share 2 more things here: 1) I am so delighted that "select all, cut, paste" in Calva may solve this problem that has been bedeviling me and my students for many years!! Thank you a million times for doing whatever it is that makes this work (if it does, but so far it looks good!). 2) While it is of course fine for anyone to say they don't want to support unstructured editing in a tool they are providing to the world (especially for free!!), they should not fool themselves into thinking that they are welcoming beginners if they do this. Beginners will almost certainly want to move parentheses (for example) because they haven't yet fully appreciated the significance of their placement, and when adding a "(" also adds a ")" where they don't want it, and trying to delete a ")" just beeps, OMG that will be frustrating and embarrassing, especially if they are doing it in front of a classmate or professor. Telling them (which the software doesn't even do directly -- it just won't let them do basic editing) that they either have to work in some kind of unsupported idiot mode or learn an entirely new way of typing and editing is not helpful, unless what you are really trying to do is to say "That's right, you really don't belong here." Yes, there will be some beginners (maybe this was you?) for whom hitting that wall will be no big deal or maybe even a pleasant challenge. And for some coders (but not this one) strict structural editing is wonderful or even indispensable. But I can guarantee you that for many beginners it is a slap in the face, especially if they are members of underrepresented groups who are already questioning whether they belong here at all. They are trying to grapple with a strange new syntax and new concepts like immutability, and then when they try to make their code look like the example in the book the editor won't even let them type. "Are you kidding me?" will be a perfectly reasonable response, and they may very well take this as a signal that they should run away and never look back. Again, I can't emphasize enough how grateful I am to tool builders in the Clojure community! But if you really want to support beginners, and not just the specific kind of beginner that you yourself perhaps once were, then it's important to consider their perspectives. IMO it doesn't take much to support a much wider class of beginners: really just free form editing, bracket matching, and re-indentation of incomplete code (down to whatever line breaks the structure, which is where the student will need to look to fix things). All of the other fancy features of Clojure IDEs will be helpful in time, but if the student isn't allowed to type/cut/paste with their existing skills, or to see the structure of what they've typed via bracket matching and re-indentation, then they aren't being supported.

👍 1
flowthing08:04:24

(I know it's probably a distinction without a difference, but for what it's worth, if you try to delete a close paren, ParEdit moves your cursor to the left of the paren, rather than beeping.) I appreciate your point that users who are experienced with structural editing probably suffer from the curse of knowledge in this regard. However, I don't think anyone is trying to gatekeep here. To me, at least, the risk of prospective new users being driven away by having to manually balance parentheses seems greater than the risk of them giving up when their editing environment prevents them from typing invalid S-expressions. At my previous job, I was involved in the process of bringing in three people brand new to Clojure. Two of them used tools that managed parentheses for them automatically, the third one didn't. Watching them work, it seemed to me that the third user had significantly more trouble working with their code than the other two, precisely because of having to balance parens manually. That is not to discount the sort of experience you're describing. However, at least for me, it would be helpful to see in practice the sorts of things they're struggling with. Perhaps there are other ways than better support for incomplete S-expressions of easing their journey. Furthermore, I do think supporting re-indentation of code with syntax errors is not a trivial task. For example, given the scenario of indenting a form with a missing close paren, at which point should the editor give up looking for the close paren? In this scenario, if the form is toward the beginning of a large buffer, the editor may hang for a significant time while looking for the close paren. There's also quite a number of other kinds of possible syntax errors to account for, in my mind.

flowthing08:04:14

Also, sending (complete) S-expressions from your editor to the Clojure runtime is, to me, absolutely central to Clojure programming. It is what initially drew me to Clojure, and a big part of what keeps me here. It seems to me that Clojure editing tools ought to do everything they can to support it. Prioritizing structural editing is one good way of doing that, I feel.

pez08:04:32

I think you are very, very wrong about structural editing, @U06BV1HCH. It’s a super power of Lisp, and benefits the experienced along with the newcomer. Your observations about how it fares out there are probably very tainted by your own dislike for it. Calva’s strict mode is very gentle. And it only affects backspace and delete when brackets are involved. There are no beeps. If Calva thinks the bracket is balanced it will skip deleting it and instead move the cursor. If Calva thinks the bracket is not balanced it will delete it as normal. To force delete, there is alt+backspace. Instructing a user about the strict mode behaviour is to my experience the only thing necessary for them to go on editing Clojure code without this causing them problems. (In Calva support there is only ever one person who have complained about strict mode, and that is you, who do not even use it.) The adding of balancing parens is not part of strict mode. It is standard VS Code behaviour, and standard in any modern code editor. If you select the code you want to wrap in parens first, that is what will happen. Because this is standard in modern editors this is what many people will expect will happen, so expectations have changed in the last couple of decades regarding this. If it is the first time someone uses a code editor, then it will be a slight surprise, but shouldn’t be too alarming that a code editor does not behave like a word processor. What Paredit brings to the table is a way to quickly get that closing bracket in place, for cases when you have created something like (|)code I wanted to wrap. But this is on demand, and will not be in the way for anyone not using it. These are my suggestions for how to help someone get started with editing Clojure code using Calva: 1. Inform about strict mode, and recommend about keeping it enabled. a. Inform about force-delete 2. Inform about that selecting code is best done with Expand Selection (a standard VS Code command, that is enhanced for Clojure code in Calva) 3. Inform about that parens are inserted as pairs in modern code editors, and recommend to keep it enabled a. Inform about that selecting code first and then inserting an opening paren will wrap the selected code. 4. Let them know about that there are many more structural editing commands at their disposal, should they want to expand on the Expand Selection power they have just learnt about.

lspector09:04:47

Thanks for your perspectives and work on this, @U4ZDX466T and @U0ETXRFEW. First let me say that I definitely do not think that you or anyone else in the community is intentionally gatekeeping! In fact I know that you and many others are trying to be welcoming, for which I am thankful. Second, I agree that having to "manually" manage parentheses is a nightmare if there is no bracket matching and re-indentation, but those features solve the problem quite elegantly, in a way that doesn't require the user to learn any new skills or mental model of typing or editing. Third, I don't believe that there are non-trivial coding problems involved in re-indenting from the top down (and stopping when the structure is broken or incomplete), because many editors have done what I'm asking for for decades (since my first Lisp coding in the early 80s at least), What happens once one gets to a difficult mess is irrelevant. In fact, if it remains a mess then that's a good clue that that's where attention is needed. As long as the correct indentation is applied from the top of the file (or selection) until something is incomplete or otherwise wrong, mission accomplished. Finally I will say that we are probably talking about different classes of beginners. Yours may mostly be professional programmers or people who know they want to be professional programmers, and they may be willing to learn new typing habits because of that. They may also just be self-selected to be people who won't mind this. Mine might be biology or art majors who signed up for my course (along with a full schedule of courses having nothing to do with programming) because they are intrigued by the concepts of evolutionary computation. They've taken one course that used Python or R or Java and I'm going to make them learn Clojure because I think that's the best language for doing what we'll be doing. They're going to have a steep climb to learn a lot of things they didn't know they had signed up for (functional programming, immutability, etc.), and they will actually be fine with this, but having to learn new ways to type just to get started may be a bridge too far. I've taught hundreds such students Lisps over the last 32 years, and sadly watched some students quit in frustration or secretly edit their code in Word when I wasn't looking. It's true that I dislike strict structural editing myself too, and will never willing work in such an environment, but these experiences of students are not things I am making up. While nobody else may be sharing these experiences in Clojure communities, that may be because only people who have made it through the (unintentionally-built) gate are here, and/or because those here who teach only teach people who are or know they want to be professional programmers. Really finally: Sorry I got the story wrong on the beeping! That must have been some other strict structural editing environment. But as @U4ZDX466T said, for me this really is a distinction without a difference. Hitting backspace should do what hitting backspace does in nearly every other editing context that students have used: delete the character before the cursor, or the entire selection if there is one. Doing anything else is a problem, even if there's some way that one could possibly learn to make backspace (and all other typing/cutting/pasting) behave normally.

lspector09:04:03

BTW I see students frustrated with the autotyping of closing brackets and quotes in other languages too! IMO this is an unnecessary pain point for first time programmers, and a bug rather than a feature if you want to welcome new programmers. It's a lovely feature for people to be able to turn on if they want it, but I don't think it should be on by default. When I teach an intro programming course in Java the students curse it, and I do too.

pez10:04:40

Let me return to the Select all plan. I have understood that you do not care about what happens after the broken structure. Going from that it behaves like you want from the single example here is deceiving. The way it works in Calva is not like your ideal re-indenter works, so it will work less times doing Select all than when done on the suspected broken code. So I advice to select the top level form and do the cut/paste instead.

👍 1
pez10:04:41

As for the ideal indenter, I think it is better done in a separate extension than in Calva. Mostly because the Calva infra-structure is not created with this use-case in mind. Some simpler parser that really only cares about brackets is probably a better way. I’m not going to write this extension, but if it is simple enough, I can offer to maintain it and keep it published on the VS Code marketplace.

pez10:04:19

For when Calva’s healer may fail, there is one more feature that could be used. There’s a command Calva Format: Infer Parens (from the indentation). It works on the whole file, so can mess things up in places where you might not be looking, so a safer way to use it is to: 1. Select the code you want to have fixed 2. cmd/ctrl+n to open a new Untitled document 3. Paste the code 4. If VS Code does not automatically set the language to Clojure for the untitled document: Set it manually (cmd/ctrl+k m) 5. Indent the code as you think it should be indented 6. Calva Format: Infer Parens (from the indentation) 7. Copy the fixed code and paste it back in the original document

flowthing10:04:55

A couple of points: • You brought up bracket highlighting. Is there some aspect of that you find unsatisfying e.g. in Calva? I believe most modern editors support that out of the box. (Apologies if you mentioned it above and I missed it, there's quite a lot to read in this thread.) • Inspired by Clooj, I think Tutkain could do a better job at highlighting an unbalanced open paren. I'll take a look at whether that can be done. • I gave Clooj a quick try to better understand what you're looking for. I still don't think it's trivial to implement re-indentation in a performant way in the general case, but I now have a better understanding of the kind of thing you're looking for. I only gave it a very brief try, but I think Clooj (or something like it) could be a very useful starter tool for prospective Clojure developers. It is immaterial to me which editor anyone uses for Clojure, but I do care about extending Clojure's use base. If you believe Clooj could be a useful tool for you, I can take a look at forking it and making it run under newer Java versions, but I can't offer continued maintenance for it. Perhaps it is something clj-commons would accept under its umbrella if it turns out to be a useful tool in a classroom setting.

lspector11:04:15

Thanks so much. A few replies: • "Select the definition/expression you're working on, cut, paste" is just as good as "select all, cut, paste" I think. So that should be my instruction if I understand correctly. • Separate extension: Could be fine. I think re-indentation is so necessary that I could imagine teaching and working using only the CLI, a generic plain text editor, and a web page with a single field in which you could paste code and press a button to fix the indentation (from the top down until it fails). Much better to have all the bells and whistles of Calva, of course, assuming there's a re-indentation solution for incomplete or messed up code. • Inferring parens is so thoroughly backwards from the way I think about coding and that I want students to think about coding that I can't express how wrong it feels to me in words 😱. And aside from that there seem to be some pretty hairy complications in following the related suggestion you describe above. I appreciate the lengths to which you have gone to think it through and explain it! But I am pretty confident it is the wrong path for me and my students. • Bracket highlighting in Calva (and many other Clojure editors) is great! No complaints! I only mention it because it is really the only other true necessity for a Clojure editor IMO, aside from allowing free-form editing and simple re-indenting of possibly-incomplete/malformed code. • Clooj was great not only because it met my needs for minimal Clojure editing features (as far as I recall -- it has been a while!) but also because there was zero setup/config or learning curve. If I recall correctly you downloaded a thing, double clicked on it, and could immediately write code (with freeform editing, bracket matching and re-indentation) and evaluate it too. Lovely. That's extremely welcoming to newcomers. If something like this could be made to work in a modern environment then that would be lovely. Some of us might just work in it forever, while some might take the plunge into more feature-rich tools with more of a learning curve.

flowthing11:04:21

I made the bare minimum changes to make Clooj run on modern JVMs (on Mac and Ubuntu, at least, which is all I have access to). If you're interested, you can try it out like this:

$ git clone 
$ git checkout develop
$ lein run

❤️ 2
🎉 1
flowthing11:04:18

If that is something you'd find useful, I can work on it some more (e.g. make a GitHub release so that folks can just download the JAR file and double-click on it as the README advertises etc.)

lspector11:04:19

Oooo thank you @U4ZDX466T! Must run again (I am supposedly on holiday and I'm getting some glares from my wife) but I will try it ASAP! (Oh one little thing is that installing lein is itself a barrier for some students, especially on Windows... but if this basically works then maybe there will be a way around that... oh I see that maybe the comment you just posted about a JAR file will do that...). More later and thanks again!

flowthing11:04:25

Sure thing. There's no getting around having to install a JDK and Clojure CLI / Leiningen anyway, I don't think.

pez11:04:39

Packaging it as a native binary gets around it, but that may be a lot of work.

flowthing11:04:23

Hmm, will the REPL work in that case?

flowthing11:04:37

I haven't done much GraalVM work, so it might. 🤷

pez11:04:23

@U06BV1HCH about separate extension. The instruction would be Install Calva, and Lisp-Reindenter. Where the latter only provides one command: Reindent file.

pez11:04:56

I don’t know what Clooj is, thought it was an editor. 😃

flowthing11:04:23

It is, but a Clojure editor needs a runtime to connect to. 🙂

Noah Bogart14:04:15

I’ve never heard of clooj, but this is cool!

lspector16:04:35

I do think one has to install JDK, but leiningen isn't necessary in some contexts. FWIW I think that Cursive will install a JDK for you and doesn't require anything else. And I think CLI presents fewer barriers than lein, which completely stumped some of my Windows students and I couldn't help because I don't understand Windows. @U0ETXRFEW: Installing Calva + one more thing in VS Code sounds completely manageable.

pez16:04:09

The Clojure CLI can be messy to install on Windows. What I do is that I download deps.clj from https://github.com/borkdude/deps.clj/releases and rename the binary from deps.exe -> clojure.exe and place that somewhere on the binary path.

flowthing16:04:10

Yes, Intellij IDEA comes bundled with a JDK, but you still need one. You also need some way of starting a Clojure runtime, be it Leiningen or the Clojure CLI.

pez16:04:59

Regarding infer parens, @U06BV1HCH. It was for when all else fails. Which may or may not happen. But now you know.

👍 1
lspector16:04:17

Thanks for that re: installing CLI in Windows @U0ETXRFEW. I will make a note of that for my students in the fall. It would be fabulous if the CLI main page included that pointer or something equivalent, although I gather that none of us can make that happen.

pez16:04:53

I think the plan is to make deps.clj the new clojure cli.

lspector16:04:50

@U4ZDX466T yes, I gather that that is the case. FWIW now that CLI exists and seems to be well supported and less of an installation headache than lein, I hope that that will suffice.

flowthing16:04:19

Yeah, could have a look at how difficult it would be to replace the Lein-specific bits of Clooj with CLI.

flowthing16:04:17

> I think the plan is to make deps.clj the new clojure cli. The core team is planning that? I didn't know, that's interesting.

pez16:04:55

I hope I haven’t just dreamt it up.

pez16:04:50

Here is at least some trace of that I haven’t been dreaming this up: https://ask.clojure.org/index.php/12901/installing-clojure

👍 1
pez18:04:26

And in #CFN4QDHPS there are quite a few messages from Alex where he says he’s endorsing the MSI install (which is based on deps.clj).

1
borkdude08:04:26

Ah yes. It would also be possible for clooj to use deps.clj and have it not download anything on startup, just refer to a bundled tools jar. Cursive is now also using deps.clj

1
😍 1
borkdude08:04:01

Ping me in #C05ND742UAC if you need help

flowthing08:04:46

Oh, that’s a great idea, thanks! I’ll look into that.

ChillPillzKillzBillz16:04:53

Hello All, I have a quick question about performing fourier transform in clojure. 1. Is clojure an appropriate language to perform intensive maths on large data sets? 2. Is there any available math library which provides higher math funciton libraries? 3. Is there any example of using JWave with clojure? If anybody has any idea about this I'll be grateful for some pointers... Many thanks!!

Ben Sless19:04:28

I'd look into one of the Clojure matrix libraries, either Neanderthal or core.matrix. For higher order math operations I'd probably use Emmy (port of sicm utils) No idea regarding jwave

genmeblog12:04:43

@U022LK3L4JJ what do you mean by "higher math functions`?

genmeblog12:04:22

Regarding FFT I plan to add com.github.wendykierp/JTransforms in future. Here is the example of usage FFT from JTransforms: https://github.com/genmeblog/soundsynth/blob/master/src/sound/waveforms.clj#L24-L41

ChillPillzKillzBillz12:04:14

Hello @U1EP3BZ3Q I am sorry didn't see your messages till now. By higher math functions I mean functionality like what is offered by numpy.

ChillPillzKillzBillz12:04:32

I'll checkout JTransforms

genmeblog12:04:14

JTransforms will be soon added to a fastmath 3 (there will be a snapshot within a week or two)

🙌 1
alpox20:04:40

I did think that filter is lazy. Can someone explain me why (take 100 (filter #(< % 10) (range))) hangs (respectively probably tries to evaluate the whole range)? (Tested on clojure 1.11.1)

1
Bob B20:04:40

there are only 10 values that match the pred, so clojure is still trying to find another 90

2
alpox21:04:21

Oh dear me, how could I miss that 😅 should not program with temperature… Thanks for the hint @U013JFLRFS8

👍 1
alpox21:04:05

I see I was actually looking for the functionality of take-while

👍 1
adi10:04:03

The expression is lazy. We can bind it to a foo in the REPL.

user=> (def foo (take 100 (filter #(< % 10) (range))))
foo

user=> (take 10 foo) ; returns immediately
(0 1 2 3 4 5 6 7 8 9)

user=> foo ; blocks indefinitely
user=> (take 11 foo) ; also blocks indefinitely
Evaluating foo in the REPL will cause the whole expression to evaluate fully, i.e. evaluation is "eager" in the REPL context. Since the filter for pred #(< % 10) will never find more than 10 items in (range), any ask for 11 or more items will force (range) to evaluate fully.