Fork me on GitHub
#clojure
<
2021-06-18
>
Adam Kalisz00:06:00

Perhaps an update of this talk would be in order 😉 Some performance tips on ClojureScript would also be great.

ghadi00:06:53

The unrolled vecs/maps were a case of bad benchmarking

ghadi00:06:32

Something that sounded good but didn’t prove out in the wild

Adam Kalisz00:06:45

Ok, seems to be still half open/ there is no closing words for that proposal if I understand it correctly.

ghadi00:06:57

IIRC, the benchmarks were oriented around monomorphic calls, which HotSpot excels at optimizing, but the increased # of classes seen at critical call sites (e.g. RT/assoc) turned those call sites megamorphic. Those kinds of call sites have the worst performance

ghadi00:06:07

microbenchmarks stress monomorphism... real world programs have more dynamism

ghadi00:06:30

It would be nice to close those tickets with a nice summary

ghadi00:06:31

some really good analysis of a similar failed optimization effort here https://github.com/google/guava/issues/1268 @adam.kalisz

ghadi00:06:58

> Guava ImmutableList (and others) offer awful performance in some cases due to size-optmized specializations

Adam Kalisz00:06:33

Thanks! I have added a comment below the YouTube talk. I would appreciate a summary, these tickets leave a sour taste when the case wasn't so clear after all. 🙂

Alex Miller (Clojure team)00:06:18

I don’t think the idea is dead. Rich would still love to have tuples. They just need to be performant. There are lots of new tools in Java since this work that may offer new choices

👍 3
seancorfield00:06:50

@adam.kalisz I think Rich’s comments in CLJ-1517 are pretty clear on the performance problems (but CLJ-1610 could benefit from a comment at least pointing to why CLJ-1517's original approach wasn’t accepted).

👍 3
seancorfield00:06:29

(and it’s nice to see the link to the Guava issue — in Colin’s comment on CLJ-1517 — I had stopped following the issue by that point I think so it’s very interesting to read about that failed optimization too!)

Alex Miller (Clojure team)00:06:42

It was weirdly contemporaneous iirc

ghadi02:06:56

I don't think anyone has tried v8-style or SELF-style hidden classes in Clojure

ghadi02:06:13

(such a map optimization would still have to defeat the megamorphic issue mentioned above)

Ovidiu Stoica02:06:55

In learning clojure, what is the ONE thing that if you practice regularly, you will become a a great clojure programmer?

seancorfield02:06:19

You might get better responses in #beginners since folks there have opted in to helping/teaching new folks…

seancorfield02:06:59

But, to answer your question: REPL-Driven Development.

18
seancorfield02:06:33

Always work in your editor, never type into the REPL. Always evaluate every change you make as you make it (you don’t even need to save the file!). Use Rich Comment Forms (i.e., (comment ..)) for all of your “scratch” ideas and exploration — and usually keep it in place in the final code so you can see how you got there. Be able to run tests via the REPL from your editor, so you’re not switching contexts. Develop good REPL hygiene so you can keep a REPL running for days (or weeks, or even months) without restarts or “refresh” tooling.

thanks2 3
3
Ovidiu Stoica03:06:59

Thank you, @U04V70XH6! I will ask in #beginners but thank you a lot for the response!

coby04:06:54

> Always work in your editor, never type into the REPL. As an autodidact I dabbled in Clojure for at least a year before I really understood this, or REPL-driven development generally. Wish I'd heard it stated plainly like this! 😅

seancorfield04:06:46

I hadn’t really thought about it being an issue until Stu Halloway said in one of his talks that he was “always baffled when he saw people typing into the REPL”…

seancorfield04:06:39

https://github.com/matthiasn/talk-transcripts/blob/master/Halloway_Stuart/REPLDrivenDevelopment.md — “Save everything. Your interactions with the REPL -- I am baffled when people type things into the REPL. We will talk about that a bit more in a second.”

🎉 3
seancorfield04:06:52

(the whole talk is excellent)

coby04:06:36

Thanks, I'll check it out.

coby04:06:20

> Develop good REPL hygiene so you can keep a REPL running for days (or weeks, or even months) without restarts or “refresh” tooling. While we're on the subject, do you have any recommended reading about this? I've been using mount and generally getting better at this but my progress has been pretty piecemeal. Usually have to restart my REPL at least once a day to hack around various things I don't fully grok yet.

coby04:06:47

hah, he's drinking a Lagunitas...I think I'll go grab a beer 😎

vemv04:06:50

> In learning clojure, what is the ONE thing that if you practice regularly, you will become a a great clojure programmer? Jumping to the source to understand what you're invoking. And favoring the use of said source over querying documentation, issue trackers, etc

seancorfield04:06:45

@UH85MNSKE “While we’re on the subject, do you have any recommended reading about this?” Well, there’s the whole https://clojure.org/guides/repl/introduction series — especially the last two sections (Enhancing…, Guidelines…). Stu’s talk is excellent, @U0522TWDA just posted a video of RDD (in the #news-and-articles channel: https://clojurians.slack.com/archives/C8NUSGWG6/p1623857070131800 ) and that thread contains more links — including Stu’s talk and my demo to London Clojurians from December showing RDD. And there’s also a link in that thread to this set of resources: https://clojure.org/guides/repl/annex_community_resources

3
3
coby06:06:07

Thanks so much, really appreciate your efforts. 🙂

3
Ben Sless06:06:17

With regards to unrolling, I did find significant performance improvements when unrolling rest-args or keys sequences when they're known at call site. The difference between (get-in m [k1 k2]) and (-> m (get k1) (get k2)) is significant and surprising. Same with assoc

Joshua Suskalo15:06:37

You might enjoy looking at https://github.com/redplanetlabs/specter which has similar performance to unrolled, but has an interface which is even nicer than the higher-level clojure functions, especially when dealing with highly-nested data.

Ben Sless15:06:38

I'm familiar with spectre but my design goal was to stick as close as possible to Clojure's semantics, with the ability of providing a drop in replacement, so users don't have to learn a new language. My implementation passes the core test suit.

dpsutton06:06:41

get-in being faster i'm assuming?

Ben Sless06:06:36

about 2x slower

Ben Sless06:06:53

not to mention that iterating over the keys with reduce1 is slower than reduce

didibus06:06:28

Did you check if it has that same megamorpgic issue the ticket refers too for real-life use cases? (No idea what megamorpgic refers too so I don't really know of relevant to get-in)

hiredman07:06:51

https://en.m.wikipedia.org/wiki/Inline_caching jitted jvm method calls will emit certain instruction sequences depending on if a call is monophonic (only invoked with a single target type), polymorphic (some small, maybe 2, different target types), or megamorphic (many target types, full real dispatch through a method table)

stardiviner10:06:04

I got error Clojure can't invoke "java.util.concurrent.Future.get()" because "fut" is null.

stardiviner10:06:13

Here is the details, the source code caused error:

(defn long-func []
  (let [p (promise)]
    (.start (Thread. (fn []
                       (Thread/sleep 5000)
                       (deliver p "hello!"))))))

;;; this expr will wait for 5 seconds.
(deref (long-func))
And here is the stacktrace:
Show: Project-Only All 
  Hide: Clojure Java REPL Tooling Duplicates  (13 frames hidden)

1. Unhandled java.lang.NullPointerException
   Cannot invoke "java.util.concurrent.Future.get()" because "fut" is null

                  core.clj: 2304  clojure.core/deref-future
                  core.clj: 2324  clojure.core/deref
                  core.clj: 2310  clojure.core/deref
                      REPL:    8  user/eval7232
                      REPL:    8  user/eval7232
             Compiler.java: 7181  clojure.lang.Compiler/eval
             Compiler.java: 7136  clojure.lang.Compiler/eval
                  core.clj: 3202  clojure.core/eval
                  core.clj: 3198  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1977  clojure.core/with-bindings*
                  core.clj: 1977  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  831  java.lang.Thread/run
My computer environment: - operating system: macOS Big Sur (Version 11.4) (M1 Apple Sillicon) - Clojure version: “1.10.3” - Java version: OpenJDK 17

Karol Wójcik10:06:42

You have to return the promise

Karol Wójcik10:06:49

Which you're not doing

Karol Wójcik10:06:08

(defn long-func []
  (let [p (promise)]
    (.start (Thread. (fn []
                       (Thread/sleep 5000)
                       (deliver p "hello!")))
p
)))

stardiviner10:06:29

I see. Let me take a try. Thanks @UJ1339K2B

stardiviner10:06:00

Problem solved. the returned p need to put out side of (.start …) sexp. Thanks a lot! I thought this might be a bug…. how stupid.

noisesmith14:06:04

unrelated to your question, the core future function returns a dereffable thing to get the result of a thread (like you have here) but with proper dynamic binding behavior and using a thread pool for efficiency

stardiviner16:06:46

I see. Thanks for your suggestion. I will learn.

Jakub Šťastný16:06:39

Hey guys, Clojure programming in Emacs, any tips? Of course I know about Cider, but apart from that? I understand the Clojure people are keen on ParInfer (for Emacs that'd be https://github.com/justinbarclay/parinfer-rust-mode then I imagine) more so than on the default Emacs paredit? (I'm yet to try ParInfer, I used to use Lispy and the Lispy EVIL thing before when working in Common/Emacs Lisp.) I'm an EVIL user (the Vim emulation for Emacs that is). So working in multiple modes is really preferred, being all the time in insert mode is really not my preference. Still – I'm happy to try anything that has potential.

isak17:06:11

You can become pretty productive if you just learn 2-3 paredit type commands. Some that I use all the time that I recommend learning the keybinds for: • Slurp forwards (makes it easy to surround code with an if/when/let, etc) • Splice Sexp killing backwards (makes it easy to undo the above) • Kill sexp

👍 3
Darrick Wiebe01:06:08

I like to use both paredit and parinfer at the same time. For me parinfer is awesome for straight-line coding but awful for significant editing, where paredit really shines. Parinfer also has two modes, and I find it indispensible to be able to toggle easily between them, so I added this lifesaver keybinding:

(evil-global-set-key 'insert (kbd "C-p") 'parinfer-toggle-mode)
(evil-global-set-key 'normal (kbd "C-p") 'parinfer-toggle-mode)
I can see which mode I'm in because the strict mode has rainbow parens visible, and the regular mode does the grey-out parens thing, which I find very intuitive, but took some work to get working. I also ran into problems with the rust variant of parinfer so fell back to the original which is slightly slower on huge files but in practice is never a problem. I agree with the above comment on learning some basic paredit commands, but would encourage you to gradually learn them all 🙂 Even the weirdo C-? is awesome about once every couple of days...

👍 3
noisesmith16:06:31

in my experience parinfer is more popular for people without lisp experience(?) but ymmv, and agreed that parinfer is great when first writing the code but becomes inconvenient when editing later (but I haven't touched it in years)

dpsutton16:06:52

I am not a fan of parinfer and stay with paredit. Simple, always works, and I direct it rather than it trying to figure out what indentation i need. it seems basically backwards. things should indent on the nesting i provide, not assume the nesting based on indentation. as for modal editing, I don't know of any reason that you should have to switch to non-modal editing

12
Joshua Suskalo17:06:25

I think the main thing with clojure people being keen on parinfer is that it's a useful tool for beginners to get into the language if they're unused to lispy syntax. Almost every clojure developer I've spoken to who started with parinfer has "graduated" to other structural editing packages.

lspector17:06:04

FWIW I regularly teach new Clojure programmers and I strongly discourage parinfer. I think it is actively counterproductive, although I appreciate that it is clever and that some people like it. What I seek both for the new Lispers I teach and for myself (a lisper for 35+ years) is something that acts as close as possible to a generic text editor, but with (crucially) good bracket matching and auto-re-indentation. No parinfer, no paredit, nothing that gets in the way of existing typing/cutting/pasting habits.

Jakub Šťastný17:06:45

Right, I wasn't aware of that. So basically on front of structural editing, I might as well stick to Lispy/Lispyville then.

🎯 3
Jakub Šťastný17:06:32

So structural editing + Cider would then be the typical way of working with Clojure in Emacs then I reckon?

blak3mill3r18:06:33

That is how I work. Lispy(ville) + Cider. I think if you can get really good lispy muscle-memory, that is about as good as any structural editing support out there.

👍 3
afleck18:06:48

i’ll second lispy(ville), it’s nice.

emccue17:06:07

@suskeyhose I've been using parinfer for years now. Can't work with anything else productively

emccue17:06:02

its the main reason I haven't really tried calva

seancorfield17:06:59

I’ve gone back and forth between Paredit and Parinfer multiple times over the 11 years I’ve been doing Clojure. I like some aspects of both and I dislike some aspects of both. For the last several years, I was using them together, first in Atom, then in VS Code, but now I’m using Calva (+ Chlorine) so I’m only using Paredit. I think Paredit takes a lot more getting used to and it’s a lot more powerful — but you need to learn/remember a lot more commands to be productive. I found Parinfer mostly did what I expected/needed but missed the more powerful structural stuff that Paredit adds. When I used them together, I disabled some of Paredit’s indent/paste stuff in order to let Parinfer do its thing.

🚀 3
pez17:06:21

Calva Peredit plays pretty well with the Parinfer extension. You'll need to disable Calva's indent-as-you-type, though.

isak18:06:43

In cursive with paredit, you can be up to normal code-editing productivity, if not higher, just by knowing 1 keybind (Ctrl-W for "Expand selection")

👆 3
kennytilton21:06:30

I just live off indentation and re-indentation, and parens-balancing highlighting. I know how to edit, thanks. But I have been Lisping for 25 years... I had a fling with Paredit, not sure what ended the relationship. I might have forgotten its birthday...

👍 5
😂 3
kennytilton21:06:51

Hmm, The Google is failing me. I saw some ADR templates somewhere, cannot find it now. Any recommendations? 🙏

kennytilton05:06:38

That's it! Thx. 🙏

didibus23:06:58

I found aggressive-indent-mode + adjust-parens-mode in Emacs is my favourite combo. Works better than paredit and pareinfer for me. Though I use a few smartparens commands on top.