Clojurians Log v2

A clarification is needed – does this happen with inline eval (`C-x C-e`, C-c C-c) or when you evaluate a form in the REPL buffer?

C-c C-c

In the past year we've greatly improved the rendering performance of large evalution outputs in the inline eval scenario – they are trimmed much earlier now, so the hang-ups should no longer happen.

Can you come up with some simplified reproducible example?

I think the issue might be related to lazy data structures. I'll try to suss out when the problem starts, and write an example. Thanks!

To check if it's really the eval printing that is slow and not the realization of lazy collections, try doing this:

(count (pr-str <my-slow-form>))

Okay, I need to do some more digging. Somewhere, I've done something confusing. Basically, I have a vector of maps. This is quite long (say, 150'000 records). I then (filter) to 25'000 records. This is my data. If i run (take 1000 my-data) this takes 10+ seconds to evaluate inline (C-c C-c). If I make some dummy data with the same number of records and run (take 1000 dummy-data) this takes < 1 s. The original data has a bit of text in some keys, so on purpose, I made the dummy data more text heavy (but with same number of keys).

(count my-data)
;; => 24914
(count dummy-data)
;; => 25000

(count (.getBytes (pr-str my-data) "UTF-8"))
;; => 15927416

(count (.getBytes (pr-str dummy-data) "UTF-8"))
;; => 83525001

(instance? clojure.lang.LazySeq my-data)
;; => true

(instance? clojure.lang.LazySeq dummy-data)
;; => true

The maps in the examples are quite long, so I didn't want to paste. I can paste it in if thats helpful.

Does (count (.getBytes (pr-str my-data) "UTF-8")) also take 10 seconds to return?

No, more like a second.

Weird indeed. In any case, nREPL should hard-cut the output at 1MB payload. Can you try issuing the same command in the REPL buffer? Do you see that the output is rendered slowly and gradually?

The output is rendered gradually (i.e. text is visibly being written to the reply in chunks)

Given that you create a bigger dummy data that "prints" faster, I'm more convinced that this has to do with computation costs, not printing. Anyway, I wouldn't be able to help much without a reproducer.

Perhaps, you could record a video of your Emacs when this stuff happens, to at least verify if the delays look like they come from printing

Idk if this is at all relevant, with my experience being many years old by now, but I was put off of using emacs after experiencing heavy slowdowns when scrolling up and down a buffer containing a reasonably-sized (cond) (with further nested control structure calls). I don't know what exactly caused it, but I assumed it was some kind of syntax rendering that couldn't handle deeply nested and/or "wide" data literals very well. Maybe you have some plugin causing a similar issue? Is there some way to debug render times in emacs? I'd be surprised if there weren't 😁

Talking about running it with certain flags to get statistics or tracing or a flame graph or whatever.

This was my cond BTW. I was a baby Clojurian at the time 😁 https://github.com/Reefersleep/derpa/blob/7274abfc1edb62a9dbcffb1ddab4064e59ebb52e/src/derpa/core.clj#L31

Hey! Sorry, I forgot to update the thread, it got solved in a private message. This issue was, as far as I understand, a call to (string-width display-string) that was organized poorly. This should be fixed by https://github.com/clojure-emacs/cider/pull/3774 (thanks to @alexyakushev) For anyone with the same problem, a quick fix is to set cider-use-overlays to "Display in echo area"

Very nice 🤓

So, it turns out that it is the value in the key-value pairs in the maps that slows everything down. In my specific example, it depends on whether or not the abstract of the article is included. However, I don't understand why (def var-name <the form>) is instant, but showing the data takes like ten seconds seconds (using C-c C-c). It doesnt matter whether the data is lazy or not. For example:

;; Is instant
(def var-a (take 10000 small-maps-lazy))
(def var-b (into [] (take 10000 small-maps-lazy)))

(instance? clojure.lang.LazySeq var-a)
;; => true
(instance? clojure.lang.LazySeq var-b)
;; => false

;; Takes long
var-a
var-b

I realize this is maybe outside the scope of the original question. I really appreciate that you took time to help me. If I should post this question in a different channel, let me know.

I checked the same operations in Cursive, and everything runs/prints in a second.

Is the data sensitive? Can you dump it into EDN and sent to me, maybe in DM?

No, not at all. I've sent you the data behind "small-maps-lazy".

Again, appreciate it. No hard feelings if you don't want to spend more time on it.