clojure

techno7 2025-10-06T07:45:01.559219Z

Hello! Question about macro expansion. A direct call to a loop-unrolling macro is fast. A defn that wraps the exact same macro call is ~15x slower. (defmacro generate-work [data n] <some macro that transforms data using n>) ;; Fast: (generate-work data 20) ;; Slow: (defn wrapper [data] (generate-work data 20)) (wrapper data) Why is the wrapper-fn version so dramatically slower than the direct macro call, when its body is expanded into the exact same code at compile time?

p-himik 2025-10-06T07:58:44.467659Z

It probably isn't. How do you measure? Are there any side-effects in the body of the macro, any memoization, access to external resources? Can you share the impl of the macro?

techno7 2025-10-06T09:09:23.427269Z

You're right, I oversimplified it. data is a lazy-seq. It also takes stateful target and does some work on it. I'm measuring with time and doing some benchmark using criterium. I'm constantly getting 5 sec with direct macro call, and getting 70-75 seconds with wrapper version.

p-himik 2025-10-06T09:12:38.557309Z

Need to see the code.

2025-10-06T14:32:39.129759Z

I agree with p-himik, there's not enough information in the question to generate any hypotheses really

techno7 2025-10-06T16:30:04.599119Z

(defmacro write-columnar-csv-unrolled [writer data num-cols] (dotimes ['row-idx (count (first data))]` (doto ~writer ~@(mapcat (fn [col-idx] [(.write (str (get (get data col-idx) ~'row-idx)))` ~@(if (< col-idx (dec num-cols)) [(.write "," 0 1)]` [(.write "\n")])])` (range num-cols)))))

techno7 2025-10-06T16:35:17.197829Z

And I'm calling it like (with-open [w (io/writer writer)] (doseq [chunk chunks] (write-columnar-csv-unrolled w chunk 20) (.flush w)))

p-himik 2025-10-06T16:56:59.543959Z

Add ^java.io.Writer to the wrapping function's signature, in front of the writer argument. But before that, just to see how to diagnose problems like this, add (set! *warn-on-reflection* true) right under the (ns ...) form where you've defined that function.

👍 1
p-himik 2025-10-06T16:58:51.572589Z

That's why providing proper minimal reproducible examples is of paramount importance. :) Absolutely nothing in your OP indicated that there might be reflection. I guess I can fine-tune my crystal orb now a bit so I can immediately call something like this out in future. :D

techno7 2025-10-06T17:35:14.416939Z

Thanks, I see the issue now. I thought there are some overhead when the code is not inlined. I was wrong and It's reflection issue.

J 2025-10-06T09:26:19.340209Z

Hi! Is anyone using https://github.com/HtmlUnit/htmlunit here?

2025-10-06T18:31:18.552519Z

I have used htmlunit from time to time for web scraping and shockingly some unit tests, but not actively using it for anything right now

2025-10-06T18:32:36.946269Z

The web is so js heavy these days it can be challenging to do stuff without a real browser. The project where I was using htmlunit for tests ended up switching to playwright.