Fork me on GitHub
#off-topic
<
2022-02-07
>
Zach Mitchell, PhD15:02:00

https://tinkering.xyz/fmo-optimization-story/ I've put this in off-topic because there's no Clojure content, but I thought it would be interesting nonetheless. I'm an (experimental) physicist working on a simulation written in Python/NumPy and it was taking about 8 hours to run. I decided to see how much faster I could make it and ended up getting it 50x faster (would be faster if my laptop had more than 2 cores). I decided to write up my optimization process. I will say that I really missed Clojure's instant feedback when I was debugging my Rust code :)

🔥 5
teodorlu16:02:51

Nice writeup! Exciting to read about the problem, and the (quite hardcore) numeric programming required to solve it. out of curiosity, could you have used clojure libraries like https://github.com/techascent/tech.ml.datasethttps://github.com/techascent/tech.ml.dataset? Not sure if there are fast Clojure options for what you're using ndarray for in your code.

Zach Mitchell, PhD16:02:17

Potentially, though there are hard abs soft reasons I write my lab software in Python.

👍 1
Zach Mitchell, PhD16:02:21

Soft: because it's very approachable for other scientists. I'm the only person in my lab at the moment, so the next person who comes along after me is going to have to bring themselves up to speed mostly on their own after I leave.

Zach Mitchell, PhD16:02:58

Hard: I started this around the same time I started learning Clojure at all. I'm still a Clojure baby lol I think the library you linked would suffice for all of the array operations I need, but the part I'm unsure about (because I haven't done any research into it) is whether it links to LAPACK for matrix diagonalization algorithms.

Zach Mitchell, PhD16:02:52

In the end I suspect I would have landed somewhere between Python and Rust in terms of performance.

teodorlu21:02:36

That makes sense! tech.ml.dataset shows a lot of promise, but the amount of man hours spent on it compared to numpy / lapack is tiny in comparison. There's also https://github.com/clj-python/libpython-clj, which lets you call into a Python interpreter from Clojure, if you want to take the middle way.

p-himik11:02:23

Another potentially useful thing here that exists in JDK itself: https://openjdk.java.net/jeps/417

👍 1
👀 2
🔥 1
teodorlu11:02:48

@U2FRKM4TW that's potentially very interesting for numerical programming in Clojure.

p-himik11:02:15

Indeed. :) Just stumbled upon it thanks to the video on floating point operations shared by ghadi in a different thread.

👍 1