This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-02-07
Channels
- # announcements (38)
- # asami (14)
- # beginners (35)
- # biff (3)
- # calva (29)
- # cider (20)
- # clj-kondo (7)
- # cljdoc (38)
- # clojure (64)
- # clojure-art (2)
- # clojure-australia (1)
- # clojure-dev (6)
- # clojure-europe (30)
- # clojure-nl (4)
- # clojure-spec (4)
- # clojure-uk (9)
- # clojured (3)
- # clojurescript (87)
- # cursive (17)
- # datahike (2)
- # datomic (10)
- # defnpodcast (2)
- # emacs (2)
- # events (1)
- # fulcro (25)
- # gratitude (1)
- # introduce-yourself (1)
- # jobs-discuss (21)
- # lsp (103)
- # malli (41)
- # meander (8)
- # minecraft (3)
- # missionary (3)
- # nextjournal (20)
- # off-topic (10)
- # pedestal (1)
- # polylith (15)
- # portal (6)
- # releases (2)
- # ring (1)
- # ring-swagger (2)
- # sci (4)
- # shadow-cljs (5)
- # spacemacs (3)
- # sql (11)
- # xtdb (3)
https://tinkering.xyz/fmo-optimization-story/
I've put this in off-topic
because there's no Clojure content, but I thought it would be interesting nonetheless. I'm an (experimental) physicist working on a simulation written in Python/NumPy and it was taking about 8 hours to run. I decided to see how much faster I could make it and ended up getting it 50x faster (would be faster if my laptop had more than 2 cores). I decided to write up my optimization process.
I will say that I really missed Clojure's instant feedback when I was debugging my Rust code :)
Nice writeup! Exciting to read about the problem, and the (quite hardcore) numeric programming required to solve it.
out of curiosity, could you have used clojure libraries like https://github.com/techascent/tech.ml.datasethttps://github.com/techascent/tech.ml.dataset?
Not sure if there are fast Clojure options for what you're using ndarray
for in your code.
Potentially, though there are hard abs soft reasons I write my lab software in Python.
Soft: because it's very approachable for other scientists. I'm the only person in my lab at the moment, so the next person who comes along after me is going to have to bring themselves up to speed mostly on their own after I leave.
Hard: I started this around the same time I started learning Clojure at all. I'm still a Clojure baby lol I think the library you linked would suffice for all of the array operations I need, but the part I'm unsure about (because I haven't done any research into it) is whether it links to LAPACK for matrix diagonalization algorithms.
In the end I suspect I would have landed somewhere between Python and Rust in terms of performance.
That makes sense!
tech.ml.dataset
shows a lot of promise, but the amount of man hours spent on it compared to numpy / lapack is tiny in comparison.
There's also https://github.com/clj-python/libpython-clj, which lets you call into a Python interpreter from Clojure, if you want to take the middle way.
Another potentially useful thing here that exists in JDK itself: https://openjdk.java.net/jeps/417
@U2FRKM4TW that's potentially very interesting for numerical programming in Clojure.