Fork me on GitHub
#clojure
<
2023-08-23
>
pesterhazy07:08:55

https://clojure.org/guides/dev_startup_time (via clojure.core/compile) can produce significant speed ups (in our case, reduce startup time by 3 seconds) Suppose our devenv starts with clojure -M -m my.entrypoint. When should we be calling compile? There are two naive options, both with issues: 1. Run only once. E.g. "(when-not (dir-exists? "classes") (make-dir "classes") (compile 'my.entrypoint))". But then as code and dependencies evolve, compile output will be stale and .class files won't be used (because .clj files are newer). 2. Run every time. This will ensure that .class files are up to date, but cancels out the benefit of AOT compilation. What approaches are teams using?

p-himik07:08:40

(2) does not completely cancel things out as compile won't recompile everything - only the things that have been changed. That being said, I had a couple of problems with AOT solving which cancelled out that 3 second benefit for the next few years. :)

pesterhazy07:08:43

Interesting, I hadn't considered that

pesterhazy07:08:17

We're thinking of AOT'ing the frontend compiler pipeline, based on shadow-cljs. I don't foresee any caching problems there For the backend, we'll have to try and see if there are issues. Alex Miller has written encouraging words here: https://ask.clojure.org/index.php/11656/could-an-aot-compilation-cache-speed-library-loading-times?show=11666#c11666

p-himik07:08:44

When you compile CLJS, most of the time is spent doing actual compilation/optimization. And AOT'ing things will only speed up namespace loading.

pesterhazy08:08:17

Yes, loading namespaces is 3s or more on an M1 for us

pesterhazy08:08:14

A significant chunk of the 16s from process start to the compile assets being ready-to-use

pesterhazy08:08:31

I just timed it: • compile takes 4s with an empty classes/ • on subsequent runs it takes 1s So it's faster but not 100% free

p-himik08:08:41

Ah, alright. In my projects, UI compilation takes ~30 seconds usually, so I wouldn't bother with 3 seconds at all. :)

vemv09:08:50

This is what I've done for 2+ years: • make the -Dclojure.compile.path= dynamic, it's target/<Lein profiles or clojure aliases><hash of classpath> • only compile, programatically, namespaces that are detected as 3rd party dependencies (e.g. clj-http.core -> yes, my.project.core -> no) This way there aren't staleness or other issues. I don't necessarily recommend implementing any of this yourself, unless you intend to create a fully reusable tool / workflow.

p-himik10:08:32

There's still room for issues, e.g. when some third-party dependency does anything non-pure at namespace load time, apart from just defining new things.

vemv10:08:47

I haven't encountered that. Anyway I'd consider that a bug that would make me at the very least report the issue. And make me reconsider the quality of the lib :)

p-himik10:08:22

Right, of course. But just a single incident like this negates all the gains you could possibly get from AOT, at least when you're a solo developer on a project.

👍 2
vemv10:08:25

It depends, because I use the same AOT setup in all projects. And I think I'd be happy to encounter a problematic side-effect - would be a net gain for the lib, and maybe for me / my project in some indirect way

jpmonettas12:08:22

I guess it also depends on how many times you restart your repl and need to reload everything from scratch, if you set it up for long runs (like even days) then some extra seconds should be nothing

Joshua Suskalo16:08:57

For my own part I just don't bother to care about dev startup time and just don't restart my repl more than a couple times a week at the most.

💯 2
jpmonettas17:08:49

though it is quite an art form to get everything right so the repl doesn't need to be restarted

Joshua Suskalo17:08:58

For the most part I consider AOT a release-only activity, at which point you get every ounce out of it you can including direct linking, stripping metadata, etc.

Joshua Suskalo17:08:50

That's a good point @U0739PUFQ. It's also a common hangup. I wrote an article about it a little while back that may help some people though! https://srasu.srht.site/var-evaluation.html

jpmonettas17:08:35

Apart from what is described in that article, in my experience I think the problem is that I need to constantly keep an eye and be very aware of where the state is, which is manageable in my own projects code but as soon as I bring other people libraries (specially java ones) then as soon as I get some non obvious error I tend to want to restart the repl, just to discard something being in an inconsistent state. I find easier to accomplish in projects I tend to touch every day since I tend to have most of it on my head and put effort to tweak my flow so I don't need to restart the repl, but maybe not so easy in a multi person project. What varies on each project is how confident I am that the state can't get into inconsistent states by messing with it

jpmonettas17:08:25

but 100% I prefer to put my time in tweaking so I don't have to restart my repl, than into making it easier and faster to restart

kennytilton15:08:44

Man, it has been a long time since I did backend Clojure. Forgot my workflow! I am looking for the equivalent of CLJS/CLJD auto-build-reload, in which instead a Clojure app wll re-run, and have no idea what I used to do, My goal is simply to save in IntelliJ/Cursive and have everything rebuilt and a designated function automagically dispatched. I happen to be using Leiningen, so I have my eye on https://github.com/weavejester/lein-auto. Good solution? Any alternates? Thx!

Ben Sless15:08:47

Tools.namespace + hook in intellij to refresh in save?

👍 4
kennytilton15:08:17

Yes, I was wondering about a hook in IntelliJ. But it just occurred to me that Cursive has a "run last test" command which I have already bound to a keychord (and another keychord runs the test under the cursor), so really I am good to go, I just have to set up tests as needed. Thx!

👍 2
phill15:08:24

You might be working under different circumstances, but I like to work with the REPL, and eval a form, and the eval'd form instantly becomes effective in, say, my Swing app or web server. For times when I wish to shut down the app, reload "changed" namespaces, and restart the app, I like to combine Sierra's "Component" (which is competent in the area of shutdown/startup sequences) and this little library https://github.com/ciderale/quick-reset, last commit 9 years ago!, whose README cites Sierra's "reloaded" essay.

☝️ 2
gnl16:08:29

Shameless plug (though is it still shameless when you're not charging anything...?) of https://github.com/gnl/playback/, which has some middleware that you can configure to call reload/refresh functions when loading stuff in the REPL. Have only used that functionality in a ClojureScript context so far, but it should work just fine in Clojure. You might find it useful (and do let me know how it goes if you try it.)

Ben Sless16:08:53

You can also mix tools.namespace with hawk https://github.com/wkf/hawk

kennytilton17:08:26

Thanks all! The nice thing about ^Q, the keychord I have bound to "Re-run last test" is that wherever I happen to finish my editing, Bam, I just hit ^Q. Evalling forms in a comment is great for small, ad hoc tasks, but after hacking here and there across a project it is great to have a keychord save, recompile and re-run. Thx again. 🙏

DeepReef1117:08:25

With Babashka, is there a way to call another cli program that is installed on the system? For instance, can I call qutebrowser kinda like I would be able to call from a terminal emulator?

borkdude17:08:26

Yes. This is usually done with babashka.process/shell

borkdude17:08:02

(require '[babashka.process :refer [shell]])
(shell "qutebrowser") ;; or whatever

DeepReef1117:08:14

Thanks! Is shell a choice or is this code working as is?

borkdude17:08:50

babashka.process is a built-in library, if that's what you're asking

2
DeepReef1118:08:40

What is wrong with this code in babashka? I'm trying to use hiccup

#!/usr/bin/env bb
(ns ptouch-label)
(require '[babashka.process :refer [shell]
           ' [hiccup2.core :as h]])

(shell "echo hi")
(prn [:div [:div [:p "ok"]]])

borkdude18:08:40

Why do you think it's wrong? What is the expected behavior and what behavior are you seeing?

borkdude18:08:04

btw, you're nesting the second libspec in the require form under the first

dpsutton18:08:06

your require vector includes the other

2
dpsutton18:08:53

(require '[babashka.process :refer [shell]]
         '[hiccup2.core :as h])
;; vs
(require '[babashka.process :refer [shell] ;; note this one is not closed here
           '[hiccup2.core :as h]])

borkdude18:08:56

(require 
  '[babashka.process :refer [shell]] 
  '[hiccup2.core :as h])

DeepReef1118:08:01

Great thanks! How can I get the hiccup printed as html?

borkdude18:08:17

(h/html [:div ...]))

4
DeepReef1118:08:08

Thanks a lot!