Often the Clojure community survey receives many requests for faster startup time. There are however several distinct startup time use cases. One of those most frequently mentioned is development startup time (whether that's starting a repl that loads a user.clj or running a dev web server). It is not widely appreciated how easy it is to use Clojure's existing mechanisms to AOT compile your startup dependencies and use them while you develop, shaving 10s of seconds off of every start. Hopefully this guide will help. I am certain there is a lot that can be done in frequently used templates to leverage this idea and I would encourage any toolsmiths/template makers out there to do so! It requires no changes to Clojure itself, very few changes to how you work, and you can leverage it right now. There are of course, other use cases (lambda/serverless, production servers, etc). The core team continues to evaluate a range of options on those.

> If you are using a user.clj in dev, you need to force a reload because it has already been loaded automatically I'm not sure I get that. So it was loaded before I called compile. So what? If it wasn't changed, why would I want to reload it?

compiling is a side effect of loading. If it's already been loaded, compilation won't happen

Gotcha, thanks! Huh. Would it then make sense to be able to just dump an already loaded ns into a class file? Assuming that it's theoretically possible.

Is this something you expect projects will setup themselves, or users?

Thank you Alex, this is something new I learned today. A sample project or more detailed guide where this technique is used on a large number of external dependencies could be even more helpful I believe especially for newer members of the community.

Just created this gist with the code that I use in production. Don't use in development yet because I didn't set up any invalidation, and forgetting to recompile stuff when you changed some dependency can lead to some non-trivial errors. https://gist.github.com/p-himik/fcf638b05d70d908c74d98e25fd9e935

@U2FRKM4TW that partially already exists - normally as you load namespaces, they are compiled and added to the dynamic classloader (at a function level). But, on disk things are a little different to facilitate file-oriented loading. I think you'd end up basically redoing the same work, doesn't seem like it would help much to have this.

@U064X3EF3 Hmm, I see. Could you please check out the comments in my gist linked above? Not sure if they have enough details, but maybe you have some ideas on how to deal with jars in general.

@U09LZR36F what do you mean by "projects"?

@U064X3EF3 you're saying normally as you load namespaces, they are compiled and the guide article says compile ... compiles that namespace and all the namespaces it requires into *compile-path* - I wonder why 'normally' loading a namespace isn't doing that?

I must be missing something obvious I feel

@U08BJGV6E example project https://github.com/puredanger/startup-time

normally loading a clojure namespace will compile the namespace into a bunch of classes and load those into the dynamic classloader

that all happens at runtime, dynamically

the process in this article describes how to save and reuse that work

I see that, I was just wondering why the saving isn't happening automatically (provided that *compile-path* is on the classpath)? Thank you for the explanation and the example project!

because it's not always needed - while you're working on your project, you probably don't actually want this at all

for the namespaces in your own project

Huh. Interesting. I'm getting

Syntax error (IOException) compiling fn* at (user.clj:1:1).
No such file or directory

mkdir classes

24.75 -> 6.68 on my machine, that is a significant improvement!

A minor nit: this line near the start has a smart quote in it, instead of a regular quote:

(compile ‘my.namespace)    ;; writes .class files to *compile-path*

Creating the dir did it. The error about fn* was especially unhelpful.

Oh, several of the code examples have those pesky smart quotes lower down too.

I’ll fix those, wish I could uninvent those

@U064X3EF3 by project I mean something like tools.deps or the application I work on daily.

I'm not sure why but whenever I call compile on a ns, only that ns is compiled into classes directory - nses required by that ns are NOT compiled at all... I'm using leiningen.

I think lein does magic for that.

It seems like this approach would go well with some caching information. Library incoming...

Not sure about the classes issue above, but it it is git, this is normally solved by having a .gitkeep file inside it.

maybe we should move this to #tools-deps b/c none of what you're saying makes sense to me

@U064X3EF3 sorry, not related to tools.deps at all. What I mean is, should something like core.cache be adding a dev/user.clj and compilation stuff? Or meander, oz, the lein-based project I work on daily, etc. Or should I, a developer, be setting this up for myself as a "personal workflow"?

I think the answer for libs is generally, no

user.clj is primarily used by people building applications (particularly common in web app templates, reloaded stuff, etc)

if you are doing dev on a project, and it has a lot of deps, you might want to consider this. for those libs you mention, that's probably not the case.

I'm not suggesting libs would include this. Sorry, I'm not doing a good job. I mean for while you're developing those libraries.

there is no reason to add a user.clj if you don't already have one

Okay. I think you've figured out what I mean now :).

it's called out explicitly because it's special, and common for applications

Better phrasing would perhaps be, I work on 3 projects in a week. Should I update all 3 of them. Or should I update ~/.clojure/deps.edn.

it depends

if the alias is identical, then sure, but I'm not sure if it would be

Wouldn't it be always extra paths classes? (Or maybe dominic_classes or something unique).

not if you put it in a different path

I can't answer these questions for you - they are dependent on your environment

I'm trying to figure out what the best current practice is. What I need to get projects to do, or my colleagues :).

35sec -> 11sec.

@U064X3EF3: can you clarify why this might be a problem: https://clojurians.slack.com/archives/C06MAR553/p1584044181213700?thread_ts=1584038194.208900&cid=C06MAR553

if you have files that you are actively changing all the time (b/c doing dev on them), aot compiling those namespaces is useless. but aot compiling your deps is still worthwhile

sure; but wouldn’t it only do it if they had actually changed?

clojure is compiling anyway in memory when the repl evaluates; so why worry about putting those outputs on disk?

I don't understand what you're asking

Sorry, you seemed to imply earlier that a user wouldn’t want their project namespaces to be compiled into classes/. I understand that because they’re changing frequently there’s no benefit to storing them. What I’m questioning though is whether there is a cost or problem if it is done. Otherwise it’s extra work for tooling to figure out which namespaces to pass to compile. Why do that work if doing so doesn’t cause people problems and is almost as quick?

I don't think there's a problem, just saying it's not giving you a benefit

Ok cool. That’s all I wanted to clarify.

I think you can do a simple algorithm like: 1. ig/load-namespaces 2. analyze deps.edn for paths that are pulled in by the current project 3. collect all ns’s loaded from those path’s sources 4. (apply compile (set/difference (all-ns) *1))

Please do not have conversations in the #announcements channel.

Or stick to a single thread under the original announcement.

Oops sorry sean — I thought this was a different channel!

@U08E8UGF7: That seems a lot more fiddly than what I was thinking. Won’t it be awkward to know what paths/aliases your repl was running with? I was thinking you’d just implement compile-namespaces… probably something like:

(defn compile-namespaces [config keys]
  (doall (->> (ig/dependent-keys config keys)
              (mapcat #(conj (ancestors %) %))
              (mapcat ig/key->namespaces)
              (distinct)
              (compile))))

The whole point is to not compile the namespaces that you are working in on in the current project, since those will change frequently. You can’t determine where those integrant ns’es are coming from just from the config

anyway, let’s move this to #integrant