This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-12
Channels
- # announcements (65)
- # aws (1)
- # babashka (12)
- # beginners (111)
- # bristol-clojurians (1)
- # cider (32)
- # clj-kondo (55)
- # clojars (3)
- # clojure (71)
- # clojure-europe (17)
- # clojure-france (4)
- # clojure-italy (36)
- # clojure-losangeles (8)
- # clojure-nl (6)
- # clojure-uk (115)
- # clojurescript (2)
- # datomic (99)
- # fulcro (32)
- # graalvm (12)
- # graphql (20)
- # hoplon (203)
- # meander (56)
- # mount (3)
- # off-topic (17)
- # pathom (17)
- # reitit (22)
- # shadow-cljs (32)
- # spacemacs (9)
- # tools-deps (19)
- # vim (25)
- # vscode (3)
A new guide on how to use trusty compile
to improve dev startup time https://clojure.org/guides/dev_startup_time
Often the Clojure community survey receives many requests for faster startup time. There are however several distinct startup time use cases. One of those most frequently mentioned is development startup time (whether that's starting a repl that loads a user.clj or running a dev web server). It is not widely appreciated how easy it is to use Clojure's existing mechanisms to AOT compile your startup dependencies and use them while you develop, shaving 10s of seconds off of every start. Hopefully this guide will help. I am certain there is a lot that can be done in frequently used templates to leverage this idea and I would encourage any toolsmiths/template makers out there to do so! It requires no changes to Clojure itself, very few changes to how you work, and you can leverage it right now. There are of course, other use cases (lambda/serverless, production servers, etc). The core team continues to evaluate a range of options on those.
> If you are using a user.clj in dev, you need to force a reload because it has already been loaded automatically
I'm not sure I get that. So it was loaded before I called compile
. So what? If it wasn't changed, why would I want to reload it?
compiling is a side effect of loading. If it's already been loaded, compilation won't happen
Gotcha, thanks! Huh. Would it then make sense to be able to just dump an already loaded ns into a class file? Assuming that it's theoretically possible.
Thank you Alex, this is something new I learned today. A sample project or more detailed guide where this technique is used on a large number of external dependencies could be even more helpful I believe especially for newer members of the community.
Just created this gist with the code that I use in production. Don't use in development yet because I didn't set up any invalidation, and forgetting to recompile stuff when you changed some dependency can lead to some non-trivial errors. https://gist.github.com/p-himik/fcf638b05d70d908c74d98e25fd9e935
@U2FRKM4TW that partially already exists - normally as you load namespaces, they are compiled and added to the dynamic classloader (at a function level). But, on disk things are a little different to facilitate file-oriented loading. I think you'd end up basically redoing the same work, doesn't seem like it would help much to have this.
@U064X3EF3 Hmm, I see. Could you please check out the comments in my gist linked above? Not sure if they have enough details, but maybe you have some ideas on how to deal with jars in general.
@U09LZR36F what do you mean by "projects"?
@U064X3EF3 you're saying normally as you load namespaces, they are compiled
and the guide article says compile ... compiles that namespace and all the namespaces it requires into *compile-path*
- I wonder why 'normally' loading a namespace isn't doing that?
@U08BJGV6E example project https://github.com/puredanger/startup-time
normally loading a clojure namespace will compile the namespace into a bunch of classes and load those into the dynamic classloader
that all happens at runtime, dynamically
the process in this article describes how to save and reuse that work
I see that, I was just wondering why the saving isn't happening automatically (provided that *compile-path*
is on the classpath)?
Thank you for the explanation and the example project!
because it's not always needed - while you're working on your project, you probably don't actually want this at all
Huh. Interesting. I'm getting
Syntax error (IOException) compiling fn* at (user.clj:1:1).
No such file or directory
when trying to compile the example project.mkdir classes
A minor nit: this line near the start has a smart quote in it, instead of a regular quote:
(compile ‘my.namespace) ;; writes .class files to *compile-path*
Oh, several of the code examples have those pesky smart quotes lower down too.
I’ll fix those, wish I could uninvent those
@U064X3EF3 by project I mean something like tools.deps or the application I work on daily.
I'm not sure why but whenever I call compile
on a ns, only that ns is compiled into classes directory - nses required by that ns are NOT compiled at all...
I'm using leiningen.
It seems like this approach would go well with some caching information. Library incoming...
Not sure about the classes issue above, but it it is git, this is normally solved by having a .gitkeep file inside it.
maybe we should move this to #tools-deps b/c none of what you're saying makes sense to me
@U064X3EF3 sorry, not related to tools.deps at all. What I mean is, should something like core.cache be adding a dev/user.clj and compilation stuff? Or meander, oz, the lein-based project I work on daily, etc. Or should I, a developer, be setting this up for myself as a "personal workflow"?
I think the answer for libs is generally, no
user.clj is primarily used by people building applications (particularly common in web app templates, reloaded stuff, etc)
if you are doing dev on a project, and it has a lot of deps, you might want to consider this. for those libs you mention, that's probably not the case.
I'm not suggesting libs would include this. Sorry, I'm not doing a good job. I mean for while you're developing those libraries.
there is no reason to add a user.clj if you don't already have one
it's called out explicitly because it's special, and common for applications
Better phrasing would perhaps be, I work on 3 projects in a week. Should I update all 3 of them. Or should I update ~/.clojure/deps.edn.
it depends
if the alias is identical, then sure, but I'm not sure if it would be
Wouldn't it be always extra paths classes? (Or maybe dominic_classes or something unique).
not if you put it in a different path
I can't answer these questions for you - they are dependent on your environment
I'm trying to figure out what the best current practice is. What I need to get projects to do, or my colleagues :).
@U064X3EF3: can you clarify why this might be a problem: https://clojurians.slack.com/archives/C06MAR553/p1584044181213700?thread_ts=1584038194.208900&cid=C06MAR553
if you have files that you are actively changing all the time (b/c doing dev on them), aot compiling those namespaces is useless. but aot compiling your deps is still worthwhile
sure; but wouldn’t it only do it if they had actually changed?
clojure is compiling anyway in memory when the repl evaluates; so why worry about putting those outputs on disk?
I don't understand what you're asking
Sorry, you seemed to imply earlier that a user wouldn’t want their project namespaces to be compiled into classes/
. I understand that because they’re changing frequently there’s no benefit to storing them. What I’m questioning though is whether there is a cost or problem if it is done.
Otherwise it’s extra work for tooling to figure out which namespaces to pass to compile
. Why do that work if doing so doesn’t cause people problems and is almost as quick?
I don't think there's a problem, just saying it's not giving you a benefit
Ok cool. That’s all I wanted to clarify.
Interesting — I’d been meaning to look at what would be involved in doing something like this with some of our integrant systems which are essentially dynamically composed from configurations that are loaded through ig/load-namespaces
.
I’ll need to read this thoroughly to see if I can finally cobble something together. This is like we used to compile clojure code in the good old days! 😁
I think you can do a simple algorithm like:
1. ig/load-namespaces
2. analyze deps.edn for paths that are pulled in by the current project
3. collect all ns’s loaded from those path’s sources
4. (apply compile (set/difference (all-ns) *1))
Or stick to a single thread under the original announcement.
@U08E8UGF7: That seems a lot more fiddly than what I was thinking. Won’t it be awkward to know what paths/aliases your repl was running with? I was thinking you’d just implement compile-namespaces… probably something like:
(defn compile-namespaces [config keys]
(doall (->> (ig/dependent-keys config keys)
(mapcat #(conj (ancestors %) %))
(mapcat ig/key->namespaces)
(distinct)
(compile))))