Fork me on GitHub
Jungin Kwon03:02:33

I am getting this error, when I build with clojure -T:build uber. I am using v0.6.7 Sometimes the build succeeds and sometimes it fails with this error. Anyone know why?

Syntax error compiling at (clojure/tools/namespace/parse.cljc:55:19).
No such var: reader/read


What version of tools.deps? There was a code loading race condition that could manifest oddly like that


0.12.1098 and later should have the race I am thinking of fixed

Alex Miller (Clojure team)03:02:00

Yeah, I would bump to latest, that should be fixed


FWIW, 0.6.7 brings in t.d.a 0.12.1071 so, yeah, use a later version to see if it helps @jungin.kwon1

Jungin Kwon03:02:10

Thank you! 👍


I'm migrating a project from Leiningen to deps.edn and In my project.clj, there's this:

:profiles {:dev {:aot [db.migration.migrations]}}
My (possibly incorrect) understanding is that this tells Leiningen to AOT-compile db.migration.migrations such that the generated classes are in the classpath for REPL use. In other words, if db.migration.migrations changes, I don't have to manually recompile it before running lein repl — Leiningen will take care of that for me. With, I can define a task like this:
(defn compile-migrations
    {:src-dirs ["src"]
     :basis basis
     :ns-compile ['db.migration.migrations]
     :filter-nses ['db.migration.migrations]
     :class-dir "classes"}))
Then do clj -T:build compile-migrations before starting a REPL (with "classes" in the classpath) to make the classes available for dev. This does mean that I need to run clj -T:build compile-migrations manually whenever db.migration.migrations changes, though. That's not a big deal, but I just wanted to make sure my understanding's correct and that this is indeed the way to do this with


@flowthing why AOT-compile db-migrations at all for dev?


These migrations generate a bunch of Flyway classes via gen-class. I don't think there's an alternative to AOT-compiling them?


I guess you can always call compile manually from the REPL if those sources change


Right, but I'll have to restart the REPL after that, though.


I guess you could put this in your user.clj


(ns user)
(binding [*compile-path* "target/classes"]
  (compile 'db.migrations))


That was my first thought, but I'm not sure that there's a benefit over making a task that you need to run beforehand, since you need to restart the REPL anyway.


Also, :filter-nses is useful here, since I really don't want to AOT-compile anything except db.migration. migrations for dev.


the benefit would be preventing starting up 2 JVMs, so it will likely save 5 seconds or so of startup time


tools build runs in another JVM and then starts another JVM for compiling clj and then you start another JVM for your REPL


which can be a bit slow


But I only need to run the compile-migrations task (and spin up the second JVM) if the migrations change.


No need to run it every time I start a REPL.


Personally I would make a bb.edn with a bb dev task which checks if sources have changed using fs/modified-since which would then invoke tools build if necessary and then invokes your REPL


but you can do something similar in your user.clj as well


why use for this if you can just compile from the same dev REPL process, it's much cheaper to do so. (compile 'db.migration) is the same as :ns-compile '[db.migration]. Compilation is always a transitive process: all transitively required namespace will also be compiled, no matter what filter-nses is.


That's an interesting thought, although I'm not sure I want to bring in Babashka just for that. Also, there are many ways to start a REPL, so it's not something I'd be willing to bake into something like bb dev.


Sure, whatever tool you use to start the REPL (make, bash, manual invocation), it's conceptually the same idea: you need to do something, but only sometimes, before starting a REPL. Or you can do that work inside your REPL.


> why use for this if you can just compile from the same dev REPL process, it's much cheaper to do so. I don't know that it's cheaper since I need to restart the REPL anyway?


it's cheaper in terms of how much work is done. you're invoking 3 JVMs instead of just 1 REPL JVM.


but if that's not a problem, go for it ;)


Well, 3 vs. 2. 🙂 I'm not sure I'm convinced that the REPL approach will take up less time.


3 vs 1. Why do you think there's 2?


Need to restart the REPL.


Oh, then it's 4 vs 2.


1. Start REPL. 2. Eval (compile 'db.migration.migrations). Class files appear in classes/. 3. Shut down REPL. 4. Start new REPL with classes in classpath.


No :) Restart REPL. user.clj: check if sources have changed, then call compile then load the rest of your app. Just 1.


All right, I have no idea what's going on anymore...


To keep it simple, I think just a (compile 'db.migrations) in your user.clj as the first thing you do (regardless if anything's changed), will be sufficient.


When you change source, just restart your REPL and that's it.


Do not require any other namespaces from your app prior to that


Oh, right. That might be what's messing things up.


Just to be clear: do you mean having (compile 'db.migration.migrations) at the top level of user.clj?


Caused by: java.lang.RuntimeException: *compile-path* not set


If I do that.


(binding [*compile-path* "target/classes"] (compile '...))


Oh, I see. :thumbsup: Let me try that.


or whatever your classes dir is.


Cool, that seems to work! So now I have this:

(binding [*compile-path* "classes"]
  (compile 'db.migration.migrations))

(ns user (:require ,,,))

;; etc
I just need to ensure that classes/ exists. I guess the reason I'm confused is that I feel quite certain I tried a variant of this and couldn't get it to work, but maybe the key is having the (compile ,,,) call at the top level before the ns form? Another (entirely plausible) option is that I made a stupid mistake previously that left me under the impression that I would need to restart the REPL after compiling for the changes to take effect.


You can also write:

(ns user)

(binding [*compile-path* "classes"]
  (compile 'db.migration.migrations))

;; rest of your user.clj
(require ')


as an optimization you could do a check with fs/modified-since or similar to check if compilation is necessary, but I assume compilation itself doesn't take that long that it makes a huge difference


Yep, could use require instead. :thumbsup:


Yeah, compilation doesn't seem to take that long, but I'll keep that in mind.


To ensure classes exists: (fs/create-dirs "classes")


Well, I tried putting (binding [*compile-path* "classes"] (compile 'db.migration.migrations)) into (user/start) instead of at the top level and it still works. Better to have it at the top level so that the migrations are compiled even when I don't run (user/start), though. I have zero idea why it works now and didn't before, though, but I think Stupid User Error is the only possibility. 🙂


Congrats on getting it working


Nonetheless, many thanks for the help! This is much better than having the compile-migrations task. 🙂


Oh, not quite there yet, actually.

λ rm -rf classes/*
λ clj -T:build compile-migrations
λ ls classes
λ rm -rf classes/*
λ clj -M:dev -r # with (compile ,,,) in user.clj
Clojure 1.10.3

λ ls classes
clojure cprop   db      hugsql  myapp


Having other AOT-compiled classes than db.migration.migrations messes things up in dev. Well, need to revisit this tomorrow.


You cannot have (compile 'db.migration) only have the db space compiled. If it depends on other namespaces, those are also going to be compiled. This will be the same with But what filter-nses does is that it only copies some of the compiled files from a temporary dir to the class dir.


You could have the same if you set compile-path to some tmp-dir and then only copy over the db dir to the class dir.


Yes, I figured it must be something like that. Well, need to think about this a bit.


but since these are dependencies, they shouldn't be changing, so maybe it doesn't hurt to have their compiled namespaces in here too.


if you remove them, clojure will compile them into memory bytecode anyway, again 🤷


some people use compile to make an AOT cache for their deps to have their REPL start up faster, which is basically what you're doing here, but only for the transitive deps of db.migrations


It does seem to hurt, but I don't know why yet. I evaluated (tools.namespace.repl/refresh ,,,) and got an exception related to one of the AOT-compiled namespaces, but I didn't have the time to look further into it yet.


omg, tools namespace refresh... let's not go there ;)


I don't want to use t.n.r, but that's the way the project is set up, unfortunately. 🙂


Guess I could try to rip it out, maybe...


ok, delete everything from the classes dir except db and then you should have a similar thing as with tools.builder filter-nses


Yes, I'll consider that, thanks. :thumbsup: