This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-12-20
Channels
- # adventofcode (29)
- # announcements (7)
- # aws (1)
- # babashka (3)
- # beginners (43)
- # biff (20)
- # clj-kondo (44)
- # cljs-dev (20)
- # clojure (74)
- # clojure-europe (24)
- # clojure-finland (2)
- # clojure-nl (13)
- # clojure-norway (3)
- # clojurescript (31)
- # code-reviews (1)
- # community-development (12)
- # cursive (3)
- # datomic (6)
- # emacs (1)
- # fulcro (25)
- # interop (7)
- # introduce-yourself (2)
- # leiningen (30)
- # nbb (3)
- # overtone (1)
- # podcasts-discuss (5)
- # polylith (24)
- # practicalli (1)
- # reclojure (1)
- # reitit (13)
- # rum (7)
- # shadow-cljs (12)
- # sql (23)
- # squint (51)
- # test-check (1)
- # testing (2)
- # tools-deps (2)
I'm trying to build an uberjar without sources.
The :omit-source
isn't enough, as I learned here: https://github.com/technomancy/leiningen/issues/1357
However, the https://github.com/technomancy/leiningen/issues/1357#issuecomment-27005055 with :uberjar-exclusions
doesn't seem to work for me.
This is my uberjar profile
:uberjar {:aot :all
:omit-source true
;; :omit-source isn't enough;
;; make sure to omit sources from dependencies like analysis lib:
:uberjar-exclusions [#"\.(clj|java)"]}
The trouble is that the JAR fails to start because of java.io.FileNotFoundException: Could not locate clojure/core/specs/alpha__init.class, clojure/core/specs/alpha.clj or clojure/core/specs/alpha.cljc on classpath
at clojure.lang.Util.loadWithClass(Util.java:251)
at my.main.<clinit>(Unknown Source)
Caused by: Syntax error macroexpanding clojure.core/extend-protocol at (0:0).
at clojure.lang.Compiler.checkSpecs(Compiler.java:6989)
at clojure.lang.Compiler.macroexpand1(Compiler.java:7005)
at clojure.lang.Compiler.macroexpand(Compiler.java:7092)
at clojure.lang.Compiler.eval(Compiler.java:7178)
at clojure.lang.Compiler.eval(Compiler.java:7149)
at clojure.core$eval.invokeStatic(core.clj:3215)
at clojure.core$eval.invoke(core.clj:3211)
at clojure.core.matrix.impl.persistent_vector__init.load(Unknown Source)
at clojure.core.matrix.impl.persistent_vector__init.<clinit>(Unknown Source)
... 574 more
Caused by: java.io.FileNotFoundException: Could not locate clojure/core/specs/alpha__init.class, clojure/core/specs/alpha.clj or clojure/core/specs/alpha.cljc on classpath.
at clojure.lang.RT.load(RT.java:462)
at clojure.lang.RT.load(RT.java:424)
at clojure.lang.Compiler.ensureMacroCheck(Compiler.java:6975)
at clojure.lang.Compiler.checkSpecs(Compiler.java:6987)
... 582 more
Please try unzipping the jar file and looking inside of it. You will have to do this anyway to ensure that you have successfullly removed the source code.
• Without :omit-source
and without :uberjar-exclusions
• With :omit-source
and without :uberjar-exclusions
• With :omit-source
and with :uberjar-exclusions
Once unzipped you can use tree
command to list files and maybe compare the output using diff --color
I did that already multiple times.
This is how I found plain :omit-source
doesn't work (it contains both class files and source code)
There are so many clj files but I guess I need to have a look at clojure/core/specs and see what's in the sources that doesn't get compiled..
Yes - focus on clojure.core to start off.
Possible approach: • If you have internal libraries then you can run aot + omit source on them first when you compile/install them. • Then build your application uberjar with aot + omit-source to generate the final uberjar then none of your own source code will be in the uberjar but there will be code for open source libraries that may not matter to you.
I considered that option but didn't want to complicate the build. Then I didn't know exactly how to do it O:-).
> This is how I found plain :omit-source
doesn't work (it contains both class files and source code)
For code in your application project or libraries?
Comparing two versions of the jar (omit-source without exclusions VS omit-source with exclusions) core specs seems to be missing
# this is in the uberjar without exclusions
ll clojure/core/specs/alpha.clj
-rw-r--r-- 1 jumar staff 7.7K Sep 14 2021 clojure/core/specs/alpha.clj
# this directory in the built uberjar is empty
ls clojure/core/specs/
https://clojurians.slack.com/archives/C0AB48493/p1671546736854739?thread_ts=1671543091.023669&cid=C0AB48493 Definitely for libraries.
Yeah - so it looks like it's uberjar + aot is not going to aot compile all of your dependencies. Then your exclusion is removing the required clj files.
I think you can just add :aot true
+ :omit-source true
to all of your internal libraries + the final application.
That way the final uberjar will have none of your code but will have open source code.
Would that be ok for your usecase?
Yes, I need to try that. But as a special profile or something because I don't want to do that during normal (dev) workflow. Do you know what's the easiest way to do it?
I have a feeling that approach will work although haven't done it in a while.
I mean what we do right now is simply
cd A && lein install
cd ../B && lein install
cd ../C && lein uberjar
Yup - so process is the same - just adding :aot :all
+ :omit-source true
to each project.clj file.
@U06BE1L6T when you do :aot
you should automatically AOT all of your dependencies and transitive ddeps - this would include clojure.spec stuff. You should then be able to just do the :uberjar-exclusions
you did before. I don’t see why that wouldn’t work. I don’t see why you’d need a setup any harder than putting these extra options in a profile you use only for uberjar’ing
:uberjar {:omit-source true :aot :all :uberjar-exclusions [#"\.(clj|java)"]}
Clojure AOT wil aggressively compile all clj files that are encountered on any use
/`require`/`load` call. If you are using a lein version newer than 1.4.1 then the option :clean-non-project-classes
should be false/off by default - meaning it all AOT class files will remain in your compile path (and get into uberjars). https://github.com/technomancy/leiningen/blob/2.10.0/NEWS.md#141--2010-12-16
@U0LK1552A Yes, that's what I hoped for. However, it breaks on core specs for some reason. I'm really using the uberjar config that I pasted above - the same one as you stated here:
:uberjar {:omit-source true :aot :all :uberjar-exclusions [#"\.(clj|java)"]}
But it just doesn't work 😞
If it matters, I'm using this version:
Leiningen 2.9.10 on Java 1.8.0_302 OpenJDK 64-Bit Server VM
I'm not sure I understand.
What do you mean by loading the missing spec ns?
We use spec in many places and require it as needed.
However, I don't think we ever reference directly clojure.core.specs.alpha
As long as it required transitively somehow. Otherwise you wouldn’t get an AOT classfiles produced
I tracked it down to the use of core.matrix library: https://github.com/mikera/core.matrix
I created a minimal reproducer here: https://github.com/jumarko/uberjar-sample/blob/master/app/README.md
I don't understand in detail how clojure.core.specs.alpha
work but I noticed it's loaded automatically in Compile.java https://github.com/clojure/clojure/blob/b1b88dd25373a86e41310a525a21b497799dbbf2/src/jvm/clojure/lang/Compile.java#L58
I even tried to add an explicit require for clojure.core.specs.alpha
but it didn't help: https://github.com/jumarko/uberjar-sample/commit/3e3e63db7ab48f49130d8a386d8f733e7c3b65a6
I had to whitelist clojure/core/specs/alpha.clj
to make this work: https://github.com/jumarko/uberjar-sample/commit/6d26f3cceb48b2ab2acc0b6553cc1db8eb49b580
:uberjar-exclusions [#"(?<!clojure/core/specs/alpha)\.(clj|java)"]
That looks like a good solution.
It's been unfortunately much more complicated in the real project, due to the usage of duct/integrant, ring params middleware and more. This stuff loads namespaces dynamically and simply breaks if sources are ommited. I'm in the process of excluding only specific set of namespaces instead of trying to exclude everything...
Maybe try adding a big (:require [.....])
statement to your main namespace listing all of the namespaces that you are likely to end up loading dynamically. Obviously this may not be ideal for some situations.