This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-06-25
Channels
- # announcements (2)
- # asami (16)
- # babashka (55)
- # beginners (27)
- # calva (14)
- # cider (5)
- # clj-kondo (16)
- # cljs-dev (22)
- # clojure (72)
- # clojure-europe (89)
- # clojure-nl (10)
- # clojure-uk (7)
- # clojured (1)
- # clojurescript (14)
- # community-development (4)
- # core-async (15)
- # emacs (10)
- # events (2)
- # fulcro (3)
- # graalvm (1)
- # graalvm-mobile (71)
- # helix (7)
- # honeysql (2)
- # introduce-yourself (1)
- # jobs-discuss (17)
- # juxt (3)
- # lsp (62)
- # malli (13)
- # meander (7)
- # off-topic (14)
- # pathom (54)
- # polylith (6)
- # re-frame (11)
- # releases (1)
- # sci (22)
- # sql (9)
- # tools-deps (84)
- # vim (37)
- # xtdb (18)
@smith.adriane @raspasov It's worth trying to not use --initialize-at-build-time
but using this approach:
https://github.com/oracle/graal/discussions/3476#discussioncomment-897705
to see if this helps.
I'm trying it out. Just blindly following the example produces the following errors:
To see how the classes got initialized, use --trace-class-initialization=primitive_math$variadic_proxy,primitive_math$using_primitive_operators_QMARK_,primitive_math$use_primitive_operators,primitive_math$variadic_predicate_proxy,primitive_math$unuse_primitive_operators
[mobiletest-uber:95490] analysis: 40,021.43 ms, 3.22 GB
Error: Classes that should be initialized at run time got initialized during image building:
primitive_math$variadic_proxy was unintentionally initialized at build time. To see why primitive_math$variadic_proxy got initialized use --trace-class-initialization=primitive_math$variadic_proxy
primitive_math$using_primitive_operators_QMARK_ was unintentionally initialized at build time. To see why primitive_math$using_primitive_operators_QMARK_ got initialized use --trace-class-initialization=primitive_math$using_primitive_operators_QMARK_
primitive_math$use_primitive_operators was unintentionally initialized at build time. To see why primitive_math$use_primitive_operators got initialized use --trace-class-initialization=primitive_math$use_primitive_operators
primitive_math$variadic_predicate_proxy was unintentionally initialized at build time. To see why primitive_math$variadic_predicate_proxy got initialized use --trace-class-initialization=primitive_math$variadic_predicate_proxy
primitive_math$unuse_primitive_operators was unintentionally initialized at build time. To see why primitive_math$unuse_primitive_operators got initialized use --trace-class-initialization=primitive_math$unuse_primitive_operators
com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
primitive_math$variadic_proxy was unintentionally initialized at build time. To see why primitive_math$variadic_proxy got initialized use --trace-class-initialization=primitive_math$variadic_proxy
primitive_math$using_primitive_operators_QMARK_ was unintentionally initialized at build time. To see why primitive_math$using_primitive_operators_QMARK_ got initialized use --trace-class-initialization=primitive_math$using_primitive_operators_QMARK_
primitive_math$use_primitive_operators was unintentionally initialized at build time. To see why primitive_math$use_primitive_operators got initialized use --trace-class-initialization=primitive_math$use_primitive_operators
primitive_math$variadic_predicate_proxy was unintentionally initialized at build time. To see why primitive_math$variadic_predicate_proxy got initialized use --trace-class-initialization=primitive_math$variadic_predicate_proxy
primitive_math$unuse_primitive_operators was unintentionally initialized at build time. To see why primitive_math$unuse_primitive_operators got initialized use --trace-class-initialization=primitive_math$unuse_primitive_operators
I'll try to use --trace-class-initialization
and see if I can get a working list@smith.adriane you'll need to use the namespace list of your specific application, generated using the sniippet I provided
just noticed that your list starts with "clojure", but it doesn't show up in mine
(->> (map ns-name (all-ns)) (remove #(str/starts-with? % "clojure")) (map #(str/split (str %) #"\.")) (keep butlast) (map #(str/join "." %)) distinct (map munge) (cons "clojure"))
see the endwhoops. I think just forgot how to read for a second
trying with "primitive_math" included
execute the above in a REPL, but first do:
(require 'your.main)
(require '[clojure.string :as str])
I added the following function:
(defn initialize-at-build-time-list [& args]
(println
(->> (map ns-name (all-ns))
(remove #(clojure.string/starts-with? % "clojure"))
(map #(clojure.string/split (str %) #"\."))
(keep butlast)
(map #(clojure.string/join "." %))
distinct
(map munge)
(cons "clojure")
(clojure.string/join ","))))
so I can have it referenced in my compile script:
INITIALIZE_AT_BUILD_TIME=`clojure -X com.phronmophobic.mobiletest/initialize-at-build-time-list
`
...
--initialize-at-build-time="$INITIALIZE_AT_BUILD_TIME" \
set -x
will print it as its executed
+ INITIALIZE_AT_BUILD_TIME=clojure,tech.v3.datatype,sci.impl,babashka.nrepl,tech.v3.resource,bencode,flatland.ordered,com.phronmophobic,babashka.nrepl.impl,datascript,edamame.impl,tech.v3,tech.v3.parallel,sci.addons,tech.v3.datatype.ffi,sci,primitive_math
com.phronmophobic
primitive_math$variadic_predicate_proxy was unintentionally initialized at build time. com.phronmophobic.mobiletest caused initialization of this class with the following trace:
at primitive_math$variadic_predicate_proxy.<clinit>(primitive_math.clj:27)
does order matter?
It's used by dtype-next
wasn't me! 😛
you'll probably run into lots of classes from this library that you should explicitly list...
is it fixable without graalvm changes?
just by adding primitive_math$variadic_predicate_proxy
and whatever else it complains about?
you could perhaps do this using the list of class files in the classes/primitive_math
directory or so
I've added all the classes that were printed
compiling now
if only they returned the error as edn 😛 (or even json).
{:unintentially-initialized ["primitive_math$variadic_predicate_proxy" ...]
:error "Classes that should be initialized at run time got initialized during image building"}
the snippet was a wishful example of native image output
it seems like it's compiling
it compiled!
rather than using:
(defn initialize-at-build-time-list [& args]
(println
(->> (map ns-name (all-ns))
(remove #(clojure.string/starts-with? % "clojure"))
(map #(clojure.string/split (str %) #"\."))
(keep butlast)
(map #(clojure.string/join "." %))
distinct
(map munge)
(cons "clojure")
(clojure.string/join ","))))
It seems like it would be more robust to just examine the classes in the uberjar. Would that work?@smith.adriane well, the point in the linked discussion is that not all classes should be initialized at build time, only the clojure ones
so if there are other Java classes in your jar then those should not be listed preferably
oh, interesting
and the deadlock can happen because of initializing at build time for some of the Java stuff I think
but it remains to be seen if the above approach has really solved the issue. did you push the change? I could try it locally
Will do. one sec
I will also try this approach for https://github.com/borkdude/tools-deps-native-experiment which has a similar issue
ok, pushed the changes
It seems like it should be possible to generate the list of clojure classes since projects like depstar need to produce a list for AOT compilation
Using the package for initialize at build time can have false positive since it's possible to have a java class that shares the same package as clojure code. I tried filtering all classnames in an uber jar by matching them with namespaces:
(defn list-resources [path]
(let [jar (java.util.jar.JarFile. path)
entries (.entries jar)]
(loop [result []]
(if (.hasMoreElements entries)
(recur (conj result (.. entries nextElement getName)))
result))))
(defn namespace->namespace-key [ns]
(-> ns
ns-name
str
(clojure.string/split #"\.")
(->> (map munge))))
(defn class-path->namespace-key [fname]
(-> fname
(clojure.string/replace #"(\$[^.]+)?.class$" "")
(clojure.string/split #"/"))
)
(defn class-path->classname [fname]
(-> (subs fname 0 (- (count fname)
(count ".class")))
(clojure.string/replace #"/" ".")))
(defn clojure-classes-in-jar [jar-path]
(let [resources (list-resources jar-path)
ns-keys (into
#{}
(map namespace->namespace-key)
(all-ns))
classes (into #{}
(comp
(filter #(clojure.string/ends-with? % ".class"))
(filter (fn [class-path]
(contains? ns-keys (class-path->namespace-key class-path))))
(map class-path->classname))
resources)]
classes))
(comment
(spit "initialize-at-buildtime-classes.txt"
(clojure.string/join ","
(clojure-classes-in-jar "target/mobiletest-uber.jar")))
;; INITIALIZE_AT_BUILD_TIME=$(cat initialize-at-buildtime-classes.txt)
;; --initialize-at-build-time=clojure,"$INITIALIZE_AT_BUILD_TIME" \
)
This doesn't quite work because it doesn't include classes created by defrecord
. It's also a comically large list.However it should be possible to modify the above code to include defrecord
classes if including java classes that share packages with clojure namespaces isn't a big deal
> if including java classes that share packages with clojure namespaces isn't a big deal sometimes it is, in the case of httpkit for example
I guess it should also be possible to list all classes created by defrecord and include them when filtering classes in an uberjar