This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-11-28
Channels
- # announcements (1)
- # babashka (9)
- # beginners (82)
- # calva (6)
- # cider (3)
- # clj-kondo (69)
- # cljdoc (4)
- # cljs-dev (10)
- # cljsrn (2)
- # clojure (74)
- # clojure-europe (11)
- # clojure-italy (9)
- # clojure-nl (15)
- # clojure-spec (18)
- # clojure-uk (89)
- # code-reviews (8)
- # core-async (42)
- # cursive (22)
- # datomic (26)
- # fulcro (13)
- # graalvm (31)
- # graphql (1)
- # leiningen (20)
- # malli (19)
- # music (1)
- # off-topic (4)
- # pathom (56)
- # re-frame (3)
- # reitit (26)
- # shadow-cljs (40)
- # spacemacs (5)
- # tools-deps (25)
@taylor i just noticed that one of my projects no longer builds with the changes in this commit: https://github.com/taylorwood/clj.native-image/commit/b3823a48be75122b9671c86ce5353a85589ef15f (it does build with the commit right before it) the output from native-image looks like this:
Error: Unsupported features in 12 methods
Detailed message:
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved type during parsing: clojure.tools.reader.reader_types.IPushbackReader. To diagnose the issue you can use the --allow-incomplete-classpath option. The missing type is then reported at run time when it is accessed the first time.
Trace:
at parsing rewrite_clj.reader$unread.invokeStatic(reader.clj:115)
Call path from entry point to rewrite_clj.reader$unread.invokeStatic(Object, Object):
at rewrite_clj.reader$unread.invokeStatic(reader.clj:112)
at rewrite_clj.reader$unread.invoke(reader.clj:112)
at clojure.tools.reader.default_data_readers.proxy$java.lang.ThreadLocal$ff19274a.equals(Unknown Source)
at java.util.HashMap.getNode(HashMap.java:579)
at java.util.HashMap.get(HashMap.java:557)
at com.oracle.svm.jni.access.JNIReflectionDictionary.getClassObjectByName(JNIReflectionDictionary.java:128)
at com.oracle.svm.jni.functions.JNIFunctions.FindClass(JNIFunctions.java:314)
at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_FindClass_3ec1032c6cb9443725d1e68194130533bfb04076(generated:0)
i tried what you mentioned in: https://github.com/taylorwood/clj.native-image/issues/15#issuecomment-550573681 , but got a result similar to what you got.
i had "--initialize-at-build-time"
before (build unsuccessful) and changed this to "--initialize-at-build-time=clojure"
i've added some info to https://github.com/taylorwood/clj.native-image/issues/15
I’m really interested about the thing where upgrading that dependency introduced a problem
hmm, not quite sure what you mean about upgrading tools.deps introducing a problem. are you referring to: https://github.com/taylorwood/clj.native-image/commit/b3823a48be75122b9671c86ce5353a85589ef15f ? atm, it looks to me like it's tools.namespace that is a culprit.
i have toggled the version back and forth and tried building with just the version changed -- it appears reproducible here, as in, with the newer version i get errors, with the older version it's fine
the project i tried with apart from fc4, is: https://github.com/sogaiu/adorn -- i'm not sure if that one will work for other folks because one of its dependencies may have been broken within the last day, but perhaps it's worth a try?
i compared the -imagecp values for the 2 runs of building adorn and found some differences. tn 0.2.11 (build succeeds), has:
tools.namespace-0.2.11.jar
tools.reader-1.2.2.jar
tn 0.3.1 (build fails), has:
tools.namespace-0.3.1.jar
java.classpath-0.3.0.jar
tools.reader-1.3.2.jar
i checked the value of:
(mapcat (comp find-namespaces-in-dir io/file) (:paths deps-map))
and in both runs it was:
(script)
so may be it's not like there was change in the behavior of find-namespaces-in-dir
i also checked the 2 runs of building fc4 and found the following differences among the -imagecp values: tn 0.2.11, has:
tools.namespace-0.2.11.jar
tools.reader-1.0.0-beta4.jar
tn 0.3.1, has:
tools.namespace-0.3.1.jar
java.classpath-0.3.0.jar
tools.reader-1.3.2.jar
the values for: (mapcat (comp find-namespaces-in-dir io/file) (:paths deps-map))
were identical:
(fc4.files fc4.image-utils fc4.integrations.structurizr.express.chromium-renderer fc4.integrations.structurizr.express.export fc4.integrations.structurizr.express.format fc4.integrations.structurizr.express.snap fc4.integrations.structurizr.express.spec fc4.integrations.structurizr.express.yaml .cli.main .cli.util .dsl .render .util fc4.io.watch .yaml fc4.model fc4.rendering fc4.spec fc4.styles fc4.util fc4.view fc4.yaml)
yeah that’s the first thing that came to my mind is checking to see if the calculated classpaths were different :thinking_face:
the type of error i'm seeing is:
Error: Unsupported features in 12 methods
Detailed message:
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved type during parsing: clojure.tools.reader.reader_types.IPushbackReader. To diagnose the issue you can use the --allow-incomplete-classpath option. The missing type is then reported at run time when it is accessed the first time.
Trace:
at parsing rewrite_clj.reader$unread.invokeStatic(reader.clj:115)
Call path from entry point to rewrite_clj.reader$unread.invokeStatic(Object, Object):
at rewrite_clj.reader$unread.invokeStatic(reader.clj:112)
at rewrite_clj.reader$unread.invoke(reader.clj:112)
at clojure.tools.reader.default_data_readers.proxy$java.lang.ThreadLocal$ff19274a.equals(Unknown Source)
at java.util.HashMap.getNode(HashMap.java:579)
at java.util.HashMap.get(HashMap.java:557)
at com.oracle.svm.jni.access.JNIReflectionDictionary.getFieldNameByID(JNIReflectionDictionary.java:278)
at com.oracle.svm.jni.functions.JNIFunctions.ToReflectedField(JNIFunctions.java:836)
at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_ToReflectedField_80d8233579d5215df0227b770e5c01228a0de9b9(generated:0)
some graal issues with similar errors include: https://github.com/oracle/graal/issues/1664 https://github.com/oracle/graal/issues/1513 https://github.com/oracle/graal/issues/1508
building with --allow-incomplete-classpath
succeeds -- startup of the resulting binary is successful without a crash, but when i try to use it for what it's supposed to do, it crashes with:
Exception in thread "main" java.lang.NoClassDefFoundError: clojure.tools.reader.reader_types.IndexingReader
at rewrite_clj.reader$position.invokeStatic(reader.clj:125)
at rewrite_clj.reader$position.invoke(reader.clj:122)
at rewrite_clj.reader$read_with_meta.invokeStatic(reader.clj:131)
at rewrite_clj.reader$read_with_meta.invoke(reader.clj:128)
at rewrite_clj.parser.core$parse_next.invokeStatic(core.clj:36)
at rewrite_clj.parser.core$parse_next.invoke(core.clj:34)
at rewrite_clj.parser$parse.invokeStatic(parser.clj:13)
at rewrite_clj.parser$parse.invoke(parser.clj:10)
at rewrite_clj.parser$parse_all$fn__2947.invoke(parser.clj:18)
at clojure.core$repeatedly$fn__6176.invoke(core.clj:5089)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.RT.seq(RT.java:528)
at clojure.core$seq__5124.invokeStatic(core.clj:137)
at clojure.core$take_while$fn__5638.invoke(core.clj:2896)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.RT.seq(RT.java:528)
at clojure.core$seq__5124.invokeStatic(core.clj:137)
at clojure.core$dorun.invokeStatic(core.clj:3125)
at clojure.core$doall.invokeStatic(core.clj:3140)
at clojure.core$doall.invoke(core.clj:3140)
at rewrite_clj.parser$parse_all.invokeStatic(parser.clj:20)
at rewrite_clj.parser$parse_all.invoke(parser.clj:15)
at rewrite_clj.parser$parse_string_all.invokeStatic(parser.clj:35)
at rewrite_clj.parser$parse_string_all.invoke(parser.clj:32)
at rewrite_clj.zip.base$of_string.invokeStatic(base.clj:73)
at rewrite_clj.zip.base$of_string.invoke(base.clj:69)
at script$find_defn_zloc_at.invokeStatic(script.clj:12)
at script$find_defn_zloc_at.invoke(script.clj:10)
at script$_main.invokeStatic(script.clj:37)
at script$_main.doInvoke(script.clj:27)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at script.main(Unknown Source)
i tried adding clojure.tools.reader.reader_types.IndexingReader
to reflection.json but during a subsequent native-image build i see:
WARNING: Could not resolve clojure.tools.reader.reader_types.IndexingReader for reflection configuration.
@taylor oddly enough if i use lein to build an uberjar first, native-image doesn't appear to have any issues
3 years later, I’m hitting the same thing. And I guess it must be a classpath issue. The classpath running
clojure -M -e "(System/getProperty \"java.class.path\")"
is very different from the one running
clj -Anative-image
The second one has a shit ton more classes and - curiously - if I compile my main namespace with
clojure -M -e "(compile ')"
I see all the clojure.tools.reader.reader_types classes compiled.
If I let clj.native
do the compile, they are not there.
As I’ve pointed out a few times: tools.deps is not a stable build tool. The classpath order is not defined and the second classpath might have an older version of clojure.tools.reader packaged up somewhere and that’s why the build breaks.in that case clj.native-image requires tools.namespace 1.0.0 which in turn requires an older version of clojure.tools.reader
I am a complete Graal neophyte, and I keep hearing stuff like "Graal is based on a closed world assumption". Here's what I'm wondering: 1. How does this tie into the dynamic nature of Clojure? 2. Which subset of Clojure's features and/or constructs are compatible with Graal? 3. Do the answers to the above two questions depend on whether we're talking about GraalVM HotSpot mode vs native image?
Compiling stuff to a native image does prohibit a lot of dynamic runtime stuff you could otherwise do with Clojure
@jaihindhreddy I'm only familiar with native image. Closed world means that it analyses reachable code at compile time and will strip everything not needed.
See https://github.com/lread/clj-graal-docs/blob/master/doc/external-resources.md for a list of projects that are currently working in graal