Fork me on GitHub
#graalvm
<
2019-06-05
>
ikitommi19:06:36

oh, that is awesome. Will test if that helps with my Immutant Voyage

lvh19:06:25

Hi, I'm tyring to use native-image with specter and it sometimes works -- I ran into this issue: https://github.com/nathanmarz/specter/issues/271 when using a path that's not known at compile time. I've tried select* but apparently that's not good enough I've put the code up here: https://github.com/latacora/casper The exception looks like it's hitting from closed-code:

Exception in thread "main" com.oracle.svm.core.jdk.UnsupportedFeatureError: Unsupported method java.lang.ClassLoader.defineClass(String, byte[], int, int) is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class, compiling:(NO_SOURCE_PATH:0:0)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:7010)
	at clojure.lang.Compiler.analyze(Compiler.java:6773)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6991)
	at clojure.lang.Compiler.analyze(Compiler.java:6773)
	at clojure.lang.Compiler.analyze(Compiler.java:6729)
	at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6100)
	at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6420)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:7003)
	at clojure.lang.Compiler.analyze(Compiler.java:6773)
	at clojure.lang.Compiler.analyze(Compiler.java:6729)
	at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6100)
	at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5460)
	at clojure.lang.Compiler$FnExpr.parse(Compiler.java:4022)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:7001)
	at clojure.lang.Compiler.analyze(Compiler.java:6773)
	at clojure.lang.Compiler.eval(Compiler.java:7059)
	at clojure.lang.Compiler.eval(Compiler.java:7025)
	at clojure.core$eval.invokeStatic(core.clj:3206)
	at com.rpl.specter.impl$closed_code.invokeStatic(impl.cljc:617)
	at com.rpl.specter.impl$fn__2322$eval_PLUS___2330.invoke(impl.cljc:636)
	at com.rpl.specter.impl$mk_dynamic_path_maker.invokeStatic(impl.cljc:915)
	at com.rpl.specter.impl$magic_precompilation.invokeStatic(impl.cljc:949)
	at latacora.casper$_main.invokeStatic(casper.clj:14)
	at latacora.casper$_main.doInvoke(casper.clj:8)
	at clojure.lang.RestFn.invoke(RestFn.java:410)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at latacora.casper.main(Unknown Source)
Caused by: com.oracle.svm.core.jdk.UnsupportedFeatureError: Unsupported method java.lang.ClassLoader.defineClass(String, byte[], int, int) is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class
	at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:102)
	at java.lang.ClassLoader.defineClass(Target_java_lang_ClassLoader.java:642)
	at clojure.lang.DynamicClassLoader.defineClass(DynamicClassLoader.java:46)
	at clojure.lang.Compiler$ObjExpr.getCompiledClass(Compiler.java:4979)
	at clojure.lang.Compiler$FnExpr.parse(Compiler.java:4109)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:7001)
	... 27 more
The path isn't terribly complicated:
(def INDEXED
  "A path that visits v and collects k in [[k v], ...].

  This is useful if you want to collect a path to something, see [[path-walker]]
  and [[path-finder]]. "
  [sr/ALL (sr/collect-one sr/FIRST) sr/LAST])

(def INDEXED-SEQ
  "A selector that visits all elements of a seq, and collects their indices.

  This is useful if you want to collect a path to something, see [[path-walker]]
  and [[path-finder]]."
  [(sr/view #(map-indexed vector %)) INDEXED])

(def path-finder
  "Finds the first entry matching `pred` in a deeply nested structure of maps
  and vectors, and collects the path on the way there."
  (sr/recursive-path
   [term-pred] p
   (sr/cond-path
    (sr/pred term-pred) sr/STAY
    map? [INDEXED p]
    coll? [INDEXED-SEQ p])))

lvh20:06:09

I guess the real question is how to avoid that eval; FWIW I tried the native-image tracing tool linked earlier and setting it up was easy but that just gets me to:

com.oracle.svm.core.util.UserError$UserException: Error parsing reflection configuration in file:/home/lvh/src/Latacora/casper/resources/META-INF/native-image/reflect-config.json:
Class cheshire.core$eval1001 not found

lvh20:06:19

of course that doesn't work, that class doesn't "exist yet" 🙂

borkdude20:06:20

I have yet to try out that tracing tool. Maybe it’s also something @taylor would like to incorporate in his tools?

taylor20:06:49

Hope to have time to try it out soon 😌

borkdude20:06:33

Me too. 🙂

lvh20:06:24

I just ran:

#!/usr/bin/env bash
set -Exeou pipefail
CFG_DIR="resources/META-INF/native-image/"
mkdir -p "${CFG_DIR}"
JAVA_HOME="${GRAALVM_HOME}" PATH="${GRAALVM_HOME}/bin:${PATH}" \
         clj \
         -J-agentlib:native-image-agent=config-merge-dir="${CFG_DIR}" \
         $@
... and it worked fine

lvh20:06:02

you want resources/, not classes/ because something, possibly @taylor's thing, cleaned out classes and then compiled it again

taylor20:06:40

Sorry, don’t understand what this is referencing

lvh20:06:40

the documentation for the tracing tool suggests using META-INF/native-image as a target directory

lvh20:06:58

the rationale is that as long as there's a META-INF/native-image on the classpath when you're running native-image, it'll pick up the configs

lvh20:06:21

my first guess was to use classes/META-INF/native-image since IIUC by default . isn't on the classpath but classes is

lvh20:06:42

that didn't appear to do anything, and then looking at classes/ I realized the META-INF dir had vanished after running native-image

taylor20:06:44

I see, will try to look into this!

lvh20:06:12

so, my hack is to use resources/ because nobody tries to delete resources/

taylor20:06:09

Yeah it does clear out everything under *compile-path*

taylor20:06:20

Did you try using the extra paths key?

lvh23:06:55

Nope I was just hacking to get it to work

lvh23:06:05

I’m sure extra paths would just work also

lvh23:06:26

If you feel that shell script is useful feel free to use it btw

lvh23:06:33

It’s almost trivial?

taylor00:06:11

Thanks! I’ve only used reflection config files before and just specified the path to them explicitly, but I’ll have to try this out when I get some laptop time

lvh20:06:21

this conveniently got rid of META-INF so the generated config did not get picked up

taylor20:06:05

I saw the profiling thing earlier today and hope to have time to at least add some examples of how to use it

lvh20:06:22

i just always use config-merge-dir, I don't understand why the other option exists

lvh20:06:30

if you don't have config files yet it'll warn

lvh20:06:38

if you really want fresh files, just delete the directory