Fork me on GitHub
#graalvm
<
2019-07-04
>
sogaiu03:07:05

regarding that, i read somewhere that it can help to label the issue appropriately. in this case i guess substratevm might be a good tag.

borkdude08:07:14

feel free to make a small repro yourself, if you want, I'm not able to run Windows at the moment on this borrowed laptop

borkdude08:07:51

I'd say: - make a small leiningen project code:

(defn -main [& args]
  (println (slurp *in*)))
- add instructions how to build an uberjar - compile the uberjar in linux and windows - show that the windows version behaves differently / incorrectly - attach a link to the repo + the uberjar itself (so they don't have to build it if they don't want to) to the issue

borkdude08:07:50

I think they're going to be looking at the uberjar .class files directly, and they don't care too much if it was built with Java or Clojure. That's my assumption but I could be wrong

sogaiu09:07:50

i will consider. btw, i tried the following variation of the code above and found the produced native image binary on windows works in all 4 shells (cmd.exe, sdk cmd.exe, windows powershell, powershell core):

(defn -main [& args]
  (binding [*in* (java.io.InputStreamReader. (java.io.FileInputStream. "project.clj"))]
    (println (slurp *in*))))

borkdude10:07:26

what if you wrap that in a LineNumberingPushbackReader like Clojure itself does?

final static public Var IN =
		Var.intern(CLOJURE_NS, Symbol.intern("*in*"),
		           new LineNumberingPushbackReader(new InputStreamReader())).setDynamic();

sogaiu10:07:07

You mean like this?

(defn -main [& args]
  (binding [*in* (LineNumberingPushbackReader. (java.io.InputStreamReader. (java.io.FileInputStream. "project.clj")))]
    (println (slurp *in*))))

sogaiu10:07:57

for the console, i have tried that combo and the thing that made a difference was setDynamic -- i haven't tried for a on-disk file yet, though so i will

sogaiu10:07:58

btw, do you recall how long your clj-kondo builds were taking on windows?

borkdude10:07:47

Maybe test for stdin:

(defn -main [& args]
  (let [in (clojure.lang.LineNumberingPushbackReader. (java.io.InputStreamReader. System/in))]
    (println (slurp in))))
to see if the problem is with binding dynamic vars, or with LineNumberingPushbackReader

borkdude10:07:55

(so slurping from stdin, not a file)

borkdude10:07:12

@sogaiu not sure, you can check appveyor, the badge is in the README on Github

sogaiu10:07:25

the first test of clojure.lang.LineNumberingPushbackReader made no difference

borkdude10:07:23

what about:

(defn -main [& args]
  (binding [*in* (java.io.InputStreamReader. System/in)]
    (println (slurp *in*))))

sogaiu10:07:56

i tried your let version and same results -- no stacktrace, worked fine. will try your latest binding code

sogaiu11:07:04

same results for latest binding code. i tried various combinations before (and have taken notes). the main difference i've noticed so far is the combination of in with its default leading to problems, but also defining a dynamic var and setting it to match what in gets initialized to leads to the stacktrace. what is quite noticeable is that the stacktrace happens in some of the shells, but interestingly, not all of the shells.

sogaiu11:07:34

my latest candidate for repro code is:

(defn -main [& args]
  (println "started")
  (slurp *in*)
  (println "finished"))
this leads to stacktraces in some shells.

sogaiu11:07:52

i do see output for "started" before the stacktrace, fwiw.

borkdude11:07:35

@sogaiu and if you rebind *in* to (java.io.InputStreamReader. System/in), what happens?

sogaiu11:07:19

that's your latest binding code, right?

borkdude11:07:59

yeah, so:

(defn -main [& args]
  (println "started")
  (binding [*in* (java.io.InputStreamReader. System/in)]
    (slurp *in*))
  (println "finished"))

borkdude11:07:42

if that works fine, there must be a problem with LineNumbering...Reader?

sogaiu11:07:15

no, i don't think so

sogaiu11:07:41

it does work fine, fwiw.

sogaiu11:07:19

i can get a reproduction of the problem if i def a separate var as dynamic and assign it what in is assigned.

sogaiu11:07:28

if i remove the ^:dynamic, there is no problem

borkdude11:07:05

how does this behave?

(defn -main [& args]
  (println "started")
  (binding [*in* *in*]
    (slurp *in*))
  (println "finished"))

sogaiu11:07:45

that works with no problem my bad, it doesn't work in some shells

borkdude11:07:07

interesting

sogaiu11:07:20

i forgot to comment out the older code

sogaiu11:07:36

sorry, trying again

sogaiu11:07:39

i got stacktraces for both flavors of powershell

sogaiu11:07:47

cmd.exe and sdk cmd.exe worked ok

borkdude11:07:56

does cmd.exe have a problem with clj-kondo right now, as is?

borkdude11:07:00

or only powershell?

sogaiu11:07:12

you mean your latest commit?

borkdude11:07:12

you could try the binary from appveyor

sogaiu11:07:17

i just pulled your latest commit and built. running w/o arguments, i get no output for: windows powershell, cmd.exe, and a fresh sdk cmd.exe however, in the sdk cmd.exe that i built clj-kondo.exe in, it runs w/o the help output...

sogaiu11:07:36

it runs w/ help output in powershell core

sogaiu11:07:54

i can also try with the appveyor version if you like

borkdude11:07:12

please do, as that's the one that's going to be distributed

sogaiu11:07:17

i just tested that one and found: works: powershell core, sdk cmd.exe that i did a build with doesn't work: fresh sdk cmd.exe, windows powershell, cmd.exe

sogaiu11:07:44

it is odd that the behavior differs between the 2 sdk cmd.exe instances

sogaiu11:07:58

i love mutable state

borkdude11:07:33

this doesn't make sense to me, sorry ๐Ÿ™‚

borkdude11:07:07

maybe it's a dll dependency problem again, but I don't know enough about this to solve it

sogaiu11:07:17

ah, on that front

sogaiu11:07:44

i got a side-by-side private assembly to work for clj-kondo.exe -- that means it skips the traditional dll search order

sogaiu11:07:15

some other projects seem to do something similar (e.g. ruby's windows installer)

borkdude11:07:45

if you combine the appveyor exe with those dlls, does that work?

sogaiu11:07:00

as far as i understand, there is only one dll that clj-kondo.exe needs that doesn't get supplied by a windows 10 system, and that's what comes with the vs c++ redit 2010 -- msvcr100.dll. i haven't been testing with the side-by-side thing, but i can do that in a bit.

borkdude11:07:13

what we ultimately want is to be able to build on appveyor (or circle in case @marc-omorain is watching) and then combine that with some redistributable dll in an installer (like scoop, chocolatey, whatever works)

sogaiu11:07:02

yes, i understand that automating the builds is important. i think it'd be worth having a conversation with the graal folks about whether they have either recommendations for doing that sort of thing with their current tools, or if that doesn't work so well with their current setup, whether they might be willing to make some changes in that direction going forward.

borkdude11:07:35

yeah, that's what this issue is about essentially: https://github.com/oracle/graal/issues/1407

sogaiu11:07:23

i believe it's in their interest too that people try their stuff and for that, automated builds help -- or put another way, without it, it's quite a disincentive to working on at least the windows portions because of the sheer amount of manual time that gets sucked away ๐Ÿ™‚

borkdude11:07:02

I wonder if there are any other clojure projects like clj-kondo that distribute binaries for different platforms

sogaiu11:07:18

the static stuff is also along those lines, because it should eliminate dll loading issues as well as ease deployment.

borkdude11:07:18

(using Graal)

sogaiu11:07:21

i wonder too.

sogaiu11:07:35

i have a few, but i'm waiting for this to get fixed -- and mine are small ๐Ÿ™‚

sogaiu11:07:22

i thought may be lvh or bruno bonacci might have some project along these lines

borkdude11:07:28

it would be effective if we could make repros that are clear and post them to graal, but this is difficult to do right now with Windows problems

sogaiu11:07:55

yes, it is a bit of a catch 22

borkdude11:07:59

or "black box" is a nicer way of putting it

borkdude11:07:08

so is GraalVM, but at least it works for linux and mac

sogaiu11:07:26

i have been reluctant to post because the results i get seem inconsistent and it's hard to find someone else who can reproduce

sogaiu11:07:34

yes, at least linux and macos seem to work pretty well

sogaiu11:07:07

what would be great is someone who knows clojure well working on the graal team...i can dream, can't i? ๐Ÿ˜‰

borkdude11:07:10

I'd say let's focus on a standalone cmd.exe (no SDK installed). if that works, using some redistributable, then I think it's good to look at VSCode on Windows. if that works, then Powershell as a nice to have

sogaiu11:07:46

hmm...well, i've been finding that which shells produce better results seem to vary with the releases, unfortunately

borkdude11:07:52

might take some time for GraalVM to mature on Windows

sogaiu11:07:00

yes, i am thinking so too

sogaiu11:07:21

did you try running via docker on windows?

borkdude11:07:49

didn't try, since I'm running Windows in a VM. Running Docker inside of that is too taxing on my machine

borkdude11:07:57

I can't even run the VM right now, my own laptop is in repair

sogaiu11:07:09

yes, sorry about that. hope it gets fixed soon.

borkdude11:07:22

I also ordered a new machine with 32GB memory, so then I could try again ๐Ÿ™‚

sogaiu11:07:44

ah, i hope that works out well. out of curiosity, which machine did you choose?

borkdude11:07:51

new Macbook Pro.

sogaiu11:07:07

wow, how nice ๐Ÿ™‚

borkdude13:07:25

very good! now I would post the text in the README with a link to your repro repo and the jars.

borkdude13:07:40

as an issue at graal/issues

sogaiu13:07:05

thanks, i will review before posting -- off to sleep. take care!

sb15:07:23

Hello, I try to use GraalVM on my macbook.. long time ago I didnโ€™t use.. and now I added the new Graal for OSX.. 1.8.0_212, x86_64: "GraalVM EE 19.1.0" /Library/Java/JavaVirtualMachines/graalvm-ee-19.1.0/Contents/Home .. and I get every time that error.. at test projects too.. eg https://www.astrecipes.net/blog/2018/07/20/cmd-line-apps-with-clojure-and-graalvm/ or cljtree

sb15:07:41

Exception in thread "main" java.lang.ExceptionInInitializerError
	at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:290)
	at java.lang.Class.ensureInitialized(DynamicHub.java:451)
	at clojure.lang.Namespace.<init>(Namespace.java:34)
	at clojure.lang.Namespace.findOrCreate(Namespace.java:176)
	at clojure.lang.Var.internPrivate(Var.java:153)
	at testgraal.core.<clinit>(Unknown Source)
	at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
	at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)
	at java.lang.Class.ensureInitialized(DynamicHub.java:451)
Caused by: java.io.FileNotFoundException: Could not locate clojure/core__init.class or clojure/core.clj on classpath.
	at clojure.lang.RT.load(RT.java:463)
	at clojure.lang.RT.load(RT.java:426)
	at clojure.lang.RT.doInit(RT.java:468)
	at clojure.lang.RT.<clinit>(RT.java:336)
	at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
	at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)
	... 8 more

sb15:07:49

Any idea why I got this..? lein clean && lein compile && lein uberjar and as in this docs native-image -jar target/cljtree-graalvm-0.1.0-SNAPSHOT-standalone.jar -H:Name="cljtree" --no-server J-Xmx3G -J-Xms3G -H:+ReportUnsupportedElementsAtRuntime

sb15:07:44

I tried -J-Xmx3G -J-Xms3G --no-server flag too.. but similar error. The jars work fine.. with java -jar

sb15:07:06

Just I want to write a command line application.. and pff.. ๐Ÿ™‚

sb15:07:22

maybe the problem .. native-image --version >> GraalVM Version 19.1.0 CE and I have GraalVM EE ? ๐Ÿ˜•

sb15:07:59

I installed the CE version.. not.. so I donโ€™t have idea.

sb15:07:19

Solved:

native-image --report-unsupported-elements-at-runtime \
             --initialize-at-build-time \
             -jar ./target/cljtree-graalvm-0.1.0-SNAPSHOT-standalone.jar \
             -H:Name=./target/cljtree

sb16:07:31

--report-unsupported-elements-at-runtime \ --initialize-at-build-time \

sb16:07:36

that solve the issue..

sb16:07:10

sorry, lot of โ€œspamโ€ here.