This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-06-25
Channels
- # announcements (1)
- # beginners (338)
- # calva (41)
- # cider (19)
- # cljdoc (10)
- # cljsrn (6)
- # clojure (116)
- # clojure-europe (15)
- # clojure-italy (25)
- # clojure-nl (5)
- # clojure-spec (19)
- # clojure-uk (52)
- # clojurescript (99)
- # clojurex (14)
- # cursive (47)
- # data-science (1)
- # datomic (5)
- # duct (1)
- # figwheel (13)
- # fulcro (58)
- # graalvm (93)
- # jobs (3)
- # joker (9)
- # luminus (4)
- # nrepl (21)
- # off-topic (41)
- # pathom (25)
- # re-frame (7)
- # reitit (8)
- # ring-swagger (13)
- # tools-deps (13)
i've replicated the problem with using the chocolatey-based windows-sdk-7.1 -- attempts to build a native image result in the error mentioned here: https://github.com/oracle/graal/issues/1258#issue-442767728 -- this was in a windows 10 vm. will try a manual set up later.
looks like uninstalling the chocolatey windows-sdk-7.1 package and then manually installing the windows sdk makes things work out -- at least the build is proceeding further now.
@borkdude by modifying PATH on a machine where clj-kondo.exe did not work, i had success in getting it to work
i compared some dependency walker info for a clj-kondo.exe working in one vm with one that did not work on a host machine. some differences i noticed include: 1) the working side used dlls from openjdk for some things while the not-working side used dlls from powershell core 2) it didn't seem to be a case of the working side was finding more dlls than the not-working side -- if anything it looks like at least for the set of things satisfied by openjdk vs powershell core, the working side found less i tweaked PATH (so on windows, i guess this also affects how dlls are located...) for the not-working side so that it would pick up some dlls via openjdk instead of powershell core, and the result was that clj-kondo.exe started working.
my current impression is that it's not clear whether the problem is a single dll -- a different set of dlls is being picked up so it may be more than one
i will try to see if dependency walker can export more info about the files -- i only saw paths when i did the comparison
so maybe this problem can be solved with scoop or chocolatey by setting a path to the right dlls or having a dependency on openjdk? (cc @rahul080327)
possibly -- don't know if you looked at the stackoverflow info on /mt vs /md, but my impression is that the choice the graal folks have made at the moment by not supporting static is one of the contributing factors to problems like this. i don't know if it's practical for them to support --static any time soon, but that seems like a better way to address the issue -- and get more folks being willing to test native-image.
iiuc, on windows, PATH affects which dlls get used by a program -- i think this explains why i kept seeing such varied behavior. installing other software in / uninstalling software from your environment can destabilize execution of existing software because that can lead to changes in PATH, which unfortunately can affect which dlls a program sees / uses.
if scoop or chocolatey can arrange to have a startup exe or script that sets PATH appropriately right before launching clj-kondo, may be that can address things?
maybe yes. I think Rahul mentioned something about an installer, that bundles dlls, but I don't know how that works
possibly - i think it might be good to give the graal folks some feedback though. two points i see as problematic for people to test out native-image on windows are: 1) the environment set up should be automatable (e.g. doable via appveyor and the like), 2) --static should be supported to eliminate dll-related issues, but also for deployment reasons.
sorry for the late chime in, the thing that I was mentioning to @borkdude was https://sourceforge.net/projects/nsis/
this could be a (last ditch) way to package the generated exe along with the DLLs from whatever build env it was produced in
AFAIK, the exe should pick up the DLLs if they are in the PWD too
so if we can install the exe and DLLs in to say C:\Program Files\clj-kondo
it should work i think
and NSIS is fully scriptable. can be packed onto appveyor
i remember using nsis -- i think there's also innosetup -- is the former preferred these days?
my exp is most with NSIS as it produced the most compatible setup.exe
and the fact that the scripting is quite scriptable
personally i dont see a fool proof way of getting this to work till we have a control of the kind of flags used by native-image
when calling the linker i guess
apart from using a bundling installer. the good old wizard based windows setups 🙏:skin-tone-3:
also i may be woefully wrong as my windows muscles are getting rusty 😕
btw, i came across this: https://docs.microsoft.com/en-us/windows/desktop/Dlls/dynamic-link-library-search-order may be there is some relevant info -- or there might be a better doc about dll search order
if the dlls are picked up from the same dir as the .exe, then it could also work with scoop which I think allows downloading dlls along with the main zip and places them in the same dir... not sure
so from @sogaiu’s link:
The search order is as follows:
- The directory from which the application loaded.
- The current directory.
so it may just work
most of my windows dev revolved around Qt, which had its fair weight of DLLs and it did work from the PWD of the exe
so if we can get clarity of which dlls those are, then I could try to add them to the scoop bucket for testing
so i guess one way could be: - install the 7.1 SDK - dump the dependency list - copy over the exe and the DLLs into a folder - pack it up
yeah, an MVP could be as simple as a zip of the stuff
and the trust on the fact that we arent supplying malicious DLLs. (this is my old paranoia typing)
> oh yes, just the entire thing as a .zip Essentially thats what NSIS is with a more human, interacting and democratic way of installing
@borkdude there are other options to check for dep DLLs: https://www.raymond.cc/blog/check-what-dll-or-ocx-dependency-files-is-needed-for-a-software/
a lot of them show the path to the DLL too. can try zipping them?
yeah theres a lot of noise
this looks kinda nicer: https://ntcore.com/?page_id=388
i have a feeling we are gonna solve https://github.com/oracle/graal/issues/1407 before they do
LOL. I don't have a Windows virtual machine right now because my work laptop had to go in for repair, and the current one has a limited disk. Isn't there something non-GUI that's available on appveyor maybe? I could give the GUI approach a stab, but not soonish
I can try that too but in a similar situation of limited disk space 😕
@borkdude apparently there is: https://lucasg.github.io/2018/04/29/Dependencies-command-line/
pretty active project! TIL
looks wayyy less noisy too
yeah last time it just took 19 failed builds. have about 5000 more to go!
yeah its time/build thats billed AFAIK
billed in the sense there is a limit
for OSS
like CircleCI's 1000 build minutes per month
@borkdude I'd suggest using the Appveyor UI to mess around as its way less roundtrips and time waste as making new commits
ok, I might try it, but currently I'm focusing on a different clj-kondo issue, namely inlining some deps and this breaks the tests 😕
right
Berlin's approaching 40 here. 🤯
That too when its the most vegan place i know 😛
added the suggestion to https://github.com/borkdude/clj-kondo/issues/276 so I won't forget
@taylor I have inlined some deps (rewrite-clj) using mranderson. now I see that those files are also compiled by clj-native-image. somehow the protocol defined in one of those inlined files is unbound now:
Caused by: java.lang.IllegalArgumentException: Unbound: #'clj-kondo.impl.rewrite-clj.v0v6v1.rewrite-clj.node.protocols/Node is not a protocol
any ideas?
the full output is here: https://circleci.com/gh/borkdude/clj-kondo/4074Hmm not familiar with the inlining thing but maybe something is being compiled out of order?
it seems compiling out of order is the problem. I've got it working with an uberjar now it seems, so I might revert to that, least friction
in case you're curious, this is the branch: https://github.com/borkdude/clj-kondo/tree/fork-rewrite-clj
I think (in-ns 'clj-kondo.impl.rewrite-clj.v0v6v1.rewrite-clj.parser.core)
in that file might be tripping it up because I’m using clojure.tools.namespace.find but not sure yet
find-namespaces-in-dir
returns different results if I comment that out, though it still doesn’t resolve the issue
got past that particular error by just not using defprotocol+
but then I reach a different one about cognitect.transit$reader
being unreachable, so I’m assuming there’s something pretty fundamentally broken about how this is getting AOT’d in clj.native-image :thinking_face:
btw, this is interesting too: https://clojurians.slack.com/archives/C03S1KBA2/p1561487749183700
regarding search order, i also found this: https://github.com/numpy/numpy/wiki/windows-dll-notes -- it has some reflections / observations regarding the previous link, plus other info.
Has any one ever tried compiling a binary for alpine linux? using graalvm and native image?
I've just seen an option for --static is that how to do it?
@sfyire here's how I did it without static: https://github.com/borkdude/clj-kondo/blob/master/Dockerfile#L21