This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-11-01
Channels
- # announcements (7)
- # babashka (41)
- # beginners (117)
- # cider (3)
- # clj-kondo (145)
- # cljdoc (25)
- # cljs-dev (19)
- # clojure (197)
- # clojure-dev (14)
- # clojure-europe (4)
- # clojure-italy (3)
- # clojure-nl (2)
- # clojure-spec (11)
- # clojure-uk (21)
- # clojuredesign-podcast (5)
- # clojurescript (29)
- # code-reviews (4)
- # cursive (87)
- # data-science (11)
- # datomic (29)
- # duct (2)
- # emacs (10)
- # graalvm (1)
- # lumo (13)
- # malli (2)
- # nrepl (5)
- # off-topic (25)
- # onyx (1)
- # pathom (6)
- # reagent (20)
- # reitit (4)
- # rewrite-clj (7)
- # shadow-cljs (114)
- # spacemacs (16)
I would appreciate it if someone could go through the steps in this issue to verify if it's clear to reproduce: https://github.com/oracle/graal/issues/1804
my lein classpath execution fails at:
Could not transfer artifact binaryage:env-config:pom:0.2.0 from/to clojars ( ): Read timed out
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
extra double quote at end of:
./clj-kondo --lint "$(lein classpath)" > /tmp/log.txt"
?i did not observe an exception for:
./clj-kondo --lint "$(lein classpath)" > /tmp/log.txt
not that it is terribly important but https://github.com/oracle/graal/files/3799384/clj-kondo-2019.10.26-standalone.jar.zip contains __MACOSX
dir
> unzip -l ~/Downloads/clj-kondo-2019.10.26-standalone.jar.zip
Archive: /Users/lee/Downloads/clj-kondo-2019.10.26-standalone.jar.zip
Length Date Time Name
--------- ---------- ----- ----
8704607 11-01-2019 15:35 clj-kondo-2019.10.26-standalone.jar
0 11-01-2019 15:45 __MACOSX/
985 11-01-2019 15:35 __MACOSX/._clj-kondo-2019.10.26-standalone.jar
--------- -------
8705592 3 files
Also kind of obvious, but you might want to replace "Build the following .jar" to "Unzip then build the following .jar"
as someone walking through the issue it would be helpful to have the reflection.json come before the command because the native-image command requires it.
@sogaiu could you try this also with the binary from github, and/or on a macos machine?
@sogaiu one you can download here: https://github.com/borkdude/clj-kondo/releases
in your code snippet, comment # this directory will be used to create a RandomAccessFile in
might not be finished or has extra word in
?
delete trailing double quote in code snippet?: ./clj-kondo --lint "$(lein classpath)" > /tmp/log.txt"
And could you also verify that the problem does not occur when linting with the jar?
am on that last step now... should mention that java -jar ...
command should executed from /tmp/project
as it needs project.clj
and maybe.. redirect output from java -jar
to different log file as to not overwrite previous results.
the results are not important and probably the log file from the first step is completely empty
the only important thing is the exception. the error code is probably on me for not catching the exception from the main function, so not for them to sort out
oh I get what you mean with the exit code: they should ignore it even it it's not zero
looks like it eh? apparently is build number: https://support.apple.com/en-is/HT201260
maybe it has to do with too many file handles or something, don't know, low level stuff!
hmmm just noticed macOS 10.15.1 is available, I doubt that would make a difference for your issue...
but they are compiled more or less the same, using lein uberjar + the script/compile script
ya, super helpful. I'm going the tools deps route, and figured out what I need for that today.
but I found nothing that came close to this using deps.edn. sogaiu mentioned an interesting tool recently which might do it
ah it was this one: https://github.com/EwenG/badigeon
cambada has support for native: https://github.com/luchiniatwork/cambada
yeah well there's also clj.native-image, but I ran into limitations. basically all tools which wrap the native-image command will be limiting, in my experience an AOT-ed uberjar + shell script works better
I am using currently using depstar to build my jar and am following instructions for aot compiling from here: https://clojure.org/reference/compilation#_gen_class_examples
one problem I ran into with clj.native-image was that it tried to compile my namespaces, but somehow missed a few or didn't do it in the right order
I also ran into problems with it on Windows. Eventually I made a script/compile and script/compile.bat
did you consider compiling your tests with graal (along with your src) and running them that way? I think that's the approach typically taken for plain old cljs testing.
might not make sense for your command line tools... but for a pure library without a command line.. maybe?
@lee No, I run the tests from the JVM, but only change the way how to call the thing under test. When doing native I call the binary using clojure.shell/sh
I haven't tried compiling clojure.test to GraalVM but if that works, then that's a fine approach I'd say.
I haven't got any GraalVM tests in edamame, but since I'm using that extensively in sci and babashka, I test it that way
what's weird is that the size of the classpath makes a difference for getting a FileNotFoundException
I suppose, if you were curious, you could do some println debugging prior to opening the file... check what returns you get for .exists
and .canRead
, maybe try .getAbsolutePath
.
that's a good suggestions. this API should work even when the file doesn't exist yet, but it's good to add that debugging.
@U04V15CAJ, I’m actually looking at the code https://github.com/borkdude/clj-kondo/blob/f6ab3624defa04708887bc9687805b4303f5dbb4/src/clj_kondo/impl/cache.clj#L58
don’t you need a .close
somewhere for that RandomAccessFile?
Maybe that would be good. So then this part would be wrapped in with-open
:
https://github.com/borkdude/clj-kondo/blob/f6ab3624defa04708887bc9687805b4303f5dbb4/src/clj_kondo/impl/cache.clj#L65
yeah, just a guess but problem occurs when classpath is long, so maybe the misleading file not found exception due to a resource leak blowing some os limit?
so .exists returning either true or false doesn't say that much, since it "should just work TM" in both cases
for a bit of extra confidence, i went through the issue steps on an ubuntu vm (though with the binary downloaded from releases) -- no exception in the vm either
ah -- it's probably totally unrelated, but i keep hearing on #shadow-cljs about jvm issues on macos
what could be tried is not only retrying the locking of the file, but also the creation of the file:
(defmacro with-cache
"Tries to lock cache in the scope of `body`. Retries `max-retries`
times while sleeping 250ms in between. If not succeeded after
retries, throws `Exception`."
[cache-dir max-retries & body]
`(let [lock-file# (io/file ~cache-dir "lock")
_# (io/make-parents lock-file#)
raf# (RandomAccessFile. lock-file# "rw")
channel# (.getChannel raf#)]
(loop [retry# 0]
(if-let [lock#
(try (.tryLock channel#)
(catch java.nio.channels.OverlappingFileLockException _#
nil))]
(try
~@body
(finally (.release ^java.nio.channels.FileLock lock#)))
(do
(Thread/sleep 250)
(if (= retry# ~max-retries)
(throw (Exception.
(str "clj-kondo cache is locked by other process")))
(recur (inc retry#))))))))