This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-01-19
Channels
- # aws-lambda (1)
- # babashka (3)
- # beginners (97)
- # cider (20)
- # clojure (9)
- # clojure-spec (8)
- # clojure-uk (9)
- # clojurescript (25)
- # core-async (2)
- # cryogen (7)
- # cursive (4)
- # fulcro (9)
- # graalvm (14)
- # graphql (8)
- # luminus (2)
- # off-topic (18)
- # pathom (3)
- # re-frame (1)
- # reagent (4)
- # shadow-cljs (36)
- # spacemacs (1)
- # vscode (2)
It seems there's an issue in FileWatcher
. If a new directory is created with a bunch of files and then removed shortly thereafter, there can be a java.nio.file.NoSuchFileException
:
Exception in thread "async-thread-macro-1" java.nio.file.NoSuchFileException: /%python-subdir-within-project%/__pycache__/flow_control.cpython-37.pyc.140019814933808
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.base/java.nio.file.Files.readAttributes(Files.java:1763)
at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373)
at java.base/java.nio.file.Files.walkFileTree(Files.java:2760)
at shadow.util.FileWatcher.registerAll(FileWatcher.java:41)
at shadow.util.FileWatcher.pollForChanges(FileWatcher.java:114)
at shadow.util.FileWatcher.pollForChanges(FileWatcher.java:76)
at jdk.internal.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:167)
at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:438)
at shadow.cljs.devtools.server.fs_watch_jvm$poll_changes.invokeStatic(fs_watch_jvm.clj:17)
at shadow.cljs.devtools.server.fs_watch_jvm$poll_changes.invoke(fs_watch_jvm.clj:16)
at clojure.core$map$fn__5866.invoke(core.clj:2753)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:51)
at clojure.lang.RT.seq(RT.java:535)
at clojure.core$seq__5402.invokeStatic(core.clj:137)
at clojure.core$apply.invokeStatic(core.clj:660)
at clojure.core$mapcat.invokeStatic(core.clj:2783)
at clojure.core$mapcat.doInvoke(core.clj:2783)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at shadow.cljs.devtools.server.fs_watch_jvm$watch_loop.invokeStatic(fs_watch_jvm.clj:46)
at shadow.cljs.devtools.server.fs_watch_jvm$watch_loop.invoke(fs_watch_jvm.clj:33)
at shadow.cljs.devtools.server.fs_watch_jvm$start$fn__73779.invoke(fs_watch_jvm.clj:77)
at clojure.core.async$thread_call$fn__34862.invoke(async.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Just to give some context - my project is a mix of Clojure, ClojureScript, and Python. I work with the Python code just by calling shell/sh
.
So it seems like if a file is deleted when Files.walkFileTree
is doing its job, there's a chance the whole walk will fail because of it.
I guess somewhat of a fix would be to keep calling Files.walkFileTree
until there's no NoSuchFileException
.
thats not currently configurable but I could add an option to let you exclude __pycache__
?
The Python sources are not tmp though. But Python creates __pycache__
right next to the sources.
I could of course move Python sources out of the classpath altogether. But having them there gives a nice ability to not have to deal with arbitrary paths when some files are shared between CLJ and Python code.
Not sure that code above would work though.
E.g. I have src
on the classpath. Somewhere in there are both CLJ and Python sources. __pycache__
will be created there as well - not at the top level.
Also, just excluding something doesn't solve the potential issue itself since you can reproduce it with arbitrary code. But one could argue that that's a problem with Files.walkFileTree
. Not sure.
I'd advise keeping it off the classpath though ... I doubt python will be able to read them from there if you move to a uberjar
The check at shadow.build.classpath/get-classpath-entries
? It would leave src
in there. And then a FileWatcher
would be created for src
. And FileWatcher
doesn't exclude anything.
If I move to an uberjar - how would I even use Python sources stored in there? I don't think I can use an uberjar at all at this point.
There's a plan to get rid of Python, which is nice. But the current plan is to replace it with Swift, which will make some things even worse. :)
Hi, I’m new to Clojure world here. I’m using ClJS to create and deploy firebase functions. I’m using shadow-cljs to perform the compilation to the JS lib. I’m not sure where to ask this. But is it possible to jump to js library implementation during the development phase ? Say for eg: I am importing an express js library and want to jump to implementation.
Is there a way to write a hook for shadowjs to "run a clojure function" if a certain file changes in my repo ?
I’ve got a cljs-dependency which references a transitive dependency via js/ prefix (js/snabbdom), but even though I’ve added snabbdom to npm deps it fails
I can reference snabbdom directly though, so I know it has been properly added to shadow-cljs
@odinodin if the library is just using js/snabbdom
without any require for it you can create it by exporting that global variable in a namespace that included before that lib
just need to make sure it is included before the lib tries to access js/snabbdom
Q: I’ve started running nodejs tests using the :autorun feature. this works great. when tests fail, the line numbers are for the .js artifact i.e. source maps are not used (by default). Is it possible to configure source maps in this mode so the line numbers are for the cljs source?
ah, but you are looking at the autorunner... hmm - dunno there
thanks. I’ll try that out. worst case I’ll stop using the autorunner and use node watch to run the tests instead