This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-02-27
Channels
- # aleph (7)
- # beginners (80)
- # boot (1)
- # cider (3)
- # cljs-dev (277)
- # cljsjs (52)
- # cljsrn (1)
- # clojure (69)
- # clojure-gamedev (4)
- # clojure-italy (1)
- # clojure-losangeles (2)
- # clojure-russia (89)
- # clojure-spec (92)
- # clojure-uk (196)
- # clojured (1)
- # clojurescript (70)
- # cursive (5)
- # data-science (1)
- # datascript (84)
- # datomic (9)
- # defnpodcast (12)
- # docker (1)
- # emacs (4)
- # events (1)
- # fulcro (112)
- # graphql (1)
- # jobs (1)
- # lumo (1)
- # nrepl (21)
- # off-topic (2)
- # onyx (3)
- # protorepl (10)
- # re-frame (23)
- # reagent (66)
- # reitit (2)
- # rum (13)
- # shadow-cljs (144)
- # spacemacs (14)
- # sql (4)
- # unrepl (29)
- # vim (16)
I see shadow-cljs uses thheller/cljs-dev Docker image for running CI tests. Is it recommended to use this image for our CI tests as well?
It appears not? 😬 https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/cljs/npm/cli.cljs#L528
Just kidding 🙂 https://github.com/thheller/shadow-cljs/blob/master/packages/shadow-cljs-jar/src/shadow/cljs/npm/deps.clj#L36
Is there a reason the wait time is hardcoded for 60s? https://github.com/thheller/shadow-cljs/blob/d59524d3ef9dfd41a2b6e5ec58fa69a1ed5002e2/src/main/shadow/build/compiler.clj#L683 Seems like this should be configurable. My CLJS is failing to compile from hitting that limit (it's due to using some generators at macro time to generate some code).
@thheller https://github.com/DogLooksGood/shadow-cljs-with-react-native-note can't wait to have a :react-native
target.
@kenny I can make that timeout configurable. the "slowest" compile for a single ns I saw so far was ~15sec. what are you doing that it takes 60s+?
@kenny I changed the timeout condition in 2.1.28
which might just fix your problem. if not you can set :build-options {:par-timeout some-high-number-in-ms}
private maven repos might work. I'm not sure. Using the same lib as lein
but not sure if I'm maybe not setting something I would need to set. lein or deps.edn also work though.
I really appreciate the lein integration - if only because it helps Cursive users 🙂 but also the ecosystem of plugins is quite nice
@thheller Thanks! I set it to 2 mins and I'm still having the issue. It appears to only happen on the CI though, and it's not clear why it is happening. The namespaces that it is printing out should not take a long time to compile. For example, DataScript is failing on the CI:
:shadow.build.classpath/resource "datascript/pull_api.cljc"] #error {
:cause "aborted par-compile, [:shadow.build.classpath/resource \"datascript/pull_api.cljc\"] still waiting for #{datascript.pull-parser}"
:data {:aborted [:shadow.build.classpath/resource "datascript/pull_api.cljc"], :pending #{datascript.pull-parser}}
:via
[{:type clojure.lang.ExceptionInfo
:message "aborted par-compile, [:shadow.build.classpath/resource \"datascript/pull_api.cljc\"] still waiting for #{datascript.pull-parser}"
:data {:aborted [:shadow.build.classpath/resource "datascript/pull_api.cljc"], :pending #{datascript.pull-parser}}
:at [clojure.core$ex_info invokeStatic "core.clj" 4739]}]
Is there a way to get more information printed out? I'd bet something else is going on here.I added the -v
flag and I get more information. I don't understand why some namespaces are taking a long time to compile. For example
Cache write: datascript/impl/entity.cljc
<- Compile CLJS: datascript/parser.cljc (78929 ms)
Could this be happening as a result of another namespace taking a long time to compile? Or are these all compiled independently and DataScript is just taking a long time for some reason?
Turns out the DataScript thing was a red herring. I removed the code created by generators and it worked (getting a No binary for ChromeHeadless browser on your platform.
error not but that's something else I'm guessing). Confusing debugging process though. Not sure why it was telling me DataScript was taking a long time to compile.
No binary for ChromeHeadless browser on your platform.
is no error from shadow-cljs. probably karma?
Right. Trying to figure that one out now. The vanilla example in the docs does not work.
What do you use on the CI? I only suggested it because it was suggested here: https://github.com/karma-runner/karma-chrome-launcher
Simply adding the steps they have there does not work. It appears other binaries are missing to start the headless chrome.
this? https://hackernoon.com/running-karma-tests-with-headless-chrome-inside-docker-ae4aceb06ed3
as for the datascript issue I can't find anything that would cause this slowdown. it might just be the CI acting up. I hear they have issues with rogue crypto miners running on their VMs
It was a red herring. Turns out it was a completely different ns causing the issue but DataScript was the one printed out for some reason.
if namespaces take too long to compile other namespaces "behind" it may get delayed a lot if there aren't enough threads
It would've been helpful if the problem ns was the one printed out instead of a tangential one.
But yes that was the case, I guess. I have code that runs at macro time which generates some static data for the runtime environment (using test.check generators). It does not take a long time to run on my machine but it does on the CI.
Sure but there are so many messages printed. Would be very time consuming to go through.
Above that I have about 100 lines of
Feb 27, 2018 3:03:01 AM shadow.build.compiler invoke
WARNING: [:shadow.build.classpath/resource "cljs/test.cljs"] waiting for #{cljs.pprint}
with the namespaces in the log messages varying.There's a lot of noise to sort through. Maybe printing a summary of all the namespaces still compiling and how long they are taking?
@lilactown you use piggieback with shadow-cljs? I deleted my original question, because I still wasn't sure about how to formulate it. As I'm trying to connect to node repl, I'm trying to find out if that's an edge case, but I'm realizing it's actually the same procedure.
what I started to do, without success, was to make a huge emacs function for starting shadow-cljs node repl, start cider connect, paste the code in, and fork the shadow-cljs process to the cider-repl buffer. To get the print callbacks in 1 buffer. But I didn't get success, as it's no way to know when the subprocess has finished starting shadow-cljs node-repl for cider to start the connection.
I feel somehow this all could be simpler 🙂 so only thinking out loud, nothing is broken
well, best would be if cider could start shadow-cljs, just open a ticket and ask if the maintainers have the time in the future to implement shadow-cljs connection like they do with lein and boot....
I think @hlolli wants an Emacs fn that starts up a build and connects to it with CIDER
similar to how you can run cider-jack-in
and it uses lein to start a headless REPL and connect to it
bingo, so no problem really, it's a minor distraction that I need a terminal window next to my emacs, and the connection it a 4 step. cider-connect -> select root folder -> go to repl buffer -> go to end of buffer -> type in (shadow.cljs.devtools.api/nrepl-select :node-repl) -> press enter twice
and on top of that, all print/console.log is in the terminal buffer, but I get the return value tough to my emacs.
yes that's right, browser repl connection doesn't do that, don't think it makes sense there, but this is only an extra feature, I know it's possible tough to implement in emacs with subprocesses
hehe, I feel like @lilactown understands me. And could clairify if I'm explaining this inaccurately? But this is nothing that requires change from shadow-cljs itself, as far as I see.
if you look at cider and clojurescript in leiningen, you see that when you close emacs with running cider, it will prompt the user for closeing 3 processes, cider process/cljs process and nrepl process. So I can imagine cider startup function, where shadow-cljs would be started as a process within emacs.
to proof, I did hack something here which almost works https://gist.github.com/hlolli/d8a28c88ee90c5a57d15299f21528030
ok .. let me first take out all emacs stuff you are doing an deconstruct the commands you are running
ah nice, ok! Glad I posted this work in progress then, the explicit port is there because it's a work in progress, guess I was debugging/ruleing out. Did this month ago..
and also, I'm pipeing the stdout to the message buffer. I'd want to change that to the cider-repl buffer. Essentially printing all output to 1 buffer.
ok, I will try to make something better out of this, I had commented line 60 to bottom, because I was giving up starting shadow-cljs as a subprocess, due to exacly the problem that, I didn't know when shadow-cljs repl was ready, but if the behaviour is like you say, the file nrepl.port exists when the port is open and available, I have then a solution. I recommend not reading this until I've made changes 🙂
this currently doesn't work since I'm unsure how I want to handle external processes
(shadow.cljs.devtools.api/node-repl)
does since simply by reading until eof on stdin
Wish I knew the nrepl protocol mechanics better, but if you're keeping the nrepl connection alive by simply look for eof. Don't know what to say, seems magic to me that nrepl can know which stdout to take as return value and send across the wire and which text to ignore (ex. the print stuff).
uhm thats what I meant .. nrepl works differently. thats why the wait-for-eof method doesn't work
nrepl in a nutshell just sends messages in the form of {:op "do-a-thing" :some "thing"}
and gets messages {:out "this is printed"}
or {:value "foo"}