Fork me on GitHub
#shadow-cljs
<
2017-11-07
>
thheller09:11:25

CPU usage goes up

thheller09:11:59

but that delay typically doesnt matter much

thheller09:11:27

this matters more and thats the highest it will go

thheller09:11:47

it is 2 sec on OSX

thheller10:11:02

do you think the delay is too much?

denik13:11:45

The hot loading delay is one of my biggest pains developing web apps in cljs. Is there a on-save hook you can expose? Editor integrations could hook into that. It should also cause a massive speed increase by only reevaluating the forms that changed (as we would through repl).

denik14:11:20

My current workaround is a fn like

(defn with-reload [& _]
  (my-app.frontend.core/mount-root))
that I wrap components I’m editing with and simply eval the top level form:
(with-reload
 (defn my-comp []
   [:div "abc"]))
This hack that saves me a lot of time during dev. Almost browser dev-tools-like responsiveness.

thheller14:11:34

I could easily expose a hook that the editor could call.

thheller14:11:50

I did some related work with the Language Server Protocol a while back

thheller14:11:21

the proof of concept did exactly that. on file save inform the compiler and have the compiler return compile information so the editor can show warnings/errors

thheller14:11:39

LSP is a lot of work, if you just want a file-save hook I can probably add something http-ish where you post some data about which files were saved?

thheller14:11:57

partial compile is tricky, not sure how to do that

denik15:11:52

Thanks theller! Looks like the on-save hook is low-hanging fruit and editor agnostic. Please add it!

thheller15:11:57

what do you need? I’m not aware of any editor that already has a hook for that

thheller15:11:50

yes but in what shape 🙂

thheller15:11:59

http endpoint you can curl? do you have EDN support or would JSON be simpler? or just one line per file saved?

denik15:11:29

hmm, don’t know much about the workings of editor extensions

thheller15:11:53

same here 😛

denik15:11:59

emacs and cursive are probably the two major editors with vim and vscode in the back

thheller15:11:14

In cursive you can connect to the shadow nrepl and send (shadow.cljs.devtools.api/compile :my-build) to the REPL on keypress

thheller15:11:28

maybe the closest thing already available

thheller15:11:06

but triggers a full build, watch is faster since it maintains some state

denik15:11:20

I agree. This is more about removing latency for UI code. Chrome devtools is instant. The editor knows exactly when files are edited, and most editors know which files were edited. This allows us to get rid of the whole file-watching process, which should enable major improvements in reload speed.

denik15:11:50

Regarding compile per form, that’s essentially like top-level-form repl eval. Editors know the top level forms of files. So between saves, we could track all edited forms and compile them one by one. I think this will enable immense speed improvements.

thheller15:11:48

well you can just use the REPL to eval the form you changed

denik15:11:17

that’s what I’m doing with the snippet above. As you can see it takes some wrapping, because we also want to call the reload-hook for the app to re-render the root ui component

denik15:11:23

It’s puzzling for me that compile times of entire projects are used as a significant measure when the major road blockers for DX lie in the waits between incremental compilation steps. It’s a bit like starting a clojure repl. It’s slow but the idea is that it will run forever. Once it’s up, it’s blazingly fast. In the long run, that’s what matters.

thheller16:11:01

I think its a tooling issue. You can bind “reload-my-app” to a button in Cursive, which just sends the proper function call to the REPL

thheller16:11:20

I have that for all my clojure setups, in CLJS live-reload+watch is usually good enough for me.

thheller16:11:27

but yeah clojure I do just about everything at the REPL

denik17:11:00

@U050RLRRQ and I discussed this. In his terms what’s needed is a) form-level change tracking and b) maintaining a dep graph per var. For forms without vars it could have get assigned an unique identifier in a). Instead of save, a keyboard shortcut evals all edited forms. Lastly, a after-eval hook reloads the app. Yes, if the hooks are available, both are tooling features. cc @U0567Q30W

mhuebert17:11:43

yeah this seems pretty involved

mhuebert17:11:15

we’re looking at doing it in maria, that’s a much more controlled environment

thheller17:11:45

I usually just load-file when I make changes in a file. clojure again though, it might be slower in CLJS.

denik19:11:22

@thheller yes it’s much slower. I’m glad we’re starting this conversation. Sometimes the hardcore tool developers are not the hardcore users. I’m glad we’re having the opportunity to share perspectives.

thheller20:11:51

I’m a hardcore user that just got fed up with the tools 😉

denik20:11:14

🙂 then you must know the pain of 8+ secs between incremental compilations

thheller21:11:22

not really, its very rarely this slow. shadow-cljs handles this slightly differently and only recompiles files that changed and those that directly required it.

thheller21:11:30

when I’m working on a framework level thing thats used in a whole lot of places I usually isolate the thing I’m working on because the usual “app” would just be annoying at that point

thheller21:11:43

so my live-reloads are usually less than 1sec

thheller10:11:17

I thought about replacing it with something faster before

thheller10:11:47

unfortunately everything I looked at consumed a whole lot more CPU (which translates to battery)

thheller10:11:22

figwheel uses https://github.com/wkf/hawk, no idea if that is faster?

denik13:11:37

figwheel is similarly slow. With large projects I had waits of 8+ seconds. Possibly because the cljs compiler has :recompile-dependents true by default, but it could also just be recompiling all the project-owned namespaces required in the files that changed. For UI development, this is very painful.

thheller10:11:37

its only slow OSX, windows/linux are pretty fast

thheller10:11:55

I think hawk fixes it for OSX

mhuebert12:11:32

not a huge deal but am curious how it would feel to be faster. currently working on a thing where a typical incremental compile is 0.4-0.5s, changes appear to show up on average 1.5s

thheller12:11:03

yeah I also sometimes feel like it could be faster

thheller12:11:32

looking at the browser and nothing happens, switching back only to realize I wasn’t patient enough

thheller12:11:29

that surprised me when testing on windows the other day. it felt pretty much instant there.

mitchelkuijpers13:11:46

How do people here work with libraries like for example om which need cljs.react and when you use npm-deps which needs react?

thheller15:11:09

using hawk is pretty straightforward and actually a whole lot faster (on OSX, did’t test others yet)

thheller15:11:27

but it doesn’t seem to detect deleted files

thheller15:11:26

not super critical though. I might just add a config option to switch between the two mechanisms

mitchelkuijpers15:11:29

Ah ok thnx let me see if I can fix this

mhuebert15:11:03

that sounds great

mitchelkuijpers15:11:42

That works like a charm thnx @thheller

thheller15:11:45

just pushed [email protected] which uses hawk to watch files

thheller15:11:53

it definitely reacts way faster on OSX

thheller15:11:22

added a 250ms delay … could reduce that even further

thheller15:11:10

but I think this is good enough while providing some safety that batch saves do not trigger 2 separate compiles

thheller15:11:56

I also made a small compiler improvement earlier today which should make it about 20% faster overall

thheller15:11:04

but that needs more benchmarks to confirm

mitchelkuijpers15:11:58

Anyone else ran into: The required namespace "com.rpl.specter.navs" is not available, it was required by "com/rpl/specter.cljc". I found this issue https://github.com/nathanmarz/specter/issues/72 but i don't run bootstrapped clojurescript

thheller15:11:57

I have a suspicion

mitchelkuijpers15:11:58

It only happens when I do compile instead of watch

thheller15:11:49

nah it doesn’t work because of the ns form

thheller15:11:08

I have my own ns parser in shadow-cljs and it is a bit too strict

thheller15:11:03

this ends up as (:use) for CLJS, which my parser thinks is invalid

mitchelkuijpers15:11:56

Ah you have your own parser for the nice error messages?

thheller15:11:07

I have my own because the default is pretty hard to work with, less painful to write a custom one.

mitchelkuijpers15:11:50

Damn you are fast

thheller15:11:27

change + to *, easy fix 😉

mitchelkuijpers16:11:17

The only thing I have now is that I have a lot of cljs.pprint warnings is that normal or am I screwing up?

thheller16:11:32

what kind of warnings?

mitchelkuijpers16:11:19

------ WARNING #40 -------------------------------------------------------------                                                          
 File: cljs/pprint.cljs:3144:3                                                                                                            
--------------------------------------------------------------------------------
3140 | ;;; This is the equivalent of (formatter-out "~:<~1I~@{~w~^ ~_~}~:>"), but is
3141 | ;;; easier on the stack.                                                             
3142 |                                                       
3143 | (defn- pprint-simple-code-list [alis]                                                
3144 |   (pprint-logical-block :prefix "(" :suffix ")"                                  
---------^----------------------------------------------------------------------
 js/cljs.pprint.*current-level* not declared ^:dynamic                                       
--------------------------------------------------------------------------------
3145 |     (pprint-indent :block 1)                                                   
3146 |     (print-length-loop [alis (seq alis)]                                       
3147 |       (when alis                                                             
3148 |         (write-out (first alis))                                                     
3149 |         (when (next alis)                                              
--------------------------------------------------------------------------------            
                                                                                        
------ WARNING #41 -------------------------------------------------------------  
 File: cljs/pprint.cljs:3144:3                                                          
--------------------------------------------------------------------------------
3140 | ;;; This is the equivalent of (formatter-out "~:<~1I~@{~w~^ ~_~}~:>"), but is
3141 | ;;; easier on the stack.                                                             
3142 |                                                                  
3143 | (defn- pprint-simple-code-list [alis]                                                
3144 |   (pprint-logical-block :prefix "(" :suffix ")"                                  
---------^----------------------------------------------------------------------
 js/cljs.pprint.*current-length* not declared ^:dynamic                                      
--------------------------------------------------------------------------------
3145 |     (pprint-indent :block 1)                                                   
3146 |     (print-length-loop [alis (seq alis)]                                       
3147 |       (when alis                                                 
3148 |         (write-out (first alis))                                                     
3149 |         (when (next alis)                                              
--------------------------------------------------------------------------------            
                                                                                  

mitchelkuijpers16:11:30

Not sure if it's a derp from me

thheller16:11:05

hmm never seen those before

thheller16:11:22

I don’t get them either

mitchelkuijpers16:11:58

I run from withing leiningen I'll investigate some more

mitchelkuijpers16:11:09

Maybe I have a an old clojurescript or something weird

mitchelkuijpers16:11:02

Should I not add clojurescript dep myself and rely on shadow-cljs?

thheller16:11:30

yes, and try adding shadow-cljs first so its deps get picked up first

thheller16:11:39

better yet, don’t use it with lein 😉

mitchelkuijpers16:11:01

Hmm does that work if you have macros serverside?

thheller16:11:16

try lein deps :tree and see what it complains about

thheller16:11:38

yes, shadow-cljs uses the JVM compiler.

mitchelkuijpers16:11:43

Yeah I already fixed a lot of issues there, but maybe not running from lein will be nicer

thheller16:11:43

not a clue what could be causing those warnings

mitchelkuijpers16:11:02

Yeah I saw a conflict about the shaded closure compiler

thheller16:11:27

thats not involved in here

thheller16:11:33

at least I can’t think of a reason why it would be. clojurescript version will have an impact but since everything else seems to work …

thheller16:11:11

do you have a REPL open? try ( "cljs/pprint.cljs") and see if thats from 946

thheller16:11:33

also check ( "cljs/pprint.cljc") just in case

mitchelkuijpers16:11:02

I saw I had two profiles with different shadow-cljs deps, maybe that is the issue

mitchelkuijpers16:11:57

I think I am going to try without leiningen, that will probably save me lots of time

thheller16:11:35

is lein deps :tree this bad? shouldn’t be too bad

thheller16:11:56

its a bit painful to use shadow-cljs for CLJS and lein for CLJ when using Cursive

thheller16:11:09

since cursive only reads the project.clj

thheller16:11:42

I use lein myself when developing shadow-cljs itself, so it definitely works fine

thheller16:11:50

just conflicts are bad 😛

mitchelkuijpers16:11:01

Hmm even without leiningen I get the same errors

thheller16:11:30

what are you doing with cljs.pprint?

mitchelkuijpers16:11:55

I thinkg nothing:

mitchelkuijpers16:11:02

I get this errors while it is trying to build:

thheller16:11:06

(:require [cljs.pprint :refer (pprint)]) (pprint {:x [1 2 3]}) definitely works

mitchelkuijpers16:11:26

Nov 07, 2017 5:39:45 PM clojure.tools.logging$eval347$fn__350 invoke                                                                                                                                                                                                                                                                                                       
WARNING: failed to get source excerpt for jar:file:/home/mitchel/.m2/repository/org/clojure/clojurescript/1.9.946/clojurescript-1.9.946.jar!/cljs/pprint.cljs at {:warning :dynamic, :line nil, :column nil, :msg "js/cljs.pprint.*current-length* not declared ^:dynamic", :extra {:ev {:name js/cljs.pprint.*current-length*, :tag js, :ret-tag js, :ns js}, :name js/clj
s.pprint.*current-length*}}                                                                                                                                                                                                                                                                                                                                                
java.lang.NullPointerException                                                                                                                                                                                                                                                                                                                                             
        

mitchelkuijpers16:11:23

And then all the warnings start popping up

thheller16:11:40

how are you calling pprint?

mitchelkuijpers16:11:23

[cljs.pprint :refer [pprint]]

mitchelkuijpers16:11:30

that is the require

thheller16:11:24

…. I think I found it

thheller16:11:39

hmm no wait

thheller16:11:07

/.m2/repository/org/clojure/clojurescript/1.9.946/clojurescript-1.9.946.jar!/cljs/pprint.cljs it is using the correct file

thheller16:11:13

and your require is correct

mitchelkuijpers16:11:19

I thind these very weird warnings:

Nov 07, 2017 5:46:55 PM clojure.tools.logging$eval347$fn__350 invoke                        
WARNING: failed to get source excerpt for jar:file:/home/mitchel/.m2/repository/org/clojure/clojurescript/1.9.946/clojurescript-1.9.946.jar!/cljs/pprint.cljs at {:warning :dynamic, :line nil, :column nil, :msg "js/cljs.pprint.*current-length* not declared ^:dynamic", :extra {:ev {:name js/cljs.pprint.*current-length*, :tag js, :ret-tag js, :ns js}, :name js/clj
s.pprint.*current-length*}}                                                     
java.lang.NullPointerException                                                               
        at clojure.lang.Numbers.ops(Numbers.java:1018)                          
        at clojure.lang.Numbers.minus(Numbers.java:137)                               
        at clojure.lang.Numbers.minus(Numbers.java:3717)                              
        at shadow.build.warnings$get_source_excerpts$make_source_excerpt__20342.invoke(warnings.clj:28)
        at shadow.build.warnings$get_source_excerpts$iter__20344__20348$fn__20349$fn__20350$fn__20352.invoke(warnings.clj:43)
        at shadow.build.warnings$get_source_excerpts$iter__20344__20348$fn__20349$fn__20350.invoke(warnings.clj:42)
        at shadow.build.warnings$get_source_excerpts$iter__20344__20348$fn__20349.invoke(warnings.clj:41)

thheller16:11:51

do you get any warnings on startup?

thheller16:11:59

just run shadow-cljs clj-repl nothing else

thheller16:11:07

does that print a whole bunch of warnings?

mitchelkuijpers16:11:33

I run it like this btw: node_modules/.bin/shadow-cljs clj-repl does that matter?

mitchelkuijpers16:11:35

I have no warnings

thheller16:11:50

no thats ok, try npx shadow-cljs clj-repl, same thing but shorter 😉

mitchelkuijpers16:11:26

Ah cool dit not know that

thheller16:11:54

so on the clj-repl (shadow/compile :build-id {:verbose true})

thheller16:11:53

issues like this one https://github.com/braintripping/re-view/issues/8 used to cause trouble in the past

thheller16:11:02

but I added warnings for that

thheller16:11:58

I’m really clueless

mitchelkuijpers16:11:24

I see no weird things in the output

thheller16:11:32

I don’t know how it would get into this :line nil, :column nil situation (which causes the NPE)

thheller16:11:13

( "cljs/core.cljc")

mitchelkuijpers16:11:15

HED] cljs/tools/reader/edn.cljs                                                       
[CACHED] cljs/reader.cljs                                                                 
[CACHED] cljs/tagged_literals.cljc                                              
[CACHED] cljs/analyzer.cljc                                                      
[CACHED] cljs/analyzer/api.cljc                                                             
[CACHED] om/next.cljc                                                                                                                                                                                                                                                                                                                                                      
-> Compile CLJS: cljs/pprint.cljs                                                           
<- Compile CLJS: cljs/pprint.cljs (2208 ms)                                             
[CACHED] fulcro/client/logging.cljc                                             
[CACHED] yahoo/intl_messageformat_with_locales.cljs                                          
[CACHED] fulcro/i18n.cljc                                                          
[CACHED] clojure/walk.cljs    

thheller16:11:17

what does that say?

mitchelkuijpers16:11:57

#object[java.net.URL 0x62f11ebb "jar:file:/home/mitchel/.m2/repository/org/clojure/clojurescript/1.9.946/clojurescript-1.9.946.jar!/cljs/core.cljc"]

thheller16:11:16

ok, so the cljs sources are definitely correct

mitchelkuijpers16:11:35

I am using om.next could that break something?

thheller16:11:56

it shouldn’t. let me try. which version?

mitchelkuijpers16:11:44

[org.omcljs/om "1.0.0-beta1" :exclusions [cljsjs/react-dom cljsjs/react org.clojure/clojurescript]]

thheller17:11:28

compiles fine, no warnings

thheller17:11:03

try nuking the target/shadow-cljs directory, maybe somehow the cache got corrupted

thheller17:11:13

I have no other explanation at this point

mitchelkuijpers17:11:31

No problem, I have to go will play some more with it

mitchelkuijpers17:11:37

Thank you for all the help though

mitchelkuijpers17:11:03

Nuking the cache does not help btw

mitchelkuijpers17:11:14

I'll try to reproduce with a smaller example

thheller17:11:46

thats really weird. I have no clue at this point.

Jon17:11:15

ran into a new problem. Code works in watch mode, but throws error after release build Uncaught Error: No protocol method ISwap.-swap! defined for type object: [object Object] in http://repo.tiye.me/mvc-works/coworkflow/

Jon17:11:26

probably problem in my code, but unfriendly error message...

thheller17:11:59

no warnings during compile?

Jon17:11:21

I think I've located the bug, not shadow-cljs' fault..

Jon17:11:55

however I had a bug in my build script and accidentally loaded "lib.js" twice, but without "main.js"

Jon17:11:12

confirmed..

Jon17:11:41

just can't relate this error message to my problem 😅

Jon17:11:31

BTW are we going to use https://github.com/shadow-cljs now?

thheller17:11:50

probably not for the project itself. I just needed a place to put the example.

thheller17:11:03

I want to write a proper guide for it but that isn’t going very well

Jon17:11:58

too many features to tell?

thheller17:11:56

not knowing what to tell. I have no idea which things need explaining, in which order, etc.

thheller17:11:46

I also don’t like blogging, I much rather write code. 😉

Jon17:11:33

for me shadow-cljs has covered nearly all my basic needs(except for dynamic import and CSS bundling) at currently. blogging won't help very much.

Jon17:11:54

maybe other will, though..

thheller17:11:18

gotta go, bbl

thheller17:11:37

dunno what you mean by “dynamic import”. feel free to explain, I can take a look later.

Jon17:11:15

I would like to learn your updates from Twitter(messages on slack also fine). I would reply to ask anyway when I want to learn more about the feature, as long as I know shadow-cljs supports it.

Jon17:11:42

https://webpack.js.org/guides/code-splitting/#dynamic-imports actually I barely use it since my cljs projects are not large enough, just nice-to-have, my app always grows.

thheller20:11:55

thats whats shadow.loader does, but you need to do it statically. I’m not convinced the dynamic approach is good. Might be doable though.