Fork me on GitHub
#shadow-cljs
<
2018-04-10
>
tony.kay00:04:52

Is there a setting similar to figwheel’s :figwheel-always?

thheller07:04:16

I'm sort of against adding this. The only reason I can think of why this would be necessary is "bad" macros which also means their output isn't cacheable at all. For cljs.test I added this hack for my testing stuff. https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/build/cljs_hacks.cljc#L783-L811 Basically all it does is call register-test to register the test var at RUNTIME. meaning I can completely bypass all macros to discover new tests at runtime.

thheller07:04:49

Given that you already have a spec macro in place would it not make sense to also do this?

thheller07:04:26

this is basically cljs.test rewritten to work completely without macros

thheller07:04:35

which means all the restrictions that come with the macros go away ... as does the need for :figwheel-always.

thheller07:04:52

I added some metadata support for configuring live-reload recently. https://shadow-cljs.github.io/docs/UsersGuide.html#_lifecycle_hooks

thheller07:04:28

I added :dev/once (or :figwheel-no-load) but didn't add :figwheel-always since I think there are better solutions

thheller07:04:38

I can add it however if you'd prefer that.

tony.kay13:04:38

Thanks, I appreciate the offer. I’ll look at your improved solution and see if I can make it work with both standard compiler and shadow in fulcro-spec. I agree that a test registry is a good idea. The original intention with fulcro-spec was to base off of the standard testing lib so that I wasn’t reinventing the wheel, just augmenting it with some nicer tires.

tony.kay13:04:51

thanks for the code links

tony.kay13:04:38

Yeah, I think you’ve got the right idea…interestingly, I already have a wrapper for “dynamically focusing tests” at runtime which already augments the deftest output to register with that system…so it really should not be a stretch.

tony.kay04:04:16

So, I imported your code, and ended up with some issues: 1. When you remove a test, there is no clear way to remove it from the registry. The macros actually work fine with that case. 2. The CI test version no longer will run using shadow…the new CI tests are fine with the regular cljs build. Not sure why on that one. I assume it has something to do with your test gathering code? I’m getting pretty tired, so not up to investigating further tonight. 3. Adding a new file is not picked up by hot reload code.

tony.kay04:04:19

On the plus side: new tests within a namespace are picked up, which was my main complaint. Still, the overall experience isn’t nearly as nice as I have it working with figwheel, though it is true that you have to manually list the nses there.

tony.kay04:04:47

Actually, I see why CI broke. You’re using your test hack inside of it. I didn’t understand what you meant…so, in order for this to work right, I think I’d have to change fulcro-spec to depend on shadow-cljs 😕

tony.kay04:04:05

I think we originally misunderstood each other. Fulcro spec emits deftest. My macro is just a thing that makes the reporting prettier for deftest. So, your “hacked” deftest should already be picked up by it without me having to import anything. So, the problems I’m seeing were already present in shadow, it seems.

tony.kay04:04:23

I also noticed that your “run-all-tests” has a different argument order than in cljs.test. I guess that should not matter, but now I’m realizing that perhaps I have to call your runner, which means I don’t end up with a general solution that works for figwheel and shadow.

tony.kay04:04:38

If you’re at all interested I made a YouTube video (5 mins) that shows you some of the problems I’m seeing, along with how to reproduce them. https://youtu.be/QHc8wTprKHo

tony.kay05:04:43

I'm sure the majority of the problems are the fact that that macro is side-effecting. I only see two ways of fixing that: one is to have an always re-load command for the compiler, and the other is to rewrite the testing library.

tony.kay05:04:24

And I don't have time right now for a rewrite:(

thheller06:04:19

what I was suggesting was changing the specification macro to no only emit deftest but also registering the spec at runtime with your fulcro-spec. so basically

(defmacro specification
  "Defines a specification which is translated into a what a deftest macro produces with report hooks for the
   description. Technically outputs a deftest with additional output reporting.
   When *load-tests* is false, the specification is ignored."
  [& args]
  (let [{:keys [name selectors body]} (fss/conform! ::specification args)
        test-name (-> (var-name-from-string name)
                      (with-meta (zipmap selectors (repeat true))))
        prefix (im/if-cljs &env "cljs.test" "clojure.test")]
    `(do (~(symbol prefix "deftest") ~test-name
           (im/when-selected-for ~(fss/conform! ::sel/test-selectors selectors)
             (im/with-reporting {:type :specification :string ~name
                                 :form-meta ~(select-keys (meta &form) [:line])}
               (im/try-report ~name
                 [email protected]))))
         (fulcro-spec.runtime/register-spec ~test-name {:any-addition-info-for-the-runtime ...}))))
But I do understand that a rewrite just for the sake of a rewrite is not ideal. I will add support for :figwheel-always.

thheller06:04:49

this macro infection of fulcro-spec is pretty much the same that cljs.test but IMHO there is no good reason to do it this way.

thheller06:04:08

whoever wrote fulcro-spec probably looked at cljs.test to see how to do it and just copied the approach.

thheller09:04:38

fixed in [email protected]. either (ns ^:dev/always some.ns ...) or (ns ^:figwheel-always some.ns ...) works.

thheller09:04:05

also tracked down the problem you reported a few days ago with the refactor rename problem. https://github.com/cursive-ide/cursive/issues/1978

thheller09:04:56

I might just lift the strict name requirement in shadow-cljs then there should be no issue. might just turn it into a proper warning not flat out ignore the file.

tony.kay13:04:46

I see. I agree with your opinion on the macros. The guy I had working on spec was just out of college, so he did pretty well considering. I might have made the same choice at the time.

tony.kay13:04:10

Thanks for the addition, it will save me headaches for now.

thheller13:04:02

just finished a bit of a refactoring of the fs-watch. new files and deleted files should now be detected better. just finishing up some testing with that, should be released soon.

tony.kay00:04:03

running tests on hot code reload (as I do when I use Fulcro Spec) needs it…I’m having to re-save my test main every time I add a new test to get it to reload the test runner (and scan for new tests)

lwhorton02:04:22

i was just thinking — because cljsjs and other solutions are pretty tedious (foreign libs, npm-deps, etc), and shadow’s npm support is so good .. should I just use shadow to compile all my third party dependencies and then <script> that bundle onto the page before my lein/some-other-method bundle is loaded?

lwhorton02:04:51

would that require some sort of “umd” :target that allows me to export namespaces so that a downstream cljs project can simply js/someExportedNS?

justinlee02:04:50

@lwhorton why not just include them in your source code and let shadow handle it?

lwhorton02:04:44

because shadowcljs has other hurdles like the repl, which is quite annoying to work with

lwhorton02:04:00

i want the niceities of lein as well as shadow

cmal02:04:07

Hi, can I make the app.js to load compiled cljs-runtime from another location other than /js/cljs-runtime? I found there are some var CLOSURE_BASE_PATH = '/js/cljs-runtime/'; in the beginning of the app.js.

justinlee02:04:59

Well first of all, we should figure out how to get you a repl you’re happy with. have you opened up any issues on the things you’d like to see? The problem is that @thheller doesn’t use the repl so I don’t think he’s aware of the issues. He has said repeatedly he actually appreciates it when people open up github issues.

justinlee02:04:27

Second, doesn’t shadow run in lein mode? I don’t do it myself, but I see lots of people who try to do that

lwhorton02:04:40

the first big issue is that the repl just dies with any syntax error you input

justinlee02:04:50

@cmal what are you trying to do?

lwhorton02:04:57

so you have to restart it, re-require your namespaces, move into the right namespace, etc.

justinlee02:04:12

that’s weird and you should definitely open a bug

cmal02:04:25

maybe change var CLOSURE_BASE_PATH = to another location.

justinlee02:04:50

if you just want to put files somewhere else, you can use the :output-dir option

cmal02:04:19

because the url is reverse proxied and I want those files to be another location, so the http service can access the actual location(maybe >) of the files, not .

lwhorton03:04:15

this doesn’t really allow me to use lein’s repl/figwheel unfortunately

lwhorton03:04:53

and without a lein repl I can’t use my vim-fireplace, and that’s another no-go for shadowcljs

thheller07:04:00

vim-fireplace can connect to an nREPL server no? https://shadow-cljs.github.io/docs/UsersGuide.html#nREPL

thheller07:04:41

just configure the port and connect to that. cider-nrepl works as well and emacs started adding support for shadow-cljs recently. should be simple to add to vim-fireplace as well.

thheller07:04:00

Or just go through lein. Just need to configure the :middleware then. https://shadow-cljs.github.io/docs/UsersGuide.html#_embedded_nrepl_server

cmal02:04:57

the :output-dir will change the compiled files's location, not the value of CLOSURE_BASE_PATH

justinlee02:04:32

maybe i’m being dense, but could you explain the reverse proxy setup a bit more?

cmal02:04:40

I found

(defn closure-defines [{:keys [build-options] :as state}]
  (let [{:keys [asset-path cljs-runtime-path]} build-options]
    (str "var CLOSURE_NO_DEPS = true;\n"
         ;; goog.findBasePath_() requires a base.js which we dont have
         ;; this is usually only needed for unoptimized builds anyways
         "var CLOSURE_BASE_PATH = '" asset-path "/" cljs-runtime-path "/';\n"
         "var CLOSURE_DEFINES = " (closure-defines-json state) ";\n")))
in source code, I will try to change :cljs-runtime-path.

thheller07:04:54

@lwhorton I'm more than happy to fix all the REPL issues you run into. The repl crash on invalid input for example was fixed recently. https://github.com/thheller/shadow-cljs/issues/233 I only use the REPL via Cursive where its pretty unlikely malformed input so I never noticed.

thheller07:04:18

@cmal why I are you trying to change cljs-runtime? you can change :asset-path just fine but cljs-runtime should basically never be touched. certain things need to be in this structure for source maps and stuff to work properly. I didn't want to spam the :output-dir with files so I chose the cljs-runtime sub directory

thheller07:04:43

if you reverse proxy the /js you can just reverse proxy /js/* no?

cmal07:04:44

Thanks. I've changed the :asset-path

👍 4
cmal07:04:37

@thheller Thanks for your source codes. They helped a lot. What could the problem of

shadow-cljs - socket connect failed, server process dead?
shadow-cljs - starting ...

cmal07:04:52

I am trying to run shadow-cljs server.

thheller07:04:40

it doesn't start?

thheller07:04:38

uhm can you translate that message? is it a ClassNotFoundException clojure.main?

cmal07:04:17

Error: class not found or cannot load the main class clojure.main

thheller07:04:27

will add a check for this now

thheller07:04:52

did your ~/.m2 directory get lost/deleted?

thheller07:04:04

delete .shadow-cljs/classpath.edn to fix it for now

cmal07:04:06

yeah , I just rynced my code and generated code to a remote machine. maybe because of that.

cmal07:04:19

Thank you!

thheller07:04:32

yeah its probably not safe to sync the .shadow-cljs directory to a different server

thheller07:04:44

but with the check that should be fine

👍 4
cmal07:04:57

ok, it works!

cmal07:04:17

I removed .shadow-cljs/ directory.

cmal07:04:27

Thanks a lot!

troglotit14:04:41

Hey! It’s me with yesterday’s problem. I created a repo https://github.com/troglotit/fulcro-shadow-bug - the gist is that in watch mode everything works, but in release/production build - I have React is not defined exception. I wasted a lot of time on it, maybe someone could try it on their machine

troglotit16:04:57

solved. it was because of namespaces ordering in :require

claudiu16:04:47

Wow thats a bit strange. In what file did the ordering make a difference ?

troglotit21:04:16

in root.cljs - I had to put fulcro.client.routing lower than fulcro.client & fulcro.client.dom. I had this idea because sourcemapped stacktrace of exception was in routes.cljc (in fulcro) - it didn’t had any reference to React, so I didn’t even try that at first.

claudiu14:04:41

@troglotit prod isn't working on my machine also.

lwhorton15:04:28

this AM I just tried adding shadow-cljs to an embedded repl (lein) as described here: https://shadow-cljs.github.io/docs/UsersGuide.html#_build_tool_integration . Now when I try to lein repl I get Unable to find static field: COMMON_JS_MODULE_LOAD in class com.google.javascript.jscomp.DiagnosticGroups .. and if I undo all the changes to my project.clj and blow away my .m2 dir, I still get this issue and can no longer use my repl 😞

lwhorton15:04:22

i think it’s somehow tied to https://github.com/clojure-emacs/piggieback/issues/78 , and I’m not really sure it’s a shadowcljs problem now that I think about it. it looks like something downloaded an ancient version of the closure compiler when I referenced shadowcljs in my dependencies

mitchelkuijpers15:04:49

@lwhorton Make sure you are using the closure compiler version from shadow-cljs and remove piggieback you won't need it

lwhorton15:04:56

well, I’m trying to use lein’s repl + figwheel (which offers a repl history, less-annoying HUD warnings, better error handling) as well as shadow compilation. the more i think about it the more impossible it seems, though

thheller15:04:30

@lwhorton if you use the latest CLJS Version for figwheel then both should work

lwhorton15:04:36

i’m just not sure how to hook up shadow’s watcher/builder to a figwheel-inject across the various repl, nrepl, etc.

lwhorton15:04:45

and it seems strange to do it that way

thheller15:04:01

thats not possible no.

thheller15:04:32

either tool needs to be in charge of the CLJS compiler. they can't both be.

thheller15:04:31

although I wonder what do you mean by less-annoying HUD warnings. do you mean that shadow-cljs actually warns about stuff and figwheel just doesn't display them after the first compile at all? I consider this a feature.

thheller15:04:42

better error handling and this one?

lwhorton15:04:43

two weeks ago I tried to use the shadow repl for ~2 days and found it really rough. every time there was a syntax error it would eject me from the repl, and there’s no repl-history feature so I couldn’t just up-arrow myself back into a good state

thheller15:04:50

the syntax error has been fixed. history can be fixed by using rlwrap currently. didn't get to fixing that yet.

thheller15:04:52

or use the lein repl. I suppose it works there?

lwhorton15:04:30

after a few hours of fiddling I wasn’t able to get vim-fireplace to connect to the nrepl seamlessly (even though there’s an .nrepl-port file spit out I see)-- but i’m pretty silly sometimes so Ill give it another shot

lwhorton15:04:06

> mean that shadow-cljs actually warns about stuff and figwheel just doesn’t display them after the first compile at all? I consider this a feature. this is nice until suddenly you have a transitive dependency somewhere that isn’t under your control and refuses to fix it’s issues

thheller15:04:56

it should only warn once about those now as well

thheller15:04:10

it will keep bugging you for warnings in your code though

lwhorton15:04:18

oh? I didnt see that in the changelogs

lwhorton15:04:29

the repl fix + no more annoying HUD is pretty great

thheller15:04:19

I have no idea if the annoying HUD is fixed since I never found the HUD annoying. If you have something that is reproducabely annoying when it shouldn't be: report it! I do want to make things more user friendly.

❤️ 4
lwhorton16:04:02

do the repl launchers support specifying something other than shadow.user as the root namespace? i dont see that possibility in the docs (other than when using the repl embedded in another repl), but i could be missing something

thheller16:04:52

not currently but soon

thheller16:04:10

I'm tweaking the REPL currently and that is one of the things I'm fixing

thheller16:04:51

In Cursive I have a few key binding to help my workflow. It allows me to specify a string of clojure code which I can send to the REPL on keypress. One of those is (require 'repl) (repl/go). Maybe you can make a keybinding for this in vim? Or maybe you can make it send something on REPL connect? Don't know anything about vim-fireplace.

thheller16:04:44

configuring which namespace you start in is IMHO something the client should do. seems weird that the server has to decide where the client starts.

lwhorton16:04:24

i see.. i can probably get away with an ide-specific solution

lwhorton16:04:34

my intent is for the dev setup to be as painless as possible — so simply launch a server (provide a namespace with some convenient fns, frequently used requires such as cljs.spec.alpha), and invoke (watch :my-proj)

thheller18:04:23

everyone: how would you name something that is REPL like but only does Eval and Print but not the Read?

thheller18:04:13

calling it a REPL would be misleading because it isn't

thheller18:04:25

but when you never call read in eval it is identical

thheller18:04:13

probably going with something generic like runtime-eval or so

justinlee18:04:17

seems reasonable, though runtime seems maybe redundant? when else would you eval? maybe eval-form or eval-whatever depending on what structure it expects. or eval-print

thheller18:04:44

runtime is because there are different runtimes you can eval in. eg. clj, browser, node etc

thheller18:04:12

maybe platform-eval

thheller18:04:24

its also for an entire namespace, not a single function 🙂

thheller18:04:56

building a new utility namespace to work around some REPL issues. tired of having no proper abstraction for this.

thheller18:04:32

meh doesn't matter much. can always rename the ns later 🙂

justinlee18:04:12

what’s the deal with these new repl things coming down the pipeline. prepl, unrepl

justinlee18:04:54

so far what i’ve read about them is so vague that it just seems like they are removing structure from nrepl and just forwarding streams

thheller18:04:34

mine is not that. basically what I want is to have a REPL interface built for the shadow-cljs UI

thheller18:04:40

so instead of dealing with the command line you get http://localhost:9630/repl or something like that which displays a nice editor with syntax highlighting, parinfer and all that stuff

thheller18:04:29

I want to do this properly so it can talk to CLJ and CLJS

thheller18:04:59

tried building this on top of nREPL ... wanted to jump out the window after a day or so

thheller18:04:14

prepl, unrepl both don't really work for CLJS

thheller18:04:35

well kind of but I'm not assuming a stream interface

thheller18:04:48

since the browser doesn't have a stdin

thheller18:04:17

will make more sense when its done ... doubtful that this will be exposed or usable for other tools

thheller18:04:24

will be pretty tied to the UI I have in mind for this

thheller18:04:07

fun and also extremely not fun at times 😉 I really don't understand how anyone can work with nREPL. so I really understand why people want alternatives.

thheller18:04:19

no idea what Rich has planned for prepl. not enough infos yet.

thheller18:04:42

unrepl is neat but sort of breaks apart for CLJS

justinlee18:04:22

I still don’t get why this is so hard. It still seems to me that you send a form from the client to the server, the server evals it and then returns the value and streams back and stdout.

thheller18:04:01

well REPL is really hard because you can (eval '(read)). they can be nested.

thheller18:04:37

this is how you can run a CLJS repl inside a CLJ repl

thheller18:04:08

but you can't (eval '(read)) in CLJS anyways since there is nothing to read from in the browser

thheller18:04:24

so I'm compromising in my solution and just don't allow it for CLJ either

thheller18:04:49

but the nested aspects of REPLs is really really powerful

thheller18:04:07

so people try to preserve that as best as possible

thheller18:04:58

REPLs are also stateful since you can switch namespaces and stuff but that part is relatively easy

justinlee18:04:06

what would (eval '(read)) actually do? i thought that was the job of the repl itself. why would you do that inside a repl?

thheller18:04:48

(eval '(start-another-repl))

thheller18:04:18

the REPL is just a dumb stream with EDN being the protocol

thheller18:04:42

but it just doesn't work when you go cross runtime like we do with CLJ -> CLJS

justinlee18:04:46

that’s insane. i haven’t thought about church numerals in forever

justinlee18:04:59

i do like this idea of meta programming that isn’t purely syntactic

lwhorton19:04:13

can we use read-string in a shadow-cljs.edn file for parameterized builds?

lwhorton19:04:35

(mostly for things like {:closure-defines {’foo (System/getenv “FOO”)})

thheller19:04:19

no. but there is {:closure-defines {foo #shadow/env "FOO"}}

thheller19:04:57

don't need to quote in shadow-cljs.edn

thheller19:04:43

be careful with env vars though. when running in server mode the env of the server will be used.

lwhorton19:04:33

how do you handle that use case where you need some value from “on high” that sits outside of an inline config?

thheller19:04:21

never needed that. my build configs are very static. anything more dynamic will be passed to the build at runtime.

thheller19:04:19

you can use clj-run and construct the build config however you please though

thheller19:04:59

(shadow.cljs.devtools.api/compile :foo) is basiscally short for (shadow.cljs.devtools.api/compile* (get-build-config :foo))

👍 4
thheller19:04:23

so you can run (shadow.cljs.devtools.api/compile* {:target :browser :whatever-config-you-want})

thheller19:04:40

config doesn't have to be in shadow-cljs.edn

thheller19:04:53

thats just convenience for the usual cases

eoliphant21:04:44

I’m getting the following error on hot reloads for some reason.

eoliphant21:04:13

invariant.js:43 Uncaught Error: Target container is not a DOM element.
    at n.exports (invariant.js:43)
    at Ka (react-dom.development.js:16426)
    at Object.render (react-dom.development.js:16514)
    at Object.reagent$dom$render_comp [as render_comp] (dom.cljs:30)
    at Function.reagent.dom.render.cljs$core$IFn$_invoke$arity$3 (dom.cljs:54)
    at Function.reagent.dom.render.cljs$core$IFn$_invoke$arity$2 (dom.cljs:49)
    at Function.reagent.core.render.cljs$core$IFn$_invoke$arity$2 (core.cljs:75)
    at Object.sqdplatform$core$mount_components [as mount_components] (core.cljs:79)
    at Object.sqdplatform$core$init_BANG_ [as init_BANG_] (core.cljs:86)
    at app.cljs:7
n.exports @ invariant.js:43
Ka @ react-dom.development.js:16426
render @ react-dom.development.js:16514
reagent$dom$render_comp @ dom.cljs:30
reagent.dom.render.cljs$core$IFn$_invoke$arity$3 @ dom.cljs:54
reagent.dom.render.cljs$core$IFn$_invoke$arity$2 @ dom.cljs:49
reagent.core.render.cljs$core$IFn$_invoke$arity$2 @ core.cljs:75
sqdplatform$core$mount_components @ core.cljs:79
sqdplatform$core$init_BANG_ @ core.cljs:86
(anonymous) @ app.cljs:7
shadow$cljs$devtools$client$browser$script_eval @ browser.cljs:58
shadow$cljs$devtools$client$browser$do_js_load @ browser.cljs:64
(anonymous) @ browser.cljs:78
(anonymous) @ env.cljs:134
shadow$cljs$devtools$client$env$do_js_reload_STAR_ @ env.cljs:118
shadow.cljs.devtools.client.env.do_js_reload.cljs$core$IFn$_invoke$arity$3 @ env.cljs:142
shadow$cljs$devtools$client$browser$do_js_reload @ browser.cljs:67
(anonymous) @ browser.cljs:152

thheller22:04:09

@eoliphant looks like you are calling (reagent/render ... the-node) on a DOM node that was maybe removed/overriden? check the dom node you are passing to render if it actually exists

eoliphant22:04:12

ok so, hmm, this works on the initial load, but only blows up when I’ve made a change and shadow pushes the new code over

thheller22:04:29

so something removes the dom node. shadow definitely does not do that

eoliphant22:04:01

this is so weird. I have the pretty much the same config on a another couple projects and it works fine. The weird thing is that if the say root render node got hammered somehow, wouldn’t the page, prior to the update be I dunno, gone?

thheller22:04:58

did you try logging the node before you use it?

thheller22:04:22

what are you using as arg for reagent/render?

eoliphant22:04:11

it’s a luminus template project (r/render [#'page] (.getElementById js/document "app")) where page is a component func that sets up the nav, page though, hmm. I was mucking around with the map of pages that is uses. . Let me double check that

thheller22:04:52

I think there was one issue with the HTML5 History which seemed to replace the entire dom when reloading code?

eoliphant22:04:39

yeah, but it works find in my other config. Interestingly the only thing that’s in the generated dom isa hidden history state input

thheller22:04:37

can you comment out the history stuff and see if it works?

eoliphant22:04:59

yeah trying that and reverting my changes

eoliphant22:04:57

I just reverted all my changes and it’s fine, even with the history. Going to walk them back forward until something breaks