This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-02-08
Channels
- # announcements (2)
- # aws (1)
- # beginners (134)
- # calva (26)
- # cider (48)
- # cljdoc (41)
- # cljs-dev (12)
- # clojure (178)
- # clojure-brasil (1)
- # clojure-europe (16)
- # clojure-italy (30)
- # clojure-nl (13)
- # clojure-spec (118)
- # clojure-uk (81)
- # clojurescript (209)
- # community-development (77)
- # cursive (7)
- # datomic (23)
- # duct (6)
- # emacs (15)
- # events (2)
- # figwheel (13)
- # figwheel-main (18)
- # fulcro (4)
- # jackdaw (4)
- # jobs (6)
- # jobs-discuss (6)
- # kaocha (2)
- # lein-figwheel (3)
- # off-topic (4)
- # other-languages (22)
- # pathom (2)
- # pedestal (9)
- # perun (10)
- # portkey (1)
- # re-frame (41)
- # reagent (6)
- # reitit (4)
- # remote-jobs (1)
- # ring-swagger (6)
- # rum (5)
- # shadow-cljs (300)
- # sql (3)
- # test-check (6)
- # testing (7)
- # vim (1)
- # yada (9)
@thheller I’m seeing flaky builds on our build with shadow related to (I think) macroexpansion
in a macro we expect a Clojure var to exist (we resolve
a symbol) but the result of resolve
is coming up nil
on and off
that probably is related to parallel compilation (which is why resolve
is a bad idea)
that makes sense
resolve
is on the Clojure side btw
during macroexpansion
if you have no explicit dependency on a namespace there is no guarantee it will be compiled when you expect it to
we have an explicit dependency on that namespace under a :clj
reader conditional
which is why this feels puzzling to me
I’ll try with :parallel-build false
how is that not supported? 😄
I thought it was
let me give you a proper example
(defmacro lazy-load
[module-id comp-sym]
(let [comp-var (ana/resolve-var (dissoc &env :locals) comp-sym)
comp-name (with-meta (:name comp-var) {:cljs.analyzer/no-resolve true})]
`(LazyComponent. ~(get-query @(resolve comp-sym))
(fn []
;; shadow loader callback, not important right now
(-> (~(with-meta 'shadow.loader/load {:cljs.analyzer/no-resolve true})
~(name module-id))
(.then (fn []
(resolve# (cljs.core/js-obj "default" ~comp-name)))))))))
@thheller ^ this is literally the code I’m running
the important part is ~(get-query @(resolve comp-sym))
that’s a fully qualified symbol for which we have a require
in the NS declaration (on the Clojure side)
the goal here is basically to inline the Om Next query to allow for code splitting - but that’s not important right now
hmm I don't get it? why are you calling resolve
? that looks like you are trying to resolve a CLJS var using CLJ resolve?
(with-meta 'shadow.loader/load {:cljs.analyzer/no-resolve true})
this also doesn't make sense to me?
that’s not the part that’s throwing
I actually elided the code a little bit, resolve#
is a js/Promise.resolve
~(get-query @(resolve comp-sym))
this is the part that runs in Clojure
because we need that value without requiring the namespace where it’s in
Om Next needs to know the query for the entire app statically
which is at odds with code splitting
so we inline it at build time
open to other suggestions if you’re seeing something I’m not
they are available at compile time
these are all cljc
files
or they should be available
we have require
s for all these namespace in the clj
side
these builds are flaky, they’re not 100% wrong all the time
sometimes they work, sometimes they don’t ¯\(ツ)/¯
FWIW I'm using code-splitting in fulcro without issues regarding queries. they are just adjusting based on what the current route is
yeah I know
we use a different scheme
for historical reasons it’s too much code to move
we need to adapt
I really don’t think that matters in this case
they’re standard (defui Component static om/IQuery (query [this] [:query :here]) ...
we have a different setup, it’s not really om next anymore
we don’t need any metadata
(defmacro lazy-load
[module-id comp-sym]
{:pre [(keyword? module-id)
(qualified-symbol? comp-sym)]}
`(LazyComponent. ~(get-query @(resolve comp-sym))
(fn []
;; shadow loader callback, not important right now
(-> (shadow.loader/load ~(name module-id))
(.then (fn []
(cljs.core/js-obj "default" ~(symbol "js" (comp/munge comp-sym)))))))))
this shouldn’t be reloading anything actually
it’s just building the test builds from scratch
@thheller I think you missed what the issue was too
you changed the part that has the comment that says not important right now
the part that’s actually failing is the othe rone
also in this build we have :build-options {:cache-level :off}
I think I’m slowly disproving your theories, apologies
let me give you the entire build config
instead of throwing pieces at you
{:cache-root (str temp-dir "/.shadow-cljs")
:builds {:karma {:target :karma
:output-to (str out-dir "/test.js")
:ns-regexp (str "^(" (string/join "|" namespaces) ")$")
:build-options {:cache-level :off}
:js-options {:node-modules-dir node-dir}}}}
ok sec
what’s missing?
oh yeah sec
there’s definitely base-config (edn/read-string (slurp shadow-edn))
in here
yeah we’re merging that in
I think those are the relevant bits anyway
ok so, relevant parts of the stack trace:
File: /var/lib/buildkite-agent/builds/buildkite-5dc4755b74-2mfgf-1/ladder-life/ladder/buck-out/gen/src/ladder/__views-next_cljs_cljs_classpath__/views-next_cljs_symlink_tree.jar/ladder/views_next.cljc
null
#:clojure.error{:source "ladder/views_next.cljc", :line 438, :column 32, :phase :macroexpansion, :symbol ladder.ui/lazy-load}
ExceptionInfo:
cljs.analyzer/macroexpand-1*/fn--2050 (analyzer.cljc:3792)
cljs.analyzer/macroexpand-1* (analyzer.cljc:3789)
cljs.analyzer/macroexpand-1 (analyzer.cljc:3835)
cljs.analyzer/analyze-seq (analyzer.cljc:3870)
cljs.analyzer/analyze-form (analyzer.cljc:4061)
cljs.analyzer/analyze* (analyzer.cljc:4109)
cljs.analyzer/analyze (analyzer.cljc:4129)
cljs.analyzer/analyze (analyzer.cljc:4114)
this is not very informative
Caused by:
NullPointerException:
clojure.core/deref-future (core.clj:2300)
clojure.core/deref (core.clj:2320)
clojure.core/deref (core.clj:2306)
ladder.ui/lazy-load (ui.cljc:344)
ladder.ui/lazy-load (ui.cljc:336)
^ this is more
try :compiler-options {:parallel-build false}
. forgot that I added support for that not too long ago
oh so that actually does something?
do you think adding cache-blockers
in this case would do anything?
yeah I’m gonna try something different too
related to our internal setup
if you say caching is affecting things here
yeah we don’t
buck extracts them
right
I meant other type of caching
we have a “tainted clojure worker”
which is a type of clojure worker that we keep around in a pool
that has done other compilation stuff before
that we reuse, to avoid starting up
so if you say caching might be a problem here, I’m gonna force this to run in a regular worker
which is what’s happening here 🙂
the NPE likely happens when you try to (lazy-load :foo 'some.other/thing)
and some.other/thing
isn't loaded on the CLJ side
which may be a missing require that would sometimes get loaded by other namespaces during parallel compile
without parallel-build and a clean worker the build should always fail with the NPE in the case of a missing require on the CLJ side
@thheller the require is 100% not missing
and it’s in the classpath for sure too
I’ve triple checked
the thing I haven’t ruled out yet is the tainted persistent worker
I did
yeah the result of resolve
is nil
you can add the (prn [:loading-that-namespace])
to the namespace the resolve is referencing
indeed, I’ll try that
yeah that’s the last option
yeah requiring-resolve
might help me here
or however it’s called
otherwise one thread may start the require and another thread may start using it while it is still loading
@thheller I think moving it to the non-tainted worker made it work - even though the build is taking a lot longer (expected). so I’ll just use requiring-resolve
, otherwise the build time is infeasible
parallel-build shouldn't make that much of a difference overall, depends on how parallelizable your compile is
no I mean we have a bunch of “targets” that we build separately (because Buck)
so we really need to have some compile cache (especially on the Clojure side) between building and running those targets
that’s what the tainted worker does for us
btw this is also why we want Clojure to have a stable ABI
it would make all this much easier
@thheller maybe I can nerdsnipe you into making ClojureScript have a stable ABI 😛
e.g. through shadow
requiring-resolve
made the build work instantly
it means that if you can have truly separate compilation
say you have namespaces:
A
/ \
B C
triggering a change in C shouldn’t need for A or B to get recompiled
and this goes further than having the transit cache too
it’s not, really
because if you compile A and exit
then compile B
the 2nd run still does some work for A
I don’t think that’s the only work it does
it doesn't do any compiler work, it does a bit to restore cache into the right places and stuff
maybe in the example I gave it is
but once you have a more complex dep graph there’s lots of work happening that wouldn’t need to happen
hmm no not really. the caching stuff ensures that only the work that needs to happen actually does happen
validating the cache and loading it is a bit of work yes but involves no actual compiler work
I’m pretty sure I’m forgetting something here
I’m fuzzy on the details
the gist of the issue is: if you have a build tool that calculates your dep graph already, you don’t need the cljs compiler trying to do it for you too
and then all the consequences that that entails
ie. parse all the ns
forms of all namespaces, account for macros, npm
dependencies, resolve all those with all the "browser"
rules and stuff
Buck does
we parse the NS deps
and hand it to buck
then we auto generate a file with all the deps
given it needs to be static
a rather big chunk of shadow-cljs is determining when to cache and when to recompile
it’s not that we want
it’s that Buck already does it for us
and has a distributed cache, so if we built something on CI, Buck just downloads it to our laptops
in fact we actually let Shadow take over when building CLJS
so we don’t actually enforce the Buck best practices when using Shadow
simply because we’d need to get out of our way to do it with lots of custom / brittle code
I’m not sure. I’m gonna raise that question internally
I don’t think it’s that easy
the whole compilation model would need to be different
this is very outdated
but it’s how we used to do it with the bare CLJS compiler
shadow now takes over
> The biggest benefits of using buck come from using the artifact cache. It's especially useful if you have several developers working on the same codebase since the cache can be distributed. You may also see better CI times with buck caching if you configure your CI to cache the buck-out directory between runs.
right but we already bought into buck
just so we get that for Clojure code too
to be fair this might be one of those things where you have to experience it to believe it
I was this way too
so rephrasing my earlier statement
I believe that Shadow and the ClojureScript compiler do have intermediate caching and cached builds
my problem is that the ABIs are not stable or public
imagine that in my earlier A, B, C example
if you added one function to A that neither B or C used
those are gonna get recompiled anyway
that kind of fine grained control is something that we could implement if there was actually an ABI
yes that is true. given the dynamic nature of CLJ(S) that is probably never going to change
brb lunch
it is something I have been thinking about quite a bunch since it could also speed up live-reloads
you change it to (defn foo [] 2)
all namespaces that require this will be fully recompiled
so during namespace compiling you track what signatures you used from other namespaces and check if they changed
that is quite doable ... but then you add macros and everything goes completely crazy 🙂
now you get it 🙂
@anmonteiro this is somewhat related. how would you teach Buck that
also depends on demo/test.md
. https://clojureverse.org/t/using-none-code-resources-in-cljs-builds/3745
oh that’s trivial I think
Buck targets depend on other targets
so you’d make one target that encompasses the resources folder
and make your JS bundle target depend on that
Hi! When coding clojure I’m using a socket repl (in conjunction with REBL). Now I’m trying to add clojurescript using shadow-cljs, I’m a bit stuck on how to get everything working together. I’m using deps.edn. Can anyone point to a working example?
@jlmr not sure that works yet. I haven't run REBL yet since it doesn't seem to like Java11 and Windows
Hello I’m trying to use clojure/tools.cli with shadow-cljs but I’m seeing a couple of warnings:
------ WARNING #1 - :undeclared-var --------------------------------------------
Resource: clojure/tools/cli.cljc:126:17
--------------------------------------------------------------------------------
123 | (recur options (into extra-args (vec (rest args))) nil)
124 |
125 | (and (opt? opt) (nil? spec))
126 | (throw (Exception. (str "'" opt "' is not a valid argument")))
-----------------------^--------------------------------------------------------
Use of undeclared Var clojure.tools.cli/Exception
--------------------------------------------------------------------------------
127 |
128 | (and (opt? opt) (spec :flag))
129 | (recur ((spec :assoc-fn) options (spec :name) (flag-for opt))
130 | extra-args
--------------------------------------------------------------------------------
------ WARNING #2 - :undeclared-var --------------------------------------------
Resource: clojure/tools/cli.cljc:228:25
--------------------------------------------------------------------------------
225 | (when *assert*
226 | (let [unknown-keys (keys (apply dissoc map spec-keys))]
227 | (when (seq unknown-keys)
228 | (binding [*out* *err*]
-------------------------------^------------------------------------------------
Use of undeclared Var clojure.tools.cli/*err*
--------------------------------------------------------------------------------
229 | (println (str "Warning: The following options to parse-opts are unrecognized: "
230 | (s/join ", " unknown-keys)))))))
231 |
232 | (select-keys map spec-keys))
--------------------------------------------------------------------------------
the warning looks legit. CLJS doesn't have a Exception
class. I guess someone didn't actually test this much.
I'm getting a compilation error every time I run yarn shadow-cljs watch app
:
[:app] Build failure:
------ ERROR -------------------------------------------------------------------
File: jar:file:/mnt/c/Users/conan/.m2/repository/org/clojure/clojurescript/1.10.439/clojurescript-1.10.439.jar!/
cljs/core.cljs:999:14
--------------------------------------------------------------------------------
996 | (if (js/isFinite o)
997 | (js-mod (Math/floor o) 2147483647)
998 | (case o
999 | ##Inf
--------------------^-----------------------------------------------------------
No reader function for tag Inf
--------------------------------------------------------------------------------
1000 | 2146435072
1001 | ##-Inf
1002 | -1048576
1003 | 2146959360))
--------------------------------------------------------------------------------
Any ideas why? I found some discussion of this error that referred to tools.reader
, but it's old and my lein deps :tree
says I'm using [org.clojure/tools.reader "1.3.2"]
.
https://clojurians-log.clojureverse.org/shadow-cljs/2017-10-03Anyone ever run into Invariant Violation: Target container is not a DOM element.
with shadow?
I do have the script tag after the root node in the body
(re/render [admin-panel] (.getElementById js/document "app"))
lol, copypaste from old deps.edn code. should be "root" 😂
(re/render [admin-panel] (.getElementById js/document "app"))
lol, copypaste from old deps.edn code. should be "root" 😂
@conan that is indeed a tools.reader conflict. you can check if you actually get the version you expect by using shadow-cljs clj-repl
and (
Yep, here's the output:
[1:0]~shadow.user=> ( "cljs/tools/reader.cljs")
#object[java.net.URL 0x1d579f90 "jar:file:/mnt/c/Users/conan/.m2/repository/org/clojure/tools.reader/1.3.2/tools.reader-1.3.2.jar!/cljs/tools/reader.cljs"]
no profiles, I'm using :lein true
and :target :node-library
, maybe they affect the classpath? here's my shadow-cljs.edn
:
{:lein true
:builds {:app {:target :node-library
:exports {:handler conan.core/handler}
:output-dir "target"
:output-to "target/main.js"}}}
here's my project.clj
:
(defproject conan "0.1.0-SNAPSHOT"
:min-lein-version "2.7.0"
:dependencies [[com.cemerick/url "0.1.1"]
[com.taoensso/timbre "4.10.0"]
[io.nervous/kvlt "0.1.4" :exclusions [org.clojure/clojurescript]]
[org.clojure/clojure "1.10.0"]
[org.clojure/core.async "0.4.490"]
[thheller/shadow-cljs "2.7.11"]]
:source-paths ["src/cljs"])
i have clojure 1.10 but clojurescript 1.10.439. i don't actually specify clojurescript in my deps as i assumed it would be better to let shadow-cljs choose, should i be explicit?
I don't know honestly .. debugging dependency issues is hard. just don't use project.clj
oh good idea, but i don't actually have a .lein/profiles.clj
. didn't think of it though
yeah that seems to be the problem. i'm guessing there are no compilation errors in there...
the error you posted is a rather old one and was caused by invalid tools.reader versions
if you remove lein
and move the dependencies to shadow-cljs.edn
and still run into this issue let me know
thanks very much for your help. i'll try starting a new project from scratch and slowly introducing stuff until it breaks (or doesn't). maybe it's just a gremlin ¯\(ツ)/¯
do NOT change anything other than removing :lein true
and copying :dependencies
to shadow-cljs.edn
Hey there, I'm trying to get a quick turnaround when developing a library against another application I have by running watch
on the library and copying the bundle over to my app's node_modules
folder and restarting the (create-react-app, which wraps webpack) dev server there. It appears to work fine when I run release
, but if I watch
, I get Error: browser bootstrap used in incorrect target
. Any reason for this or way I can get it to work? I'm using es6 modules, which I believe compile down to require
calls. I tried the solution here: https://github.com/thheller/shadow-cljs/issues/376 but it's not exactly my situation and it seemed to make things worse.
I know in a macro definition we can use (boolean (:ns &env))
to detect if this macro is being used in CLJS code, but is there a way to detect, in a macro, that this macro is being called in regular browser cljs or nodejs?