This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-10-29
Channels
- # announcements (2)
- # babashka (2)
- # beginners (76)
- # boot (6)
- # calva (7)
- # cider (12)
- # clara (4)
- # clj-kondo (11)
- # cljdoc (9)
- # cljs-dev (21)
- # cljsrn (7)
- # clojure (72)
- # clojure-dev (158)
- # clojure-europe (2)
- # clojure-italy (3)
- # clojure-losangeles (3)
- # clojure-nl (5)
- # clojure-spec (29)
- # clojure-uk (93)
- # clojurescript (40)
- # cursive (7)
- # data-science (1)
- # datomic (28)
- # defnpodcast (5)
- # duct (5)
- # emacs (7)
- # events (2)
- # figwheel-main (5)
- # fulcro (55)
- # graalvm (2)
- # instaparse (1)
- # jobs (5)
- # juxt (1)
- # luminus (3)
- # nyc (2)
- # pathom (3)
- # planck (25)
- # re-frame (2)
- # reagent (4)
- # reitit (23)
- # shadow-cljs (381)
- # spacemacs (6)
- # sql (19)
- # tools-deps (7)
- # xtdb (4)
Hello! Newb here! I’m having a problem requiring javascript modules from cljs. I’ve created a super-simple reproduction (~4 lines). Googling my error comes up with nothing. Can anybody help me? I’m compiling with npx shadow-cljs compile npm
.
@gdwarner you should have builds configuration in your shadow-CLJS.edn
If you target node, you can check this repo
In perticular this file
https://github.com/minimal-xyz/minimal-shadow-cljs-nodejs/blob/master/shadow-cljs.edn
Hi. I have a problem with a macro, after I switched from figwheel
to shadow-cljs
. My macro (defmacro assert ...)
lives in kunagi_base/utils.cljc
. I have no :require-macros
or anything like that in my code. With figwheel this worked fine. Shadow-cljs gives me this error: Use of undeclared Var kunagi-base.utils/assert
. Any suggestions? Thank you!
There’s a stacktrace in the bottom of the snippet: ReferenceError: shadow$provide is not defined
ok that error is because :npm-module
defaults to :runtime :browser
but you are using it in node
you need an actual build config like :builds {:npm {:target :npm-module :runtime :node :output-dir "node_modules/shadow-cljs"}}
@witek see https://code.thheller.com/blog/shadow-cljs/2019/10/12/clojurescript-macros.html you might have been relying on https://clojure.atlassian.net/browse/CLJS-2454
When using release
, I get Uncaught TypeError: Cannot read property 'g' of undefined
. Wasn't there a way to create release output with hints to the real (not shortened) names?
@witek shadow-cljs release app --pseudo-names
will make the names slightly more recoginizable but you probably want to turn on https://shadow-cljs.github.io/docs/UsersGuide.html#infer-externs if you haven't already
Trying to use shadow-cljs in Cursive with the pom.xml
method. Should Cursive be able to resolve references to node modules (or javascript references such as (.log js/console "...")
), or is it expected that it doesn’t?
Found previous questions regarding this, sorry about that 🙂 https://clojurians.slack.com/archives/C6N245JGG/p1572023469012800
Does that apply for js in general (including the npm packages)?
I'm not sure about the state of JS support in Cursive. it has never worked for me so I'm not sure if its suposed to work.
Got it - thanks 🙂
I’m still seeing problems when restarting the shadow-cljs build. The process is, I kill shadow-cljs to add a new dependency in deps.edn, then start it again. Then a bunch of errors appear, at their root:
TypeError: module$node_modules$react$index.createContext is not a function
at eval (routes.cljs:10)
at eval (routes.cljs:10)
at eval (<anonymous>)
at Object.goog.globalEval (main.js:827)
at Object.env.evalLoad (main.js:2174)
at main.js:2412
If I delete all compiled output, the shadow-cljs cache, and node_modules, then starting the build again, everything compiles fine.
I get a lot of similar errors too: TypeError: _react.default.createContext is not a function
, TypeError: React.forwardRef is not a function
, TypeError: module$node_modules$react$index.memo is not a function
and similar.
Roger that. I’ll keep a copy of the .shadow-cljs, node_modules and the compiled output.
also keep more errors ... excerpts of errors are pretty much pointless. it is likely one error cascading out and causing all the rest
It looks like that React itself has some issue. It could be some dependency but I can’t be sure at this point.
Yep, that seems to be the case. One thing that I see in the compiled output is that while I have a “main.js” file which is 11MB, there’s also a whole bunch of individual files under cljs-runtime
.
@neo2551 I’m targeting node, but with the npm-module
variant. The docs say that I can use the default shadow-cljs.edn via the ‘covenience mode’. https://shadow-cljs.github.io/docs/UsersGuide.html#_convenience_mode .
does it matter: hi vs “hi”. Or do I need to include “hello” since that’s what I’m trying to require?
but you only need to list the "entries". meaning namespaces accessed from elsewhere (eg. require in node). it'll include its dependencies automatically
any particular reason you are using :npm-module
? :node-library
and :node-script
are much better generally
I want to call my clojure lib/module from javascript. I don’t think I want node-script
, but node-library
may be appropriate. I have no intention of actually creating an npm module.
If one of my deps is described as “cljs support coming soon”, but contains (temporarily) incorrect .cljc
files, is there a way to tell shadow to ignore that dep during cljs compilation? I don’t intend to use it from cljs, but do want it on my classpath at the clj REPL.
@cjsauer not sure what you mean. what is the actual problem you see? just files on the classpath shouldn't hurt unless you use them?
When I add this specific dependency to my project, it breaks cljs compilation by throwing a bunch of warnings about undefined vars in my test files. Removing the offending dep fixes things. I’m guessing that the bad .cljc
files in that dep are causing cljs compilation to fail in some way.
or do you add the dependency via deps.edn
:local/root
or gitlibs and use a test target?
I don’t require this specific dep in the *-test.cljs
files that the warnings point to… :thinking_face:
It points at about 6 of my deftest
macros with an undeclared var
warning, for example:
(deftest entity-ids->lookup-set
Use of undeclared Var my-app.client-test/entity-ids->lookup-set
Yeah…not sure…removing the dependency also removes all of those warnings strangely enough.
[io.replikativ/datahike "0.2.0"]
https://github.com/replikativ/datahike
Attempting to compile the .cljc
files in that repo fails with all kinds of exceptions, because the cljs support isn’t there yet. My thought was those files were somehow tripping up my own compilation. But:
>it doesn’t matter whats on the classpath UNLESS you actually :require
a file
Ah sorry, by that I meant cloning that repo and running their cljs tests doesn’t work (which is expected). It fails compilation.
This was when I thought maybe datahike’s cljc files was to blame >BUT your issues seem related to something else This is likely the case now that I know shadow is likely not touching any datahike code
a) Yeah, I have working tests using the :browser-test
build target. I can open the resulting build up in the browser and see All tests passed
.
b) I add the datahike dependency to my deps.edn
file, and then rerun my :browser-test
build, which results in a bunch of warnings, but I just found that there is one interesting one:
Resource: datascript/core.cljc:586:36
Use of undeclared Var cljs.reader/register-tag-parser!
c) Correct. None of my code has made any attempt to require or use datahike yet.
So, there seems to be some conflict between datascript and datahike that I think is likely unrelated to shadow (apologies for the noise here). The key thing missing from my understanding of shadow was this:
>it doesn’t matter whats on the classpath UNLESS you actually :require
a file#object[java.net.URL 0x72d5daf9 "jar:file:/Users/calvin/.m2/repository/org/clojure/clojurescript/1.10.520/clojurescript-1.10.520.jar!/cljs/reader.cljs"]
Seems to exist and load just fine. Likely a dep conflict like you mentioned. I’ll keep investigating. Thanks for your help 🙏Hm…not intentionally. I can’t seem to find that directory anywhere on my filesystem. Commenting out the datahike
dependency causes all of these conflicts to disappear…this /private
folder doesn’t exist in the datahike
jar file either :thinking_face:
I'll remove this particular behavior so that extra "bad" output doesn't cause trouble
Aha, it looks like replikativ’s hasch
lib is the culprit for that specific conflict:
( "private/js/out/cljs/reader.js")
#object[java.net.URL 0x698e4e6c "jar:file:/Users/calvin/.m2/repository/io/replikativ/hasch/0.3.5/hasch-0.3.5.jar!/private/js/out/cljs/reader.js"]
Any suggestion on how to move everything into a new directory, e.g. dist/
when run npm run build
? I understand this might not be shadowcljs’ job, but wonder how people do it.
It’s the npm-script. I usually config something like
"scripts": {
"start": "shadow-cljs watch app",
"build": "shadow-cljs release app"
},
so why do you want to copy it? why not just have shadow-cljs write the output dir you actually want?
if you really need to copy you can just use "build": "shadow-cljs release app && cp dist somewhere"
or so?
It’s just my preference to have everything built to a separate folder. If you dev and release in the same folder, say public/js/
, then do you commit this folder in VCS, say git? I do want to commit the release version, but then I will always see it changing in git status
when I dev.
something like shadow-cljs release app && cp dist somewhere
would work, but then before I copy, I assume I need to clean up the dev files? But shadow-cljs doesn’t have a clean command, so…
in https://github.com/filipesilva/create-cljs-app/blob/master/template/package.json I instead use a clean command
"start": "shadow-cljs watch app",
"build": "yarn clean && shadow-cljs release app",
"clean": "rimraf public/js"
for the same reasons really
when deploying dist/
, I don't want to include the non-prod js files that get produced there
@filipematossilva do you have any plans to add something like react-scripts for create-cljs-app?😁
so abstracting away all the tools in a separate package, right?
I thought about it but I think that runs a bit contrary to the clj ethos... although this starter is directed to js folks
I also don't feel that confident in my knowledge of the "best" way of doing things
@filipematossilva would you mind changing the default generated config a bit. I'd rather it used the newer :dev-http
config
so instead of
{:builds
{:app
{:asset-path "/js",
:devtools {:http-port 3000, :http-root "public"},
:modules {:main {:init-fn app.core/main}},
:output-dir "public/js",
:target :browser},
:e2e
{:ns-regexp "e2e.*",
:output-to "out/e2e.js",
:target :node-test},
:test
{:ns-regexp "app.*-spec$",
:output-to "out/test.js",
:target :node-test}},
:dependencies [[reagent "0.8.1"]],
:nrepl {:port 3333},
:source-paths ["src" "e2e"]}
you do {:builds
{:app
{:asset-path "/js",
:modules {:main {:init-fn app.core/main}},
:output-dir "public/js",
:target :browser},
:e2e
{:ns-regexp "e2e.*",
:output-to "out/e2e.js",
:target :node-test},
:test
{:ns-regexp "app.*-spec$",
:output-to "out/test.js",
:target :node-test}},
:dependencies [[reagent "0.8.1"]],
:nrepl {:port 3333},
:dev-http {3000 "public"}
:source-paths ["src" "e2e"]}
Good to know!
And where would :http-resource-root
and :preloads
fit into this new pattern?
regarding the config changes, yes let me add it
regarding the indentation, that is the default for zprint-clj
https://github.com/kkinnear/zprint
I couldn't find another formatter that had a good npm distro
is there a canonical config for these formatters that matches what people use?
btw you can always run clojure code using https://shadow-cljs.github.io/docs/UsersGuide.html#clj-run
I had no idea, that sounds pretty handy!
so right now shadow-cljs run
lets me run a given function in a clj namespace I have
but to use it directly as shown in https://github.com/kkinnear/zprint#clojure-cli might not work currently, is that right?
I trying just doing run zprint.main
without any modification to shadow-cljs.edn besides adding the dependency, and it seems to print the formatted output
kamik@RED-X1C6 MINGW64 /d/work/create-cljs-app/template (master)
$ shadow-cljs run zprint.main <shadow-cljs.edn
shadow-cljs - config: D:\work\create-cljs-app\template\shadow-cljs.edn cli version: 2.8.64 node: v10.16.0
{:builds
{:app {:asset-path "/js",
:modules {:main {:init-fn app.core/main}},
:output-dir "public/js",
:target :browser},
:e2e {:ns-regexp "e2e.*", :output-to "out/e2e.js", :target :node-test},
:test
{:ns-regexp "app.*-spec$", :output-to "out/test.js", :target :node-test}},
:dependencies [[reagent "0.8.1"] [zprint "0.5.1"]],
:dev-http {3000 "public"},
:nrepl {:port 3333},
:source-paths ["src" "e2e"]}
it's very slow though
@filipematossilva have you read about server mode and how it functions? (regarding the "slow" comment)
I'm familiar with it insofar as using shadow-cljs start
in the background to speed up further commands.
I didn't try it with this command though
let me try
but yeah the point is to get rid of the startup delay. so in general the server should always be running when you are developing
Yes you are right, it is much faster now. Especially the second and subsequent runs.
I need to think a bit about this usage pattern... It feels pretty powerful insofar as it provides direct access to all these clj tools. But I'd need to see if I could get a 1-to-1 correspondence as far as functionality goes. The npm zprint wrapper allows formatting all files and writing them back to the same place, and that sounds like it'd need a custom function to do. But a custom function is also a good example of how a user can customize it.
I just wanted to mention it. just in case there is something available in clj but not on npm
the zprint defaults seem to suck for code but there was a big discussions about formatters a while ago
I remember seeing a discussion on clojureverse I think
it looked stalled
yeah its a bit easier in JS and especially JSON configs since they are least maintain insertion order
I had an unrelated question for you
are the npm wrappers today the same as described in https://code.thheller.com/blog/shadow-cljs/2018/06/15/why-not-webpack.html?
so something like
// node_modules/lib/foo.js
shadow$provide["module$node_modules$lib$foo"] = function(module, global, process, exports, require) {
var foo = 1;
exports.hello = function() { return foo; };
}
// node_modules/lib/bar.js
shadow$provide["module$node_modules$lib$bar"] = function(module, global, process, exports, require) {
var foo = require("module$node_modules$lib$foo");
foo.hello();
}
in regards to module usage, I assume these wrappers can never avail from code motion, right?
GCC has a nice feature called code motion, where it can "split" the content of a given JS file if part of it is used in one module but not the others
so for instance
// node_modules/lib/foo.js
shadow$provide["module$node_modules$lib$foo"] = function(module, global, process, exports, require) {
exports.hello = function() { return 1; };
exports.world = function() { return 2; };
}
is there a way to exclude certain modules from the wrapper?
I was trying to imagine how would code motion be used for npm libraries, assuming the library was in a GCC friendly format
I experimented with all of this a very long time. basically it never works. there are barely and "strict" ES6+ modules on npm (that also only depends on other strict ES6+ code)
I don't mean for the general case though
yeah, now that I think about it I do
so for instance I could try to import @angular/core
. It is published in ES2015 among other formats. It is also annotated for GCC via tsickle
(a TS wrapper to annotate TS output for GCC).
nevermind that there's no way in hell angular will properly work in cljs
but that library is full ES2015
then separately I also had momentjs, which is in crazy format
I import them both from my app
but neither has an interdependency
I would like to tell shadow-cljs "try to import angular/core in full GCC mode, but keep momentjs in compat"
you can "fake" it by copying node_modules/@angular/core/esm5/*
to src/main/angular
or so
yes, that limitation I did expect
so back to the old externs way
yeah but that kinda defeats the purpose of trying to optimize these further
because that's es5 code in ESM format
there's also es2015 code in another folder
the esm2015 one
when you said this doesn't work, do you mean that it's not a feature that is available or that there's no reasonable way such a feature could be developed?
well, not going to call it impossible. I certainly can't implement it in a reasonable way.
You mentioned there is no extern inference for js code. I imagined that the wrapped npm libs got some sort of extern inference.
is it that they don't actually get any extern inference?
the :advanced
parts need externs for the code not part of the advanced compilation (ie. npm code)
AND if you have npm libs accessing the advanced code you'll need externs for those parts
the current state of npm just doesn't allow anything more than what is currently done
no I don't think there's ever going to be a solution for the re-entrant case
where by re-entrant I mean going from "compat" back to "full gcc" code
which is probably me misusing the word
yeah but no one is going to bother getting all that
which is why shadow-cljs filled up this need so well
full npm compat is sweet
tbh that's what drew me to shadow-cljs instead of any other setup
I'm from a JS background and it's a hard sell to have a huge overhead to all the stuff you already know
yeah, same for me with Clojure .. already had 10+ years of experience with Java so having access to that was sort of the killer argument
we'll see how this evolves. I forsee a lot of breakage if they ever adopt more ESM code
atm some libraries publish it under non-standard package.json entry points, like module
, esm
, or es0215
which means that the resolver needs to know to look these up... and it gets messy
I sorta expected the wrapper to handle the interop well enough, it's similar to the webpack wrapper
which means all code moving to the strict variant will break the packages not yet moved
for all the good webpack did, it also messed stuff up a bit by letting people write packages that only work on webpack pipelines
like all the require('something.css')
and defaulting resolving .wasm
extensions by default
huge burden on consumers
I understand the need for all of that. just wish they hadn't built it on top of require
yeah something that just wasn't overloading the default stuff that much
webpack ended up doing something similar to what I was asking about, where they pulled out esm code into one giant scope, but bail on a number of circumstances
So imagine this case then: - I have cljs code - I have some es2015 libs that sometimes call each other, but never call non-es2015 libs - I have non-es0215 libs that call each other, but not es2015 libs would this case work by having the es2015 libs in advanced mode?
I know this isn't something that would just generally work, I'm mostly thinking of a case where I am very familiar with my dependencies
and if I switched it a bit to the point where the es2015 libs called non-es2015 libs, but not the other way around?
couldn't those get figured out by shadowcljs the same way it does when cljs calls a compat-mode npm lib?
JS is just handed to the GCC to do its thing. it would need to do the same kind of processing that is done for CLJS
yeah that makes sense
it looks for module.exports
(and variants thereof) and collects the exported names as externs
but in practice that is actually not a good idea and doesn't work when the JS code isn't processed by shadow-cljs (eg. react-native, node-script, node-library, ..)
I didn't actually understand that... I thought "extern inference" meant "looking through js code to try and figure out externs" but now I understand it means "looking through CLJS to figure out npm lib externs"
it looks at the code and tries to figure out if something you accessed will be coming from npm
that's why it would warn https://shadow-cljs.github.io/docs/UsersGuide.html#infer-externs
if I wanted to take a stab at doing that three tier approach, would you be open to accept contributions?
yes I'd provide involved test cases
it seems like an interesting entry point in contributing to shadow-cljs
like something that people would actually use. not some hypothetical thing that doesn't actually exist in the wild
you can fake it with zero code changes and just a few symlinks today (or file copies)
yeah a real example would also give a good indicator of the benefit
I imagine the benefit is that the es2015 libs get a dramatic cut in size
but maybe that ends up not being true
oh I think I have an interesting example
yes I don't have any illusions that code on npm will always find way to break
npm was never really built with forwards compat in mind
so imagine I have a bunch of non-es-2015 libs
and then I bundle them via rollup
kinda of how the cljs recommended they were bundled via webpack instead
but this time the output is actually a giant es2015 module
(that probably is broken because of some weird lib)
if it's not broken though, GCC takes over and might do some good because it got a lot of es2015 to work with
and since it's a giant es2015 module, gcc can do codemotion
no, you can config for commonjs compat
actually let me check that this produces es2015...
sorry this is the right one https://github.com/rollup/rollup-plugin-commonjs
compat isn't as good as webpack, but it exists
this of course is subject to all the pitfalls in random code we discussed
in angular cli we're actually experimenting with doing rollup inside webpack... because webpack has a lot of trouble making code in lazy chunks tree shakeable
so we process all the es2015 code with rollup, then pass on that output (including lazy modules) to webpack
with webpack the problem is this: - if you have a single chunk, maybe webpack will do module concatenation and everything is in a single scope, then Terser can do tree-shaking - if you have multiple chunks module then concatenation breaks down, and the webpack module loader indirection stops Terser tree-shaking
rollup, similar to GCC, performs tree-shaking with bundling information
so those are the only two JS tools today that can actually handle tree shaking across chunks/modules
they probably could but are trying to do build caching as their main focus atm I think
which is why I think shadow-cljs is in a really good place to take advantage of es2015 libs
contrary to most of the js ecosystem, the clj ecosystem is not averse to non-js toolchains like java
the pure js GCC implementation is pretty much unusable, so realistically GCC needs java
anyway I'll try to setup a few realistic test cases, then some mock cases that explore it a bit more, then try a implementation
happy to guide you through it. it will mostly be about changing the resource type from :shadow-js
to just :js
😛
cool, thank you for the pointer
I'll look there after the test cases