This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-02-20
Channels
- # announcements (1)
- # beginners (65)
- # calva (16)
- # cider (44)
- # clara (16)
- # clojure (84)
- # clojure-dev (48)
- # clojure-europe (5)
- # clojure-finland (4)
- # clojure-houston (1)
- # clojure-italy (19)
- # clojure-nl (27)
- # clojure-russia (6)
- # clojure-spec (37)
- # clojure-uk (123)
- # clojured (11)
- # clojurescript (21)
- # datomic (40)
- # duct (4)
- # emacs (6)
- # figwheel (4)
- # figwheel-main (5)
- # fulcro (34)
- # jackdaw (8)
- # juxt (117)
- # kaocha (3)
- # klipse (1)
- # leiningen (33)
- # luminus (2)
- # nyc (3)
- # off-topic (29)
- # om (1)
- # pedestal (7)
- # planck (4)
- # re-frame (27)
- # reagent (8)
- # reitit (5)
- # rum (2)
- # shadow-cljs (428)
- # spacemacs (5)
- # tools-deps (15)
- # yada (6)
is there a way to force shadow-cljs to output a node script that packages all code and dependencies into a single file? it looks like the default behavior requires me to npm install
where i end up running node, ideally i only send a single self-contained file, much like an uberjar
hi everyone. are release artifacts supposed to look for things in /workdir/.shadow-cljs/builds/{{project}}/dev/out/cljs-runtime/
?
@levitanong no, that is not part of a release
build. note the dev
in the path.
@currentoor that is currently experimental and still has a bunch of issues that need to be resolved https://github.com/thheller/shadow-cljs/issues/290
Everything I ever want from shadow-cljs is either there or already in the process of getting there. You’re awesome!
I need testers for that feature. you can test it today on your builds and let me know about packages that break or don't compile
@thheller so this is weird. I’m building a node script, ran a release on it, and when i ran it in docker, it gave me a bunch of errors about not finding '/workdir/.shadow-cljs/builds/image-gen/dev/out/cljs-runtime/goog.debug.error.js'
won’t a release build simply overwrite a dev build artifact if they have the same name?
And i don’t think i’m doing anything weird like renaming things… so I don’t see how i could be using the wrong file
a dev build can be identified pretty easily it will start with something like
#!/usr/bin/env node
(function(){
var SHADOW_IMPORT_PATH = __dirname + '/../../.shadow-cljs/builds/script/dev/out/cljs-runtime';
ooh, that’s useful. thanks!
I'm trying to use shadow-cljs with https://github.com/juxt/kick.alpha and https://github.com/juxt/edge
The issue with the first one: it has a file called injector.cljs
that imports figwheel which I don't use and don't have in the classpath. I think shadow-cljs tries to compile every single cljs file on the classpath, and it fails on this one while building the :npm
build.
Is this really an issue (I haven't gotten the project to work yet, so can't check)? If so, what would be the correct way of fixing it? If not, can I remove the :npm
build from the shadow-cljs configuration data to remove the error message?
The issue with the second one is that it wants me to add all publicly available web resources to the classpath, including the ones that are generated in target
. There are some JS files there as well and because of that, shadow-cljs spams a huge number of warnings about "provide conflict", like provide conflict for #{pathetic.core} provided by pathetic/core.cljs and {"/cljs-runtime/cljs-runtime.pathetic.core.js" #{pathetic.core}}
.
I think it's possible to fix it by not adding target
to the classpath and altering request handlers so that they look there. But to be honest, I have no idea whether that would work with the release uberjar.
Another potential way to fix it would be to still have target
in the classpath for the server and not have it for shadow-cljs. The only issue here is that I have to write some code to create the correct set of aliases for :deps
in the shadow-cljs configuration data, but I think it could be done.
But is there another way to solve this that's perhaps more preferable?
@p-himik shadow-cljs only compiles files that are actually required. so injector.cljs
must be required from somewhere.
can't find an injector.cljs in the edge repo? ah its in the kick repo whatever that is 😛
@thheller I am trying to use compile-str
in bootstrapped clojurescript instead of eval-str
and it returns a null
for a string like
"(.log js/console \"HI\")"
Please do try and let me know what you get. I tried the same thing with lein based project and it works, so could it be due to some way shadow bootstraps the compiler?
@metacritical I don't even know what compile-str is so I can't say whats its supposed to return. if you paste the full code I can maybe tell you what is wrong
@thheller Well it is supposed to compile cljs to js
It is in the same namespace as cljs.js
I think it will be better if i give you the context of what i am trying to do. I am trying to write a electron app that has bootstrapped cljs so that it can eval config files at runtime. but i am unable to get the ambient environment functions when i eval it so i thought i will do a global eval against node ‘vm’ by translating the cljs code to js.
(ns demo.bootstrap-script
(:require
[cljs.js :as cljs]
[cljs.env :as env]
[shadow.cljs.bootstrap.node :as boot]))
(defn print-result [{:keys [error value] :as result}]
(prn [:result result]))
(def code "(.log js/console \"HI\")")
(defonce compile-state-ref (env/default-compiler-env))
(defn compile-it []
(cljs/compile-str
compile-state-ref
code
"[test]"
{:eval cljs/js-eval
:load (partial boot/load compile-state-ref)}
print-result))
(defn main [& args]
(boot/init compile-state-ref {} compile-it))
@thheller It's very strange - I can't find it anywhere. The relevant ns is indeed added to the list of preloads but only if I explicitly tell kick.alpha
to use figwheel. And there are no messages in the logs mentioning that if was used. Is there any way to find out what exactly has made shadow-cljs compile a particular file?
@p-himik shadow-cljs clj-repl
then (shadow/find-resources-using-ns 'that.thing.inector)
but if the edge
stuff modifies the classpath or so it may not find whatever is using it
there is this in the repo. not sure what this is about. https://github.com/juxt/kick.alpha/blob/master/src/juxt/kick/alpha/providers/shadow_cljs.clj.txt
I don't know what edge is about or what it needs so I'm absolutely clueless about whats going on
Thanks, I'll try that. The file that you linked is not used because the authors of kick don't use shadow-cljs. I just copied that file to my project and modified it a bit for my needs.
I won't resolve to such easy fixes. I need to know the truth! 🙂 I'll try to dig around for a bit.
Ah, gotcha. But I'm almost positive it's injected dynamically in some way. But I'll do that, sure. "Any of this" - you mean kick? Not sure to what directories you're referring.
Oooh, neat!
But here's the issue - the problem is only reproducible with the :npm
build that's injected automatically. And I don't see this build in the UI. Maybe I can try just copying it so it becomes an explicit build.
configure it like this
{:npm
{:target :npm-module
:entries [your.main.ns]
:output-dir "node_modules/shadow-cljs"}}
Wait a second, I don't use it anywhere. I have no idea what it's for even - I just copied your code from load-cljs-edn
.
It's a build system that should make it easy to automate stuff.
E.g. restart your whole build chain and server when you enter (reset)
in the REPL. It can also help in situations where you don't want to restart everything, e.g. you need to reload some components without dropping all active connections.
But I just started using it and I've never used anything like it before, so I can't really tell you anything really useful about it.
Well, I can add one thing.
In my previous project, I had to either specify some paths two times or to parse shadow-cljs.edn
within the main code to make the server know about all of the resources.
Here, it's launched using the single source of truth, config.edn
. It contains everything you want it to contain, and since it uses aero
, you can avoid duplication entirely.
I need to restart shadow-cljs if I e.g. change a path. In some cases I had to restart it when I added something - don't really recall the details but I think it had to do something with a macros and a new namespace.
But I can make it so that shadow-cljs
is not restarted if I don't want it to be restarted. And since it's all in the same JVM that's still running, it doesn't take that much time anyway.
Maybe their example will give you a better idea of what it can do: https://github.com/juxt/edge/blob/master/main/resources/config.edn
yeah shadow-cljs isn't really meant to be used that way. the assumption that there is a shadow-cljs.edn
and hardcoded and not optional (currently)
Hmm, I thought I could get away with just using (shadow.cljs.devtools.server/start! (dissoc config :builds))
and (shadow.cljs.devtools.api/watch (assoc build :build-id build-id))
while feeding it the config that was preprocessed the same way you do it.
eg. when you edit the config it will automatically trigger a recompile with the new config
Hmm. But using shadow-cljs.edn
again brings that problem of having either to copy some values or to parse shadow-cljs.edn
. I'll see how it works out with the common config - maybe that's "some degree" is well within my needs.
To be honest, I have never made a CLJ app release, so I'm yet to find that out.
:output-dir
needs to be specified for both shadow-cljs and for the server to serve the generated files.
True. But just having it being specified in multiple places makes things harder to reason about, especially when you work with other people and have different projects that use different approaches while still using shadow-cljs. And I don't like to rely on anyone's memory when it can be avoided, including my own. 🙂
I'm trying to understand the architecture. I suppose the main/deps.edn
is the primary source for deps
so why not put shadow-cljs.edn
right next to that main/deps.edn
. make it use :deps {:aliases [:dev]}
but if you move the config into the config.edn
thing I don't understand how you build a release build then?
Since I've never done that before, I'm not really sure what issues you have in mind.
In the edge
repo they have bin/onejar
:
#!/bin/sh
clojure -Sdeps '{:deps {pack/pack.alpha {:git/url "" :sha "0878b52b7718e4b348a500dce0b3edc5b555a4f1"}}}' -m mach.pack.alpha.one-jar "$@"
And in the documentation it says to release the app with:
$ ../bin/onejar -A:prod --args '-m edge.main' project.jar
I certainly hope that figwheel/scss/shadow-cljs aren't running in production servers 😛
lib.ig.yada/deps.edn
3: integrant {:mvn/version "0.6.3"}
phonebook-api/deps.edn
6: integrant/integrant {:mvn/version "0.7.0"}
And they work like local libraries. And dependency resolution works the same way as it does in Java in general.
ok, I don't have to like it. I can still help you figure out how to best integrate everything
Thank you. I'll keep banging against all the new walls here, and if my head starts to hurt I'll come back.
I think the original question was here: https://clojurians.slack.com/archives/C6N245JGG/p1550649905397600
There's a oneshot multi method for performing production asset builds, the code lives mostly in kick, but has a main in edge under lib.edge.kick iirc
I don't love the production build code yet. So I suspect it will go through some evolutions.
You're right about the reset, but we solve it by only restarting servers if the config has changed.
It used to be a var def'd in a namespace, but that was changed to integrate it with config.edn better.
I think we had some projects where shadow-cljs.edn was not expressive enough for us, and we wanted to use the API and aero tags because of that.
@thheller what would this do in practice? https://github.com/thheller/shadow-cljs/commit/e8ed39ad2081b34f219c74c9c893143c3ee794ea
I feel like upgrading shadow from 2.7.29 (?) to 2.8.2 may have broken some of our code
but I haven’t pin pointed exactly what
I did
I’m on 2.8.5 now I think
yeah we’re on 2.8.5 and seeing broken stuff
this is related to jQuery
so only in advanced
I haven’t exactly pinpointed what is broken, but I’m getting this error: TypeError: b.modal.modal is not a function
where b
should be a React component instance, modal
a ref
ref to js/$
and modal.modal
the Bootstrap modal function
bad naming but this.jqueryref.modal()
basically
me neither but the code hasn’t been touched since March 2018
no we upgraded React 2 weeks ago and this worked fine since
this is either (missing) externs or JS deps stuff
did anything change in shadow recently wrt externs?
k reverting the PR
maybe?
oh yeah we do 😄
(:require ["bootstrap"]
for jQuery plugin injection
so the commit I pointed out is the bad one?
somehow it jumped to sight
I’m gonna revert and upgrade to the next one once you fix this
thanks for the help
@anmonteiro release is building. give it a min or so. will be 2.8.6
oh kk
so, question about shadow.loader: we are heavily using it in a project to load modules as needed on our web pages. it sounds like 2.8 upgrades GCC which might impact this. anything I should be aware of out of the gate?
assuming you are using 2.8.6
. the previous releases were all bad due to a bad :language-out default
I tried my best to make it work again. it changed a whole bunch if the closure library but it works fine in all my builds
we're also currently troubleshooting an issue with IE 11 where a few certain modules just won't load. I'm heading into the office now to investigate 😛 one of the problems we're running into is that it's hard to know why a module might fail to load/run. any tips on how to get more insight into that?
e.g. another issue we had was, a module wouldn't load when cookies was off; turns out it was referring to js/localStorage
at the top level and throwing an exception, but we weren't getting any kind of indication that was happening. just the promise returned by shadow.loader never resolved
@thheller btw to make module loading work for us we had to monkey patch google closure
which is… fun
(.setSourceUrlInjection loader false)
(.setDebugMode loader true)
(set! (.-usingSourceUrlInjection_ loader) (fn [] false))
we need to force this google.loader.moduleLoader.prototype.usingSourceUrlInjection_
function to always return false
to force the module loader into debug mode
otherwise it uses XHR + eval and our content security policy doesn’t allow that
we want it to add script
tags to the DOM
to be clear what we disallow is eval
, not xhr
it was actually fun to debug (in hindsight), because our bundles were coming back with status 200 but there was a stupid runtime error from the loader
wonder why there is no built-in flag for this. I'd expect google to also disallow eval
yeah.. I wondered the same
the Closure Library documentation is also not the best
I don’t think many people use it outside Google + ClojureScript
hopefully fixed in Shadow?
and we can just keep doing what we were doing downstream? 🙂
var trustReason = goog.string.Const.from("generated by compiler");
?
I might have to touch that code
so weird. closure library is full of those "safe" constructs to ensure things are "trusted"
is setModuleUris
not there anymore?
we’re monkey patching that code too
to add the CDN prefix for our modules
then we need to be able to prefix the module URIs with the CDN path
we don’t know it at compile time
it’s a chicken-egg problem
we need to compile the CLJS for Buck to output the compiled hash
that’s how we guarantee we don’t do work if nothing changes in our CLJS
the hash stays the same
in fact, our server “tells” the HTML what the CDN asset path is
so I’m currently doing:
(->> (js/Object.keys old-uris)
(run! (fn [k]
(let [links (->> (gobj/get old-uris k)
(map (fn [path]
(build/static-url path)))
into-array)]
(gobj/set uris k links)))))
(gobj/set js/goog.global.shadow$modules "uris" uris)
(js/shadow.loader.mm.setModuleUris
(gobj/get goog.global.shadow$modules "uris"))
you can use :module-loader :no-inject
so it doesnt inject the shadow$modules
variable
having a way to not do this would be appreciated, but it’s not a strong need from our side
ah, interesting
the user guide only mentions :module-loader true
😛
what does “inject the loader data” mean?
inject the shadow$modules
var into the code (was shadow$loader
previously, renamed because of a weird renaming bug)
hrm so ideally I’d like something in the middle
like, a function to map over the URIs and prefix them 😛
but I might be asking too much
it’s OK to say no 😛
also, it might be that the deploy hasn’t fully completed, but I’m seeing the same runtime error with shadow 2.8.6
yeah I only know the prefix at runtime
in the JS thing
@thheller yeah 2.8.6 didn’t fix our thing
reverting to 2.7.29
same one
modal.modal
thing
agreed
this could be between 2.7.29 and 2.8.X
I don’t have too much insight into what the problem would be
hmm if you handle the caching and stuff it might be the new minimize-require
default
what’s that
caching is definitely sorted out
https://clojureverse.org/t/reducing-the-build-size-when-using-lots-of-js-dependencies/3742/2
k trying
trying that now
@thheller wait so, minimize-requires
is disabled in dev, and enabled in prod?
so would it also break in bad ways if I enabled it in dev?
to experiment
I mean, assuming that’s what’s breaking our bundle
would it be repro’ed in dev?
trying
shadow-cljs takes extra care to ensure everything is recompiled and invalidated correctly
but I'm unsure what all your buck stuff does so maybe it decides to not recompile something when it should?
in dev we don’t have buck
so it’d be shadow all the way
doesn't have any effect on builds and stuff. I think the only case that actually used it was removed a while ago
OK, good. I was starting to think I'm becoming blind. 🙂
So, an interesting thing. Right now kick
only supports figwheel. And both release build and debug watch are ran with explicitly specified :source-paths
. The reason is that the output dir inside target
is added to the classpath - I think just to make the server aware of the resources there during the runtime.
But when you specify {:deps true}
, shadow-cljs ignores :source-paths
.
I can fix it in my project by explicitly specifying a filtered list of aliases for :deps
(the directories inside target
are added to the classpath by dedicated aliases). But would be interesting to know if there's a better way.
Well if I use shadow-cljs dynamically then I can change anything. But not when {:deps true}
is set, yes.
Ohhh. So when I use shadow-cljs by using server/start!
and api/watch
, I'm reusing the classpath, and none of :deps
, :lein
, :source-paths
(and probably other keys) are in effect.
Damn. So I cannot add target
to the classpath at all. Hence, I must have an additional static files handler that looks specifically under target
.
The edge
project has the server set in such a way so that the one and only static files handlers looks for files only in the public
directory inside any of the classpath paths. resources
is on the classpath, so resources/public
is used for some regular static files. target/dev
or target/prod
is also on the classpath, so target/{dev,prod}/public
is used to serve some generated static files. At least, that's my understanding.
Yeah, it's added because the manual says to use -A:dev:build:dev/build
where :dev/build
is just {:extra-paths ["target/dev"]}
.
@thheller :minimize-requires false
still makes it break
I’m on 2.8.6
@thheller I’m bisecting locally but there might be a change this was introduced between 2.8.2 and 2.8.5
2.8.3 changed shadow$loader
to shadow$modules
so with your hack active that will break depending on which one you rewrite
we don’t have modules in production yet
I don’t know what’s happening
but our upgrade from 2.8.2 to 2.8.5 is what broke it 100%
https://github.com/thheller/shadow-cljs/commit/2d4d49d70fc03781763b631f1ecbf84023238588
don't set :language-out :ecmascript6
or higher as certain optmizations don't yet work with es6+
need :language-in
es6+ since closure library now contains es6 code (eg the new module loader stuff)
trying language-in
ok so it isn’t language-in either 😕
the only other commit that stands out to me is https://github.com/thheller/shadow-cljs/commit/e8f85964bebbcd830f81514b203dd47038815f4d
@thheller does this commit look fishy to you ^ ?
so state is being updated to just contain :virtual
?
I might be ways off, but it looks weird
all the other commits in the version range look fine to me
oh I get it
->>
got me
I’ll try 2.8.3, and 4
see which one is broken
we’re currently running 2.8.2 in prod and all is fine
2.8.5 and 2.8.6 are broken for us
https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/build/compiler.clj#L1220-L1225
what about this?
otherwise the closure lib files got converted twice which worked but produced annoying warnings
I’m not sure what that means
converted twice because Closure’s last pass would also convert them?
closure started including files that can't be loaded like the other closure files in dev mode
in release mode that would lead to compiling the files twice, once for the conversion and once for the optimizations
eg. this can't be loaded without processing first https://github.com/google/closure-library/blob/master/closure/goog/loader/activemodulemanager.js
we’re not using code splitting in these builds
I really don’t think Buck is the problem here
we’re only using it to know where targets should be output to
shadow is still in control
we’re literally calling (shadow/release* build-config {:verbose true})
trying 2.8.3 now
the feedback loop is a little slow for this
so 2.8.3 is broken
therefore there are only 2 possible commits that could have done this 😛
and we’re not using code splitting so I don’t think it could’ve been the loader
-> modules
one?
I don’t think I can produce a minimal example easily
(ns cljsjs.ajquery
(:require ["jquery" :as jquery]))
(js/goog.exportSymbol "$" jquery)
(js/goog.exportSymbol "jQuery" jquery)
this is our jquery shim
ok nice
that you can repro
I wonder if $
is being mangled so some thing?
@thheller wait so if you revert https://github.com/thheller/shadow-cljs/commit/e8f85964bebbcd830f81514b203dd47038815f4d
does it work then?
thanks
https://github.com/thheller/shadow-cljs/commit/e8ed39ad2081b34f219c74c9c893143c3ee794ea
I only disabled half of it .. the code that actually removed the require call was still being used
hahaha
thanks