Fork me on GitHub
#shadow-cljs
<
2019-02-20
>
currentoor02:02:16

is there a way to force shadow-cljs to output a node script that packages all code and dependencies into a single file? it looks like the default behavior requires me to npm install where i end up running node, ideally i only send a single self-contained file, much like an uberjar

levitanong06:02:48

hi everyone. are release artifacts supposed to look for things in /workdir/.shadow-cljs/builds/{{project}}/dev/out/cljs-runtime/?

thheller06:02:29

@levitanong no, that is not part of a release build. note the dev in the path.

thheller06:02:05

@currentoor that is currently experimental and still has a bunch of issues that need to be resolved https://github.com/thheller/shadow-cljs/issues/290

currentoor15:02:27

Everything I ever want from shadow-cljs is either there or already in the process of getting there. You’re awesome!

thheller15:02:20

I need testers for that feature. you can test it today on your builds and let me know about packages that break or don't compile

levitanong06:02:12

@thheller so this is weird. I’m building a node script, ran a release on it, and when i ran it in docker, it gave me a bunch of errors about not finding '/workdir/.shadow-cljs/builds/image-gen/dev/out/cljs-runtime/goog.debug.error.js'

thheller06:02:29

then you are not using the correct file

thheller06:02:44

a release build will definitely not contain any references like that

levitanong07:02:53

won’t a release build simply overwrite a dev build artifact if they have the same name?

thheller07:02:14

yes it would

levitanong07:02:44

And i don’t think i’m doing anything weird like renaming things… so I don’t see how i could be using the wrong file

thheller07:02:06

well run shadow-cljs release the-build and look at the output

thheller07:02:11

a dev build can be identified pretty easily it will start with something like

#!/usr/bin/env node
(function(){

var SHADOW_IMPORT_PATH = __dirname + '/../../.shadow-cljs/builds/script/dev/out/cljs-runtime';

thheller07:02:55

a release build will not contain that path and start directly with minified code

levitanong07:02:00

ooh, that’s useful. thanks!

p-himik08:02:05

I'm trying to use shadow-cljs with https://github.com/juxt/kick.alpha and https://github.com/juxt/edge The issue with the first one: it has a file called injector.cljs that imports figwheel which I don't use and don't have in the classpath. I think shadow-cljs tries to compile every single cljs file on the classpath, and it fails on this one while building the :npm build. Is this really an issue (I haven't gotten the project to work yet, so can't check)? If so, what would be the correct way of fixing it? If not, can I remove the :npm build from the shadow-cljs configuration data to remove the error message? The issue with the second one is that it wants me to add all publicly available web resources to the classpath, including the ones that are generated in target. There are some JS files there as well and because of that, shadow-cljs spams a huge number of warnings about "provide conflict", like provide conflict for #{pathetic.core} provided by pathetic/core.cljs and {"/cljs-runtime/cljs-runtime.pathetic.core.js" #{pathetic.core}}. I think it's possible to fix it by not adding target to the classpath and altering request handlers so that they look there. But to be honest, I have no idea whether that would work with the release uberjar. Another potential way to fix it would be to still have target in the classpath for the server and not have it for shadow-cljs. The only issue here is that I have to write some code to create the correct set of aliases for :deps in the shadow-cljs configuration data, but I think it could be done. But is there another way to solve this that's perhaps more preferable?

thheller08:02:20

@p-himik shadow-cljs only compiles files that are actually required. so injector.cljs must be required from somewhere.

thheller08:02:23

can't find an injector.cljs in the edge repo? ah its in the kick repo whatever that is 😛

metacritical09:02:25

@thheller I am trying to use compile-strin bootstrapped clojurescript instead of eval-strand it returns a nullfor a string like "(.log js/console \"HI\")"

metacritical09:02:39

Please do try and let me know what you get. I tried the same thing with lein based project and it works, so could it be due to some way shadow bootstraps the compiler?

thheller10:02:47

@metacritical I don't even know what compile-str is so I can't say whats its supposed to return. if you paste the full code I can maybe tell you what is wrong

thheller10:02:51

without any code I can't help

metacritical10:02:45

@thheller Well it is supposed to compile cljs to js

metacritical10:02:16

It is in the same namespace as cljs.js

metacritical10:02:56

I think it will be better if i give you the context of what i am trying to do. I am trying to write a electron app that has bootstrapped cljs so that it can eval config files at runtime. but i am unable to get the ambient environment functions when i eval it so i thought i will do a global eval against node ‘vm’ by translating the cljs code to js.

thheller10:02:15

can't say anything about that but I can compile things just fine

thheller10:02:22

(ns demo.bootstrap-script
  (:require
    [cljs.js :as cljs]
    [cljs.env :as env]
    [shadow.cljs.bootstrap.node :as boot]))

(defn print-result [{:keys [error value] :as result}]
  (prn [:result result]))

(def code "(.log js/console \"HI\")")

(defonce compile-state-ref (env/default-compiler-env))

(defn compile-it []
  (cljs/compile-str
    compile-state-ref
    code
    "[test]"
    {:eval cljs/js-eval
     :load (partial boot/load compile-state-ref)}
    print-result))

(defn main [& args]
  (boot/init compile-state-ref {} compile-it))

thheller10:02:31

outputs [:result {:value "console.log(\"HI\");\n"}]

p-himik12:02:50

@thheller It's very strange - I can't find it anywhere. The relevant ns is indeed added to the list of preloads but only if I explicitly tell kick.alpha to use figwheel. And there are no messages in the logs mentioning that if was used. Is there any way to find out what exactly has made shadow-cljs compile a particular file?

thheller13:02:58

@p-himik shadow-cljs clj-repl then (shadow/find-resources-using-ns 'that.thing.inector)

thheller13:02:38

but if the edge stuff modifies the classpath or so it may not find whatever is using it

thheller13:02:13

I don't know what edge is about or what it needs so I'm absolutely clueless about whats going on

p-himik14:02:15

Thanks, I'll try that. The file that you linked is not used because the authors of kick don't use shadow-cljs. I just copied that file to my project and modified it a bit for my needs.

p-himik14:02:36

Yeah, of course that call returns just #{}.

thheller14:02:04

you can just replace that file with one that just defines the ns and nothing else

p-himik14:02:39

I won't resolve to such easy fixes. I need to know the truth! 🙂 I'll try to dig around for a bit.

thheller14:02:40

well if you do that it will be much easier to find out why it is included

thheller14:02:23

but can you explain a little bit what any of this does?

thheller14:02:00

why are there a billion directories with only one file each?

p-himik14:02:11

Ah, gotcha. But I'm almost positive it's injected dynamically in some way. But I'll do that, sure. "Any of this" - you mean kick? Not sure to what directories you're referring.

p-himik14:02:06

Still #{}.

thheller14:02:28

if you have a build running that has the empty version injected

thheller14:02:44

navigate to the build and recompile it

thheller14:02:50

there is a "Build Namespaces" at the bottom

thheller14:02:02

select the injector and it will tell you how it got included

p-himik14:02:28

Oooh, neat! But here's the issue - the problem is only reproducible with the :npm build that's injected automatically. And I don't see this build in the UI. Maybe I can try just copying it so it becomes an explicit build.

p-himik14:02:42

For my :main build, there's no injector in the list of namespaces.

thheller14:02:30

ah ... I missed the part where you are using the built-in default :npm build

thheller14:02:38

that indeed does compile everything unless you configure it

thheller14:02:49

so don't use that 🙂

thheller14:02:41

configure it like this

{:npm
 {:target :npm-module
  :entries [your.main.ns]
  :output-dir "node_modules/shadow-cljs"}}

p-himik14:02:55

Wait a second, I don't use it anywhere. I have no idea what it's for even - I just copied your code from load-cljs-edn.

thheller14:02:58

that will only include the :entries and their deps

thheller15:02:43

ah now I get it

thheller15:02:02

you are just loading the config and start every build defined in it

thheller15:02:05

yeah don't do that 🙂

p-himik15:02:31

Ah, I see! In hindsight, that was kinda obvious and stupid of me. 🙂 Thanks!

thheller15:02:22

what is this kick stuff even for?

p-himik15:02:48

It's a build system that should make it easy to automate stuff. E.g. restart your whole build chain and server when you enter (reset) in the REPL. It can also help in situations where you don't want to restart everything, e.g. you need to reload some components without dropping all active connections. But I just started using it and I've never used anything like it before, so I can't really tell you anything really useful about it.

thheller15:02:24

but why would you want to restart shadow-cljs when you restart your server?

thheller15:02:46

they shouldn't be coupled in the first place?

p-himik15:02:35

Well, I can add one thing. In my previous project, I had to either specify some paths two times or to parse shadow-cljs.edn within the main code to make the server know about all of the resources. Here, it's launched using the single source of truth, config.edn. It contains everything you want it to contain, and since it uses aero, you can avoid duplication entirely. I need to restart shadow-cljs if I e.g. change a path. In some cases I had to restart it when I added something - don't really recall the details but I think it had to do something with a macros and a new namespace. But I can make it so that shadow-cljs is not restarted if I don't want it to be restarted. And since it's all in the same JVM that's still running, it doesn't take that much time anyway.

p-himik15:02:42

Maybe their example will give you a better idea of what it can do: https://github.com/juxt/edge/blob/master/main/resources/config.edn

thheller15:02:31

yeah shadow-cljs isn't really meant to be used that way. the assumption that there is a shadow-cljs.edn and hardcoded and not optional (currently)

thheller15:02:44

so your builds need to be defined in there

thheller15:02:52

everything else is fine

thheller15:02:14

so basically you'll need a shadow-cljs.edn with the builds define like normal

thheller15:02:16

:kick/shadow-cljs
{:build-to-start [:foo :bar]}

thheller15:02:33

you then start those builds in the provider thing

thheller15:02:37

you could define the builds "outside"

thheller15:02:47

but currently there are several things that are tied to the config

p-himik15:02:54

Hmm, I thought I could get away with just using (shadow.cljs.devtools.server/start! (dissoc config :builds)) and (shadow.cljs.devtools.api/watch (assoc build :build-id build-id)) while feeding it the config that was preprocessed the same way you do it.

thheller15:02:03

eg. when you edit the config it will automatically trigger a recompile with the new config

thheller15:02:36

you can get away with it to some degree yes

thheller15:02:04

just absolutely not recommended and I make no guarantees that it will work reliably

thheller15:02:15

also how do you do a release build in that setup?

p-himik15:02:18

Hmm. But using shadow-cljs.edn again brings that problem of having either to copy some values or to parse shadow-cljs.edn. I'll see how it works out with the common config - maybe that's "some degree" is well within my needs. To be honest, I have never made a CLJ app release, so I'm yet to find that out.

thheller15:02:31

copy which values?

p-himik15:02:47

:output-dir needs to be specified for both shadow-cljs and for the server to serve the generated files.

p-himik15:02:00

Same with :asset-path, I think.

thheller15:02:26

how often do you change the :output-dir? I haven't changed mine in 4 years 😛

p-himik15:02:20

True. But just having it being specified in multiple places makes things harder to reason about, especially when you work with other people and have different projects that use different approaches while still using shadow-cljs. And I don't like to rely on anyone's memory when it can be avoided, including my own. 🙂

thheller15:02:16

I'm trying to understand the architecture. I suppose the main/deps.edn is the primary source for deps

thheller15:02:17

so why not put shadow-cljs.edn right next to that main/deps.edn. make it use :deps {:aliases [:dev]}

thheller15:02:23

and run them separately

thheller15:02:41

or if you really prefer you can run the embedded version just fine

thheller15:02:10

but if you move the config into the config.edn thing I don't understand how you build a release build then?

thheller15:02:53

how is this done for the figwheel sutff?

thheller15:02:24

I can't figure out where the release build happens in the repo?

p-himik15:02:35

Since I've never done that before, I'm not really sure what issues you have in mind. In the edge repo they have bin/onejar:

#!/bin/sh

clojure -Sdeps '{:deps {pack/pack.alpha {:git/url "" :sha "0878b52b7718e4b348a500dce0b3edc5b555a4f1"}}}' -m mach.pack.alpha.one-jar "$@"
And in the documentation it says to release the app with:
$ ../bin/onejar -A:prod --args '-m edge.main' project.jar

thheller15:02:00

that seems to be the server side stuff

thheller15:02:14

how is the CLJS compiled in :advanced?

thheller15:02:24

I can see the config but don't quite get how it is built

thheller15:02:13

I certainly hope that figwheel/scss/shadow-cljs aren't running in production servers 😛

p-himik15:02:54

That's not my plan, at least. 🙂 Let me see...

thheller15:02:11

so this entire edge architecture commits several sins in my view

thheller15:02:53

how do you even deal with dependency conflicts?

thheller15:02:33

lib.ig.yada/deps.edn
3:        integrant {:mvn/version "0.6.3"}
phonebook-api/deps.edn
6:  integrant/integrant {:mvn/version "0.7.0"}

thheller15:02:03

how does that not get even more out of date the longer the project lives? 😛

thheller15:02:35

why are they spread all over the place 😛

thheller15:02:50

how do you ever audit all deps ...

p-himik15:02:57

These pieces are not really parts of the main projects. They're local libraries.

thheller15:02:15

edge.asciidoctor/deps.edn
4:  integrant/integrant {:mvn/version "0.8.0-alpha2"}

p-himik15:02:23

And they work like local libraries. And dependency resolution works the same way as it does in Java in general.

thheller15:02:37

ok, I don't have to like it. I can still help you figure out how to best integrate everything

p-himik15:02:09

Thank you. I'll keep banging against all the new walls here, and if my head starts to hurt I'll come back.

thheller15:02:32

let me know if you find where the release build for CLJS is produced

thheller15:02:38

so far I can't find it

thheller15:02:41

or the scss for that matter

lilactown15:02:49

@dominicm might be able to provide some help fyi

thheller15:02:38

I think I figured it out

dominicm15:02:24

Haven't read up, but ask away

dominicm15:02:39

The dependency problem is real, but no worse than when you have libraries.

thheller15:02:03

yeah I couldn't figure out how the release build is produced for the CLJS code

thheller15:02:20

but I think its the build alias and the kick oneshot! thing

dominicm15:02:40

There's a oneshot multi method for performing production asset builds, the code lives mostly in kick, but has a main in edge under lib.edge.kick iirc

dominicm16:02:37

I don't love the production build code yet. So I suspect it will go through some evolutions.

dominicm16:02:04

You're right about the reset, but we solve it by only restarting servers if the config has changed.

dominicm16:02:04

It used to be a var def'd in a namespace, but that was changed to integrate it with config.edn better.

dominicm16:02:18

I think we had some projects where shadow-cljs.edn was not expressive enough for us, and we wanted to use the API and aero tags because of that.

thheller16:02:05

suppose you have a (:require ["react" :as r]) in your code but don't use it

thheller16:02:15

or only use it on a function that itself was not used

anmonteiro16:02:20

I feel like upgrading shadow from 2.7.29 (?) to 2.8.2 may have broken some of our code

thheller16:02:27

previously react would still be included in the build

anmonteiro16:02:29

but I haven’t pin pointed exactly what

thheller16:02:33

upgrade to 2.8.5

thheller16:02:40

that should fix most of the funkiness

anmonteiro16:02:53

I’m on 2.8.5 now I think

thheller16:02:21

do you use lots of the closure lib stuff?

anmonteiro16:02:24

yeah we’re on 2.8.5 and seeing broken stuff

anmonteiro16:02:32

this is related to jQuery

thheller16:02:35

broken how?

anmonteiro16:02:44

so only in advanced

anmonteiro16:02:08

I haven’t exactly pinpointed what is broken, but I’m getting this error: TypeError: b.modal.modal is not a function

thheller16:02:23

.modal.modal?

anmonteiro16:02:25

where b should be a React component instance, modal a ref

anmonteiro16:02:39

and modal.modal the Bootstrap modal function

anmonteiro16:02:48

bad naming but this.jqueryref.modal()

thheller16:02:06

ah hmm not sure why that would be different?

anmonteiro16:02:18

me neither but the code hasn’t been touched since March 2018

thheller16:02:24

did you accidentally also update react when upgrading shadow-cljs?

anmonteiro16:02:41

no we upgraded React 2 weeks ago and this worked fine since

thheller16:02:41

given all the version range crap defaults in node and stuff?

anmonteiro16:02:11

this is either (missing) externs or JS deps stuff

anmonteiro16:02:29

did anything change in shadow recently wrt externs?

anmonteiro16:02:00

k reverting the PR

thheller16:02:00

do you maybe have some JS requires for only the side effects?

anmonteiro16:02:20

oh yeah we do 😄

anmonteiro16:02:28

(:require ["bootstrap"]

thheller16:02:33

hehe oops yeah they will be removed now

anmonteiro16:02:35

for jQuery plugin injection

thheller16:02:37

doh didn't think about that case

anmonteiro16:02:32

so the commit I pointed out is the bad one?

anmonteiro16:02:42

somehow it jumped to sight

anmonteiro16:02:01

I’m gonna revert and upgrade to the next one once you fix this

anmonteiro16:02:04

thanks for the help

thheller16:02:45

@anmonteiro release is building. give it a min or so. will be 2.8.6

thheller16:02:09

ok should be available. sorry about that.

thheller16:02:52

the idea sounded good in my head and worked nicely for the tests I did

thheller16:02:12

just forgot that you may want things only for their side effects 😛

lilactown16:02:59

stupid side effects!

lilactown16:02:50

so, question about shadow.loader: we are heavily using it in a project to load modules as needed on our web pages. it sounds like 2.8 upgrades GCC which might impact this. anything I should be aware of out of the gate?

thheller16:02:09

no it should work as before

thheller16:02:53

assuming you are using 2.8.6. the previous releases were all bad due to a bad :language-out default

lilactown16:02:04

ok. good to know

thheller16:02:37

I tried my best to make it work again. it changed a whole bunch if the closure library but it works fine in all my builds

thheller16:02:46

so it should be fine

lilactown16:02:19

we're also currently troubleshooting an issue with IE 11 where a few certain modules just won't load. I'm heading into the office now to investigate 😛 one of the problems we're running into is that it's hard to know why a module might fail to load/run. any tips on how to get more insight into that?

lilactown16:02:27

e.g. another issue we had was, a module wouldn't load when cookies was off; turns out it was referring to js/localStorage at the top level and throwing an exception, but we weren't getting any kind of indication that was happening. just the promise returned by shadow.loader never resolved

thheller16:02:30

given the awful state of the IE11 devtools not really

thheller16:02:00

well do you have an error handler attached?

thheller16:02:41

(-> (loader/load "extra")
    (.then fn-to-call-on-load fn-to-call-on-error))

thheller16:02:48

so the second fn is also never triggered?

lilactown16:02:58

i'll verify but that's the behavior we were seeing

thheller16:02:36

IE11 also doesn't like console IIRC

thheller16:02:51

only when the devtools are open

lilactown16:02:55

yeah I'm exploring that too.

thheller16:02:18

I would expect the loader to properly propagate errors

thheller16:02:21

but who knows

lilactown16:02:23

I almost gave up on web dev the first time I had to track down a bug due to that 😛

anmonteiro16:02:48

@thheller btw to make module loading work for us we had to monkey patch google closure

anmonteiro16:02:57

which is… fun

lilactown16:02:26

what did you end up patching?

anmonteiro16:02:29

(.setSourceUrlInjection loader false)
         (.setDebugMode loader true)
         (set! (.-usingSourceUrlInjection_ loader) (fn [] false))

anmonteiro16:02:53

we need to force this google.loader.moduleLoader.prototype.usingSourceUrlInjection_ function to always return false

anmonteiro16:02:06

to force the module loader into debug mode

thheller16:02:09

why is that a problem?

anmonteiro16:02:19

otherwise it uses XHR + eval and our content security policy doesn’t allow that

anmonteiro16:02:32

we want it to add script tags to the DOM

thheller16:02:02

interesting

anmonteiro16:02:08

to be clear what we disallow is eval, not xhr

anmonteiro16:02:56

it was actually fun to debug (in hindsight), because our bundles were coming back with status 200 but there was a stupid runtime error from the loader

thheller16:02:00

wonder why there is no built-in flag for this. I'd expect google to also disallow eval

anmonteiro16:02:09

yeah.. I wondered the same

anmonteiro16:02:25

the Closure Library documentation is also not the best

anmonteiro16:02:40

I don’t think many people use it outside Google + ClojureScript

thheller16:02:45

yeah the module loader also now requires a method call that wasn't there previously

thheller16:02:53

not documented at all. fun times.

anmonteiro16:02:02

hopefully fixed in Shadow?

anmonteiro16:02:14

and we can just keep doing what we were doing downstream? 🙂

thheller16:02:15

yeah it injects the needed code

anmonteiro16:02:45

var trustReason = goog.string.Const.from("generated by compiler"); ?

thheller16:02:08

yeah that was a fun one too

anmonteiro16:02:27

I might have to touch that code

thheller16:02:44

so weird. closure library is full of those "safe" constructs to ensure things are "trusted"

thheller16:02:55

I really wonder how google injects the module loader config

anmonteiro16:02:02

is setModuleUris not there anymore?

thheller16:02:08

nope its gone

anmonteiro16:02:31

we’re monkey patching that code too

thheller16:02:33

setModuleTrustedUris now

anmonteiro16:02:42

to add the CDN prefix for our modules

thheller16:02:07

don't monkey patch stuff. just tell me that you need a setting for something 😉

thheller16:02:15

but :asset-path takes care of that no?

anmonteiro16:02:41

then we need to be able to prefix the module URIs with the CDN path

anmonteiro16:02:44

we don’t know it at compile time

anmonteiro16:02:47

it’s a chicken-egg problem

anmonteiro16:02:58

we need to compile the CLJS for Buck to output the compiled hash

anmonteiro16:02:21

that’s how we guarantee we don’t do work if nothing changes in our CLJS

anmonteiro16:02:24

the hash stays the same

anmonteiro16:02:49

in fact, our server “tells” the HTML what the CDN asset path is

anmonteiro16:02:09

so I’m currently doing:

(->> (js/Object.keys old-uris)
              (run! (fn [k]
                      (let [links (->> (gobj/get old-uris k)
                                       (map (fn [path]
                                              (build/static-url path)))
                                       into-array)]
                        (gobj/set uris k links)))))
         (gobj/set js/goog.global.shadow$modules "uris" uris)
         (js/shadow.loader.mm.setModuleUris
          (gobj/get goog.global.shadow$modules "uris"))

thheller16:02:27

you can use :module-loader :no-inject so it doesnt inject the shadow$modules variable

thheller16:02:35

which you can just set via HTML instead for example

anmonteiro16:02:36

having a way to not do this would be appreciated, but it’s not a strong need from our side

anmonteiro16:02:49

ah, interesting

anmonteiro16:02:14

the user guide only mentions :module-loader true 😛

thheller16:02:36

yeah its well hidden 😉

thheller16:02:09

basically the build output generates the module-loader.edn file (or .json)

thheller16:02:24

you can use that data. rewrite it and emit in into html

thheller16:02:45

<script>var shadow$modules = <the-json>;</script>

thheller16:02:56

somewhere before the JS include

thheller16:02:56

I've done this in the past but stopped caring about busting caches along the way

thheller16:02:45

but yeah there had to be a different way of getting that info there

thheller16:02:57

since adding the hashed urls to the file changes the hash of that file 😉

anmonteiro16:02:26

what does “inject the loader data” mean?

thheller16:02:42

inject the shadow$modules var into the code (was shadow$loader previously, renamed because of a weird renaming bug)

anmonteiro16:02:51

hrm so ideally I’d like something in the middle

anmonteiro16:02:14

like, a function to map over the URIs and prefix them 😛

anmonteiro16:02:29

but I might be asking too much

anmonteiro16:02:34

it’s OK to say no 😛

thheller16:02:50

so you only know the prefix at runtime not at build time?

anmonteiro16:02:55

also, it might be that the deploy hasn’t fully completed, but I’m seeing the same runtime error with shadow 2.8.6

anmonteiro16:02:01

yeah I only know the prefix at runtime

anmonteiro16:02:18

in the JS thing

anmonteiro17:02:35

@thheller yeah 2.8.6 didn’t fix our thing

anmonteiro17:02:40

reverting to 2.7.29

thheller17:02:46

what is the problem?

anmonteiro17:02:46

modal.modal thing

thheller17:02:09

the code removal is definitely disabled

anmonteiro17:02:43

this could be between 2.7.29 and 2.8.X

anmonteiro17:02:23

I don’t have too much insight into what the problem would be

thheller17:02:49

hmm if you handle the caching and stuff it might be the new minimize-require default

thheller17:02:34

that will definitely break if you use files from different builds together

anmonteiro17:02:41

what’s that

anmonteiro17:02:47

caching is definitely sorted out

thheller17:02:08

I enabled that by default

thheller17:02:40

:js-options {:minimize-require false} to disable

anmonteiro17:02:19

trying that now

anmonteiro17:02:30

@thheller wait so, minimize-requires is disabled in dev, and enabled in prod?

thheller17:02:12

no need to optimize for size in dev builds

anmonteiro17:02:19

so would it also break in bad ways if I enabled it in dev?

anmonteiro17:02:23

to experiment

thheller17:02:42

no its fine, just causes some additional recompiles if your JS deps change

anmonteiro17:02:00

I mean, assuming that’s what’s breaking our bundle

anmonteiro17:02:05

would it be repro’ed in dev?

thheller17:02:00

hmm it might?

thheller17:02:28

shadow-cljs takes extra care to ensure everything is recompiled and invalidated correctly

thheller17:02:53

but I'm unsure what all your buck stuff does so maybe it decides to not recompile something when it should?

thheller17:02:58

I don't know 😛

anmonteiro17:02:10

in dev we don’t have buck

anmonteiro17:02:16

so it’d be shadow all the way

thheller17:02:32

yeah then it will probably work

p-himik17:02:56

Is the --npm flag actually used? I can't find where it's handled.

thheller17:02:51

no it just tells the server that it was started by the npm script

thheller17:02:18

doesn't have any effect on builds and stuff. I think the only case that actually used it was removed a while ago

p-himik17:02:30

OK, good. I was starting to think I'm becoming blind. 🙂 So, an interesting thing. Right now kick only supports figwheel. And both release build and debug watch are ran with explicitly specified :source-paths. The reason is that the output dir inside target is added to the classpath - I think just to make the server aware of the resources there during the runtime. But when you specify {:deps true}, shadow-cljs ignores :source-paths. I can fix it in my project by explicitly specifying a filtered list of aliases for :deps (the directories inside target are added to the classpath by dedicated aliases). But would be interesting to know if there's a better way.

thheller17:02:47

uhm I don't follow

thheller17:02:58

you can't add :source-paths dynamically

p-himik17:02:39

Well if I use shadow-cljs dynamically then I can change anything. But not when {:deps true} is set, yes.

thheller17:02:46

I'm confused. you cannot change the :source-paths once the JVM is running

thheller17:02:07

figwheel isn't actually changing source paths, it is changing input paths

thheller17:02:09

different things

thheller17:02:47

:source-paths in figwheel are used as inputs to the CLJS compiler

thheller17:02:54

so the JVM doesn't know about them

thheller17:02:59

meaning they can't contain macros

thheller17:02:05

which I found profoundly annoying

thheller17:02:03

or I'm completely missing what you are talking about ... also possible.

p-himik17:02:20

Ohhh. So when I use shadow-cljs by using server/start! and api/watch, I'm reusing the classpath, and none of :deps, :lein, :source-paths (and probably other keys) are in effect.

thheller17:02:33

they only apply when shadow-cljs is started

thheller17:02:42

when you use the embedded mode they have no effect at all

thheller17:02:54

and just use the classpath you are on

p-himik17:02:28

Damn. So I cannot add target to the classpath at all. Hence, I must have an additional static files handler that looks specifically under target.

thheller17:02:21

who uses target? why do you need it on the classpath?

p-himik17:02:58

The edge project has the server set in such a way so that the one and only static files handlers looks for files only in the public directory inside any of the classpath paths. resources is on the classpath, so resources/public is used for some regular static files. target/dev or target/prod is also on the classpath, so target/{dev,prod}/public is used to serve some generated static files. At least, that's my understanding.

thheller18:02:12

ok but that sounds like it is already added to the classpath?

p-himik18:02:12

Yeah, it's added because the manual says to use -A:dev:build:dev/build where :dev/build is just {:extra-paths ["target/dev"]}.

anmonteiro17:02:45

@thheller :minimize-requires false still makes it break

thheller17:02:14

does your dev build work with 2.8+?

anmonteiro17:02:20

I’m on 2.8.6

anmonteiro18:02:48

@thheller I’m bisecting locally but there might be a change this was introduced between 2.8.2 and 2.8.5

thheller18:02:04

2.8.3 changed shadow$loader to shadow$modules so with your hack active that will break depending on which one you rewrite

anmonteiro18:02:25

we don’t have modules in production yet

thheller18:02:28

hmm so its just missing the boostrap stuff? thats weird

anmonteiro18:02:12

I don’t know what’s happening

anmonteiro18:02:24

but our upgrade from 2.8.2 to 2.8.5 is what broke it 100%

thheller18:02:51

wait I thought all 2.8 releases broke? you downgraded to 2.7.x?

thheller18:02:45

this may cause all sorts of weird issues

thheller18:02:08

maybe try :language-in :ecmascript6?

thheller18:02:28

don't set :language-out :ecmascript6 or higher as certain optmizations don't yet work with es6+

thheller18:02:35

could try less though

thheller18:02:01

need :language-in es6+ since closure library now contains es6 code (eg the new module loader stuff)

anmonteiro18:02:23

trying language-in

anmonteiro19:02:18

ok so it isn’t language-in either 😕

anmonteiro19:02:07

@thheller does this commit look fishy to you ^ ?

thheller19:02:31

looks like it does exactly what it should?

anmonteiro19:02:34

so state is being updated to just contain :virtual?

anmonteiro19:02:52

I might be ways off, but it looks weird

thheller19:02:06

virtual sources are generated by the compiler and should not be compiled by closure

anmonteiro19:02:08

all the other commits in the version range look fine to me

thheller19:02:42

its just copying the source to :output without compiling

thheller19:02:51

but that code isn't used in :release mode at all anyways

anmonteiro19:02:04

I’ll try 2.8.3, and 4

anmonteiro19:02:07

see which one is broken

anmonteiro19:02:17

we’re currently running 2.8.2 in prod and all is fine

anmonteiro19:02:24

2.8.5 and 2.8.6 are broken for us

anmonteiro19:02:29

what about this?

thheller20:02:50

:release doesn't call convert-goog only :dev does

thheller20:02:19

otherwise the closure lib files got converted twice which worked but produced annoying warnings

anmonteiro20:02:15

I’m not sure what that means

anmonteiro20:02:47

converted twice because Closure’s last pass would also convert them?

thheller20:02:34

closure started including files that can't be loaded like the other closure files in dev mode

thheller20:02:54

so they need to be converted in dev to something that can be loaded

thheller20:02:38

in release mode that would lead to compiling the files twice, once for the conversion and once for the optimizations

thheller20:02:50

but I don't see how this is related to your problem

thheller20:02:20

but there haven't been many commits since 2.8.2 so I don't know whats up

thheller20:02:38

did you try a release build without buck?

thheller20:02:46

just to rule out issues there?

anmonteiro20:02:43

we’re not using code splitting in these builds

anmonteiro20:02:04

I really don’t think Buck is the problem here

anmonteiro20:02:19

we’re only using it to know where targets should be output to

anmonteiro20:02:23

shadow is still in control

anmonteiro20:02:16

we’re literally calling (shadow/release* build-config {:verbose true})

thheller20:02:04

did you try 2.8.3/4 yet?

anmonteiro20:02:09

trying 2.8.3 now

anmonteiro20:02:19

the feedback loop is a little slow for this

anmonteiro20:02:13

so 2.8.3 is broken

anmonteiro20:02:44

therefore there are only 2 possible commits that could have done this 😛

anmonteiro20:02:29

and we’re not using code splitting so I don’t think it could’ve been the loader -> modules one?

thheller20:02:00

depends on whether your do anything with shadow$loader?

thheller20:02:57

can't see how that would possible affect a missing bootstrap .modal either

thheller20:02:00

and you are sure 2.8.2 is working and wasn't just a fluke?

thheller20:02:22

so you require "jquery" and "bootstrap"? maybe I can reproduce something on my end

anmonteiro20:02:15

I don’t think I can produce a minimal example easily

thheller20:02:10

did you try logging whatever b was in b.modal.modal?

thheller20:02:27

I can actually reproduce it

thheller20:02:37

just include jquery and bootstrap

anmonteiro20:02:41

(ns cljsjs.ajquery
  (:require ["jquery" :as jquery]))

(js/goog.exportSymbol "$" jquery)
(js/goog.exportSymbol "jQuery" jquery)

thheller20:02:42

in release mode the .modal is gone

anmonteiro20:02:52

this is our jquery shim

anmonteiro20:02:04

that you can repro

anmonteiro20:02:21

I wonder if $ is being mangled so some thing?

thheller20:02:28

["jquery" :as jq]
    ["bootstrap"]
    ))

(js/console.warn "modal" (.-modal (jq "#app")))

thheller20:02:39

undefined in release

anmonteiro20:02:48

does it work then?

thheller20:02:30

it does not

thheller20:02:38

I'll bisect a bit

thheller20:02:42

gah I'm an idiot

thheller20:02:02

no idea why 2.8.2 still worked for you. the first commit you linked is responsible

thheller20:02:22

I only disabled half of it .. the code that actually removed the require call was still being used

thheller20:02:44

should work again in 2.8.7

thheller20:02:52

thats what I get for working on 2 things at once 😛

anmonteiro21:02:33

@thheller confirmed fix in 2.8.7

👍 10