Fork me on GitHub
#shadow-cljs
<
2021-05-02
>
sh5406:05:47

I am getting an issue that when some required in javascript file changes, the final product shadow-cljs produces does not change.

sh5406:05:55

I have some fork of some javascript project that I am working on then “compiling” and copying over relevant js files into my clojure project. shadow-cljs is certainly noticing that this happens and starts compiling and hot reload triggers. However the actual change in functionality does not get picked up. If I reload the page there is still no change. But if I go to the shadow-cljs dashboard and press stop then watch then the change appears.

sh5406:05:06

Not sure if this is a general issue or something very specific here. What I am actually changing is a c++ file that gets compiled to js via emscripten. For this project the wasm binary is just base64 encoded and dumped into the js file. So when I am changing something the only change in the js file is that some def var wasmBinaryFile="data:application/octet-stream;base64,ABCD"; is getting altered. Just for context.

thheller07:05:36

@slack1003 hard to say without knowing how you actually include those files. why not load the actual .wasm file over the ajax or so? inlining it via base64 will make it needlessly large?

sh5407:05:04

I agree that it should be compiled to its own file and loaded. I am just on a fork. It is something I have thought of changing. There is a certain simplicity in just having a single js file include though

sh5407:05:17

I am requiring it in via (:require #?(:cljs ["imgui-js" :as imgui]))

thheller07:05:19

so its in node_modules/imgui-js? node_modules files are not hot-reloaded. to invalidate the cache it has you can touch node_modules/imgui-js/package.json which will then trigger a recompile

sh5407:05:34

so I have kept it out of the node_modules right now. in my own build it just copies it into a directory on the class path. then in my shadow-cljs config I do :js-options {:resolve {"imgui-js" {:target :file, :file "src/cljc/imgui/imgui.js"}}}

thheller07:05:04

do NOT use :resolve if you have stuff on the classpath anyways

sh5407:05:13

not sure if that is best practice by any means but seems easiest for development

sh5407:05:23

i’ll get that changed then

sh5407:05:57

thanks, i’ll see if that helps.

sh5407:05:13

ahh, I remember there was a particular reason for why i did that! There is a separate js file I include that I have done that style: #?(:cljs ["/imgui/imgui_demo.js" :as imgui-demo]) . And it tries to do require('imgui-js') to get in that same file. Since it looked like I needed that resolve bit to make imgui_demo.js happy i just used that in my cljc file too.

sh5407:05:59

can I leave the :resolve in the shadow cljs settings to keep the other include happy and move to use (:require ["/imgui/imgui.js" :as imgui]) in my cljc file then?

thheller07:05:20

sorry I don't have a clue what you are doing and would need to see some actual code

thheller07:05:49

otherwise there are far too many unknowns for me but it looks like you are trying to hack this together in the wrong way

thheller07:05:20

using :resolve for packages you already control is almost guaranteed to be wrong

sh5407:05:52

maybe… main issue is I don’t have “full” control. Just on a fork to add some functionality. and if I don’t have to change some of the build fundamentals in that project then I would like to leave them be and deal with things on the clojure side.

sh5407:05:32

sorry if my explanation here is lacking. I can get back with a better summary of how things are wired together

thheller07:05:04

wasm in general requires rather specific "glue" code that currently isn't very bundler friendly

sh5407:05:14

It all works fine outside of the hot reload.

thheller07:05:47

yeah, I'm gonna need to see some code to comment on that

sh5407:05:14

Yeah I have done my own wasm tests a while back and I remember having quite a few issues with the glue code then.

sh5408:05:57

k incoming essay: 🙂

sh5408:05:59

So I include https://github.com/flyover/imgui-js as a git submodule.

sh5408:05:02

I have a little build tooling that just runs its make file and copies the relevant compiled js files over somewhere into my class path.

sh5408:05:05

Here I guess I may be going wrong and that I should register it as a node module. Not sure how to do that but I’m sure I can figure it out if thats the way to go.

sh5408:05:08

Anyway, the project compiles down to two important files. imgui.js really is the thing that does everything. imgui_demo.js is several thousand lines of js that is recommended to include when developing since it both serves as documentation and a settings tweaker. Again all not really my code, though I can always change things up if necessary in the fork.

sh5408:05:12

It seems like it is the interplay between those two files that causes some issues with shadow-cljs. The top of imgui_demo.js has one of those standard check all sorts of places to figure out how to require imgui.js.

sh5408:05:18

// imgui_demo.js top
    (function (global, factory) {
        typeof exports === 'object' && typeof module !== 'undefined' ? factory(exports, require('imgui-js')) :
        typeof define === 'function' && define.amd ? define(['exports', 'imgui-js'], factory) :
        (global = typeof globalThis !== 'undefined' ? globalThis : global || self, factory(global.ImGui_Demo = {}, global.ImGui));
    }(this, (function (exports, ImGui) { 'use strict';
    ... );

sh5408:05:32

The important thing being the require('imgui-js') it seems?

sh5408:05:55

Right now I really only include them in my wrapper files.

sh5408:05:57

What works with the requires is when I do the following:

sh5408:05:03

(:require
      #?(:cljs ["imgui-js" :as imgui])
      ;;#?(:cljs ["/imgui/imgui.js" :as imgui]) ;; this does not work!
      #?(:cljs ["/imgui/imgui_demo.js" :as imgui-demo]))

sh5408:05:07

Then everything builds fine. But alterations to the js file do not get picked up and require a stop and watch. But hardly the end of the world.

sh5408:05:10

and in my shadow-cljs config include:

sh5408:05:14

:js-options {:resolve {"imgui-js" {:target :file :file "src/cljc/imgui/imgui.js"}}}

sh5408:05:19

Now I have tried with the following instead:

sh5408:05:25

(:require
      #?(:cljs ["/imgui/imgui.js" :as imgui])
      #?(:cljs ["/imgui/imgui_demo.js" :as imgui-demo]))

sh5408:05:28

Right now this results in imgui.js somehow being included twice! The imgui_demo.js file seems to get its own copy.

sh5408:05:30

I can see this by looking at my shared.js which shadow-cljs is compiling.

sh5408:05:33

In this setup it has the lines:

sh5408:05:37

SHADOW_ENV.evalLoad("module$imgui$imgui.js", true , ......
    SHADOW_ENV.evalLoad("module$src$cljc$imgui$imgui.js", true , ......

sh5408:05:41

so two copies of imgui.js which compiles fine but crashes at runtime.

sh5408:05:44

I guess I could try to keep them in my node_modules instead? Maybe it will give better results. Though it feels like it should be possible to handle them being files in the class path too.

thheller08:05:54

first of all forget about :resolve. as I said that this is NOT relevant here

thheller08:05:28

WHY did you add it in the first place? WHY do you think require('imgui-js') is a problem?

thheller08:05:28

again this is would be much much much much much easier if you just put the code into some repo so I can look at it together

sh5408:05:09

Without the :resolve bit then shadow cljs is not able to build anything: X Compilation failed.

The required JS dependency "imgui-js" is not available, it was required by "imgui/imgui_demo.js".

sh5408:05:09

How I have things required in is the only way that the project is building, and not throwing at runtime, and not double including imgui.js

sh5408:05:05

I can try to reproduce in a simpler repo.

thheller08:05:21

WHERE is imgui-js from? it is not the file you generate from the wasm right? did you modify that file in any way?

sh5408:05:05

as stated before the require('imgui-js') part is in the helper files like imgui_demo.js that are also a part of that project

thheller08:05:30

I asked where the file is from, not where it was included from

thheller08:05:58

src/cljc/imgui/imgui.js WHERE is this from? is this also wasm generated code?

sh5408:05:52

that is just copied from the build output of the project. Copied unmodified like all the rest of the build artifacts I am relying on

thheller08:05:08

ok, so you have two files in a directory?

thheller08:05:16

ok, my first suggestion would be to just modify the generated demo.js to replace require('imgui-js') with require('./imgui.js') instead

thheller08:05:33

since you are generating that file anyways that should be trivial to add

sh5408:05:54

yeah I can try that

thheller08:05:04

the goal of this is to make everything self-contained so it doesn't rely on build config and the code expressing its intent correctly

thheller08:05:04

right now require('imgui-js') is interpreted as "require the imgui-js npm library" but it is not an npm library. avoid resolve to fix this, much better to fix it in the source directly

sh5408:05:16

totally. I guess you have to deal with all sorts of different js files doing some funky things when it comes to dependencies sometimes 😕

thheller08:05:54

the problem really is :resolve to :file. I should probably remove that altogether since it is basically never the correct way to do anything

sh5408:05:43

good to know

sh5409:05:30

hey, so your suggesting on changing the require to ./imgui.js works nicely. I eliminated my use of :js-options :resolve too. just going to alter my tooling to alter the files then I can check if hot reload works

sh5409:05:43

thanks a bunch for your help and time

sh5409:05:17

I was going in with the idea of not altering the build artifacts and fudging things on the clojure side

thheller09:05:18

hot-reload was likely broken because of the :resolve

thheller09:05:56

can you send me the generated demo.js file? I'd be interested to see how that looks in full

sh5409:05:07

but it looks like the way to go is to provide files with the right refs

sh5409:05:07

the two files where imgui_demo.js has been patched

thheller09:05:47

hot-reload is probably going to be a problem. the code doesn't really allow it. but recompile and browser reload should be ok without :resolve

thheller09:05:13

the caching problem should be gone though

sh5409:05:34

yeah as long as the browser reload is all good then that is plenty good enough! I figured hot reload would be a problem given the important stuff happens in the wasm

thheller09:05:12

would be a little easier to accomplish without the wasm inlined into the source

sh5409:05:13

yeah the recompile is now working! thanks again for the help

👍 4
Sam Ritchie20:05:04

hey all.. I am having the toughest time converting to shadow-cljs from lein, for my CIDER repl workflow. I run cider-jack-in-cljs, select shadow-cljs then shadow as the repl type,

Sam Ritchie20:05:21

and everything boots, but then when I try to eval a form, I see

No available JS runtime.
See 

Sam Ritchie20:05:00

my goal is to get a node repl running

Sam Ritchie20:05:38

so… MAYBE I am supposed to run the generated file and it will get a repl going? except this is my test target, so it probably will not…

Sam Ritchie20:05:19

I must be missing something obvious here, for getting up a workflow that uses node to evaluate forms without needing a browser. sorry for the fumbling question… I’ve been at CLJS a long time and build stuff can still be mystifying

Sam Ritchie20:05:00

node-repl is saving me… before it was failing with an error I can’t find, but I may be in business!! apologies for the flailing, I think I’m good 🙂