FWIW we do have relatively slow hot reload using the webpack HMR in this configuration. it successfully loads the new code. so perhaps I should be disabling shadow-cljs hot reload and leaning on webpack, with the accept logic to keep it from refreshing the whole page.

(that's working. I'd be glad for it to be faster, though. how do JS devs work like this? 😜 )

in general its a bad idea to have two separate hot reload mechanisms as they will most probably inferere with each other

the error goog.provide is not a function usually means that something is being loading the wrong "context"

there should be a global goog, but depending on what webpack did it might no longer be global

do you set :runtime :browser in the build config?

https://github.com/metabase/metabase/blob/master/shadow-cljs.edn#L13 no

ah nvm thats the default. so no need to set it

config looks fine

did you consider using :target :esm instead?

where can I see the JS/TS code and webpack config?

well, it's working nicely so far as it goes. I'm trying to figure out how to get the React plugins to detect this and re-render. it's all library code, but if I change how eg. display-name works, any display names on screen don't re-render.

• https://github.com/metabase/metabase/blob/master/webpack.config.js • https://github.com/metabase/metabase/blob/master/frontend/src/metabase-lib/breakout.ts#L1 CLJS imports> and https://github.com/metabase/metabase/blob/master/frontend/src/metabase-lib/breakout.ts#L21 • https://github.com/metabase/metabase/blob/master/src/metabase/lib/js.cljs#L178

how hard is it to do a local build and getting this to the point where the error occurs?

it requires some changes I haven't pushed yet. with my branch published, it's yarn install; yarn dev and open http://localhost:3000. I don't think there's any active issues in scope for shadow-cljs though. it doesn't seem possible to get webpack's HMR to not reload the files again a few seconds later.

I think webpack has an ignore option for watch, that should block it from trying to hot reload

can't remember what that was called though

it does, but that makes it ignore the files completely, so they never get loaded with the initial build either

maybe a sufficiently precise regex could exclude everything except the entry point that never changes? I haven't dug into that since the shadow-cljs reloads were failing anyway.

is it possible for webpack/JS/TS to leave an import as it is instead of trying to bundle it?

like if you do import * as CLJS from "./foo.js" it is leasing ./foo.js as it and will let ESM load it? 😛

I have so many nightmares from looking at webpack output, that my first instinct is to always eliminate webpack from the equation

I'm not sure about that. probably not - webpack seems to assume it's got full control in the usual configurations.

at least so that it doesn't touch the CLJS code 😛

that is one big repo

that would be nice, but I don't think we have the option as it stands.

yarn has been going for a while now 😛

I think it's the largest open source Clojure repo

didnt' get to the clj(s) parts yet 😛

the cljs-repl branch has my changes now. if you want to see the error live, you'll need to set :autoload true in the shadow-cljs.edn and then once the app is loaded make an edit to something in src/metabase/lib, like adding a print statement to some function in src/metabase/lib/equality.cljc.

{"via":[{"type":"java.lang.Exception","message":"Failed to load template 'frontend_client/index.html'. Did you remember to build the Metabase frontend?","

you'll see it choke on goog.provide, then a moment later the webpack HMR will succeed.

when opening :3000

something else I need to run?

2023-09-20 19:38:35,425 ERROR routes.index :: Failed to load template 'frontend_client/index.html'. Did you remember to build the Metabase frontend?
[backend] java.lang.IllegalArgumentException: No implementation of method: :render of protocol: #'stencil.ast/ASTNode found for class: nil
[backend]       at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:584)
[backend]       at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:576)
[backend]       at stencil.ast$eval97201$fn__97202$G__97192__97211.invoke(ast.clj:19)

yarn dev should be everything, that's odd.

[backend] 2023-09-20 19:38:28,680 INFO util.fonts :: Reading available fonts from /mnt/c/Users/thheller/code/oss/metabase/resources/frontend_client/app/fonts
[backend] 2023-09-20 19:38:28,694 ERROR routes.index :: Failed to load template 'frontend_client/index.html'. Did you remember to build the Metabase frontend?
[backend] java.lang.IllegalArgumentException: No implementation of method: :render of protocol: #'stencil.ast/ASTNode found for class: nil
[backend]       at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:584)
[backend]       at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:576)
[backend]       at stencil.ast$eval97201$fn__97202$G__97192__97211.invoke(ast.clj:19)
[backend]       at stencil.core$render.invokeStatic(core.clj:80)
[backend]       at stencil.core$render.invoke(core.clj:74)
[backend]       at stencil.core$render_file.invokeStatic(core.clj:87)
[backend]       at stencil.core$render_file.invoke(core.clj:83)

the files exist AFAICT

https://github.com/metabase/metabase#quick-setup-dev-environment I think that should cover everything? yarn dev should run the BE + FE with hot reloading, but maybe we missed something. try bin/build.sh to make sure any one-time assets got created? maybe we broke that first-contact workflow on master.

seems to work on second run

weird, but okay.

[static-viz] Watchpack Error (stats): Error: EACCES: permission denied, lstat '/mnt/c/pagefile.sys'
[frontend] Watchpack Error (stats): Error: EACCES: permission denied, lstat '/mnt/c/pagefile.sys'
[frontend] Watchpack Error (stats): Error: EACCES: permission denied, lstat '/mnt/c/pagefile.sys'

why is this watching this? 😛

no idea, actually.

so I'm in your cljs-repl branch

shadow-cljs seems to be running

but the client does not connect?

not can I tell if actual CLJS is getting loaded?

ah nvm.

websocket.cljs:15 Refused to connect to '' because it violates the following Content Security Policy directive: "connect-src 'self'     *:8080 ws://*:8080 ws://*:9630".

I see the message about the client connecting in the browser console?

already had shadow-cljs running, so 9630 was taken

is there a CLJ repl I can connect to? the one where the CLJ stuff is running? or is that running in shadow-cljs?

I can connect to the shadow-cljs nrepl fine, but is that the backend one?

I don't like these commands where one command runs everything. can I run things separately?

too much terminal noise 😛

ok changed the port, now it seems to connect

(afk for 30ish)

I debugged this before ... I just can't remember what the fix was 😛

the double hot reload problem you can solve by outputting to node_modules/metabase-cljs (or pick a dir you like more)

in the JS parts then just from "metabase-cljs/whatever.js" instead of from "cljs/whatever.js". the extra webpack alias entry for this you can then remove

then in the webpack config search for watchOptions and change it to

watchOptions: {
        ignored: /node_modules/
    }

now if I make a change it still fails

but webpack reports [webpack-dev-server] Nothing changed. and does nothing

maybe that also works with the alias setup, tried the node_modules way first but that still webpack HMR'd

I'll try to remember what the fix was for the CLJS reload though

did you try the :target :esm thing? that might change the imports to things that work.

sorry this lead me down a rabbit hole of quirks. I'll see what I can figure out over the weekend probably

😨 I'm concerned that you're diving in really hard on a non-breaking issue. the current state (`:autoload false` and relying on webpack's HMR) is getting the job done - the CLJS code reloads, the REPL works and doesn't need constant reconnecting.

if this has exposed a weak spot you want to dig in and fix fully, that's great. but I don't want you to work hard on this with the goal of unblocking a use case, since we're not really blocked.

don't worry about it. not much related to your case, rather :npm-module in general

sounds good, then.

it's also worth saying out loud that I really appreciate the superb support you give to shadow-cljs users! Metabase has a rare use case - "a CLJS library consumed by JS code" is ✅ but "through webpack" is 😱 - and this isn't the first time I've asked a tricky question and got great answers. 👏

hmm, maybe this runs deeper than I thought. I can't seem to eval new things in some contexts? I keep getting errors like

TypeError: Cannot set properties of undefined (setting 'empty_env')
    at ../cljs/cljs.test.js (test.cljs:252:1)
    at options.factory (react refresh:6:1)
    at __webpack_require__ (bootstrap:24:1)
    at fn (hot module replacement:62:1)
    at ../cljs/metabase.lib.js.js (test.cljs:605:1)
    at options.factory (react refresh:6:1)
    at __webpack_require__ (bootstrap:24:1)
    at Object._requireSelf [as require] (hot module replacement:102:1)
    at Object.apply (jsonp chunk loading:444:1)
    at hot module replacement:344:1 

metabase.lib.equality_test.untyped_map_test = (function metabase$lib$equality_test$untyped_map_test() {

(maybe there's some missing init code when I do this, so cljs.test.test_vars and other namespaces don't exist in JS land?)

perhaps I'm "holding it wrong" here - I am trying to eval things and use deps like clojure.test that weren't transitively in the :entries! but with :npm-module as it stands it's hard to know how to have the tests in scope in :dev mode without having to list them all 😞 if you supply both :entries and :ns-regexp the latter is ignored.

I'm guessing this is the same issue as hot reload itself

the scope is screwed up and it is mixing things in different scopes

I swear I fixed this before

but can't remember or find it

but this is mostly due to :npm-module being so extremely hacky

I think I follow. I wonder if there's a way to tell webpack in dev mode to treat the (`^:export`ed) functions in the CLJS library as externs or similar, basically to say "these are globals just trust me". then the input and compiled output looks like this:

// foo.ts
import * as Lib from "cljs/some.namespace";

export function someFunction(x: SomeType): AnotherType {
  return Lib.some_function(x);
}


// app.bundle.js or whatever, the compiled output in dev mode
exports.someFunction = (x) => $CLJS.some.namespace.some_function(x);

problem is that you want webpack to see some of the output, so it can provide the npm dependencies

you could try https://shadow-cljs.github.io/docs/UsersGuide.html#js-provider-external

and then just use the exported vars by name

however that has the problem that this is designed so that the JS parts are inactive, as in used by the CLJS parts but not doing anything on its own

but your use case is the reverse of that as JS seems to be the primary driver

I guess you could setup some sort of coordination mechanism so that the JS only starts once CLJS is "ready"

basically the CLJS just exports global vars

and JS just uses the globals, without any imports

assuming TS can even do that without yelling at you 😛

I'll play around with it.

🎉 I managed to get this working, at least to the point of successfully loading the dev mode app. (120% certain I broke the prod build in the process, but one thing at a time.) here's how it works in dev mode now: • separate shadow-cljs target :dev, :target :browser and :js-provider :external (requiring that generated file from frontend/src/metabase/dev.js, which is dev mode only) • that outputs into target/cljs, ie. outside the frontend/src tree that webpack sees. • webpack has an import alias cljs_dev/foo.bar, so that import "cljs_dev/some.namespace"; really loads the output files (webpack understands this and bundles the files properly) • someone has to require those, so I copied the :entries list into the same dev.js as above, doing import "cljs_dev/each.namespace"; for all of them (that sucks and needs automating but it works for now; see below) • webpack has an externals: [function(...) { ... }] that spots the original "cljs/some.namespace" imports and transforms them into "var" style, which produces the same code as before

var cljs_metabase_lib_js__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(/*! cljs/metabase.lib.js */ "cljs/metabase.lib.js");

ah, I think I solved the staleness/no REPL issue. the yarn build-hot command launches the CLJS and webpack watchers simultaneously. in the old config, both builds start, shadow-cljs finishes, then webpack finishes, then immediately notices the changed CLJS output and builds again. manually editing a regular JS file caused that the reload and now the stale warning is gone and REPL is connected. that's easy enough to fix with separate commands for people who care about CLJS development.

I can't quite find a winning combo here. :target :browser produces a singular JS file and import "cljs_dev/main"; fails with browser bootstrap used in incorrect target because goog.global.document isn't defined. perhaps there's a more right target to use in this different configuration? :npm-module has the same issue with hot reloading that started the conversation, so I tried :browser but it doesn't like being embedded (probably because it's wrapped in a webpack IIFE.

can I see the setup somewhere? I don't quite understand what you are doing now? what is import "cljs_dev/main";? webpack should ONLY see the one :external-index file. it should not be aware of the other CLJS output in any way, i.e. no processing done whatsoever

basically you just add the external index to the webpack build

and let it build all other JS/TS code together with it

then the :browser CLJS build is independent

don't use the $CLJS var. that is entirely a development only thing that doesn't exist in release

any ^:export you have is available is the global scope under that name

so metabase.lib/whatever is just metabase.lib.whatever, or maybe it that makes typescript happier window.metabase.lib.whatever

note that you can also supply the full name from the CLJS, it does not need to be your exact namespace structure

so (defn ^{:export "myCLJS.foo"} hello-world [x] ...) will be available under myCLJS.foo, not whatever.namespace.hello_world

so if I follow correctly your only problem is importing the shadow-cljs :modules into the JS/TS build, instead of just the :external-index file

I juggled several possible setups yesterday; let me take another look now. import "cljs_dev/main"; is exactly requiring the single output file (`main.js`, since there's only one module) from somewhere early in the webpack world, so the file gets included. let me make sure I'm testing the setup you suggested and see how it goes. my memory says it failed on the goog.global.document check, but I'm only ~70% confident I'm remembering the right attempt.

Uncaught Error: browser bootstrap used in incorrect target
    at Object.<anonymous> (app-main.hot.bundle.js?1f108a291b8ab1aed5c6:557959:11)
    at ../../../target/cljs/main.js (app-main.hot.bundle.js?1f108a291b8ab1aed5c6:558100:3)
    at options.factory (runtime.hot.bundle.js?b6ce8fd471095ef10d4c:704:31)
    at __webpack_require__ (runtime.hot.bundle.js?b6ce8fd471095ef10d4c:35:33)
    at fn (runtime.hot.bundle.js?b6ce8fd471095ef10d4c:361:21)
    at ./dev.js (app-main.hot.bundle.js?1f108a291b8ab1aed5c6:510135:71)
    at options.factory (runtime.hot.bundle.js?b6ce8fd471095ef10d4c:704:31)
    at __webpack_require__ (runtime.hot.bundle.js?b6ce8fd471095ef10d4c:35:33)
    at fn (runtime.hot.bundle.js?b6ce8fd471095ef10d4c:361:21)
    at ./app-main.js (app-main.hot.bundle.js?1f108a291b8ab1aed5c6:503708:70)

ah, yeah. goog.global = this || self; but this is set as module.exports in webpack's module loading context, so it's {} not window. I'll see about adjusting how the main.js is loaded to be an unwrapped, global <script> tag.

okay I did that but the timing is busted. before the main.js finishes running, other imports are already failing because window.metabase isn't defined (yet). I'm poking around with the "promise" style of webpack externals to see if I can make it wait properly.

no dice. that broke things in other ways. not surprising; I think you'd have to do async loading "all the way down" for that to work.

but now I think I'm properly stuck. there's two incompatible timing constraints: • I need to load the :js-provider :external webpack bundled file first, so that shadow$bridge is defined before the main.js build loads. • but the main webpacked code is trying to consume main.js globals before it finishes loading in script style, so main.js would need to load first, like in a <head><script> tag.

my gut feeling now is that the way forward is to give :npm-module some love, so that it supports shadow-cljs hot reloading and the REPL, in a way that still plays nice with webpack. (and I'll rework our setup so the output isn't under frontend/src, so webpack's HMR isn't involved.)

Uncaught Error: browser bootstrap used in incorrect target
    at Object.<anonymous> (app-main.hot.bundle.js?1f108a291b8ab1aed5c6:557959:11)
    at ../../../target/cljs/main.js (app-main.hot.bundle.js?1f108a291b8ab1aed5c6:558100:3)

all this tells me is that you are loading the CLJS output after it has been processed by webpack

but it shouldn't even touch it

but yes, that is the loading order issue I mentioned above

you either setup some kind of coordination mechanism, so that the JS waits for the CLJS to load and then starts

or :browser is not an option

I assume there is some kind of "start" function in your JS code

if that start point can wait for everything else to finish it should be fine right?

or do you need to access the CLJS code immediately while the files are loading?

that won't work

I've just finished getting two branches ready with my best effort for both :browser and :npm-module. see https://github.com/bshepherdson/metabase/tree/cljs-repl-browser and https://github.com/bshepherdson/metabase/tree/cljs-repl-npm (I've replaced the README.md with usage notes and a sketch of the approach for each one).

the globals are needed statically, unfortunately. webpack was replacing the imports of CLJS with const Lib = window.metabase.lib.js; or similar, at the top level.

hmm and if you get rid of those? I mean this isn't really needed right?

the actual code can just call metabase.lib.whatever() instead of Lib.whatever()?

theoretically I could replace all the Lib.foo() calls with window.metabase.lib.js.foo() calls and it should work.

or if thats possible with webpack you could take the :external-index file, run it through webpack

then load that file

then the CLJS output

then the webpack regular output?

directly calling the globals would be plausible but unpopular, but it's also not minifier friendly IIUC. or maybe ^:exported functions are always globals even in a release build?

:export is always global yes, well not in :npm-module but in :browser yes

I'm just not sure if its possible to make webpack split out the :external-index content and load it first

I'd assume that is possible but thats way beyond what I know about webpack 😛

I see. that's a plausible path if there are no other options. probably there's even a way to persuade webpack to generate code on seeing import * as Lib from "cljs/some.namespace"; that will do nothing at the top level but access the global when used in the function calls.

I'll play around with that, I don't think it's a big step from the current cljs-repl-browser branch.

I mean :esm is also an option, but since you are using namespaces directly is going to be a little annoying

I don't follow that.

:npm-module you can access each namespace directly, as you are doing in import * as Lib from "cljs/some.namespace";

:esm you setup the modules in the build config

so you'd have import * as Lib from "cljs/that-module"

but rather than just having (defn ^:export foo []) in the CLJS code

you have {:exports {foo some.namespace/foo}} in the build config

so if you have a lot of exports that gets a bit tedious

could also to :exports-var some.namespace/exports and you do the (def exports #js {:foo foo}) in code

ah no wait nevermind, that does't work for ESM

we do have a lot of functions, yeah. but that could be a hook in the build config? (that scans the classpath for .clj{s,c} namespaces and for ^:exported functions inside them.)

no, as a module is the grouping of multiple namespaces

unlike :npm-module where you get a file per namespace

hoist the hook a level higher, where it generates the :modules entirely? 😜

but :modules control what gets built, so without it doesn't know where to start 😛

what bothers me about :npm-module as well as :esm I guess is that the code gets processed twice

so first shadow-cljs+closure compiler does its thing

then webpack does it again

which is wasteful and not optimal

I have hopes for the above scheme to make the access to the globals come later so :browser would work.

yeah I think the best option is 3 way split of the files, that way CLJS and JS stay entirely seperate

and the "shared" JS just gets pushed into the 3rd file

I can make the import return a Proxy so the globals won't be touched until one of the functions is actually called. that plus delaying the app kickoff until the CLJS code is loaded should do it, I think.

do you do anything special that you actually need webpack for? I mean the config looks a bit intimidating but do you actually need it? 😛

ah nvm. I guess you are using images and fonts and other resources

I was wondering that yesterday too. we use it, but I think only for things that shadow-cljs could do... but I don't know how comfy folks would be with that, and I fear how tightly integrated some of the FE folks' tools are with webpack.

let me think about it a bit over the weekend, these all have been issues that have come up many times before

but now having the real project to look at might give me some ideas

maybe I just back of the one reason that makes :npm-module so hacky

could be much less hacky after all if I don't try to isolate the scope

okay, we'll talk next week. I'll let you know if I can get the Proxy hack going. (seems plausible, but who knows.)

ok, should all be fixed with 2.25.6

just use :npm-module, no special settings. should just work out of the box

just need to remove that one js/module reference you have in a CLJS file

CLJS hot-reload and repl works just fine

webpack always says [webpack-dev-server] Nothing changed. so thats maybe a problem or maybe not? not really sure 😛

the CLJS parts however all now just work as intended

😍 fantastic stuff!

hm, I'm seeing shadow-cljs reloading the code successfully, which is great. but I'm also seeing webpack HMR detecting and reloading it, so I'm missing something to stop it. I'm trying to bring back the js/module.hot hacks.

was this working for you in the Metabase repo? I wonder what's different here.

webpack always said nothing changed? and didn't reload anything itself

can't bring back anything working with js/module since that doesn't exist when shadow-cljs reloads the code

I tested on the metabase repo, but I still had :output-dir "node_modules/metabase-cljs" in the build config

and then import ... from "metabase-cljs/whatever.js" in the code

nothing extra in the webpack configs

hmm. maybe there's magic for node_modules. I'll try that.

no, webpack is still recompiling and HMRing for me. can you push your branch?

I found my issue with webpack; I hadn't spelled the watchOptions.ignored properly. it needs to be **/node_modules/metabase_cljs_dev/** or similar; I had node_modules/metabase_cljs_dev/**.

why are you doing this dev thing?

well, the reload and REPL are working nicely. loading namespaces that are outside of the :entries doesn't work, but I'm not overly surprised by that.

why would it not work?

we have cause to let dev and release builds coexist; hence the :dev {:output-dir ".../dev"} split.

what do you gain by doing that? seems just like extra config noise you don't really need?

I mean if you want to try a release build you just make one?

on watchOptions.ignored: I think it wants to match relative import paths like ../../../../node_modules/metabase_cljs/some.namespace.js

on /dev: we often end up doing release builds while having the dev mode watch open at the same time. that was clobbering things and causing issues some months ago.

I agree it makes life more complicated, but I think the alternative is worse, and webpack HMR is sorted out now.

(still no idea why it detects changes for me but not for you; with the watchOptions.ignored set it doesn't emit any HMR messages.)

on loading things that aren't in :entries: is it expected to work if I eg. try to eval a test namespace that isn't part of the build? I'm seeing it doesn't properly require things that aren't already part of the build (eg. test utils) and also that actually running a test fails because in cljs.test.test_vars is undefined. (I'd be happy to include the tests in the build if I were either (a) confident they'd be tree-shaked out of a release build; or (b) there was like :dev {:extra-entries [...]} or :dev {:extra-ns-regexp #"-test"} that get merged together with the regular :entries.

it should work fine to load any namespace over the repl

just remember that cljs.test is extremely macro heavy

but the namespaces should load absolutely fine just like any other

assuming you require or load-file them of course

just in-ns won't do anything

I use Conjure in NeoVim, so :EvalBuf is calling eval-str with the contents of the buffer. IIUC that's not the same thing as a require. I guess I'll take that up with #CK143P6D7

thanks again for all your help on this! this will make our work on sharing business logic between the Clojure backend and JS/TS frontend a much better experience. hot reloads are great, but hooking into the running browser to play with the same query the UI is building is the real killer feature.

if you have some repro steps for stuff not working in the REPL let me know. happy to look at it, just need the actual executed REPL commands as I don't use Conjure

I think it's: • :target :npm-module in the browser like Metabase • connect the REPL • choose a namespace not already in :entries, like a test namespace. • (eval-str (slurp the-ns-file)) through the REPL • receive errors

that is not a valid thing to eval at a CLJS REPL

eval-str does not exist, neither does slurp

I guess didn't mean actually run that code in the browser. like as nrepl commands.

also don't exist as nrepl commands 😛

I've got to run but I'll be back later. Conjure (and Vim REPLs generally) are weird.

thats why I mean I need the actual things that are getting eval'd. every editor does its own thing in some way

you can also run clj -A:cljs -M -m shadow.nrepl-debug 3333 .shadow-cljs/nrepl.port

and then connect your editor to nrepl port 3333 instead of the shadow-cljs port

this creates a debug logfile of actual nrepl traffic in target/nrepl-debug

that has all the infos I need

hm, I can't get that to work. it shows [:server-ready 3333 50655] which looks plausible, but then on connecting to 3333 I get "no available JS runtime" even though the browser is connected. if I connect directly to 50655 it works fine.

ah, there we go. https://drive.google.com/file/d/1VS5Aunl6DC0dcZDJ-gePC1cSFJujJdbq/view?usp=sharing that's me eval-ing a (js/console.log "foo") then trying twice to eval a test namespace. that always seems to fail the first time (or maybe I don't wait long enough after landing in the ns?) but the second attempt is what I've typically been seeing.

(if you're not familiar with how vim REPL integration generally works: there's no terminal interface where you type into the REPL. you send the inner or top-level form under the cursor, or the whole file, to the REPL and an output window displays the results. the sticky part here seems to be that "eval the whole file" is done in the same way as any other form, ie. just passing the whole text of the file to be eval'd like anything else.)

your codebase is weird. code I can execute just fine in my shadow-cljs project, blows up in the metabase project

as in it runs very slowely and eventually runs out of memory

do you modify the clojure runtime in some way? its too much code to easily figure out what might be going on

not that I know of. there is a custom test assertion, but that's just a defmethod call I think. mostly it's just a huge codebase, ~113kLOC of non-test Clojure(Script). it's become a performance regression test for clj-kondo and clojure-lsp.

the webpack part of the JS build does crash for OOM every few hours though, if you keep making changes so it does hot reloads.

(for shadow-cljs purposes, the CLJS-facing codebase is much, much smaller than the BE CLJ codebase.)

it appears to be cider-nrepl, at least it showed up in a jstack dump where it shouldn't even be involved

removed it and the thing I was looking at works fine

weird. maybe one of us has a ~/.clojure/deps.edn that's doing something different? I think all the versions are up to date, but perhaps not.

I use Cursive. so no cider-nrepl at all ever

but that wasn't the problem I was looking at, just one that prevented me from looking what I actually wanted to look at 😛

FWIW I just updated my PR with the latest integration of 2.25.6, https://github.com/metabase/metabase/pull/34007/files in case there's something diverging between your tests and mine.

so I'll leave that one for later

I'm fighting with your dev setup right now

its driving me nuts to always start everything in one command, so trying to split it up

but now I started shadow-cljs separately but for some reason webpack isn't using that build output?

or rather the REPL is not connecting

basically I just removed 'yarn build-hot:cljs' from the dev command

but I guess something doesn't like that

with my PR, you can do clojure -M:run, yarn build-hot:cljs and yarn build-hot:js all in separate terminals.

(it's important that the CLJS build finishes before the yarn build-hot:js starts)

huh, that should have worked. I dunno.

ok, if I run that instead it works

ok, I think I found the problem. just not sure what causes it. too tired, will take a look tommorrow

good news! have a good night.

found the cider problem https://github.com/clojure-emacs/cider-nrepl/issues/819, not related to your issue but I'll be looking at that next

webpack 5.85.0 compiled successfully in 72958 ms this isn't normal right?

ok found a bunch of other problems in the REPL code. should all be fixed as of 2.25.7. at least I can't find anything that doesn't work anymore. let me know if you do 😉

I regret to say that the webpack time seems normal enough to me, depending on the speed of the machine and disk. it takes 23300ms on my M1 Mac with an SSD and all the files already in the disk caches because it just ran. just Node things; it touches tens of thousands of JS files in node_modules before it even starts to process the actual input. (the more I work with webpack, and standard JS tooling generally, the more glad I am that it's not my main job.)

seems to be WSL making it so slow, but yeah webpack seems rather slow overall

Bun seems like it'll help, but currently it just generates thousands of errors really fast 😅

hehe

yeah the problem is that webpack added so many quirks to the building stuff, that many packages use those quirks and rely on them.

so using anything other than webpack yields a bunch of errors. sometimes I just have to give up in shadow-cljs and direct people to webpack

let me give 2.25.7 a try momentarily.

I had some thoughts on the webpack hot reload stuff too, since webpack just seems to ignore node_modules files updating entirely

so you need to actually restart webpack to get updated CLJS code loaded

at least sometimes, couldn't really figure out a pattern

that's actually still an unexplained difference between yours and mine - it does notice node_modules changes for me.

not sure if that's a file-watcher thing or what

could be WSL related in some way, since it also tried to watch my c:\pagefile.sys for some reason

I wonder if that's a V8 internal thing that the file watcher is wrongly seeing under WSL? many GCs play games with the memory map to put an inaccessible page at the far end of the allocation area. (so they can move the allocation pointer without checking it's still in bounds, and the hardware generates a page fault if they've run out of Eden space. makes the common case a free hardware test instead of a branch instruction.)

it works beautifully! I loaded a test file not already in the classpath, and see all the transitive loads in the console, and the tests run neatly. chefskiss.png!

thank you again so much for your superb support! Metabase is an odd case in several ways, but everything is working very nicely now.

happy to fix it, very rare to have access to setups like yours and I want this to work