what is the context here? does it happen when you first run the output? does it happen over the REPL?

it’s as result of compiling or watching a dev build - as soon as i run my script:-

$ node ./out/index.js

since I’m using leiningen to manage the dependencies i start a dev build with the following:-

lein with-profile +dev run -m shadow.cljs.devtools.cli --npm watch cli

lein with-profile +live run -m shadow.cljs.devtools.cli --npm release cli

i also appreciate this is a little vague so am happy to put a little github repro together … just from what I’ve been reading and trying to get through my thick skull is whether it’s due to the :keep-as-import bit which might exclude it for development builds but happily bundle it for release builds? i must admit i struggle with the complexities of esm vs common js etc … this is the first time I’ve tried to build a simple project using a dependency that’s esm only :s - i really do appreciate any help but also understand you’re insanely busy!

since its node you can just run :js-options {:js-provider :import} maybe that fixes it?

repro always helps

which shadow-cljs version do you use?

latest version i believe - i'll give the js-options a try and if no joy i'll get a repro for you :) thanks for the continued help

yep .. pretty sure im using 2.23.3

curious why you are doing the +live +dev switch? not related to esm, just curious

oh ... i might not actually need it but the project is eventually meant to connect to one of our various firebase projects and i was attempting to use different profiles to add additional src paths for the various google service json files

(and i needed to use leiningen as that supported connecting to our private maven repo on google cloud)

ok, just tried with :js-provider :import and it’s the same deal, if I build a release then my script runs fine, if I compile/watch a dev build i get the same ReferenceError: esm_import$ink is not defined error 😞 i’ll get a repro to you, if not over the weekend then definitely Monday!

so I had some time to look over this and identified the issue

per spec the module loading order is guaranteed as long as there is no async import

https://stackoverflow.com/questions/35551366/what-is-the-defined-execution-order-of-es6-imports

the shadow-cljs output assumes this is always the case

however, ink has a top level async import, which then apparently makes everything async and totally messes the loading order of shadow-cljs

not sure what to do about that

ah fair enough @thheller - really appreciate the update especially over the weekend! i think for the time being I could at least go back to a pre 4.0 release of ink (prior to the esm module rewrite) - it’ll mean I can’t use react 18, but that’s not an issue at all really

I had an idea how to at least ensure this sort of works

am all ears! if there’s anything I need to change/reconfigure in my project/code

problem appears to be bigger than expected but I got some stuff working

oh nice - that’s encouraging! if there’s anything I can try and do to help? although the various ways commonJS and esm work and conflict with each other currently confuse me!

(even if it’s to send you some beer money!) 🙂

just need to tweak the output thats generated so node is happy. just need to find a way to work around one last issue

wow! thanks for this!

try version 2.23.4, should hopefully take care of it

legend! that’s dev and release builds working … am I right in that I shouldn’t be loading this module dynamically, I should be able to (:require [“ink” :as ink :refer [Text render]]) .. as normal?

actually ignore that … that’s what i’m already doing and it’s working fine thank you 🙂

just spotted one little issue which might be due to hot reloads in dev builds … if I change my code and run my script again I get:-

file:///Users/benj/Development/fireadmin_2.0/out/cljs-runtime/shadow.esm.esm_import$ink.js:4
import * as esm_import$ink from "ink";
^^^^^^^^^^^^^^^^^^^^^^^^^^

SyntaxError: Identifier 'esm_import$ink' has already been declared
    at ESMLoader.moduleStrategy (node:internal/modules/esm/translators:119:18)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:468:14)

if I quit the watch process and restart it and then run the script everything is fine

what exactly do you do to trigger this?

you say hot reload and then "run my script again", so what is it? a hot-reload or node that-script.js?

Sorry … the script just renders a Text component to the terminal and then exits. If I simply change the color property on the text component, the watch process recompiles and produces new output - then I run the script again and get the error. If I restart the watch, and run my script it all works fine

oh right

the change I made is not cache aware, so it just writes it every time and duplicates the imports

should I add ^:dev/once metadata to the namespaces that might require “ink”?

no you should not. try 2.23.5

kk

yep! all good 🙂

can I contribute to your beer fund in anyway for all the help?

ah! nvm … found your paypal link … 🙂

thanks

pleasure … and thank you for all your hard work on shadow - it’s been a game changer for me!

@thheller sorry, it’s me again … not a big issue - dev build is still happy etc, but I can’t get a REPL connection with a :target :esm build - previously it was :node-script and this was fine - now I just get repeated Waiting for Shadow CLJS runtimes, start your CLJS app even though my node script is running? Don’t want this to be a time sink so if the :esm target doesn’t support connecting via a REPL then fair enough

yeah, :esm currently doesn't support a node REPL

ah … that’s an easy one then 🙂 thanks!

REPL in general is very tricky with ESM, so have to figure some stuff out for that eventually

kk no worries … wish I hadn’t started playing around with this “ink” library now … will see how far I get without my REPL crutch 😄

you could just stick with a regular :node-script and use a dynamic import to load the ink lib

oh! this?

(ns 
  (:require
    [shadow.esm :refer (dynamic-import)]
    [shadow.cljs.modern :refer (js-await)]))

(defn foo []
  (js-await [mod (dynamic-import "ink")]
    (js/console.log "loaded module" mod)))

no you need to expose mod somehow to the outside and make sure the import finishes before you run anything else

so (def ink nil) as a normal def and then (set! ink mod) or so in the await body

ah gotcha … will give that a go thanks

for shadow.esm to work you probably need :prepend "global.shadow_esm_import = function(x) { return import(x); }" in the build config

still need to hide that from the closure compiler so it doesn't complain

does :prepend go in the :modules section and does the :node-script target support it?

node-script doesn't have a modules section 😉

ah … 🙂

just in the build config should be fine, not actually sure but I think so

cool thanks 🙂

well that all works a treat! back to :node-script target with REPL, using the pesky ESM ‘ink’ module … script waits until the dynamic import has completed at which point it (set!’s …) all the stuff I need to expose, then I just require my new ink namespace instead and :refer to everything as normal … brilliant!

(I’ll have to wait until next payday to send you some more beer money!) 🍻

dont worry about it, happy to help 🙂

I should probably make that a properly documented recommendation. going full ESM so full of so many problems 😛

😄 … the Javascript eco system (common Js and ESM) makes my brain hurt … if it wasn’t for shadow-cljs I’d be in a proper state :s

hey @thheller - one more quick question, if i needed to import another esm module what would i change my :prepend to in my build config? i currently have

"global.shadow_esm_import = function(x) { return import(\"ink\"); }"

var ink = import("ink");
var other = import("other");
return merge(ink, other) // whatever the js equivalent is :)

uhm why did you hardcode "ink" there? thats supposed to use the x argument?

oh!

didn’t realise - thought ‘x’ was a placeholder for the module I wanted to import :s

doh!

nah we just can't call the import directly because the closure compiler complains

so we setup an alias function, and shadow.esm/dynamic-import calls that instead

cool! thanks for setting me straight 🙂

:target :esm sets up that alias automatically, but :esm then just has other problems which you are aware of 😛

😄

i don't know of a flamegraph but the shadow cljs console usually reports how long it took to write out cljs files. i assume this ia your bottleneck and not closure's optimizations misread your post, oops

does it just take long or does it also produce excessive amount of JS code?

otherwise profiling is just regular profiling. nothing special in regards to the compiler. if you suspect macros just record time spent in anything but cljs.* or shadow.*. that should reveal macro issues

did you try to just eval form for form in the REPL? should be noticable if something is slow

hmm yeah I guess I'll try and hook in clj-async-profiler or something

I haven't gone form for form. the curious thing is that in another build, a change to the file takes 5s. so my hypothesis that it's a form in the file might be off.

it could be a difference in the build type too. one build is an :npm-module the other is a regular :browser build

build targets really only control how the output is organized. they are not a factor in actual compilation. in fact they aren't involved in that code.

so, more than likely it is the macros. what is done within those I cannot say. if you have something looking at the target setting that is certainly possible

I had moderate success at profiling Clojure using YourKit https://www.yourkit.com/java/profiler/features/