run node the-output.js --help

thanks

When I println I don't see it in the console. How can I debug my tests / run println?

don't know what you mean. whereever you run node the-output.js it should show the println

Project is here: https://github.com/squint-cljs/squint/blob/main/shadow-cljs.edn

the error simply means that cljs.pprint is required in multiple modules and the configured hierarchy doesn't work

compiler is the main module

I think it might be due to shadow using it in its own stuff

This used to be the case for node too which you fixed

if I remember correctly

yes, it likely is because of the devtools injected into the browser. it does a bunch more stuff than the node one (e.g. the hud)

I can disable that right?

if you don't want a repl or hot-reload yes

yes that's fine

:devtools {:enabled false} in the build config

that fixed the problem, awesome, thanks

FWIW :runtime :node basically also just disabled this since that doesn't exist still 😛

:browser actually exists, so that just enabled it. when I eventually get to actually implementing :runtime :node it'll break as well since the devtools themselves require cljs.pprint

https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/remote/runtime/obj_support.cljc#L4

I'm still dealing with the root problem though, I'm getting some kind of warning about "fs" in the browser which I cannot copy paste. I'm pretty sure this part of my code doesn't use "fs"

well something must be, shadow-cljs doesn't

when I do a "release" build it works

this is why I switched to :runtime :browser

but to no avail

I can make a small repro if you're interested

how do I get it to the failure state?

I pushed a branch here: https://github.com/squint-cljs/squint/tree/module-in-browser Run bb dev (or the equivalent shadow command for this bb task) And then host a web server in the root http-server --dir . on port 8090 And then visit localhost:8090 (or change the port in index.html

I can make a smaller repro if that's better for you

When you run bb build instead, the example works

can I do it without bb? not installed

yes, just look inside the bb.edn for what to run

dev (t/watch-squint) that doesn't tell me much 😛

oh sorry :)

just a sec

(defn watch-squint []
  (fs/create-dirs ".work")
  (fs/delete-tree ".shadow-cljs/builds/squint/dev/ana/squint")
  (spit ".work/config-merge.edn" (shadow-extra-test-config))
  (bump-core-vars)
  (shell "npx shadow-cljs --aliases :dev --config-merge .work/config-merge.edn watch squint"))

ah yeah found that

fuck, this is hard to do without running bb ;)

just let me make a smaller repro later today if this is too much hassle for you

I just make the changes to the build config manually I guess

works for me? can't see any fs errors?

with watch?

{:deps {:aliases [:cljs]}
 :dev-http {3001 "."}
 :builds
 {:squint
  {:js-options {;; don't bundle any npm libs
                :js-provider :import}
   :compiler-options {:infer-externs :auto}
   :target :esm
   :runtime :browser
   :output-dir "lib"
   :devtools {:enabled false}
   :modules
   {:compiler {:exports
               {compileString squint.compiler/compile-string}}
    :compiler.sci {:depends-on #{:compiler :compiler.node}
                   :init-fn squint.compiler.sci/init}
    :compiler.node {:depends-on #{:compiler}
                    :exports
                    {compileFile squint.compiler.node/compile-file-js
                     compileString squint.compiler.node/compile-string-js}}
    :cljs.pprint {:entries [cljs.pprint]
                  :depends-on #{:compiler}}
    :node.nrepl_server {:depends-on #{:compiler.node :cljs.pprint}
                        :exports {startServer squint.repl.nrepl-server/start-server}}
    :cli {:depends-on #{:compiler :compiler.node}
          :init-fn squint.internal.cli/init}}
   :build-hooks [(shadow.cljs.build-report/hook
                   {:output-to "report.html"})]}}}

npx shadow-cljs watch squint yes

index.html changed to import { compileString } from '/lib/compiler.js'

ah I used /index.js

I'll try yours

also works as it just reexports the same thing

what might be relevant here and what I don't have because of no bb is this (bump-core-vars) thing

yeah. weird, perhaps some kind of caching problem

whatever that does

ok, I'll check some more. gotta run now. thanks for verifying on your side

bump-core-vars checks squint/core.js for what exports are defined there and dumps it in an edn file

not sure if I'm supposed to do anything special to trigger this

I can click the compile! button and it spits out some code and an alert

so seems working to me

yeah, weird. I'll try cloning in a fresh directory later today

@U05224H0W Sure enough, I got it working when I make a fresh clone. In my regular checkout, no matter what I do: delete .shadow-cljs, .work, the "fs" problem keeps coming back. Like, wtf :)

But thanks for double checking again

I'll just swap the fresh clone with my regular dev folder checkout and be done with it

crap, no, I made a mistake, my index.html was loading my non-local version. problem also persists with the fresh checkout

maybe it's due to this:

$ rg 'from "fs"'
cljs-runtime/shadow.esm.esm_import$fs.js
3:import * as esm_import$fs from "fs";

commenting out:

// import "./cljs-runtime/shadow.esm.esm_import$fs.js";
// SHADOW_ENV.setLoaded("shadow.esm.esm_import$fs.js");
// import "./cljs-runtime/shadow.esm.esm_import$path.js";
// SHADOW_ENV.setLoaded("shadow.esm.esm_import$path.js");

I suspect this is due to the test module also being compiled and fs + path (which are used in there) are maybe pushed to the (most common) compiler module

although when I just remove those lines, the tests still work

well, yes the test file has a direct require for fs?

not sure why its moved to the compiler module though, let me check

it is moved to compiler because :compiler.node and :squint_tests require it and their only common other dependency is :compiler

I see

you should be able to add a :node {:entries [] :depends-on #{:compiler}} module and have the node things depend on that to catch all the node things

it isn't necessary for shadow to move these deps anywhere since they are built-in to node right?

well due to the way imports are handled it is necessary

ok, I'll try your recommendation

can you put js libs in entries?

unfortunately still have to work arround the closure compiler and hiding any traces of ESM 😛

you can, but not necessary

I don't have to put anything in entries?

just use :entries []. that should be enough. entries is allowed to be empty in this case yes

it worked! thanks

In a similar project, I ran into the same "fs" issue. I tried to introduce a :node module, but this gives me:

Module Entry "clojure.string" was moved out of module ":clojure.string".
It was moved to ":cljs.core" and used by #{:cljs.core :clojure.string :compiler :cli}.

npx shadow-cljs watch cherry

well its the same reason

you are trying to force clojure.string into a separate module, but the module :depends-on structure doesn't allow it

so the compiler has to move it someplace it fits, but can't because you forced it

adding the :node module in this instance does absolutely nothing

in the other build the :node module was used as a collection spot where the namespace "mover" can put stuff

a namespace can only be in one module, and the "mover" has to pick where to put it

is there something I can do to still have the clojure.string module but not make this complain in watch? it's only a watch problem

no clue why its getting moved

ah nvm

add :devtools {:enabled false :console-support false}

this is being added https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/cljs/devtools/client/console.cljs for watch via preloads

that's it 🙌

what are you after?

I mean would it make more sense to get an output like :npm-module would produce? i.e. a .js file per CLJS ns?

feels like that is what you are going for there?

just in a much more manual way than :npm-module would need?

I want to have ES6 output

yes, I get that. but it looks like you are trying to get a .js file per namespace

nevermind for a second that :npm-module isn't ESM

but what if there was a mode for :esm where you didn't need to specify :exports and instead just give a list of namespaces you want ESM-ified

ie as in :entries [cljs.core clojure.string clojure.walk ...] and you get a ESM cljs.core.js, clojure.string, etc

with all public vars exported as actual ESM export

I mean :npm-module has the rule of needing ^:export which you can't add to those namespaces above

but it could just as well pick any public var in the namespace

not sure how useful that all would be in the end, but to me that is what you are going for there with that config setup

that's true, I'm exporting core, string, walk, etc because cherry is going to import that as its standard library via import (ESM). Exporting all public vars from a namespace is something that could be handy but not sure how many more I'll add to this project

also I may want to exclude stuff too

I'm programmatically creating the configuration for this reason

but for the repro, I didn't

now I'm back to the "fs" problem with cherry too btw. inserting the node module as I did with squint doesn't help, I'm seeing:

import "./cljs-runtime/shadow.esm.esm_import$fs.js";
SHADOW_ENV.setLoaded("shadow.esm.esm_import$fs.js");

I don't know how to explain it further. any JS require is basically a namespace in how shadow-cljs is designed

every namespace can only be in one module, so your namespace require structure decide what goes into which module

and if your config moves stuff into places it can't use then things break

introducing the extra :node module just gives the compiler a place to put stuff, that still satisfies the dependency graph

the :node name here has no special naming for the compiler, it is just an extra node in the graph

so in cherry you either need to make whatever module also depend on :node, or change your require structure

That’s what I figured too, all clear on that. The compiler module has no fs require and also no transitive fs require. Still I get this issue. I understand that node isn’t a special name, just an extra place for moving stuff to

thats the wrong way at looking at it, the compiler module doesn't need a fs require. the compiler module is the first common dependency module it can be moved to, thus it ends up there

Can I lock fs to the node module so I can debug what other modules must depend on node? (AFAIK I already have this currently, just not the locking down of fs).

sure, just :entries ["fs"], at least I think that should work. never actually tried 😛

I think I tried that yesterday but I’ll try in an hour or so. Thank you for your patience :-)

if you have a repro I'm happy to take a look, maybe I can think of a better error description. that error message has always been less then helpful :P

alright, I have :node {:entries ["fs"] :depends-on #{:compiler :cljs.core}} which isn't complained about, but also I still get the "fs" error message in the browser console 😆 I'll try to make a repro

This is what node.js looks like:

import "./compiler.js";
import "./cljs.core.js";
import "./cljs-runtime/shadow.module.node.prepend.js";
SHADOW_ENV.setLoaded("shadow.module.node.prepend.js");
import "./cljs-runtime/shadow.module.node.append.js";
SHADOW_ENV.setLoaded("shadow.module.node.append.js");

ah when I change "fs" to fs I get:

Module Entry "fs" was moved out of module ":node".
It was moved to ":compiler" and used by #{:cherry.tests :node :cli}.

yep, all good now!

cli didn't depend on node yet, which was the issue

thanks a bunch!

You can do this with a macro possibly, or perhaps goog-define will work too? Or do you change the shadow config using an env var?

https://shadow-cljs.github.io/docs/UsersGuide.html#shadow-env

https://shadow-cljs.github.io/docs/UsersGuide.html#config-merge

> but you should consider using --config-merge instead. Gotcha, that's what I also use in combination with bb

https://shadow-cljs.github.io/docs/UsersGuide.html#_release_specific_vs_development_configuration

https://shadow-cljs.github.io/docs/UsersGuide.html#clj-run

many options to do pretty much anything, without more specific details of what you actually need I cannot say what would be best

build hooks are definitely NOT recommended. I regret adding them since they are used for so much stuff they shouldn't be used for.

Thanks for the pointers, super helpful

trying to configure a few endpoint URLs at build time, and were using a very convoluted way to doing it involving babashka, S3, dynamically building the index file.. all sorts of stuff.. trying to get it way simpler to make the whole building-in-a-container thing a bit less complicated 🙂

why do runtime configuration at build time at all?

why not use actual runtime configuration?

or you can use a macro or goog-define ;)

Ah, not enough coffee yet, I meant build-time!

anything like endpoint URLs are what I call runtime configuration in general I keep out of the build config at all

and instead often load it via html, or have the runtime fetch some json/edn/transit or whatever on startup

allows changing stuff easily without even having to recompile

Our deployment story is several single-tenant instances of the front-end app, and we don’t want to deal with the client changing which tenant its pointed at.. but I mean your approach makes sense

well if the client wants to change something you are not stopping them via some build constants 😛

well, ok, there is that 😛

env.clj

(defmacro env [s]
  (System/getenv s))

(:require-macros [env :refer [env]])

(js/fetch (env "CLIENT_URL"))

this is a bad idea, do not do this. caching is not aware of this, thus will not recognize that something may need recompilation and give you cached output

I agree with Thomas, but if you're not able to do it at runtime, that's also an option. Yeah, this is true about caching, but you can disable that for specific namespaces or you can bust stuff in .shadow-cljs related to that thing ;)

and else, goog-define is really what is made for this I guess

yeah just don't. if you want environment variables use what is provided. otherwise your build output will be unreliable and annoying to debug.

"use what is provided" - what exactly? goog-define?

all the links I linked above. yes, for environment variables goog-define + shadow/env

does vanilla CLJS also cache macro usage?

there is not such thing as a "macro usage cache". there is a namespace cache, based on a few things the compiler decides to either use the cache (not expand macros at all) or recompile (actually expand macros)

I don't know what the heuristics in regular CLJS are anymore, so can't say

but given how many people run CI systems caches are often wiped completely after a build, so that point may be irrelevant

btw FWIW I also injected these configs into the HTML when I worked on an app the needed this kind of config

I just use https://github.com/thheller/shadow-graft

so many things to consider… this is all great food-for-thought.. we already leverage some env stuff in the index file directly.. which is at least easy to sanity check with view-source.. and getting them there via goog-define seems like a good way to go.. thank you both for all your insights.. my job would be no fun at all without bb and shadow-cljs 🙂

I suppose it's worth verifying that shadow connects its websocket to the same port as the file serving, but on the /api/ws path, correct?

In case it's useful, I'm proxying using something like this:

# Front-end.
  server {
    listen 8000 ssl;
    http2 on;

    include /etc/nginx/snippets/ssl-params.conf;
    include /etc/nginx/snippets/self-signed.conf;

    location / {
      proxy_pass ;
      proxy_http_version 1.1;
      proxy_set_header Host ;
      proxy_redirect  ;
      proxy_redirect  ;
    }

    location /api/ws {
      proxy_pass ;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
    }

    access_log on;
    error_log /var/log/nginx/error.log debug;
  }

what is the error that occurs? the screenshot is missing the actual error?

the default will take the document host and combine it with the shadow-cljs port

so if this nginx is running on a different host this likely doesn't work?

you can set :devtools {:use-document-host false} in your build config to disable that and make it connect to localhost instead

Hey! Thanks so much for the response. I was thinking that the screenshot was the actual error. Just an exception in attempt-connect!. There is no other info around that.

Lemme try that document host flag.

Ohhhh, my man! That works. 🙂

I'll explore with/without proxy configurations to ensure everything still works when it's set to false. Again, thanks for your time.

see https://shadow-cljs.github.io/docs/UsersGuide.html#proxy-support btw, if you really want to proxy the websocket over your nginx

I originally looked at that, but I don't think I interpreted it correctly. Now knowing the issue, I suppose you're saying that if I set :devtools-url that I don't need to set :use-document-host?

either one is fine really, just in some setups the actual host shadow-cljs is running on may not be accessible. so the :devtools-url would allow you to specify what it should talk to, i.e. your proxy

if its directly reachable :use-document-host is enough

Perfect. Thank you. 🙂

say you want to access the app over wlan on your phone or something

it trying to access localhost won't work

Understood.