Fork me on GitHub
#shadow-cljs
<
2022-10-21
>
mkvlr06:10:28

when implementing server-side-rendering, I'm thinking about switching to :target :esm so I'm able to keep a single bundle that I can run in the browser and in node (or graal). Are there any reasons I'm missing why this would be a bad idea?

thheller06:10:02

well for node you usually wouldn't bundle all the dependencies and let node load them

thheller06:10:09

for graal/browser you'd need to

thheller06:10:43

but I guess for node it doesn't matter too much if you just bundle it

thheller06:10:54

graal IIRC doesn't support ESM, so only single module release builds would work

mkvlr06:10:38

right, I think you convinced me regarding getting things working in node vs in graal first.

thheller07:10:10

I'd recommend :node-script, it just is the most straightforward way for node

thheller07:10:23

ESM is a little bit of a journey given the state of ESM in node

mkvlr07:10:52

oh, there's still issues with it? Ok, will try :node-script on the first glitch I run into

mkvlr07:10:47

or actually, trying :node-script first, thanks for the rec!

thheller07:10:44

ESM needs to be opt-in for node. only ESM code can include ESM packages thought, so depending on what you want to use that may be required

mkvlr07:10:31

and for the browser do you recommend :esm these days or also sticking with :browser?

thheller07:10:21

:browser, unless you want to load ESM code dynamically via import

👍 1
mkvlr07:10:24

ok, so I actually do load esm modules in node now, so trying :esm instead of :node-script

mkvlr07:10:18

running into ReferenceError: WebSocket is not defined at shadow$cljs$devtools$client$websocket$start (full error in 🧵 ) with target :esm and trying to run it via node .

mkvlr07:10:28

node public/js/viewer.js
Installing CLJS DevTools 1.0.3 and enabling features :formatters :hints :async
file:///Users/mk/dev/clerk/public/js/cljs-runtime/shadow.cljs.devtools.client.websocket.js:5
var socket = (new WebSocket(ws_url));
             ^

ReferenceError: WebSocket is not defined
    at shadow$cljs$devtools$client$websocket$start (file:///Users/mk/dev/clerk/public/js/cljs-runtime/shadow.cljs.devtools.client.websocket.js:5:14)
    at Object.attempt_connect_BANG_ (file:///Users/mk/dev/clerk/public/js/cljs-runtime/shadow.cljs.devtools.client.shared.js:482:128)
    at Object.shadow$cljs$devtools$client$shared$init_runtime_BANG_ [as init_runtime_BANG_] (file:///Users/mk/dev/clerk/public/js/cljs-runtime/shadow.cljs.devtools.client.shared.js:970:16)
    at file:///Users/mk/dev/clerk/public/js/cljs-runtime/shadow.cljs.devtools.client.browser.js:957:36
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:541:24)
    at async loadESM (node:internal/process/esm_loader:91:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12)

Node.js v18.7.0
mk@studio ~/d/clerk (main) [1]> node public/js/renderer.js
file:///Users/mk/dev/clerk/public/js/renderer.js:5
var SHADOW_IMPORT_PATH = __dirname + '/../../.shadow-cljs/builds/renderer/dev/out/cljs-runtime';
                         ^

ReferenceError: __dirname is not defined in ES module scope
This file is being treated as an ES module because it has a '.js' file extension and '/Users/mk/dev/clerk/package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.
    at file:///Users/mk/dev/clerk/public/js/renderer.js:5:26
    at file:///Users/mk/dev/clerk/public/js/renderer.js:1797:3
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:541:24)
    at async loadESM (node:internal/process/esm_loader:91:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12)

Node.js v18.7.0

thheller07:10:48

currently :target :esm presumes browser runtime as the default

thheller07:10:59

(and has no implemented support for node repls)

thheller07:10:13

set :runtime :custom in the build config

mkvlr07:10:24

nice, no more error!

mkvlr08:10:18

so I have a module in public/js/viewer.js that I can load without errors in node but am too stupid to write another script that imports and uses it 🙈 The js ecosystem often makes me feel this way…

thheller08:10:23

first of all drop the notion of public for node. to me that look like it would end up as a public script in http sense, which I assume what you are not building here

thheller08:10:40

> write another script that imports

thheller08:10:44

you mean other .js script?

mkvlr08:10:02

my module has this at the end shadow.cljs.devtools.client.env.module_loaded("viewer");

thheller08:10:23

not sure what you mean by that

mkvlr08:10:48

tried various ways to import (also moving it to the ./ path but didn't figure out how to make it work

thheller08:10:49

this is something shadow-cljs internal, it is not relevant for anything besides that

thheller08:10:59

you mean importing from a .js file?

mkvlr08:10:13

should I be able to import "viewer.js"?

thheller08:10:14

import { somethingYouExported } from "./public/js/viewer.js"

thheller08:10:33

path depends on where the .js file lives relative to the viewer.js

thheller08:10:10

import "viewer.js" is viewed by node a npm package and it'll try to load an npm package

mkvlr08:10:12

doh, I could swear I tried this but it seems like not

mkvlr08:10:24

and your thing above does work, thank you!

thheller08:10:31

so always use a relative path of some kind

thheller08:10:42

so, starting with ./ or ../

mkvlr08:10:06

ah, I tried public/js/viewer.js and /public/js/viewer.js but not ./public/js/viewer.js

thheller08:10:30

/ would be an absolute path, so absolute on your filesystem

thheller08:10:41

public/js/viewer again would look for the public npm package

👍 1
Hankstenberg10:10:23

Is there any guide that says something about how to use the output of the "release" target? I see various files lying around in ".shadow-cljs/builds/app/release", but.. what do I do with them?

1
thheller10:10:12

you don't do anything with those, you only use the actual configured output files. ie. :output-dir in a browser build

thheller10:10:58

the files in .shadow-cljs are build supporting files (eg. caches). they are build internal and aren't used anywhere besides that

🙌 1
Hankstenberg11:10:23

Ah, I see thanks. 🙂 Okay, now I need to find out why the release build doesn't work.

thheller11:10:50

whats the problem?

Hankstenberg11:10:52

"Error: No item 1 in vector of length 0"

Hankstenberg11:10:42

There's some of my own log messages in the console before the error, so I guess I need to see where it crashes by putting in more console logs.. or is there a better way?

thheller11:10:17

thats the error message you'd get for (nth [] 1)

thheller11:10:51

I recommend using npx shadow-cljs release app --debug to debug issues

thheller11:10:41

or npx shadow-cljs release app --pseudo-names but that doesn't have source maps, so a little lower level and may require looking over some JS code

thheller11:10:53

generally --debug is best

Hankstenberg11:10:15

Damn, I actually have a single stupid "nth" in my code. I'll check that right away. Thanks so much!

Hankstenberg12:10:34

Got it working. Awesome!

😎 1
mkvlr12:10:36

now running into https://clojurians.slack.com/archives/C6N245JGG/p1627574180362500 with the advanced build. @borkdude you wrote you replaced this, is there a way to let shadow fix this now?

borkdude12:10:18

Check nbb's build, I don't remember, it's been a while :)

mkvlr12:10:07

(spit "lib/nbb_core.js"
        (str/replace (slurp "lib/nbb_core.js") (re-pattern "self") "globalThis"))

mkvlr12:10:19

thanks!

👍 1
thheller13:10:33

uhm this should not exist?

thheller13:10:43

how did you build it?

mkvlr13:10:03

shadow release, nothing else

thheller14:10:28

clerk$ npx shadow-cljs release viewer --pseudo-names
shadow-cljs - config: /mnt/c/Users/thheller/code/oss/clerk/shadow-cljs.edn
shadow-cljs - starting via "clojure"
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate shadow/cljs/devtools/cli__init.class, shadow/cljs/devtools/cli.clj or shadow/cljs/devtools/cli.cljc on classpath.

Full report at:
/tmp/clojure-13405443538645289994.edn

thheller14:10:35

how do you compile?

mkvlr14:10:14

clojure -M:sci:demo:dev release viewer

thheller14:10:54

yeah upgrade

thheller14:10:04

that shadow-cljs version didn't handle esm properly

borkdude14:10:18

so I can perhaps also remove that workaround then

mkvlr14:10:34

@U05224H0W will do, thanks so much for all your support, greatly appreciated! 🙏

mkvlr14:10:02

@borkdude a careful removal, are you afraid it’s coming back?

borkdude14:10:53

no, it was more a lazy removal ::P

Lone Ranger15:10:38

Want to confirm there's no shadow-cljs easy pathway for this kind of js:

import 'draft-js/dist/Draft.css';
The only way to get that to work is to webpack compile it then require the webpack compiled javascript with shadow via the normal js interop routes? aka
// ./src/js/project/css.js
import 'draft-js/dist/Draft.css';
(defn project.css
  (:require "/project/css"))  ;; <-- Invalid namespace declaration    
etc?

thheller16:10:06

correct, shadow-cljs does not support css. use a tool that actually only handles css and don't try to turn css into js

thheller16:10:18

just postcss or so is fine

thheller16:10:07

and absolutely never even try to include some webpack output in a shadow-cljs build just to get css. thats madness 😛

😅 2
hlship16:10:13

So, I have an idea that I'm working on, and I'm trying to figure out how to proceed. In essence, I want a command line tool that executes, reads some data from the local directory, and then launches a browser to interactively show the results. I'm just trying to find my way around all the browser UI stuff I've ignored for about 10 years, and wrap my head around shadow-cljs concepts. First off, is this a normal use-case?

hlship16:10:02

As in, I would want my browser FE on launch to chat with a local server, passing EDN or Transit or JSON back and forth.

thheller17:10:46

yes, you can use shadow-cljs to build the cljs code that ends up running in the browser

thheller17:10:03

and the node script in case you want to run node for the command line parts

thheller17:10:05

(or use CLJ for the command line parts)

hlship17:10:19

Right, so a Clojure CLI command that, at launch, starts up a web server, serves up the previously compiled JS & assets plus additional routes specific to the tool, and opens a browser to the URL?

thheller17:10:44

yes, that would be one way

hlship17:10:10

Is that a well trodden path, or do I have some bad preconceptions?

thheller17:10:00

well running it locally is probably not done too often. but any CLJ website works that way, so yes very good path

thheller17:10:04

its basically what shadow-cljs does during development ;)

thheller17:10:36

ie shadow-cljs server and open (its UI)

thheller17:10:00

serves the precompiled JS code and talks to the CLJ process via websocket

hlship17:10:42

I'm a little lost into how I can add my own routes to http://localhost:8080 when running shadow-cljs watch frontend

thheller17:10:50

well that should be the server you intend to run for "production", which shadow-cljs should not be a part of

thheller17:10:08

dunno what you use for :8080 now? :dev-http?

thheller17:10:35

it should be your own server. ring, pedestal, whatever you like

hlship17:10:09

So far, no server support. I guess we would set up :8080 for JS and other assets, and another port, say :8081, for data?

thheller17:10:30

no, just one server. as far as JS and other assets are concerned, it just needs to serve the static files. any webserver can do that

hlship17:10:54

So far, all I have is this:

;; shadow-cljs configuration
{:deps true

 :dependencies
 []

 :dev-http {8080 "public"}

 :builds
 {:frontend
  {:target :browser
   :modules {:main {:init-fn }}}}}
Ah, ok. I need to specify a :handler for my non-static assets.

thheller17:10:22

you create a server to run your CLJ code. the one you intend to use for your command line tool

thheller17:10:28

shadow-cljs is not involved in that in any way

thheller17:10:40

you build it like you would for a regular website

thheller17:10:56

eg. pedestal. no knowledge of shadow-cljs whatsoever. separate worlds.

hlship17:10:24

Right, that's what I was thinking when I said "data on port 8081".

thheller17:10:41

yeah, just remove :dev-http {8080 "public"} then

thheller17:10:46

its not needed if you have your own server

thheller17:10:31

just make that server serve the files in the public dir

hlship17:10:25

So I want all the live reloading goodness when developing, but then I want to package it all up and serve static assets off the classpath; so it feels like you are telling me about the second part of the story (one production server to handle static and dynamic) but I'm not quite sure what it looks like during development (where I want all the shadow-cljs live reloading goodness, at least for client components).

thheller17:10:57

all REPL and hot-reloading is handled independently of the :dev-http server. it is not involved in that

thheller17:10:19

that is handled by the main shadow-cljs server running on :9630, which also serves the UI parts

thheller17:10:34

the server serving the JS doesn't need to do anything special, just static files. it'll work with any server and needs no knowledge of shadow-cljs

hlship17:10:10

Oh, it's coming into focus for me; so the shadow-cljs code is doing its thing and it's writing stuff intoo my public/js folder and using websockets to force the client to reload, but the server on :8080 is vanilla, and I can just replace that with my own app-specific server that servers assets from public but also hands the routes I need.

hlship17:10:34

Maybe I can do a documentation PR that lays some of this out?

thheller17:10:04

> IMPORTANT These are just generic web servers that server static files. They are not required for any live-reload or REPL logic. Any webserver will do, these are just provided for convenience.

thheller17:10:09

its sort of there already?

hlship17:10:50

Yes, I'd love to expand on this a bit: "As you develop your client side application, you will reach the point where it needs to communicate with a backend; at that point, you should remove :dev-http and provide your own server, that serves the assets in public but also provides the additional routes your frontend requires." Something like that.

lilactown19:10:41

you might not. we use a reverse proxy to route our requests to either our local UI, local APIs or dev env APIs

lilactown19:10:10

we continue to use shadow's dev-http server with that setup

thheller19:10:21

there are just too many possible setups to cover everything I'd say

Lone Ranger21:10:50

Ok I am trying to figure out how this sorcery works:

import cljs from "goog:cljs.core";
Is this a shadow thing or a Google Closure Compiler thing?

thheller21:10:05

closure compiler