Fork me on GitHub
#shadow-cljs
<
2024-03-28
>
martinklepsch09:03:01

Hey! I just added [cognitect.transit :as t] to a :target :esm + :js-provider :import build and it appears to create an invalid import statement:

1 | import "./cljs_env.js";
    ^
SyntaxError: Unexpected string literal "./cljs_env.js". import call expects one or two arguments.

thheller09:03:38

hmm in what context does that happen? doesn't look like a shadow-cljs error? and its a perfectly valid import?

martinklepsch09:03:09

It's in bun โ€” maybe that is the issue.

thheller09:03:10

its also not an import call? call being import()

thheller09:03:56

I'd assume that bun understands ESM?

thheller09:03:34

dunno if thats relevant but it shows up as first hit on google ๐Ÿ˜›

๐Ÿ˜ 1
thheller09:03:23

could be that its attempting to load the file as commonjs

martinklepsch09:03:28

Weirdly when I just import the file in a standalone JS file and run it with bun its all fine

thheller09:03:35

in node you need to set type: "module" in package.json

โœ… 1
thheller09:03:42

dunno if there is an equiv for bun

thheller09:03:15

I would have assumed that bun would fix some of the node nonsense

thheller09:03:17

but guess not ๐Ÿ˜›

martinklepsch09:03:58

Hm now I tried logging whatever cljs_env.js exports and got this error

1 | (function (entry, fetcher)
    ^
SyntaxError: Missing 'default' export in module 'server-out/cljs-runtime/cljs_env.js'.

thheller09:03:31

why would it need a default export?

thheller09:03:53

also dunno what that line is, its not shadow-cljs code?

martinklepsch09:03:29

no, it's just bun file.js

import env from "./server-out/cljs-runtime/cljs_env.js";

console.log(env)

thheller09:03:43

ah, yeah it doesn't have a default export, so that won't work

thheller09:03:50

it doesn't export anything at all

thheller09:03:04

in esm development builds everything is running using globalThis

thheller09:03:26

so exports are only used for release builds and the defined :modules in the build config

thheller09:03:04

$ bun js/main.js
Module {
  Box: {
    $$typeof: Symbol(react.forward_ref),
    render: [Function: Box],
    displayName: [Getter/Setter],
    defaultProps: {
      flexWrap: "nowrap",
      flexDirection: "row",
      flexGrow: 0,
      flexShrink: 1,
    },
  },
  Newline: [Function: Newline],
  Spacer: [Function: Spacer],
  Static: [Function: Static],
  Text: [Function: Text],
  Transform: [Function: Transform],
  measureElement: [Function: measureElement],
  render: [Function: render],
  useApp: [Function: useApp],
  useFocus: [Function: useFocus],
  useFocusManager: [Function: useFocusManager],
  useInput: [Function: useInput],
  useStderr: [Function: useStderr],
  useStdin: [Function: useStdin],
  useStdout: [Function: useStdout],
}

thheller09:03:14

just tested a dummy build, runs fine in bun

martinklepsch09:03:23

Can you require transit?

martinklepsch09:03:42

I haven't had an issue until that, it all worked totally fine

martinklepsch09:03:15

[com.cognitect/transit-cljs "0.8.280"]
[cognitect.transit :as t]

thheller09:03:24

just requiring that causes the problem

martinklepsch09:03:11

Yeah, same here

thheller09:03:34

ah .. you didn't show the full error

thheller09:03:36

1 | import "./cljs_env.js";
    ^
SyntaxError: Unexpected string literal "./cljs_env.js". import call expects one or two arguments.
      at <parse> (/mnt/c/Users/thheller/code/shadow-cljs/out/esm-node/js/cljs-runtime/com.cognitect.transit.js:1:1)

thheller09:03:26

I suspect it gets confused by this

if (TRANSIT_NODE_TARGET) {
    module.exports = {reader:transit.reader, writer:transit.writer, makeBuilder:transit.makeBuilder, makeWriteHandler:transit.makeWriteHandler, date:types.date, integer:types.intValue, isInteger:types.isInteger, uuid:types.uuid, isUUID:types.isUUID, bigInt:types.bigInteger, isBigInt:types.isBigInteger, bigDec:types.bigDecimalValue, isBigDec:types.isBigDecimal, keyword:types.keyword, isKeyword:types.isKeyword, symbol:types.symbol, isSymbol:types.isSymbol, binary:types.binary, isBinary:types.isBinary, 
    uri:types.uri, isURI:types.isURI, map:types.map, isMap:types.isMap, set:types.set, isSet:types.isSet, list:types.list, isList:types.isList, quoted:types.quoted, isQuoted:types.isQuoted, tagged:types.taggedValue, isTaggedValue:types.isTaggedValue, link:types.link, isLink:types.isLink, hash:eq.hashCode, hashArrayLike:eq.hashArrayLike, hashMapLike:eq.hashMapLike, equals:eq.equals, extendToEQ:eq.extendToEQ, mapToObject:transit.mapToObject, objectToMap:transit.objectToMap, decoder:decoder.decoder, 
    UUIDfromString:types.UUIDfromString, randomUUID:util.randomUUID, stringableKeys:writer.stringableKeys, readCache:caching.readCache, writeCache:caching.writeCache};
  }

martinklepsch09:03:31

Ahh, just realizing that now, sorry!

thheller09:03:44

yep, if I remove that it works fine

thheller09:03:27

I guess it has some detection for module.exports and doesn't do the partial if (TRANSIT_NODE_TARGET) evaluation

thheller09:03:03

blame transit since that should never be in that file ... but it has always been in there ...

thheller09:03:57

I guess you could make a copy of that file and just put it on your classpath?

thheller09:03:30

just remove the two ifs at the bottom

martinklepsch09:03:27

Trying but getting

--- com/cognitect/transit.js:9
A file cannot be both an ES6 module and a script file that contains at least one goog.provide.

thheller09:03:18

in what context is this? WHEN do you get this?

martinklepsch09:03:08

[:server] Compiling ...
[:server] Build failure:
Closure compilation failed with 1 errors
--- com/cognitect/transit.js:9
A file cannot be both an ES6 module and a script file that contains at least one goog.provide.

martinklepsch09:03:29

Sorry, I just made the same mistake again. This appears in the shadow build process

thheller09:03:25

hmm weird. I'm not sure why this works when its in a jar but not a file?

thheller09:03:51

could be a shadow-cljs getting confused since it doesn't have a goog.provide or goog.module call

thheller09:03:07

dunno why this weird format was chosen

thheller09:03:52

ah! I think the mere presence of module.exports makes the closure compiler think this is commonjs

thheller10:03:04

so it doesn't use the default ESM

thheller10:03:13

but if you remove the module.exports thats gone and now it gets confused

thheller10:03:16

gotta love JS ...

thheller10:03:00

I don't have a simple solution. easiest is probably forking transit and rewriting it a use goog.module or actual closure-js. dunno why this goog.scope stuff was used

thheller10:03:07

but also not sure why this works for all the other files but not the transit.js. could totally be shadow-cljs just being confused, but I'm not sure why

martinklepsch10:03:21

Hm bummer... the road to ESM will be bumpy ๐Ÿ˜„

mkarp15:03:15

Hey there! Is there anything I can look at to speed up subsequent shadow-cljs release builds? I can see a significant difference between the calls (140s the first time, 80s after that) but I wonder if thereโ€™s something special to our codebase (too many macros?) that doesnโ€™t allow subsequent calls to be almost a no-op?

$ clj -M:shadow -m shadow.cljs.devtools.cli release app
[:app] Build completed. (4437 files, 4214 compiled, 0 warnings, 143.61s)

$ clj -M:shadow -m shadow.cljs.devtools.cli release app
[:app] Build completed. (4437 files, 25 compiled, 0 warnings, 80.61s)

thheller15:03:53

try with --verbose to see where actual time is spent. but my guess is that all leftover time is spent in optimizations

๐Ÿ‘ 1
thheller15:03:15

second pass uses cache, so it skips recompiling most of the files

thheller15:03:27

but optimizations are not incremental, so it cannot really be sped up

thheller15:03:06

4437 files is kinda nuts ๐Ÿ˜›

mkarp15:03:22

Yeah ๐Ÿ™‚

thheller15:03:23

otherwise probably worth checking the build report https://shadow-cljs.github.io/docs/UsersGuide.html#build-report

๐Ÿ‘ 1
thheller15:03:34

and look for stuff to maybe make the build smaller

mkarp15:03:38

Just checked --verbose, youโ€™re right, Closure - Optimizing ... (75233 ms) which is 80%. Pretty much no cljs file was compiled except a few with macros inserting contents from non-cljs files. I guess not much we can do to speed up the optimising step?

thheller15:03:17

> Pretty much no cljs file was compiled except a few with macros inserting contents from non-cljs files.

thheller15:03:20

what kind of content?

mkarp15:03:45

We have a custom solution generating clojure constants from css styles

thheller15:03:25

and are those inserted as EDN? deeply nested EDN data optimizes very poorly and slowly

thheller15:03:41

as in a macro ending up as (def styles {:huge {:nested "map"} ...})

thheller15:03:17

I'd recommend starting with the build report

๐Ÿ‘ 1
thheller15:03:36

files that remain large after optmization often also mean they took the most time to optimize

thheller15:03:44

unfortunately there are no more detailed stats available, so kinda hard to guess but I helped people before that inlined some rather big EDN data and after my recommendation to remove and fetch at runtime reduced build time by like 90%

wow 1
thheller15:03:44

not that you should ever even consider inlining anything that large (100kb+) to begin with

๐Ÿ‘ 1
mkarp15:03:52

> and are those inserted as EDN? deeply nested EDN data optimizes very poorly and slowly Iโ€™ll check! Thanks @U05224H0W

chaos18:03:47

Hello, I'm encountering an issue where an external npm package attempts to import lodash.flow using the import flow from 'lodash/flow' path syntax (using a slash), which is considered valid according to https://www.labnol.org/code/import-lodash-211117. I'm experiencing the same error when trying to do this from the starter-app. Could it be that this path require method is not supported in Shadow? Everything functions correctly otherwise when I use the lodash.flow syntax (with a dot). Thank you for your help.

The required JS dependency "lodash/flow" is not available, it was required by "starter/browser.cljs".

Dependency Trace:
	starter/browser.cljs

Searched for npm packages in:
	c:\clj\shadow-browser-exp\node_modules

thheller18:03:22

which lodash version do you use? its one of those packages that kinda constantly changes how its organized

thheller18:03:11

if there is a c:\clj\shadow-browser-exp\node_modules\lodash\flow.js it should be ok

thheller18:03:48

maybe you just don't have lodash installed at all and only lodash.flow, which is a separate package

chaos18:03:57

I'm using "lodash.flow": "^3.5.0", in my test app (original issue was seen in a dep package)

thheller18:03:25

well if you don't have the lodash package installed it obviously can't be found

thheller18:03:45

lodash/flow means its a file in the lodash package

chaos18:03:46

`ls -al node_modules/lodash.flow/ total 34 drwxrwxrwx 1 chaos None 0 2024-03-28 15:53 . drwxrwxrwx 1 chaos None 20480 2024-03-28 15:55 .. -rw-rw-rw- 1 chaos None 1951 2024-03-06 08:24 LICENSE -rw-rw-rw- 1 chaos None 439 2024-03-06 08:24 README.md -rw-rw-rw- 1 chaos None 11640 2024-03-06 08:24 index.js -rw-rw-rw- 1 chaos None 737 2024-03-06 08:24 package.json

`

thheller18:03:02

lodash.flow is an entirely separate independent package with no relation

thheller18:03:18

npm install lodash if you want to use lodash/flow

chaos18:03:24

I see, let me have another look at the dep package

chaos18:03:52

indeed, the dep package, although it only declares lodash.flow as a dependency, it does eventually pull in the lodash pkg. Thanks for you help, I'll pick it up from here