Fork me on GitHub
#shadow-cljs
<
2022-02-01
>
thheller07:02:35

@eliascotto94 move it to a foo.config namespace that you can reference everywhere. the problem is having this in foo.core so just move it out. I personally have a my.app.env namespace in every app for such purposes

🙌 1
thheller07:02:11

otherwise a circular reference is not allowed

shivekkhurana08:02:34

hey guys, I have run into a really weird issue, and am not able to figure out the root cause. In my setup, I'm importing a js source file (compiled to commonjs), directly inside my CLJS source. Th error is that a required module is not defined:

var s = _interopRequireWildcard(require("@solana/web3.js"));
^ this line is from a compiled file. This compiled file is from a package that the js code depends on. Somewhere deeper in the file when this function is used, I get an error saying that s is not defined. Has anyone faced this before? I've been stuck on it for the past 2 days. I have made sure that the module is present. I have also manually imported this module in my core file. Its present there (I'm able to console.log), but it isn't available for a js package that relies on it.

thheller08:02:15

so you mean you have this fine on the classpath imported via (:require ["/some/thing.js" :as x])? ie. not a file in node_modules

shivekkhurana08:02:46

Yes. This works fine in my app/core.cljs

thheller08:02:18

files on the classpath are meant to be ESM. it should not contain babel-rewritten commonjs

thheller08:02:42

standard ESM is fine

shivekkhurana08:02:55

I can convert the commonjs stuff to esm

thheller08:02:47

what is it converted from in the first place?

shivekkhurana08:02:49

its standard es6 with some JSX stuff

shivekkhurana08:02:02

so I use SWC to convert it to Commonjs

shivekkhurana08:02:09

and then use that source inside a CLJS app

thheller08:02:41

well commonjs should be fine in theory

shivekkhurana08:02:10

The problem is that the dep is available in cljs source (node module k can be required in cljs), but not available when js source (`(:require ["/some/thing.js" :as x])`) requires it. Ie thing.js requires it (ie node module k is not available in thing.js).

thheller08:02:05

that should be fine but I can't comment much without more info

thheller08:02:35

(:require ["@solana/web3.js" :as sol]) what is (js/console.log sol)?

shivekkhurana08:02:52

The console log si the package contents

thheller08:02:55

then it should be fine in the JS result

shivekkhurana08:02:34

Okay, thanks for the quick response.

shivekkhurana08:02:38

I'll checkout whats up.

thheller08:02:40

note that you can just output the JS source to packages/that-stuff and use :js-package-dirs ["packages" "node_modules"] in your build config

thheller08:02:50

then that-stuff can pretend to be a full npm package

thheller08:02:58

and you can (:require ["that-stuff"])

thheller08:02:07

it doesn't need to be on the classpath for that

shivekkhurana08:02:37

okay, this is news. I'll try this out.

shivekkhurana08:02:17

there is no docs on js-package-dirs ?

shivekkhurana10:02:22

I removed my custom js code, required the npm deps directly. The same problem occured in the dist file of the package. Then I used the same package in nextjs, it worked fine there 😭

shivekkhurana12:02:34

I finally forked the problematic package. It was using rollup under the hood to create a browser build. The terser plugin in rollup minified in a way that was incorrect for CLJS (and Dart). For some reason, it works fine for Nextjs based projects.

zimablue08:02:11

re the problem I had yesterday, I tested more and got EXTREMELY confusing results, they seem to imply the existence of shadow-cljs (or connected js tooling) global state outside of "yarn link" elements (which are in ~/.config/yarn/link/...). The only global state I can think of are global shadow-cljs installs and globally installed NPM modules but maybe there is more?

shivekkhurana08:02:20

in my case, I'm being naughty and sym linked a js package source to my project root. and then started requiring it in my cljs source 🤐

thheller08:02:21

shadow-cljs doesn't use globally installed npm modules. it just uses whatever is in node_modules in your project

thheller08:02:45

if yarn puts something global in there it is no longer global

zimablue08:02:00

jar: 2.16.12 cli: 2.16.12 deps: 1.3.2 config-version: 2.16.12 Does the "deps" here refer to the version specified in "deps.edn"?

thheller08:02:40

when using deps.edn nothing in info matters basically

thheller08:02:51

deps.edn overrides all of it

zimablue08:02:39

shadow-cljs doesn't use globally installed npm modules you say, is there still a "global" state in terms of whether shadow-cljs is "globally" installed

thheller08:02:14

no, no global state ever

thheller08:02:24

the global shadow-cljs install is just the actual shadow-cljs CLI tool. you can just remove it, it won't affect compilation in any way

thheller08:02:41

only trouble it can cause is using a global install that is out of sync with the local install too much

thheller08:02:51

ie. global install v1 but project install v2

thheller08:02:35

basically I recommend to not use the global install but I do so myself because of the convenience 😉

zimablue08:02:57

hahaha I respect that

zimablue08:02:03

the reason I started thinking "global state" was because two disconnected projects both failled and then succeed simultaneously when only one of them was having commands run/being modified

thheller08:02:17

define failed?

zimablue08:02:27

strange errors, the one I got above where I got a stack overflow in shadow.umd_helper.js was the initial problem, then after hacking away there was a point where compile was succeeding but the exports when "required" through vanilla nodejs cli were undefined

zimablue08:02:58

then after much hacking away config and dependencies to try and find the problem, it started working, but extremely weirdly the original project (I'd made a copy) also started working

thheller08:02:43

> I got a stack overflow in shadow.umd_helper.js

thheller08:02:48

got a stack overflow where? in what context? part of the compilation? part of running shadow-cljs?

zimablue08:02:19

I used yarn link which I thought was the problem but I purged all traces of it and got the same error later, unless it had snuck state into something, AFAIK the only shadow-cljs "state" is .shadow-cljs and .cpcache

thheller08:02:43

shadow-cljs only .shadow-cljs yes

zimablue08:02:44

when importing the node-library build using javascript in a node repl, when accessing a property of the defined module, got a stack overflow

zimablue08:02:49

so compile etc worked

thheller08:02:56

.cpcache is not something shadow-cljs ever looks at or uses

thheller08:02:27

it really is much easier if you give me stack traces and full errors

thheller08:02:24

ie. in the node-repl no JS is compiled by shadow-cljs and it just emits require("whatever") if you use a node library

zimablue08:02:34

and where, that's the name of some file that shadow-cljs generated, which seems to play a part in exports when making a node-library, full name is shadow.umd_helper.js, location in ($out_dir/cljs-runtime)

thheller08:02:35

so was that a node stack overflow?

zimablue08:02:52

node library, not node repl

thheller08:02:08

the node part implies the same, unless you override :js-provider

zimablue08:02:20

it was a node stack overflow when importing the file generated by shadow-cljs compile and then attempting to access one of the exported functions

thheller08:02:47

yeah sorry I can't help with this without more accurate errors

thheller08:02:58

happy to look at traces and stuff but this is going nowhere 😛

zimablue08:02:23

sorry for my poor communication, the error itself has mysteriously disappeared

zimablue08:02:36

probably I will accidentally summon it in a minute

thheller08:02:46

projects don't share any state ever. excepts maybe if you link the same npm dependency in the local node_modules

thheller08:02:03

then they share that npm dependency and if you modify that then all bets are off

thheller08:02:50

but they still won't share caches or whatever but if that package is doing something crazy then both projects will be affected by it

thheller08:02:33

umd_helper is just the wrapper for the :node-library output, so it'll be on the stacktrace but itself it doesn't do anything that matters

zimablue08:02:01

the stacktrace was all umd_helper though, whatever I'd done, there was some sort of circular reference being created inside it, otherwise how can you get a stacktrace like:

backend.test_file_store
Thrown:
RangeError: Maximum call stack size exceeded
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:5:16)
    at Object.get [as test_file_store] (/home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js:4:114)

thheller08:02:09

and what is in /home/zimablue/projects/kangrok-ui/.shadow-cljs/builds/node/dev/out/cljs-runtime/shadow.umd_helper.js?

thheller08:02:44

that should be an extremely simple file. I don't see how this could possible generate a stackoverflow?

thheller09:02:04

oh are you maybe trying to setup some :exports that exports itself?

thheller09:02:45

like setup npm packages in a way where (:require ["something"]) would be referring to itself?

zimablue09:02:12

that's not what I was trying to do, I was just trying to make a cljs with exports, the reason this all gets funky is that there's a stack of typescript depending on clojurescript depending on typescript

zimablue09:02:41

which is why I was trying to use "yarn link" in the first place, to eliminate some shell scripting parts of the build

thheller09:02:10

if you want deep integration use :npm-module

thheller09:02:23

:node-library is really meant to create a standalone actual node library

thheller09:02:50

if you setup circular stuff then there is nothing to prevent that

p-himik18:02:17

Stumbled upon https://github.com/funcool/promesa/issues/86 and decided to investigate for a bit. Could reproduce it only with shadow-cljs (at least on 2.16.12). This is enough to trigger the error:

(defprotocol P
  (a [_]))

(a (throw ""))
which results in:
------ WARNING #1 - :undeclared-var --------------------------------------------
 File: /home/p-himik/tmp/cljs-compiler-bug/src/a/b.cljs:6:1
--------------------------------------------------------------------------------
   3 | (defprotocol P
   4 |   (a [_]))
   5 | 
   6 | (a (throw ""))
-------^------------------------------------------------------------------------
 Use of undeclared Var a.b/ignore
--------------------------------------------------------------------------------
Maybe this line https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/build/cljs_hacks.cljc#L823 is simply missing the symbol ignore?

thheller18:02:33

teh fuck is that? 😛

thheller18:02:46

why would the symbol ignore be in that set?

thheller18:02:51

I don't have a clue where that ignore is coming from?

thheller18:02:56

can you share the repro? I'm curious

p-himik18:02:43

ignore is defined here: https://github.com/clojure/clojurescript/blob/a4673b880756531ac5690f7b4721ad76c0810327/src/main/clojure/cljs/analyzer/impl.cljc#L29 And it ends up being as a tag because of this, I think: https://github.com/clojure/clojurescript/blob/5e7fd72f7fa25e10241f229998e5620d79bc0aad/src/main/clojure/cljs/analyzer.cljc#L1537 The repro is literally those 3 lines that I posted, and a bare-bones shadow-cljs.edn. But I can create a repo with the repro if you want.

thheller18:02:23

still faster to just run npm install and not create all the other stuff 😛

thheller18:02:01

I've literally never seen the ignore stuff before when looking through analyzer code 😛

thheller18:02:14

you might be right though

p-himik18:02:35

And, of course, npm i && npx shadow-cljs watch main.

mauricio.szabo13:02:13

I can't believe that we're finally finding the source of that error! 🎉

mauricio.szabo13:02:41

I was already considering "one of the ClojureScript weird things that happen from time to time" and living up with it 😄

p-himik13:02:43

Just @ me when you find the next unexplainable thing. :D

Brandon Stubbs15:04:55

I am running into this issue now, was there any solution to this problem?

p-himik16:04:50

Can still reproduce on the latest shadow-cljs version, created https://github.com/thheller/shadow-cljs/issues/1112.

thheller17:04:44

and which code is triggering this? the repo is helpful to reproduce but quite pointless

thheller17:04:49

like what kind of actual code is triggering this?

p-himik17:04:08

You mean, code in shadow-cljs?

thheller17:04:45

no, user code causing this warning

thheller17:04:16

(a (throw "")) I don't know why this causes the warning, but this is code you'd never do in actual code, so why have it?

p-himik17:04:39

That's not a warning, that's an error. And the code is in the repo?.. > this is code you'd never do in actual code, so why have it? It's an issue when you're using a macro that expands into similar code. E.g. (or val (throw "")) wouldn't be an issue but a macro - like apparently Promesa's do - can write stuff like (or (a val) (a (throw "")).

p-himik17:04:05

Even if we were not able to come up with an explanation of why such code could be written, it is still syntactically valid, and apparently people do still stumble upon the issue, and only with shadow-cljs.

thheller17:04:51

well, if it always points at invalid code I'm fine with that

thheller17:04:36

that isn't to say that I won't look into it, just that it seems very low priority

p-himik17:04:49

But why is it invalid? CLJS accepts it, CLJ also does.

thheller17:04:03

not invalid, pointless

thheller17:04:21

its a throw, so why call a. it is unreachable code.

p-himik17:04:09

Because then every single macro that wraps its arguments in function calls will have to check that it's not a (throw ...) form.

thheller17:04:02

that is why I'm asking for a real world example of something the user might write

thheller17:04:12

I need context, I don't know whats going on and I want context

thheller17:04:32

if there is a macro generating bad code that is context, placing this in low priority for me

thheller17:04:44

if this is something that happens commonly, this is high priority

thheller17:04:54

just trying to understand the issue before I go digging into the compiler

p-himik17:04:24

@U015Y1A1N8Y Did you use exactly what's described in the summary of https://github.com/funcool/promesa/issues/86 ?

thheller17:04:46

CLJS happily accepts a bunch of bad code with no warnings, shadow-cljs is a bit stricter in many places

thheller17:04:59

so promesa is the only known cause? besides your minimal repro?

p-himik17:04:45

To me - yeah. Unless Brandon has stumbled upon this via some other means. And of course it's trivial to write a macro that looks useful but that would also be prone to this issue.

thheller17:04:33

I still don't have the slightest clue what the function of the ignore symbol is, so just trying to get context

p-himik17:04:25

Seems like the only current usage in CLJS is type inference of forms like

(if x
  ^boolean val
  (throw (ex-info "No x" {})))

Brandon Stubbs21:04:27

Sorry just got back to the computer now, for me it was also using the promesa library. But for my use case the code was:

;; [applied-science.js-interop :as j]
;; [promesa.core :as p]

(defn- throw-not-2xx
  [res]
  (if (<= 200 (j/get res :status) 299) res
      (p/let [{:keys [msg] :as body} (-> res .json js->clj)]
        (throw (ex-info msg body)))))

Brandon Stubbs21:04:32

I rewrote the function in another way to continue, but for this I get a response from fetch - I don’t want to touch the body only if it was an error status then I want to get the body to add info into the error

mauricio.szabo17:04:55

Came late to the party - but yeah, if you're using promesa your can't ever throw

mauricio.szabo17:04:00

Otherwise it'll cause this error. It's quite bad to not support this because there are some good cases for throw, like, you use p/let to await a couple of promises, then check the value of one and if it's invalid, throw

thheller17:04:00

fixed in 2.23.2

🎉 4
mauricio.szabo04:04:55

Amazing, as always! 👏

Mitchell Harris20:02:53

I’m getting an error running npx shadow-cljs release app

The required JS dependency "buffer" is not available, it was required by "node_modules/amplitude-js/amplitude.umd.js".

Dependency Trace:
	viderahealth/events/common_test.cljs
	viderahealth/events/common.cljs
	viderahealth/amplitude.cljs
	node_modules/amplitude-js/amplitude.umd.js

Searched for npm packages in:
	/Users/mitchellharris/src/patient-fe/node_modules

See: 
We do have shadow-cljs listed in our package.json, so It isn’t likely to be https://clojurians-log.clojureverse.org/shadow-cljs/2020-05-27 Any ideas?

thheller20:02:39

npm install buffer? it should already be installed but I guess its not?

Mitchell Harris20:02:09

Oh. I thought that was a node.js only package. That does work. 🤷 I was under the impression that was bundled with shadow-cljs so it wasn’t necessary. Sorry to bug you with such a trivial thing. Thank you.

thheller21:02:32

its not bundled with shadow-cljs but installing shadow-cljs in the project brings in node-libs-browser which does have buffer as a dependency. so it should be there. dunno why it isn't there

skelter21:02:53

should I expect using a string with a absolute path in a :require (edited) to work? Assert failed: (symbol? sym)

thheller21:02:27

@skelter in shadow-cljs yes. everywhere else no. also assuming by absolute path you mean this https://shadow-cljs.github.io/docs/UsersGuide.html#classpath-js

skelter22:02:23

@U05224H0W Yes, re 12.2.1 . Ok...so it should work. So something is weird with my use of it. Maybe the underscores in the filename. I'll start peeling away at it. Thanks.

skelter22:02:54

the simplest emptiest "/acme/myproto/a.js" worked. I think problem is something in my .js file

thheller07:02:23

what kind of js file is it? ESM works best. if it is closure JS (ie. goog.provide or goog.module) you must refer to it via a symbol as any other regular CLJS/Closure ns. (ie. (:require [acme.myproto.a])) matching whatever name is in goog.provide or goog.module

alex21:02:04

Hi there, I'm attempting to do some code splitting, but it seems like the loader is trying to load a module that I didn't specify. I have a configuration and source files similar to the below:

;; shadow-cljs.edn
{:builds {:client {:target     :browser
                   :module-loader true
                   :modules    {:client {:entries [my-app.client]}
                                :room {:entries [my-app.components.room]
                                       :depends-on #{:client}}}}}}

;; my-app.client
(require-lazy '[my-app.components.room :refer [Room])
client and room modules are generated in my output-dir and the module mapping file When I navigate to the page that loads the room module, I get a runtime exception from failing to load the module. However, it's not failing because it can't load room; rather it seems like shadow attempts to load a module with components moduleID when that module doesn't exist Is that because I have a namespace defined at my-app.components.room, and it expects the existence of a parent components module? I don't have a my_app/components.cljs file

ag21:02:30

what's the best way of feeding custom options (declared in shadow-cljs.edn) into the app, that can be read using cljs.env - http://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/env.cljc

thheller22:02:16

@rahx1t namespace names don't matter to modules, so no there is no such thing as a parent components module. I don't know what require-lazy is though? if that is emitting a module load for components then that is out of shadow-cljs control

alex22:02:18

awesome, thanks for confirming in that case, require-lazy might be the source of the bug and is extracting the wrong module name. it's from the Rum library: https://github.com/tonsky/rum/blob/9feb2a1781e3ef716361f9788150a01ea7f791b6/src/rum/lazy_loader.cljc#L31-L52

thheller22:02:49

yeah thats it

alex22:02:45

thanks! i don't have much experience with writing macros, but since i know the expected output, i'll try to figure the bug and patch it. appreciate your help!

thheller22:02:55

doesn't look like a bug to me. looks quite intentional

thheller22:02:39

just seems to assume that your namespace structure actually matches your module structure

alex22:02:49

Ah interesting. In your experience, would you suggest I add a my-app.components namespace that re-exports all of the components from my-app.components.*? Then create an associate :components module?

thheller22:02:44

no. that doesn't make sense to me at all but I didn't write that macro

thheller22:02:10

but that also has a helper macro

alex17:02:24

thanks for pointing me to that!

alex17:02:53

I appreciate not having to call (set-loaded! :module-id) if I use the macro you shared

thheller22:02:01

@ag I don't know what you mean by that. if you are building a self-hosted build then there is no way to feed anything from shadow-cljs.edn into that. what are you trying to get there?

ag22:02:51

I just need some custom config options things like :base-api-root, etc. I'm not sure what's the best way of doing something like this.

ag22:02:48

Nevermind. I think I've found "Using Environment Variables" in the documentation. Will try that

thheller22:02:32

its unclear why you are referring to cljs.env. that is something from the compiler. you wouldn't use it in a regular build

thheller22:02:40

its not about environment variables at all

ag22:02:45

yeah, sorry about the confusion. I thought that's one of the ways how it can be done. I initially found this gist: http://gist.github.com/metametadata/bb00917e09463f6ce04f0e50ccc0740a

thheller22:02:48

ah ok. yeah that works too but only in macros

thheller22:02:03

:closure-defines and environment variables are probably easier

ag22:02:39

I'm still confused. How do I access those vars from the app?

thheller22:02:08

(ns some.ns) (goog-define FOO "default-value")

thheller22:02:22

think of goog-define as def so (def FOO "default-value")

thheller22:02:28

you use it like any other def

thheller22:02:35

but you can set the value from the build config

ag22:02:50

I still can't seem to get it right. I set in shadow-cljs.edn: [:builds :app :closure-defines] to something like this: {:base-api-root #shadow/env ""} Now what should I do?

ag22:02:40

(goog-define) is in Clojurescript file? It's not working for me in Cljs repl

thheller22:02:42

yes, it should be in a file

thheller22:02:17

{:base-api-root #shadow/env ""} this is not quite right. this will look up the environment variable called ""

thheller22:02:32

and the key must be a namespaced symbol

thheller22:02:49

so my example from above some.ns/FOO

ag22:02:40

Yay, it worked. Thank you Thomas!