Fork me on GitHub
#shadow-cljs
<
2020-04-21
>
haywood00:04:43

I’m not finding a build hook that’s working for me, looking for some advice. I need to run some code that generates a new routes file (details not really important) based on files in a directory. This should really run before shadow does anything, but there’s not really a hook for that. I’m trying to repurpose my function to run on the :configure step, and the :compile-prepare step, but that’s not working so swell

justin01:04:37

@thheller thanks for the project 🙂. Was running into issues getting react and some npm react components to play nice together, and noticed some funky javascript output with fighweel. switched to shadow-cljs and it fixed my issue.

👍 4
haywood03:04:01

++ shadow is the gold standard @justtaft

thheller06:04:43

@haywood if you want something to run before shadow-cljs does anything then run it separately 😛

yenda08:04:35

I'm writing an override of require for node-test target how can I make sure it's added at the top of the js file

thheller08:04:58

override how?

thheller08:04:25

I'm assuming for all the custom mods we talked about last time?

thheller08:04:08

you can't really override actual node require

yenda08:04:28

(function(){
    var Module = require('module');
    var originalRequire = Module.prototype.require;

    Module.prototype.require = function(){
        // if react-native module return mock
        return originalRequire.apply(this, arguments);
    };

thheller08:04:20

right. I would keep this totally separate probably.

thheller08:04:05

and when you run node you run node --require=./your-require-override.js the-output.js

thheller08:04:13

you can make it part of the build also but I prefer to keep hacks in a place I can easily experiment with 😛

yenda08:04:53

but if I want to write it in cljs then I need to have a separate build config for it?

thheller08:04:21

write what in CLJS? the override mechanism?

yenda08:04:53

yeah I was thinking of giving it a shot since I already have all the mocks in cljs

thheller08:04:33

hmm the issue is that its not easy to ensure that its actually loaded before require is first used

thheller08:04:01

so I wouldn't recommend trying that

thheller08:04:43

it is easier in development because you can abuse :preloads but there is no :preloads-that-also-apply-in-release

yenda09:04:30

before I go further it seems like my first issue with the react-native modules in node is that they are using import

SyntaxError: Cannot use import statement outside a module

yenda09:04:51

it looks like the issue is that I get a require when it should be an import shadow.js.shim.module$react_native_languages = require("react-native-languages");

thheller09:04:59

yes. most react-native modules cannot be loaded in node. they are packaged with the assumption of going through metro first.

thheller09:04:13

I thought that is why you are mocking them?

yenda09:04:55

yes that is not a useful path to explore indeed, even if I manage to get them to run in node they will most likely yield odd results or errors

Aron10:04:00

so, i removed my question because I found in the docs the environment variable part but actually it's not what I need, or at least only part of it. I still don't know how I can read the value of anything I pass to shadow-cljs inside my app. I am happy to use --config-merge or anything else, but that's just setting the value, what is the keyword to search for to find how to read what I set there?

yenda10:04:20

@thheller looks like the solution with a separate js file works, on every error I mock the lib that fails and move to the next error. Since I already have the mocks in cljs and I find it more pleasant to write what would be the right approach? So far I have a frankenstein 😄 I put my mock cljs file in the path that is compiled by :test and I wrote an override.js that imports it, probably the worst possible way by copy pasting most of the shadow init code https://gist.github.com/yenda/eae7a492f2a7ea94cb4124ff5ef46f4a

yenda10:04:38

(it does work though)

thheller10:04:37

but in general I advise everyone against using environment variables in CLJS builds. just pass that data into your code at runtime instead

Aron10:04:52

@thheller what do you mean pass code at runtime

thheller10:04:14

pass the data

thheller10:04:47

assuming you are following the recommendations of having a init fn that is called when your code is loaded

thheller10:04:05

<script>your.app.init("with-the-data-you-need");</script>

thheller10:04:15

(defn init [the-data] ...)

thheller10:04:31

can pass the data in as a regular JSON object

thheller10:04:35

or edn encoded string

thheller10:04:43

whatever you want really

Aron10:04:26

how is going to help me to have different values for different builds

Aron10:04:12

The problem I am trying to solve is that there are a number of servers I need to connect to depending on the environment

Aron10:04:32

django runs on one port for me, another for others and definitely different port in production

thheller10:04:02

then you adjust whatever generates <script>your.app.init("with-the-data-you-need");</script>

Aron10:04:20

my devenv is devcards, thats' one port but when my boss checks my work, he will use the development build which is served by django in development mode which of course has it's own dev config

Aron10:04:34

so I need something that generates that, thanks

Aron10:04:44

we just use static files at the moment

thheller10:04:56

just have different html files ...

thheller10:04:16

index.html index-dev.html index-boss.html etc

4
Aron10:04:20

... that just exports the problem from shadow-cljs into django, I don't want to learn python 🙂

thheller10:04:51

if that pattern doesn't fit then use the env+closure-defines

Aron10:04:55

the thing is, no one wants to use cljs just me, if I have to commit code that changes everyone's codebase, there will be comments

thheller11:04:16

I don't have a clue what you are doing so you are going to have to do something that works for you

Aron11:04:31

I just realized something, I am not doing anything special, I created this project with create-cljs-app. There are no js files with init functions. I am trying to find how to pass data to my apps still, but so far only the closure-defines is the only thing that is somewhat similar, and even that I haven't yet been able to figure out how it actually expected to work. 🙂

Aron12:04:33

I think this create-cljs-app thing uses clojurescript to do this init calling thing since I have something like a main function exported and called with render, and the main function is configured in :init-fn. So if I understand your previous suggestions correctly, I should just write my configuration inside my core.cljs and cards.cljs files that call the render functions

thheller13:04:30

if the build config contains :init-fn some.ns/init then yes that calls init without args. you can will put the config into the html via <script>_CONFIG = "whatever";</script> and access it in into via (js/console.log js/MY_CONFIG)

Aron13:04:15

this also makes sense now, in hindsight.

Aron11:04:37

🙂 thanks for the help, I will figure out something

yenda11:04:54

I'm getting a maximal call stack size exceeded when running my tests

SHADOW import error /home/yenda/status-react/target/test/cljs-runtime/shadow.module.main.append.js
/home/yenda/status-react/node_modules/source-map/lib/util.js:343
function compareByGeneratedPositionsDeflated(mappingA, mappingB, onlyCompareGenerated) {
                                            ^

RangeError: Maximum call stack size exceeded
    at compareByGeneratedPositionsDeflated (/home/yenda/status-react/node_modules/source-map/lib/util.js:343:45)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:88:11)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)

yenda11:04:19

could it be because I have most of the SHADOW_IMPORT statements twice with my require override?

martinklepsch12:04:36

(let [react-dom-server (js/require "react-dom/server")]
      (.renderToStaticMarkup react-dom-server src))))
I’m just wondering, is this kind of approach something that Shadow’s npm module system supports?

thheller13:04:09

I don't understand the question? shadow-cljs does not bundle js/require calls in browser builds. if this runs in an environment that has an actual js/require then it would work but shadow-cljs has no hand in that?

martinklepsch13:04:50

This is for a node build. I guess I’m wondering if in node builds the ns require is needed for the npm module to be included in release builds? I hope that question makes more sense, sorry 🙂

thheller13:04:46

in node builds shadow-cljs doesn't include anything into the build

thheller13:04:11

it literally just writes require("a-thing") if you (:require ["a-thing" :as x])

martinklepsch13:04:09

got it, thanks!

thheller13:04:13

but yes ... if you want shadow-cljs to bundle things then it must be in the ns form. it does not pick up js/require calls in the code

thheller13:04:45

it could pick up js/require calls but that might give people ideas of using it dynamically and so on which won't work

martinklepsch13:04:50

uhm. what’s the difference between “bundle” and “include anything into the build”?

thheller13:04:22

in node builds in general it will create a regular .js file that still has require calls in the code

thheller13:04:31

node will take care of those and provide the dependencies at runtime

thheller13:04:49

in the browser we obviously can't do that so shadow-cljs does not generate require calls

thheller13:04:08

and instead bundles all the required code into the output directly so it can be loaded at runtime

thheller13:04:34

so a browser build would contain react and so on while a node build will not (by default)

martinklepsch13:04:00

Ok, I think that makes sense. Bundling isn’t really a (necessary) thing in node.

thheller13:04:56

you can still enable it if needed but it is off by default since node can just provide it

Aron13:04:52

I don't know what's with me and with clojre(script) tooling, but this continuous struggle of reading thousands of words with no results is really upsetting. I've given up for 8 years because of this, and I really feel like shadowcljs works, but to not be able to do something as simple as to set a build dependent variable for hours, despite getting help from the creator himself, is just upsetting^2. In js I know I have a bundler, browserify or webpack, I select the env plugin, use bash env variables in the terminal, then I can read them either through some module or on global/window. Not nice, but the whole process took less than 30 minutes the first time I had to do it. I've been trying to do this for 4 HOURS continously. I've read everything twice, I don't know what I missed but at this point I wish I could just quite and go fishing instead.

dpsutton13:04:46

{...
 :builds
 {:app
  {:target :browser
   ...
   :modules {:app {:entries []}}
   ;; to enable in development only
   :dev {:closure-defines { true}}
   ;; to enable always
   :closure-defines { true}
   ;; you may also enable it for release as well
   :release {:closure-defines { true}}
   }}

4
Aron13:04:20

as I described it above, the problem for me here is that this is INPUT, and I would rather read the variable, not just write it

thheller13:04:51

(ns )

(goog-define VERBOSE false)

thheller13:04:17

think of that like (def VERBOSE false)

Aron13:04:24

trying to (prn VERBOSE) I only got the value that I defined inline in the file, the config didn't overwrite it

thheller13:04:25

so you use it like any other def

thheller13:04:45

well how did you set it?

Aron13:04:58

the way you see it in the file at the root of this thread

thheller13:04:35

which one? those are multiple different examples rolled into one?

thheller13:04:22

do you use project.clj or deps.edn instead of just shadow-cljs.edn? in that case you might have the wrong clojurescript/closure-library versions in which case closure-defines don't work properly

Aron13:04:10

oh, i am so stupid

Aron13:04:27

of course it didn't work

Aron13:04:35

I tried to do two things at once. thanks again @thheller and @U11BV7MTK

yenda13:04:24

Is there a known cause for these sort of maximal call stack size exceeded errors?

Testing status-im.test.chat.models.message
SHADOW import error /home/yenda/status-react/target/test/cljs-runtime/shadow.module.main.append.js

/home/yenda/status-react/node_modules/source-map/lib/util.js:373
function strcmp(aStr1, aStr2) {
               ^
RangeError: Maximum call stack size exceeded
    at strcmp (/home/yenda/status-react/node_modules/source-map/lib/util.js:373:16)
    at compareByGeneratedPositionsDeflated (/home/yenda/status-react/node_modules/source-map/lib/util.js:354:9)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:88:11)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)
    at doQuickSort (/home/yenda/status-react/node_modules/source-map/lib/quick-sort.js:99:5)

yenda13:04:35

I run the tests with yarn shadow-cljs compile mocks && yarn shadow-cljs compile test && node --require ./test-resources/override.js target/test/test.js override.js is here https://github.com/status-im/status-react/pull/10217/commits/80e7685cc772b8e0aa08f6c9325aeed7266902d5#diff-31223e9c64942d5ed5214807141c977e

yenda14:04:16

this particular test didn't yield any error with the doo+lein stack. I also noticed that there was another test that was failing with the same kind of error, but in that case it was just a test that wasn't run with doo and was wrong, but instead of a proper assertion fail it was throwing this callstack error

thheller14:04:10

source-map also modifies require I think so you might just get in its way?

thheller14:04:49

try running against a release build. that won't include the source-map package (unless you do manually)

yenda14:04:46

interesting start with a warning in release build:

Resource: mocks/js_dependencies.cljs:47:1
 constant ReactNative assigned a value more than once.
Original definition at externs.shadow.js:2

yenda14:04:09

but yeah looks like it is related to sourcemap require because I see

Testing status-im.test.chat.models.message
ERROR in (add-received-message-test) (TypeError:NaN:NaN)
Uncaught exception, not in assertion.
expected: nil
  actual: #object[TypeError TypeError: lV.a is not a function]
DEBUG [status-im.native-module.core:259] - [native-module] is24Hour

yenda14:04:19

so probably some function call that wasn't mocked

thheller14:04:07

you can turn off source maps then it won't include that package

thheller14:04:18

just makes debugging a bit annoying 😛

yenda14:04:02

well it's tests anyway

yenda14:04:18

easier to debug

ERROR in (add-received-message-test) (TypeError:NaN:NaN)
Uncaught exception, not in assertion.
expected: nil
  actual: #object[TypeError TypeError: status_im.chat.models.message.add_message.cljs$core$IFn$_invoke$arity$2 is not a function]
than random max call stack exceeded 😄

thheller14:04:53

shadow-cljs release tests --debug also helps

yenda14:04:07

what does it do? I just get an extra warning about tufte

yenda14:04:24

it looks like it's because of the with-redefs in the test

thheller14:04:08

if you need with-redefs when you should set :compiler-options {:static-fns false}

yenda14:04:20

thanks! totally worked

Andrea Russo17:04:50

Did someone have success in importing react bootstrap in a shadow-cljs project?

haywood17:04:18

should just be able to npm install it and start using it right?

haywood17:04:30

reagent’s react interop is great, I’m using the EUI library just fine.

Andrea Russo17:04:53

npm installed react-bootstrap and I’m not able to do a simple (:require [“react-boostrap/Button” :default Button])

haywood17:04:14

:as Button?

Andrea Russo17:04:58

it seems that the react-bootsrap npm module is written in TypeScript

Andrea Russo17:04:05

full of .ts files

Andrea Russo17:04:23

@haywood yes, tried that too

haywood17:04:50

not to be overly pedantic, but I refer to this chart a lot https://shadow-cljs.github.io/docs/UsersGuide.html#_using_npm_packages

haywood17:04:30

I think the ts files are just the source files, if you look in the package.json it’ll point to the package’s main directory

haywood17:04:38

(in the node_modules folder)

Andrea Russo17:04:56

I’ve found the js files, in node_modules/react-bootstrap/esm

Andrea Russo17:04:29

but requiring [“react-boostrap/esm/Button” :as Button] doesn’t work

Andrea Russo17:04:01

Search in: /Users/arusso/Development/clojurescript/dias-web/node_modules You probably need to run: npm install react-boostrap/esm/Button

Andrea Russo17:04:39

maybe I can force the resolver by telling something like this?

Andrea Russo17:04:47

:js-options {:resolve {“react-bootstrap” {:target :npm :require “react-bootstrap/esm”}}}

Andrea Russo17:04:57

doesn’t work 😩

Aron18:04:57

I wouldn't want to use esm modules yet if I could avoid it. Is that package published using exclusively those?

Andrea Russo18:04:12

I don’t know

Andrea Russo18:04:17

I nee to check that

Aron18:04:44

in any case, the link above to the UserGuide explains how to load npm packages, the npm error saying that you have to iunstall react-bootstrap/esm/Button sounds wrong

Aron18:04:20

have you tried just ["react-bootstrap/Button" :as Button]?

Andrea Russo18:04:29

yes I tried it also

Aron18:04:34

and what was the error?

Andrea Russo18:04:57

always the same:

Andrea Russo18:04:58

Search in: /Users/arusso/Development/clojurescript/dias-web/node_modules You probably need to run: npm install react-boostrap/Button

haywood18:04:30

I’m gonna try it locally, have me curious

Andrea Russo18:04:51

I’m using this template project:

Andrea Russo18:04:21

lein new re-frame <project-name> 

Andrea Russo18:04:50

just do a npm install react-bootstrap

haywood18:04:27

eh I’m just installing into my project

haywood18:04:41

this package is pretty weird though looking at the node_modules directory

haywood18:04:11

yea just not working and I’m not sure why

haywood18:04:37

the package.json points to main: cjs/index.js, and if you go to that file it’s exporting everything

haywood18:04:15

["react-bootstrap" :refer (Button)]

Andrea Russo09:04:28

Did it work for you? I’m trying it but without success

Andrea Russo09:04:40

It works! Wonderful, I was sure I tried also that! Thank you very much @haywood

haywood18:04:51

you can do [Button] too

haywood19:04:00

I need to run a process that reads files from one of my apps directories, this process creates and writes a new clojurescript file to disk, which Shadow knob should I be using? It should run during startup and on changes.

klausharbo19:04:19

I’m quite new to Clojurescript but have quite a bit of experience with Clojure. I’m interested in interacting with Excel using @microsoft/office-js``. I use the shadow-cljs re-frame example for experiments and modified the namespace declaration in views.clj`` to be

(ns demo.views (:require [demo.routes :as routes] [demo.subs :as subs] [re-frame.core :as re-frame] ["@microsoft/office-js" :as office]))
and get the error
[:app] Compiling ... [:app] Build failure: module without entry or suffix: @microsoft/office-js
…
ExceptionInfo: module without entry or suffix: @microsoft/office-js
	Clojure.core/ex-info (core.clj:4739)
	clojure.core/ex-info (core.clj:4739)
	shadow.build.npm/find-package-require (npm.clj:207)
	shadow.build.npm/find-package-require (npm.clj:165)
	shadow.build.npm/find-file (npm.clj:387)
	shadow.build.npm/find-file (npm.clj:358)
	shadow.build.npm/find-resource* (npm.clj:618)
	shadow.build.npm/find-resource* (npm.clj:610)
Any hints to where I might start looking for the problem?

haywood19:04:47

it’s sorta saying you can copy the file in the dist directory to your static server, or you need to compile it yourself?

haywood19:04:20

easiest thing to do for now would be to `

<script src="/assets/office-js/office.js"></script>

haywood19:04:33

where you copy it to resources/public/js]

haywood19:04:22

and just refer to the global variable that file creates via js/office

thheller19:04:44

yeah microsoft things are very unfriendly towards packaging

thheller20:04:04

probably best to follow their instructions

klausharbo20:04:17

I’ll try that - thanks

haywood21:04:31

ok @thheller thanks for the tough love. I think I got what I needed by hooking into the :configure stage on startup, and then just running my own fs-watch that runs independently, so it’ll run even if shadow fails. Thanks