Fork me on GitHub
#shadow-cljs
<
2021-04-15
>
Vincent Cantin05:04:06

I am compiling 2 independent CLJS projects A and B with shadow-cljs, and load them both into the same browser webpage as compiled JS scripts. I am trying to make A read a CLJS hashmap produced by B. The transmission of the data is made via JS property js/window.b-data. This part is working. Now, when A runs (get b-data :foobar) it returns nil. The keyword is not found because :foobar in A is different than :foobar in B.

;; We are in A's runtime.

(def b-foobar (first (keys js/window.b-data)))

(keyword-identical? :foobar b-foobar) ;; false
(identical? (.-fqn k) (.-fqn :ui/alert)) ;; true
(keyword? b-foobar) ;; false
The problem of (get b-data :foobar) is that b-foobar fails keyword?.
(core/defmacro keyword? [x]
  (bool-expr `(instance? Keyword ~x)))
Maybe Keyword is a different in A and in B ? Would there be a way with Shadow-CLJS to make b-data pass the predicate keyword? ?

Vincent Cantin06:04:50

I found the definition of Keyword inside cljs/core.js :

cljs.core.Keyword = (function (ns,name,fqn,_hash){
this.ns = ns;
this.name = name;
this.fqn = fqn;
this._hash = _hash;
this.cljs$lang$protocol_mask$partition0$ = 2153775105;
this.cljs$lang$protocol_mask$partition1$ = 4096;
})
Since both compiled CLJS runtime are using a different function (the same implementation but different "instance"), it makes sense that the instance? fails.

Vincent Cantin06:04:01

I guess that there is no way for shadow-cljs to help, here.

thheller06:04:57

yeah what you are trying to do is not possible

thheller06:04:30

and I strongly recommend against doing it at all, two separate builds means two separate cljs.core and whatever other dependencies they might be sharing

thheller06:04:35

lots more for the user to download

thheller06:04:52

if you need to share data you are doing to need to use transit/edn

thheller06:04:10

but ideally don't have separate builds in the first place

Vincent Cantin07:04:40

My use case is Cypress tests (`A`) with my webapp (`B`) via https://github.com/viesti/cypress-clojurescript-preprocessor

Vincent Cantin07:04:00

I try to read the values in re-frame.db/app-db

thheller07:04:52

then I'd recommend just having the test build include your regular app?

thheller07:04:58

or create an exported function in your main app that you can call to get the DB as transit or so

馃憤 3
thheller07:04:39

including two separate development builds at the same time is guaranteed to cause issues as well

thheller07:04:06

two release builds can work but even they might cause problems with each other

Vincent Cantin07:04:34

I succeeded using a hack, but it is very brittle

(defn rewrite [x]
  (walk/prewalk
    (fn [x]
      (case (pr-str (type x))
        "cljs.core/Symbol" (symbol (.-ns ^js x) (.-name ^js x))
        "cljs.core/Keyword" (keyword (.-ns ^js x) (.-name ^js x))
        x))
    x))

thheller07:04:38

you are basically asking for trouble, nothing will work as expected. don't do this. I can't do more than warn you 馃槈

馃槄 3
Vincent Cantin09:04:07

I tried your approach and exported a function which spits the db as edn in a string. It works very well. Thank you !

Karol W贸jcik09:04:43

1. How can I provide "react" and react-dom using :provider :external and using webpack.ProvidePlugin? This is what I've tried:

const path = require('path');
const webpack = require('webpack');

module.exports = {
    entry: './public/js/index.js',
    mode: 'development',
    output: {
        path: __dirname + '/public/js',
        filename: "libs.js"
    },
    devServer: {
        stats: 'errors-only',
        contentBase: path.resolve(__dirname, '/public/js'),
        publicPath: '/'
    },
    plugins: [
        new webpack.ProvidePlugin({
            react: 'react',
            ReactDOM: 'react-dom',
        })
    ],
    module: {
        rules: [
        ]
    }
};
Still getting:
app.js:1563 Error: Dependency: react-dom not provided by external JS. Do you maybe need a recompile?
    at shadow$bridge (index.js:14)
    at eval (shadow.js.shim.module$react_dom.js? [sm]:3)
    at eval (<anonymous>)
    at Object.goog.globalEval (app.js:497)
    at Object.env.evalLoad (app.js:1560)
    at app.js:1699
env.evalLoad @ app.js:1563
(anonymous) @ app.js:1699
11:27:52.383 
2. Btw: Specifying global require like (:require ["react"]) does not work. It has to be (:require ["react" :as react]). Why?

Karol W贸jcik09:04:45

Actually 2. is more suprising for me.

thheller10:04:10

don't know why you are messing with provideplugin?

thheller10:04:15

just compile the file with webpack without any plugins or special config

thheller10:04:34

the external index js does not require that, only if you add additional custom js you may need it for that?

Karol W贸jcik10:04:59

Stopped messing with providePlugin. I've used something like this instead:

(ns web.globals
  (:require
   ["react" :as react]
   ["react-dom" :as react-dom]))
Still I don't understand why it needs this :as 馃槙

Karol W贸jcik10:04:24

I need webpack config. I want to tree-shake antd-design.

Karol W贸jcik10:04:09

@thheller Could you please elaborate what you mean by custom js?

Karol W贸jcik10:04:16

Ok. I think I found a bug in shadow$bridge. Without :as the dependency is not registered. Shouldn't (:require ["react"]) generate require('react')? in external-provider file? This is with :as:

// WARNING: DO NOT EDIT!
// THIS FILE WAS GENERATED BY SHADOW-CLJS AND WILL BE OVERWRITTEN!

var ALL = {};
ALL["@ant-design/icons"] = require("@ant-design/icons");
ALL["antd/lib/divider"] = require("antd/lib/divider");
ALL["antd/lib/input/Search"] = require("antd/lib/input/Search");
ALL["antd/lib/layout/Sider"] = require("antd/lib/layout/Sider");
ALL["antd/lib/layout/layout"] = require("antd/lib/layout/layout");
ALL["react"] = require("react");
ALL["react-dom"] = require("react-dom");
global.shadow$bridge = function shadow$bridge(name) {
  var ret = ALL[name];

  if (ret === undefined) {
     throw new Error("Dependency: " + name + " not provided by external JS. Do you maybe need a recompile?");
  }

  return ret;
};

shadow$bridge.ALL = ALL;
This is without :as:
// WARNING: DO NOT EDIT!
// THIS FILE WAS GENERATED BY SHADOW-CLJS AND WILL BE OVERWRITTEN!

var ALL = {};
ALL["@ant-design/icons"] = require("@ant-design/icons");
ALL["antd/lib/divider"] = require("antd/lib/divider");
ALL["antd/lib/input/Search"] = require("antd/lib/input/Search");
ALL["antd/lib/layout/Sider"] = require("antd/lib/layout/Sider");
ALL["antd/lib/layout/layout"] = require("antd/lib/layout/layout");

global.shadow$bridge = function shadow$bridge(name) {
  var ret = ALL[name];

  if (ret === undefined) {
     throw new Error("Dependency: " + name + " not provided by external JS. Do you maybe need a recompile?");
  }

  return ret;
};

shadow$bridge.ALL = ALL;

thheller11:04:17

first of all regarding the :as. without it is is basically unusable so I don't understand why that is even a question?

thheller11:04:11

what is the overall goal for this?

thheller11:04:12

> I want to tree-shake antd-design

thheller11:04:36

what does that mean for you? I mean you already have separate requires which basically is tree shaking already?

thheller11:04:41

I'm unsure why the external index doesn't include requires without :as and that could be considered a bug but I'd still like to understand why you don't want the :as? I mean you do intend to use react right?

thheller11:04:50

by custom JS I mean JS code you write that you want to compile with webpack and include alongside the external index

Karol W贸jcik11:04:30

@thheller Ok let me explain myself a little bit. I don't know whether I will ever use "react" in the code like react/createElement or so. "react" and "react-dom" are only required so that I can use reagent, therefore requiring "react" :as react is now something I don't need. (it's just triggers clj-kondo warning) Regarding ant-design yes you're right. Now it's treeshaked since I'm directly specifying the path to component, but I would rather use ant plugin for webpack and simply refer the components I want. Another story is including css which I also plan to do with webpack. At some point I want to require less in shadow-cljs like so: (:require ["./core.less"]). Don't know whether it will work with shadow-cljs. https://ant.design/docs/react/use-with-create-react-app

thheller12:04:10

ok so. reagent.dom will be including react-dom so if you don't plan on using it in your code then don't

Karol W贸jcik12:04:47

Ahhh.. I think I got you!

thheller12:04:56

tree shaking in webpack is based on the import statements but since the shadow-cljs external index generates require that won't work

thheller12:04:35

I did start to implement with supporting :refer to emit import instead but that'll have a few compatibility issues so I used require for now

thheller12:04:03

(:require ["./core.less"]) will not work since classspath relative imports will always be processed by shadow-cljs directly and only supports JS

Karol W贸jcik12:04:10

Ok I will somehow live with using full path for importing ant components. Is it possible to somehow bypass shadow-cljs so that "./core.less" will not be processed by shadow-cljs?

thheller12:04:50

in ns :require no

Karol W贸jcik13:04:35

Oh. But still I can use js/require am I right? 馃檪

Karol W贸jcik13:04:37

Probably import is special statement and cannot be used like js/require 馃槃

thheller13:04:32

adding either import or js/require is not valid

thheller13:04:48

since ONLY the external index file will be processed by webpack

thheller13:04:57

not the CLJS parts

thheller13:04:16

I could add an extra mechanism to tell webpack to also include other stuff

thheller13:04:24

but it will not be part of ns :require

thheller13:04:30

could be ns metadata or so

thheller13:04:06

would be much easier to talk about this if you setup a demo repo of what you'd like to have

thheller13:04:28

doesn't need to be functional but something code wise we can talk about. its all a bit abstract in a slack thread

Karol W贸jcik13:04:36

Sure. I will provide a demo.

Karol W贸jcik08:04:33

@thheller Here is a demo of what I'm trying to achieve: https://github.com/FieryCod/ant-design-in-shadow-cljs Basically I want to import less, and scss files into .cljs so that webpack could handle compilation of both.

Karol W贸jcik11:04:42

Yep. It's needed by reagent. Alternatively I can put those line in app.cljs.

thheller11:04:25

it is NOT needed by reagent

thheller11:04:28

what gives you that idea?

thheller11:04:18

I'd like to understand why you are repeating it?

Karol W贸jcik11:04:15

Ok. Just please remove those lines. You will see that shadow does not include react.

Karol W贸jcik11:04:21

We're going through the stuff I already described above.

thheller11:04:30

ok so a couple notes

thheller11:04:16

but you only seem to want to include it for the side effects right? it is never actually used anywhere

thheller11:04:28

the problem is that webpack is not aware of the classpath

thheller11:04:43

so the path you include that way needs to be relative from the output file, not the source file

thheller11:04:05

and the point of this repo is that you want to replace ["antd/lib/divider" :default AntDivider] with just ["antd" :refer (Divider)] or whatever that would be?

thheller11:04:56

or is it to get less support?

Karol W贸jcik11:04:00

I want to 1. Replace ["antd/lib/divider" :default AntDivider] with ["antd" :refer (Divider)] 2. Include less, scss via webpack.

Karol W贸jcik11:04:10

I want both 馃槃

thheller11:04:24

and antd doesn't require any additional setup to support this? thought it required a plugin to do that?

thheller11:04:43

yes but it isn't setup anywhere. I'm trying to figure out what is actually needed from the shadow-cljs side

Karol W贸jcik11:04:47

There are two things. 1. Generate imports not require (you said that it requires some work therefore I'll probably wait for it) 2. Allow to require assets in external provider.

Karol W贸jcik11:04:38

I will add babel-loader and babel-plugin-import to project in couple of hours.

thheller11:04:55

as for requiring assets

thheller11:04:50

you are now using webpack to process public/js/index.js. you could write a manual public/js/index-with-styles.js or whatever that just does require("./index.js"); require("./ant.less") etc

Karol W贸jcik11:04:53

I'm wondering whether it's possible to create a .js file which I import in shadow-cljs that would point to assets.

thheller11:04:02

so not sure that is really something shadow-cljs needs to do

thheller11:04:11

generating imports is a problem for one simple reason. the convention is that commonjs packages such as react must be imported as import React from "react" which in JS terms is a singular default export

thheller11:04:29

but all CLJS code uses [react :as react] so :as**

thheller11:04:40

which would be import * as react from "react"

thheller11:04:09

emitting an ESM import is difficult for that reason

thheller11:04:17

the problem is that pretty much all CLJS libs break because of their now "incorrect" requires

thheller11:04:05

for the code you control you could easily just switch the references

thheller11:04:24

but that is not so easy for library code such as reagent

thheller11:04:43

so the only reason it uses require in the first place is compatibility

thheller11:04:20

I don't really have a solution for this but the problem will be present until all JS libraries uniformly written in ESM

thheller11:04:27

and everything will likely change again once packages such as antd adopt https://webpack.js.org/guides/package-exports/

thheller11:04:11

that react via reagent alone doesn't work is a bug

Karol W贸jcik11:04:10

I've forgotten how much js world is a mess by working with cljs for 2 years.

Karol W贸jcik11:04:39

Totally understandable why you're resisting to adopt asset require in shadow-cljs.

thheller12:04:58

well the external code already has partial support for (ns {:external/assets ["./foo.less"]} (:require ...))

鉂わ笍 3
thheller12:04:27

it is useful to be able to express such things in code, just doing it via require is bad IMHO

thheller12:04:46

but JS doesn't have namespaces or metadata so they have a harder time to get something like this 馃槢

Karol W贸jcik12:04:44

:external/assets are great! Love it

pinkfrog13:04:51

@thheller reading your post: https://code.thheller.com/blog/shadow-cljs/2019/08/25/hot-reload-in-clojurescript.html. What will be reloaded when a cljs file changes. Will the ns containing the init-fn be reloaded, of the ns just containing the modified code.

thheller13:04:02

the ns you changed + all those that have a direct require for that one

pinkfrog13:04:12

so are you saying the ns has in-direct require to the changed ns won鈥檛 be re-required ?

pinkfrog14:04:55

I see: shadow-cljs聽will only automatically recompile the direct dependents since in theory dependencies further up cannot be directly affected by those interface changes.

thheller16:04:09

> On the backend the shadow-cljs watch app process will compile a namespace when it is changed and also recompile the direct dependents of that namespace (ie. namespaces that :require it). This is done to ensure that changes to the namespace structure are reflected properly and code isn鈥檛 using old references.

thheller16:04:23

you can change the reload strategy to reload all files or set :dev/always metadata if you want to force recompile a namespace on every change

thheller13:04:43

init-fn ns only if that has a direct require, otherwise no

thosmos19:04:11

I鈥檓 getting some errors when watching my project after updating the version to 2.12.5. Not sure how to troubleshoot this:

[2021-04-15 12:37:05.387 - WARNING] :shadow.cljs.devtools.server/nrepl-ex
CompilerException Unexpected error macroexpanding if-ns at (cider/piggieback.clj:22:1). 
...
[2021-04-15 12:44:11.955 - WARNING] :shadow.cljs.devtools.server.util/handle-ex - {:msg {:type :start-autobuild}}
NoClassDefFoundError Could not initialize class cljs.repl__init

thheller19:04:10

@thosmos version conflict. not using the updated 1.10.844 CLJS with 2.12.+ won't work

thosmos19:04:34

OK thanks will upgrade to that

thosmos20:04:53

That worked