Fork me on GitHub
#shadow-cljs
<
2021-05-04
>
zimablue04:05:01

Hi, what is the recommended way if any to build something intended for a (node + typescript) consumption? I think the issue is that you need to do :keep-native-requires for node to work, but :keep-native requires leaves as you'd imagine, the requirement as "require" whereas typescript seems to hate that, throwing this error: WARNING in ../node_modules/datahike/datahike.js 112:12-19 vs-sprot-mm: Critical dependency: require function is used in a way in which dependencies cannot be statically extracted

thheller07:05:41

@zhuxun2 no, that is not supported and probably never will be

thheller07:05:14

@zimablue don't know what that error is about, would need to see some code. don't know what :keep-native-requires has to do with anything? when you are using node anyways you absolutely should not be using :js-provider :shadow?

zimablue07:05:46

I did not know that I should not be using that js-provider, sorry. In fact that is the one thign I never tried varying

zimablue07:05:53

what should I be using?

zimablue07:05:22

I pulled the first configuration that I tried from this thread: https://github.com/thheller/shadow-cljs/issues/290

thheller07:05:49

well, I need to know more about what you are building to answer

zimablue07:05:24

trying to compile datascript (cljs library) so that it works through an application that uses typescript to target a node runtime

zimablue07:05:45

in fact it's datahike, which is a fork

thheller07:05:50

so you use :target :node-library?

zimablue07:05:10

yes, although I've tried others in my scattershot attempts

thheller07:05:42

well, it sounds like :node-library would do what you need. if you describe what the problem with that was I can maybe help further

thheller07:05:04

completely basic config, no :js-options config whatsoever

zimablue07:05:18

I will strip out all the config and then come back

zimablue07:05:28

you recommend to use "release" right, not compile

zimablue07:05:54

does infer-externs matter? it is on auto currently

thheller07:05:38

you must use release if you intend to release the actual library to be used in projects. compile or watch will only work on the machine it was compiled on in the directory it was compiled in

thheller07:05:14

matter for what? it only affects the warnings you'll see for your code. nothing else

zimablue07:05:29

for working when consumed as a module by typescript I guess

zimablue07:05:40

I'll leave as-if and follow your advice

zimablue07:05:51

I think I will have already tried that combination, but it will at least give us the relevant error

thheller07:05:08

you don't need to set it at all, auto is the default nowadays (unless you are on a really old version)

zimablue07:05:43

I will set it just to let it know I'm serious

thheller07:05:16

it does not affect the code that is generated in any way so I'm unsure what you think it does

zimablue07:05:24

sorry, it was a stupid joke

thheller07:05:57

just wanted to be clear that it doesn't affect the generated code ๐Ÿ˜› it really only controls whether you see the warnings or not

zimablue07:05:20

with the settings that you suggested, the runtime error is this: $environ$core$fs$$.$existsSync$ is not a function. When I look at the code, it's trying to do existsSync on ".lein-env". I'm actually trying to run it in a vscode extension and now wondering whether vscode blocks raw node calls so I'm checking on that also

thheller08:05:37

that looks like you are maybe doing (def fs node-fs) with (:require ["fs" :as node-fs]) or something like that?

thheller08:05:31

for that you'll need to add manual externs I guess

thheller08:05:28

or just get rid of that entire library. it doesn't seem to be doing all that much ๐Ÿ˜›

Aron08:05:31

this is such an evergreen advice. I love libraries, because of this :)

thheller08:05:21

yes, maybe but unfortunately very many libraries are designed for CLJ and then someone just adds support for CLJS later as an afterthought

thheller08:05:28

that doesn't always work out well

Aron08:05:36

I remember the first time I realized how far cljs and clj are from each other. I almost broke down crying. ๐Ÿ™‚

Aron08:05:52

although, in this situation it seems to me there might be other assumptions going on that could be breaking https://imgur.com/a/rZzPfWl

zimablue08:05:31

yes, :require "fs" as fs is happening, both seemingly in that dependency and also in the library main

zimablue08:05:58

the problem is when this all gets pushed through a typescript=>node-electron thing though, it all works fine in shadow-cls node-repl

thheller08:05:27

@zimablue the problem is the library uses js/require so externs inference is "hindered" a little. that is why you are seeing the existsSync (and likely the others) getting renamed

zimablue08:05:48

so it's expected that it works in node-repl but not when compiled and pushed through typescript?

thheller08:05:51

if that were (:require ["fs" :as fs]) externs inference would be much smarter and generate the proper externs

zimablue08:05:12

ah, so it might just be the dependencies having that requirement that's the problem, not the project "main"?

thheller08:05:13

please take a second to understand what externs are

Aron08:05:20

js requires explicit exports, which is not kept without all this being in place

zimablue08:05:21

but "manual externs" should work?

thheller08:05:36

they ONLY prevent the renaming of stuff. when you run node-repl NOTHING is getting renamed since it does not run through :advanced compilation

zimablue08:05:38

I have taken a second before, it's taken me longer than that definitely xD from memory "variables which should not be renamed"

thheller08:05:46

when you run release things will get named and minified

thheller08:05:55

so with externs you prevent those renames

thheller08:05:32

externs-inference is meant to do this automatically for you but it can only do so when the code is written appropriately. which this library is not. so you need to help it along manually

Aron08:05:51

appropriate is es6 import/export?

thheller08:05:28

FWIW if you use the proper (:require ["fs" :as fs]) then shadow-cljs will automatically add externs for whereever fs/* is used. with (js/require ...) this is not done. basically since you are never supposed to use that ...

Aron08:05:09

thanks for getting back on this, sorry for causing confusion :/

Aron08:05:45

I can see why js/require is not helpful

thheller08:05:28

well often it is used as a "hack" to get a dynamic require sort of behavior. like that library does. from a build tool perspective this pattern is horrible but I understand why it is done.

thheller08:05:42

a simple ^js hint would also fix problems with this but the lib doesn't have it

Aron08:05:00

or some cljs convention

thheller08:05:13

there is no such thing as es6 import/export in CLJS, but lets please stay on topic here

thheller08:05:50

@zimablue what you need is a externs/your-build-id.txt and just one line for existsSync (and maybe more lines for other stuff getting renamed)

thheller08:05:17

that will make it work fine in release and typescript

thheller08:05:11

normally this should be fixed directly in the code but since it is a library you can't easily do that. unless you are ok with forking it

pez08:05:59

@zimablue have a look at the Clover extension and see how things are done there. iirc it uses shadow-cljs as the central part of the toolchain. I can also recommend using the #vscode channel for getting some pointers and help from others fighting a similar battle. ๐Ÿ˜ƒ

thheller08:05:46

I'd honestly just replace that library. Unless you actually want your electron app to be configurable via a .lein-env file?

zimablue08:05:54

It's not my personal project to pull libraries out of, I'll follow the explanation you gave, applied to this project and see where I get to

zimablue08:05:12

@pez, thanks for that advice I will take a look

Adam Kalisz14:05:21

@thheller it has been more than a month. I couldn't load my dev-environment in Chromium on Debian Linux and had to resort to working in Firefox without some of the useful tools. The loading of the project was so slow, it basically was just hanging. The issue seems to have been with :devtools {:loader-mode :script} which was there for some (maybe already obsolete) issue. Switching to {:loader-mode :eval} seems to fix the problem. Everything is snappy again

Adam Kalisz14:05:07

I don't know any details, my colleague found it to fix some of his performance issues and it helped me even more.

Prabu Rajan14:05:52

Hi, How do I import a react library component in a shadow-cljs / re-frame app? Below isnโ€™t working

//JS version
import { Button } from "@chakra-ui/react";
import { ChakraProvider } from "@chakra-ui/react";

class App extends Component {
  render() {
    return (
      <ChakraProvider>
        <Button
                px="3"
                py="2"
                bg="green.200"
                rounded="md"
                _hover={{ bg: "green.300" }}>
            Click me
        </Button>
        <h1>Hello World</h1>
      </ChakraProvider>
    );
  }
}

//cljs version

(ns todo.views
  (:require
   [re-frame.core :as re-frame]
   [todo.events :as events]
   ["@chakra-ui/react" :refer (ChakraProvider)]
   ["@chakra-ui/react" :refer (Button)]
   [todo.subs :as subs]))

(defn button-panel []
  [:div {:id "button-panel"}
   [:> Button {:px 10 :py 5 :bg "green.200" :on-click
     #(re-frame/dispatch [::events/update-counter 1])} "+"]
   [:> Button {:px 10 :py 5 :bg "green.200" :on-click
     #(re-frame/dispatch [::events/update-counter -1])} "-"]])

(defn main-panel []
  (let [counter (re-frame/subscribe [::subs/counter])]
    [:> ChakraProvider
     [:div
     [:h1 @counter]
     (button-panel)]]))

Prabu Rajan15:05:04

I was able to fix it like this

(ns todo.views
  (:require
   [re-frame.core :as re-frame]
   [todo.events :as events]
   ["@chakra-ui/react" :as cui]
   [todo.subs :as subs]))

(defn button-panel []
  [:div {:id "button-panel"}
   [:> cui/Button {:mr 1 :bg "green.200" :on-click
     #(re-frame/dispatch [::events/update-counter 1])} "+"]
   [:> cui/Button {:bg "green.200" :on-click
     #(re-frame/dispatch [::events/update-counter -1])} "-"]])

thheller16:05:54

["@chakra-ui/react" :refer (ChakraProvider Button)] would be ok too

Prabu Rajan17:05:35

Cool, works too, not sure what was missing earlier

thheller17:05:37

there seems to be a bug. you had the "@chakra-ui/react" require twice and the second one replaced the first one instead of adding to it

Prabu Rajan17:05:08

Yes, thats right. Thanks for pointing it out. You mean its a bug with shadow-cljs or the code above?

thheller17:05:25

bug in shadow

mss18:05:27

anyone had any luck getting react-pdf or generally pdfjs working nicely with shadow? feel like Iโ€™m mashing a square peg into a round hole, mostly because Iโ€™m not super familiar with pdfjs

zalky18:05:26

Hi all: when using :js-provider :external , is it required to have a node_modules installed before compiling with shadow? I'm getting an error without it:

The required namespace "react" is not available, it was required by "reagent/core.cljs".
I'm getting my js bundle via another source (not npm), and wondering if there's still some way to make it work with shadow.

thheller18:05:42

well the problem there is that reagent.core does (:require [react ...]) as in a symbol

thheller18:05:14

the only way to know that this symbol is referring to an actual npm library it needs to be in node_modules

thheller18:05:39

if that was (:require ["react" ...]) that check is not necessary since we know that strings are reserved for JS libs

thheller18:05:21

currently there is no way to tell it manually that certain symbols should be treated as a npm lib, well besides installing the actual npm lib

thheller18:05:24

I tried advocating for enforcing strings for JS requires but that feedback was ignored so we have this problem unfortunately

zalky18:05:55

Gotcha, the string require does make a lot of sense, it's basically js interop, no sense hiding it. I guess just assuming every missing dependency is externally provided is not feasible?

zalky18:05:01

I suppose not since you still have to check the cljs ones.

zalky18:05:00

It wouldn't make sense to turn it into a warning, would it? "This is missing, better be provided externally" something along those lines?

thheller18:05:45

I won't just make every symbol valid assuming its a npm dependency. there could be typos and stuff

๐Ÿ‘ 3
thheller18:05:21

I'd be open to adding a config option to treat certain symbols as npm requires without requiring their presence

thheller18:05:00

maybe I could also just use the :npm-deps from deps.cljs for that

lispers-anonymous20:05:06

Does shadow-cljs have a tool built in for printing out a graph of namespace requires? I am trying to figure out why cljs.pprint is being included in my production bundle, which I learned using the shadow-cljs.build-report tool.

dpsutton20:05:42

i don't remember where i heard/read it, but i have a memory that if you use the equivalent of format it will drag pprint in

lispers-anonymous20:05:34

I don't believe we are using any functions called format . Do you know what namespace that lives in?

dpsutton20:05:40

it had been a while. i looked it up and its cljs.pprint/cl-format so probably not the insight i remembered it to have been ๐Ÿ™‚

lispers-anonymous20:05:23

Yeah definitely not using that lol, I appreciate the thought though. I feel like exhausted all my options for finding the culprit

nivekuil20:05:26

expound, maybe from guardrails? if you use fulcro at all it'll get pulled in

lispers-anonymous20:05:20

Expound may be it actually. looking

lispers-anonymous20:05:05

Yeah that's it. I'm not using fulcro or guardrails, but we do have one usage of expound. This must be it. Thank you!

๐Ÿ‘ 3
thheller21:05:15

@dannyfreeman that graph does not currently exist no. requires a bit of manual digging currently to find it. do you use fulcro?

thheller21:05:39

ah nvm thread. yeah expound is one source

lispers-anonymous21:05:21

Yeah that was the only culprit thankfully. I can happily code split on the namespace where I use expound and get cljs.pprint out of my main bundle.