Fork me on GitHub
#shadow-cljs
<
2021-04-13
>
dvingo04:04:54

is there a way to apply preloads only to some modules and not all. I am trying to create a web-worker, but the preloads assume the main thread js environment is present and are resulting in an exception when the worker starts.

thheller06:04:46

@danvingo you can set :preloads per module yes. just {:modules {:main {:init-fn :preloads [foo.bar]}}}

dvingo12:04:57

Thanks! That worked, but these are dev-time only preloads, is there a way to include them only during dev and not a release? I tried nesting them under :dev but they aren't being included now:

{:modules {:main { :dev 
;; moving to here:
{:preloads [...]} }
          :worker{}}
:devtools          {:after-load         app.client.client-entry/refresh
                                          :http-port          8044
                                          :http-resource-root "public"
                                          :http-root          "resources/public"
;; moving from here
                                          :preloads           [com.fulcrologic.fulcro.inspect.preload
                                                               app.client.development-preload
                                                               com.fulcrologic.fulcro.inspect.dom-picker-preload]}}

thheller13:04:53

:preloads are ALWAYS developmen only and do not apply to release builds

thheller13:04:00

so just put it into modules like I said

dvingo13:04:51

perfect, thank you!

grounded_sage12:04:39

I have been digging through old Clojurians messages and issues on Github related to connecting a repl to different runtime environments we need to deal with when doing Chrome extensions. Still unclear to me if this is possible... let alone how to do it. It seemed that @wilkerlucio mentioned that he had it working on background and devtool but not content-script. Though I have not been able to get any working. Wondering what the current state of this is and whether it is worth investing any more time in trying to get it working. Seems that there is a lot of nrepl and sourcemap related issues..

pinkfrog13:04:40

Using shadowcljs$aot version, constantly seeing this error:

[:mobile] Build completed. (203 files, 91 compiled, 0 warnings, 13.29s)
Exception in thread "async-dispatch-2" java.lang.IllegalArgumentException: No implementation of method: :close! of protocol: #'clojure.core.async.impl.protocols/Channel found for class: nil
        at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:583)
        at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:575)
        at clojure.core.async.impl.protocols$eval1077$fn__1078$G__1066__1083.invoke(protocols.clj:21)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:979)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:975)
        at clojure.core.async$ioc_alts_BANG_$fn__3907.invoke(async.clj:384)
        at clojure.core.async$do_alts$fn__3839$fn__3842.invoke(async.clj:253)
        at clojure.core.async.impl.channels.ManyToManyChannel$fn__1491.invoke(channels.clj:265)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:832)

thheller13:04:45

@i that means you are also using a conflicting too old core.async version as a dependency

thheller13:04:10

with all AOT code you basically need to have the version that the AOT code used, otherwise you'll get errors such as that

pinkfrog12:04:47

I wonder why that makes a difference. With or without aot, I thought using an old library will both fail, not aot alone.

thheller12:04:12

just an artifact of how clojure AOT code works, doesn't mean the non AOT code will work. it might fail at a different point in the code and not directly on load

thheller12:04:39

might also just work but either way you should ensure to use a compatible version

thheller13:04:26

@grounded_sage I cannot help without seeing your config or what you are actually doing. chrome extensions are rather difficult since they do have all sorts of restrictions but the repl and hot-reload do work. I cannot tell you more since I don't know what you have tried. this is still pretty much the only "documentation" that exists https://github.com/thheller/shadow-cljs/issues/279

grounded_sage13:04:58

Here is the shadow-cljs.edn and the manifest.edn @thheller. Currently I get a tonne of few websocket errors that are thrown in the console and a bunch of sourcemap errors. https://github.com/homebaseio/datalog-console/blob/extension/shadow-cljs.edn https://github.com/homebaseio/datalog-console/blob/extension/shells/chrome/manifest.edn

grounded_sage13:04:53

Have mostly been aligning with the file structure and set up done for fulcro-inspect so that it is easier to reference across the two and see differences.

thheller13:04:00

then you are likely just blocking yourself with your CSP rules?

thheller13:04:25

I can never remember the exact rules for CSP but it needs to be able to connect to localhost:9630 (the shadow-cljs instance server)

grounded_sage13:04:32

ah sorry on my local that is set to 9630. Must have been a recent change I did not push.

thheller13:04:42

I mean if you get warnings/errors already then they are the hint to fix it as well

thheller13:04:02

you can paste some and I can maybe tell you what they mean

grounded_sage13:04:30

Okay. Seems that changing the CSP url to be the same as devtools-url removes the ws errors that are thrown. But this makes it different to what is happening in fulcro-inspect. Which has 9632 in the devtools-url and then 9630 in the CSP rules.

grounded_sage13:04:41

I get this when I open up the chrome console inside of the chrome-panel. Unchecked runtime.lastError: Could not establish connection. Receiving end does not exist.

grounded_sage13:04:05

In both background (when clicking the url of extension inspect views) and the chrome panel I get a lot of source map loading errors DevTools failed to load source map: Could not load content for : HTTP error: status code 404, net::ERR_UNKNOWN_URL_SCHEME

thheller13:04:05

well is it actually running on 9630 or 9632? it is logged on startup

thheller13:04:56

well if you have multiple shadow-cljs instances running locally and 9630 is taken it'll use 9631 then 9632 and so on

thheller13:04:13

don't use :devtools-url at all if you want to avoid that headache

thheller13:04:25

ah wait no you have to

thheller13:04:41

but yeah use your port ...

thheller13:04:54

for source maps try :source-map-inline true in :compiler-options

grounded_sage14:04:45

:source-map-inline true worked for chrome-extension target but doesn't seem to be working for chrome target https://github.com/homebaseio/datalog-console/blob/extension/shadow-cljs.edn#L21. Though perhaps I have something else configured wrong here? I also seem to be able to connect to the background runtime with the repl when I set my devtools-url to 9630. When the devtools-url is set to 9632 I get WebSocket connection to '' failed: followed by shadow-cljs - remote-error several times before it gives up trying to connect.

thheller14:04:57

I do not understand why you are trying to connect to 9632. shadow-cljs ONLY starts ONE port. there is no such extra magic port with regards to chrome-extension

thheller14:04:25

so stick with 9630 if that is the port you are starting shadow-cljs with

grounded_sage14:04:26

I was just following the configuration from fulcro-inspect

thheller14:04:45

but it clearly doesn't apply to your setup. it can only connect to a port that actually exists, so don't configure one that doesn't

thheller14:04:37

as for the REPL connection. open http://localhost:9630/runtimes

thheller14:04:49

that will list all the connected runtimes with their respective ids

👍 2
thheller14:04:04

you can switch to a specific one but the default rule is to connect to the first one that connects

grounded_sage14:04:23

okay. We are doing similar things. Plan to also build an electron app so most of what was done there I thought would apply.

grounded_sage15:04:27

Thanks for pointing out the url for open runtimes. Was a big help. I am able to connect to them with the repl now. Only thing left is source-maps not loading in the panel. But I think this is a path related issue on my end.

thheller15:04:09

inline should fix it but might be a bug in the :chrome-extension target not supporting it properly or something

grounded_sage17:04:34

yes it fixed it for this one actually. It was the :browser target for the panel UI that is still throwing the source map error.

thheller15:04:15

don't know, can't check right now

lispers-anonymous16:04:50

I have a question about an issue I've run into with the :output-feature-set :es5 setting with shadow-cljs. When my compiled javascript is loaded I get an error that says

Uncaught TypeError: $jscomp.arrayFromIterable is not a function
    memoizedFn memoize.js:60
    setEnabled history.js:503
    <anonymous> events.cljs:458
    re_frame$fx$do_fx_after fx.cljc:59
    ... blah blah ...
    <anonymous> babel-polyfill.min.js:1
    m babel-polyfill.min.js:1
    b babel-polyfill.min.js:1
Using the browser console, I can see that the $jscomp object is defined, but the function arrayFromIterable is not defined on it. Other functions are. I was able to fix this by defining the function in a :prepend statement for my module.
:prepend "$jscomp.arrayFromIterable = function(a) {return a instanceof Array ? a : $jscomp.arrayFromIterator($jscomp.makeIterator(a));};"
It fixed my problem. My javascript loads but it feels dirty. Is there another way to solve this?

thheller16:04:47

there is some issue with $jscomp I havent been able to track down yet

thheller16:04:07

normally it just goes away after wiping the cache and recompiling. just try deleting .shadow/builds/<build-id>

lispers-anonymous16:04:18

I attempted that, but it did not fix my issue

thheller16:04:20

and then restarting the watch or release

lispers-anonymous16:04:49

It goes away when I target :es6 but I can't do that because of internet explorer 😞

thheller16:04:50

I mean if you can make a reproducible minimal repro that would help a lot

thheller16:04:03

IE is dead 😉

lispers-anonymous16:04:23

Not for my enterprise customers unfortunately lol

lispers-anonymous16:04:21

I will see if I can make a reproduceable small build. That might take me a while to do. I'm working on migrating a really large cljs application to shadow so there are a lot of moving parts. In the mean time, maybe that little hack will help someone.

emier16:04:56

In testing, what’s a good way to run some js code from a file (`myPreloads.js` ), which mocks some things before running the main test function? Basically, achieving the same thing as copy-pasting the code from myPreloads.js to the beginning of the test/target/testable.js file (which obviously isn’t a good idea).

lispers-anonymous19:04:50

I was able to accomplish this by writing my own preloads and using those in the test build.

lispers-anonymous19:04:01

If running clojure code won't work for you, you should be able to require the js file in the namespace declaration of your preload file.

emier14:04:20

Thanks for the reply @dannyfreeman, that seems to do what I want it to! However, now I get a problem with the modules that are required from the preloaded js file. So right now I have my preload namespace:

(ns preloads
  (:require ["./myPreloads.js"]))
and the js file I want to run, which requires its own modules myPreloads.js
...
require("module-alias/register");
...
I get the following error when trying to run the tests:
SHADOW import error /app/.shadow-cljs/builds/test/dev/out/cljs-runtime/myPreloads.js

/app/test/target/testable.js:67
    throw e;
    ^
Module not provided: shadow.js.shim.module$module_alias$register
Looking at the generated testfile testable.js, this is where it does the import.
SHADOW_IMPORT("shadow.js.shim.module$module_alias$register.js");
Can’t really figure out how to require these modules so that shadow can find them. Am I missing something here?

lispers-anonymous14:04:55

That I am unsure about. I did run into some issues around testing code meant for a browser for a node-test target that seems similar. To fix those issues I had to set this option in my :test profile in shadow-cljs.edn file

:js-options {:entry-keys ["module" "browser" "main"]}

lispers-anonymous14:04:48

I'm really unsure what the implications of doing that are, but combining that with stubbing out the global window object got my tests to run in node.

lispers-anonymous14:04:43

Also, if the contents of myPreloads.js are pretty simple, you could try re-writing them in clojurescript directly in your preloads cljs file. Perhaps shadow-cljs doesn't understand the code in your js file.

emier14:04:52

Okay, from the docs it seems like :js-options determines which build variant to use for the npm package, e.g. CommonJS or ESM. Unfortunately, adding it doesn’t do anything for me. Unfortunately, myPreloads.js is quite extensive so I’m trying to avoid rewriting it in cljs.

emier14:04:46

@thheller any ideas on how to fix this error related to shadow-cljs being unable to import npm modules that are required from a local js-file which in turn was required in a preload namespace? More info in my first reply of this thread

emier14:04:30

Not sure if it makes a difference, but the modules it can’t find seems to be devDependencies

thheller15:04:13

how are you running the tests? it is :node-test I presume?

emier06:04:16

yes, using :node-test

thheller06:04:09

if your preloads only setup node related things and don't interact with CLJS in any way you can just use node -r ./preloads.js your-test-output.js

thheller06:04:35

there are ways to get it into the build of course but this would be the fastest and easiest

emier06:04:17

okay, it seems it could work, is there a way to make :autorun run this as well though?

thheller07:04:29

no there is not but I don't recommend using that anyways

thheller07:04:23

you are much better off using something like https://www.npmjs.com/package/chokidar-cli to watch the test output file and have that rerun the tests when needed

thheller07:04:42

that also gives you full control over how tests run and so on

emier07:04:40

Thanks for the tip, I’ll look into it! Just curious, what’s the reason behind the nested requires not working when using preloads?

thheller07:04:43

it should work but what the code does actually matters and I don't have enough information to say

thheller07:04:52

if you have some reproducible code I could probably tell you

thheller07:04:03

but without its kinda hard to say

thheller18:04:14

@dannyfreeman you can do it a little cleaner by setting :compiler-options {:force-library-injection ["es6/util/arrayfromiterator"]}. the problem is that the polyfill detection logic somewhere makes a mistake and that part gets lost but you can always just tell it which polyfills to include

👀 2
lispers-anonymous18:04:49

I just tried to use :force-library-injection ["es6/util/arrayfromiterable"] in my compiler-options and that did not work. arrayfromiterator also didn't change anything but that function is available already.

thheller19:04:08

oh right. force-library-injection only triggers for release builds

thheller19:04:19

I guess that could be adjusted too yeah

lispers-anonymous19:04:44

I will try this on a release build real quick

lispers-anonymous19:04:52

Yeah that :force-library-injection key works for my release build, but not a development build.

lispers-anonymous19:04:17

Thank you 🙂 is there anyway I could help out with that?

thheller19:04:32

too late 🙂 should be fixed in 2.12.5. also just added the interable by default so should be ok even without config.

lispers-anonymous19:04:55

Awesome. I'll be on the lookout for that release. Thank you very much for the help.

valsen20:04:21

Hi, noob question here. What is a common way to set and access env variables, for example to use different firebase configs for dev/prod? I’ve tried using js/process.env but get a ReferenceError: process is not defined. Is there some trick to getting this to work in shadow or cljs in general?

valsen14:04:01

I’ve configured my env variables and added them to my shadow-cljs.edn as described in those links, but how can I now reference/use them in my cljs code? For example, if I have :closure-defines {app.core/APIKEY #env "APIKEY"} , I don’t understand how to reach it from within the app.core namespace

valsen14:04:52

If I try something like {:api-key #shadow/env "APIKEY"} I get a compile error No reader function for tag shadow/env . And simply referring to it by its name, APIKEY , I get an undeclared var error

thheller15:04:39

you (goog-define APIKEY "default value") in app.core and then use it like any other clojure def

valsen15:04:41

Ok, thanks:) It does however feel a little strange having to use system environment variables for setting config values for the frontend. Isn’t it more common do that in runtime instead of when compiling?

thheller16:04:46

yes, but a browser environment will hardly have access to your system environment

thheller16:04:58

so either way it is something that will become part of your build at compile time

thheller16:04:38

I do recommend keeping some kind of configuration in your HTML directly, so that it becomes actual "environment" configuration and is not hardcoded at build time