Fork me on GitHub
#shadow-cljs
<
2020-04-07
>
thheller07:04:56

Clojurists Together is coming up and I'm thinking about applying for shadow-cljs. I just don't have anything in particular I want to work on. Do any of you have any suggestions about stuff I should be working on?

πŸ‘ 28
sogaiu07:04:41

more remote / inspect development?

thheller07:04:14

yeah that is kinda what I want to work on but its too speculative and really not shadow-cljs specific

thheller07:04:30

maybe I'll make an update for the Inspect stuff I already built and apply with that separately

thheller07:04:49

It is quite different now from the last video after all

sogaiu07:04:50

ah, so is the non-speculativeness an important criteria for the application? i thought the results here: https://www.clojuriststogether.org/news/q2-2020-survey-results/ sort of suggested that may be speculative things might be appreciated too?

thheller07:04:27

yes, but I'm asking specifically if there is something in shadow-cljs itself people would like to see

sogaiu07:04:42

understood

thheller08:04:12

graaljs support has some up in the past and that would be interesting

sogaiu08:04:24

that does sound interesting

thheller08:04:33

but probably not more than a few hours work

mkvlr08:04:56

I’d love extra-mains support in shadow

thheller08:04:38

I know what extra-mains does in figwheel but I don't see why I would use it over separate builds?

teodorlu08:04:30

I'd also love to see more work on Shadow-CLJS Inspect. I'm aware that you intend for Inspect to be usable from outside of Shadow-CLJS, but that doesn't make it a "too risky" project, in my opinion. The fact that it can be decoupled from Shadow seems like a pure advantage to me. Sorry for adding a second reply that avoids your constraints, but Inspect is something I think could become fantastic. It might make sense to identify some Shadow-CLJS maintainance tasks to go along. I don't have any good recommendations there. Note that whereas one of the points of Clojurists Together is to fund the "boring work", there's plenty of interest in doing speculative work. Image source: https://www.clojuriststogether.org/news/q2-2020-survey-results/

teodorlu08:04:25

I also view Shadow as quite stable in its current state, with great docs. Maintainance work and documentation might be more of a critical task in other projects.

mkvlr08:04:19

@thheller because compiling once instead of x times is less work?

mkvlr08:04:51

we are probably an outlier with a large number of targets in our codebase

thheller08:04:45

well that depends I'd say. for my 6core/12thread CPU running two builds in parallel that may use the same namespace usually end up completing at the same time?

thheller08:04:33

and extra-mains produces overhead for the "main" build making that always slower and recompiling when it might not need to

thheller08:04:07

I'm not against adding it since it is fairly trivial to do so. I just don't have use for it myself so I need a proper use-case setup I can use for testing

thheller08:04:56

what do you mean by large number of targets?

thheller08:04:34

are they all rolled into one with extra-mains during development and then just split out for release builds?

mkvlr08:04:07

we have six targets total

mkvlr08:04:28

including devcards

thheller08:04:42

can you clarify what target means for you? :target in shadow-cljs is likely not what you mean?

thheller08:04:09

so just different builds in shadow-cljs terms?

mkvlr08:04:12

sorry for the unclear language

mkvlr08:04:46

I guess we should start by trying to switch to shadow an measuring the performance impact, this has been on our list for a while

thheller08:04:59

and your main concern about using separate builds is that namespaces will be compiled once for each build?

mkvlr09:04:54

yes, our setup also is such that we compile the same main namespace for different targets, e.g. browser, web-worker and graaljs

thheller09:04:53

well I always recommend making web workers part of your browser build via :modules

thheller09:04:02

but you are hosting the entire self-hosted env in a worker right?

thheller09:04:27

that way the worker and main can share code and the user only downloads cljs.core etc once

mkvlr07:04:03

oh, I missed that and didn’t know about that. That should cut things down quite a bit for us, thank you!

mkvlr07:04:17

:init-fn is shadow-specific, right?

thheller08:04:45

{:init-fn foo.bar/init} is basically sugar for {:entries [foo.bar] :append-js "foo.bar.init();"} but :append-js per module is also shadow-cljs specific

mkvlr08:04:53

hehe, sounds very useful for us though

mkvlr08:04:36

did you ever try to get it upstream?

mkvlr08:04:14

I do think it’s nice to have a config that’s compatible with vanilla clojurescript

thheller08:04:20

I gave up on getting any of my ideas upstream a long time ago

😿 4
thheller08:04:25

but the entire :modules integration in shadow-cljs is so completely different that it will be non-trivial to adopt this for core

mkvlr08:04:55

alright, no more excuses for us to move to shadow then

thheller08:04:45

I wondered why you are using :simple compiled CLJS for the "normal" pages that aren't even using CLJS?

thheller08:04:02

wouldn't it be better for your users to have one :advanced build for regular page stuff

thheller08:04:23

that can load self-hosted CLJS separately like I assume you do for other languages?

mkvlr08:04:25

we’re also using self-hosted in the browser as an internal feature

mkvlr08:04:46

for debugging mainly

mkvlr08:04:25

but that should eventually go away

thheller08:04:34

hmm ok. always scares me to see 10mb+ of JS when that could be <500kb πŸ˜›

thheller08:04:52

ok can always optimize that later I guess

mkvlr08:04:10

we tried a few times to get advanced going

mkvlr08:04:43

I always say it’s still smaller than a news page’s ad 😼

mkvlr08:04:04

would it be possible to have an advanced build and load self-hosted into the same target?

thheller08:04:48

as long as the advanced and self-hosted builds only interact via "remote" RPC basically. so each serializes to EDN or whatever and the other loads it

thheller08:04:54

they cannot otherwise interact directly

mkvlr08:04:18

and you lose any sharing of the base…

thheller08:04:24

easy if you load the self-hosted env in a worker or so since that has to talk that way anyways

thheller08:04:08

well each will have its own base yes but the :advanced one will be much much smaller

thheller08:04:26

and the self-hosted load on demand so no impact on overall page performance that might not use it

mkvlr08:04:01

yes, we already do that on demand

thheller09:04:48

also multiple builds in shadow-cljs works a bit differently than elsewhere due do the caching

thheller09:04:04

so they'll be a lot faster after the first compile than elsewhere

mkvlr09:04:39

yes, we have the self-hosted compiler in a worker

mkvlr09:04:14

this sounds like we should really give this a try before asking for more features

thheller09:04:18

how come the link you gave me about the graal stuff loads the :simple compiled version of cljs?

thheller09:04:38

doesn't seem to need any self-hosted things? seems like a large chunk of JS for a page that doesn't use much?

thheller09:04:22

but your entire setup seems rather "custom" and the defaults in shadow-cljs might not be a good fit for a setup like that

thheller09:04:15

I get worried anytime I see 10mb+ JS downloads for any site :P

Chris McCormick09:04:49

> graaljs support has some up in the past and that would be interesting @thheller i am very curious about what this would mean in the context of shadow-cljs

thheller09:04:15

graaljs meaning the actual graaljs script engine

thheller09:04:34

NOT the usual graalvm native-image stuff people are excited about

thheller09:04:38

that will never work for shadow-cljs

thheller09:04:42

graalvm native-image cannot be supported since the CLJS compiler needs to be able to load code dynamically for macros and stuff. native-image cannot do that.

thheller09:04:16

the graaljs stuff you can use in any JVM so isn't coupled to graalvm in any way

Chris McCormick09:04:18

very interesting

thheller09:04:38

graaljs is just an interesting replacement to nashorn basically

thheller09:04:18

and since nashorn is going away I need to find a replacement for shadow-cljs anyways (since that uses nashorn)

Chris McCormick09:04:05

i'm quite ignorant about the graalvm space so forgive me if this is a stupid question, but is it conceptually possible to build a Clojure based graalvm native image which embeds the graaljs execution environment inside it as they have done in the example you linked, so you would end up with a native binary which runs cljs?

Chris McCormick09:04:48

a friend of mine @retrogradeorbit is shipping really neat stuff with Clojure + GraalVM and I'm in awe of it (bootleg, spire)

thheller09:04:39

don't know why you would do that πŸ˜›

thheller09:04:38

graalvm native image is neat because it starts fast

thheller09:04:54

but that comes at the code of not being dynamic anymore and clojure is cool because it is dynamic

thheller09:04:58

so ... tradeoffs πŸ˜›

Chris McCormick09:04:28

i guess there are situations where shipping a single binary to the end user is advantageous

Chris McCormick09:04:39

if the end user doesn't know or care about nodejs

Chris McCormick09:04:47

i know there is nexe too

thheller09:04:41

I still see graalvm native-image in the same way as an :advanced compiled CLJS build. highly optimized for one particular thing and not extensible at all πŸ˜›

thheller09:04:07

so it is absolutely useful for certain stuff, not so much for other things πŸ™‚

David Pham10:04:30

@thheller How would you do you cli β€”config-merge with a edn files? (Having a $(cat config.edn) I guess?)

thheller11:04:42

I recommend using the CLJ api whenever things go past a simple command line. https://shadow-cljs.github.io/docs/UsersGuide.html#clj-run

thheller11:04:06

I never had a use to pass in a full edn file

David Pham11:04:49

Hmmm... how do you modify the argument of your build then?

thheller11:04:26

don't know which one I'd need to modify

darwin12:04:09

ad Clojurist Together, I would cast a selfish vote for Dirac support: https://github.com/thheller/shadow-cljs/issues/636

βž• 8
thheller13:04:46

@darwin you can likely already do all of that. assuming that you are able to use websockets, no tcp or other remote method available yet

thheller13:04:27

only the custom analyzer stuff likely needs some work

darwin13:04:51

yep, I’ll have to dive into it

isak14:04:54

I'm getting this error when loading the page (latest version):

Exception in thread "async-dispatch-17" java.lang.IllegalArgumentException: No implementation of method: :close! of protocol: #'clojure.core.async.impl.protocols/Channel found for class: clojure.
lang.PersistentVector
        at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:583)
        at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:575)
        at clojure.core.async.impl.protocols$eval246$fn__247$G__235__252.invoke(protocols.clj:21)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:979)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:975)
        at clojure.core.async$ioc_alts_BANG_$fn__3086.invoke(async.clj:385)
        at clojure.core.async$do_alts$fn__3018$fn__3021.invoke(async.clj:254)
        at clojure.core.async.impl.channels.ManyToManyChannel$fn__670.invoke(channels.clj:265)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

isak14:04:13

It gets printed on every load

thheller14:04:54

weird indeed. maybe a conflicting core.async version?

isak15:04:40

Oh, let me try to update it

thheller15:04:41

should be [org.clojure/core.async "1.0.567"] if you are on .94

thheller15:04:58

[org.clojure/core.async "1.1.587"] with the next release

isak15:04:59

you are right, that was it, thanks @thheller :thumbsup::skin-tone-2:

pmooser16:04:52

I have a shadow-cljs question - I am using shadow-cljs with the self-hosted cljs repl, and I notice that because the CLOSURE_BASE_PATH is defined as a relative path, I get loading errors for source maps and things, depending on what URL I happen to be accessing via the embedded dev http server. Does this sound like a crazy configuration error on my part?

thheller16:04:09

shadow-cljs does not provide a self-hosted cljs repl

pmooser16:04:19

I don't know quite how to explain - the bootstrap repl?

thheller16:04:20

CLOSURE_BASE_PATH you control via :asset-path

pmooser16:04:41

Ok, I will see if I can re-read the docs to look for :asset-path.

thheller16:04:41

@pmooser you are using the :bootstrap support build target right?

pmooser16:04:25

@thheller Yes ... I think I have it all fixed, using the :asset-path as you suggested.

pmooser16:04:35

And yes I am using the :bootstrap target ...

FlavaDave17:04:45

having an issue with a new project. Build keeps on failing.

Davids-MBP:~ Dave$ cd projects
Davids-MBP:projects Dave$ cd hone-master/
Davids-MBP:hone-master Dave$ shadow-cljs watch app
shadow-cljs - config: /Users/Dave/Projects/hone-master/shadow-cljs.edn  cli version: 2.8.83  node: v13.12.0
shadow-cljs - starting via "clojure"
running: npm install --save [email protected] [email protected]
npm WARN saveError ENOENT: no such file or directory, open '/Users/Dave/package.json'
npm WARN enoent ENOENT: no such file or directory, open '/Users/Dave/package.json'
npm WARN Dave No description
npm WARN Dave No repository field.
npm WARN Dave No README data
npm WARN Dave No license field.

+ [email protected]
+ [email protected]
updated 2 packages and audited 2750 packages in 0.871s
found 3 low severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details
[2020-04-07 12:02:51.476 - WARNING] :shadow.cljs.devtools.server.nrepl04/middleware-fail - {:sym refactor-nrepl.middleware/wrap-refactor}
FileNotFoundException Could not locate refactor_nrepl/middleware__init.class, refactor_nrepl/middleware.clj or refactor_nrepl/middleware.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
	clojure.lang.RT.load (RT.java:462)
	clojure.lang.RT.load (RT.java:424)
	clojure.core/load/fn--6839 (core.clj:6126)
	clojure.core/load (core.clj:6125)
	clojure.core/load (core.clj:6109)
	clojure.core/load-one (core.clj:5908)
	clojure.core/load-one (core.clj:5903)
	clojure.core/load-lib/fn--6780 (core.clj:5948)
	clojure.core/load-lib (core.clj:5947)
	clojure.core/load-lib (core.clj:5928)
	clojure.core/apply (core.clj:667)
	clojure.core/load-libs (core.clj:5985)
shadow-cljs - HTTP server available at 
shadow-cljs - server version: 2.8.40 running at 
shadow-cljs - nREPL server started on port 49842
shadow-cljs - watching build :app
[:app] Configuring build.
[:app] Compiling ...
[:app] Build failure:
The required namespace "react" is not available, it was required by "reagent/core.cljs".
The namespace was provided via :foreign-libs which is not supported.
Please refer to  for more information.
You may just need to run:
  npm install react

FlavaDave17:04:14

npm install react does nothing

lilactown17:04:03

it sounds like you don’t have a package.json

lilactown17:04:27

run npm init in your project and then npm install shadow-cljs react react-dom

FlavaDave17:04:20

ok i tried that and then ran shadow-cljs watch app again and got a different error

NPM dependency "react" has installed version "^16.13.1"
"16.9.0" was required by jar:file:/Users/Dave/.m2/repository/reagent/reagent/0.9.1/reagent-0.9.1.jar!/deps.cljs
NPM dependency "react-dom" has installed version "^16.13.1"
"16.9.0" was required by jar:file:/Users/Dave/.m2/repository/reagent/reagent/0.9.1/reagent-0.9.1.jar!/deps.cljs
[2020-04-07 12:51:04.188 - WARNING] :shadow.cljs.devtools.server.nrepl04/middleware-fail - {:sym refactor-nrepl.middleware/wrap-refactor}
FileNotFoundException Could not locate refactor_nrepl/middleware__init.class, refactor_nrepl/middleware.clj or refactor_nrepl/middleware.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
the print out was very long but that seems to be the meat and potatoes

thheller19:04:37

@dgonsalves22 looks like you have configured to use the refactor-nrepl middleware but did not add it do the classpath

thheller19:04:43

so either add that or remove the middleware

isak19:04:04

Is it possible to disable :`browser-inject` for a build? I have a web-worker module in a seperate build, and this worked until I upgraded today. Now it chokes because the generated script references window:

(js/window.addEventListener "beforeunload"
    (fn []
      (when-let [s @socket-ref]
        (.close s))))

thheller19:04:52

in a separate build? I presume you mean module?

isak19:04:34

No, it is a separate entry under :builds

isak19:04:05

so there is no module from that build that will run in the normal browser environment

thheller19:04:05

so its a build for only a worker?

thheller19:04:56

hmm but yeah the logic may be buggy if the only module is a worker

thheller19:04:05

you can set :devtools {:browser-inject false} I guess

isak19:04:58

Cool, that worked πŸ™‚

thheller19:04:08

open an issue if you don't mind

thheller19:04:14

that shouldn't be necessary

isak20:04:14

Is inspect intended to be used with a value that can change over time? Does this require tapping it continuously, or should once be enough?

isak20:04:55

Looks like it requires tap after changes

isak20:04:22

I'm trying this:

(tap> #'re-frame.db/app-db)
with this Datafiable protocol extension:
(ns foo.utilities.debug
  (:require [clojure.core.protocols :as p]
            [reagent.ratom]
            [shadow.remote.runtime.cljs.browser]))

(extend-type reagent.ratom/RAtom
  p/Datafiable
  (datafy [x]
    @x))

isak20:04:28

Hmm I think I'm mistaken, looks like the deref is happening more than once (I added a print)

thheller20:04:59

hmm dunno about reagent atom. I only use regular atoms

thheller20:04:50

datafy is called when the object is added and "described"

isak20:04:08

Ok sweet, just saw that it updates when I navigate back and forth πŸ™‚

isak20:04:31

When I click back on the stack of items, then drill down into it again

thheller20:04:38

easy trick is (tap> {:foo re-frame.db/app-db})

thheller20:04:00

that way you can be sure you can trigger the datafy whenever you want

isak20:04:46

right ok

thheller20:04:14

but the point of datafy is to get data, so not something that changes over time πŸ˜‰

4
FlavaDave20:04:54

i removed the middleware and now I get

{:type java.lang.IllegalArgumentException
   :message "Cannot open <nil> as a Reader."
   :at [$fn__11520 invokeStatic "io.clj" 288]}

thheller20:04:39

you get that where?

FlavaDave20:04:19

when I run shadow-cljs watch app i mean there is a lot more I just wanted to avoid putting the whole thing in here because it would be very long

thheller20:04:40

well without context that error is not very useful

thheller20:04:43

any more info? I don't mind long posts? πŸ˜›

FlavaDave20:04:50

I got it to work. ended up being something totally different

FlavaDave21:04:43

Also, thanks for the help @thheller and @lilactown