Clojurians Log v2

@lilactown the last time I looked at the fast-refresh stuff it required processing the generated JS to extract the signatures of the hooks used

very very very doubtful that'll work for CLJS

It doesn't. We can easily use macros to generate the signature

I have a working poc of this

The only actual requirement is to register the signatures with the runtime somehow (Babel, macros, etc.) and to call a method to trigger a "refresh" on hot reload

The last real hurdle is to correctly get all references to my changed ns to reload correctly

Going to sleep. Will check back later

@haiyuan.vinurs how did you define your component? how is the JS code getting access to it? :npm-module + webpack?

@yyoncho has joined the channel

Hi - I have 2 npm packages generated by shadow-cljs that are a dependency of a third application. Is it possible to share the timbre configuration between the two packages in the packaging project?

sorry, I don't know anything about timbre

it is not advisable to create 2 npm packages intended to be consumed in one project. they'll contain a lot of duplicated code

I have a .clj file with a macro in it under src/asciidocs/core.clj, and then I have src/asciidocs/app.cljs file which declares (:require-macros [asciidocs.core :as core]). When I try to use the macro the shadow-cljs complains:

6 |   (println (core/load-asciidoc "resources/docs/cv.asc"))
------------------^-------------------------------------------------------------
No such namespace: asciidocs.core, could not locate asciidocs/core.cljs, asciidocs/core.cljc, or JavaScript source providing "asciidocs.core"
--------------------------------------------------------------------------------

read https://code.thheller.com/blog/shadow-cljs/2019/10/12/clojurescript-macros.html

thanks

Is there any way to get the compiler to not evaluate stuff in #?(:clj blocks? I don’t want to compile the required NS into CLJS just to get rid of an error, as it’s not otherwise used in this file.

Of course, I can just require something random, like #?(:cljs [clojure.string :as pc]) to get around it, or create a dummy NS to include for this purpose.

don't import it as a macro then?

I'm confused about your question. you ask about :clj blocks but then fix it in :cljs block?

ah sorry ... now I understand. no that is not possible

it has to be syntactically correct (so the alias needs to be resolvable)

or just don't use .cljc 😉

it is the reader failing here. not the compiler. it isn't eval'ing the :clj blocks. it is just trying to read and for that is must resolve the ::pc/input alias

might be interesting to modify tools.reader so it doesn't try to resolve whenever it is in a read-cond block that isn't active

doesn't really matter what ::pc/input resolves to in those cases

nice result from the wild: I recently converted a fairly large client codebase to compile with shadow-cljs - live-reload became ~3x faster upside_down_parrot and more reliable (was previously getting lots of errors when refreshing the page during recompile)

only 3x? 😛

Speaking of the wild, I don’t want to use the words “de facto cljs compiler” yet, but I feel like I’m seeing a lot more shadow-cljs out there these days as opposed to only a year ago or so. Have you seen any concrete indications as to this @thheller (Visitors to the site etc.)?

numbers keep going up yes. no clue how it compares to others though

the yearly clojure survey seems to be a good indicator

last one was about 25% shadow-cljs. we'll see in the next one 🙂

does anyone have a good method of running single tests in a :node-test script?

shadow-cljs node-repl (require '[your.test-ns :as x]) (x/that-test)

Thanks, that seems to work, but I'm using spectron (https://github.com/electron-userland/spectron), which is supposed to open up an electron instance to run the spec with webdriver. But running the spec in the REPL doesn't do that so maybe it's not possible from the node-repl?

or maybe it's because the tests are async

scratch that, I ran the test with a .catch and am looking into the error now

🤦 Had to run fixtures

I guess the :node-script output could take additional command line args too

so node the-output.js --only some.test/foo or so. if someone is interested in working on that

wouldn't that be possible without any changes to shadow-cljs?

sure if you write a custom runner

the default one doesn't take any command line args

hey thheller, I made a short POC of what I'm trying to do with react's HMR support (react-refresh) here: https://github.com/Lokeh/react-hmr-cljs

if you have time to look at it, I would greatly appreciate it

I'm able to generate the signatures and get hot reloading of same-file changes working just fine

however, in the example I have a dep-b and dep-c namespace where core depends on dep-b, and dep-b depends on dep-c. if I change the greeting in dep-c to be "Bonjour!", it won't pick up the change on hot reload until the next render

thats missing an example how the use actually looks

or I'm just blind 😛

hm?

yeah I'm just blind sorry 😛

sorry yeah it's laid out a little weird. react-hmr.core has the example usage

now you only need to figure out how to get rid of all the related code in production 😉

yes, it’s easy enough to wrap in (when goog/DEBUG ...

not so easy to get rid of the react-refresh require though 😉

true. could be done with a preload and some alter-var-root / global JS shenanigans

but does my problem I need help with make sense?

don't know enough about the refresh stuff to comment. it likely happens because core didn't change it just cancels the re-render

right, so the issue is that the react-refresh stuff only re-renders components that have had their “type” (the function component registered with the ID) changed

the way that they’re suggesting people implement it is to detect whether a file only exports components. if it exports anything else than components, then call React.render instead (and blow away the state)

which is… one solution

I would need to be able to somehow detect whether a namespace is “safe” to refresh in my after-load hook

the other option could be to reload the entire dependency chain

which if you are editing a file that has many dependents, could be very onerous

I think either way, I need some help from shadow-cljs / figwheel to make it work

sorry, don't know enough about react-refresh to make sense of that. Don't know why "only exports components" is relevant at all. makes no sense to me right now

it’s a super dumb heuristic to try and detect, “does anything other than React call something from this file”?

when i require ant-design in cljs and build a browser target like main.js, is that all ant-design files will bundled in the main.js?

@haiyuan.vinurs yes if you require antd directly

is there a method exclude it?

only import what you actually need

if I put what I’m asking for concretely, with my understanding of the problem: 1. would you be willing to add the ability for the load hooks to get passed any info about the ns that’s being loaded? symbol + metadata? 2. would you be willing to add a setting that would reload all dependent namespaces up to the entry point?

ok, understand

either 1 or 2 would potentially allow me to solve the reloading problem. 2 I feel like might have the most potential for problems, but I don’t know the complexity for 1

1) the client side doesn't have that info. 2) I kinda chose not to add it https://github.com/thheller/shadow-cljs/issues/349

could that info could be passed to the client side from the compiler state?

in theory yes.

but you'll first need to make a convincing case why it is necessary at all

I mean react-refresh is not even released yet. not going to add features for it until the design is finalized

also would kinda like to understand it first

yeah totally

but how do you even tell a component from a regular function?

seems completely pointless to me?

I agree that the heuristic they chose is not a good one

the way I understand react-refresh to work today, is each time a component is registered (e.g. via a reload of the namespace), if the ID already exists it marks that ID as dirty. when performReactRefresh is run, it then goes through the React tree from the root and re-renders all of the components marked as dirty. if the hooks “signature” has changed since the last refresh, it will re-mount the component instead of re-rendering.

so the issue shows up when: - a function gets changed that some component uses inside of it’s render that’s not a registered component - that component’s namespace wasn’t reloaded, so it’s not marked as dirty - React doesn’t re-render it since it thinks it’s clean

Is it possible for a macro to detect :dev vs :release, e.g. to elide logging code?

@tslocke in theory yes but you probably want your own setting for this

:compiler-options {:external-config {:your/key ...}}

OK that's what I've been doing but I'm having trouble switching back to having the logging active after doing a release build. TBH I think I'm not clear on the correct way to do a release build, deploy it to the server, and then go back to developing.

https://shadow-cljs.github.io/docs/UsersGuide.html#_release_specific_vs_development_configuration

how did you access it exactly? and where did you put it?

if you put it into the incorrect places or access it incorrectly the cache won't be invalidated

the correct way to make a release build is shadow-cljs release your-build

I have it in my shadow-cljs.edn, under :builds -> :app -> :dev

have what in that

When I run shadow-cljs release, it overwrites the same main.js as shadow-cljs watch - is that expected?

if you configured it that way yes

(have what) :compiler-options {:external-config {:your/key ...}}

:release {:output-dir "somewhere/else"} would change it

OK, maybe that's the simplest way to fix the issue then - I'll have the release build go somewhere else

rm -rf public/js && shadow-cljs release app && copy-public-js-to-server

And I access it with (get-in @cljs.env/*compiler* [:options :external-config :logging])

ok looks good then. release builds don't share any cache with the watch/compile builds so they usually don't interfere

is there a way to specify the :lein option outside of the top level? context is that im trying to use lein with shadow and specify a different lein profile for :dev and :release within a build

@mazin just call lein directly then

lein with-profiles +release run -m shadow.cljs.devtools.cli release app

same as calling shadow-cljs release app (just with the extra profile)

@thheller thanks so much. I'm once again amazed by how available you are to help out. Don't you get swamped?

right now I'm just hanging out and reading stuff

helping out is more interesting 😉

🙂

gotcha, thanks so much.

I went ahead and opened an issue on React to try and see if we can solve this without needing to change shadow-cljs 😛 https://github.com/facebook/react/issues/17281

@lilactown so is the concept of react-refresh that re-registered components get refresh, but nothing else does?

exactly

re-registered components get re-rendered (possibly re-mounted if the hooks signature has changed), otherwise all components are left alone

so it's essentially partial dependency invalidation

which kinda makes sense because react only know how to invalidate and hot-reload a certain class of things, namely components

react is saying "if you tell me when my stuff changes, I can make an honest effort at refreshing them because I know their semantics"

but that's always been the problem at HMR in general... your app needs to work in a way amenable to HMR

HMR concerns bleed into the application architecture

The thing that puzzles me right now is that, if React re-rendered the entire app, it would fix this most of the time I think

So I'm not sure why they only re-render the components which have changed

re-rendering the whole app is the sort of hot reload shadowcljs already has today, right?

It is except calling React.render also re-MOUNTS everything

So you lose all local state

If you could re-render the whole app without losing local state, you have achieved the dream 😂

and this refresh thing re-renders without re-mounting?

Yes

then I think what you need to make this work is the dependency graph that shadowcljs has around

then you can reason about it and say things like "namespace.path changed so I will mark all components that depend on it dirty"

but I guess that's why you wanted the component metadata in the first place?

if you go down to the symbol level and still have the dependencies it might be easier

then you could have some reverse mapping that says "this react component in refresh stuff maps to that cljs function" and also "this reload affected all these cljs symbols" and cross reference those

yes, you can take the surgical approach where you analyze each components dependencies and detect whether any have changed

you can also take the sledgehammer approach, which is that shadow-cljs or figwheel could reload ALL dependent namespaces which would in turn mark any components within them as dirty

both of these require quite more work on the tooling side, which I understand why thheller is skeptical 😉

you could just keep a registry of all and :dev/after-load make it dirty yourself

I saw a register! call somewhere already

right, yes hmmm

each component gets roughly this emitted:

(def set-my-component-signature! (create-signature))

(def my-component ...)

(set-my-component-signature! my-component "hooks signature goes here")

(register! my-component "my-app.core/my-component")

I’m not sure how I would make it dirty. I suppose I could wrap each in a fn? that seems gross

dunno either. does it use identity? like regular react or does react-refresh have different rulres?

I assume it does

(register! my-component "my-app.core/my-component" (fn [new-component] (set! my-compoent new-component)) maybe

probably needs an indirect call then though

so (defn actual-component* ...) and (defn component [...] (actual-compoent* ...))

hmm not sure I get it yet

I think the gist of the thing is to give the call to react-refresh-runtime/register a new function

how much work would it be to add support for doing a full recompile of dependent namespaces? if just to experiment with it in relation to react-refresh

I want to see how bad it would be with a large project number of namespaces

not easy to add right now. full recompile is also madness.

Yeah def not full recompile

@aiba has joined the channel

you can sort of if you pass the runtime-id when switching the REPL

(shadow/nrepl-select :the-build {:runtime-id ...})

Ok. What I'm doing right now that works is:

:reql/quit
(shadow/repl-runtime-select :browser-extension
    (:runtime-id (first (shadow/repl-runtimes :browser-extension))))
(shadow/repl :browser-extension)

hmm or maybe not

I don't know anymore to be honest

the nREPL stuff has been a little frustrating

I didn't realize I could pass a :runtime-id to nrepl-select, so I'll try that.

No worries, I realize I'm way in edge-use-case land here, but it's interesting nonetheless.

you can't actually pass :runtime-id to the nrepl

it currently only expects that as part of the nrepl message (which no tool actually sends)

but I could change it so it respects it if set via nrepl-select

but all of this is part of the stuff I'm currently working through

so I hope it'll become a bit clearer in the future

No worries at all, I appreciate that it works as well as it does. I'm not really impeded here, just trying to make it more ergonomic so I can eval in different JS environments really efficiently.

that should take care of it

but headed to bed now. will test and probably release tommorrow

It might require more tooling integration (eg from Cursive) but in my "ideal future" I'd be able to configure rules such that "send to repl" from a certain file automatically gets eval'd in a particular environment, rather than switching a session singleton. I'm not sure how practical such a system would be beyond what I'm doing, but maybe there's a more general "route eval requests to environments" behavior hiding in there.

Cool, thanks for your help!

yes, I wish tools had support for "runtimes"

but the regular CLJS repls don't expose them so it would be shadow-cljs only right now

I have an interesting problem I’m trying to tackle

React recently shipped native support for hot reloading components

actually, before I type all this out and bother y’all with thinking through this, I’m going to do some experiments and see where I actually fall over

the problem I’m currently afraid of is if I use React’s way of performing an update without restarting the app, it might have some stale references to e.g. a utility function I just changed

so I might need some way of determining what “kind” of change was done (in webpack they use some heuristic to determine of a file only exports components), and decide whether to run performRefresh or render

after writing it out, I'm pretty sure this will be an issue I need to tackle if I want to use the React hot reloading stuff

I don't suppose there's any information that the before-load/`after-load` fns can be given to help with this?

since all def and defns are exported by default, I don't think I can rely on that heuristic to ensure only the React components in a particular file are used elsewhere. I'm thinking right now if I could annotate an ns with some metadata, and get fed that in my after-load hook, then I could rely on the programmer to reasonably discern if a ns is safe to just refresh and add the meta themselves

ex:

(ns ^:react/refresh my-app.components
 ,,,)

ah interesting. I was reading figwheel docs, and it distinguishes between reloading and recompiling dependent namespaces. it's possible that by reloading dependent namespaces (not recompiling them) we can have the desired behavior.

my preliminary tests show it actually all works fine 😅

ah shoot. it only works one ns deep in the tree

e.g. if I have namespaces a, b and c where a requires b requires c: a -> b -> c then if I change c, only b is reloaded.

if we reloaded the entire dependency tree of c -> b - a then I think things would work correctly. I don't know how heinous that would be in large apps 😬

is there any example that shows how to call my component in react component, my own component is defined in cljs

@thheller thanks for the info! 🙂