you can technically use shadow.remote to talk to the runtime directly

so you could create an external tool that also connects to shadow.remote and then gets the data

https://github.com/thheller/shadow-cljs/blob/master/doc/remote.md

not many docs on this subject though

right now only websocket is supported

but every remote interaction shadow-cljs does is done over shadow.remote and its all accessible to anyone, this includes REPL, hot-reload, etc

very nice, thanks for the info. I'll try to figure out how to go about it based on the shadow codebase.

I added an example client in CLJ https://github.com/thheller/shadow-cljs/blob/master/src/dev/shadow/remote_example_ws.clj

just like :clj-eval there is :cljs-eval, just need to find the correct :to via :request-clients https://github.com/thheller/shadow-cljs/blob/master/src/dev/shadow/remote_example_ws.clj#L155

so you could either do it on top of eval

or implement some client side code, but given that it'll be mostly macro driven I guess eval is required either way

if you know the runtime id you can also just use shadow.cljs.devtools.api/cljs-eval

amazing, thanks for the example - I was going down the route of trying to use runtime.shared/add-extension with a new op like :write-malli-config. I don't think there's any macros involved. After your suggestion to do instrumentation at runtime I refactored it to do so https://github.com/metosin/malli/blob/master/src/malli/instrument.cljs . so at runtime all the schema data is available and a simple data transform is run on it and then printed: https://github.com/metosin/malli/blob/master/src/malli/clj_kondo.cljc#L208 So my hope was I could do something like

(shadow.remote.runtime.shared/call @runtime-ref 
  {:op :write-malli-config :data kondo-config})

ok so I have it working with the method I was thinking, I just guessed at a few pieces, but the communication is happening clj side, loaded by including this ns in a shadow build hook:

(defmethod worker/do-relay-msg ::write-kondo-config
  [worker-state msg]
  (println "Got kondo config" msg)
  worker-state)

(shared/call
    (deref client-shared/runtime-ref)
    {:op ::write-kondo-config
     :to env/worker-client-id
     :data (get-kondo-config)}
    {})

(shadow.cljs.devtools.api/cljs-eval :the-build "(whatever.ns/get-kondo-config)") and then write the result to disk?

oh man, that's so cool, ha - yea that's perfect! I can put that in a compile finish hook and I think that's it! Thanks Thomas

it's working fairly well, but there is an off-by-one problem/non-determinism at play with hot-reload. The hook I have runs on :flush and then invokes cljs-eval but sometimes the data returned contains the old values from the previous compilation. So I think the cljs-eval is happening before the latest code is loaded. Ideally I wouldn't have to update the cljs side of things and could do this all from the build hook, so was wondering if there is a way to determine if the latest code was loaded in the client and not only written to disk.

well the server side implementation does not synchronize with the client side. it just sends messages

got it, I just used a simple counter on the client side and busy loop until it gets increment on the jvm side to know when the code is loaded. Thanks again for the help!

I'm still not sure what you are doing, do you have code to share?

oh haha, yea this is what I got working: https://github.com/metosin/malli/pull/829

Malli has a feature that takes a function schema and converts it to equivalent clj-kondo config and writes that to the kondo config directory. We want the same capability in cljs but the constraints are: • we need to evaluate the cljs code because the schemas may contains Vars which can only be resolved at runtime • once we have the output of the schemas-> clj-kondo config we need to persist it to the filesystem so given those constraints I realized that shadow-cljs is already doing this communication and can we utilize that somehow? and that's how I landed on this solution Hope that helps clarify what the goal is.

and where or how do you generate it for CLJ?

in CLJ: • all schemas are collected and then transformed to kondo config and then saved to disk that happens here: https://github.com/metosin/malli/blob/cfbc8eaa05bedb1e7e7b132d52bbf2e61b143f92/src/malli/dev.clj#L35

so this is done once on start?

and then again via that watch on the https://github.com/metosin/malli/blob/cfbc8eaa05bedb1e7e7b132d52bbf2e61b143f92/src/malli/dev.clj#L32 so as you dev and change any function schemas the new kondo config is output

as far I can tell you already have the preload ns calling instrument! right?

then your previous attempt might be better?

ie. the (defmethod worker/do-relay-msg ::write-kondo-config [worker-state msg]... )

and in the preload ns (defn ^:dev/after-load kondo-transfer! [] (whatever-to-send-message-to-relay))

then you don't need a hook at all

and that do-relay-msg would be in the malli clojure codebase?

could be yeah. in the preload ns you could ensure its loaded via :require-macros [malli.shadow-helper]

I see - I think that would work! from the user's perspective they would then only need to add the preload. And users not using shadow, would be unaffected by this

yep

or could just remove that part of the code

I also have a late-cljs feature in a branch for a couple months now

which basically just lets you annotate a ns to be compiled last

which would be perfect for this 😛

since you could just make the preload part of mall, without needing to add all the requires

yesss! that was my next question - how to make this a library only concern and not need the user to deal with it

if (defn ^:dev/after-load kondo-transfer! [] (whatever-to-send-message-to-relay)) this lives in the library in a ns that cannot include the user's code, will it always run after the user's code?

well it runs after all code is loaded which is enough right?

but not sure if the instrument! call is relevant?

nvm. that also executes before after-load

from a user's point of view they're using ! in an after-load hook and that is where the latest schemas are picked up after hot reload. So this kondo preload would have to run after that, which I'm not sure about the order of multiple after-load hooks

I thought they are using the preload ns only?

the order is somewhat unreliable, it should be in dependency order but isn't guaranteed to be

"the preload ns only" meaning the one in the library?

and not their own?

why is (md/start!) not in the preload?

the idea is that a user will call ! however they like - could be in a preload but doesn't have to be - it also takes an options map which we want to make visible to the user

I don't really know how any of this works, so I'm just guessing what parts do

I assume the goal should be to keep malli.dev.cljs out of the build entirely for release

so best way to achieve that would be the preload?

(ns com.my-org.client.preload
  {:dev/always true}
  (:require
    [] ;; must require all namespace here that potentially get instrumented
    [my.app.util]
    [malli.dev.cljs :as md]
    [malli.dev.shadow-hooks]
    [malli.instrument :as mi]))

(mi/instrument!)
(md/start!)

something like that?

yea agreed there - I think we're saying there is a preload ns the library provides which will deal with the kondo transmission, but the user will have to provide their own separate preload to include the namespaces in their app to ensure the preload runs last

yes, right now

just need the start:

(ns com.my-org.client.preload
  {:dev/always true}
  (:require
    [] ;; must require all namespace here that potentially get instrumented
    [my.app.util]
    [malli.dev.cljs :as md]
    [malli.dev.shadow-hooks]
    [malli.instrument :as mi]))

(defn {:dev/after-load} dev []
   (md/start!))

(ns malli.dev.shadow-instrument
  {:shadow.build/late-cljs true
   :dev/always true}
  (:require
    [malli.dev.cljs :as md]
    [malli.dev.shadow-hooks]
    [malli.instrument :as mi]))

(mi/instrument!)
(md/start!)

if I ever add the late-cljs part 😉

although for your case that isn't even relevant much

but looking at this I think another option is to tell the user to just invoke this call:

(ns com.my-org.client.preload
  {:dev/always true}
  (:require
    [] ;; must require all namespace here that potentially get instrumented
    [my.app.util]
    [malli.dev.cljs :as md]
    [malli.dev.shadow-hooks]
    [malli.instrument :as mi]))

(defn {:dev/after-load} dev []
    (md/start!)
    (malli.shadow-kondo/emit!))

since you don't do stuff in macros anymore right?

the schema collection is still in a macro

which does need to get the latest code

ok so the instrument! is the macro?

emits something like:

(malli/collect [long list of all namespaces])
(malli/instrument!)

the instrument is all in js

that is confusing. what emits the collect?

https://github.com/metosin/malli/blob/cfbc8eaa05bedb1e7e7b132d52bbf2e61b143f92/src/malli/dev/cljs.cljc#L35

so the user-facing api is just !

so the start is the macro

why is the instrument! call in the preload then?

yea

or is that just old docs?

probably old docs - is that in their now?

I looked at the PR link you sent

hmm, let me double check

ooh yea sorry I overlooked that, that is incorrect

yeah, I'd recommend to move the start! into the preload

right that's what that should say

and inside a function with dev/ater-load right?

(ns user.app.shadow-instrument-preload
  {:dev/always true}
  (:require
    []
    [malli.dev.cljs :as md]
    [malli.dev.shadow-hooks]
    ))

(md/start!)

no

like that

the :dev/after-load hook lives in .shadow-hooks

not required for the user to define it

user code otherwise doesn't contain anything

i see

unless I'm missing something that should be enough

i think so - if this runs after their latest code changes that should work

thats what the [] will ensure

if thats the :init-fn ns it will ensure that the preload is actually compiled after those

preload is a bad name, doesn't mean its actually preloaded just injected into the build

https://github.com/metosin/malli/blob/master/docs/clojurescript-function-instrumentation.md#releaes-builds

also none of this is needed then

got it - ok yea that should work then. and then the malli side can be a preload like you suggested

yea I updated the docs with that PR to suggest the preload strategy first

and fwiw could have shortened without all the CLJ code

yea, if you know a way around that that'd be awesome to know

{:main
 {:target :browser
  :output-dir "resources/public/js/main"
  :asset-path "js/main"
  :dev {:modules {:main {:init-fn com.my-org.client.entry/init}}}
  :release {:modules {:main {:init-fn com.my-org.client.release-entry/init}}
            :build-options {:ns-aliases
                            {malli.dev.cljs malli.dev.cljs-noop
                             com.my-org.client.malli-registry com.my-org.client.malli-registry-release}}}
  :closure-defines {malli.registry/type "custom"}
 ,,,

nice!

didn't know you can change the init-fn for release

https://shadow-cljs.github.io/docs/UsersGuide.html#_release_specific_vs_development_configuration

can override anything, only trouble is vectors since they just append not replace

but init-fn just replaces

but this entire switching becomes unnecessary with the preload calling start

since none of the dev code is included by the user code, only the build config

exactly - thanks for the info on that

this is really helpful. I'll try out the setup you suggested using the multi-method and clean up those docs. I really appreciate the help on this.