I'm using vs code and calva

can't really do this on top of cljs-eval. really need access to the compiler env which is on the clojure side.

There's no way to smuggle in a clojure command from the browser?

I'm okay with finding a hacky approach to this; doesn't need to go through cljs_eval

well the common tools people use for autocomplete and such all run over nrepl

there is no direct way to gain access to nrepl from the browser though

Maybe there should be? ;)

I don't like nrepl at all so I would be building this kind of thing over shadow.remote

You've been working on some really cool browser-based stuff with the shadow-cljs UI. It seems to me like you could come up with a lot of use-cases for something like that

yeah, I do have plans for all of that. just no time to do it all 😛

Haha I can imagine

the UI also uses shadow.remote for everything, everything it uses you also have access to

but yeah totally undocumented and might be changing a lot still 😛

Cool! Just read the rationale for shadow.remote and I think it sounds great

I don't really care if it's undocumented/alpha

If i wanted to use it to do stuff like this, how would I get started? :)

https://github.com/thheller/shadow-cljs/blob/b8466f176b6e8f6bf61cd70fbfbaf4762c581eb1/src/main/shadow/cljs/devtools/client/shared.cljs#L430

so cljs-eval already uses it too 😉

so you would add a namespace to you build loaded via :preloads

that would need to call this fn https://github.com/thheller/shadow-cljs/blob/b8466f176b6e8f6bf61cd70fbfbaf4762c581eb1/src/main/shadow/cljs/devtools/client/shared.cljs#L67

so (add-plugin! ::foo #{} init-fn stop-fn)

init-fn gets one parameter being the environment, including the runtime and all the dependencies

the runtime you can use to receive and send messages

I mean it gets very involved very quickly but the REPL is implemented on top of all of this

https://github.com/thheller/shadow-cljs/blob/b8466f176b6e8f6bf61cd70fbfbaf4762c581eb1/src/main/shadow/remote/runtime/eval_support.cljs

https://github.com/thheller/shadow-cljs/blob/b8466f176b6e8f6bf61cd70fbfbaf4762c581eb1/src/main/shadow/cljs/devtools/client/shared.cljs#L383-L386

basically you add the plugin, it on init then registers extensions

and then they talk over the shadow.remote socket

alright

the entire UI inspect and tap thing are also just plugins https://github.com/thheller/shadow-cljs/blob/b8466f176b6e8f6bf61cd70fbfbaf4762c581eb1/src/main/shadow/cljs/devtools/client/shared.cljs#L374-L381

but yeah not documented anything here and the CLJ side is currently not using this plugin mechanism

so extending that side is harder

got it

so if I wanted to add a plugin that (as a first step) lets me list all loaded namespaces, how would I set it up?

the worker side on the server implements everything using a multi method

https://github.com/thheller/shadow-cljs/blob/e9093aed586d5fb49efd94555787f0bef08fb164/src/main/shadow/cljs/devtools/server/worker/impl.clj#L688

from what you wrote earlier I assume it has to deal with clj?

and the worker you'd need to talk to to get autocomplete and stuff

alright

so the clients sends this https://github.com/thheller/shadow-cljs/blob/b8466f176b6e8f6bf61cd70fbfbaf4762c581eb1/src/main/shadow/cljs/devtools/client/shared.cljs#L248 and the worker handles it in the multi-method above

does all the cljs code you linked run in the browser? or is there a cljs part on the backend too?

there is no cljs part in the backend no

okay, great

I think this should be enough to get me started

are there any specs or similar that describe the messages being passed around here?

you probably shouldn't use any of the existing messages so no

basically you would (defmethod impl/do-relay-msg ::my-auto-complete [worker-state msg] ...) somewhere in your code on the CLJ side

alright, so my messages can be arbitrary data?

yes, just clojure maps

the two mandatory fields you need are these

https://github.com/thheller/shadow-cljs/blob/b8466f176b6e8f6bf61cd70fbfbaf4762c581eb1/src/main/shadow/cljs/devtools/client/shared.cljs#L248-L249

what about e.g. the stuff that gets passed to the add-plugin fns?

so an :op you choose and the :to

any specs for that?

got it

:to tells it to direct the messages to the worker, which can then answer via the multi-method

the multi-method must return the worker-state it gets. in the worker-state there is a :build-state which has :compiler-env

which has the :cljs.analyzer/namespaces you'd want to look at for autocomplete

okay

hey also, do you think it would be possible to use this approach to get require working from my browser-based repl?

(add-plugin! ::plugin #{}
  (fn [{:keys [runtime] :as env}]
    (api/add-extension runtime ::auto-complete {})

    (shared/call runtime
      {:op ::complete-me
       :to env/worker-client-id
       :foo "bar"}
      
      {::complete-result
       (fn [msg]
         (js/console.log "result from worker" msg))})

    env)

  (fn [{:keys [runtime]}]
    (api/del-extension runtime ::auto-complete)))

basic could look something like this on the client side

alright, thanks

seems straightforward enough 🙂

(defmethod impl/do-relay-msg ::complete-me
  [worker-state {:keys [foo] :as msg}]
  
  (impl/relay-msg worker-state msg
    {:op :complete-result
     :yo []})

  worker-state)

server side

note that you need to add the multi-method implementations before starting the worker

otherwise it may not pick them up properly

what do you mean get require working? cljs-eval can require just fine?

but yeah, given that the entire REPL is based on this you can do whatever too

hmm okay

I had some problems with require earlier, but it might just have been an error on my part

I'll take a closer look at it

anyway thanks; looking forward to testing this out 🙂

I think these are Calva issues. Please feel invited to post in #calva about it.

Oh, the reason I don't just help you here is that I am short on time. But hopefully in the #calva channel someone else can help.

Thanks. I will repost this in #calva.

https://clojurians.slack.com/files/UPWHQK562/F03CX51NS8P/untitled.cpp It's the only user.clj I have actually.

$ find . -iname "user.clj"
./src/dev/user.clj

the find doesn't mean anything since I don't know your full classpath setup

but none of the above is related to shadow-cljs in any way

just try running clj -A:shadow-cljs

I assume that gives you the same errors?

if you try (enumeration-seq (.getResources (.getContextClassLoader (Thread/currentThread)) "user.clj"))

I assume you mean clj -A:shadow-cli because that's the name of the alias I have. Nope, it doesn't give me errors.

(#object[java.net.URL 0x2e01e8ae "file:/home/alex/repos/personal-projects/example/src/dev/user.clj"])

and without this file?

or just without src/dev declared?

If I don't include :extra-paths ["src/dev"], that's when I get the above exception.

and you are running this all directly from the command line directly AND manually

no editor involved of any kind? or any other tool supposed to "help out"

I mean something must trigger the loading of all that stuff in the exception.

there seems to be a socket server involved in some way?

I am running it from my command line, no editor. When you gave me that snippet to look for user.clj, I used clj -A:shadow-cli clj-repl to evaluate it.

that already does too much

I meant literally clj -A:shadow-cljs

maybe need to comment out the :main-opts in deps.edn

just trying to narrow down on when and how the error happens

might also be related to core.async in some way. or some AOT classes you have somewhere

You do mean :shadow-cli as I have it in my shadow-cljs.edn, yeah?

I'm only guessing since I can't see your full setup

I assume you mean deps.edn. yes, that one

I mean you can just go with clj. that should be fine as well

trying to get you to a blank clojure REPL that doesn't crash on startup somehow

do you have other :main-opts or :jvm-opts configured in your deps.edn?

Not at the top level, only under aliases that I'm not activating.

so what happens with just clj or clj -A:dev

Same exception. Unless I add src/dev.

Should I take this to #clojure instead? I guess this rules out shadow.

do you have anything in ~/.clojure/deps.edn?

or I guess you can try clj -Srepro -A:dev to rule that out?

I mean your entire setup is full of red flags for me but should still be possible to figure out exactly what is causing your issues

I've been making do with it for local dev, but now that I'm looking at deployment stuff is indeed screaming at me.

Yes, I do have some stuff in my global deps:

{:aliases
 {:cider-clj
  {:extra-deps {org.clojure/clojure {:mvn/version "1.10.1"}
                cider/cider-nrepl {:mvn/version "0.21.1"}}
                ; fipp {:mvn/version "0.6.23"}}
   :main-opts ["-m" "nrepl.cmdline"
               "--middleware" "[cider.nrepl/cider-middleware]"]}
  :cljs
  {:extra-deps {nrepl/nrepl {:mvn/version "0.7.0"}
                cider/piggieback {:mvn/version "0.4.1"}}}
                ; fipp {:mvn/version "0.6.23"}}}
  :nREPL
  {:extra-deps {nrepl/nrepl {:mvn/version "0.8.1"}}}
  :repl/reveal {:extra-deps {vlaaad/reveal {:mvn/version "1.3.270"}
                             lambdaisland/deep-diff2 {:mvn/version "2.0.108"}}
                :ns-default vlaaad.reveal
                ;; note Fira Code is a ligature font so it's not supported in Reveal, whjch only uses monospaced
                :jvm-opts   ["-Dvlaaad.reveal.prefs={:theme,:light,:font-family,\"Fira Code\",:font-size,8}"]
                :repl-options {:nrepl-middleware [vlaaad.reveal.nrepl/middleware]}
                :exec-fn repl}}}

as long as its still is aliases that aren't activated that is fine

did you at some point experiments with AOT maybe? are there a lot of .class files in your project somewhere?

Not that I am aware of. I can't find any .class files from my root and I haven't knowingly used any AOT stuff. Should I try just blowing away my .m2 and pulling everything anew?

check which core.async` version you end up with just in case

clj -A:dev -Stree or so

nevermind I'm seeing things. those are spec issues not core.async

I mean yeah killing your .m2 might help in some way

I have a largely unfounded suspicion this is somehow related to https://github.com/fulcrologic/guardrails, but don't really have any evidence to back up the hunch.

could be, but finding how it gets loaded would help

Let me kill .m2.

Sorry for the noob troubles being so time consuming here @U05224H0W. Thank you for helping out.

I think it worked...

I have no idea how I could have broken anything in the Maven cache. I guess I'll just get in the habit it nuking it when things stop working smoothly.

Is there some way I could verify the output to make sure it built correctly?

{:clojure.main/message
 "Execution error (OutOfMemoryError) at java.util.Arrays/copyOf (Arrays.java:3537).\nJava heap space\n",
 :clojure.main/triage
 {:clojure.error/class java.lang.OutOfMemoryError,
  :clojure.error/line 3537,
  :clojure.error/cause "Java heap space",
  :clojure.error/symbol java.util.Arrays/copyOf,
  :clojure.error/source "Arrays.java",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type java.lang.OutOfMemoryError,
    :message "Java heap space",
    :at [java.util.Arrays copyOf "Arrays.java" 3537]}],
  :trace
  [[java.util.Arrays copyOf "Arrays.java" 3537]
   [java.lang.AbstractStringBuilder ensureCapacityInternal "AbstractStringBuilder.java" 228]
   [java.lang.AbstractStringBuilder append "AbstractStringBuilder.java" 682]
   [java.lang.StringBuffer append "StringBuffer.java" 393]
   [java.io.StringWriter write "StringWriter.java" 122]
   [java.io.PrintWriter write "PrintWriter.java" 541]
   [java.io.PrintWriter write "PrintWriter.java" 558]
   [java.io.PrintWriter print "PrintWriter.java" 685]
   [clojure.data.json$write_string invokeStatic "json.clj" 317]
   [clojure.data.json$write_string invoke "json.clj" 290]
   [clojure.data.json$fn__262$G__257__269 invoke "json.clj" 286]
   [clojure.data.json$write_array invokeStatic "json.clj" 348]
   [clojure.data.json$write_array invoke "json.clj" 342]
   [clojure.data.json$fn__262$G__257__269 invoke "json.clj" 286]
   [clojure.data.json$write_object invokeStatic "json.clj" 335]
   [clojure.data.json$write_object invoke "json.clj" 319]
   [clojure.data.json$fn__262$G__257__269 invoke "json.clj" 286]
   [clojure.data.json$write_object invokeStatic "json.clj" 335]
   [clojure.data.json$write_object invoke "json.clj" 319]
   [clojure.data.json$fn__262$G__257__269 invoke "json.clj" 286]
   [clojure.data.json$write_array invokeStatic "json.clj" 348]
   [clojure.data.json$write_array invoke "json.clj" 342]
   [clojure.data.json$fn__262$G__257__269 invoke "json.clj" 286]
   [clojure.data.json$write_object invokeStatic "json.clj" 335]
   [clojure.data.json$write_object invoke "json.clj" 319]
   [clojure.data.json$fn__262$G__257__269 invoke "json.clj" 286]
   [clojure.data.json$write invokeStatic "json.clj" 475]
   [clojure.data.json$write doInvoke "json.clj" 424]
   [clojure.lang.RestFn invoke "RestFn.java" 425]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.RestFn applyTo "RestFn.java" 132]
   [clojure.core$apply invokeStatic "core.clj" 671]],
  :cause "Java heap space"}}

There are some places in shadow-cljs where JSON writing could easily be made more memory efficient, like e.g. in this code:

(spit target
  (json/write-str sm-index))

just set something like :jvm-opts ["-Xmx2G"] when building

which java version is this using? older JVM versions didn't detect container memory correctly, so they used the actual machine memory as a reference

which often ends up consuming way more than allocated to the container image

if you use a newer jvm version that usually sorts itself out too, older require manual settings

fine to leave this in always, shadow-cljs doesn't really need that much memory. the default jvm settings are a bit greedy

Yeah, that would require doubling my client's spending on Heroku because AFAIC you can't get a separate build dyno - it all runs with the same limits and the current RAM limit is 1 GB, which is plenty for the app itself. JDK 17

hmm then maybe your build is actually too large an breaks the build 😉

I'll probably switch to building an uberjar or a full-on container locally. It's been a long time coming.

never ever inline json files by the way if they are above 20 bytes or something

oh nvm it was the source map generation

Well, I do inline KBs worth of static data in other projects - where it really makes sense. :) But not here.

yeah dunno. I've seen it do big builds with just one gig allocated

FWIW the resulting release bundle is 5 MB and the source map is 29 MB. No clue where it's at on the "big builds" scale.

yeah release source maps can get a bit out of hand