https://github.com/thheller/shadow-cljs/issues/127 @thheller I am really interested in th is feature. It would make the life for my coworker much easier. I would like to work on it, if you don't mind? Could you point me to the right starting point? 🙂

@timrichardt how so? this really isn't that relevant anymore? it only affects ES6 files in node_modules? why does that affect you?

There are people working on jsx Files. I babel them and then bundle with shadow. Errors inside the babel-output are not source mapped to the original jsx Files. Do you have any idea how to tell the browser the mapping from jsx->js?

how do you include them in the project?

They live in a separate folder /src/js along with the .js.map -Files.

ok so you are including via https://shadow-cljs.github.io/docs/UsersGuide.html#classpath-js ?

Yep.-

and the files do have the sourceMappingURL comment?

They do.

then source maps should already be working

Mh, ok.

I can take a look if you make a reproducible example. I haven't tested this in a long time so maybe its broken.

but it isn't what #127 is about so it would be a separate issue

I'll do a minimal example.

I basically only need a .jsx file and its converted form + map

https://github.com/timrichardt/jsx-cljs-template

I have included the js and map

@timrichardt I think the problem is that the map file needs to contain the sourcesContent. closure will not find the file otherwise.

ah nvm its already in there

is it possible to make babel inline the source map into the file?

I also saw this warning now.

------ WARNING #1 -  -----------------------------------------------------------
 Resource:
 Failed to resolve sourcemap at Component.js.map: Component.js.map

Babel has this option, let me try.

Inlining in principle works, but Chrome and Firefox seem to not recognize the source map comment.

I'll check it out later. can you add the inline option to the repo?

OK, i'll do. Thanks anyway.

@thheller Do you know if there is a way to use both :handler and :proxy-url with the :dev-http feature? After a quick look at the code, I think the answer is "no" but I wanted to check in case I missed something

ultimately what I am after is the ability to define what should be wildcarded to my SPA and what should be forwarded to the proxy

The follow-up question is: if there is not, would you be interested in a patch that does?

@ghaskins "define what should be wildcarded to my SPA" what does that mean? we are talking about :dev-http, what does the SPA have to do with that?

what I mean is: i have a backend with different endpoints...lets just say there is an identity provider on /oauth, an api backend on /graphql, and an admin ui at /admin....and /admin is a shadow-cljs compiled SPA

if you have a backend anyway why not use that directly? I mean why do you need the proxy?

what id like is to define something like a push handler that says /admin/* should be index.html, anything else should be forwarded

its just during development of the SPA

why do you need the proxy? does it proxy to some remote production server?

or is your backend running locally anyways?

right, though its a sandbox deployment, but same difference

no, its not local

its running in k8s in aws, except for my ui

the dev-proxy works great, except for the inability to specify what is a route handled in the spa

you are confusing me with your wording

sorry

"route handled in the SPA" to me means something handled by the client that never even reaches the server

we have a kubernetes architecture with a bunch of microservices...one type of microservice is a shadow-cljs compiled ui that has an SPA in it

during develoment, its nice to work on the UI code locally where I want any of the paths that would normally hit the UI backend to be served by dev-proxy, and all others to forward on

the SPA has kee-frame/reitit routes, so things like /admin/foo/bar are actually served from /admin/app.js

but everything else needs to forward to the server

ah you are using push state routes and don't want to forward those to the proxy?

like the server always give you the same html regardless of what path you ask for?

i think that is correct, though I am unsure of the terminology

right, or really once the /admin/[index.html, app.js] are running, it knows that foo/bar is a route inside the browser

which works great

but the problem is if someone tries to navigate to /admin/foo/bar externally, the proxy forwards that to the backend

i.e. it only works if you come in through the front index.html

yes but you handle that in production too right?

so what I was thinking is something like a push-handler that can be configued in conjunction with :proxy-url, and the push-handler can decide if it should be handled locally or forwarded

yes, prod is fine

once its in the k8s backend, the routing all works

this is just when the :dev-proxy is in front during dev

so the first question is: do you already allow this and I missed it? and if you don't, would you be willing to consider a patch for the feature?

looking at the :dev-proxy code, I suspect we can put an undertow handler in front of the LoadBalancingProxyClient in a fairly straight forward way

sorry, :dev-http code, i meant

what im thinking is something that would basically allow this and :proxy-url to both be configured, details TBD

hmm I'm undecided. usually I recommend running your own server if you need more complicated setups

thats an acceptable response, though I think what i need/suggest is not too far off from what you already offer

like how would you want to configure this?

one person may say that you want to exclude certain paths, another may want to include certain paths

im thinking it would look something like

{...
 :dev-http
 {8000
  {:root "public"
   :handler 
   :proxy-url ""}}}

yes but that doesn't work since the proxy always answers

> one person may say that you want to exclude certain paths, another may want to include certain paths Yeah, so to that point, the protocol for the :handler would need to allow it to express whether to serve locally or move on to the proxy

im thinking its either a path to a local file (e.g. "index.html" or nil, in which case the proxy is queried

the issue is that at the point where the handler is called the http request is in blocking mode and you can't proxy anymore

its been a while since I wrote an undertow backend, but I think one thing that may work is if we add an undertow handler in front of the reverse-proxy

since undertow already has handler chaining, IIRC

yes you can add a handler to decide which other handler to take next

right

thats what i was thinking was the integration point

but how would you configure that?

im thinking its the same :handler spec as above, and if it returns a path, we serve that, and if it doesnt, we move to the next handler

so in my example, I could map "/admin/*" to "index.html", and everything else is nil

or I could code it for the opposite, e.g. "/oauth/* -> nil

just trying to figure out how to express this in config without getting too complex

whatever the use case is

some kind of :proxy-skip

right

pointing to a (defn should-this-proxy [req] true|false)

or :proxy-skip [#"^/something" #"^/else"]

right..if we can assume that the non-proxy behavior of shadow.http.push-state/handle is what we want, then the new config really just needs to be a predicate

thats why I was thinking I wouldnt propose a new configuration knob, just allow the :handler to decide

but id be happy with the above, too

the handler can't decide because of how undertow works

ah, ok

as soon as it enters "ring" mode (which the handler is) I switch it to "blocking" mode

but proxy only works when its in the async mode

let me check how I did this in protojure

and as far as I know there is no way to go back to async mode

granted that worrying about this at all for a dev server is overkill

but I still want to do it correctly

i understand completely

so, details TBD, obviously, but this is what I was thinking

as per https://undertow.io/undertow-docs/undertow-docs-2.1.0/index.html#handlers

specifically "a handler can pick the next handler to invoke based on the current request."

yes I understand that

in protojure, im currently only installing one handler, just like shadow

https://github.com/protojure/lib/blob/master/src/protojure/pedestal/core.clj#L300

but im wondering if I hook it at the Undertow/builder level, if we can get both the :handler and :proxy-url components working together

i get what you are saying about blocking vs async mode

might have to play around with that

yeah the way this is built is quite strange but it made sense to me at the time I built it

i also understand that completely, heh

basically before the :handler is called it is passing through https://undertow.io/undertow-docs/undertow-docs-2.1.0/index.html#blocking-handler

ok, let me understand how you did that part

but this breaks when its in blocking mode https://undertow.io/undertow-docs/undertow-docs-2.1.0/index.html#reverse-proxy

i also forget the nuances of the undertow api...i remember there are things that you do that implicitly set the mode

I think I have an idea how to add what you want without getting too complicated

oh? that would be awesome

I sort of created a DSL for describing the handler setup

looks something like

[:shadow.cljs.devtools.server.dev-http/file-recorder
 {:on-request
  #object[shadow.cljs.devtools.server.dev_http$start_build_server$file_request_fn__19970 0x6fce3691 "shadow.cljs.devtools.server.dev_http$start_build_server$file_request_fn__19970@6fce3691"]}
 [:shadow.undertow/soft-cache
  [:shadow.undertow/ws-upgrade
   [:shadow.undertow/ws-ring
    {:handler-fn
     #object[shadow.cljs.devtools.server.dev_http$start_build_server$http_handler_fn__19965 0x6109c518 "shadow.cljs.devtools.server.dev_http$start_build_server$http_handler_fn__19965@6109c518"]}]
   [:shadow.undertow/compress
    {}
    [:shadow.undertow/file
     {:root-dir
      #object[java.io.File 0x7ee0cc9 "out\\demo-test-browser"]}
     [:shadow.undertow/classpath
      {:root "shadow/cljs/devtools/server/dev_http"}
      [:shadow.undertow/blocking
       [:shadow.undertow/ring
        {:handler-fn
         #object[shadow.cljs.devtools.server.dev_http$start_build_server$http_handler_fn__19965 0x6109c518 "shadow.cljs.devtools.server.dev_http$start_build_server$http_handler_fn__19965@6109c518"]}]]]]]]]]

ah, that probably explains the [::undertow/xx] stuff I saw

i didnt dig into that, but I was curious

yeah constructing the handlers by hand was quite annoying otherwise

it probably overkill but it solves the issues I wanted to solve 😛

i get it

very clever

its not quite clear to me what I need to do to express this in the DSL though...guidance appreciated

you can't express it in the DSL

ah, sorry...misunderstood

problem is that the "next" logic of the proxy handler requires this returning null

https://github.com/undertow-io/undertow/blob/cc993bcad219f449f452ac3a36021398d3cc48f1/core/src/main/java/io/undertow/server/handlers/proxy/ProxyHandler.java#L177

but it never does

https://github.com/undertow-io/undertow/blob/8d911b58acfc74caa3dc0f00b216dd78d4b8f76c/core/src/main/java/io/undertow/server/handlers/proxy/LoadBalancingProxyClient.java#L254

but I guess I can subclass that proxy client and have that return null whenever the clojure fn returns false

that sounds reasonable

or just leave all that alone and add a regular handler instead

probably simpler .. especially since subclassing in clojure is annoying

yeah...

the handlers are a simple reify, so probably easier than a full gen-class

though the 'next' concept seems to be only in the deprecated constructors

maybe they've changed the way the chaining works

please help me debug my build configuration:

:builds       {:app {:target     :npm-module
                      :output-dir "node_modules/shadow-cljs"}}}

[:app] Configuring build.
[:app] Build failure:
Invalid configuration
-- Spec failed --------------------

  {:target :npm-module,
   :output-dir "node_modules/shadow-cljs",
   :build-id :app}

should contain key: :entries

| key      | spec            |
|==========+=================|
| :entries | (coll-of        |
|          |  simple-symbol? |
|          |  :distinct      |
|          |  true           |
|          |  :min-count     |
|          |  1              |
|          |  :kind          |
|          |  vector?)       |

-- Relevant specs -------

:shadow.build.targets.npm-module/target:
  (clojure.spec.alpha/keys
   :req-un
   [:shadow.build.targets.shared/output-dir
    :shadow.build.targets.npm-module/entries]
   :opt-un
   [:shadow.build.targets.npm-module/runtime
    :shadow.build.targets.shared/devtools])
:shadow.build.config/build+target:
  (clojure.spec.alpha/and
   :shadow.build.config/build
   (clojure.spec.alpha/multi-spec
    shadow.build.config/target-spec
    :target))

-------------------------
Detected 1 error

altho

{:frontend
                 {:target :browser
                  :modules {:main {:init-fn }}
                  }}

:target :browser has different config that :target :npm-module

for :target :npm-module you need to specify :entries [at-least-one.of-your.namespaces]

basically you need to configure which namespaces you want to have available from JS

@thheller how do I choose which ns to add?

whichever you want to have available from JS?

why are you using :npm-module in the first place?

this is from react native app

react-native should be using :target :react-native and not :npm-module

I think I get the gist of your DSL now

@thheller im struggling to see where you serve the local-files in front of the proxy from

oh, i think i see it now

see :shadow.undertow/file in the DSL example from above

yep, i see it now

ty

it checks if the file exists and calls the next handler if it does not

@ghaskins https://github.com/thheller/shadow-cljs/commit/0e1d8f86921c69745b68c8e127dc78af19167f05

will make a release in a bit, just want to look at something else first

there are some cases where it may be easier to whitelist or blacklist what should be proxied, so I wonder if a predicate-fn is more flexible

for example, I have two UIs, one at /foo and another at the default / page, so the exclude works well for the former but not the latter...but a simple predicate-fn would work for both

so what im thinking is, maybe :proxy-pushstate-predicate returns true means invoke the :handler, false means invoke the proxy

thoughts?

and there could be a default pushstate-predicate that handles exclude/include options, if we wanted

is it possible to get the absolute path of a file (alternatively relative path + project directory)? in vanilla cljs cljs.analyzer/*cljs-file* does the trick, but with shadow-cljs I only seem to get the part of the path after the source directory

e.g. "hello_world/other.cljs" rather than "/Users/.../src/hello_world/other.cljs"

@saikyun what for? and what would it report for files from libraries in .jar files?

I’m trying to use the :target :bootstrap feature in Shadow and am running into trouble with macros. The :entries vector of my bootstrap build contains instaparse.core, and the build fails with:

The required namespace "instaparse.macros" is not available, it was required by "instaparse/core$macros.cljc".
"instaparse/macros.clj" was found on the classpath. Maybe this library only supports CLJ?

The required namespace "" is not available, it was required by "cljs/repl$macros.cljc".
"clojure/java/io.clj" was found on the classpath. Maybe this library only supports CLJ?

@adamrenklint the most likely explanation is that the library just doesn't support self-hosted use. especially if it imports namespaces like http://clojure.java.io

That’s the weird thing, I have created this analyzer cache before, and it definitely included Instaparse, but it was all done with Boot and is horribly slow to develop with, so I want to use Shadow instead.

But maybe you can explain the first error. Why is importing macros implemented using CLJ instead of CLJC throwing an error? The namespace is obviously there, the error message even states it.

I can't explain it sorry. It has been several years since I looked at and implemented this stuff and I cannot remember the fine details

there is a secondary pass that does the macro compilation

maybe you just need to exclude some

I honestly cannot remember any of this 😞 self-hosted confuses me when it comes to macros

@ghaskins 2.10.19 lets you specify :proxy-predicate some.ns/some-fn. should be (defn some-fn [ex config] true|false) where ex is the undertow HttpServerExchange (eg. (.getRequestURI ex) to get request path) and config is the :dev-http config map

should return true if the request should go to the proxy and false if the handler should be taken instead

@timrichardt I looked over the source map issue but I can't figure out how to make it work

Ty @thheller

Does anyone have a clue why I am getting DOMException: Failed to construct 'WebSocket': The URL 'ws://:8237/chsk?... Exceptions in my console, when I install a service worker? Everything works fine as far as I can see. Live reloading is working..

no clue. looks like you are trying to connection to ws without a host?