Fork me on GitHub
#aleph
<
2023-02-22
>
Tom Fenton23:02:19

Hey all! I've been trying to get figwheel-main up in a command prompt, running alongside aleph. WebSocket connection etc. appears to be fine, BUT I can't live-reload code because aleph locks resources served via wrap-resource (i.e. the .js files figwheel-main is trying to update). This behaviour is not present in http-kit (does it serve the files without locking them?); however, http-kit doesn't support ssl directly so it's not an option. Is this locking intended and/or configurable behaviour in aleph? Apologies for any errors in the above, I'm very new to web dev 🙂

hiredman00:02:34

what makes you think a resource is being locked?

hiredman00:02:10

most likely you aren't actual using wrap-resource when testing with http-kit and instead are serving files

hiredman00:02:51

resources are blobs of data read from a classloader, they may be backed by files on disk, or may be entries in a jar, or where ever a classloader chooses to reader them from

hiredman00:02:41

in some cases classloaders will cache the contents of a resource, and not reread it again

Tom Fenton09:02:55

Hi @U0NCTKEV8, I used figwheel-main to reload my code under different circumstances. 1. Clojure REPL not running 2. Clojure REPL running 3. Clojure REPL running, aleph started 4. Clojure REPL running, aleph started, page served In the first attachment you can see that 1-3 are successful but 4 fails. Reloads continue to fail until I end the REPL process. I'm also unable to delete the file from the file system because it's locked by the java.exe process. I'm using wrap-resource as part of wrap-defaults (secure-site-defaults), but if I explicitly use the bare-minimum middleware (see second image) I get the same behaviour as soon as I add wrap-resource. I don't get the same behaviour if I swap in http-kit - the files remain writeable even after being served. Thanks for the insight about the classloader - maybe the two servers use different classloaders, or have them configured differently? But I don't know nearly enough to go under the hood in Netty 🙂

Matthew Davidson (kingmob)04:02:10

@UAH1JFQV9 Hmm, wrap-resource comes from Ring, so you may need to ask around on #C0A5GSC6T for help if the issue is unique to that middleware. If you're returning a deferred in Aleph and using Ring middleware that alters responses, you will have to wrap the middleware to handle that, because Ring doesn't understand deferreds. However, I suspect that's not the case here, because you don't have a custom handler based on your screenshot.

Matthew Davidson (kingmob)04:02:45

Looking at it a bit more closely, I suspect the classloader may be the culprit. Netty maintains its own threadpool, which doesn't use the clojure classloader. Try (wrap-resource "public" {:loader (.getClassLoader clojure.lang.Compiler)}) , something like that might work.

Tom Fenton22:02:37

Hi @U10EC98F5, thanks for taking the time to look into it! I tried experimenting with classloaders but no joy. After a lot of trial and error I think I've been barking up the wrong tree. Running with SSL results in resource locking, running without SSL does not (and http-kit doesn't support SSL so was unaffected...). I still don't fully understand why yet, but at least I have a workable compromise now - -main starts a secure server, (start-dev) starts an insecure one.

Matthew Davidson (kingmob)05:02:27

Hmmmm. Can we see some code? A minimal reproducing example would be great. It would help debug what's going on if we could see how you're initializing the server. What ssl-provider and trust-store are you passing in?

Matthew Davidson (kingmob)05:02:44

Also, have you talked to Bruce Hauman @U064J0EFR, the figwheel author, yet? He might have some insight.

Matthew Davidson (kingmob)05:02:37

Only other thing that immediately comes to mind is that placing the generated Js files under resources/ seems odd. As @U0NCTKEV8 mentioned, if the resources/ files are being cached, you'll have an issue. I double-checked, and the standard figwheel location for files is under target/, which seems more reasonable. I would place only finished files under resources/ as part of the final build steps.

Tom Fenton22:02:31

No problem - I've messaged you with details of a repo I've set up to show you where I'm at. Feel free to share if you think it's useful. It would be great if Bruce has any pointers, either about the locking issue or my poor folder structure 🙂 since I wasn't getting the behaviour in http-kit I'd assumed the issue was aleph-specific, but now I know SSL is a factor, I'll experiment next week - will let you know if target vs resources and e.g. Jetty SSL makes any difference. My project is very new (learning how to use WebSockets) so at the moment I'm just using a SelfSignedCertificate (io.netty.handler.ssl.util).

Matthew Davidson (kingmob)06:02:30

@UAH1JFQV9 Quick question before I look too closely at this: have you tried changing the :target-dir to "target", or somewhere that's not resources? It would help if you could rule that out first.

Tom Fenton00:02:59

Tested again - merged my code into a fresh project based on the official figwheel-main template. Template actually uses resources for the :target-dir, but even after changing it to target (and adding a wrap-resources to serve from target), the result is the same. Fine for http, locked when enabling SSL.

Matthew Davidson (kingmob)06:03:46

Hmm. What's weird is that Aleph literally does nothing with resources. It's possible it's a netty issue, but a quick search didn't turn up anything immediately relevant. Tom, I took a look at your minimal example, but it doesn't seem to be working, even in insecure mode. For starters, it would be helpful to strip out anything unnecessary to demonstrate the problem, like Sente, Timbre, and maybe the CSRF stuff. Next, I got some figwheel warnings about the classpath that I've never seen before (probably because your watch dir src/minimal_example/cljs instead of src):

❯ lein fig:dev
[Figwheel:WARNING] The watch directory "src/minimal_example/cljs" is not on the classpath! A watch directory must be on the classpath and point to the root directory of your namespace source tree. A general all encompassing watch directory will not work.
[Figwheel:WARNING] Attempting to dynamically add "src/minimal_example/cljs" to classpath!
[Figwheel:WARNING] Source directory "src/minimal_example/cljs" is not on the classpath
[Figwheel:WARNING] Please fix this by adding "src/minimal_example/cljs" to your classpath
 I.E.
 For Clojure CLI Tools in your deps.edn file:
    ensure "src/minimal_example/cljs" is in your :paths key

 For Leiningen in your project.clj:
   add it to the :source-paths key
After that, I ran lein repl, and the instructions are wrong. I assume it should be minimal-example.clj.core/start-dev, and not minimal-example.core/start-dev which doesn't exist. This runs and opens up the app page, but the reagent code never renders, the web page still says "Please wait - loading". (Fixing the watch dir didn't fix this) Next, I looked at the console, and saw a bunch of timbre and sente errors:
Uncaught SyntaxError: unexpected token: identifier
timbre.js:2:8
Uncaught TypeError: can't access property "call", taoensso.timbre.swap_config_BANG_ is undefined
    taoensso$sente$set_min_log_level_BANG_ sente.cljc:105
    <anonymous> sente.cljc:109
sente.cljc:105:2
Uncaught TypeError: can't access property "call", taoensso.sente.make_channel_socket_client_BANG_ is undefined
    <anonymous> web.cljs:18
web.cljs:18:7
Your readme says you tested this on Azul. I have no idea if that makes a difference, but it may be safer to run this on a more common JDK, just in case. (FWIW, the cljs community seems to have moved on to shadow-cljs. If you try it, I'd be very curious to know if you see the same problem.) If you get your minimal example working, I'll take another look

Tom Fenton20:03:39

Hi @U10EC98F5, As requested: 1. Removed Sente which clears up the code, but doesn't affect the observed behaviour 2. Changed :watch-dirs to prevent the figwheel-main classpath error 3. Corrected the readme 4. Corrected the Timbre/Sente errors which prevented the page from loading (serves me right for using an alpha version of Sente 🙂) but of course moot due to point 1 5. Tested in Oracle Java 1.8.0_361 I'll see if I can reproduce in Netty directly, maybe Jetty for good measure. I might also try shadow-cljs, though I'm sure this would exhibit the same behaviour since even Windows reports the files are locked - see attached. This only happens when serving https, and only after the page is served for the first time. For personal use I'll stick with figwheel-main - it works and it makes sense to me, and as non-Node.js user, the cost/benefit of migrating doesn't add up.

Matthew Davidson (kingmob)06:03:49

OK, I'll take a look when I can. Also, FWIW, shadow-cljs isn't specifically for node users. Plenty of cljs users use it for their front-end builds. Most, probably. It's big initial selling point was better npm compatibility, but not for backend node, iiuc.

Matthew Davidson (kingmob)06:03:29

@UAH1JFQV9 Can you move this to a GH Issue? I don't want to forget about it