looks fine to me

need to track the cljs-runtime files in development since that contains the JS code tailwind can read

since each namespace is a separate file. in release builds its all in one file.

how does it oom? I mean the files aren't that large?

you don't really need to track the sources, only if you also have CLJ server side code

Well, this happens even on the skeleton project

I have minimal cljs setup, but still

Let me check if I can make a very quick repro case for you

1. Have deps-new (https://github.com/seancorfield/deps-new) as a tool 2. Add an alias in your clojure.edn to use clojure 1.12 by adding :1.12 {:override-deps {org.clojure/clojure {:mvn/version "1.12.0-alpha7"}}} in your global aliases (just used for the template generation below) 3. Create a sample repo from using my template with clojure -A:1.12 -Tnew create :template io.github.agorgl/clj-fullstack%agorgl/clj-fullstack :name com.example.sample-poc 4. Change the pedestal.jetty dependency in deps.edn to version 0.7.0-SNAPSHOT (template fetches latest non-snapshots dynamically, but I've been using some stuff from the latest pedestal snapshot, this is required for build) 5. Run npm install to install package.json deps 6. Run npm run watch for the first time, let it finish 7. Stop npm run watch and run it again, now let it finish and wait a few seconds 8. Boom in the tailwindcss process (OOM)

By the way, why do we need to track cljs-runtime directory in addition to original src directories?

Boom in the tailwindcss process (OOM)

what does that mean exactly? what is the actual error?

I mean how much memory does it consume when you run it without watch?

It prints a stack trace from node (tailwind) process I have a RAM widget and I can see it glide up 4Gigs in 3-4 seconds before the OOM stacktrace in node

I can repro and copy paste the stacktrace here really easily (so can you with the steps above if you want)

can you create the repo for this? I can't actually help debug tailwind but I'm curious if it happens on my machine

Yes everything is set, just follow the steps above

It is not a repo, but it generates one from my template, its pretty easy to get it going

I can make a repo if you want to skip steps 1-4 though

@U05224H0W Here is a poc to make it easy for you: 1. git clone 2. npm install 3. npm run watch and wait till you reach [:app] Build completed. then Ctrl+C 4. npm run watch (again) and wait a few seconds after you reach [:app] Build completed. 5. You should see the oom crash on the tailwindcss node process, something in the lines of:

[shadow:watch  ] shadow-cljs - HTTP server available at 
[shadow:watch  ] shadow-cljs - server version: 2.27.5 running at 
[shadow:watch  ] shadow-cljs - nREPL server started on port 8777
[shadow:watch  ] shadow-cljs - watching build :app
[shadow:watch  ] [:app] Configuring build.
[shadow:watch  ] [:app] Compiling ...
[shadow:watch  ] [:app] Build completed. (172 files, 0 compiled, 0 warnings, 3.33s)
[tailwind:watch]
[tailwind:watch] <--- Last few GCs --->
[tailwind:watch]
[tailwind:watch] [18140:0x5fece4307220]    31011 ms: Mark-Compact (reduce) 2047.1 (2083.3) -> 2046.4 (2083.6) MB, 308.73 / 0.00 ms  (+ 2.9 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 326 ms) (average mu = 0.313, current mu = [18140:0x5fece4307220]    31502 ms: Mark-Compact (reduce) 2047.4 (2083.6) -> 2046.6 (2083.8) MB, 341.71 / 0.00 ms  (+ 0.7 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 360 ms) (average mu = 0.308, current mu =
[tailwind:watch]
[tailwind:watch] <--- JS stacktrace --->
[tailwind:watch]
[tailwind:watch] FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
[tailwind:watch] ----- Native stack trace -----
[tailwind:watch]
[tailwind:watch]  1: 0x5fece042b083  [node]
[tailwind:watch]  2: 0x5fece0830434 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
[tailwind:watch]  3: 0x5fece083082b v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
[tailwind:watch]  4: 0x5fece0a3c2dc  [node]
[tailwind:watch]  5: 0x5fece0a539d7 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
[tailwind:watch]  6: 0x5fece0a308d0 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
[tailwind:watch]  7: 0x5fece0a31919 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
[tailwind:watch]  8: 0x5fece0a136e3 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
[tailwind:watch]  9: 0x5fece0de728a v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
[tailwind:watch] 10: 0x5fec811e53f6 

apart from the duplicated namespace structure the repo looks fine

cannot really help with debugging tailwind, dunno what it has issues with

https://github.com/tailwindlabs/tailwindcss/issues?q=is%3Aissue+JavaScript+heap+out+of+memory+is%3Aclosed

doesn't seem uncommon though

I'm starting to believe that the parallelism of the watchers maybe is the problem in this

Like the tailwind watcher could start working before the shadow-cljs compilation finishes

how much actual memory do you have?

16G

Yeah I could increase the node vm memory

You want to see if this happens with bigger available memory too?

you can limit the memory shadow-cljs uses, but no clue about tailwind

I added NODE_OPTIONS="--max-old-space-size=8192" to tailwind:watch command and kinda circumvented the oom, although the behavior is still strange

A plain npx tailwindcss -i ./src/css/tailwind.css -o ./resources/public/css/compiled/site.css (without watcher) in another terminal after a shadow-cljs compilation takes 1 sec

Even with the node options above when in watcher mode it seems that shadow-cljs triggers the tailwindcss watcher multiple times and that is what causes the problem

I need some kind of debounce for this

could be that tailwind is just a bit eager and triggers for every file watched?

I suppose it could be

really no clue, I don't use tailwind anymore

Yeah I've seen your shadow-css I might switch to that at some point

I just needed compatibility with stuff like shadcn-ui for now 🙂

I'm starting to get somewhere with this: • I start tailwindcss watcher on one terminal using npx tailwindcss -i ./src/css/tailwind.css -o ./resources/public/css/compiled/site.css --watch, all good • I compile my cljs sources in another terminal using npx shadow-cljs compile app everything works well and fast. I repeat this multiple times, no problems appear • I start shadow-cljs watcher in the terminal I was doing the compile runs, with npx shadow-cljs watch app and I see that the tailwindcss watcher is rapidly consuming memory till the oom So my question is, does npx shadow-cljs watch app do anything different from npx shadow-cljs compile app command apart than watching the cljs sources? When starting npx shadow-cljs watch app shouldn't behave like calling npx shadow-cljs compile app just once (until some source changes?)

watch injects more code, for hot-reload/REPL and stuff

so compile is a bit more compact I guess

Hmm I see

watch also consumes more memory, since it holds the entire build state in memory longer than compile would

well my problem is not in the shadow-cljs watcher, but for some reason the js it produces makes tailwindcss watcher to blow up

watch doesn't inject that much more code, so dunno why that would be a problem

I'll make a diff and I'll try to pinpoint this

does it work if you run a watch, let it build and then shut it down? as in shadow-cljs shut down, not tailwind

Case 1: • Launch npx shadow-cljs watch app let it build, Ctrl + C • Launch npx tailwindcss -i ./src/css/tailwind.css -o ./resources/public/css/compiled/site.css --watch in another terminal, it works, no OOM Case 2: • Launch npx tailwindcss -i ./src/css/tailwind.css -o ./resources/public/css/compiled/site.css --watch leave it open • Launch npx shadow-cljs watch app in another terminal, let it build, wait a few seconds, tailwindcss watcher in the first terminal crashes with OOM

So I suppose according to this, the problem aren't the sources per se, but the way that shadow-cljs watch writes the files in the disk, triggering tailwind watcher multiple times or something?

so its running OOM because shadow-cljs + tailwind consume too much memory together?

so you could try to limit the JVM memory used, and as such make more available to node/tailwind

The OOM is not in the system, its JUST in the npx tailwindcss process

In the above examples, I use 2 terminals, one for each watcher

So I suppose with 2 npx invocations we get 2 node vms behind them

I don't know what you mean. OOM is still OOM, doesn't really matter how you ran the things that consume memory

When I say OOM I'm refering to the crash that npx/node prints out to console and shuts down the command

E.g.

❯ npx tailwindcss -i ./src/css/tailwind.css -o ./resources/public/css/compiled/site.css --watch                                         49s

Rebuilding...

Done in 1099ms.

<--- Last few GCs --->
 =[108186:0x62f74ab9a220]   282162 ms: Mark-Compact (reduce) 2047.3 (2083.1) -> 2046.6 (2083.6) MB, 303.68 / 0.00 ms  (+ 3.2 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 326 ms) (average mu = 0.473, current mu =[108186:0x62f74ab9a220]   282694 ms: Mark-Compact (reduce) 2047.6 (2083.6) -> 2046.8 (2083.8) MB, 374.33 / 0.00 ms  (+ 6.2 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 402 ms) (average mu = 0.384, current mu =

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0x62f747a2b083  [node]
 2: 0x62f747e30434 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 3: 0x62f747e3082b v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0x62f74803c2dc  [node]
 5: 0x62f7480539d7 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 6: 0x62f7480308d0 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 7: 0x62f748031919 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0x62f7480136e3 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
 9: 0x62f7483e728a v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x62f6e87e53f6
zsh: IOT instruction (core dumped)  npx tailwindcss -i ./src/css/tailwind.css -o  --watch

yes

My system still has lots of memory

node must limit the process to 4Gigs of memory or smth

apparently it doesn't

I mean how else do you explain this behavior? if you kill a shadow-cljs watch it doesn't touch the files or modify them in any way after being killed

so they are the exact same, and you just confirmed that tailwind is fine when its dead?

so the combination of both watches consumes too much memory

I think the problem is the watch trigger

you can trigger it manually

If I build with shadow-cljs watch app and kill it, I have the files written before I start the tailwindcss watcher

just touch one of the files or something without shadow-cljs running?

Yes but that does not replicate the shadow-cljs watch functionality, which I suppose touches all the files?

(at least when starting)

well, when starting it writes all files yes

on incremental updates it only touches the files that changed

so this is most probably the case

tailwindcss watcher watches all the files

shadow-cljs watcher writes all the files at startup

tailwindcss is being triggered multiple times in a few ms, and is not smart enough to 'debounce' the build

and tailwindcss process misbehaves leading to oom

well yeah I guess if it triggers immediately for each file thats a problem

btw, just tried starting shadow-cljs watch app, let it build, kill it, start the tailwindcss watcher, touch a file from the output of shadow-cljs watch app and it does not have a problem

Another question I need to ask is: What files does shadow-cljs watches? I'm afraid that it could also be a cyclic dependency like shadow-cljs builds files -> tailwindcss gets trigger and builds css file -> shadow-cljs tries to build files again or smth?

assuming that tailwind doesn't touch or generate any JS files, so it won't trigger a rebuild

but in case you are on a mac there have been reports that icloud syncing can cause additional compiles. as in the project folder being synced to icloud

Nop, I'm on a pretty bare bones Linux laptop

No fancy stuff here

well, then you can check if something funky is happening in the shadow-cljs terminal log. it'll log when it compiles something, so you should be able to identify recompile loops

the one on stdout?

Can I somehow enable verbose logging to print a line for each file built in shadow-cljs?

[:app] Build completed. (172 files, 0 compiled, 0 warnings, 3.33s)

that one

I want to see the rate that it writes files on startup

Yeah no build loops seem to happen

:log {:level :debug} in shadow-cljs.edn top level logs a bit more

npx shadow-cljs watch app --verbose logs more compile progress

I suppose the write operations are the flush ones?

correct

I guess you could try skipping the cljs-runtime dir and only run over the sources

but in the past tailwind didn't recognize some cljs code properly and didn't find many aliases

Yeah that is my main question now, why do I even watch cljs-runtime dir

I'm using tailwindcss with hiccup and :class "<tw space separated classnames here>"

Is there a specific case it won't work?

its been years, can't really remember

class strings should be fine

The jackshae repo that comes first in google does not have an explanation either

an argument for running over the output files is that it can also find tailwind uses in libraries, but I guess those aren't very common

for release builds I'd definitely run over just the output, but for dev I guess it doesn't matter too much

Hi @U03PYN9FG77 I'm using tailwind_watcher.clj which is a stripped down version of the one you are using. Basically I'm not using the release bits. I can confirm that I had a Javascript out of memory once. I don't know how to repeat but i suspect I've clicked force compile in a already watched project.

I'm using this conf in my shadow-cljs.edn app build to pass parameters to the watcher, and it to be invoked more integrated with the cljs compilation.

:tailwindcss {:input "./styles/globals.css"
              :config "./tailwind.config.js"
              :output "./resources/public/css/app.css" }
:build-hooks [(tailwind-watcher/start-watch!)

babashka/process {:mvn/version "0.5.22"} ;; Necessary to invoke tailwindcss

An update for anybody that stumbles upon this in the future: I replaced the tailwindcss watcher with the more generic watchexec that supports debounce and seems way more stable. Now my package.json is:

{
  "name": "@com.example/clj-fullstack-sample",
  "scripts": {
    "watch": "run-p -l *:watch",
    "release": "run-s -l *:release",
    "shadow:watch": "npx shadow-cljs watch app",
    "shadow:release": "npx shadow-cljs release app",
    "tailwind:watch": "node -p \"require('./tailwind.config').content.map(x => x.replace(/^\\.\\//, '')).join('\\\\n')\" > .twpath; RUST_LOG= WATCHEXEC_FILTER_FILES=.twpath watchexec -q -d 1000 --no-vcs-ignore -- npx tailwindcss -i ./src/css/tailwind.css -o ./resources/public/css/compiled/site.css",
    "tailwind:release": "NODE_ENV=production npx tailwindcss -i ./src/css/tailwind.css -o ./resources/public/css/compiled/site.css --minify",
    "babel:watch": "npx babel src/js -d gen/js -x \".js,.jsx,.es6,.es,.mjs,.cjs,.ts,.tsx\" --watch",
    "babel:release": "npx babel src/js -d gen/js -x \".js,.jsx,.es6,.es,.mjs,.cjs,.ts,.tsx\""
  },
...
}

I use node to extract the content paths from the tailwind.config.js, write them to a file and pass that to watchexec to avoid duplication of the watch paths. Then watchexec uses debounce time of 1 sec to call the regular npx tailwindcss build command to run the tailwindcss build every time a relevant file changes

still don't understand how it runs out of memory? seems like memory use should be constant if it just runs over the same sources multiple times?

hmm I guess maybe it triggers while its still building the previous run?

@U05224H0W the problem is not the compile process but the watcher

yes exactly

it does not have any debounce mechanism so I believe the tailwindcss watcher triggers multiple times in a short timespan and this leads to this behavior