Based on my rough mental calculation, if I render one HTML page at a time with one "synchronous" SCI script in one node.js process, I get a mixed I/O + CPU workload. For mixed I/O + CPU workloads, it seems running 3~5 times as many parallel tasks as the number of CPU cores is good enough for saturating CPU cores. If I rent a cloud computer with 8 CPU cores and 16GB RAM, it doesn't make sense to have more than 40 processes for my use case. With only 40 processes, I can easily set node.js max old size to 128MB. I would still have enough RAM left for other things.... With 256MB for each process, I would need to reduce the process count down to 4 processes per core. It seems this multi-process architecture is good enough for running untrusted SCI scripts against mixed I/O + CPU workloads. I can maybe even think about using babashka processes for server-side SCI HTML rendering instead of node.js. For IO workloads, I don't think you need custom computation which SCI is useful for. Thus, I think babashka/node.js processes are currently the best substrates for untrusted SCI scripts. If you are going to saturate a cloud computer's CPU cores with untrusted SCI scripts, then it makes sense to reserve a computer for SCI scripts. If it's reserved for SCI scripts, just give it enough RAM.
@jeroenvandijk I think this is my solution to your SCI problem. It's not the only possible way, but it's currently the only way. It is simple and makes sense.
Saving memory at all costs doesn't really make sense for CPU workloads and mixed I/O + CPU workloads.
Mixed I/O + CPU workloads are how ruby puma web server manages to be competitive.
node.js asynchronous code execution only makes sense for I/O-bound workloads.
I suppose your workloads are bottlenecked by CPU cores.