I used core.async's pipeline in the past and it worked well for me. I don't want to pick on old warts, IIRC core.async is not the best choice for using with nbb (if it works at all). Can someone please point to an example where something gets processed in parallel, like starting a bunch of encoding jobs and waiting for them to finish, stuff like that. Thanks!
I would use the promesa library for this or just plain promises.
promesa is built-in to nbb
Yeah, I know that, it's just been a while since I touched that lib, I forgot everything, I was hoping to see some refresher of a concrete example
depends. does your encoding lib have an async API?
if so, then it would be something like:
(let [x (encoding/job) ;; returns promise 1
y (encoding/job) ;; returns promise 2
(p/all [x y]))
The result of this would be one promise with the final result of the two promises in a vectordoes your encoding lib have an async API?Hmm... not using any lib, I just need to start off a bunch of ffmpeg jobs (in parallel) and then after that feed the resulting file(s) to some other thing, etc.
so you would shell out to ffmpeg then right?
yup
any specific reason to use nbb for this and not bb (which does have core.async btw)
you can still do this with nbb like the above, but the (encoding/job) would have to be replaced with a call from the "process" API
Let me look something up, I do have an example somewhere
any specific reason to use nbbAhem... yeah, that's an interesting question 🙂 I started playing around with nbb, for some completely unrelated reason, and now I want to add this thing that I conceptualized in my head
import { spawn } from 'node:child_process';
function cmd(...command) {
const p = spawn(command[0], command.slice(1), { stdio: 'inherit' });
return new Promise((resolveFunc) => {
p.on("exit", (code) => {
resolveFunc(code);
});
});
}you need to translate this to nbb code, but you'll manage
then you just run (cmd "ffmpeg" "etc") which returns a promise
and combine it with what I said before
Yeah, no.. but the question is, how do I do all that in parallel? If I need to run like dozens of ffmpeg jobs?
call cmd a dozen times
gotta go now, sleep time, but I think I provided all the building blocks that you'll need here
note that cmd is async, not synchronous, so calling it multiple times will kick off jobs in parallel
Yeah, okay. I'll fool around this thing, will see how it plays out. Thank you! Have a good night.
I ended up using bullmq
nice