nbb

ag 2024-10-05T22:00:30.420749Z

I used core.async's pipeline in the past and it worked well for me. I don't want to pick on old warts, IIRC core.async is not the best choice for using with nbb (if it works at all). Can someone please point to an example where something gets processed in parallel, like starting a bunch of encoding jobs and waiting for them to finish, stuff like that. Thanks!

borkdude 2024-10-05T22:01:34.458529Z

I would use the promesa library for this or just plain promises.

borkdude 2024-10-05T22:01:40.785839Z

promesa is built-in to nbb

ag 2024-10-05T22:02:44.479219Z

Yeah, I know that, it's just been a while since I touched that lib, I forgot everything, I was hoping to see some refresher of a concrete example

borkdude 2024-10-05T22:06:20.220409Z

depends. does your encoding lib have an async API?

borkdude 2024-10-05T22:08:25.353019Z

if so, then it would be something like:

(let [x (encoding/job) ;; returns promise 1
      y (encoding/job) ;; returns promise 2
  (p/all [x y]))
The result of this would be one promise with the final result of the two promises in a vector

ag 2024-10-05T22:09:13.281699Z

does your encoding lib have an async API?Hmm... not using any lib, I just need to start off a bunch of ffmpeg jobs (in parallel) and then after that feed the resulting file(s) to some other thing, etc.

borkdude 2024-10-05T22:09:54.719139Z

so you would shell out to ffmpeg then right?

ag 2024-10-05T22:10:06.966229Z

yup

borkdude 2024-10-05T22:10:25.774919Z

any specific reason to use nbb for this and not bb (which does have core.async btw)

borkdude 2024-10-05T22:11:44.952799Z

you can still do this with nbb like the above, but the (encoding/job) would have to be replaced with a call from the "process" API Let me look something up, I do have an example somewhere

ag 2024-10-05T22:12:12.180509Z

any specific reason to use nbbAhem... yeah, that's an interesting question 🙂 I started playing around with nbb, for some completely unrelated reason, and now I want to add this thing that I conceptualized in my head

borkdude 2024-10-05T22:12:51.371899Z

import { spawn } from 'node:child_process';

function cmd(...command) {
  const p = spawn(command[0], command.slice(1), { stdio: 'inherit' });
  return new Promise((resolveFunc) => {
    p.on("exit", (code) => {
      resolveFunc(code);
    });
  });
}

borkdude 2024-10-05T22:13:04.824199Z

you need to translate this to nbb code, but you'll manage

borkdude 2024-10-05T22:13:25.808969Z

then you just run (cmd "ffmpeg" "etc") which returns a promise

borkdude 2024-10-05T22:13:36.791999Z

and combine it with what I said before

ag 2024-10-05T22:13:43.420899Z

Yeah, no.. but the question is, how do I do all that in parallel? If I need to run like dozens of ffmpeg jobs?

borkdude 2024-10-05T22:13:57.279669Z

call cmd a dozen times

borkdude 2024-10-05T22:15:22.055309Z

gotta go now, sleep time, but I think I provided all the building blocks that you'll need here

borkdude 2024-10-05T22:15:42.165349Z

note that cmd is async, not synchronous, so calling it multiple times will kick off jobs in parallel

ag 2024-10-05T22:16:47.992079Z

Yeah, okay. I'll fool around this thing, will see how it plays out. Thank you! Have a good night.

ag 2024-10-07T20:16:26.461749Z

I ended up using bullmq

borkdude 2024-10-07T20:17:56.572279Z

nice