clojure

Ovi Stoica 2025-10-03T03:58:48.031899Z

Hello! What is the usual way or common way people build async workers? Let's say you have a monolith and you need to spawn, at regular times, or based on events, workers to do specific jobs. Usually this comes with a persistence layer like redis to maintain jobs that weren't finished if the server restarts

p-himik 2025-10-03T04:26:30.439399Z

> at regular times, or based on events Sounds like a perfect fit for a core.async go block with an alt! over a timeout and a channel that receives events. > Usually this comes with a persistence layer That would just mean that something like that go block put stuff onto this layer, and some other process (in the conceptual way, not in the "an OS process" sense) monitors it and executes stuff from it. So, somewhat orthogonal to the "timeout and events" part - "somewhat" because of the part below. That leaves the bit of run time between something triggering an event and the go block putting it onto the persistent layer. If you care about this gap, the event itself could be delivered via such a persistent layer. Easy to do if you don't need the "exactly once" guarantee. Otherwise, core.async is, I think, a wrong thing to use here.

👍 1
Ovi Stoica 2025-10-03T05:30:48.672069Z

Found this post which is similar to your suggestion: https://blog.janetacarr.com/creating-a-dead-simple-asynchronous-job-system-in-clojure/ I also found https://github.com/msolli/proletarian which is tied to SQL DBs but still is nice

2025-10-03T10:29:09.439029Z

#caveman also uses proletarian in case you want to see an example: https://caveman.mccue.dev/tutorial/clojure/23_run_jobs_in_the_background

🚀 1
Ovi Stoica 2025-10-03T10:30:03.660169Z

Thank you, that’s very helpful!

Ovi Stoica 2025-10-03T10:30:30.744799Z

Does anybody know of example job “systems” using carmine? My setup uses datomic so I’m not SQL based

lukasz 2025-10-03T14:21:53.278249Z

I used Carmine's worker stuff in the past, it worked fine

2025-10-03T16:28:38.206679Z

There's also https://github.com/nilenso/goose And you can also use Quartz https://github.com/michaelklishin/quartzite Goose can be Reddis backed.

danielsz 2025-12-09T16:04:25.145229Z

I use carmine + redis. https://github.com/danielsz/system/blob/master/src/system/components/redis_pubsub.clj is the pub sub component for system.