This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-05-16
Channels
- # announcements (22)
- # beginners (4)
- # biff (4)
- # cider (5)
- # clerk (3)
- # clojure (28)
- # clojure-chennai (1)
- # clojure-europe (23)
- # clojure-gamedev (7)
- # clojure-korea (5)
- # clojure-madison (3)
- # clojure-my (1)
- # clojure-nl (1)
- # clojure-norway (49)
- # clojure-sweden (7)
- # clojure-uk (4)
- # clojuredesign-podcast (14)
- # clojurescript (10)
- # clr (5)
- # cursive (4)
- # datascript (17)
- # datomic (2)
- # events (1)
- # garden (1)
- # introduce-yourself (2)
- # jobs-discuss (14)
- # lsp (23)
- # malli (14)
- # missionary (9)
- # off-topic (109)
- # overtone (7)
- # polylith (5)
- # releases (5)
- # shadow-cljs (7)
- # sql (13)
- # testing (30)
- # xtdb (10)
- # yamlscript (44)
Has anyone tried to build an async task runner / workflow system with dependent tasks using Missionary? Like Airflow or Celery.
I'll need (to buils) one soon
I've been thinking about something similar from another angle, but this also relates to distribution of work and state persistence. If we can distribute work between client and server, how can we distribute work among servers in a similar manner? I want (via remote ...)
what I've done in the past (without missionary) is have the servers pick up tasks when they have capacity. For that I have a job table in postgres, and use a skip lock in a transaction to process the task:
(postgres/execute-one! postgres
{:delete-from :scheduled-tasks
:where [:=
:id
{:select :id
:from [:scheduled-tasks]
:order-by [:execute-at]
:where [:<= :execute-at now]
:for [:update :skip-locked]
:limit 1}]
:returning [:*]})
if the server fails to execute the task (rolling back or timing out the transaction) then the task is picked by anotherI would just use https://github.com/manetu/temporal-clojure-sdk
I started working on something like that. I didn't quite get it to a usable state for lack of time or focus, but you can look at it if you are interested https://github.com/AdamFrey/missionary-pipelines
@U06060CK0 seems like a private repo