Fork me on GitHub
#missionary
<
2023-09-05
>
martinklepsch11:09:52

Hello ๐Ÿ™‚ I'm trying to use missionary to launch a few dozen independent processes. Those processes should do something on startup and then also execute certain tasks in specified intervals. Ideally I'd also be able to introduce some offset between launching those processes. After a fixed interval of time I'd like to shut down all processes. I think missionary is the right tool for this but I'm not entirely sure how to structure it...

xificurC11:09:59

could you describe an MVP that would form the basis to improve towards the final solution? Or extend the problem description, right now the definition is too vague to propose solutions. E.g. what do the tasks do on startup (computation, IO), how are the specified intervals specified (is it an external input, is it known AOT)

martinklepsch11:09:32

Hey, and sorry about the vague-ness ๐Ÿ™‚ The startup is IO (network/promise) and the intervals are defined AOT.

martinklepsch12:09:38

I'm trying to generate load on a UI system for performance testing. This effectively means simulating users in a chat via database writes. The users activity can be described as follows: 1. joining the chat = "startup" 2. sending messages while active 3. sending presence pings while active

martinklepsch12:09:52

So each user is basically a task that continuously executes some simple IO. I was thinking I need to use a flow for the "executing stuff in intervals" but I'm wondering if maybe this also could just be cancellable tasks.

xificurC12:09:26

you can run this from the comment block. It concurrently spawns 10 tasks that do some work (you can see the prints in stdout) and shut down after provided ms

xificurC12:09:22

> I was thinking I need to use a flow for the "executing stuff in intervals" but I'm wondering if maybe this also could just be cancellable tasks indeed tasks can work as well, in the above prototype loop/recur with an m/sleep inside acts as a long-running task

martinklepsch12:09:59

yeah the loop/recur is nice actually. Thanks for this I think I'm understanding most of it even ๐Ÿ™Œ

๐Ÿ™‚ 2
martinklepsch12:09:07

Let me fiddle around a bit more ๐Ÿ™‚

๐Ÿ‘ 2
martinklepsch12:09:08

Thanks, I managed to get everything working to my liking! โค๏ธ

๐ŸŽ‰ 4
martinklepsch12:09:23

Very cool that you can just loop recur in an async context like this

clojure-spin 2
martinklepsch12:09:37

Ah โ€” one thing I just noticed I dropped is the m/via m/blk but I think since I'm in cljs-land it probably isn't relevant? (there's no impl for it in missionary.cljc)

xificurC13:09:08

glad to help! m/via is for offloading cpu/io-intensive tasks on a JVM thread, so yes, irrelevant in cljs context

martinklepsch13:09:21

erm... as I now incorporated the promises into startup-task things stopped working and now I'm like "how do I convert a promise to a task" thinking-face but can't find anything in the docs

xificurC13:09:51

are the promises third party or can you change the implementation to return tasks?

martinklepsch13:09:33

ultimately they're from a third party library. I would try to convert my functions to tasks, but in the end I'll need to interop with promises.

telekid13:09:28

If you haven't already stumbled across https://github.com/leonoel/task, I'd give it a read โ€“ it helped me intuit things like promise interop a bit better

xificurC13:09:45

if you want to fully understand the await-promise implementation I opened a https://github.com/leonoel/missionary/discussions/61 a while back where Dustin links to his https://gist.github.com/dustingetz/d3fb5338b43ef86df5f97387f1d5e249#file-await_promise-clj

๐Ÿ™ 2
telekid13:09:22

oh this is so great, thank you

martinklepsch13:09:44

I remember reading the task spec a while ago actually. And the reasoning for await-promise also makes sense. At the same time it would be nice if it was a bit more out-of-the-box supported to work with promises. In JS it's just the bread and butter of everything

xificurC13:09:52

I think Leo tries to keep missionary small and free of helpers. In any case, today you can copy the impl from the wiki page into your codebase ๐Ÿ˜‰

Dustin Getz13:09:10

electric repo has a contrib.missionary-contrib ns where i collect the helpers, send them to me and i will add them. will break it out at some point. i try to review all contrib helpers with Leo first

martinklepsch13:09:18

I just ran into a slightly surprising error handling behaviour that I'd like to understand... ๐Ÿงต

martinklepsch13:09:52

With this kind of code I could see {:sending uid} being tapped and then :sent would never show up. The overall thing (wrapping this with m/race ) would exit successfully. Upon further investigation it turned out that there was a synchronous error in post-chat-msg! that was not surfaced. await-promise was never called, instead the .add line caused the exception.

martinklepsch13:09:26

Interesting... after a REPL restart I can't repro this...

leonoel13:09:29

race swallows errors if at least one other input succeeds

martinklepsch13:09:36

Even if the first completed input fails? Would this mean combining m/race with m/sleep would always succeed?

leonoel13:09:12

yes. as long as there is still an input running, the race is not over (because there is still a chance to succeed)

martinklepsch13:09:39

Gotcha, what would be the recommended way to fail fast if something fails?

leonoel13:09:19

timeout may be exactly what you need

๐Ÿ‘Œ 2
leonoel13:09:58

the implementation of timeout uses attempt and absolve for this exact reason