I have a question about shutdown of m/sp. I have a situation where I need to do some cleanup inside the task that is being cancelled. So I catch the cancelled exception inside the m/SP block. My question is if I am allowed to use m/? after the task has been cancelled? Thanks!
Yes. If you call m/? after cancellation, the task process will be spawned and cancelled immediately. Use m/compel if you want to force it to run to completion.
I have written a logger that writes every 500ms batches to a file. https://github.com/clojure-quant/quanta-missionary/blob/main/src/quanta/missionary/logger.clj that works fine, and an example is here: https://github.com/clojure-quant/quanta-missionary/blob/main/demo/src/demo/log_perf.clj Where I have an issue is in the shutdown. In shutdown I use a rdv to make sure the already accumulated data gets flushed to disk. And then since I watch an atom, I use an eduction to stop my imput flow, so the missionary ap task stops. What I had to do to make it work is add a (Thread/sleep 10). I think what I really need is a function that waits until all missionary threads are finished. any ideas?
The real issue seems to be that I am not receiving m/sp success callbacks in the shutdown of my app. It seems missionary cancels it's thread factory at some point.
I think I found out my issue. When I instantiate a task with (task success error), I get the dispose function. As soon as the dispose funciton goes out of scope (and possibly is garbage collected), the underlying task dies wihtout going either to success or error path. Is this intended behavior?
I doubt ignoring the dispose function could have any effect on the task process. Help me reproduce this conclusion
thanks @leonoel This is part of my shutdown process. So the app already has receivec CTRL+C. Perhaps that explains it? In any case I will try to make this reproduceable.
continue-a (atom true)
log-f (m/eduction
(take-while (fn [_] @continue-a))
log-f)
This pattern is dubious - log-f may not terminate after continue-a becomes false, because it will not check until the next value is available on input.
Prefer this :
eof (m/dfv)
log-f (m/eduction
(take-while (complement #{::end}))
(m/ap (m/amb= (m/?> log-f) (m/? eof))))
;; in the dispose function
(eof ::end)
This change makes flow-logging-task correctly terminate on disposeThanks @leonoel