Clojurians Log v2
missionary
I have a few questions about how missionary works. Looking at the implementation I like the idea of the notify followed by deref but, i'm confused as to how one avoids race conditions with this approach. Let's say we have a simple iterator, lets call it Map that is taking inputs from A, transforming them via f then providing them to B. From what I understand the controlflow is:
A calls notify on Map
Map derefs A, transforms the value, and notifies B
B derefs Map with the value. All is good
But what happens if while Map is transforming the value, A notifies Map that there's yet another new value. What if while we're transforming the value? It seems like Map has to somehow track that there's a new value available, and hold onto that information until B derefs Map then I assume as part of that deref Map would notify again.
But it also seems like there is a race condition between B derefing the value and Map being notified that there's an update. Sure the moment Map gets the deref called there may not be a new value available, but right after it looks A may come in and notify.
Have you seen https://github.com/leonoel/flow and https://github.com/leonoel/task?
Yes, these race conditions are expected. If a node has many inputs, the inputs can also race with each other in addition of racing with output. How they're resolved depends on the operator.
Note - missionary generally tries to minimize buffering so a simple Map operator would rather work like this :
1. A calls notify on Map
2. Map calls notify on B
3. B derefs Map
4. Map derefs A
5. Map transforms value returned by A and returns it to B
That's how e.g. m/latest works, try it with a couple of m/watch as inputs
I want to clarify the discussions for the readers. Race conditions are described on wikipedia as > [...] leading to unexpected or inconsistent results Missionary operators don't exhibit unexpected or inconsistent results (other than open bug tickets). Missionary operators use concurrency primitives to ensure consistency. @tbaldridge your insight is correct though, missionary operators have to take care of these conditions.
Sorry if this is all convoluted, but it feels like every input/output type node has a racecondition, and I'm not sure how that's normally solved