✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅=> {ee 420073}

You will need RCF master branch to run these (not the published maven coords)

Why p/for on 50 and not jut core for ?

Do you have another layer on top of photon to mange dom updates ?

p/for is concurrent and does diffing to stabilize the body over time. There is an object lifecycle (mount/unmount). See the final test in photon-1-hello-world, line 127

Yes, we have a photon-dom library not shown yet in these tutorials

Ok, now that I know a bit more thx to @U09K620SG’s excellent onboarding examples, I have created a sample issue:

(tests "duplicate watches via fn's"
  (def !x (atom 0))

  (p/defn comp-y []
    (! (+ 40 (p/watch !x))))
  
  (p/defn comp-z []
    (* 2 (p/watch !x)))
  
  (def dispose
    (p/run
      (let [z (comp-z.)
            y (comp-y.)]
        (! (+ y z)))))

  % := 40

  (swap! !x inc)

  % := 42 
  % := 44

  (dispose))

p/watch is a macro to hide something from tutorial usage, look at the impl

Pull latest to see this new tutorial

Oh, I thought watch was how we read a value and established the reactive dependency, such that changes would flow thru automatically. Not so?

Only diff I see in the new example is using new instead of dot notation. Did I miss sth else?

The point of the example is if you only want one subscription to the atom, call new once

m/watch does not actually perform side effects, it returns a pure functional "IO value" that describes a side effect

better explanation - m/watch returns a flow, but it does not run the flow. You have to instantiate it

p/watch is different then m/watch btw - did you see that they are different? (p/watch !x) macroexpands to (new (m/watch !x)), bundling the instantiation (new) with the flow definition (m/watch)

p/watch maybe should have a ! as in p/watch!

So nice to have Kenny and Dustin in the same room. 😄 I was reading you quite intently about Cells in e.g. 2006, Kenny, and a few years ago when Matrix surfaced too. What was your conclusion about this discussion, did everything check up as being glitch-free?

No, @U0514DPR7, if a derivation D pulls from the same source data point S via more than one path, a single change to S can lead to two or more computations of D, possibly with earlier computations then being transiently inconsistent with the change to S. Aka, a "glitch". These inconsistencies are rectified by the time propagation of the original change is complete. These glitches are accepted by the reactive streams crowd as far as I can make out. They propose avoiding glitches by not coding multiple possible dependency paths back to a given source. ie, Glitches are "pilot error". Cells had the same glitch until 3.0, and I was able to write a lot of quality code despite the theoretical flaw. Luck ran out with an extreme case of duplicate dependency paths, where all derivations led back to a single source. RoboCup simulation, in fact.

kenny we would appreciate if you would speak with less certainty wrt missionary’s design. duplicate watch is not the same thing as a glitch. missionary, in addition to being a glitch-free streaming library, is also a pure functional programming library with referentially transparent primitives (like Haskell). registering callbacks on an atom with watch is a side effect and pure FP requires us to treat that with a certain reverence and that is the essence of why missionary works differently here than you want it to. the behavior you observed is not caused by the propagation engine but rather the clojure IWatchable interop.

we could change the behavior trivially if we wanted to but we feel it is a violation of FP and unvigorous

I will be honest, @U09K620SG, I did not know what to make of your response to my demonstration of a glitch in the code above, so I let it go. Also, you had already described one glitchy example as a user error, so I thought you like Stalz were OK with the double updates (with one logically inconsistent update). But here we are. Was there something wrong with my example? It does update twice, and at one point comes up with a value logically inconsistent with the application state. If that is nominal behavior, fine, our definitions differ.

> Was there something wrong with my example? yes

> If that is nominal behavior, fine, our definitions differ. the busted program you provided is trivially fixed (subscribe to the atom once, not twice) and the issue has nothing to do with the propogation engine; it is a clojure interop issue

I have provided passing tests documenting all behaviors in the repo with many comments

> I thought you like Stalz were OK with the double updates (with one logically inconsistent update) I want to be crystal clear to all in the future who will stumble upon this thread 1. we are NOT okay with double updates 2. the updates are NOT logically inconsistent, missionary has done exactly the correct thing because the program explicitly subscribes to the foreign clojure atom twice, and missionary correctly propagates immediately after the first foreign callback firing from the atom, and again after the second foreign callback firing from the atom. the programmer did not express the intended DAG, and we have great deeply thought out reasons for it being this way which are essential properties for client/server distribution

"the busted program you provided is trivially fixed (subscribe to the atom once, not twice) " I tried to address this point ^^^ by pointing out that the dup subscribe to S could arise, not thru deliberately subscribing twice, but because the developer in a real app trying to derive D may well end up calling an interesting function IFN that happens also to refer to S along the way. But D still needs the raw value of S, which IFN obscures producing its own interesting result. Cells 2.0 was broken by RoboCup's big strategic module tripping up the big skill execution module. Staltz suggested copying all of IFN's logic into the derivation of D, or at least refactoring so dup subs do not arise. So I would have to roll the strategic and execution code into one blob, because of the reactive engine I chose? And would it always be trivial? I am not saying FB engineers are any good, but there are a lot of them and Redux got invented because they simply could not figure out how to display accurately an unread message count: https://www.youtube.com/watch?v=nYkdrAPrdcw&t=783s As you listen to the speaker's breakdown of the challenges they faced with that simple message count, one can almost hear the glitches in their manual orchestration attempts. They would fix it and refix it constantly. That is precisely why Redux got invented, because an ad hoc solution eluded them. So telling users to just reorder their logic may not scale. I was asked here what I had concluded about Missionary being susceptible to glitches, so I answered that it is, where a glitch is defined as a reactive value transiently taking on an inconsistent state visible to application code. We can call that a feature of Missionary instead of a glitch since it has been documented, but the window of risk is the same: complex applications will unavoidably lead to duplicate subscriptions and transient incorrect program state during state change propagation.

Kenny i'm sorry you're extremely confused, the issue is at the clojure IWatchable interop layer, not the propogation engine

If you are interested in understanding, the documentation is here: https://github.com/hyperfiddle/photon/blob/567860954be3db440ff663309ad2f54f301bdff5/src-docs/user/photon_missionary_interop.clj#L20-L41

"Kenny i'm sorry you're extremely confused,..." It is the long COVID, I swear... "..., the issue is at the clojure IWatchable interop layer, not the propogation engine" So when my app screws up a $50b trade, my users sue...IWatchable? To a reactive engineer, things like IWatchable are our problem to work around, and no excuse for the behavior of the code I produce. If we want to write serious reactive code, we have to own the responsibility for it. If perchance IWatchable was a great solution, sure, we use it. But if using it leads to glitches.... The response I have heard consistently is "hey, that is not our behavior, that is the behavior of this random tool we pulled off the shelf." How does that help my users? I chose the tool, I have to make it work correctly.

Using IWatchable does not lead to glitches. Subscribe to each atom once and you will have the intended program. This is elementary stuff

"Subscribe to each atom once" Could HF/Missionary automagically detect multiple subscriptions? Cells does not support cyclical dependency, ut they can be coded. If they are, the runtime cycle is detected by Cells. This does happen to me every six months or so, and indeed I am able to find some sloppy thinking and refactor the way it should have been in the first place. No more cycle. The question I am raising to no avail is whether only elementary HF applications, and trivial test cases, can stay within the subscribe-once constraint. In my twenty-five years experience of deeply reactive systems, my prediction is "no", but quite a bit of HF code will be OK. Some of it will require a simple Staltz-ian refactoring. The FB unread message count? Could force users to roll their own anti-multi-subscribe safety net. Hopefully not Redux. :)

> Some of it will require a simple Staltz-ian refactoring. It will not, unless someone finds an actual bug in the propogation engine, in which case we will fix the bug

> Could HF/Missionary automagically detect multiple subscriptions? We could yes, we do not choose to do this today. Our mission is to make it possible to express correct programs. It is not part of our mission to automatically turn buggy programs into correct programs. Fix your bug!

> In my twenty-five years experience of deeply reactive systems, my prediction is "no", but quite a bit of HF code will be OK. Some of it will require a simple Staltz-ian refactoring. The burden of proof for continuing to make comments like this is to produce a test case. You did produce a pentagram of death test case (thank you), which we aced.

Since I'm the one who asked if this discussion had reached its fixpoint, I feel a need to come back to it a bit. 1. I very much want to thank both of you, Kenny and Dustin, for trying out as best as you could to answer my question in this space and stretch a bit more to make your point understood! 2. Then I want to pause and recognize how hard it can sometimes be on each viewpoint holder, when we feel we need to make sure our angle is understood and well considered. Especially when we know these things will be read again and again through the coming years. 3. Finally, reading you, I felt we share common sentiments on this: we all strive for the best. That's very clear to me. And therefore I wish that having discussed this now might lead to the overall best. And I'm already glad that Photon's and/or Missionnary's docs have been or will be augmented thanks to this exchange.

Ha-ha, I was wondering where you had gone to, @U0514DPR7! What did you learn from the exchange? Not the kumbaye my lord feel-good crap, I mean about how Missionary works. Think you can pass a review quiz?

@U0PUGPSFR, the answer reduces to "no, I'd have to invest in learning the ins and outs of the thing". Here's a NLE;GTR (complement of TL;DR): • After CL, I went to Haskell and Qi-then-Shen (both to stretch my mind) and Clojure (hoping I could one day apply it at work, which is now the case). And also to learn English better. And obviously the imperatives of my day jobs through the years. • Although I keep a close watch on Hyperfiddle (HF), I haven't yet started doing the same on Missionary (M). I'm still at the stage of feeling M is unreadable, for not having invested in learning it. I know it's expected. I'm curious if I'll ever justify learning its usage. I'm more in pull-based learning mode than in push-based since I started being in the business of having 5 children. • Although I've kept a watch on React and its various clojure wrappers, I haven't invested in the fundamentals of FRP and reactive programming. I thus don't feel qualified to judge M. • What I keep from your exchange for now is that the current implementation will always converge to a coherent fixpoint but if our code calls code that instantiates more than one watch on the same atom and some 'parallel' intermediaries fire side effects, then we might sometimes observe incoherent side effects fired. Thus M (nor HF) doesn't protect us from that and we should pay attention to make sure no more than one watch per atom is instantiated. • I don't have a practical intuition to determine in advance if it's going to be hard or easy to guard the number of watches per atom. Your experience seems to suggest this can easily get off of hand. And even though we can build quite complex systems even with this caveat, you strongly suggest it's important to add this check box to the fundamentals now. I don't know if it's practical or possible to add that to M, or possible to add that to HF as a kind of higher-level supervision. If I understand well, there's a purity constraint here, a bit like adding Cut to a Prolog makes it impure, and Dustin feels M should be kept pure (which feels like a noble argument, obviously, although one might judge that to be impractical in a grander scheme of things - I'm, again, not yet inclined to judge on this). • I'm still wondering what parts of your vast and hard-won wisdom with Reactive Programming (and FRP, or not?) might bring benefits to either or both M and HF. For example if your glitch claims are really a problem of M's current implementation or not, or of HF on top of M only, or even of M or HF on top of clojure's current watch implementation (which I believe we shouldn't expect to change, unless...). • I see the ravages of such glitches in some redux-based react apps, and also I've seen the chat bug in recent years on Fb. (BTW Facebook created Flux, not Redux, although they now employ Redux's creator.) • I also wondered if by "kumbaye etc." you were targeting my response written yesterday, hoping to prevent me from (ever) writing another one like that, and/or triggering me to really engage in the argument for its worthy sake. ("ok Dan, you wrote enough human-political sweetening stuff, let's get back to the argument at hand because we can still learn and win more if we continue it".) If that's the case, I hope this answer is more favorable to you than my previous one.

"BTW Facebook created Flux, not Redux...." Correct, though Redux was created to fix Flux and did not change the fundamental design idea of exiling state to a secondary store. But good catch. The rest of your summary shows excellent appreciation of the thread.

To complement dustin's answer, let me add some technical context about what could be the source of confusion here. • Rx streams are discrete time only. Operators with continuous time semantics like combineLatest are necessarily glitchy, we all agree glitches are bad and stalz-refactoring is not an acceptable solution, no argument here. The problem is essential to the GoF observer pattern, clojure watch mechanism has the exact same problem. • Matrix cells are a continuous time abstraction. The engine is able to maintain a DAG of cells with dynamic topology, and propagate a change through it without glitches. Discrete time events are not managed, so all effects interacting with the engine must be written imperative style, with callbacks and direct mutation of reference objects. • Missionary aims to support both discrete and continuous time, to combine the benefits of managed effects AND glitch-free propagation. The flow abstraction supports both semantics and you can switch from one to the other with functional composition. This approach scales extremely well, you can go a very long way without having to ever mutate a single reference. Photon inherits this property. @U0PUGPSFR The purpose of watch is purely interop. It is provided as a convenience for setting up a simple communication channel, to write unit tests or experiment at the REPL. Atoms and watches are sparingly used in a real application. There are some very specific use cases that require it, like representing cycles, and doing so requires special care anyway because then you need to ensure the propagation loop eventually converges. Here the atom is local state, it's watched immediately and you definitely don't want to expose it. In practice, the problem you're describing is really a non-issue if you understand what you're doing. We may reconsider this choice in the future if newcomers happen to struggle with it but as Dustin said this is a cosmetic change.

"you can go a very long way without having to ever mutate a single reference" How does anything ever happen? How does a chess app player move P-K4? The question at hand is state management--if we are not mutating references, what is there to manage? Confused. Anyway, all I had to work from were simple examples from my HF onboarding and Missionary tutorials. Those let me play a little and set up that glitchy example. Now I learn that I was not really using Missionary, I was doing sth else. OK. So fine. I will guess that I misunderstood the idea that Missionary apps never change, and that they do process inputs. Is there a working example of realistic Missionary code that shows a new input propagating thru a DAG and obeying these constraints: • all derived values are updated; (duh :)) • no derived values are updated more than once; (we need some way to count/observe updates); • corollary: derivations never see obsolete values, defined as values that need to be recomputed because of the same change being processed. Mind you, that ^^^ presumes a model David Gelernter christened a "trellis", one in which everything moves forward in discrete steps as determined by some arbitrary "generation" ID. Perhaps that does not apply to Missionary?

If you're interested, pull latest – there are more and better examples now. I think you were the second person I onboarded (due to your level of expertise), so what you saw was super early. I demo with real web app examples now

https://github.com/hyperfiddle/photon/tree/master/src-docs/user https://github.com/hyperfiddle/photon/blob/be908870b5cf43d8fd7f003488e632a4d091994c/src-docs/user/demo_webview.cljc#L46 single watch on database

btw, The RoboCup example involved an app that had to play soccer by • receiving every 100ms a complete visual inventory of landmarks, other players, and the ball as a single JSON blob; and • responding with actions such as turn, run, and kick. So all dependency chains led back to the single visual inventory input fed from a handler on a UDP socket.

Thx, Dustin! I will check those out. 🙏

This dom/input example doesn't even have a userland watch, the dom event subscription is abstracted by the input component https://github.com/hyperfiddle/photon/blob/be908870b5cf43d8fd7f003488e632a4d091994c/src-docs/user/demo_system_properties.cljc#L17 (there is a watch-like construct inside, m/observe which is how you implement m/watch)

I would love to try robocup when we get time, maybe an opportunity to collaborate

btw, RoboCells was in Common Lisp. And the league itself seems retired as of 2017. Lemme look on SourceForge...

That was easy. https://sourceforge.net/projects/robocells/ Caveat: I remember doing one such archaeological dig and coming up with an old version, so this may not be the version that finally triggered a breaking glitch.

RoboCup for real bots seems quite active, but I do not see a simulation league still active: https://www.robocup.org/