Clojurians Log v2

Ah yes, "what (we seek)" is indeed what I had in mind.

Maybe you already demo-ed that in your write-up, let me take another look...

Aha! > fm-ancestor parent chain search > Usage: : (fm-ancestor _what_ _where_) Example: (fget (fm-ancestor :my-radio-group me) :current-choice) Parameters: what : a test function indicating if a model is the one sought; where : the starting model of ancestor search; options : me? : [false] should the starting where model be tested for a match? wocd? : [true] should dependencies /not/ be formed when cell properties are read? Short for "without-cell-dependency". must? : [true] should an exception be thrown if search fails? ie, "Must the navigation find the target what? > In brief, search recursively up testing parents for a match.

I think I got distracted in the middle of that and never got to fm-navig, the one for which I got the Apple Pen so I could draw depth-first, left-right. 🙂 I can offer a quick tl;dr on that if needed--I will be heads down coding for a couple of days, otherwise.

tl;dr: A full search • starts with me, then • kids recursively, depth first, left to right, then • parent recursively, skipping me. Note: if we say :inside? false, we do not search the kids of the starting 'wher', but when we head up to the parent the resursive call always says :inside? true. ie, :inside false is just for the original "where".

It flies indeed! 🚀 Thanks! Ah okay haha, yeah I was a bit confused by that. Thought maybe it meant because it's not guaranteed to succeed and run or something.

Hmm, that :async? true combined with await has my attention. Lemme check sth....

[By the time I got to the end of this I decided a good quick experiment would be to simply remove the :async? true. Then what happens? -kt] Below is the handling of :async? true. Tl;dr: it expects a Future as the computed value. Once `connected? is true, we return --- well, is it a vector of services or a Future? Perhaps we could capture the return value in a local var, print it, then return it. Looking at the MX handling it should throw an exception if it does not get a future (or nil). await is tricky. If it is working as we would like, then your formula will not return a Future. Sadly, await never works the way I expect. :)

(do
                       (assert (or (nil? raw-value)         ;; someday support other default future cell values, mebbe :pending
                                 (dart/is? raw-value Future))
                         (str "cnset-future got non future: " raw-value dbgid dbgdata))

                       (if (dart/is? raw-value Future)
                         (do
                           ;; (dp :got-future :defchg cty/*defer-changes* :wii cty/*within-integrity*)
                           (.then ^Future raw-value
                             (fn [fu-val]
                               ;(dp :then-callback-sees :defchg cty/*defer-changes* :wii cty/*within-integrity*)
                               (with-integrity [:change :future-then]
                                 ;; todo if a cfu is meant to run repeatedly as dependencies change,
                                 ;;      do we need to clear :then? Or is opti-away not a problem
                                 ;;      since it would have happened were there no users??
                                 (rmap-meta-setf [:then? c] true)
                                 (c-value-assume c (if-let [and-then (:and-then? @c)]
                                                     (and-then c fu-val) fu-val) nil))))
                           ;; forcing nil pending future
                           ;; TODO support :pending-future-placeholder-value and force that instead
                           (c-value-assume c nil propagation-code))
                         (c-value-assume c nil propagation-code)))

That last sequence improved and as code (but check it! I just banged it in):

:services-lookup (cF+ [:watch (fn [_ _ lookup-future _ _]
                                    (when lookup-future
                                          (.then lookup-future
                                                (fn [results]
                                                      (with-cc :svs-lookup
                                                           (mset! me :services 
                                                               (mapv (fn [service]
                                                                              (md/make
                                                                                    :service service
                                                                                    :characteristics (.-characteristics service)))
                                                                  results)))))]
                         (when (mget me :connected?)
                             (.discoverServices device))
   :services (cI nil)

Hmm, howseabout I extend things so we have instead of :async? and :async parameter, which can be true for the current behavior or a map with two optional keys:

:until _pending_value_, default nil
    :then _result_handler_, defaulit `identity

Wow, I must be overloaded, forgot I supported this (from the TodoMVC demo):

(cF+ [:async? true
                         ; optional chance to filter async response before it
                         ; gets set as the mx slot value
                         :and-then? (fn [c lookup]
                                      (= 200 (.-statusCode ^dht/Response lookup)))]
                     (dht/get (.https Uri "" "drug/event.json"
                                {"limit"  "1"
                                 "search" (str "patient.drug.openfda.brand_name:"
                                            (mget me :title))})))

Untested (but the Todo version works):

:services  (cF+ [:async? true
                 :and-then? (fn [c services]
                             (mapv (fn extract-services
                                     [^ble/BluetoothService service]
                                     (dc/print [:got-svc (.-characteristics service)])
                                     (md/make
                                       :service service
                                       :characteristics (.-characteristics service)))
                               services))
                 :watch obs-slot-new]
             (when (mget me :connected?)
               (.discoverServices device)))

Btw, for anyone wondering why non-boolean :and-then? ends with "?" ... https://www.google.com/search?q=dude+wheres+my+car+and+then+scene&oq=dude+wheres+my+car+and+then+scene&aqs=chrome..69i57j0i10i512l2j0i390l4.10081j0j7&sourceid=chrome&ie=UTF-8#fpstate=ive&vld=cid:b28ec2d0,vid:oqwzuiSy9y0

Hrm, so this is interesting. (.discoverServices device) gets called when expected on connection. :and-then? ... runs not when the future resolves, but after :connected? is set back to false.

Didn't notice at first because it disconnected quickly enough that it happened in close proximity in original testing. @hiskennyness

Trying manual version without and-then? above...

Oh wait, maybe the future isn't finishing when I expect...

Oh, wait, maybe (.discoverServices device) is a future that returns yet another future??

Nope.

Hmm, this is not what I expected. :got-services!!!!!!!!! prints right when expected. Future does indeed return. :running-the-with-cc!!!!! on the other-hand, does not run until connection is closed. I suspect now would be a very good time to upgrade the f/mx sha to the latest and see if it is mx isolation related.

Aha, need with-mx-isolation around with-cc in the and-then? and now it works!

(Or at least, that's how I got it working in my and then substitute above.)

@hiskennyness wrapping this here with with-mx-isolation ought to do the trick: https://github.com/kennytilton/flutter-mx/blob/9a23237807af77ba09d397114b7bc2aec7655ae7/src/tiltontec/cell/evaluate.cljd#L205 only I don't know if that might break something else...

Should be okay right? Cause .then is a top-level event?

Good point. Did you run into something, or are you just thinking ahead?

I will give it a try....

Yep, this problem here was caused by it: https://clojurians.slack.com/archives/CKCBP3QF9/p1671574944766179?thread_ts=1671230925.880929&cid=CKCBP3QF9

Uh-oh. I worried a bit about that, but mebbe not enough. Do you have a larger chunk of code to remind me of the context, or has this been pushed to "huh"?

Has been pushed, yes. What is currently there works by side-stepping and-then?

brb...

Hmm. When connected? goes to false the services formula should return nil and no Future should get dispatched, and any and-then should not run. How does connected? get set to false?

Correct, but the future already had been dispatched from when it was first connected.

I find myself wondering if a Future can fire a second time kind of on its own. Do you see this fire after connected? goes to false?

(dc/print :!connected--discovering-services)

Nope.

I think it was simply an interference due to the lack of with-mx-isolation around the with-integrety

Let me push the new version with a with-mx-iso around that. Tests out OK.

IE. the reason I did not see the results of and-then was because

(with-integrity [:change :future-then]
                                 ;; todo if a cfu is meant to run repeatedly as dependencies change,
                                 ;;      do we need to clear :then? Or is opti-away not a problem
                                 ;;      since it would have happened were there no users??
                                 (rmap-meta-setf [:then? c] true)
                                 (c-value-assume c (if-let [and-then (:and-then? @c)]
                                                     (and-then c fu-val) fu-val) nil))))

SHA 28420793ebf4d2639da3c8ccd622e379c8729039

I am a bit groggy, but I think your analysis is spot on. That and-then handling is just mset! in-lined, and we have learned we need with-mx-isolation in CLJD+Flutter around seemingly top-level mset!s

👍🏻 New sha works beautifully :)

Looking forward to an example of matrix navigation with a qualification function now that I'm done (for now 🤞🏻) distracting you with other things! :)

Hey, thanks for pushing f/mx and diagnosing the breakage! The idea of dynamic vars being visible in callbacks from Dart/Flutter has me scratching my head as to whether its a bug or a feature. 🙂 A dynamic binding is sth we find in the operative call stack. Our callbacks from Dart/Flutter are seeing bindings extant when the first-class function callback is created, not in the call stack. In Common Lisp, If we want to capture dynamics, we have to convert them to locals and close over them:

(let [my-dyno *some-dyno*] ...return first-class function that references my-dyno...)

"an example of matrix navigation with a qualification function" Qualification function?

If by qualification function you mean that the "what (we seek)" parameter can be a function, that is just a general principle of making a low-level utility as flexible as possible. In CL, all the list comprehension utilities take optional parameters :key :and :test, and :from-end as well. The defaults there being, in CLJ-ese, identity, =, and false. I leave the options to macrology that supplies a test, except no from-end is possbible.

Looks like I should start from pbvm-admin.blue? @zenflowapp

I am crashing back into bed. I saw this nifty stuff:

(await (.delayed dc/Future (dc/Duration .milliseconds 2000)))

(.listen (.-state device)
              (fn [state]
                (mset! device-mx :status state)))

Hmm, I should try running the thing for real. Still shooting for noon.

Hey, she came up fine, just not finding the connections. More in a few hours, got tennis and brunch next.

A bit puzzled. Before I get into anything, on the "huh" branch pbvm-admin.core starts out with

(ns pbvm-admin.core
  (:require
   ["dart:core" :as dc]
   [pbvm-admin.blue :as bt]
   ["ble.dart" :as bb]
   ["package:flutter/foundation.dart" :as foundation]
   ["package:flutter_blue_plus/flutter_blue_plus.dart" :as
    ble]
   ["dart:async" :as async]
...etc...

@zenflowapp ^^^

@hiskennyness Oh, yep, ble.dart was a prebuilt example, guess I never checked that into git, but not needed or relevant currently. > Looks like you know your way around Dart futures. Not really, that tidbit was thanks to the DartClojure translator ;). That being said, I do think that would work for stubbing out the ble library. > Next, the app does not find any bluetooth devices, but then I also see that scan! and repl-ish do not run. Oops, sorry, yeah that is more dormant code. Right now I'm using the little plus button to trigger the scan.

If you wanted to have it scan on every reload, you could add a (widget m/Text "") somewhere in the active tree. That would cause repl-ish to run on every hot reload.

I'm game for pairing at some point. Although I can't say I've ever done that before, so I'm not sure what tech is ideal for that. I do have rustdesk installed.

For bluetooth to show up, you need to manually turn on bluetooth and location services. Although I have not tried on Apple, more config might be needed there.

I think we can use google hangouts or sth. I am not much of a pairer either. Tech interviews with coding really bug me. 🙂 Lemme see how hangouts works.

Ah, they want us to use google chat now. I need your email to send you an invite.

Sure, it's <mailto:ben@devcarbon.com|ben@devcarbon.com>

Ah, this should do it: http://meet.google.com/hye-auqr-gyf

Looks like you connected.

Pushed changes to huh . Ought to have renamed that to aha :P

Hey, check out as-dart-callback:

(defmacro as-dart-callback [[& cb-params] & body]
  `(tiltontec.flutter-mx.core/with-ctx+as-is [~'me ~'ctx]
     (fn [~@cb-params]
       (tiltontec.cell.base/with-mx-isolation
         ~@body))))

Ah, cool! I remember now reading that and wondering what it did. :P For some reason in my mind I was thinking it was only relevant for widgets. Probably thought it only mattered when context was needed.

Hmm, maybe there is a way to detect when a change is being triggered from a vanilla fn as a callback and warn about it?

🤔

This look kosher to you? I see the :!!!-Do-I-even-run????!!!! and the service print out, but the watch never runs.

Oh forget the :emphemeral? true , I added that but haven't tested it yet.

Problem I'm running into is that even though I have verified that the child device matrix has properly had its :connected? property set to true, :connections is still empty (on the parent matrix).

Perhaps this is not kosher?

:connections (cF (let [devices (mget me :device-candidates)]
                      (filter (fn [device]
                                (mget device :connected?))
                              devices)))

Hang on, checking sth simple...

I see:

(with-cc (mset! me :scan-candidates candidates))

Oh, that might solve two-for-one!

So (with-cc :set-candis (mset! ....etc ))

That snippet looks fine:

:connections (cF (let [devices (mget me :device-candidates)]
                      (filter (fn [device]
                                (mget device :connected?))
                              devices)))

Okay, so just tried again with the debug keyword passed to with-cc, and no change as far as :connections goes. (havn't tried replacing my temporary state atom yet)

Oh, let me look back at what you said about those.

Printing the first :device-candidates gives this:

{:host nil, :parent Instance of 'Atom', :status BluetoothDeviceState.connected, :device-id 48:23:35:0C:1E:87, :cells-flushed ([:device-name 25]), :manufacturer-data {43981: [0, 0, 1, 12]}, :connected? true, :device BluetoothDevice{id: 48:23:35:0C:1E:87, name: PBVM, type: BluetoothDeviceType.le, isDiscoveringServices: false, _services: [], :device-name "PBVM"}

Ah, still reading, but we might need to deal with async, right? I am thinking the connections filtering runs before the candidates get their :connected property set. Hmmm, but then the state change would propagate. Still reading...

printing :connections gives empty list (). Yep, 'tis async. I'll see if I can reproduce on a smaller scale.

Just really quickly first though, is there a chance that because I'm manually pulling in a list of matrixes that the calculation of changes isn't happening?

matrixes -> matrix kids

Where do you pull manually?

Hmm, manually would not be the right word, I s'pose. Meant instead of navigating there with fx* and/or friends

In the connections rule, can we put a print statement of the devices right after they are bound in the let?

Sure thing

Do you think I could run this against my bluetooth devices over here? I'd love to pitch in, no charge.

Lessee, the connections will depend...hang on. Can we add a DOALL on the filter? Lazy is the bane of MX! 🙂

We need a print in the filtering function as well, so we know it is actually hitting the :connected? properties.

Ah oooh okay

I do wonder if there isn't a better way to isolate the bluetooth interop IE make it stub-able for tests etc

We can also make the connections formula a :

(cF+ [:watch (fn [_ _ _ _ cell]
                         (dp :connections-used (tiltontec.cell.base/c-useds cell)))]
     ...etc...)

Yeah, I was thinking stub, too.

What we want to see is that the let picks up the devices you expect, and that the formula useds includes the connected? cells of each.

Sometimes I wrestle to get lazy lists realized. Not sure DOALL has always worked for me. Maybe I was just thrashing.

I see gen-device-mx does a mapv, that shouldn't be lazy.

Ah, OK, the-kids does a DOALL. I guess I developed trust in that at some point. 🙂

btw, as each device's :connected? property changes, the :connections rule will re-run and all the candidates will get scanned. No harm.

Sorry for the delay, I've got some really weird thing going on, so I just restarted EVERYthing.

np. Don't mind me. I know you are busy. I'll just throw things over the wall as I spot them.

> btw, as each device's :connected? property changes, the :connections rule will re-run and all the candidates will get scanned. No harm. This is what I expected, but doesn't seem to be happening.

Thanks :)

I would try the print statements to be sure what we think is happening is happening. Might be a surprise in there.

Okay, so it prints while scanning no problem. But it does not print under :connections after the device matrix is updated.

Does the print after the LET in the connections rule ever fire?

Which gets triggered from the .listen in here:

(defn gen-device-mx [scan-candidates]
  (mapv (fn [^ble/ScanResult sr]
          (let [^ble/BluetoothDevice device (.-device sr)
                device-mx (md/make
                           :device (cI ^ble/BluetoothDevice device)
                           :device-name (cF+ [:watch (fn [_ me new-value _ _])]
                                             #_ (.-name (mget me :device))
                                             (.-name device))
                           :manufacturer-data (cI (.-manufacturerData (.-advertisementData sr)))
                           :device-id (.-id device)
                           :status (cI nil
                                       )
                           :connected? (cF (let [status (mget me :status)]
                                             (= ble/BluetoothDeviceState.connected status))))]
            (.listen (.-state device)
                     (fn [state]
                       (mset! device-mx :status state)))
            device-mx))
        scan-candidates))

Yes, but only while scanning. IE before connection

When it prints does it show N devices?

Note that the population of :device-candidates will not change again, so we need the state flow from the :connected? dependencies to trigger the :connections formula again.

Oh, the watch print never triggers. Guess that's to be expected given :connections doesn't currently update.

Which watch?

> Note that the population of :device-candidates will not change again, so we need the state flow from the :connected? dependencies to trigger the :connections formula again. Correct.

The :conncetions one

(cF+ [:watch (fn [ cell] (dc/print [:connections-used (cty/c-useds cell)]))] (let [devices (mget me :device-candidates)] (dc/print [:devices devices]) (doall (filter (fn [device] (dc/print [:device device]) (mget device :connected?)) devices))))

It should fire at least once, when the model is created and all formulas are forced awake.

Oh probably does way back....

OK.

Do you think it is because the mget (mget device :connected?) is under the filter higher-order-function?

Well then that print is worth finding, I think. It needs to show dependencies on the :connected?'s

Aha, #{Instance of 'Atom'} let me deref that..

Did we get a DOALL around that filter invoking the HOF?

Ya

Dependency tracking works by binding a special var to the cell that is running, so the mget inside the HOF should see that.

Okay, here it is:

^ from the :cadidates watch

Ah, looks like dependency only on -- guessing -- the list of candiadtes. Try (mapv tiltontec.cell.base/cinfo (c-useds cell)).

Oh, sorry, you printed the deref.

[[:device-candidates "anon" nil ()]]

The value is (). So we never see the device candidates. Lemme stare at some more code.

Okay. I'll work on a mini-ified version.

Does gen-device-mx get input scan-candidates?

tbh, I think we are better off adding prints to what we have. It does look correct, so a minified version will prolly just work. 🙂

We are good. We have established :connections never sees candidates. Does gen-device-mx?

I am starting to wonder if there is a flaw in my ephemeral? handling.

Hmm. Yes, it does.

Aother thought: maybe lose the the-kids in :device-candidates formula. Should be OK, but I have never done that.

We might also do a watch on the :device-candidates formula to see the new value.

Okay, so :device-candidates updates during scanning, but not when it's child matrix gets updated later via connection.

IE the watch runs

That sounds nominal. :device-candidates gets populated once. After that each element then changes.

But :connections never sees the dev-candidates, so it never reads their :connected? properties.

We did see that :connections had a depenency on :dev-candidates, but saw nil. When :dev-candidates got populated, :connections did not run again.

Maybe have the watch on :device-candidates print out c-callers of the cell the way we printed c-useds?

K...

:callers [[:connections "anon" nil ()]]

Awesome. Hmmm... and the watch shows a new-value full of devices?

You know, I might be able to knock off a parallel recreation using a different async mechanism, if you would like me to wrestle this to the ground.

Have we removed the the-kids? I am just trying to reduce variables.

Oh, and maybe make it :ephemeral? false.

Yep, no more the-kids.

Okay

Hmmm, I would also arrange for the test function to start by making a new bluetooth model each time.

And yep, seeing new-values full of devices

Great. I think the problem may be that we are re0using the same model, if I have that right.

I've been doing numerous hot-resets, would that change that?

Lessee. Deos that :connected formula run each iteration? TBH, now that we are confused I would eliminate the hot restart thing as a variable as well.

I presume you mean hot-reload?

At this point I do not trust anything. 🙂

Good point :)

I do see some unorthodoxy in what we have. It looks OK but I am starting to fade. It might be better if I recreate this tomorrow. I also have some debug machinery at the cells level I can activate if the failure to propagate persists.

But I would begin by creating a function that instantiates a new bluetooth on each experiment. Then hot reload is fine, I think.

Curious about what the orthodox version would look like, but by all means, do get rest when it is needed.

You are not far off! And I might have done the same, you seem to be on top of things nicely.

I just have to wrestle with it a bit.

My recreation will stay very close to what you have until I see what is going on.

Guessing this is how you test? (scan! :eager-connect true :target "PBVM")

Yep, although :eager-connect not necessary currently

Would there just be one scan in the currrent setup? Or could another get kicked off.

Manual, but I have only been doing one typically

Is :scan-results vestigial?

Looks like the test driver sets :scan-candidates directly.

Ah, it used to set :scan-results

I am wondering about initialization now. I wonder if that watch on :scan-results gets fired off and messes with things. I would ## out that guy if we can. (Note to self: document lifecycle.)

Hmm, interesting!

If we had a fresh bluetooth each run I would not worry, because initialization would run and then a scan would set the scan-candidates.

Yeah, scan results is because mostly I didn't know how to filter beforehand.. I think

So the scan-results watch could do its thing and be done, with no lasting harm.

Some things have changed a bit as I was working around different problems. (working around instead of solving for the time being)

Understood. Sounds like programming. 🙂

Haha yep! > So the scan-results watch could do its thing and be done, with no lasting harm. Yep, its sole purpose is to (potentially) filter by target

OK, I should be on this by noon ET tomorrow. Looking forward to it!

I also often have a 3AM shift... 🙂

Okay thank you very much! I think I may have been too slow getting this on github to actually be helpful, but I sent you an invite to the repo.

Got it. Thx!

most recent work on the huh branch :P

Ah, understood. huh is my last escalation before wtf.

But I spell it hunh

Haha I almost wrote mine huhp since "?" are not allowed in branch names. :P

Very lispy!

OK, I am off to the pub. Can't wait to dig into this later. I will just build the same structure but without the bluetooth, using my favorite async lookup to simulate the connection.

Alrighty, thanks again for all your help, it is greatly appreciated! :D