Clojurians Log v2

A flow graph when instantiated can absolutely become a process, forwarding life cycle events shouldn’t be too hard either.

Wondering if there is a generalized way to do the conversion

I haven't touched it in a while, so may be out of date with the most recent flow but I was playing with similar stuff here https://git.sr.ht/~hiredman/resderelictae/tree/master/item/irc.clj#L140

My approach is to use unique proc ids which allows to build large flows from simpler, smaller flows.

(defn merge-flows
  "Combine :conns and :procs from all gs."
  [& gs]
  (reduce
   (fn [g1 g2]
     (merge
      (when (or (:procs g1) (:procs g2))
        {:procs (merge (:procs g1)
                       (:procs g2))})
      (when (or (:conns g1) (:conns g2))
        {:conns (into [] cat [(:conns g1) (:conns g2)])})
      (dissoc g1 :conns :procs)
      (dissoc g2 :conns :procs)))
   (first gs)
   (next gs)))

(defn gen-pid [prefix]
  (-> (gensym (str prefix "-")) name keyword))

(defmethod ->frame-flow :concat [media]
  (let [{:keys [inputs]} media

        input-flows (into []
                          (map ->frame-flow)
                          inputs)

        concat-ins (into []
                         (map (fn [i]
                                [(inkw i) "useless docstring"]))
                         (range (count input-flows)))

        proc {:proc (-> (concat-frames-proc concat-ins)
                        flow/map->step
                        flow/process)}
        
        g (apply merge-flows input-flows)
        pid (gen-pid "concat")
        
        conns (into []
                    (map-indexed (fn [i g]
                                   [(-> g :out-coord) [pid (inkw i)]]))
                    input-flows)
        g (merge-flows g
                       {:procs {pid proc}
                        :conns conns
                        :out-coord [pid :out]})]
    g))

->frame-flow will return a flow with a single output. The coord for the output connection will be be in :out-coord (alongside the normal flow keys :procs and :conns).

I've also been experimenting with composing flows that may have multiple inputs and outputs, but it's a bit undercooked at the moment.

as long as proc ids are unique, then you can combine smaller flows into larger flows just by merging :procs and concatenating :conns.

It’d be nice if pids could be passed either as the current keyword, or as a tuple like [pid-kw instance-uuid]. And if I only pass a kw, the flow could just generate an instance uuid automatically. That way monitoring stays clean. Worst case I can still encode the instance id into the keyword and patch the monitor to parse it and display it properly. Which is what you're doing.

My use case is having a library of self-contained processes and flows that I can mix and match.

No

Roger. ✅