Clojurians Log v2

Let’s say I want split a program into loosely coupled modules. First approach could be to use something like mount, and share a state (atom) and a few control functions, which is very similar to standard DI approach in OO languages. Second approach is to use channels and exchange data using them. What are pros and cons of those approaches? Channels are conceptionally very interesting and allow things like pubsub, mix, merge etc, and at first I thought that it would be possible to build things in erlang-like way with communicating processes. But, unlike erlang, I feel it raises complexity a lot, I need to think about io (is ws/send is io? Where http kit calls its callback? etc), put! vs >!!c, go blocks, their limitations, transducers, pipelines, buffering strategies, etc etc and probably it’ll be easier, say, not to build pubsub, but just iteratively loop over active sockets and call it a day, etc. especially given Clojure thread safety guarantees. Would like to hear your thoughts on the topic. I really like the idea of CSP and I really like Clojure but can’t quite join them conceptually without complexity overflow. (Just to add some context, async models I find simple: kotlin async/await (but not swift one), goroutines, erlang CSP, I feel you could just use them and not fall into thinking trap)

it's impossible to give a great answer without the context of the actual problem.

e.g. I'm not really tracking why you'd need an atom for option 1, it's not clear what you're wanting to get out of async, etc

It seems like you might be asking, "how do you initialize a core.async program?" but the options you lay out make it seem like that's not the question(?)

Yeah, sorry, I understand it may sound like a blurry description, to make it more concrete, let’s say I have a websocket service, with multiple real time connections, and flow of messages going between subset of those connections, like in slack or discord, and the question is how to structure it with core async In kotlin it could be a set of coroutines, connected by shared flows In elixir/phoenix it could be a set of genservers connected by pubsub I imagine in clojure it could be a list of conns inside an atom or alternatively it could be a core async program, but not sure how to structure it this way or does it make sense at all

is the set of connections dynamic (e.g. 1 per current user)? are you wanting synchronous communication?

it could be several per user, e.g. desktop / mobile And both sync / and async are possible

gotcha. well having thought about it for like 1.5min, my instinct would be to put the connections in some kind of shared state and look them up when you need to use them (i.e. option 1, I think).

synchronous req/response over core.async is doable, but it's kind of a pain, esp considering the underlying connection will be synchronous anyways

unless the connection is async, in which case... yeah idk I'd have to play with it more than likely

(curious to see what others think)

Thank you, makes sense, yes Seems like core async differs a lot from go or erlang where it’s kinda natural to structure the app around CSP style processes (not sure how to put it better)

do you even have a choice in erlang? I'd be surprised if it's any more un-natural than go. we just have different tools in addition to go-blocks.

but if you have a go solution, I would expect the core.async solution to more or less match go-block for go-block.

> choice in erlang You don’t:) but it’s kinda good model imo > go-block Sure, I tried (and failed) to cover this in op post, it will more or less match, but I expect less mental go-tchas in go

So I guess my struggle really boils down to this: core async is interesting but complex, and I wanted someone to tell me that I’m just using it wrong and it’s nice and simple haha

My hope is vthreads move core.async more to go in this regard

> I expect less mental go-tchas in go Yes and no. core.async offers some things go channels don't (priority selects, buffering strategies, xforms). The biggest downside relative to go is the fact that you have to manually put heavy CPU usage and I/O on a thread. That said, I've never actually locked up the core.async machinery, and in general it works okay if you miss a spot, so you shouldn't be too terribly worried about it. I'm curious whether you find it more complex than go, because most of the semantics are identical.

(vthreads will eventually alleviate having to manually put things on thread, but IMO that's more a quality of life change rather than a massive capability upgrade/simplifier)

Maybe you’re right and it’s simpler than I’m imagining, will to try to do it

Erlang doesn't use the CSP model, it uses the Actor model.

Sure, that’s my simplification of things here

What I mean is that it’s conceptually easy to split a program into multiple concurrent processes (e.g. representing a single connected client) and organize communication between them

The difference I think, is the "Actor" also contains the code. Whereas a "Process" is more like a thread.

So basically, I think the answer to your question is actually you'd do both #1 and #2

Like your modules are unrelated to core.async. You can have a namespace with functions that perform what you want done. Then you can start a go process that uses it.

Which I admit, makes it more confusing, because it's more open ended how to organize things.

For example, one approach (using Component) is something like:

(defrecord MyProcess [config state-atom in-chan out-chan]
  component/Lifecycle

  (start [this]
    (let [running? (atom true)
          state-atom (or state-atom (atom {:count 0})) ; Initial state
          in-chan (or in-chan (chan 100))
          out-chan (or out-chan (chan 100))
          
          ;; Main process loop
          process-loop
          (go-loop []
            (when @running?
              (when-let [msg (<! in-chan)]
                (try
                  ;; Process the message and update state
                  (let [new-state (process-message @state-atom msg)]
                    (reset! state-atom new-state)
                    ;; Optionally send result to out-channel
                    (>! out-chan {:status :processed 
                                  :result new-state}))
                  (catch Exception e
                    (>! out-chan {:status :error 
                                  :error (.getMessage e)})))
                (recur))))]
      
      ;; Return the component with its running state
      (assoc this
             :state-atom state-atom
             :in-chan in-chan
             :out-chan out-chan
             :running? running?
             :process-loop process-loop)))

  (stop [this]
    ;; Clean up resources
    (when-let [running? (:running? this)]
      (reset! running? false))
    
    (when-let [in-chan (:in-chan this)]
      (close! in-chan))
    
    (when-let [out-chan (:out-chan this)]
      (close! out-chan))))

;; Helper function to process messages
(defn process-message [state msg]
  (case (:type msg)
    :increment (update state :count inc)
    :decrement (update state :count dec)
    :reset {:count 0}
    state))

;; Constructor function
(defn new-process [config]
  (map->Process {:config config}))

;; Example system using the process component
(defn example-system [config]
  (component/system-map
    :process (new-process config)))

You can make process-message a multi-method if you prefer, or tweak anything. But this should show you one way to organize states/modularize using core.async

Thank you! This sample really helps a lot

would definitely read a book on core async, so much things there (which is good and bad at the same time)

Ya, I feel in general Clojure lacks articles, books, or others about the larger structuring of apps. Like all those "enterprise patterns" style things. Like, showing how to implement a chat room like app, in an enterprise way, aka, not a scrappy thing, but like proper code structure and organization, state management, handling errors, retries, logging, and all that

Probably because it's a small community. I think most articles/books are more in-the-small, walk you through functional programming, lisps, macros, the core functions, abstractions, and so on. But not many then show you how to leverage all that for more complex applications.

Agree 100% I found Clojure Applied the most useful on the topic, but also tbh (re)reading it brought me here:) bc I got the idea how I could bind modules with core async, but wasn’t sure why I needed it and how could I use it for something like you showed above

Well, it turns out I was completely wrong and core async is awesome, and what I thought was a source of complexity turned out to be a great transparency tool to understand what’s going on under the hood (as opposed to magical runtime) (and that was quite predictable tbh..) clj Thanks for advices!