Clojurians Log v2

I think you're focusing too much on what you can do to a queue, instead of what a queue enables you to do. When Rich et al say "queues are awesome, and you should use them more", I don't think they mean queues are good because you can slide over them or reorder them. I think they mean queues are good because you can put them in-between processes, and suddenly those processes don't need to know anything about each other, other than what's on the queue. The queue is primarily a decoupling mechanism. It's a way to enable CSP; the important part of CSP is the processes, and the fact that they are sequential; in order to achieve that, they communicate through queues. core.async will be invaluable in any frontend work where you conceptually have multiple streams of work you want to happen concurrently, because it will let you structure your code so each of these stream is a single, sequential code block. In other words, go is at least as important as chan.

Didn't read all this, but worth a mention that core.async cljs predates async/await, and this is an interesting article about generators/csp: https://swannodette.github.io/2013/08/24/es6-generators-and-csp/

and I recognize this maintainer but I guess it's not widely used https://github.com/jlongster/js-csp

Empirically, I guess CSP hasn't made much sense in JS land?

I wonder if you can just build some channel-like construct on top of async-await these days, eg: https://docs.rs/tokio/latest/tokio/sync/oneshot/index.html

> core.async cljs predates async/await Yes

guess so: https://github.com/NodeGuy/channel?tab=readme-ov-file#basic-use

Well, certainly

A ton of people have built libs no one is using

CSP is ultimately built on top of callbacks and state machines, so yes; Without macros though it will be quite awkward (not saying the macro approach is ideal)

async/await is IOC, channels are not that special

> A ton of people have built libs no one is using Wisdom of the crowd is not always no.1

As long as it works and solves a problem, it doesn’t have to be the most popular option

It's not the simplest option either

When this popular/not popular argument comes up, I always remember this https://paulgraham.com/avg.html

> … If they didn’t know what language our software was written in, or didn’t care, I wanted to keep it that way.

> It’s not the simplest option either That’s about that specific implementation or CSP in general?

CSP in general, for the frontend use-case

It just doesn't solve a problem anyone has

except the problem of wanting to use CSP for frontend

Well, it does for me lol…

I’ve elaborated above but it does solve problems when the problem is not super trivial

I guess it's a self-consistent framework for functions like throttle?

With only one concurrent source of events, perhaps. But even then, they are concurrent, so effectively each event is independent. Can you reason about order implicitly in JS because there’s typically only one thread? Yeah, but that’s hard and complex. If you care about order, and you only have callbacks/async/await, the order problem is on you, without something like CSP. Not to mention if you need to start computing aggregates, filtering or dropping events, etc.

I think channels with transducers is an underutilized power we have in Clojure. I.e. (chan 42 (map inc))

> I guess it’s a self-consistent framework for functions like throttle? Framework is overstating how low-level it is… Perhaps flow is closer to a framework or “architecture” https://clojure.github.io/core.async/flow.html

Hm, it's not a framework, but it creates a function-coloring problem, so async will make everything that touches it need to also run in async blocks.

There's no blocking-get and put in cljs async, right?

I think that’s the hard part of using it: there are many edge cases to know and think about, but that’s not saying much beyond “concurrent programming is hard”. But I do believe it can lead to a simpler system; without CSP, everyone ends up re-implementing it via stateful callbacks/async/await

> There’s no blocking-get and put in cljs async, right? No

coloring… In that sense, yeah… specifically in CLJS

in practice, I don’t think that’s much of a problem, since often the end result of the whole async ceremony ends up in some atom somewhere; so that’s the escape hatch

Say, an atom that drives React state, most typically

There was another thread in here, my interpretation of that is JVM and virtual threads basically deprecate go-blocks. So what's left is queues and backpressure?

I think it's because, you can kind of sprinkle promises as you need it, and often times, it's not even you that cares, it's just the API returns a promise. CSP requires you to start and think, maybe I can model this using concurrent processes... And that's more of a buy-in from the start.

> There was another thread in here, my interpretation of that is JVM and virtual threads basically deprecate go-blocks. So what’s left is queues and backpressure? In a way, yes! even though technically, the macro go-blocks are staying

But they have less utility, effectively JVM is doing that work instead, for most cases; Still some gotchas around VirtualThread but I think they are getting quite solid now (not much experience with them, yet)

We had no trouble switching a bunch of stuff over to virtual threads. It's magic.

But we're not really using core.async

I said this before, a channel is not a queue. The buffers you can optionally have on them is a sort of queue, if you want to call sliding-buffer or dropping-buffer a queue. But a channel is a synchronization primitive that does rendezvous. Like when you look at it from the abstraction point of view.

It made the profiler output harder to read, so I'm in the process of convincing people to add otel tracing 😛.

I’ve read posts of companies running into problems at scale, but it’s a net win, and I think Oracle is hard at work to eliminate or minimize those cases

OTEL… oh man 🙂 Have you used it?

Hold that thought. One of the issues I had with virtual threads is that we do a bunch of rust FFI, and that can pin virtual threads. So, I basically built my own async runtime to work around it, although we're not sure we want to go that far. I found out that a one-shot channel is essentially just a mutex, as you say, a locking/sync primitive.

I get the supposed value proposition, but last time I tried to use it, it was the biggest antithesis to the Clojure philosophy of “simple” (that’s about OTEL)

I would rather pay NewRelic or similar than deal with OTEL… if it fits the budget

I mean you now have literally unreadable stack traces, if you use virtual threads like we do, which admittedly is an overly-complicated way to do it.

new relic can't fix that for you

do virtual threads generate unreadable stack traces?

I thought that was one of the things that they had as an advantage over Netty-style async programming of the past

Every time you create one, it's like submitting a task to an executor, so you cut your stack off and can't stitch it back together, eg if an exception raises.

As in, you could achieve the same scaling gains of not hogging a thread via Netty, but stacktraces would be bizarre

Hmm…

But at least whatever is on the virtualthread does have a normal-looking stack trace?

Isn’t that good enough?

With a little restraint it could be good enough, not sure we have it 😄

There's a specific pattern we're doing using 'with-open' that will spin up a virtual thread per resource, and there's a lot of those.

That’s like saying that when you call “future” you get cut off…

I guess it’s… expected? 🙂

@(future 
   (throw (ex-info "i throw in the future" {})))

it grew out of this and it's frankly gotten out of hand: https://medium.com/@maciekszajna/reloaded-workflow-out-of-the-box-be6b5f38ea98

before we did it with virtual threads, the stacktraces were unreadable because they were too long, now they're empty

ah… hmm

I’ve been experimenting with something like this:

(defmacro try-io! [& exprs]
  `(try
     (do ~@exprs)
     (catch Throwable e#
       (timbre/warn
         (ex-info "IO failed" {:code-executed '(do ~@exprs)} e#)))))

It’s not a standard solution, but I’ve been enjoying it, at the REPL/during dev at least

(try-io! 
  (slurp "i-dont-exist"))

java.io.FileNotFoundException: i-dont-exist (No such file or directory)
   clojure.lang.ExceptionInfo: IO failed
    code-executed: (do ^{:line 2, :column 3} (slurp "i-dont-exist"))

Who needs stack traces, when I can see the code that ran and failed lol

It is a nice API for expressing complex asynchronous code, regardless of back-pressure considerations. AFAICT, JavaScript runtimes (both browser and Node.js) sweep the concept of back pressure under the carpet. Because everything is asynchronous, you never really know when you’re going to run out of resources on that single thread. It’s like a no-limit credit card. In reality, there is a limit, but you never know when it’s going to come into effect. In practice, for JS runtimes, that’s not much of a problem. Would I personally prefer to use such a runtime for data processing when it matters? No.

Isn't there only one thread in js?

Yes, by default, unless you’re using workers. But that’s almost like another process.

So, yes.

What do you consider complex async work?

i feel like core async is all about ques, when have you managed the que in front end work?

An animation 🙂

Notice the use of alts! and <! everywhere

alts! is useful here because the animation is cancelable by the user throughout, on any step.

That’s just one example, can be coordinating requests between multiple services, calling 5 different APIs, and processing the first one that responds first, etc, etc

Many use cases

What's the user experience those alts are providing?

All theoretically possible with promises, or callbacks, it’s just a question of how readable the code is.

> What’s the user experience those alts are providing? An animation starts on a mobile phone, that’s full screen

alts! cancels the animation on any user touch, via interrupt-ch

Notice the interrup-ch is a promise-chan , so it receives input once, and all takers from that channel receive that input

it’s relatively complex, but the code reads linearly

With promises, or callbacks, it would be some stateful nested mess

With a bunch of if/else everywhere

The animation just count “3”, “2", “1”, “Go!”

You can see the numbers clearly in the code, the countdown

Hmmm.

If you have not used alts! might be hard to make sense of the code

Interesting

Or promise-chan … both very powerful primitives that are hard to replicate with promises, etc

I'm not sure it would be less readable as a series of if blocks, in a way that would be more clear what the path is. What happens if a user interrupts at 2?

The whole animation stops

That would be line 62

I mean, how do i learn that reading the code?

Well, you need to know how the overall system is wired up, just like any other system; but I have not looked at this code in years, and I remember 🙂

If user interrupts at two (by touching the screen): a message is put! on interrupt-ch , which line 62 will receive, and then the code will execute all the way through instantly at all other steps… because interrupt-ch is a promise-ch

Notice, at line 42 we save interrupt-ch in a global mutable state… that is centralized for the whole app.

Gotcha.

So some other code somewhere else puts values on it

Fundamentally, core.async is stateful, but it’s well managed state

Why is it an alt here? What's the alternative to the user event?

(timeout ...) meaning that as far as the go block is concerned, we’re “parked”, aka stopped. But obviously, no actual thread blocking.

While we’re “parked”, bounce-f is running

Hmmm.

Which is actually running an animation

But that is a separate system from the go block, it actually runs in native code… (it’s a mobile app)

Is this a common thing? A way to control/encode timing?

So the animation does not get blocked, alts! just makes it “cancelable” by allowing us to “unpark” at any time, and remove the view, aka “cancel” the animation

I don’t know

That’s how I thought to do it, and worked well for my case

The most barebones one to do, is to use setTimeout

that’s the most primitive construct to “wait”

At least you're using the que

Basically, schedule something to happen in n milliseconds

But how do you now decide not to run that?

Some global mutable variable, that gets checked before execution? Maybe

Again, there’s nothing fundamentally novel here that is not possible in raw JavaScript

It’s just a question of clarity, and a bit more understandable code by using managed/structured concurrency primitives

The penalty is increased bundle size: that kind of async code does get compiled to… quite a lot of JavaScript

If that’s not a problem for you, then I generally think core.async is great for JavaScript

That’s most clear argument I’ve heard against it

My dumb brain just goes back to: i never manage a que of events in js.

You get a http response, you deal with it. You don't get 50 and decided to drop the last 10.

Most other arguments boil down to: “I don’t like it”, “hard to understand”, or “there’s bugs”; I would say most bugs are fixed; Hard to understand? Sure, it’s a different way of thinking about program structure, but that’s CSP, not core.async’s fault.

> You get a http response, you deal with it. You don’t get 50 and decided to drop the last 10. And that might be fine!

If your problem is relatively trivial, like you’re saying: just one response, no problem! Callback, and call it a day 🙂

Now… what happens when we need add retry logic though

How do we do that with callbacks?

More … callbacks? 😅

Perhaps it doesn’t matter, we just expect the user to click again, or we display some error.

Depends on the use case, always

Otherwise we can talk abstract all day

Yes, a different callback

How do we know how many times we’ve retried?

(if we’re talking about retries)

Perhaps use some library… that manages it for us, someone else has written the stateful callbacks 😅

I guess I'm missing how having a que is helping with that...

With core.async, it can be a loop/recur, that has n number, so it looks like idiomatic Clojure code

One very powerful way to use core.async, for an important part of a frontend app, is to put all requests that come from the network on a channel

That way, we know what order the requests came in

Sometimes that can be important, or convenient

I.e. let’s say we need to send request to some analytics service, and register a user at the same time, but we should not proceed with login, before the analytics service responds they got the user info

That would matter if your going to change the order of events in that que.

Anytime order matters

Or when we need to wait, a bit

And “synchronize” between different events

With callbacks, any one of those things typically ends up being a little stateful island in the app… And more often than not they are all ugly and done in a slightly different way; again with core.async, you can design one unified way that is sane and makes sense

Can you design an ugly system with core.async? Sure

But at least gives you the right primitives to design a good one! haha

It’s like a nice set of paint brushes

But it doesn’t solve the problems for you, it’s not a framework, it’s a library

I'm trying to think of a time i have had a fe application call multiple end points at the same time and it was responsible for making sure the logic was ordered. I feel like that kind of strictness always gets pushed into the persistance layer, which is why the frontend is thought of as a view.

Typically registration logic, combined with analytics

i don't follow.

Can be a hairy bit, combined with writing some persistence into the browser, SQLite, etc

Registration logic/UI for a frontend app is where you need to call multiple endpoints

And the calls often have to be ordered

Assuming the frontend is calling some analytics service

So your FE is stitching together data? Thats what i mean, that sounds like a recipe for disaster.

if that info needed to be atomic, it needed to be so at the persisted layer.

Your frontend needs to call Google Analytics

It’s a very common thing people do

Can you call that from backend? Sure

But often people want to call it from frontend also, to detect any potential problems, i.e. say if frontends sends the event to Google Analytics, but the backend does not, something is broken

I'm not tracking how core async plays a role in that though...

One potential way it might work: 1. Call Google Analytics to report Guest visit event 2. Register call to backend (BE) -> get user-UUID 3. Call Google Analytics to report Register event with user-UUID 4. Login call to BE -> get login-UUID 5. Call Google Analytics to report Login event with login-UUID

FE stitching data? yes, how else would you do it, assuming you want to track each one of those steps

If your frontend is literally just a view, then it does not matter, correct

I mean, those are discrete events, i'm making the claim, core async is useful when your dealing with a collection of events as a collection. What you have is a list of handling each event at once. I'm not seeing how coreasync plays a useful role in this.

Well, if you don’t see a problem with the way you’re dealing with async state, then there’s no problem, really!

You started on the premise that core.async is only useful for managing backpressure on the server: there’s other ways to manage backpressure on the JVM, like any queue primitive, those all provide backpressure

> a collection of events as a collection. The events are related, if that’s what “collection” means here, but I’ll stop the discussion here, because I am not sure I can give better examples 🙂

related = they share data

ok, you have 5 things in your que, what are you doing with it? is sliding? are you dropping some? are you waiting until you get 3 before you take some action?

thats what i'm assuming a que would be useful for.

Yes! You can do any of those things!

so you would drop 5 (call GA to report login) if the que size was 4?

Again, it all depends on the use case, but dropping is a very useful primitive, yes, for some cases

Not in this case, that would not make sense… let’s say it’s a browser app that tracks body pose coordinates

Sends me 100s of events, but I only care about the last 10

stuff like that https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html

it sounds like the use cases you’ve worked on just don’t need advanced logic for async handling, and that’s perfectly ok

I think the deck is stacked against core async here because the FE is a view for the user, not a processing house, so the propensity is to only be sending things to the view that can... be viewed. So that means the BE has already curated those events in some order that makes sense.

That might be the case

its been a useful discussion!

Some people don’t even want to have a backend

Local first, etc etc

It’s a minority

But it exists

I’m just saying the world views are broad, and there are always inherently complex use cases 🙂

That core.async can help with

I agree, if the browser app wasn't a "view" the queing capabilities would start to make sense.

Backend does not make things inherently less messy, assuming it still need to do all that coordination, you’re not gaining anything by first shipping all the data 100s of miles away, that’s only adding complexity

(That’s the Local First view point, if I can dare speak for them, even though I’m not super involved with the overall approach, at the moment)

i assume local first is the notion of trying to do more on the client?

All

Not “more”

so "desktop apps" 2.0

https://www.localfirstconf.com (a conference! 🙂 )

(no association on my end, just found randomly)

Smart people have written papers on it, it’s a thing https://martin.kleppmann.com/papers/local-first.pdf

I do like the idea, in the abstract sense

CRDTs

good chat, ttyl

Yeah, martin is a hero of mine. I can't believe i got to chat with him a couple times, real chill fella.

Well, then you better know about Local First! 😄

I have the extremely vague impression the real issue with CRDTS is users are trained on the centralized experience.

One of the most clear explainers of CRDTs

I don’t think it’s that

I think that problem is that it’s economically hard to build things that are truly Local First… But, examples exist; Obsidian?

It’s not “strictly” local-first but it comes close

Also, CRDTs are used commercially also, for example Apple Notes

syncs via CRDTs, almost 100% guaranteed (and the sync is damn good)

I think there’s enormous potential in them, even as a server-to-server sync primitive, or even server-client

The good thing is that CRDTs don’t care if it’s local-first, server-first, etc

CRDTs are also most know for the text syncing algos, which are the hardest, and also a limited use case

There are arguably simpler CRDTs, for syncing data structures

https://madebyevan.com/algos/

Also, the reality is that many apps don’t truly benefit from CRDTs, if the user does not care about Local First/privacy/etc

Again, I think they can be widely useful, and are underused as a cross-server syncing primitive, server-client syncing, etc

Honestly, what i tend to figure out is that if you go down the unconventional path, you end up right where you started, only wiser, but also a bit more worn down lol. Usually the reason things are they way they are is because gravity pulls them there, you can change the units, the universe remains. The big benefit is that it often lets the person who understands the ins and outs to see past the choices that don't matter, to the few that really do.

Put another way, i would start by looking at where CRDTS and local-first is already flourishing....

> only wiser, but also a bit more worn down lol. Haha I agree with wiser, but worn down does not sound very optimistic 😄

> Put another way, i would start by looking at where CRDTS and local-first is already flourishing.... That makes sense, and it aligns with my view of “conventional” places, not strictly “local first”

server-client, server-server sync

> if you go down the unconventional path, you end up right where you started Well, arguably, Clojure is a counter example of that 🙂 Before Clojure, nobody was really using persistent collections in a production environment, with good performance; many people thought it’s not possible to do

That’s, I think, a not often discussed point about Clojure, perhaps out of humility, but in my opnion, it was a very significant innovation

persistent immutable collection, to be specific

sure. It's a nice tool, i'm here and have been for years. It's a bit of an illusion though, it's mutating under the hood. That's what i'm getting at, you usually find the true distance between competing ideas is much closer then it appears verbally.

hahah…. well come on, sure, but yeah that’s were the rubber meets the road, yes

If you went all theoretical and pure functional

It would be slower than a glacier

For all intents and purposes, it’s pure, and functional, from the user’s perspective

Agreed. I came to clojure because it was the right tool for me at the right time in my journey. It clarified everything i was trying to understand before and then offered conventions around issues i was confused about.

For pure functional stuff to be fast, we probably need fundamentally different hardware; that’s a different level innovation, perhaps not for this century lol

Right, i tell people clojure is well suited for network bound computation. (is that the right way to say that?)

if your system needs to run as fast as it can on a single machine, then you should probably start lower (c, rust, etc..)

That’s … debatable; but not going to have that debate right now! 😜

me neither. I'm going to fall back asleep. Thanks again.

https://github.com/bhauman/dotsters https://rigsomelight.com/dotsters/

> What was the justification for the port to clojure script? Javascript also runs on the backend, in mobile apps, and desktop applications (like slack clients)! > i feel like core async is all about ques, when have you managed the que in front end work? Core.async has queues and they are an important part of the design, but I wouldn't say that core.async is about queues. Core.async shines in scenarios where you have multiple asynchronous inputs and outputs to your program. This happens all the time on the frontend where you have an unpredictable user, multiple unpredictable network connections, timeouts, and other asynchronous apis (workers, animations). > there is no way to apply back pressure across the network, or really much use for dropping events in the que There are multiple methods for applying backpressure across the network from the browser. There are also other inputs and outputs besides the network inside a javascript program where you may want to apply backpressure (eg. specifically on the front-end, you may want to disable user inputs to keep the user from initiating too many requests). ---- Another topic that wasn't really mentioned is dealing with timeouts. I find core.aync's timeout support very elegant compared to other async libraries.

> My dumb brain just goes back to: i never manage a que of events in js. For some events, like mousemove isn't it pretty easy to imagine wanting a sliding buffer? Like you want to keep current mouse coords in some state atom that also tracks text of an input field. You could always manage this with callbacks closing over state, but I think if you were to try to extract the essential abstractions so as not do this in an ad-hoc way each time, it be hard to end up with something better than channels+buffers.

Another simple example is a search bar where you can show results as you type. It's common to have requirements like: • don't send a request for every key stroke • update periodically • only have one request in flight a time (but timeout if it takes too long) • debounce the initial key type events

But it's not really so much about the queue as it is about dealing with multiple asynchronous inputs and outputs. It's trivial to implement a queue in javascript.

https://github.com/swannodette/async-tests

> Core.async has queues and they are an important part of the design, but I wouldn't say that core.async is about queues. Core.async shines in scenarios where you have multiple asynchronous inputs and outputs to your program. This happens all the time on the frontend where you have an unpredictable user, multiple unpredictable network connections, timeouts, and other asynchronous apis (workers, animations). Multile async inputs and ouputs...that you put on the queue that you manage?

can you give me a code example? are you talking about using the alts function?

> There are multiple methods for applying backpressure across the network from the browser. Yeah, you request less often right? what else would there be (unless were talking sockets maybe). But thats not a feature of coreasync right?_

> Multile async inputs and ouputs...that you put on the queue that you manage? Yep!

Queues are an important part of the design.

Yeah, you request less often right? what else would there be (unless were talking sockets maybe). But thats not a feature of coreasync right?_

focusing on channels (especially as simple queues) as the value prop of core.async disregards the power of decomposing subsystems into processes that communicate with one another. I posted a few links above which I believe are good examples of this kind of power.

I don't usually recommend core.async because it can be difficult to learn. It's also difficult to apply because managing multiple asynchronous sources/sinks is inherently complex. However, if you do take the time to learn it, it's quite powerful.

@joe.lane i'm reading... I think my confusion is how to judge if this is a good use of a tool like coreasync, or just over engineered, whats the critera? Here is my naive answer to that question, the logic should need to manage the queue, as in dropping-buffer or sliding buffer or maybe alts.

At least in my experience, everything goes wrong the first couple times you try to use core.async and it's very frustrating.

it refers sliding-buffer... then doesn't use it https://github.com/bhauman/dotsters/blob/master/src/dots/core.cljs 😕

Personally I find the higher level core.async functions like a/merge/`a/transduce`/`a/into`/`a/map` really nice to work with, and I like how they work just as well with promise style channels (including the result of a go block) and queue style channels, and make switching between these styles and composing them pretty nice. I agree there's definitely a learning curve, but having got passed it myself I haven't looked back.

@smith.adriane I won’t defend the onboarding experience, it can be rough, but if you have any feedback you think that would ease this frustration I’m all ears.

@joe.lane It's too late. I learned it years ago. I think a lot of the difficulty is the inherent complexity of building asynchronous systems. Maybe a guide that combines the wisdom of CSP and Making reliable distributed systems in the presence of software errors with core.async examples?

And maybe Java Concurrency in Practice for using core.async on the jvm.

Good luck! 🫡

@joe.lane why is dotstar a good example? My criteria, as stated above, was that it would need to manage the queue itself, that, to me, meant using sliding-buffer, dropping-buffer, maybe alts, (their might more queue managing functions) because that implies it's actually _using a queue instead of a over-engineered promise, that example, at a very quick glance doesn't use those functions.

or we can change the criteria, but it needs to fit into some structural framework.

i guess the a/merge a/map, etc.. functions would also count.

i haven't thought about what those imply...

what's wrong with "Core.async shines in scenarios where you have multiple asynchronous inputs and outputs to your program."?

define "shines"

excels

thrives

Merriam Webster offers "to be distinguishable by superiority : surpass others"

imagine someone said "how do i deal with multiple asynchronous inputs" and I said "use something that excels"

would they end up using core async based off that? I don't think it would help them point the way. What i'm saying is that i'm either framing my problem wrong, or the tool that reportorial helps with it.

If i needed queueing logic, i fully understand core async would be useful, i'm trying to understand when thats happening to people though, or if they are just pushing queues into scenarios that don't need them at all.

> how do i deal with multiple asynchronous inputs I would ask multiple follow up questions as well.

I think that response is a bit disingenuous Drew, can we stop with the quibbling of words and their definitions before it drives people away from this conversation?

@joe.lane i was responding to @smith.adriane

he asked "whats wrong with"

If someone asked me how to make a web app, I might suggest javascript, but I don't know how to give a short answer to that question.

I know. I’m asking you not to respond that way to them.

I think it's best to think of core.async a bit like recursion vs loops. Or for vs reduce and so on. It's a different abstraction for modeling concurrent behavior. It's not necessary to have it in JS, because JS already has abstractions to model concurrency, but it enables a choice where some might prefer the core.async abstraction in some situations

be more specific, what did i say that your having an issue with? i'm trying to give an intuition that his remark wasn't going to win people over. You can't simply claim your tool "shines/is-good" you have to show how it fits a problem the user is having.

can you be more specific*

I'm not trying to win people over. I typically don't recommend core.async because there is a significant learning curve that most people aren't interested in.

Having built many asynchronous systems in the browser and on the JVM, I have found core.async to be in invaluable tool. I've also tried recommending it before and have seen folks struggle to get over the hump.

One clear difference is that core.async is based of a stream of values. Whereas generally promise based approaches like in JS are based on single element values. Another difference is that the async stream support in JavaScript is forceful, there's no handshake. So if you need handshakes core.async is nicer.

@didibus my extremely naive question is "what is someone doing with that queue of events" because in my mind, you would need to be using the sliding-buffer or dropping-buffer, otherwise your ... processing every element.

But fundamentally, you need to have a problem where you have concurrent processes, and they need to be sequenced. The "sequencing" means they depend on each other at certain points where they have to wait and get something or communicate something to the other to coordinate themselves concurrently.

> Another difference is that the async stream support in JavaScript is forceful, there's no handshake. So if you need handshakes core.async is nicer. i'm thinking about this... what do you mean by "forceful"?

@joe.lane backing up, thanks for your input. I wasn't attempting to be dis-respectful, but the question was about why a particular phrase wasn't good enough and to re-iterate, it's because, in that case, the phrase was putting all the heavy lifting on a word "shines" which doesn't point in a meaningful direction. It doesn't lead to good follow up questions. @didibus can core async help you create a dependency order on events?

> my extremely naive question is "what is someone doing with that queue of events" because in my mind, you would need to be using the sliding-buffer or dropping-buffer, otherwise your ... processing every element. I use sliding buffers all the time, for the reason they are advertised for; whenever I only care about the most recent event. Examples mentioned earlier are keeping currnet mouse coords from a bunch of mousemove events, or a typeahead search bar where a new keyup event makes the previous string query results unnecessary. Are you saying it's strictly better to implement these things without core.async? I know core async isn't necessary to implement them, but the fact that some people prefer these abstractions should justify its existence on cljs no?

> I use sliding buffers all the time, for the reason they are advertised for; whenever I only care about the most recent event. recent event (1) or events (1+, but less then the everything)?

In those examples, (1)

why not an atom then?

i'm being honest, i feel like im missing something critical in this.

The point of framing core.async the way I did is that it's better to focus on describing and understanding the problem you are trying to solve rather than particular functions that a library has. Core.async can be applied to a wide variety of problems.

Most of your counter examples are focused on the "what" rather then "if" and "when".

When an event is processed or if an event is processed is the hard part of the problem that core.async helps with.

To be more concrete, try to implement the examples from https://github.com/swannodette/async-tests without core.async.

@smith.adriane thanks, ill look this over.

https://swannodette.github.io/2013/07/12/communicating-sequential-processes/

david nolen has a bunch of concrete examples

https://swannodette.github.io/2013/08/23/make-no-promises/

why not an atom then?

actually, I'm not sure which of these are the best examples, but you can check his old blog posts.

From another angle, do you have a chunk of code that does something decently non trivial of the category we're talking about that you feel could definitely could only be made worse by using core async?

@jjttjj you would reset! not swap for this case right? My intuition is an atom would reset! about as fast as a dropping-buffer with size 1. > From another angle, do you have a chunk of code that does something decently non trivial of the category we're talking about that you feel could definitely could only be made worse by using core async? My contention, or hypothesis, is that any interaction that could be sufficiently modeled without a queue management system would be made overly complex by adding one. The way Rich, and to some extend david, talk about core async leads me to believe they have found a way to model most systems with queues, which means they manage buffers (sliding, dropping ,etc...) Im having trouble, because of my past experience seeing front end browser work in this light and it makes me feel like there is some major "ah ha" moment. Meanwhile, a tool like re-frame, or odoyule rules, instantly speaks to the types of problems i run into in my browser work.

it's not so much "core async isn't good at something" it's that some of the literature gives me the impression i'm working in a very different context, is that because of other choices i made or because i'm not viewing a larger category of things in the same light?

I think someone above said they thought part of the issue was that the javascript runtime doesn't really give you any impression of how limited your resources are, so how are you going to go about setting a mechanism to know when to drop things in the buffer? This, plus the browser mostly being a view, as opposed to a "local first application, which persists data to the users computer" means the browser is often getting a curated, pre-ordered set of events. The notion of being streamed a bunch of things, some of which it might have to drop in favor of others, is foreign.

The move mouse example almost seems to fit, but if your only keep the most current, thats what ratoms/atoms do anyway yeah? Or i'm i wrong in this?

https://jackschaedler.github.io/goya/ https://github.com/jackschaedler/goya Another example of a larger application decomposed using core.async. Not an endorsement necessarily, but you can see the different Om components are built independently, some having their own "processes", and being connected via channels.

thanks @joe.lane!

Ok sorry, (looking after my kid as we chat haha). I fact checked myself cause I had forgotten a bit. At first you have Promise vs Channel. A promise is like a Channel with a buffer of 1 that closes after putting a single item. In fact core.async has a channel that mimics a Promise called promise-chan. But fundamentally. Promise is one value at a time. The async process is assumed to be done after that one value is delivered. A channel is fundamentally a possibly infinite stream of values. It's not assumed that the async process is done after producing one value. Now reasoning about what the material and practical and concrete implications of this difference is hard to be honest. Now JS also has await for... Or really it has async iterators. An async iterator isn't like a channel. An async iterator captures a computation and has a next() that returns a Promise. The consumer calls next() on it when it's ready, and gets a promise, then awaits the promise. It's like get me the next value when you have it I'll wait. A channel is decoupled from any one computation. That's why more than one process can put and take from the same one or even to multiple making it a many-to-many. Another difference is the async iterator isn't a buffer, a channel can optionally buffer things. So you can control 1 element at a time, 10, etc.

I want to add something, a Channel isn't a queue. Only if a channel has a buffer is it queueing. A channel is a handshake between two processes. If you put on a channel that has no buffer, the put cannot happen, nothing to put into (no queue). So the producer waits for a consumer to show up. Once a consumer shows up he doesn't take from a queue, he takes from the producer directly (the producer gives it to the consumer). It's a synchronization primitive.

If we go back to the async iterator. That's also a synchronization primitive. The consumer can say, I'm waiting for the next task. Now when the producer shows up, the consumer is ready for them. With a channel, it's the other way around, the producer says I'm ready to publish and waiting for a consumer to show up.

Thanks a lot, that makes sense. I have a lot to think about.

Well, it can go both ways with a channel. The consumer can wait for any producer to put on the channel. That's why they say it's a rendezvous

That's the thing, it's different abstractions, different semantics, but a lot of similarities as well. It's hard to go from that to like, ok so in practice what does it change? When to use which?

That's why I was saying it's a bit like for comprehension vs map/reduce. Or loop vs recursion.

I think some places the CSP model is good at: • Two or more concurrent processes that take turns waiting for each other • Complex conveyor belt style processing where items might fan-out/in, go through multiple stages, etc. • Stream handling and buffering

Thanks didibus. I learned a lot from everyone that chimed in. I'm going to go wrestle with all those ideas and hopefully come out the other end stronger. Sorry if i can on strong early on, i was trying to cut into what confused me and that's always hard to communicate.

panics are more like exceptions but discouraged from use. I was speaking about the Result<T, E> type and different ways to compose an Error type, which is a normal returned value. It's pretty normal to use an enum where each variant might be another enum type, like http error wrapping a json deserialization library error.

The library author can't make a decision for all the possible applications that might use it on whether an error is recoverable or not.

The caller can pattern match on whatever is interesting to them, it's more like checked exceptions but with better tooling. The compiler forces you to do something intentional with the error type

Common things to do are: • just pass it back up the stack to the next caller • wrap the error in another type • wrap the error in a dynamic error type (anyhow), similar to how golang does it, if you are sure the caller won't need the details (although it can still dynamically downcast to get the underlying error type, you don't have the compiler helping you like you would with a pattern-match) • create a panic! and crash the program, if you must

Library code should never call panic itself, application code can but it's kind of a bad idea.

One notable counter-example is that anything that does a malloc can panic (if you run out of memory), and that bothers some people, but it might be worse to put that error into essentially every function's return type, eg if it does something like using strings or vecs, or calls another function that does.

On the JVM, you can catch an OutOfMemory exception if you want, but you likely shouldn't do that. You can catch panics in rust, too, which is also probably a really bad idea.

So, async rust doesn't really deal in exceptions, just returned values that might wrap error types like Result<>. Heterogeneous collections are hard to do, so you can't just do anything like throw 'nil' on a channel. I agree it's weird that core.async decided to use nil that way, and I also thought it wasn't clear how to do exceptions with it when I tried ~10 years ago, before I even got into all the typed FP.

I thought the ocaml monitors were a superior way to do async exceptions, and I think I like using a language that basically avoids exceptions entirely even more.

Does Rust not have nil?

The Error trait seems the same as polymorphic nil to me.

safe code uses Option types, unsafe code can do anything C can do, so yea.

yea, generally you wouldn't use a None to convey an error when you can have a richer type with more details.

Safe code you're not allowed to use nil? Or it's more a best practice?

hm, there's no nil. But pointer derefs are just operations on numbers and nil is basically 0 in that context.

you can https://doc.rust-lang.org/std/ptr/fn.null.html

safe code just doesn't use pointers

Which would still error if trying for a method lookup?

Ok, so say a function wants to take optional arguments? What's the pattern? You sub-in Optional.empty ?

yes, optional positional arguments or a struct with optional fields

So if you lookup a key on a map, and it's not there, you get back an Optional?

yea

you always get back an optional

Ok, so they're just ham on Optional 😂

if it's not there, it's a None

I recently rewatched 'maybe not!' and my take is that in clojure, everything's optional. In a language like rust or ocaml, it's convenient that some things are non-optional

I don't think that addresses the issue of nil in Clojure. The nil in Clojure is much more like Optional already. (The interrop nil is a problem though)

Ya, I find Clojure nil is already like if everything was Optional

And in Clojure "some things are non optional" cannot be enforced. So it's just a doc-string if that's the case and relying on it is trusting the doc string.

Where nil in Clojure I've heard criticism is that it can often be ambiguous. Like (get {:a nil} :a}

So if you want to give "details", like what's the cause of nil? If the caller can benefit from knowing. If you instead return custom indicators then it breaks the generic nil handling for callers that don't care and it's annoying.

Yeah, I don't like to have to think about the possibility that a nil value in a map means something different than if the key is missing.

I try to avoid that in any codebase I'm on by stripping those out and never associng a nil myself.

I was wondering if you merged nil and errors. Say you didn't have Optional, only Result and nil None was just a type of error that implements the Error trait ?

Not every function returns a Result type, either, but you could use error types for that, sure. Anything can be an error type.

The value of a shared Option type is just that you don't have 20 option types that different people wrote.

It's just enums and structs at the end of the day

I don't think the trait system would help you hide that optionality in the signatures but keep compiler enforcement, if that's what you're asking

this is a fun one: https://doc.rust-lang.org/std/result/enum.Result.html#method.transpose

So, if a function in Rust returns just a plain string for example? It means it can't error?

yea, correct

minus the panics I mentioned earlier, String does its own heap allocation and something like running out of memory would crash you.

They have a more explicit result-returning function if you care about that https://doc.rust-lang.org/std/string/struct.String.html#method.try_with_capacity

And what happens if something can error and return optional? Result<Optional<T>> ?

I think that's rare, but I found one: https://doc.rust-lang.org/std/process/struct.Child.html#method.try_wait

Haha ok.

Maybe you want a shared error type for all the functions in a module, it would be weird to add a None case that is only used by some of them.

I still wonder what would happen say we had: nil - same as now nil/type - a "sub-type" of nil, to specify not-found, missing, contains-nil, etc. (nil/type message parent) Constructs a nil type that is like an "exception", will capture stack trace, can wrap another nil type and can include a custom nil message. And they'd all still be nil? true.

In clojure it sounds like you want a nil protocol

You'd need anything that checks nil to call that protocol instead. Effectively you've made ruby nil at that point, I think? It's probably slow.

Ya, in Clojure it would be a protocol. Or some custom thing. And it wouldn't play nice with Java. I'm more in the realm of Clojure 2.0 I think haha. Say no more thrown exceptions (apart from panics like rust). Functions return a result or nil.

I really don't like the way golang does it, but it sounds a bit like that https://go.dev/blog/errors-are-values

It's basically a convention that all functions return a tuple of [value, err] that you have to constantly check

You're not suggesting a tuple, but what's the difference?

if (second-slot) is poor-man's pattern-matching

It's not like that, all functions return either a value or nil. The success path looks just like normal Clojure

In that same talk, Rich seemed to like union types, which might be what you're looking for. I think of them as an enum with unnamed variants. In that hypothetical, a function returns value | nil , not Option<value>

I used them a handful of times in typescript and I'm not sure they're worth it, personally, but a compiler can verify it

It would be like:

(get {:a 1} :a) ;> 1
(get {} :a} ;> nil/key-not-found
(get {:a nil} :a} ;> nil
(get {} inc) ;> nil/invalid-key["Key must be hashable." [:stack ..., :line ...]]

that's all fine, but what do you return out of a function that calls those functions?

And in all cases, you could just do: (when-let [res (get ... ...)] (* 10 res))

sure, I guess it could return nil/get-or-inc that automatically wraps the other things

It's handled like normal Clojure nil is today

The only difference, is maybe if you wanted to know, ok, why is it nil? Then you could match on it.

Hmm, it seems like exceptions are a way to force something to get handled, and this loses that property

If you don't pass it back out, it gets lost

vs with exceptions, if you don't handle it, someone else has to handle

But nil in Clojure behaves that way. Most functions either "handle it" or return the nil.

Yes but I think you're talking about increasing the scope of that to include more things like actual errors, not the presence or absence of values

I suppose a TCP connection error is the absence of a valid TCP connection 😉

You'd have to be aware of it a bit like exceptions that returning a new nil or returning the same nil you got is different, since there are many types of nil. But that's similar to today

If you think as nil as nothing. Then ya, (make-tcp-conn ...) ;> nil It's like, I wanted a TCP connection, I got nothing.

(some-> (make-tcp-conn ...)
        (connect "http://...")
        (send "foo"))

And say you want to "handle errors" instead of just short-circuiting and ignoring them:

(if-let [res (some-> (make-tcp-conn ...)
                     (connect "http://...")
                     (send "foo"))]
  :success
  (do (log "Error sending" res)
      res))

You could come up with wtv convenient syntax, even try/catch could probably work:

(try
  (some-> (make-tcp-conn ...)
          (connect "http://...")
          (send "foo"))
 (catch nil/auth-error e
   (log e)
   e)
 (catch nil/something-else e
   ...))

If the try gets a nil? result, it matches it into the catch forms, otherwise it just returns it

> Hmm, it seems like exceptions are a way to force something to get handled, and this loses that property True, this basically defaults all errors to be "nil". And we assume the callers default to handling "nil" as short-circuit and ignore.

But in Clojure.... there's never any real forcing of handling an exception or a nil either. And this would do the same as in Clojure in a sense... If the nil bubbles up to the top, you see nil/not-found[at line, cause, "Could not foo"] in the REPL

Or if a function didn't propagate the nil it got but replaced it, now you see the replaced one. It's kind of same as exceptions today

So I'm thinking two things... 1. Railway style errors. So errors are returned as values, not thrown. 2. The distinction between nil and error does not matter. The returned value is still just conceptually the same "nothing" with optional details as to why nothing was returned.

I mean it seems easy to unintentionally forget to bubble, and kind of a bad default in that regard. Most apps should have a top level uncaught exn handler that forwards to something like sentry or rollbar

If you see something new there, you usually want to add code to fix it

Ya, that's possible, but I'm not sure. Because if you updated all the core "nil" handling to bubble the nil and not return a new one. Then it might be that you need to manually very rarely make sure you also bubble it back up.

For example (+ 1 nil/not-found) would return the passed-in nil/not-found

Euh, no, actually it would probably wrap it like: nil/value-not-a-number[:line 123, :cause nil/not-found, ...]

You'd even update when and company

(when nil/not-found ...) ;> returns nil/not-found

Would you make a distinction about early exits? eg a syntax like this could skip passing it into when at all (when nil/not-found? ...)

What do you mean? This shouldn't execute the body of when, but instead of returning a new nil it would bubble the one it got and return the same one

Let's say some function (not when) doesn't pass what you gave it back out again, and returns a different nil, but you prefer to bubble the first nil

Ah, well technically you can do that yourself, but I would say that function did the wrong thing, unless it did it on purpose.

eg, with get

is there a nil key in the map? does it have to be the same one?

It should wrap it:

(get {:a 1} nil/auth-error) ;> nil/key-not-found[:cause nil/auth-error]

Also I'm kind of simplifying the print here. But these capture everything an exception does, the full stack trace, line or the error, etc. So you still are able to trace this all back down to the first nil

you might need some utility to walk the returned nil tree to find things that can wrap nil/auth-error as well as the nil type itself 😄

when you're trying to match on either:

nil/auth-error
nil/key-not-found[:cause nil/auth-error]

But now we're not even talking about making nil and error the same. More about how error values should work. Like forget my idea. If you had a Result type. What happens if someone does: (get {:a 1} result.error)

Or even just: (get {:a 1} result)

In static-land, you would just have to unwrap the result to get to the value unless the map keys are also results, before you have anything to pass into get. In hypothetical clojure 2 I have no idea.

Ya, this is Clojure-land 😛. I guess maybe thrown exceptions make sense in Clojure for this reason.... You can't accidentally pass as input an exception you received from the previous function. But I also think, what I suggest here would work. Every function knows that all their input could be "the-value-they-want" or "the-value-they-don't-want" or "nil". And they have to do the right thing in each case. In the case if nil they might want to return it or wrap it.

I like the idea actually, I'm just poking at it from a type-biased perspective

Oh ya, it needs a lot of poking haha. Maybe it's massively flawed somehow. But I do feel there's something interesting in it as well.

One of those biases is if a compiler can verify correctness, it probably should. In this case, if you're matching on a type that can't happen downstream, you probably made a mistake. It should also be knowable what cases can come out of a function and automatically merged into the caller.

Ya, I won't comment on that haha, I hate checked exceptions 😛

well, 1990s java isn't all there is to say about that, I guess

Wait, is rust smart enough to look at the full call tree? Like if my top-level handles them all, everything below isn't forced to do so?

Rust makes you be explicit, but I could imagine a language that doesn't

Ah, so like 1990s java 😛

no, eg, ocaml can figure out the types without you putting them on every function signature, and can even generate the module interface files for you

if you just never write them out, that's kind of similar

of course it's sometimes useful to have names on types

Ya, like I'm ok if the compiler is super smart, and is more saying, I'll tally up and union the set of all total type or errors that could happen at every function call (including from all the function calls that function makes). And then I'll force you somewhere in the entire call tree to handle all of them so none of them "escapes".

And then, if it wants to hook into the IDE so the IDE on hover can show you what is the union of them the compiler analyzed

But if it forces the direct parent caller to have to add a bunch of declarations of it's own, and to explicitly "rethrow" them if it does not want to handle them. I hate that

yes, exactly, I could see a caller filtering out some of the cases from the things it calls or grouping them differently, idk

but the default would just be what we're already doing flying blind in a dynamic language

hopefully a little better than that

But I think the issue is, if you "return" an error, how can it "default handle it". Say:

(defn foo []
  (let [b (bar ...)] ;; Bar can return 3 types of errors
    (bazz b)) ;; Bazz can return 2 types of errors

Functions can be given narrower inputs than what they take. If a function can't handle those nils, then you probably made a mistake and should be forced to do something else like early-return or replace with a default?

Not really, because "If a function can't handle those nils" isn't true. A function can always handle a nil, it might just do so by returning an error.

Right, I think your example is the nuance I asked earlier. Do you want foo to return 2 or 3 (assuming 2 of the 3 are the same)? Does it always call bazz? Exceptions are also about control flow.

What if bazz does a bunch of side effects before returning the error, you can want those to happen or not

If foo cares to handle the 3 errors of bar. Then it could match on b. If it doesn't care to handle them, then it doesn't need to do anything.

> What if bazz does a bunch of side effects before returning the error, you can want those to happen or not That's bazz's problem though. It get garbage input, why does it go and do side-effects before validating?

Hm, motivating example, do you want jetty not to start if your ring handler is broken?

The thing is, when you look at it from a static-lens. I agree with you. Bazz has "validated" itself by declaring it's argument types to be only what is valid. But in a dynamic-lens, I think it's different.

Hm, motivating example, do you want jetty not to start if your ring handler is broken?

What should bazz do if called with nil? What should it do if called with an error ?

Hum.. actually maybe this is where the distinction between an error and nil could matter. bazz might want to handle the nil by subbing some defaults and still working. But it might want to handle error by returning a wrapped error or something like that 🤔

my take is that in Clojure everything is optional

A useful distinction in my mind is specs/schema/malli are runtime checks by all the consumer of the data structure. Static types are at the producer, you can't call the function with the wrong thing.

You still can, you simply can't call it with the wrong types. But there's still plenty of ways to call it with the wrong values or references or with its required context missing, etc.

But you're right though. I think what you bring up is the important distinction. For correctness you can put a runtime assert and then you can safely assume downstream you'll get valid inputs from that point on. But you don't get lineage or where the wrong input originated from and how it got there. Which is the part that makes debugging and refactoring harder without static types. I've always found that's the thing that I miss most, the automatic lineage tracking that just highlights in my editor where the production of the wrong types comes from.

The benefit and the costs 🙂

@gaverhae and @didibus thanks for giving more feedback. my contention, which the abstraction itself seems to promote, is that a single item channel is just a promise, and if that's all you need, you should use it, because it's more specific, informative, and less likely to introduce issues because of an overly flexible api. I'm focusing on queues because it's integral according to the rational: > This is often achieved via the introduction of queues between the producers of data and the consumers/processors of that data.

I still need an hour to review some of the examples of core async on the front end, as i think those are more informative then arguing if it's a good idea, as ideas tend to always seem good unless it's clear what the alternative would have been. The rational argues independent systems can benefit from channels, I'm saying that internal subsystems in a browser client don't need that: they're all coupled to modifying the dom. E.g one isn't crunching numbers because the server already did that. So that only avenue left is managing channels between the client and other physical distant components aka the servers. In those cases, the order of events is already managed by a more complex structure than a queue.

I think focusing on the queue though, which is the underlying implementation mechanism, takes you down a misunderstanding of the CSP model of concurrency. I think it's hard to gage if you could benefit from CSP in the frontend if you don't understand CSP and think of it like a queue. If you do, then you are only answering if you should use queues inside your frontend code. But you want to know if you should use CSP.

When you do (chan) you get an unbuffered channel, there is no queue here. You cannot put an item in it and leave it there, because there is nowhere to leave the item. So when a process wants to put an item on a channel, the process has to pause and wait, and continue to hold onto the item, until another process shows up and takes the item from them. Both the producing process and consuming process have to meet at the same time in order for them to exchange a message. This is called a rendezvous. Now, if the producing process doesn't want to wait for another process to meet up with them and take their item, you need to have a buffer. That's when you can do (chan 1), which creates a channel with a buffer of one. Now the producer doesn't need to rendezvous with the consumer, they can leave the item in the channel's buffer and go do something else, so the producer and consumer process don't have to meet at the same time, they can meet asynchronously instead, but up to a limit, if the channel buffer is full, the producer will have to wait again for a rendezvous. Or the producer can choose to drop their item (discard it), and they can choose which one to drop, the one they have now, or one from the current buffer.

So it's better to think of a channel like a "meeting place". A place where processes meet up to exchange information. That meeting place (aka channel), could optionally have a drop box (buffer), where people can leave things for others to pick up at the same meeting place.

Funny enough, "rendezvous point" is a synonym for "meeting place".

Contrast this with a promise . A promise is like a magical box, I give you this magical box, it's empty at first, and magically it will appear something in it. At the point where something appears in it, it will buzz to let you know something has appeared. Once something appears in it, it's over. Nothing else can appear, you cannot reuse the box, and you also cannot remove the thing from the box. Promise example:

(def p (promise))

(deliver p 42) ;;=> Current process doesn't need to wait

@p ;;=> 42
@p ;;=> 42

(deliver p 10) ;;=> Current process doesn't need to wait, deliver drops since promise already realized
@p ;;=> 42

(def c (chan))

(>!! c 42) ;; Current process has to wait here, this will therefore "pause" the current process, either blocks or parks depending on what the process is running on (thread or fiber). They are waiting at the "meeting place", which is `c` for another process to show up and take 42 from them.

;; Assume the below is in another process

(<!! c) ;;=> 42, our process meets with the previous process, they are now both at the meeting place `c` and the first process hands over 42 to this process
(<!! c) ;;=> Current process has to wait here, the previous process has left the meeting place `c`, and is no longer there. So there is no one else at `c` and therefore nothing to take from anyone. So our process will block or park again, waiting for another process to show up.

;; Assume we are back in the first process

(>!! c 10) ;; The first process shows up to `c` again, and there is already someone there, so they give them 10 and leave.

;; Assume we are back to our other process

(<!! c) ;;=> 10, they were already waiting here, now that someone showed up, they got 10

Now let's look at something a bit more like a Promise, and also see how it differs:

(def c (chan 1)) ;; Channel with a buffer of 1

(>!! c 42) ;;=> Current process doesn't need to wait, can put 42 in the buffer and leave the meeting place `c`

(<!! c) ;;=> 42, no waiting needed, there was something in the buffer, so when process went to `c`, even though no other process was there, it grabbed what was in the drop box and left

(<!! c) ;;=> Process has to wait here again, as it revisits `c`, there is no other process there, and the buffer is empty, so it has to wait.

;; Assume in another process

(>!! c 10) ;;=> Our process goes to the rendezvous place `c`, and the drop box is empty, so it leaves 10 in it and leaves.

;;=> There was another process already waiting in `c`, and so it immediately picked up the item that was left, which was 10, so the previous process now wakes up and gets 10, the drop box is empty again

(<!! c) ;;=> The process has to wait again, as nothing is in the drop box at `c`.

Whoever said the "process" is as important as the "channel" is correct here. In fact CSP stands for concurrent sequential processes (no mention of channel in the name 😛 ) The main difference is that in CSP, you need to start having the concept of these processes. In the Promise case, the same process could deliver to itself, there doesn't even need to have other processes involved. In the case of the channel, you really needed to have 2 or more processes to even demonstrate the channel behavior. I think in the frontend, this would be the first thing. Identity what concurrent processes you have or could have. Maybe each widget is it's own process. Now you want widgets to concurrently handle user events, make remote calls, and re-render. The browser could be seen as it's own process. When user click on the browser, you want it to tell the widgets that a click happened. Put those click events on a channel. Do you want the browser to wait for a widget to handle the click event? Probably not, okay add a big buffer, decide what to do if there are so many clicks and widgets are way behind on handling them. Maybe the click event is fanned out, to all widgets, and each widget receives it, and decides if they care about it or not. If they do, they can go and handle it. Now if one widget has to wait for another widget, they can rendezvous on another channel, etc. Say you type in one widget, it should update another, they can have a channel between them, the one where you type it puts a message to the other, when the other is ready it handles it.

Pedantic nit: CSP is 'communicating' sequential processes

I don't think IOC is part of the formalism

@didibus thanks, really, i appreciate this back and forth were having. Your example of widgets talking to each other over channels is interesting, what do you mean by widget? > Do you want the browser to wait for a widget to handle the click event? Probably not, okay add a big buffer, I feel like this is the kind of example that confuses me, because typically a browser app always reacts to the user click immediately to give them a good experience. e.g if a user clicks "buy" on an item, even if the server has to process their credit card, you still show them something right away to give them feedback. you don't treat them like a second class citizen because the browser is for them. This is maybe at the heart of what i'm confused about, the browser isn't there to process it's there to react to the user and show them something to prompt for further input from them. > Say you type in one widget, it should update another, they can have a channel between them, the one where you type it puts a message to the other, when the other is ready it handles it. Why use a channel? if you just want to display it, then you would mutate the state of the message. e.g the events startin on the left would look like: [+h +e +l +l +l -l +o] and be turned into the current state: "h" then "he", etc... the que would only be useful if you were doing to do more then just pop each one off ... right? Someone linked this video https://www.youtube.com/watch?v=AhxcGGeh5ho and in it david lists a lot of interesting examples, he says one of the reasons he wanted core async is so we could abstract away the source of the events, aka, the browser vs mobile. I'll need to think about that, intuitive it makes sense, but it keeps pushing the value protestation further away. However, it's maybe the first time i was like: "yeah, ok, i get that" because it tries to decouple the producer (servers/user) and consumer (user/servers). I think react just solved this by making all producers and consumers look the same. I feel like thats a big part of this, i feel like react over laps a bit with csp and it makes it hard to see the value proposition with whats left? A lot of his other examples are also interesting, like calling multiple web servers and taking the fastest response...however, that "feels" contrived to me. Like, why is the browser ok with any server response? Why are there multiple servers that can respond with data thats ok? Are they in different phsyical locations? Is this a speed issue? if so, why is it being solved here on the client where... its just so weird. Anyway, thanks again, ill keep poking at this.

Calling multiple web servers probably isn't something you want to do client-side, but it's a real use-case and I saw it recently in gRPC: https://grpc.io/docs/guides/request-hedging/ I agree with the 'react' assumptions, basically you want asynchronous events to appear to be processed instantaneously for the most part. Core async is about complex coordination and implies non-determinism. Maybe if you're doing a simulation or art that might be cool, but that's not the kind of application most people write.

CLJS/reframe has an event queue and used core.async initially, but they switch to a hand-rolled state machine.

you don't need all that machinery to have queues and side effects and decoupling

re: “Maybe each widget is its own process” I assume “widget” roughly means component, in the React sense. FWIW, I am not a fan of this general frontend approach, if you can avoid it. It becomes message passing and very OOP-like very quickly. As perf optimization for limited cases it’s OK (i.e. I really need to deliver data X to component C in the quickest way possible). Generally, I much prefer the approach where the state is driven, ideally, by a single atom for the whole front end app. It’s much more consistent that way and you can reason about the state as a whole. Once you start doing asynchronous message passing from any<->any, it’s effectively a mini-distributed system in your frontend. That’s hard, for no apparent benefit, usually.

When React + core.async were new, that was one of the first approaches people experimented with. I think it’s been tried, and I’ve done it personally – it was a mess.

yeah, frontend is hard enough without being a distributed system

For frontend specifically, I think core.async really shines when it handles the edges of the system. I.e. messages coming from the network, user events - clicks, taps, scrolls, key presses. Events go through core.async, get filtered, processed, ordered, etc and end up in an atom, most typically.

The more you can keep the rendering close to (= view (f @atom-state)), the better

The computer can't simply "always react to a user". You want to make it appear that way to the user to provide a nice responsive experience, but under the hood you need a way to attempt to give that illusion. That's where concurrency comes in. I think what's a bit confusing in the browser JS environment is that there's already other models of concurrency. But I'll give a better example. Say you want to implement a browser based minimal Slack app. You have three panes, on the left, a channel selector pane, on the right a channel messages pane (showing latest messages posted to the channel) and at the bottom a message pane (to type a new message in the selected channel). The message pane has to update itself in new message being posted to the channel. You have a websocket subscribed to the selected channel and your server pushes new messages to it. Can you use a promise? It would be something like (psuedo-code):

(def ws (make-ws ...))
(def p (get-message ws)) ;; Returns a promise
(then p #(update-message-pane %))

(def ws (make-ws ...))
(loop [p (get-message ws))]
  (then p #(update-message-pane %))
  (recur (get-message ws)))

(def ws (make-ws ...))
(add-event-listener ws :on-message #(update-message-pane %))

(def on-message (chan 100))
(def ws (make-ws ... on-message))

(go-loop [msg (

(def on-message (chan 100))
(def ws (make-ws ...))

(add-event-listener ws :on-message #(>! on-message)) ;; And maybe you'd need to see here if you can handle the backoff, though I'm not sure websocket supports backoff

(go-loop [msg (