This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-02-12
Channels
- # aatree (9)
- # admin-announcements (2)
- # alda (4)
- # announcements (2)
- # beginners (87)
- # boot (218)
- # braid-chat (14)
- # cbus (2)
- # cider (19)
- # cljs-dev (17)
- # cljsjs (1)
- # cljsrn (5)
- # clojure (84)
- # clojure-android (1)
- # clojure-czech (8)
- # clojure-ireland (3)
- # clojure-madison (20)
- # clojure-poland (22)
- # clojure-russia (54)
- # clojure-sanfrancisco (1)
- # clojurescript (81)
- # clojurewest (4)
- # community-development (94)
- # conf-proposals (5)
- # core-async (199)
- # css (3)
- # cursive (68)
- # datavis (2)
- # datomic (23)
- # dysphemism (138)
- # editors (7)
- # hoplon (8)
- # jobs (8)
- # jobs-discuss (7)
- # ldnclj (2)
- # liberator (6)
- # off-topic (32)
- # om (200)
- # omnext (2)
- # onyx (88)
- # proton (58)
- # re-frame (14)
- # reagent (1)
- # ring-swagger (26)
- # yada (14)
@alexmiller: Please don’t put closed? in the core.async API
I’ve been writing a lot of Go lately. The biggest problem with core.async is that you can’t create sane abstractions without exposing channels. Double bad b/c you have to expose read/write channels.
that would prevent (via an exception) writing to read-only ports and reading from write-only ports, plus closing read-only ports
After all this Go code I’m writing, if i were to return to core.async, I’d probably be writing a fair number of macros for the public interface of my abstractions
@bbloom Are you saying it's bad to put channels at interface abstractions in general? For instance, do you think that Subscription example would be better if it did not expose a channel via Updates ?
@alexisgallagher: no. note the type of the Updates method’s return value
Okay. I was reading your "The biggest problem with core.async is that you can’t create sane abstractions without exposing channels. Double bad b/c you have to expose read/write channels." and wondering if it was still "single bad" in your view to expose channels at all.
select { case sub.updates <- update: // successfully published default: // handle slow subscriber }
Ok, that makes sense.
I've never worked with channels before, but I dove in head first last Monday on a piece of work, so I'm still finding my bearings.
holistically, i don’t think you can enforce everything either statically or dynamically
I agree with that one.
note that i’m happy for that enforcement to be dynamic (exception), rather static (type error, a la Go)
Yeah, makes sense to me. But I've been marinated in Swift for the last year or so so I've become much fonder of type systems than I ever expected to.
I'm still trying to get a sense of where channels add something distinctively superior to other options, and where not. I'm not sure if they should be treated as something you only fall back to for complex cases, or something that should be used universally.
For instance, a few hours ago, I realize that I'd way overcomplicated what I've been working on lately, by adding an unnecessary channel in the middle, but that's probably just because I'm new to them.
lower level like byte streams?
I’ve used a ton of channels and go concurrency patterns for a web service that interacts with FFmpeg and some other services, and now again for a Go application that does a lot of bluetooth low energy stuff
i’ve advised strongly lots of people to remove core.async from large parts of their CLJS apps
Interesting. I remember hearing there was evn a CSP implemetnation in C (libmill) but never figured out how solid or complete it was.
makes sense.
What I spent a hunk of today realizing is that close!
should not be used as a form of back pressure, or for consumer to producer signalling. But this just opened the question of how to tell a producer to shut up. Another channel? Or just set a flag?
My stumbling journey to enlightenment unfolded in #C03S1KBA2. Didn't know about #C05423W6H.
Why another channel? Just for aesthetic consistency? Or is a (chan (dropping-buffer 1)) just to receive a :stop command functionally better than an atom with a stop flag?
Right, that's one way I was wondering about, (alts! control-channel other-chan :priority true)
When i was studying timeouts it seemed to me you'd always want priority or else you face the improbable but possible scenario of your timeout never getting tried and found closed.
even though it makes it deterministic ?
Right, but shouldn't the choice of timing out if possible but guaranteed once it is possible ?
if you alt! on some channel read and a timeout channel read, and the first channel blocks forever, the timeout read will proceed
My worry was if you had (alts! (timeout 100) chanA chanB ... chan1000)
where all channels were available, then it becomes less probable alts will "look" at timeout, and then the effect of the timeout is delayed.
Ah. Ok. New distinction for me to mull over.
That makes sense. So if I'd only put the timeout in the same alts as the read channels if I wanted to force a read timeout. If I want a process tiemout then I put it one level higher.
Thanks, this make sense. And yeah from that POV I can see why :priority seems a bit superfluous. Since you have an ordering mechanism when you really want it, and when you don't want it you really shouldn't be expecting it at all.
No surprise, but seems like the go community has a lot of deep working knowledge on how to work in this style.
That talks looks quit good.
most interesting concurrent go programs are a bunch of little machines, each one with a bunch of heterogenous channels and a go-routine running a for loop around a big select (alt)
Interesting. When i was thinking of "control channel to shut off polling" I felt like I should wrap it in api by having a function like (stop-polling [w])
. Do these APIs often the flavor of start/stop/adjust commands embedded in function names?
otherwise, you basically just need to return a map of channels and write good doc strings on what you can write to which channels when
so instead of something like .Subscribe() returning a Subscription object with an Updates() method, you’d instead return something like {:updates #<a channel> :stop #<a channel>}
and then document your subscribe method as such: “returns a map with two keys. The :updates value is a channel yielding all the changes, until you close the :stop channel”
and then you rely on the “we’re all consenting adults here” policy of dynamic typing to not do anything else to either channel (eg close the updates channel or write to the stop channel)
I like the idea of a stop channel that's not used to send a special value :stop
, but where the client just sends the only message needed by close!ing the channel. That's slick. No faff around deciding the magic token.
if you need something more complex than stop, you can use a control channel and just put keywords on it
So why would it be better for (Subscription) to create and return the channel itself, rather than take the channel as an argument and just build the go block that populates it?
"returns a map with two keys. The :updates value is a channel yielding all the changes, until you close the :control channel. You can also send [:subscribe 123] or [:unsubscribe 456] to add or remove new records to the scope of your subscription”
often it’s nice to take channels as input b/c it means callers don’t need to do as much alt! etc
b/c they can let publishers do the merging on to one channel implicitly by nature of accepting that channel as a parameter & using it
Rather than defining a mini language of special keyword values for the channel, would it be typical/advisable in public API to instead just defining functions (subscribe m num), (unsubscribe m num), etc., that wrapped the dirty details of sending value X onto the channel under keyword Y. ?
I confess: I didn't understand that comment. 😳
the most clojur-y thing to do would be to just do what im saying with messages like [:foo 1] or [:bar 2]
i was expressing my wish to alexmiller that i could get slightly better enforcement for read vs write ports
Hmm, but why not (defn subscribe-num [m num] (go (>! (:updates m) num)))
I didn't mean :updates
. I meant the control channel..
sure, you can do async sends to the control channel and it’ll probably be fine for small number of control messages
You mean within the loop in the go block ?
if you have an unbuffered “stop” channel in your go loop, then a successful write to that channel means that the stop message was delivered
I was earlier imagining a (chan (dropping-buffer 1))
control channel that just sat around waiting for :stop
to show up before you raised the idea of a "close!-only" channel. But now I'm thinking abot the case where there's a more complex control language, and wondering if then it's better to document all the valid keywords or wrap it in functions that know the keywords.
Hadn't really thought through what kind of a buffer a more complex control channel would best have.
for private/internal use, just put the messages you want on the channel and write comments
yeah, simpler.
if you have some kind of public/stable API, you can make macros that write to the channels
Thanks for taking the time to explain these points. Helps me put it all in perspective.
Why macros over plain old functions?
AH! So the user can invoke the macro inside go
.
:mostly_sunny:
That is my light finally dawning emoji.
Enjoying it?
and for all the awful things about Go, and all the complaining i’ve done about it, it’s actually quite a useful tool and super nice in many ways
as an engineer, i love Go. as a language geek, I hate Go 😛 as a concurrency nut, I love/hate Go depending on whether the problem I have is better solved by CSP or a persistet data structures + an atom 😉
happy to provide further experience report details for the other citizens of #C05423W6H another time
Sounds like you've got a real emotional roller coaster of a language there.
thanks!
@bbloom very interesting thoughts, I agree with at least parts of what you said and would love to hear more. I've probably written less Go than you, but lots of concurrent code in other languages. Curious why specifically you don't advocate using core.async in cljs? (also curious about your further thoughts mentioned on Go) I use core.async quite a bit in my current app (lots of server-push and real-time stream processing and re-processing) with great success, though the majority of my "logic" client-side is essentially conveyor belts on the edges calling to other services, server, etc, then coordinating with each other. For the UI directly, I take a simpler approach similar as in re-frame with a single conveyor and a single atom for ui updates. To that end, I'd agree that using core.async to handle each and every dom event or communicate between react components is maybe not the best plan, but not sure why else you advocate against.
@bbloom I appreciate channels and a few things in Go quite a bit, but the rest of the language makes me want to kill myself including the tooling and package management at times. I suppose some of my experience is a bit rougher having not used the latest and greatest Go versions, and writing a lot of it originally in Acme (I'm probably dating myself here, also a former Plan9 user). Anyway, a lot of Go feels like the same old same old I don't like about other languages only with some better concurrency. A win in some ways, but depending on the project, not exactly the first thing I am looking for in a language. Happy thought to accept a job in Go over most other languages, so I guess that's a compliment. I'm a firm believer in the right tool for the right job so don't really care what the language if that's the case.
@hugesandwich: mostly i’m advocating against core.async in views in cljs. React/etc give you such nice determinism, it’s a shame to throw that away
the utility of core.async approaches zero when you have a sane backend API… see all the stuff dnolen has been shouting about for a while now
my favorite thing about Go is that it’s not Scala, which is the other major language in use on my team's backend 😛
It's funny to think how prevalent "My favorite thing about language X is that it's not language Y" probably is. Like we're all fugitives from past language traumas.
Hoare's original work is very readable, imo
Strange people lurking in chat rooms handing out opinions ... that's no basis for a system of software government! Supreme executive power derives from a mandate from the masses, not from some farcical conversational ceremony.
@alexmiller: After reading a bit about CSP and the Actor model, I’ve found that CSP sounds better to me except for the fact that, from what I’ve read, it’s not fault tolerant like actors. Do you have any thoughts on fault-tolerance in CSP?
@bbloom ok, it sounds like we're in agreement on CLJS. As for Go not being Scala, totally agree. I'm a fugitive from many languages, beginning with ASM/C as my first languages along with a bit of Fortran and a dash of COBOL, then Lisp and Smalltalk, all the way through C#, F#, Java, Scala, JS, Clojure, Python, Rust, and Ruby the last few years the most among countless other (ex: Squirrel, Go, Lua, Erlang).... I can safely see almost all have left their trauma, but I remain with a great appreciation of Lisp, Clojure, and Smalltalk even though I'd reach for some others before them. Python, Ruby, and JS leave me with great trauma even though I feel comfortable in them. Let me see I have a great appreciation for the hard work of all language authors. Scala, we could go on, but I believe it was made by people who wish code was ascii art. Python traumatizing me because it was seemingly made by someone who really needs to watch Rich's hammock talk.
I was just going to say, Erlang is giving you a lot different impl of actors and tools than Scala for example
I like clojure more, but elixr/erlang seems better suited for web servers, which is what I’m working on.
I'd recommend Clojure
but I might be biased
@aria I find I write clearer code CSP style and less "magic." That's usually because a lot of the people I've seen implement actors like to pile in all sorts of other stuff and then break things. You can make bad code CSP style too, so it's really just how you want to build your code. I enjoyed writing systems in Erlang, but I don't really like the language beyond some of the messaging related constructs and actors. I find Clojure a bit easier to be productive and more versatile. That said, you could of course write actors and CSP in almost any language, so write actors in Clojure or CSP-style with something else if you want. What I like about CSP is that it is pretty simple. Any time you are writing a distributed system or anything that needs fault tolerance, you tend to need queues anyway. Moreover, the simpler the system, the easier it is to reason about to avoid doing things that break fault tolerance. Core.async provides in-memory abstractions with channels that help you communicate and do things like exhibit backpressure. Even if you build a system in actors, you're using a lot of these things elsewhere in your code, actors or not. For true fault tolerance, you generally need persistent stores and synchronization mechanisms which are more about tools, architecture, and algorithms than they are about using actors or CSP alone. You can screw that up with anything, and most people do (see https://aphyr.com/tags/jepsen)
One of the best things I find about core.async and doing anything that needs some semblance of reliability is that for one, unbounded queues are not the default. Too many times I've worked on systems that work until they don't because it's just "add more RAM" and things like that. Redis comes to mind.
@hugesandwich: wow great answer thanks.
@aria - no problem. You say you are working with web servers, but do you mean writing a server itself or just writing a web app?
@hugesandwich: I mean writing an API server for a webapp
The webapp itself is all static files thanks to frontend templating. All I really need is an API.
@aria ok in that case, either erlang or clojure is fine. If you were writing a web server itself, I'd have other suggestions for you.