This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-04-02
Channels
- # announcements (2)
- # beginners (32)
- # boot (10)
- # calva (81)
- # cider (39)
- # clojure (56)
- # clojure-europe (8)
- # clojure-italy (7)
- # clojure-new-zealand (1)
- # clojure-nl (8)
- # clojure-poland (1)
- # clojure-spec (12)
- # clojure-uk (38)
- # clojurescript (5)
- # community-development (1)
- # core-async (55)
- # cursive (3)
- # datomic (44)
- # dirac (15)
- # emacs (20)
- # events (1)
- # fulcro (57)
- # hyperfiddle (2)
- # jobs (9)
- # juxt (9)
- # kaocha (1)
- # lein-figwheel (1)
- # off-topic (93)
- # pathom (2)
- # pedestal (3)
- # planck (3)
- # reitit (15)
- # ring (10)
- # shadow-cljs (25)
- # spacemacs (7)
- # sql (19)
- # tools-deps (8)
Yeah, I'm probably going to be building something that uses a/map
to implement something that has the semantics of mix
, because you can't exactly add inputs to a map, although you can create another map that is composed of submaps
I am trying to a/put!
all incoming messages from an external API onto a chan. Ususlly consumers have no issues keeping up with producers but occasionally there will be huge bursts of 10k messages from a single producer. I cannot discard any of these messages. What are the usual strategies for dealing with this? Just make my buffer size large enough to account for the largest burst? Implement some sort of logic to group the messages prior to the put!
?
the typical amount of messages I'm dealing with is less than 10 per second, and the 10k bursts are rare but predictable, so it doesn't feel quite right to just set the buffer size to 10k, but I could be wrong
I would just do that and have a big buffer to account for the bursts, but
> Implement some sort of logic to group the messages prior to the put!
?
depending on how you are implementing the producer. You mentioned is an external API, are you requesting from it? Or your program has the API and each request gets somehow feeded into core.async?
I should also note that this is an attempt at a library, so I'm hesitant to make the decision for end users regarding buffer size when only certain types of requests require a large buffer for responses. So I guess the best way around this is to allow the users to just pass the whole chan
they require themselves with whatever buffer requirements they need
I just wasn't sure if there was some fancy buffer strategy for these type of bursty situations
Not that I am aware of, I can think of partitions, debounces, and thinks like that, but are more complex than just a big buffer. Iād go with accepting chan as parameter and let the user choose
nils are not valid channel values
you're putting nil on the channel
transducers are applied after you put it on the channel and before you take it off
with some intentional hand-waving about whether it's the producer or consumer thread (as both can occur)
after I put it on the channel and before I take it off? So, does that mean its applied as at the beginning of the put take operation?
same semantics
less hand-waving :)
if that was so, this would work (go (>! (chan 1 (map (fnil identity ::NOTHING))) nil))
right, but if the filter occurred before the put operation, then it would never reach that invariant
my expectation is that it simply would never reach out to the transducing process (the put operation)
https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L291 the transducer transforms the internal buffer add operation
https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L67-L70 values are checked at the door