Fork me on GitHub
#core-async
<
2016-12-29
>
tbaldridge06:12:29

@danboykis: nothing wrong with it, but it's not something I've needed much

danboykis23:12:57

@tbaldridge I have an app that gets events from a queue that has no restrictions in terms of call volume. I have to integrate with a slow http endpoint which can't handle more than 1000 connections. Otherwise it'll fall down. For every queue event I have to call this slow http endpoint with the info from the queue event. Using http-kit I can easily blow past the 1000 limit so my idea is to have a channel of size 1000 to which I send queue events in order for it to buffer the amount of outstanding calls to this slow http service. Here's the psuedo code:

(def slow-http-ch (chan 1000))

(go-loop []
  (let [queue-event (<! queue-ch)
  		response-ch (chan)]
    (>! slow-http-ch [queue-event response-ch])
    (write-to-db! (<! response-ch))))

(go-loop []
  (let [[queue-event response-ch] (<! slow-http-ch)]
    (http-kit-call (fn [response] (put! response-ch response)))))

noisesmith23:12:43

@danboykis do I read it correctly that you expect a channel that can hold 1000 items to prevent hitting an http server with more than 1000 in-flight requests at a time?

noisesmith23:12:08

I would typically solve that with one channel with no buffer, and (dotimes [_ 1000] (go-loop ...)) where the go loop is the same as the last one in your sample

noisesmith23:12:41

unless the http-kit-call is non-blocking...