Fork me on GitHub
#pedestal
<
2018-07-24
>
kenny16:07:07

I have an async interceptor that returns a channel. For example:

(def my-async-interceptor
  {:enter
   (fn [context]
     (println "start processing")
     (async/go
       (println "inside go block")
       ;; imitate some time consuming process
       (async/<! (async/timeout 10000))
       (assoc context :response {:status  200
                                 :body    "ok"
                                 :headers {}})))})
If I get a bunch of these request piled up (the go block takes some time to return), then the println inside the go no longer prints "inside go block" but "start processing" is still printed. Any idea why this would happen?

ddeaguiar17:07:31

@kenny the backing threadpool for go is small. I suggest using thread with io-bound operations. Keep in mind that once you go async, the remaining interceptors are processed on a go thread once your async interceptor returns.

kenny18:07:23

@ddeaguiar Do you recommend for handlers that could have lots of longish (~10s) requests pending to simply be "synchronous"? i.e. no go block?

ddeaguiar19:07:54

@kenny no, I’d move those off the servlet thread. I typically perform any heavy-lifting/integration in interceptors and limit the final handler to response handling

ddeaguiar19:07:02

if you have longish operations, you can use thread instead.

kenny19:07:59

Not sure I totally understand what you mean. In my case my handler sends a message to an external service that processes the request. The handler then takes the response from the external service and constructs a HTTP response. All of that is happening in the final handler in the interceptor chain.

ddeaguiar19:07:38

so you can move the integration piece to an interceptor and leave the response processing to the handler

kenny19:07:51

Why would that make a difference?

ddeaguiar19:07:48

Just communicating how I normally split up work. Either way, use thread not go

ddeaguiar19:07:51

caveat is this

ddeaguiar19:07:09

if your handler returns a channel then that channel is expected to return a response body

ddeaguiar19:07:42

if an interceptor returns a channel, then that channel is expected to return a context map

ddeaguiar19:07:15

I tend to avoid io-bound operations on the serving thread and go blocks

ddeaguiar19:07:22

best to keep those free

ddeaguiar19:07:46

particularly if integrations take ~10s

kenny19:07:13

Most often they will take a much shorter time 0-2s but sometimes it may take ~10s.

ddeaguiar19:07:14

still 2s is a long time

ddeaguiar19:07:23

I’d also consider introducing a circuit-breaker

kenny19:07:09

Is that more than a simple timeout?

ddeaguiar19:07:52

the pattern is more than that but a timeout is a good start

ddeaguiar19:07:57

That may be enough though. It depends on what your needs are

kenny19:07:39

The way it is written now is it waits for a response from the external service for up to 10s, if no response is returned when 10s is hit then an error response is returned.

ddeaguiar19:07:36

Depending on what type of system you are building and the traffic you expect, you can end up exhausting your thread pool

ddeaguiar19:07:03

if, for example, the system you are integrating with goes down for a long period of time

ddeaguiar19:07:32

with a circuit breaker you can detect errors and trip the breaker causing additional requests to fail fast

ddeaguiar19:07:58

then you can test some requests to see if the situation has been resolved

ddeaguiar19:07:13

and either close the breaker or leave it tripped

ddeaguiar19:07:11

but there’s a bit of ceremony involved so you’d use something like this if you really need it

kenny19:07:36

If the threadpool gets exhausted, what happens to additional requests that come in?

ddeaguiar19:07:01

I suppose you’ll see some form of queueing. You’ll see response times go up.

kenny19:07:51

Hmm. Any idea how easy it'd be to exhaust the thread pool?

ddeaguiar19:07:22

If you use go, pretty easy since it’s small (like 8ish threads)

ddeaguiar19:07:41

AFAIK, thread is unbounded so you are limited by your system

ddeaguiar19:07:04

you can change the pool size for go

kenny19:07:18

It sounds like thread is the right approach here.

ddeaguiar19:07:34

I’ve never worked outside of the defaults here

kenny19:07:46

What do you mean?

ddeaguiar19:07:08

I’ve never changed the default threadpool size in core.async

ddeaguiar19:07:42

go blocks are dispatched over an internal thread pool, which
defaults to 8 threads. The size of this pool can be modified using
the Java system property `clojure.core.async.pool-size`.

kenny19:07:03

Oh. Why would I want to do that over using thread?

ddeaguiar19:07:33

Just pointing out it’s possible. I wouldn’t

ddeaguiar19:07:40

the heuristic I use for choosing between go and thread are: cpu-bound -> go, io-bound -> thread

kenny19:07:37

That's quite useful! I'll change to thread for IO ops. Thanks.