Fork me on GitHub
#onyx
<
2018-05-08
>
sparkofreason00:05:35

I must have done something wrong with docker.

dbernal14:05:16

I'm getting a bunch of http async request exceptions on a Spandex output plugin that I built for writing to Elasticsearch. Is there a certain strategy I could employ to avoid these timeouts? Maybe something similar to the http-client? And more generally is this an issue with how thread heavy the peers are? I'm getting the exception pretty consistently after I input about 100 segments into the job.

dbernal15:05:24

I think it might be an issue with the client and not Onyx. nvm

lucasbradstreet16:05:30

Ah k. If it comes up again and you’re making a lot of async requests you may want to add some back pressure into prepare-batch/write-batch

dbernal18:05:31

is there a good example of how to properly do back pressure somewhere? Running into some issues trying to set it up

lucasbradstreet18:05:02

You can see an example of onyx-amazon-s3 doing some backpressure

lucasbradstreet18:05:02

by returning false it’ll call write-batch over and over, heartbeating in between

dbernal18:05:16

thank you!