This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-05-08
Channels
- # architecture (20)
- # beginners (140)
- # cider (155)
- # cljsjs (1)
- # cljsrn (29)
- # clojure (53)
- # clojure-dev (9)
- # clojure-italy (40)
- # clojure-nl (13)
- # clojure-poland (4)
- # clojure-russia (8)
- # clojure-uk (51)
- # clojurescript (74)
- # cursive (36)
- # data-science (1)
- # datomic (3)
- # emacs (14)
- # fulcro (11)
- # graphql (2)
- # java (21)
- # javascript (2)
- # jobs (3)
- # nrepl (7)
- # off-topic (18)
- # om-next (1)
- # onyx (9)
- # portkey (8)
- # re-frame (72)
- # reagent (64)
- # reitit (4)
- # remote-jobs (1)
- # rum (1)
- # schema (1)
- # shadow-cljs (49)
- # slack-help (1)
- # spacemacs (8)
- # specter (7)
- # tools-deps (27)
- # vim (28)
I must have done something wrong with docker.
I'm getting a bunch of http async request exceptions on a Spandex output plugin that I built for writing to Elasticsearch. Is there a certain strategy I could employ to avoid these timeouts? Maybe something similar to the http-client? And more generally is this an issue with how thread heavy the peers are? I'm getting the exception pretty consistently after I input about 100 segments into the job.
Ah k. If it comes up again and you’re making a lot of async requests you may want to add some back pressure into prepare-batch/write-batch
is there a good example of how to properly do back pressure somewhere? Running into some issues trying to set it up
You can see an example of onyx-amazon-s3 doing some backpressure
https://github.com/onyx-platform/onyx-amazon-s3/blob/0.12.x/src/onyx/plugin/s3_output.clj#L88 only take max-concurrent-uploads at a time, and return false to stay at this state in the state machine https://github.com/onyx-platform/onyx-amazon-s3/blob/0.12.x/src/onyx/plugin/s3_output.clj#L103
by returning false it’ll call write-batch over and over, heartbeating in between