This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-02-02
Channels
- # aws-lambda (1)
- # beginners (46)
- # boot (190)
- # cider (12)
- # clara (6)
- # cljs-dev (9)
- # cljsjs (8)
- # clojure (152)
- # clojure-austin (3)
- # clojure-berlin (3)
- # clojure-finland (2)
- # clojure-france (5)
- # clojure-italy (3)
- # clojure-russia (92)
- # clojure-serbia (4)
- # clojure-spec (7)
- # clojure-uk (190)
- # clojurescript (115)
- # cursive (20)
- # datomic (20)
- # dirac (4)
- # emacs (9)
- # gsoc (5)
- # hoplon (1)
- # jobs (1)
- # klipse (4)
- # lein-figwheel (1)
- # leiningen (6)
- # lumo (2)
- # mount (18)
- # off-topic (57)
- # om (68)
- # om-next (14)
- # onyx (33)
- # perun (32)
- # portland-or (4)
- # re-frame (21)
- # reagent (85)
- # ring (6)
- # ring-swagger (23)
- # schema (1)
- # uncomplicate (1)
- # untangled (13)
- # vim (7)
what is the best way to aggregate input to bundle for batched http requests?
Currently we use the http response to update our database status later down the pipeline.
a :trigger/sync seems terminal
@dspiteself You’re looking to send the result of an aggregate to a downstream task?
yea exactly
i know that probably interacts with the timeouts
@dspiteself The old way used to be to route it through another input. We have support for this in 0.10. I’m not sure if the patch for it made it into the first alpha, but if not we have all the hard stuff figured out.
Yeah, that was the difficult piece to get in place - what happens when there’s a downstream failure.
how do you "route it through another input"?
use kafka plugin
We’d usually put it on Kafka
It’s cumbersome, but it got the job done.
But anyway, check with @lucasbradstreet when he’s around. I’ve been tied down in a lot of meetings lately, sorry I can’t give you a firm answer.
ok thanks
The API for triggers is augmented with the ability to pass the aggregate value down to any other immediate downstream tasks.
oh that is released in 0.10.0
@dspiteself We have alpha releases out for 0.10.
https://github.com/onyx-platform/onyx/tree/0.10.x#onyx-0100-asynchronous-barrier-snapshotting
@lucasbradstreet would love to know how to pass the aggregate value down to any other immediate downstream tasks when you get a chance. I tried to find it in the code and there is enough indirection it is difficult.
@dspiteself @michaeldrogalis alas, we don’t have an API for that yet. I’ve held off so far, as the messaging code needs to be non-blocking, so it’s not as simple as just calling the messenger from a trigger.
I’ll see if I can come up with something temporary at least.
no dont worry
Ah, I thought that portion was complete. Sorry for the misinformation.
I will use micheal's answer
It won’t be a lot of work once we know what the API should look like.
any queue that will not require more ops work?
SQS perhaps?
yea we are in google cloud kafka will be fine
if you do drop a feature like that I would love a ping. 🙂
thanks guys
Yep, shouldn’t be much longer. Been in the making for ages.