Fork me on GitHub
#onyx
<
2018-04-16
>
sparkofreason15:04:34

Doesn't look like it, no :reduce in the source that I can find, and definitely letting all segments through.

lucasbradstreet17:04:04

Yeah, you’re right. It’ll need a bit of a refactoring to separate out the transformed segments from the triggered segments so that when we pass the segments downstream we can correctly identify which ones we want to pass down based on the onyx/type.

lucasbradstreet17:04:17

Actually it might be enough to just decide in here https://github.com/onyx-platform/onyx-local-rt/blob/0.12.x/src/onyx_local_rt/impl.cljc#L276 since onyx.core/triggered has all of the triggered segments. We need to bring it closer in line with what is in onyx core though.

Graham Seyffert23:04:27

Hey everyone - I’m writing a pretty straightforward output plugin, and I have a few questions about making network calls asynchronously The output has to make calls that are potentially very long-running (~30 seconds if it times out/errs), but I don’t want that to hold up other calls being made (esp. if the batch size is more than a few), since the results of the calls aren’t related to each other. I originally wrote it using pmap to execute them all in parallel, but I’m wondering if there’s a better way to handle that that will work better with the framework (and since I don’t see pmap being using in the other plugins I’ve checked out). I set up a test environment using the flat-workflow example in examples repo, and obviously changing pmap to map causes the calls to be executed one at a time, which I’d like to avoid

lucasbradstreet23:04:13

Hi @gseyffert, I have some comments for you, but I’ll have to get back to you on it.