Fork me on GitHub
#onyx
<
2017-09-05
>
dwarfylenain07:09:06

I'm trying to write a input plugin and for the moment it's working but the poll! Function seems to be called very very frequently like many times per second

dwarfylenain07:09:22

So is there a way to configure the frequency of the calls to poll!

dwarfylenain07:09:03

Or am I doing something wrong in the poll! Function like returning a wrong value resulting in unnecessary polling ?

lmergen07:09:55

@dwarfylenain yeah this is by design, what you can do is simply invoke a Thread/sleep command inside the poll! function as long as it is not longer than timeout-ms (last argument passed to the poll! function)

dwarfylenain07:09:37

Thank you I will try that

dwarfylenain07:09:57

I mean in this case it doesn't really matter to me because I'm receiving the input data as a stream so it doesn't "cost" me much to poll and see if there is new data in my buffer

lmergen07:09:31

i believe there was some discussion of implementing a proper backoff strategy here

dwarfylenain07:09:37

But my next input plugin will actually call a rest API where there is rate limiting so

lmergen07:09:39

but as of now, it's not there

dwarfylenain07:09:29

Yo but the sleep thing should work for me

lmergen07:09:51

@dwarfylenain i think the best thing to do is keep track of the rate limiting inside your poll! function, but just return nil when you're currently rate limited

lmergen07:09:42

when you're blocking inside the poll! function for too long, bad things could happen (since you're also blocking other lifecycle activities)

dwarfylenain07:09:15

Oki I see, it makes sense, thank you

dwarfylenain07:09:38

Just a last question

dwarfylenain07:09:52

The timeout you are talking about is the value you provide in your catalog in :onyx/batch-timeout

dwarfylenain08:09:09

Or it has nothing to do with it ?

lmergen08:09:34

yes that's the same

dwarfylenain08:09:28

Ok, great ๐Ÿ™‚

lmergen08:09:35

but, if you set it too high, it's effectively the same as just 'blocking' the thread

dwarfylenain08:09:53

I digged into the docs but not really into the code itself

dwarfylenain08:09:45

Ok so I just return nil except if I know I can make a โ€œnewโ€ call the API for fresh data (like I counted that since last call enough time has passed)