Fork me on GitHub
#onyx
<
2017-07-19
>
lucasbradstreet06:07:17

@maxk could you please provide some more information about the issue? Some helpful information would include:

lucasbradstreet06:07:29

1. Number of peers per node.

lucasbradstreet06:07:48

2. Is it all running locally?

lucasbradstreet06:07:37

3. What is the basic structure of the job? How many tasks in what kind of workflow structure, do you use any windowing etc.

lucasbradstreet06:07:52

4. Is it reproducible?

lucasbradstreet06:07:37

5. Would it be possible that there is a lot of memory pressure on the causing long GCs etc

lucasbradstreet06:07:05

6. Extra onyx log (stripped of anything confidential and sent privately would really help)

maxk13:07:56

@lucasbradstreet , thank you for your help. Below are answers to your questions: 1. 12 vp, 1 node 2. yes, running locally 3. pretty straightforward structure, no flow control or windowing so far (https://ibb.co/fDi9Kk ) 4. it is kind of reproducible. Today I was able to reproduce it twice out of 4 attempts to start a job. 5. yes it is possible 6. I'm trying to gather it and will provide as soon as job will fail next time

stephenmhopper16:07:22

I’m using Onyx (0.10.0) with SQS. At what point are messages deleted from the input queue? Does that happen after a successful write to the specified output task (in this case, an SQS queue)?

michaeldrogalis16:07:59

@stephenmhopper Should be removing items off the queue when the input task gets checkpoint invoked on it — which only occurs after its segments that flowed from it successfully made it all the way downstream.

stephenmhopper16:07:07

Also, @lucasbradstreet I was catching up on Slack and saw that you’re working on something Onyx + ML related. What can you tell me about it? I have a strong ML background, but haven’t done any ML pure streaming yet. I started building out a project for using Tensorflow in an idiomatic Clojure fashion, but found a bunch of architecture issues with Tensorflow that make the project more of a hassle than it’s worth

lxsameer18:07:44

I'm reading the user guide, my understanding about barriers is a bit fuzzy. can some one please give me an example ?

michaeldrogalis18:07:19

@lxsameer That is a very big topic. Do you have any specific questions? I’d refer you to the original paper.

lucasbradstreet18:07:52

@stephenmhopper we're not currently doing anything in the ML space, but I was interested in the patterns for doing it with a stream processor, from training all the way to deployment

lxsameer18:07:18

@michaeldrogalis ok then I'll read the original paper, thanks man