Fork me on GitHub

@maxk could you please provide some more information about the issue? Some helpful information would include:


1. Number of peers per node.


2. Is it all running locally?


3. What is the basic structure of the job? How many tasks in what kind of workflow structure, do you use any windowing etc.


4. Is it reproducible?


5. Would it be possible that there is a lot of memory pressure on the causing long GCs etc


6. Extra onyx log (stripped of anything confidential and sent privately would really help)


@lucasbradstreet , thank you for your help. Below are answers to your questions: 1. 12 vp, 1 node 2. yes, running locally 3. pretty straightforward structure, no flow control or windowing so far ( ) 4. it is kind of reproducible. Today I was able to reproduce it twice out of 4 attempts to start a job. 5. yes it is possible 6. I'm trying to gather it and will provide as soon as job will fail next time


I’m using Onyx (0.10.0) with SQS. At what point are messages deleted from the input queue? Does that happen after a successful write to the specified output task (in this case, an SQS queue)?


@stephenmhopper Should be removing items off the queue when the input task gets checkpoint invoked on it — which only occurs after its segments that flowed from it successfully made it all the way downstream.


Also, @lucasbradstreet I was catching up on Slack and saw that you’re working on something Onyx + ML related. What can you tell me about it? I have a strong ML background, but haven’t done any ML pure streaming yet. I started building out a project for using Tensorflow in an idiomatic Clojure fashion, but found a bunch of architecture issues with Tensorflow that make the project more of a hassle than it’s worth


I'm reading the user guide, my understanding about barriers is a bit fuzzy. can some one please give me an example ?


@lxsameer That is a very big topic. Do you have any specific questions? I’d refer you to the original paper.


@stephenmhopper we're not currently doing anything in the ML space, but I was interested in the patterns for doing it with a stream processor, from training all the way to deployment


@michaeldrogalis ok then I'll read the original paper, thanks man