Fork me on GitHub
#onyx
<
2015-09-25
>
michaeldrogalis13:09:42

@chrisn: Super cool. I'm interested to see what happens. simple_smile

lucasbradstreet21:09:24

@chrisn I might adjust onyx-seq to just assume your seq returns a sequential of segments. Since you control the fn it's easy enough to wrap it if needed. I'll let you know if I make that change so you can change it before upgrading

lucasbradstreet21:09:28

How'd the run go?

chrisn21:09:56

We are just now starting to fight with docker.

chrisn21:09:10

So I can't tell you and may not be able to tell you soon 😞.

lucasbradstreet22:09:55

Aww. Always fun. If you're forwarding ports make sure to forward the aeron ports on udp, not tcp

chrisn23:09:59

OK, so what we see are that only 1 machine is running the task.

chrisn23:09:23

I have 1 async input which is getting information from a seq, then N workers and 1 custom output.

chrisn23:09:46

The other peers will start the task but they don't get segments.

chrisn23:09:29

sorry, 1 machine is running the job.

chrisn23:09:56

I think most likely the other machines cannot communicate with the 1 peer that is getting segments because we setup the docker stuff + aws stuff wrong.

chrisn23:09:13

It is difficult to get that part right.

chrisn23:09:20

Put another way, my guess is that the peers cannot communicate across machine boundaries because our docker setup is wrong. This means that the peers that are on the same machine that happened to get the 1 peer that is reading segments are all happy but the other peers are not.