Fork me on GitHub
#onyx
<
2016-06-13
>
acron10:06:14

@michaeldrogalis: How's franzy progress? 🙂

lucasbradstreet10:06:20

@acron: we have a Kafka plugin update ready to merge but we're blocked on how to handle compatibility with both 0.8 and 0.9

acron10:06:36

@lucasbradstreet: that was my next question 🙂

acron15:06:06

Gah, I haven't off the top of my head.

michaeldrogalis15:06:07

We have a Kafka 0.8 and 0.9 plugin, both working, both with totally different impls and dependencies, both with the same interface.

michaeldrogalis15:06:11

Yeah it's a hard one, heh.

acron15:06:42

Is it feasible to just put 0.8 on life-support on a branch and focus mainly on 0.9 ?

michaeldrogalis15:06:09

I dont think so, too many people still use 0.8

michaeldrogalis15:06:26

You can still use the snapshot version of 0.9 though, thats on Clojars already.

acron15:06:04

Ok, thanks

michaeldrogalis16:06:42

I'd like to be able to do :kafka/version :0.8 / :0.9 in the catalog entry, but I can't see a way to not have a cage-match between the dependencies at runtime..

bfaber21:06:17

hey onyx channel, I have a vague and probably stupid question!

bfaber21:06:32

Are there any gotchas on segment retries? I have a test which is looping, but I can't understand if I caused it. We're pretty baffled. It's possible we caused it somewhere, but is there some failure that I might be tripping?

michaeldrogalis22:06:39

@bfaber: The default time that a segment must complete within before it is retried is 60 seconds. What's the mean latency for your segments to run through the workflow?

bfaber22:06:03

90-150 seconds

bfaber22:06:30

(I knew it would be something silly!)

bfaber22:06:39

@michaeldrogalis: A thousand thanks! That did it, I am now happily not looping. 🙂

michaeldrogalis22:06:58

@bfaber: Cool, glad to hear it.

Joe Lane23:06:55

Anyone know why onyx-starter pulled down the choco-solver in its dependencies?

michaeldrogalis23:06:54

@joe.lane: Onyx uses BtrPlace to schedule virtual peers on tasks. BtrPlace uses Choco Solver.

Joe Lane23:06:18

Awesome, I have no idea what that is! Unfortunately i’m new to distributed systems so I don’t know what scheduling virtual peers on tasks means, however I know what choco is does so BtrPlace must be doing some cool stuff. Thanks for the quick response, any tips on getting started in distributed systems other than just go through onyx?

michaeldrogalis23:06:15

@joe.lane: Read lots of papers 🙂 If you make your way through Onyx, there's a chapter in the User Guide dedicated to explaining how scheduling works.

Joe Lane23:06:29

If you had to name the first paper to get started, what would it be?

Joe Lane23:06:37

I’ll follow the citations from there

michaeldrogalis23:06:01

@joe.lane: It's a pretty broad field, depends on what you want to learn. Maybe something by Lamport, he started a lot of the fundamentals. http://amturing.acm.org/p558-lamport.pdf

michaeldrogalis23:06:07

Someone else might have a good suggestion though.

gardnervickers23:06:11

It's hard to jump right in without any keywords to know to look for, I found Kyle Kingsbury's posts and talks on distributed databases really great as a stepping off point

Joe Lane23:06:08

Thanks guys!