Fork me on GitHub
#onyx
<
2016-03-16
>
jasonbell08:03:56

@otfrom has a hugging list…..

lucasbradstreet08:03:45

@greywolve: I merged your onyx-metrics fix. Would you like me to cut a release? We’re on 0.9.0 alphas now but I can cut one from the 0.8.x branch if you need to deploy it

otfrom09:03:41

jasonbell: of course I do

nha09:03:21

Looks like there is a link between onyx and https://github.com/ymilky/franzy ? Just wondering what it is exactly, if this would be the recommended kafka lib for use with onyx, and also if you think something about https://github.com/pyr/kinsky ?

lucasbradstreet09:03:50

@ymilky created it out of a personal need, along with Onyx’s need for a better client than clj-kafka

lucasbradstreet09:03:41

kinsky is too opinionated for our needs

jasonbell09:03:58

@otfrom: it’s published on github right? 😉

otfrom09:03:14

jasonbell: private repo

mrjaba09:03:07

Awesome write up on the Jepsen testing you folks have been doing, really enjoyed reading it http://www.onyxplatform.org/jekyll/update/2016/03/15/Onyx-Straps-In-For-A-Jepsening.html

lucasbradstreet09:03:49

Thanks @mrjaba. It took a while to get right, but it was totally worth it.

mrjaba09:03:57

Sounds like it!

mrjaba09:03:09

thanks for sharing your experiences

lucasbradstreet09:03:13

Now that we have the framework right, we’ll be able to continue to add tests to it pretty easily

mrjaba09:03:49

I'm only working through the tutorials still at the moment, but really looking forward to getting onto the more advanced examples

lucasbradstreet09:03:23

Great simple_smile. Give us a shout here if you have any questions

mrjaba09:03:36

Certainly will, and thanks for all your hard work simple_smile

lucasbradstreet10:03:54

I just replied to your issue :)

lucasbradstreet10:03:14

I'll be pushing out a new release very shortly so let me know if you hit any troubles with that version :)

acron10:03:51

@lucasbradstreet: trying it now simple_smile is there a preferred browser?

lucasbradstreet10:03:12

To be honest I’ve only tested it in chrome. I’ll actually give it a go in Firefox now

acron10:03:29

Ok, I'll stick with Chrome for now, that's no prob

lucasbradstreet10:03:28

Nice. Works fine in Firefox

lucasbradstreet10:03:46

Sorry IE, don’t really care about you

lucasbradstreet10:03:39

I just closed 10 dashboard issues simple_smile

acron10:03:15

Yeah, I saw that, nice going.

acron10:03:34

A couple of issues showing up here, which may not be onyx-dashboard issues.

acron10:03:45

^ is giving me a 5xx ?!

acron10:03:43

does that fb link come from cljsjs ?

lucasbradstreet10:03:35

I actually don’t see any 500s in my network activity when I load it in chrome though

acron10:03:53

Cached, perhaps?

lucasbradstreet10:03:08

uBlock origin is blocking it

lucasbradstreet10:03:22

I clicked the min link sorry

lucasbradstreet10:03:41

Yeah, still 500s

lucasbradstreet10:03:06

Does anything work, or is it completely broken?

acron10:03:25

It falls over after that with base.js:643 goog.require could not find: cljsjs.react :s

acron10:03:37

So, no, completely broken 😞

acron10:03:50

But not your fault!

lucasbradstreet10:03:24

Could you try a lein clean and then try the instructions again?

lucasbradstreet10:03:38

Hate to suggest that 😕

acron10:03:01

I shall, but I think if that react link is bust then it makes sense that everything else falls over

acron10:03:55

Yeah, no change

acron10:03:10

I'm tracking down this cljs.react dependency

lucasbradstreet10:03:03

Hmm yeah. At least it should work with our advanced mode compiled jars, since I think they'll have everything in the one JS file

acron11:03:38

lucasbradstreet: I had an Onyx app running but I've torn everything down and tried to use the sample job. Starting Onyx test environment looks like it might be hanging though

lucasbradstreet11:03:00

You mean via the lein run example in the README?

acron11:03:13

Yeah so I am trying that now

acron11:03:22

lein run -m onyx.peer.dag-test

lucasbradstreet11:03:48

OK, let me know how you go with that first

lucasbradstreet11:03:18

I think it’s probably just ZK conn string problem. I just made a couple changes so you can supply that at the REPL for dev. I’ll point you to it once you’ve confirmed that this works

acron11:03:11

Do I need to be running ZK?

lucasbradstreet11:03:51

Not with onyx.peer.dag-test. That has everything you’ll need to try the dash

acron11:03:28

Still no luck

lucasbradstreet11:03:10

Weird. Stuck at zookeeper still? CI is releasing a new version now if you want to try out a jar

acron11:03:39

Yeah, lets try the jar

lucasbradstreet11:03:26

I'll let you know when it's ready

acron11:03:04

Thanks mate; sorry for the bother, I'm sure the jar or docker will work 😉

lucasbradstreet11:03:09

No bother at all. Good to know about these issues and it'll be good to have someone try out what I just released :)

acron11:03:43

often falling off it 😛

lucasbradstreet11:03:21

@acron it's ready. Could you follow along with the instructions here: https://github.com/onyx-platform/onyx-dashboard

lucasbradstreet11:03:41

Link to releases.md is at the top. Note that you'll need 0.8.11

lucasbradstreet11:03:28

hopes the release process worked (I'm on my phone at the moment

lucasbradstreet11:03:47

Don't run the jar from your onyx-dashboard repo. Clojure will actually complain about mixed mode compiles. Every time this happens I WTF

acron11:03:14

error {
 :cause KeeperErrorCode = NoNode for /onyx
 :via
 [{:type org.apache.zookeeper.KeeperException$NoNodeException
   :message KeeperErrorCode = NoNode for /onyx
   :at [org.apache.zookeeper.KeeperException create KeeperException.java 111]}]

acron11:03:29

I recall an issue about /onyx missing

lucasbradstreet11:03:57

Could be a better error message, but it's probably just not able to connect to a zookeeper which has had onyx connect to it

acron11:03:30

Hah, my onyx app wasn't running!

acron11:03:40

Ok, figwheel has started... lets take a look

acron11:03:00

Sweet beauty

lucasbradstreet11:03:15

Figwheel started? Oops haha. That shouldn’t happen from the jar

acron11:03:27

Sorry, I don't mean figwheel

acron11:03:30

I mean http-kit

acron11:03:38

I got carried away in my excitement

acron11:03:48

Works great!

acron11:03:07

otfrom: quick hangout? you gotta see this

lucasbradstreet11:03:16

Awesome. Let me know if you have any requests. I’ve been in triage mode with it, but I really want to add some features to it in the future

lucasbradstreet11:03:37

By the way, try out the time travel feature and you can step through the cluster state over time

otfrom11:03:01

acron: show me

acron12:03:20

@lucasbradstreet: Super cool, thanks a lot simple_smile We are impressed

lucasbradstreet12:03:45

Glad you like it, and I’m really happy that it’s been triaged. It was somewhat incorrect, and quite hard to use before.

ymilky14:03:35

@nha I'm asking people to try Franzy before I proclaim it should be the Kafka client the should use, but I'm currently using it without issues. As Lucas mentioned, I created it out of a real need. I had a lot of trouble with other Kafka clients, especially on 0.9 ranging from not working to inaccurate results/bugs to being slow. The reason I think it is a good fit with Onyx is that all the data to/from is maps and records. This should make it easy to not only feed records to the producer/consumer in the conventional way, but also to use metadata, metrics, and other info you can get back form the API with onyx. Moreover, since there is also an admin api, at least for my purposes, that helps, especially if I need to bootstrap a topic or add partitions or something before letting data flow out to Kafka from an onyx job.

michaeldrogalis14:03:21

@otfrom: I hope to be on this list someday 😄

acron14:03:16

ymilky: I can more than likely give franzy some production use in the next few months. Similarly to you, I find clj-kafka kinda wonky

ymilky14:03:02

@acron sounds good, I haven't encountered any bugs from Franzy itself, but surely the underlying Java client parts have some bugs, so test it well first

acron14:03:22

Ok, newbie question. Is it possible to run multiple peers on the same machine? I notice peer-ports and peer-port-range were deprecated in favour on a single port number

ymilky14:03:26

@acron the "getBytes" parts were very strange for me

acron14:03:04

Does it matter at all if peers connect on different ports?

michaeldrogalis14:03:26

@acron: Yes, you want to run roughly the number of peers per number of cores in the machine. They'll multiplex efficiently through one port. You only want one peer group per machine, though

michaeldrogalis14:03:13

A peer group is essentially a set of shared resources, which includes the Aeron networking layer. Aeron only wants one of these on the box. So yeah, add more virtual peers, keep it to 1 peer group.

acron14:03:57

:thumbsup:

acron14:03:42

To get the terminology correct: 1 peer contains n virtual peers, and 1 peer group contains n peers ?

michaeldrogalis14:03:13

This is a result of some poor naming at the outset of the project. A "peer" is meant to basically be physical machine, a "virtual" peer one of N workers on the machine, and a "peer group" a set of resources shared locally on the machine for the N virtual peers.

michaeldrogalis14:03:31

Will probably look into clearing up the naming for 1.0, but hopefully that makes sense.

acron14:03:16

@michaeldrogalis: That's cool, thanks for clearing it up

lucasbradstreet14:03:05

@michaeldrogalis: exceptions coming out of handle-exception are qualitatively worse now because we’re dropping the original stack trace, to be replaced by the handle-exception one

lucasbradstreet14:03:22

fault lies in the ex-info returned by merge-error-keys

michaeldrogalis14:03:20

I suspected that - I saw a weird stack trace last night and meant to look into it, thanks for the confirmation

lucasbradstreet14:03:27

This ex-info becomes the new stack trace that gets logged

lucasbradstreet16:03:16

Dynamic types hum

michaeldrogalis16:03:41

@lucasbradstreet: Thanks. Upgrade it to master when you're ready.

lucasbradstreet16:03:00

I’m also going to improve that “no bookkeeper” exception, which was the main reason I encountered it

michaeldrogalis16:03:28

Unsolved problems: automated tests for "good error messages".

acron17:03:41

@ymilky: "You didn't read the code, comments, documentation, or anything else I told you to" 😆

lvh17:03:45

mechanical turk 😄

acron17:03:41

@ymilky: Is there a mechanism within franzy for discovery of Kafka brokers from ZK? Or should one use zookeeper-clj for that?

ymilky17:03:29

@acron what kind of discovery do you need? You can use franzy-admin to list brokers and their associated metadata if you want

ymilky17:03:50

there's also a few methods that will return you info about the replicas, leaders, in-sync replicas, etc.

ymilky17:03:16

there shouldn't be anything that is in clj-kafka that isn't in franzy as far as I am aware, functionally speaking

acron17:03:39

@ymilky: just a broker list. something to satisfy :bootstrap.servers for a producer

ymilky17:03:39

this will read from zookeeper directly, so no need to know about another kafka broker yet

ymilky17:03:00

no prob, good luck and let me know if you find any problems

lucasbradstreet17:03:08

@michaeldrogalis: I’m checking my python version

ymilky17:03:35

@acron the returned list should look something like....

lucasbradstreet17:03:35

Python 2.7.11 is greater than 2.7.8 I assume

ymilky17:03:13

you can get the ssl channels this way for example if they are different from plaintext (or non-existent)

ymilky17:03:02

I suppose I should add a convenience method that will tranduce the results into a valid bootstrap.servers connection list

ymilky17:03:27

@acron working on some other stuff today, but I'll try to throw in something like I described in a commit tomorrow so unless you are desperate, don't bother writing tons of code to manipulate the map returned. Good to hear some real-world feedback for potential feature requests, thanks.

acron17:03:29

@ymilky: no prob simple_smile fyi, I was able to retrieve the broker list as you stated it

ymilky17:03:10

awesome, glad to hear

ymilky18:03:33

@lucasbradstreet: That's correct. You can always go to https://www.python.org/downloads/ when in doubt of versions. Thankfully the versioning is sane, the real trouble is v2 vs v3 versions.

lucasbradstreet18:03:26

Thanks ymilky. Seems like it was a bug in engraver or it’s pip package

acron19:03:55

@ymilky: hmmm best way to diagnose a send-sync! that never completes?

ymilky19:03:54

@acron I'd suggest turning on log4j and seeing what it output

ymilky19:03:09

It's unlikely something is happening in Franzy itself as that call generally just proxies

ymilky19:03:51

additionally, you could try send-async! with a callback and you'll at least get notified when kafka thinks it has ack'd your record. Your actual behavior will be dependent on the ack settings in your config for the producer/broker

acron19:03:40

I'm fairly sure the message isn't reaching kafka

ymilky19:03:42

I haven't had time to tweak, it's just a quick one i had for seeing what kafka is sending back. Obviously you'll also have to be sure you can turn on all the log4j stuff for your clojure project, which in some cases may mean pulling in some of the deps

ymilky19:03:57

or handling it via timbre if you want (there's an add-on for that somewhere)

ymilky19:03:22

all send-sync! is doing is shortcutting a deref of the future to block immediately

acron19:03:51

I'm putting log4j in now

ymilky19:03:06

I've tried sending 1 million+ entries in a single tight loop without issue, but who knows, there could be some issue with the java client itself

acron19:03:01

It's likely something in my config

ymilky19:03:10

as I mentioned, also try the send-async! with callback, and you can try manually deref as well with that since it will return the future to you

acron19:03:10

I'll keep digging simple_smile

ymilky19:03:22

no worries, exactly the kind of field testing I'm looking for

ymilky19:03:25

thanks again