Fork me on GitHub
#onyx
<
2017-07-25
>
lmergen08:07:11

@stephenmhopper just a shot in the dark, but maybe you forgot to set the region ?

stephenmhopper12:07:14

@lmergen Nope. It looks like I needed "sqs:GetQueueUrl" as well. I was passing in the region and queue-name, so I'm guessing the plugin needed permissions to fetch the URL too. I added "sqs:GetQueueAttributes" too, but I'm not sure if it was required.

theblackbox14:07:47

Hello all, I've finally got round to attempting the aforementioned SSL auth test with my Onyx setup and I was wondering if anyone might recognise the madness I've produced in my logs? Any guidance would be greatly appreciated as I expect I'm simply not providing a config parameter that it is expecting or that the plugin simply cannot make an SSL connection even though I am attempting to pass the correct config params.

theblackbox14:07:11

My apologies if this is Off Topic =D

michaeldrogalis15:07:01

@theblackbox I set up an authenticated Kafka SASL connection once. It.. was difficult. I’d take Onyx completely out of the equation until you make sure that your connection is being established from a normal consumer.

michaeldrogalis15:07:20

There’s very little in the way to do with Onyx other than provide the right consumer settings, so a plain Kafka broker is your best bet.

michaeldrogalis16:07:01

@stephenmhopper Thanks for the PR. I’m sure that will save someone a headache in the future.

stephenmhopper16:07:03

Debugging AWS permissions is never fun

michaeldrogalis16:07:44

Top 5 least favorite activities.

stephenmhopper16:07:54

What are the other 4?

stephenmhopper16:07:08

Classloader / dependency issues?

michaeldrogalis16:07:22

At least with that one, I can unpack the jar and see what’s inside. Sometimes AWS just feels like a black box for things like security.

lucasbradstreet16:07:16

We’ve put a new interactive post up discussing session windows (built with onyx-local-rt). Hope you like it http://pyroclast.io/blog/2017/07/25/patterns-session-windows.html

lucasbradstreet16:07:26

Everything in it applies to Onyx as much as Pyroclast

theblackbox17:07:09

@michaeldrogalis thanks for the heads up. I've got SSL comms established via the command line tools. I haven't checked via the REST API yet. We're using a confluent stack, but that's neither here nor there from what I understand. I was mostly just hoping the logs jogged someone's memory and I'd simply forgotten some setting.

lucasbradstreet17:07:54

ZK and Kafka versions?

theblackbox17:07:30

but I think what you are saying to me is get out of Onyx land and use something clojure native to proof my setup

lucasbradstreet17:07:07

We haven't tested against Kafka 0.11 yet.

lucasbradstreet17:07:12

And yes that would be ideal

lucasbradstreet17:07:27

We're happy to help later but it'll be a bit hard without starting from the basics

theblackbox17:07:40

@lucasbradstreet I'd asked before and was assuming that making the leap from command line consumer/producer to passing those same configs through to onxy/kafka-plugin wouldn't be too great a leap to make

lucasbradstreet17:07:04

the Kafka 0.11 mention was in case you’re using the latest

theblackbox17:07:20

definitely not v11 but don't know for certain the versions on confluent, I'd say it uses 0.10.1.2 off the top of my head though

theblackbox17:07:47

but I'll stop trying to run before I'm walking and head back to clojure and take things slowly

theblackbox18:07:22

thanks for the help as ever =D