@ymilky: Just came back to this, this morning: https://www.refheap.com/116048

Logs are showing

Exception in thread "async-dispatch-2" java.lang.Error: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

prior to that it prints the ProducerConfig and then

[2016-03-17 08:37:58,368][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka version : 0.9.0.1
[2016-03-17 08:37:58,368][INFO][org.apache.kafka.common.utils.AppInfoParser] Kafka commitId : 23c69d62a0cabf06
[2016-03-17 08:37:58,369][INFO][org.apache.kafka.clients.producer.KafkaProducer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.

Last line is a bit suspicious...

And if I put the go-loop outside the with-open, so creating a producer each time, I get...

[2016-03-17 09:48:39,314][WARN][org.apache.kafka.clients.NetworkClient] Error while fetching metadata with correlation id 1 : {command=LEADER_NOT_AVAILABLE}
...
[2016-03-17 09:48:39,314][WARN][org.apache.kafka.clients.NetworkClient] Error while fetching metadata with correlation id 79 : {command=LEADER_NOT_AVAILABLE}

A thought though; is onyx-kafka compatible with kafka 0.9.x ?

It's untested. The idea was that we are going to switch to franzy asap, as clj-kafka did not support the 0.9.0 client

Gotcha

@acron I've seen this happen with apache.curator. Are you using embedded Zookeeper or an old version of Zookeeper?

from the stack traces it doesn't seem like it's a problem with Franzy, rather either something with networking and/or Zookeeper not electing a broker

I would query some of the broker info with Franzy-Admin and see what you get back. It hits ZK directly for most operations so a down broker/ZK issue shouldn't affect it too much

I think I discussed this briefly with Lucas or Michael awhile back, the only problem I've had so far is with curator and kafka 0.9

you can also try setting the advertised host name...it's possible it can't see your broker or the broker can't see zk on your network

metadata.broker.list also might need to be set, similar issue

Gonna try writing a blog post with http://www.themostdangerouswritingapp.com Pray for me, lol

... -Flips on hard mode-

That was surprisingly productive.

haha nice one 😄

michaeldrogalis: where is the post though? 😉

@otfrom: In a Gist, getting massacred by typos and half-thoughts

Onyx 0.9.0-beta1 is now available on Clojars. See the changelog for upgrading: https://github.com/onyx-platform/onyx/blob/0.9.x/changes.md#090

This is mostly a performance-improvement release with a few, small API-breaking changes to prepare for new features in 1.0.

Has anyone seen there's a new mode for Cider giving light-table like variable watches in your functions? It's "enlighten-mode". Pretty cool

Oh wow, I gotta check that out.

@gardnervickers: yes that was I think the best feature of Cider 0.11.0

that's why cider has an edge, potentially it can include any feature in no time 😄

It’s great for some transformation stuff I’ve been doing lately

@michaeldrogalis: could you rename this when you get a minute? https://github.com/onyx-platform/twitter-plugin/issues/1

been bugging me on circle

@gardnervickers: Done

like :lightning: