@devn: A snapshot build will be cut when master finishes - https://circleci.com/gh/onyx-platform/onyx-elasticsearch/360

Er, @devth ^ Sorry.

ok, great! I'll test it out in the next few days.

Artifact deployed @ onyx-elasticsearch-0.9.9.1-20160815.153513-2.

🙂

Is there anything special that needs to be done in order to send a sentinel value in for kafka? I set the kafka/partition value and published :done to the topic/partition but the job continued to run

@camechis: Sentinel values aren't supported with multi-partition Kafka topics.

so in our case we made one partition

Reliably detecting end-of-stream for that scenario is error-prone, so we punted there.

Ah, should work alright then.

How does the data need to be formatted for the sentinel? From the kafka client

Just :done. Single keyword

ok, i sent in as a string assuming it would get serialized and deserialized. Using the same Kafka api as the kafka plugin

ah, maybe i should have used the edn Serializer?

Strings wont be coerced to keywords

thought that might be the issue so looks like i need to use the edn serializer

so it the kafka client will accept the keyword

Yup, that should do it.

Probably in the future we'll support an offset as a marker for completion, and avoid using a special value.

That'd be far more robust.

awesome, we are using this more for testing out stuff on our cluster with windowing/triggers

one thing we are still experiencing is a lot of retries according to our metrics. Need to definitely figure out whats going on there