Fork me on GitHub
#onyx
<
2017-05-30
>
jetmind09:05:32

Hi there! We have a function which submits all our jobs to onyx and we’re trying to check if any tasks have been submitted already (to prevent duplicates). We’re using onyx.api/subscribe-to-log for this. The problem is if it’s a new tenancy there’s no log in zookeeper and subscribe-to-log loops infinitely trying to connect. Is there a way to check if log exists? We’re on 0.9

maxk11:05:11

Hello everyone. I'm stuck with onyx and onyx-kafka. Got

org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
all the time from onyx peers. What I am doing wrong?

gardnervickers12:05:40

@maxk Are you sure Kafka is connected to your Zookeeper instance?

gardnervickers12:05:33

I would double check using the kafka-topics.sh script to make sure you can connect to ZK and list topics.

maxk12:05:09

@gardnervickers, thank you. Looks like you're correct

gardnervickers12:05:29

@jetmind You can query ZK directly for your tenancy under /onyx

jetmind12:05:51

I kinda hoped there’s some helper in onyx for this, but thanks 🙂

gardnervickers12:05:15

You pretty much always want to spin on waiting for the log to be created.

gardnervickers12:05:30

At least for Onyx

jetmind12:05:10

yeah, makes sense

devth20:05:09

I started a job that uses onyx-datomic read-log to stream entities to ES on an empty Datomic db. I then populated Datomic with 500 entities, but onyx-datomic didn't read them. killed the job and started a new one and it streamed as expected. will read-log not pick up new entities as they are added? the job was still spinning, and datomic/log-end-tx is unset.

michaeldrogalis20:05:00

@devth It should. Are you sure it wasn’t reading off d/db?

devth20:05:50

no, i'm not sure. hadn't considered that. looking

michaeldrogalis20:05:14

read-log shouldn’t do that, but that’s just the first thing that comes to mind from the description

devth21:05:17

i don't see a db value involved. d/log takes a conn.

stephenmhopper21:05:07

Sorry to interrupt, but what’s the best way to shutdown an Onyx environment to ensure that synchronization operations complete? I’m using the latest version of the SQS plugin and with-test-env. I can write messages to an SQS queue with one job and then successfully read them off of that queue (out through a core.async channel) with another job. But the second job only sometimes makes the call back to SQS to delete the messages and it seems to be based on how much time I let the job sleep for. Is there an onyx.api call for gracefully terminating a job? Can I just call kill-job with the job-id?

michaeldrogalis21:05:54

@devth Can’t say I’m sure what’s going on there off the top of my head. If you manually open up a log connection to Datomic, will it also follow on the new entities?

michaeldrogalis21:05:12

I mean it should - Im just hard pressed to suggest what else could be going on there.

michaeldrogalis21:05:22

@stephenmhopper kill-job will do the trick. The job is going to shutdown gracefully, and will invoke lifecycle/task-stop to finish out operations

michaeldrogalis21:05:39

Ah, you’re on 0.9?

devth21:05:41

inside the loop: (d/tx-range (d/log conn) tx-index nil)

michaeldrogalis21:05:10

0.9 Datomic is pretty actively used

stephenmhopper21:05:20

@michaeldrogalis Yeah, that’s what I thought, but it doesn’t seem to always send the delete message back to SQS. I’ll submit a bug for @lucasbradstreet to look at it

devth21:05:41

strange. i'll have to do some more testing and observing to see what's really going on.

michaeldrogalis21:05:29

Okay thanks - was just curious

michaeldrogalis21:05:42

We’ll need to get a look at it later in the week, LB is on vacation

stephenmhopper21:05:17

He released a 0.10.0.0-20170526.222812-24 version of the SQS plugin as the current beta version wasn’t calling delete messages at all due to a bug

michaeldrogalis21:05:49

You’re just seeing the last few messages on SQS not get cleared as the job spins down?

lucasbradstreet21:05:31

@stephenmhopper I could see how that would happen. You’ll probably have to wait for the next epoch to have completed before you can safely shutdown. We don’t have any helper functions for this yet, but you could make successive calls to https://github.com/onyx-platform/onyx/blob/0.10.x/src/onyx/api.clj#L262 and check the epoch on each?

stephenmhopper21:05:36

@lucasbradstreet ah, so I just have to wait for the next epoch and then call kill-job for now?

lucasbradstreet21:05:59

that said, it should be finishing the epoch before it shuts down

lucasbradstreet21:05:21

though not if you’re kill-job’ing

stephenmhopper21:05:55

@lucasbradstreet Is there a way to ask it nicely to finish the epoch and shut down?

stephenmhopper21:05:31

Can I just close the core.async :out channel?

lucasbradstreet21:05:18

that’s currently the responsibility of the input plugin, and there’s currently no way to do that with the SQS plugin. I could see adding a kill-job that completes the job on the next epoch though.

stephenmhopper22:05:35

@lucasbradstreet okay, cool. Checking the epoch and waiting for it to change seems to be working. Should I update the tests for the SQS plugin to do this instead of just sleeping for a few seconds?

lucasbradstreet22:05:24

That’d be great

lucasbradstreet22:05:33

The sleeping was a hack obviously

lucasbradstreet22:05:56

specifically the global gitignore

stephenmhopper22:05:05

@lucasbradstreet I might have to do that for open source projects. For professional / day-job work, I’ve always tried to avoid a global .gitignore so that I can ensure environment consistency for everyone on my team, but with open source projects, having a global .gitignore might save me some time. Thank you

lucasbradstreet22:05:48

Ah, that makes a lot of sense