Fork me on GitHub
#onyx
<
2018-04-28
>
Drew Verlee00:04:19

@lucasbradstreet How could i go about verifying the behavior of the window-state? I have a fairly small program and i’m not sure how to debug it. I’m expecting the :trigger/sync function to fire and it doesn’t. Hopefully i’m not missing something small here. Code for reference: https://github.com/drewverlee/ad/blob/master/src/ad/core.clj

lucasbradstreet00:04:23

I would try copying the watermark trigger code into your own code base and add some printlns. I’m out and on my phone at the moment, but I see two risky things they. One is that I’m not sure the watermark scheme supports local rt because that functionality required a lot of tracking watermarks across channels. The second problem is that I can’t remember if watermarks are a window state trigger now. It probably is, but I would want to check.

Drew Verlee00:04:51

@lucasbradstreet thanks for the input. I’ll see what i can do.

lucasbradstreet00:04:54

I have a pretty strong feeling we’re not tracking the min watermark across DAG edges in local rt

jqmtor13:04:51

I noticed that the developer's guide is outdated in regards to the branching policy (https://github.com/onyx-platform/onyx/blob/0.12.x/doc/developers-guide/branch-policy.md). The preferred way of submitting a PR nowadays is to target master, right?

jqmtor14:04:19

I created a couple of tiny PRs with stuff that I found along the way while upgrading the plugin template

lucasbradstreet17:04:30

@quimrstorres ah yes, we dropped develop and purely work from master now. It wasn’t worth the overhead.

lucasbradstreet17:04:02

Updated it. Thanks!

👍 4
joelsanchez21:04:41

hi, it seems that no one is using onyx-http? I can't get it working and I don't find examples in github

lucasbradstreet22:04:43

It should work. If you git clone the repo and run lein test do they pass?

lucasbradstreet22:04:54

If so, check the onyx.log to see if there’s an error

joelsanchez22:04:32

the tests pass but in onyx-starter the messages just seem to go through untouched

joelsanchez22:04:40

only getting Aeron status channel error in the log

lucasbradstreet22:04:29

You’ve modified onyx-starter to use onyx-http?

lucasbradstreet22:04:10

Does that job even submit OK? We don’t currently support output tasks that have other tasks downstream

joelsanchez22:04:34

yikes, that must be it

lucasbradstreet22:04:08

the success? key will probably be false

lucasbradstreet22:04:24

Also, note we have a template you can use to start a new project https://github.com/onyx-platform/onyx-template

joelsanchez22:04:03

thanks, I'll use the template when I get the basics working

joelsanchez22:04:17

result {:success? true, :job-id #uuid "085f412c-30d9-7cc1-939e-095f9213961b"}

joelsanchez22:04:40

who knows...anyway I prefer using clj-http directly, also it seems that onyx-http is not designed for my usecase

joelsanchez22:04:58

I was having much success with clj-http

lucasbradstreet23:04:09

Ok sure. If you’re going to use clj-http you may want to try onyx/batch-fn to fire off a a batch of async requests at a time.

lucasbradstreet23:04:28

Otherwise you’ll find throughput challenging

👍 4
joelsanchez23:04:19

thanks for all the help, really helped me as an Onyx beginner 🙂