Fork me on GitHub
#onyx
<
2017-12-15
>
Forrest00:12:22

hey there. I am working on a project with a number of grouped tasks and when trying to query the state of said tasks (via the onyx-peer-http-query server), I’m not receiving anything back in the :contents when not supplying the groups in my query params.

Forrest00:12:00

I’m curious if this is expected behavior and if so, where should I go to retrieve the group-ids relevant to my tasks (something I also can’t seem to figure out, lol).

lucasbradstreet00:12:17

We did fix an issue like that in peer-http-query in the last month or so

lucasbradstreet00:12:36

Sounds like you’re just getting empty results instead.

lucasbradstreet00:12:00

If you could fork http-peer-query and demonstrate the issue via https://github.com/onyx-platform/onyx-peer-http-query/blob/0.12.x/test/onyx/health_test.clj, that’d be great.

Forrest00:12:35

we’re using 0.12.

Forrest00:12:20

will take a look at forking right now

lucasbradstreet00:12:22

K. Thanks. We may have seen something similar. Hopefully it’s easily reproducible. If not, let us know and we can dig in further

Forrest00:12:36

k, so it looks like when I don’t specify a groups query param, the health test fails with the following:

FAIL in (health-test) (health_test.clj:137)

expected: (get-in (clojure.edn/read-string (:body response)) [:contents "A stealthy fo"])
  actual: nil

  values: (get-in
           {:contents {}
            :grouped? true
            :window {:window/aggregation :onyx.windowing.aggregation/count
                     :window/id :my-window-id
                     :window/range [5 :minutes]
                     :window/storage-strategy [:incremental :ordered-log]
                     :window/task :my/inc
                     :window/type :fixed
                     :window/window-key :event-time}}
           [:contents "A stealthy fo"])

Forrest00:12:18

and ran the tests

lucasbradstreet00:12:11

Oh, I realised that it was by design (probably bad design)

Forrest00:12:17

just to clarify, my expectation is that if I don’t specify groups, that I should get contents for all the groups

lucasbradstreet00:12:24

Yeah, that’s really how it should work.

Forrest00:12:28

:thumbsup:

lucasbradstreet00:12:02

k, cool, thanks. I can fix that tonight.

Forrest00:12:14

that would be awesome. thanks so much!

Forrest00:12:18

you guys are awesome!

lucasbradstreet00:12:10

How do you find the usability of that feature? I ummed and ahhed about providing a way to do it without supplying the allocation version, but I was worried that users would get at the state over an entire cluster in an inconsistent way. That said, it would make it simpler to play with.

Forrest00:12:35

overall its been very useful. just had a curveball with a grouped task. all of the other tasks i have aren’t, so it’s been really great to be able to query the state and see what’s going on in my windowed tasks

Forrest01:12:06

the params weren’t too difficult to get at either, which made it a really usable feature, imo

lucasbradstreet01:12:41

Yeah, it’s been really nice for us too. Even if you only use it for debugging.

lucasbradstreet01:12:11

Can you please give onyx-peer-http-query 0.12.0.1-20171215.010158-3 a try?

lucasbradstreet01:12:25

If it works for you I’ll release 0.12.0.1

Forrest01:12:11

well, unfortunately, i’d need to redeploy my onyx setup, which takes some time (and also my manager’s sign off 😄), so i’d probably need to test that in the morning.

lucasbradstreet01:12:35

Well, let me know whenever and I’ll cut you a release.

Forrest01:12:53

that’s awesome. thanks so much! will ping you tomorrow for sure

Forrest18:12:13

hey @lucasbradstreet, good news. I tested out your change and it looks great! without specifying group keys I get back everything instead of nil. if I specify the groups, it still works as well. kudos! thanks so much for getting that out so quickly. :thumbsup:

lucasbradstreet18:12:04

Great. Thanks for the update 🙂.

lucasbradstreet18:12:41

0.12.0.1 is going through our release process now.

Forrest18:12:53

💥 awesome