This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-03-04
Channels
- # admin-announcements (3)
- # alda (4)
- # beginners (30)
- # boot (116)
- # cbus (5)
- # cider (20)
- # clara (10)
- # cljs-dev (12)
- # cljsjs (41)
- # cljsrn (9)
- # clojars (6)
- # clojure (131)
- # clojure-bangladesh (5)
- # clojure-colombia (2)
- # clojure-dev (9)
- # clojure-ireland (4)
- # clojure-japan (3)
- # clojure-norway (10)
- # clojure-poland (6)
- # clojure-russia (59)
- # clojure-sg (1)
- # clojurebridge (2)
- # clojurescript (76)
- # clojurewerkz (4)
- # css (6)
- # cursive (21)
- # data-science (24)
- # datomic (27)
- # emacs (9)
- # hoplon (68)
- # jobs (2)
- # jobs-rus (1)
- # ldnclj (10)
- # lein-figwheel (9)
- # leiningen (21)
- # off-topic (5)
- # om (232)
- # onyx (63)
- # parinfer (2)
- # proton (25)
- # re-frame (12)
- # reagent (39)
- # untangled (6)
- # yada (122)
I'm working through the learn-onyx
challenges and notice that if I mess one up, the tests hang - what's the cause of this? is this avoidable?
Usually it’ll be something that’s logged in onyx.log
Are you using the repl to run the tests?
Generally they hang because it’s waiting to read the segments back but they never arrive. If you’re using the repl and viewing the onyx.log you can generally ctrl-c, fix it, and rerun the test from the repl
We have an extra test-helper function called feedback exception, that reads the serialized exception back from ZooKeeper, which we should start using in learn-onyx. For now you need the onyx.log
I recommend tail -F on the onyx.log while you dev
If you add this line to the tests it might save you some hassle https://github.com/onyx-platform/onyx-template/blob/0.8.x/src/leiningen/new/onyx_app/test/onyx_app/jobs/sample_job_test.clj#L27
@acron: C-c C-b should unstuck you, unless Cider changed the bindings since the last time I used it
@michaeldrogalis: wow, I never knew that! ever learning in the land of Clojure.
@lucasbradstreet: For the task bundles regarding the schemas, could you refresh my memory? We’re no longer merging with os/TaskMap correct? https://gist.github.com/gardnervickers/7705757653df28dd0e49
or os/Lifecycle
@gardnervickers: the problem is that os/TaskMap isn’t actually a map https://github.com/onyx-platform/onyx/blob/0.8.x/src/onyx/schema.cljc#L191
It’s a conditional that allows us to use different schema definitions for different types of tasks e.g. input/output
Hmm I wonder why I didn’t notice that before.
So what I prefer to do is to do this: https://github.com/onyx-platform/onyx-datomic/blob/0.8.x/src/onyx/plugin/tasks/datomic.clj#L25
Note the UserTaskMap key. That basically says, allow any other key than a :datomic one
That way they can’t make :datomic/whatever typos and it’ll be allowed
ohhhhh
I did not know you could do stuff like that with Schema. I’ll take a look at how it’s implemented.
Yeah, the conditional part is important for things like allowing :onyx/group-by-key XOR :onyx/group-by-fn
With this sample you provided me: https://gist.github.com/gardnervickers/7705757653df28dd0e49
I prefer to default a lot of those settings, e.g. fetch size
Would rather just stick them in the base task map - they can be overridden by the opts if needed
For Kafka I think at least these should just be the defaults in the readme:
:kafka/chan-capacity chan-capacity :kafka/fetch-size fetch-size :kafka/empty-read-back-off empty-read-back-off :kafka/commit-interval commit-interval
I thought we were putting the option in the function signature no?
so that IDE’s could pick them up when using the function.
I was thinking the mandatory options should be
ahhhh ok
Yea I like that better
Otherwise you’re creating more boilerplate
yup true
I remember we decided something else around the signature’s too, we would have two arity’s, one where the required args were seperate arguments and another where they were :keys
in the opts map. Do we still want to do it like that?
It’s a little more work but I’m happy to do that
There’s not much downside
I have come around to almost never wanting the non-task-opts version but I can see why people might want it
By non-task-opts version you mean the combined map version?
I think if you go that route, you should have the mandatory args in one signature
And that’s kind of the documentation for what should be in the second version
I’ll point you to what I mean
Thanks, I know we talked about this before but for some reason I’m blanking super hard
No problem. We gotta get it right
Ok, so this one: https://github.com/onyx-platform/onyx-amazon-sqs/blob/master/src/onyx/plugin/tasks/sqs.clj#L50
You take the mandatory args in one signature
Ah gotcha now I remember
awesome thanks
Probably should make a macro for this eventually
Since it’s pretty much the same pattern for every one of them
Do macros give you much though?
I don’t see much that couldn’t be done with a function
Yea you’d end up writing just as much