This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-11-07
Channels
- # bangalore-clj (1)
- # beginners (255)
- # boot (29)
- # cider (16)
- # cljs-dev (13)
- # cljsrn (6)
- # clojure (200)
- # clojure-berlin (1)
- # clojure-dev (13)
- # clojure-dusseldorf (6)
- # clojure-greece (1)
- # clojure-india (1)
- # clojure-italy (1)
- # clojure-russia (33)
- # clojure-spec (28)
- # clojure-uk (27)
- # clojurescript (47)
- # cursive (32)
- # data-science (3)
- # datascript (1)
- # datomic (40)
- # emacs (39)
- # events (4)
- # fulcro (55)
- # graphql (16)
- # immutant (2)
- # luminus (2)
- # lumo (5)
- # off-topic (142)
- # onyx (50)
- # portkey (1)
- # re-frame (45)
- # reagent (80)
- # remote-jobs (2)
- # ring-swagger (3)
- # rum (9)
- # schema (3)
- # shadow-cljs (184)
- # spacemacs (3)
- # test-check (4)
- # unrepl (2)
- # yada (5)
maybe this is fixed in newer versions (we're on 0.9.15 still), but this error was a little bit hard to get through:
------ Onyx Job Error -----
There was a validation error in your flow-conditions for key :flow/to
{
:flow/from :read-reference
:flow/to :write-size-order
^-- :write-size-order isn't a valid value.
:flow/predicate :raker.reference.job/sizechart?
}
Did you mean: :write-size-order
------
what it tried to say is that :flow/to
should be a vectorSo I've created a WIP pull request: https://github.com/onyx-platform/onyx-http/pull/3 - would love some comments there 🙂
@asolovyov Getting these errors to be as "accurate" as possible to the real problem was tough with Schema. I'd like to rip out the existing error output and replace it with Spec & Expound
Phrase is another candidate if we want to retain the amount of customization we have today.
Either way I'd like to ditch what we have today if anyone can help out with that.
I made the PR against master. There are lots of merge commits now oops
You’re almost definitely right about the sleep
@asolovyov move the resets here above the submit job https://github.com/piranha/onyx-http/blob/0.11.x/test/onyx/plugin/http_output_test.clj#L100
Then you’ll see the data on the output. What’s happening is that the job is being started and the tasks are being initialized before the atoms are setup properly
Everything else looks pretty reasonable aside from the retry handling, where I’m not really sure what the right response is. If aleph has some native way to handle it, that’d be good. Otherwise, you may want to just buffer up the retry data in an atom, and retry everything that’s past due during the write-batch / sync calls. If you do it this way you would want to make sure it only checks the retry map every X ms, since you don’t want to pay the checking cost on every batch.
I would like a more native way to do retries on the outputs now that it’s made more necessary by the ABS fault tolerance model
by "more native" you mean closer to those onyx protocols rather than that loop/recur construct?
I think we’ll have to try some different things in the plugins first though.
documentation is pretty sparse and I have too much focus-switching those days to properly dive into that 😢
Sorry about that
Lemme have a look at the blocking
hah, it's okay, I'm not complaining! I'm rather disappointed at me not understanding everything right away 😄
I am going to fix up the the plugin template soon and really document how each protocol function is supposed to work there
oh I know what the issue is
You have to return true from the write-batch call since you’re done
Since prepare-batch and write-batch are meant to be non-blocking, so you can call something async, and then decide whether to advance to the next state machine state
By making it non-blocking you can do long running stuff and still ensure that your peer is heartbeating etc
Once I made write-batch return true, all that’s left is that you’re not calling ack-fn from process-message
I should probably make prepare-batch and write-batch force a true/false, vs nil
if you push up what you have I can have another look from there
@lucasbradstreet pushed 🙂
Seems like the in flight counter is still getting stuck at 3
Hah yeah
ok, I'm going to sleep - the only thing not working now is I'm trying to put stuff on out-chan
in post-process
and it stucks there, I'm too tired of figuring out those blocking thingies :))
btw, we have custom version of onyx-http with this post-process
thingie so we can feed information about results into our system (stuff like "an email was sent successfully"), I hope it's okay I put it there 🙂
Cool, good progress