Fork me on GitHub
#onyx
<
2017-11-07
>
asolovyov10:11:43

maybe this is fixed in newer versions (we're on 0.9.15 still), but this error was a little bit hard to get through:

------ Onyx Job Error -----
There was a validation error in your flow-conditions for key :flow/to

{
   :flow/from :read-reference
   :flow/to :write-size-order
    ^-- :write-size-order isn't a valid value.
   :flow/predicate :raker.reference.job/sizechart?
}

Did you mean: :write-size-order
------
what it tried to say is that :flow/to should be a vector

asolovyov14:11:54

So I've created a WIP pull request: https://github.com/onyx-platform/onyx-http/pull/3 - would love some comments there 🙂

michaeldrogalis16:11:05

@asolovyov Getting these errors to be as "accurate" as possible to the real problem was tough with Schema. I'd like to rip out the existing error output and replace it with Spec & Expound

michaeldrogalis16:11:50

Phrase is another candidate if we want to retain the amount of customization we have today.

michaeldrogalis16:11:12

Either way I'd like to ditch what we have today if anyone can help out with that.

asolovyov19:11:27

okay 🙂

lucasbradstreet19:11:02

I made the PR against master. There are lots of merge commits now oops

lucasbradstreet19:11:45

You’re almost definitely right about the sleep

lucasbradstreet19:11:20

Then you’ll see the data on the output. What’s happening is that the job is being started and the tasks are being initialized before the atoms are setup properly

asolovyov19:11:51

I did too much shuffling code today TBH

asolovyov19:11:59

give me a second I'll check this 🙂

lucasbradstreet19:11:15

Everything else looks pretty reasonable aside from the retry handling, where I’m not really sure what the right response is. If aleph has some native way to handle it, that’d be good. Otherwise, you may want to just buffer up the retry data in an atom, and retry everything that’s past due during the write-batch / sync calls. If you do it this way you would want to make sure it only checks the retry map every X ms, since you don’t want to pay the checking cost on every batch.

lucasbradstreet19:11:32

I would like a more native way to do retries on the outputs now that it’s made more necessary by the ABS fault tolerance model

asolovyov19:11:10

by "more native" you mean closer to those onyx protocols rather than that loop/recur construct?

lucasbradstreet19:11:00

I think we’ll have to try some different things in the plugins first though.

asolovyov19:11:04

that would be great, I agree 🙂

asolovyov19:11:24

but I haven't wrapped my head around that stuff yet 😞

asolovyov19:11:57

documentation is pretty sparse and I have too much focus-switching those days to properly dive into that 😢

asolovyov19:11:10

heh, still blocked on feedback-exception

lucasbradstreet19:11:31

Sorry about that

lucasbradstreet19:11:54

Lemme have a look at the blocking

asolovyov19:11:06

hah, it's okay, I'm not complaining! I'm rather disappointed at me not understanding everything right away 😄

lucasbradstreet19:11:01

I am going to fix up the the plugin template soon and really document how each protocol function is supposed to work there

lucasbradstreet19:11:15

oh I know what the issue is

lucasbradstreet19:11:30

You have to return true from the write-batch call since you’re done

lucasbradstreet19:11:03

Since prepare-batch and write-batch are meant to be non-blocking, so you can call something async, and then decide whether to advance to the next state machine state

lucasbradstreet19:11:41

By making it non-blocking you can do long running stuff and still ensure that your peer is heartbeating etc

lucasbradstreet19:11:26

Once I made write-batch return true, all that’s left is that you’re not calling ack-fn from process-message

lucasbradstreet19:11:41

I should probably make prepare-batch and write-batch force a true/false, vs nil

asolovyov19:11:22

somehow it didn't help it seems... weird

asolovyov19:11:54

I mean I return true from write-batches now plus I call ack-fn and still..

lucasbradstreet19:11:28

if you push up what you have I can have another look from there

lucasbradstreet19:11:49

Seems like the in flight counter is still getting stuck at 3

asolovyov19:11:24

like ack-fn is still not called?

asolovyov19:11:32

okay, I can debug that I guess 🙂

asolovyov20:11:51

damn I'm getting real errors now, life is suddenly much nicer 🙂

asolovyov20:11:14

ok, I'm going to sleep - the only thing not working now is I'm trying to put stuff on out-chan in post-process and it stucks there, I'm too tired of figuring out those blocking thingies :)) btw, we have custom version of onyx-http with this post-process thingie so we can feed information about results into our system (stuff like "an email was sent successfully"), I hope it's okay I put it there 🙂

lucasbradstreet20:11:47

Cool, good progress