Fork me on GitHub
#onyx
<
2017-03-14
>
jetmind11:03:28

any idea what this error message means? ":flow/to mapped to :none value must exactly proceed :all entry” I have flow condition like this:

{
   :flow/from :write-es-product
   :flow/to :none
   :flow/action :retry
   :flow/short-circuit? true
   :flow/predicate :r.p.f/conflict?
}

jasonbell11:03:22

If a flow condition specifies :none as its :flow/to, it must come directly behind an :all condition, or first if there is no :all condition. This is because of the semantics of short circuiting.

jasonbell11:03:48

Do you just have the one flow cond?

jetmind11:03:51

No, I have a bunch of flow conditions with short-circuit true before this one. So, basically this should come first

jetmind11:03:13

yeah, this works, thanks

jasonbell11:03:27

👍 glad you got sorted

theblackbox12:03:58

Hello all, I'm working on setting up an onyx system to feed in user events and categorise them accordingly. An implementation of this I've had working in the past using pure clojure simply implements async chans in a pipeline and where I made big wins was transforming data when put to a chan using a transducer. I was hoping to implement something similar with onyx, however, when I set up a transduced channel it doesn't appear to output to the next stage in the workflow. Am I missing something about the input/output nature of chans? my implementation currently uses an "output" chan and I can see my transducer doing what it should, but I'm wondering if I need to add a stage that will read from that write chan in order to pass messages to the next stage?

yonatanel14:03:37

@theblackbox Can you show your workflow?

gardnervickers14:03:38

@theblackbox if you'd like to use your transducers, you need to run the transduction to completion at each stage of your Onyx job on the segments passed in.

gardnervickers14:03:45

Transducers don't make much sense for wiring together computation in a distributed context, other than as an API. You need to serialize/deserialize at task boundaries which takes away the benefit of not using intermediate seq's.

gardnervickers14:03:29

Stateful transducers encapsulate their state making failure recovery impossible too.

theblackbox14:03:18

excellent thanks for the heads up @gardnervickers

theblackbox14:03:40

I'll explore alternative options

gardnervickers14:03:02

What I mean by making failure recover impossible is they make failure recover without replaying everything from the beginning of time impossible. You can still use any type of transducer with Onyx as long as you run the transduction to completion over the segment passed into your task function.