Fork me on GitHub
#onyx
<
2015-10-15
>
spangler22:10:32

I'm just starting out with Onyx and I have some questions : )

spangler22:10:20

First off, everything (mostly!) makes sense, but I am having trouble figuring out how to get channels in and out of jobs

spangler22:10:13

All the examples I see just (def in-chan (chan capacity)) in the same ns as the lifecycles onyx is using for this job

spangler22:10:29

(or out-chan)

spangler22:10:44

I am wondering a few things about this

spangler22:10:05

First, if I am submitting many jobs, won't all the messages for all the jobs come through that channel? How do I access just the messages for a particular job?

spangler22:10:32

It seems I would need an input and output channel for each job, so I am wondering how this works in practice

gardnervickers23:10:49

When you start a job and inject the channels into the lifecycle with the core.async plugin, those channels will be input’s and outputs for that specific job

gardnervickers23:10:02

If you want to run multiple jobs, you need multiple channels

spangler23:10:04

@gardnervickers Yes, that is what I was hoping

spangler23:10:17

What eludes me is how to set that up

gardnervickers23:10:46

Hold on a second, it’s been a while since I used onyx. Let me verify my assumptions

gardnervickers23:10:21

So do you want to run two sets of channels in the same job or two seperate jobs?

spangler23:10:31

Two separate jobs

spangler23:10:52

Or N separate jobs really

spangler23:10:04

Ideally I would like to pass in the channels to the jobs when I submit it

spangler23:10:36

But the lifecycles are only made of keywords, not references to things

gardnervickers23:10:52

Yea, I remember that was a sticking point for me early on too

gardnervickers23:10:26

essentially you need a lifecycle map for each job

gardnervickers23:10:40

(def lifecycles
  [{:lifecycle/task :in
    :lifecycle/calls :flat-workflow.core/in-calls}
   {:lifecycle/task :in
    :lifecycle/calls :onyx.plugin.core-async/reader-calls}
   {:lifecycle/task :out
    :lifecycle/calls :flat-workflow.core/out-calls}
   {:lifecycle/task :out
    :lifecycle/calls :onyx.plugin.core-async/writer-calls}])

spangler23:10:14

Right, I have that

gardnervickers23:10:21

where in-calls and out-calls are maps that exist somewhere in some namespace on the running JVM.

spangler23:10:38

Totally, but then those maps are static right?

gardnervickers23:10:51

those maps are where you put your ref's

gardnervickers23:10:58

you cant submit the pure ref’s

spangler23:10:01

If they are def'd in some namespace how do I provide different ones for each job?

gardnervickers23:10:46

you’d have to define maps for each job

gardnervickers23:10:10

so like in the sample, where

(def in-calls
  {:lifecycle/before-task-start inject-in-ch})

(def out-calls
  {:lifecycle/before-task-start inject-out-ch})
you have that, you’d have to make two of them

gardnervickers23:10:29

in-calls1 and in-calls2

gardnervickers23:10:54

then you have to create a seperate lifecycles map

(def lifecycles
  [{:lifecycle/task :in
    :lifecycle/calls :flat-workflow.core/in-calls}
   {:lifecycle/task :in
    :lifecycle/calls :onyx.plugin.core-async/reader-calls}
   {:lifecycle/task :out
    :lifecycle/calls :flat-workflow.core/out-calls}
   {:lifecycle/task :out
    :lifecycle/calls :onyx.plugin.core-async/writer-calls}])
for each job