This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # admin-announcements (10)
- # beginners (17)
- # boot (14)
- # braveandtrue (4)
- # cider (6)
- # cljsrn (38)
- # clojure (232)
- # clojure-austin (1)
- # clojure-belgium (2)
- # clojure-dev (20)
- # clojure-greece (20)
- # clojure-japan (3)
- # clojure-poland (39)
- # clojure-russia (76)
- # clojure-sanfrancisco (6)
- # clojure-uk (4)
- # clojurescript (90)
- # cursive (2)
- # data-science (10)
- # datomic (18)
- # garden (16)
- # hoplon (244)
- # immutant (3)
- # jobs (6)
- # jobs-discuss (2)
- # juxt (1)
- # off-topic (3)
- # om (50)
- # onyx (23)
- # re-frame (5)
- # reagent (36)
- # remote-jobs (11)
- # slack-help (6)
- # spacemacs (2)
- # untangled (46)
Hi all. I’m trying to build a workflow that branches (such that at the end there are two output plugins). From everything I’ve read that should be totally possible. But any time I put a branch into the workflow, my jobs stop such that neither output is eventually hit.
For example, I took this:
and turned it into this:
[[:read-lines :original-wrapper] [:original-wrapper :determine-origin] [:determine-origin :build-row] [:build-row :prepare-rows] [:prepare-rows :write-lines]]
[[:read-lines :original-wrapper] [:original-wrapper :determine-origin] [:determine-origin :build-row] [:build-row :prepare-rows] [:build-row :prepare-rows-2] [:prepare-rows-2 :write-lines] [:prepare-rows :write-lines]]
The first workflow works as expected (the inputs are written to the DB). The second (in which
:prepare-rows-2 is a copy in the catalog of
:prepare-rows) workflow writes nothing to the DB.
Am I misunderstanding how branches work, and/or can
:onyx/type :function not be a branch?
@sirsean: Yep, totally possible and standard usage. Are exceptions in onyx.log?
Okay. You're definitely going to want to figure out the log redirection to see what its says. If it grinds to a halt, you almost definitely have a stracktrace to look at.
@sirsean: Ah, that's telling. Notice that after you submit your job, the peers dont say that they're starting any work.
e.g. your job never started in the first place before it didnt have enough resources.
Oh haha I have # peers at 6, so when I went from 5 to 7 tasks it didn’t work any more?
I’ll try that. (My original workflow with a second output had 6 tasks, this was just an attempt at a simplified version that ended up with 7. Do I need strictly more peers than tasks, or should it work if they’re equal?)
In general it should work if they're equal, with some minor caveats: https://github.com/onyx-platform/learn-onyx/blob/master/src/workshop/workshop_utils.clj#L28
I know it seems dumb that you need to care about how many peers you launch, but it pays off down the road when you need to reason about the performance profile of your program in production.
I suppose that makes sense, you wouldn’t be able to tell what’s going on if one of your peers had to perform multiple tasks and the others didn’t.
For this case, that was the trick. 😄 (Now to go see what’s up with my more complicated plugin, but I’ll make sure
N_PEERS is higher.)