This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-07-25
Channels
- # announcements (22)
- # architecture (6)
- # beginners (76)
- # cider (44)
- # clara (6)
- # clj-kondo (34)
- # cljdoc (4)
- # cljs-dev (8)
- # clojure (162)
- # clojure-brasil (4)
- # clojure-dev (32)
- # clojure-europe (3)
- # clojure-italy (16)
- # clojure-japan (4)
- # clojure-nl (3)
- # clojure-poland (1)
- # clojure-spec (25)
- # clojure-uk (45)
- # clojuredesign-podcast (5)
- # clojurescript (91)
- # clojutre (3)
- # core-async (19)
- # core-logic (8)
- # cursive (60)
- # data-science (3)
- # datomic (7)
- # defnpodcast (2)
- # duct (5)
- # emacs (7)
- # fulcro (1)
- # graalvm (9)
- # graphql (1)
- # jackdaw (12)
- # jobs (1)
- # jobs-discuss (1)
- # kaocha (3)
- # leiningen (3)
- # luminus (2)
- # off-topic (44)
- # onyx (17)
- # pathom (6)
- # planck (15)
- # re-frame (1)
- # reagent (13)
- # rum (2)
- # shadow-cljs (43)
- # spacemacs (3)
- # sql (43)
- # tools-deps (3)
- # vim (43)
- # xtdb (13)
I went through all the learn-onyx challenges & have been trying to get the onyx-template working from https://github.com/onyx-platform/onyx-template Just using the local in-memory zookeeper, I figured out how to compile & run Onyx via:
java -cp target/peer.jar myapp.core start-peers 2 -c resources/config.edn
java -cp target/peer.jar myapp.core submit-job "basic-job" -c resources/config.edn
Unfortunately, when I try to actually submit the job it is accepted by the Onyx cluster but never executed. The Onyx log gives off "Job ID 6a1f7400-f5d3-3d49-ebde-487f6964ab8e has been submitted with tenancy ID 1, but received no virtual peers to start its execution."
I got the Onyx Dashboard working and I can see the accepted job in the cluster, but it's indefinitely stalled. The Clojurians Slack logs for the previous time this came up have sadly been lost --
I'm rusty so I can't remember all the details but I believe there is essentially an npeers setting
I'm still a bit green, but I believe so. The job is the default from the onyx-template, which has the following catalog (pulled from the dashboard):
[{:onyx/plugin :onyx.plugin.core-async/input,
:onyx/medium :core.async,
:onyx/batch-timeout 1000,
:onyx/type :input,
:onyx/name :in,
:onyx/max-peers 1,
:onyx/doc "Reads segments from a core.async channel",
:onyx/batch-size 1}
... ;; rest of tasks also have :onyx/max-peers 1
Hmm, I'll take a look at what options I can pass to the first peer.jar
instantiation that starts the in-mem Zookeeper, thanks for the tip
I think that was basically it -- using too small N for number of peers. I started up with more & it took the job. From looking at the dashboard, I suspect it wants at least N >= 3. I think I had the misconception that a sequential 3-task workflow could be executed sequentially with just one peer, but that doesn't seem to be true
Huh. Thanks for the explanation. Glad I have 32gb RAM with the amount of JVMs in play here 😅