Fork me on GitHub
#onyx
<
2016-04-08
>
drankard07:04:19

Thanks for pointing me the right direction, docker now takes --shm-size=“” Im using compose though, they are not ready with this update yet.. but its comming in 1.7.

lucasbradstreet07:04:10

@drankard: yeah unfortunately you need the manual script work around + privileged mode until compuse supports it

jeroenvandijk09:04:56

I’ve generated a Onyx template (no customization yet), started a local cluster via docker-compose, submitted a job. I’ve started a dashboard succesfully. I’m figuring out what everything means. I’m seeing a recurring error in the docker logs. Is that expected?

lucasbradstreet09:04:42

Hmm, that’s not good, but it’s probably because the template was created vanilla. Lemme check how it looks

lucasbradstreet09:04:13

Hmm, so I assume you created with:lein new onyx-app my-app-name -- +docker

lucasbradstreet09:04:35

Oh, you must have done both?

lucasbradstreet09:04:50

i.e. +metrics +docker

jeroenvandijk09:04:53

yeah

lein new onyx-app load-test -- +docker +metrics

jeroenvandijk09:04:28

And I followed the README instruction to start the cluster

jeroenvandijk09:04:51

If I look at the dashboard something seems to happen (not sure what exactly :))

lucasbradstreet09:04:52

Can you change the following file:

lucasbradstreet09:04:53

src/my_app_name/tasks/metrics.clj

lucasbradstreet09:04:59

to look more like this?

lucasbradstreet09:04:11

(defn add-metrics
  "Adds throughput and latency metrics to a task"
  ([job task] (add-metrics job task {}))
  ([job task opts]
   (-> job
       (update-in [:lifecycles] conj (merge
                                       {:lifecycle/task :all ; or :task-name for an individual task
                                        :lifecycle/calls :onyx.lifecycle.metrics.metrics/calls
                                        :metrics/buffer-capacity 10000
                                        :metrics/sender-fn :onyx.lifecycle.metrics.timbre/timbre-sender
                                        :lifecycle/doc  "Instruments a task's metrics to timbre"}
                                       opts)))))

lucasbradstreet09:04:17

I think there’s a bug in the template

jeroenvandijk09:04:23

cool, thanks I’ll try

jeroenvandijk09:04:34

I think i don’t have to restart the cluster, right?

lucasbradstreet09:04:12

Alternatively you can just remove this line in src/my_app_name/jobs/meetup_job.clj (= :prod mode) (metrics-behavior/add-metrics :read-lines)))

lucasbradstreet09:04:44

It should be enough to just resubmit the job

leonfs10:04:58

Hey Lucas… Did you have an opportunity to look at the :trigger/fire-all-extents? issue we spoke the other day?

lucasbradstreet10:04:01

Sorry, I’ve been busy with other things. I’ll give it a look tonight

leonfs10:04:51

cool.. thks

jeroenvandijk10:04:18

@lucasbradstreet: Thanks, both approaches worked

lucasbradstreet10:04:00

Thanks @jeroenvandijk. I'll create an issue and we'll get that fixed up asap

jeroenvandijk10:04:12

The funny thing is that before when there was an error I saw more events in the dashboard. I still need to get my head around this simple_smile

lucasbradstreet11:04:38

Mmm you should've at least seen the job killed and the peers reassigned.

jeroenvandijk11:04:28

yeah sorry, the job is running. And i see peers reassigned. I killed the cluster as I missed your comment about resubmitting. Before I saw more activity under jobs “None"

lucasbradstreet11:04:21

Ah. Ok, kinda hard to know what was going on but let me know if you need anything clarified

jeroenvandijk11:04:55

Yeah I think i just need to get some experience before i take more of your time simple_smile

jeroenvandijk14:04:22

The onyx-template generates a 0.8 Onyx application. Is that still the recommended version?

jeroenvandijk14:04:55

I guess not, reading the Onyx readme

michaeldrogalis14:04:27

@jeroenvandijk: You can upgrade to 0.9.0. We didn't get around to upgrading the template yet. It's one of the few pieces not in our automated release pipeline.