Fork me on GitHub
#onyx
<
2016-06-14
>
joshg00:06:17

Choco is really neat. I played with it awhile back to pick fantasy basketball teams for a friend (using constrained knapsack): https://gist.github.com/hadronzoo/7916c32c0d434c9e7dc5#file-core-clj

joshg00:06:33

Probabilistic models are much better for the fantasy basketball problem, but it was still fun.

drewverlee01:06:15

@lanejo01: > any tips on getting started in distributed systems Maybe start with the CAP theorem and its critique by Martin Kleppmann. I was super lucky to have my first distributed systems talk be by Kleppmann talking about the CAP theory at Strangeloop 2015.

Joe Lane01:06:10

Watching it right now ๐Ÿ™‚ Thanks @drewverlee !

michaeldrogalis15:06:41

We're putting together a self-help guide that has a series of questions that you can answer to diagnose common problems with your Onyx program. Does anyone have thoughts on whether this would be best as a static page with links to jump around between, or something more visual like a flow chart?

devth15:06:18

Static page. Scannable and searchable.

michaeldrogalis15:06:45

Been leaning in that direction. I imagine anyone using it will eventually want to read everything in its entirety and not need to click around to navigate.

michaeldrogalis15:06:06

Oh, I should mention that the reason a flow chart might be interesting is that some of the questions to determine what the problem is are several deep. e.g. "Is X true?", yes -> "Is Y true?"

devth15:06:53

Sounds possibly interesting but hard to reason about unless I considered some of the actual flows. I wonder if you could encode some of that in a doc.

michaeldrogalis15:06:59

I think in a doc it would look like:

michaeldrogalis15:06:28

What's the state of X?
- [It is A](link-1)
- [It is B](link-2)
...

michaeldrogalis15:06:40

Where link 1/2 are anchors into the same doc

devth15:06:50

๐Ÿ‘

devth15:06:37

Looking at the onyx dashboard (very cool!) - one of my jobs shows blank under "Host Allocation". Does that mean there are no peers available to do the work?

michaeldrogalis15:06:38

@devn: I think so. Can I see a screenshot?

devth15:06:53

I think I had submitted that job with NPEERS=4

michaeldrogalis15:06:00

It says its running, so perhaps its a bug in the dash. o-o

devth15:06:14

I went back and re-submitted with NPEERS=3 and the job ran, but the old one continues to spin

michaeldrogalis15:06:30

You should kill off your old one.

devth15:06:33

(don't fully understand what the significance of NPEERS is, and why more than enough would casue it to not run)

devth15:06:52

Ok. Don't see a way to kill jobs in the dashboard.

devth16:06:00

I need to ssh in and kill from a local repl or something?

michaeldrogalis16:06:04

Check out the scheduler chapter of the user guide to learn about that, btw.

devth16:06:10

Thanks will do

michaeldrogalis16:06:13

onyx.api/kill-job on the job ID

lucasbradstreet16:06:51

I thought we left kill-job in the dashboard, and just ditched restart-job. Itโ€™d be good to add kill-job back in at least. Anyway, yes onyx.api/kill-job

devth16:06:51

Can I do that from a local repl if I point it at the prod zookeeper? Or what's the normal workflow for submitting/restarting/killing jobs?

michaeldrogalis16:06:16

Yes to the first question. Latter part depends on your set up, we've done it a few different ways. Depends on the customer

michaeldrogalis16:06:32

This might be helpful, too. See the Clojure part and HTTP server below: https://github.com/onyx-platform/lib-onyx#replica-queries

michaeldrogalis16:06:09

Just another way to see what's going on.

michaeldrogalis16:06:23

It would be nice to have a cohesive command line tool to do submission/restarts/job killing. I think that needs to come out of the community organically though.

devth16:06:33

Got it. I'm assuming a CLI would hit the endpoints provided by

(require '[lib-onyx.lib-onyx.replica-query-server :as rqs])

(def server-port 3000)

(def server (rqs/start-replica-query-server peer-config server-port)

michaeldrogalis16:06:44

Yep yep ๐Ÿ™‚

gardnervickers21:06:09

@devth: Iโ€™m working on a kubernetes example for onyx-twitter-sample and just remembered how to handle the shm issue,

- name: aeronshm
  emptyDir:
    medium: โ€œMemoryโ€
will create a volume as a section of the hosts memory

devth21:06:14

gardnervickers: nice, thanks!