Fork me on GitHub
#onyx
<
2016-02-16
>
robert-stuttaford05:02:41

are checkpoints scoped to onyx/ids?

robert-stuttaford05:02:56

that is, if i switch onyx/id, i’m also effectively switching checkpoints?

lucasbradstreet05:02:26

@robert-stuttaford: by default they are scoped to the job, however checkpoint/key allows them to be scoped to whatever you want

robert-stuttaford05:02:15

ok. so if i specify a checkpoint/key, even if i change onyx/id, it’ll continue to use that checkpoint information?

lucasbradstreet06:02:01

Yes I'm pretty sure

lucasbradstreet06:02:34

The point is to to give you a way to transition between deployments

robert-stuttaford06:02:50

ok. this helps me to understand why i’m struggling. i had this idea that checkpoints were scoped to onyx/id

robert-stuttaford06:02:15

to be honest, i don’t really know that checkpoints are actually helping us, given that our source of truth about what’s been done is our transaction log

robert-stuttaford06:02:46

they’ve certainly made things a lot more difficult to work with, given that two successive test runs have differing behaviour

lucasbradstreet06:02:00

You will still want them because you need a way for the peer to pick back up if it crashes

lucasbradstreet06:02:45

If you're just running tests then use a new checkpoint key each time or force reset the key

robert-stuttaford06:02:33

totally get that. what’s frustrating is when i do resume a checkpoint, the job doesn’t stop when i try to reset the system

robert-stuttaford06:02:10

i’m narrowing in on why, i think.

lucasbradstreet06:02:20

Sounds like a bug?

robert-stuttaford06:02:32

i think i need to learn about how to surgically clear zookeeper data. it’s a black box to me right now

robert-stuttaford06:02:35

ok. so when i ensure a fresh onyx/id and checkpoint, it gets to ‘waiting for peers / enough peers are ready’. if not, it never reaches that state. so something about what we’re doing is borking the ZK state in some way

robert-stuttaford06:02:24

oh, my, gosh. i’m an idiot.

lucasbradstreet06:02:19

Heh, what happened?

robert-stuttaford06:02:56

our test server setup redeploys the app from the build server when you restart the service

robert-stuttaford06:02:59

including the config file

robert-stuttaford06:02:09

so i’d edit config, it’d restart, wiping my edits

lucasbradstreet06:02:44

Oops. That explains that

robert-stuttaford06:02:08

i’ve made checkpoint resetting configurable now. testing...

robert-stuttaford06:02:17

right. now i can finally get back to testing multiple servers together

robert-stuttaford18:02:47

:onyx/max-pending doc on cheatsheet has a typo: "- exhibiting backpressure to upstream message produces.”, should be producers

robert-stuttaford19:02:51

nothing motivates tidying up the place like expecting respected visitors simple_smile

robert-stuttaford19:02:18

refactoring stuff as i write those docs

michaeldrogalis19:02:13

I would still like to go back and make the cheat sheet fancier. Searching was a great idea.

robert-stuttaford19:02:32

one easy way is just a SHOW ALL flag

robert-stuttaford19:02:35

then you can use page search

lucasbradstreet19:02:19

It wouldn’t be that hard to add search anyway

robert-stuttaford19:02:36

i know - i had fun doing exactly that with the codex

robert-stuttaford19:02:39

really need to update this list. so much more good stuff since i last did

michaeldrogalis19:02:01

Hah. Yeah thats exactly the functionality I'd like to see.

michaeldrogalis19:02:02

@robert-stuttaford: I'm gonna be doing a talk at Clojure/west called Inside Onyx. It's a deep dive of how everything actually works. I can give you and your team a preview if you want to demo it.

robert-stuttaford19:02:16

man that’d be awesome! via hangout?

michaeldrogalis19:02:06

Sure. Maybe late March. I need to sit down and write it, I dont think it'll take too long though.

robert-stuttaford19:02:32

that’d be seriously awesome

michaeldrogalis19:02:24

I did a full codebase walkthrough with some people last week. 2.5 hours covered .. I'd say half the core codebase. Its wild remembering how much has gone into making core featureful and stable.

michaeldrogalis19:02:50

Gonna break for lunch, back later