Fork me on GitHub
#jobs-discuss
<
2018-07-13
>
Daniel Hines23:07:13

Question for you smart folks. The big theme behind a lot of my work right now is CI/CD, and I'm trying to figure out what to invest in and how much. In my org, I'm kind of a one-man, skunkswork project guy working with only loose ties to our much larger IT department. I have zero budget, and very limited favors when it comes to opening up ports on the firewall, provisioning hardware, etc. Meanwhile, my direct supervisors have big dreams of building global scale education platforms, so I try to skate toward where they think puck will be and look for the tech that will get them there. In ya'lls more experienced opinion, which will deliver the most value to my folks: set up CircleCI in an hour and be done with it, or take the time and resources to learn and hone a self-hosted CI/CD pipeline with Jenkins, or even something like Jenkins-X on Kubernetes?

jonahbenton02:07:22

Nothing's ever 1-hour-and-done. Here's my methodology. Every technical commitment: * provides some amount of leverage for some sorts of activities * creates some amount of friction for other sorts of activities * incurs some amount of debt- what might have to be undone, both in technical terms and in human habit/process terms, to adopt a different solution with a better leverage/friction profile. * costs some amount of human time for setup and ongoing support, along with dollars My suggestion would be to dig a little deeper to specify what sorts of activities are important for the business and the product (CI/CD of what assets, for what purpose, etc), and then evaluate those options in that context. Make a spreadsheet with products in columns and leverage/friction/debt/cost in the rows, keep it terse, get feedback, and arrive at a defensible story. Then save the spreadsheet in an "Architectural Decision Record" (http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions), to be revisited in the future by you or your successors, to check on the balance of leverage and friction and activities that matter to the business. Two more thoughts about skunkworks: * the skunkworks role is an important one, but it's easy to get out of alignment. Most skunkworks folks are generalists, with a range of skills, and their greatest value is as force multipliers and fast leverage creators. What they most need to avoid are creating solutions that only they understand, turning themselves into bottlenecks. If you are skunkworks, by definition, you're not an owner, you don't have a budget. Everything you stand up needs to not require your maintenance attention. (If you find yourself wanting to be an owner, see the next point) * with skunkworks comes a lot of freedom, and a lot of reliance on personal taste and judgement. In that context, one's personal interests necessarily have an impact. It's important to keep clear the distinction between what's best for the business, and what's most interesting/valuable to you personally. You may be itching to set up and run a k8s cluster, but it is likely that what's best for the business is a CI/CD platform. You want to keep tabs on that gap and maintain an open channel with your supervisors to keep it from growing too large and ensuring that you get to grow in ways that make sense to you, as you help the business.

Daniel Hines12:07:00

Wow, you've nailed my situation exactly - thank you for this insight. Would you be so kind as to elaborate on your statement, "the skunkworks role is an important one"? My position has evolved very organically and informally (which I imagine is how most of these roles come to be); thus, the guiding principles and values are often unclear to me. What exactly is the essence and importance of the skunkworks role when done right? Your statement that they most need to avoid "creating solutions that only they understand" also gives me pause, as I fear I may have trapped myself. Do you mean my colleagues should be able to follow along in the code, and make potential changes/updates, or do you mean they should understand the intent and operation of the solution within the greater business context?

jonahbenton14:07:49

Happy to help. There is a lot to say about the importance of skunkworks and "guiding principles and values" and I'll add- risks- as well. Rather than me doing it poorly- for big picture context, here are a couple of posts from folks who have written eloquently about these dynamics. On the positive side, "the wolf" is the best description I've found of the skunkworks engineer role "in the large": http://randsinrepose.com/archives/the-wolf/. These roles have been around since there has been technical engineering at scale. On the negative side, Rachel Kroll seems to have been a wolf, or what she calls the "roaming fixer" at most of her roles, and has some recent posts that get to the risks. Here's one, but she has many others: https://rachelbythebay.com/w/2018/03/21/next/ What you're doing as a company is making a machine for making money. Formal responsibility for the parts of the machine fall the various people managers, who are owners of their domains, and guide their teams around implicit and explicit dependencies on and responsibilities to other teams. These are teams, not individuals, because of the bus factor. The "essence and importance of the skunkworks role when done right" is- there is one person that can do the work of a team, because they have generalist technical skills, spanning distinct specialities, and also have communication skills, taste, understanding of owner/manager intent, and business value. They become an unusually efficient way of turning manager/owner intent into part of the business machine. That's powerful. The risk in "creating solutions only they understand" is the risk of not being on a team. A skunkworks engineer who produces a piece of the machine without needing a team- yes, they are a force multiplier. But if other things in the machine depend on that piece, and the bus factor on that piece is 1- the skunkworks engineer- then the whole org is actually in a worst place than if the skunkworks engineer wasn't there are all. Does that make sense? So if one is not on a team that has ownership of a piece of the overall machine, one has to be careful to enable other people and teams, without making bus factor 1 systems. Either a) use a 3rd party service b) make sure some other team owns the solution or c) become a manager or lead and get a team.

jonahbenton14:07:09

In terms of trapping oneself- communication is the best strategy. Every situation is unique and distinct, but if you have concerns about being a bus factor 1 in ways that are not widely understood, in a trusting culture it is only beneficial to the business to raise them. There are always bus factor 1s, and every situation has calculated risks.