The Difficulties of Measuring Engineering

The report is so absurd and naive that it makes no sense to critique it in detail.
- Kent Beck responding to the McKinsey Report

Luckily this was a hollow threat, because a few days later he and fellow blogger Gergely Orosz released a two part blog series critiquing not exactly Mckinsey's report but... any report that tried to put “effort based” metrics at the top of the list for things to track.

The main gripes I heard throughout the multi-page retort were:

1. The easier the measurement can be attributed to an individual the more ripe it is for manipulation.
2. Engineering investments have less predictable outcomes than other business functions (see sales / marketing).
3. Team outcomes can lead to a more united workforce while individual outcomes can be divisive.

What is this uproar really saying about the industry right now?

The enterprise is ready to start treating engineering like other business units.

If this wasn’t apparent enough by the mass layoffs we’ve seen over the last 12-18 months, I’d get your eyes checked. Yet, as exposed by Beck v. McKinsey, we don't have a perfect way to measure engineering… at least not to the same degree that we have for measuring sales, marketing, or other business units.

Why?

The why is simple. Engineering is often unpredictable.

Unpredictability is the thing, as an industry, we have to address if we ever want to climb into the true business function ring.

So how do we enhance predictability and what might happen if a team of world class systems engineers worked tirelessly to build a development platform that made infrastructure, networks, DNS, and deployments standardized and predictable?

Well, that just so happens to be exactly what we’ve spent the last half decade improving at Cycle.

And what have we found during that time?

1. Less can be more.
2. Standardization isn’t optional or even secondary.
3. Self service helps more than you think.

Less Can Be More

The high performing teams I work with don’t skimp on workload. They expect their engineers to deliver on a large breadth of “tickets”, and their ambition isn’t slowing down. However, the top tier of these teams understand that in order to have a large workload, engineers need to be able to stay focused.

How can you create demanding workloads without interrupting engineering focus?

• Reduce the number of things to reason about.
• Reduce configuration options or configuration as a whole.
• Reduce total decision load.

Things to reason about, specifically those that SWE’s shouldn’t have to deeply care about, can be networking, infrastructure, deployment methods, ops interfaces (think API sprawl). Get these things out of the way, either by creating an internal development platform from scratch or using something like Cycle and engineers can spend less time straying from core work. That reduction in stray alone could bump perceived or measured productivity greatly.

Configuration options are an absolute killer. More can go wrong with an overly customizable configuration than could ever go right. You’ll hear this echoed throughout “real talk” with your engineers, they just want something that works.

Decision load can be seductive. The engineers that I work with, both internally and externally, are astonishingly bright. Unfortunately, when someone can quickly picture every possible way to do something, it can cause a huge amount of fatigue and new technology that takes longer to build on top of might seem like a better (more fun) choice. Providing a more structured environment can narrow that load and give engineering teams an easier time moving things forward.

Standardization Isn’t Optional

It’s not optional and it should be enforced at the platform level. This piece has been sooooooo overlooked and yet it has been, for many organizations, the downfall of their DevOps adoption attempt. It’s also been the birthplace of platform engineering. And I’m all for platform engineering, but that still doesn’t necessarily provide a standardized and flexible way to approach DevOps.

I’m confident in saying that Cycle users really get the best piece of the cake here.

And that's because on Cycle, standardization is part of the platform. It’s not a “ok go figure out networking however makes sense for this app”, it's “this is the network you have to work with, make sure you understand how your service is going to interact with other services as well as integrate into this available network”.

Is that really more productive?

Yes, absolutely.

Not only in initial deployment, but also with debugging, reasoning, and architecting. When engineers have major answers to how something will be implemented (every time) they’re able to make much larger strides and when they get used to those edges, it multiplies their productivity.

The Freedom to Self Serve

I’d never suggest you just give engineers the keys to the castle, but self service for basic infrastructure shouldn’t be something that is difficult to provide. If you’re going to measure anything, start measuring how long it takes your engineers to get the things that they need. Again, platform engineering champions have been pushing hard here and for good reason. Being able to spin up the necessary pieces for testing something out quickly shouldn’t be complicated.

Cycle takes this a step further. Because of the standardization and automation I’ve been talking about above, engineers have access to, very near, 1:1 environments to deploy to. This brings an additional layer of validation to any feasibility for new services tested.

Productivity: Objective or Subjective

As we move to push for parity with other business functions, engineering must consider how it can do better with measuring and as we’ve talked about here that should start with understanding the constraints of the system.

Only from this place can we build truly objective measurements of productivity, performance, effectiveness, and efficiency.

DevOps held the torch that went quite ary for many, but the dream is not lost. Whether you decide to use a platform like Cycle to help get all these pieces lined up for your team, or you build an internal development platform from scratch… either way it's essential to get a handle on the environment itself before tracking individual developer productivity.

💡 Interested in trying the Cycle platform? Create your account today! Want to drop in and have a chat with the Cycle team? We'd love to have you join our public Cycle Slack community!