Outcome Metrics – Measure What Matters
First published: February 20, 2014
What do you currently measure? Most metrics efforts seem to focus on the throughput or output measures.
Throughput measures tend to cover how fast you get through the work. Examples from Agile teams are things like ‘the number of items completed’ or ‘total story points’ completed during an iteration. Lean teams might use cycle time, or how long it takes a similar-sized item to move from the beginning of the lifecycle to when it is launched.
Output would include Lines of Code, features, or function points completed.
Throughput tells us how fast we go; output tells us how much we deliver. But why are we doing any of this work? What are the problems we are trying to solve? What are the business results we desire?
By measuring throughputs and outputs, we are incentivizing people to deliver more of them. More creates more waste. While some people may believe if they throw enough features out, it may increase the probability of hitting their target, it feels eerily similar to the belief that if you put enough monkeys in a room typing for long enough, they will produce the complete works of Shakespeare. This is not only misguided but is wasteful for our products and our planet. Yes, you heard it right. Think of the amount of resources being used for all those wasted features, let alone all the power they consume as they linger around on servers for years. When the glaciers melt away, you know who to blame.
The one thing very few organizations appear to be measuring is Outcomes. Outcomes are the value we create. They range from wanting to increase revenue for a company to improving the usability and learnability of a product.
Despite all of this, we continue to measure what is easy rather than what is important. I think the reason is that it is not as simple as measuring throughput or output. You have to think, understand and test relentlessly.
Still, the results of doing so can be massive. The first startup I turned Lean in the early 2000s was based on measuring outcomes progressively. We had a simple spreadsheet that tracked what we launched and what impact this had on our outcomes. We would share it with the whole development every month over pizza. This kept the team engaged and focused, able to make better decisions on a daily basis. As a result our revenue skyrocketed.
Now, as we introduce outcome metrics into other organizations, we are finding it an epiphany and far more transformative than any Agile and Lean adoption we have seen. Product Backlogs are thrown away as most of the items have no associated outcomes, and many are pure waste.
Management is on board faster than ever as now they understand what the impact is on their bottom line. They can see how outcome generation reduces risk by testing that you deliver value or invest elsewhere.
To help do this, we use the Mobius navigator. You can download it here. The basic steps are in a slideshare here.