Your software development teams are focused on delivering value and a strong return on investment (ROI). You have to. Gone are the days where you receive a budget without the expectation to validate the investment.
I see the daily struggle within information technology and software development organizations. You are under tremendous stress to prove your teams are worthy of funding.
To quantify that value, you may measure cost per story, or some variation such as cost per story point or iteration — same data; same purpose.
Cost per story is the team’s total cost for an increment divided by the number of stories delivered. The lower the cost per story, presumably the more value delivered for the money invested.
Unlike measuring business value points, cost per story is a tangible, objective measure based on legitimate data. It’s easy to measure, but is it meaningful?
In “You’re Not Delivering the Value You Think You Are,” I noted that the easy measures aren’t always meaningful measures. Teams often measure what seems like a value indicator, but it isn’t. Cost per story falls into this category.
The good news is you’re moving in the right direction — focused on quantifying value and return on investment. Measuring the cost per something can also be valuable. A slight tweak in perspective will help you drive this forward in a meaningful way.
Let’s explore:
- Why measuring ‘cost per’ can be a good measure
- Why cost per story is a meaningless measure
- Alternative measures that reveal value and ROI
Measuring ‘Cost Per’ Something
Knowing the cost per some output can be a crucial business performance management metric. How much it costs to produce an increment of value directly impacts your return on investment.
If you manufacture a product, cost of goods sold directly impacts the sales price or profit margin. If you’re in the transportation business, cost per mile is a vital baseline metric. If you run a data center, cost per square foot measures your utility of the facility. If you process applications/requests, cost per adjudication is a key business metric.
The software you develop supports one or more value streams. Your customers use the software to find, request, and self-manage services. Your organization uses the software to provide service and support to your customers. You want the cost to find, request, self-manage, provide, and support to be as low as possible while maintaining high customer satisfaction. Cost per is a vital metric in this regard.
Many operational cost per metrics support these value streams. Cost per support ticket. Cost per user session. Cost per application intake. Cost per document upload. Cost per GB of data transfer or storage. These metrics directly support the value streams.
Cost per story is not one of them.
Delivering a story has an assumed future benefit. Delivering stories more effectively and efficiently has no value if the intended benefits aren’t realized.
According to a previous Microsoft study, most of our ideas fail to improve the metric they were intended to improve. That means the time and money spent delivering a particular story may not actually improve any business metric.
Cost Per Story Is a Meaningless Measure
This is the heart of the matter — value isn’t in what you do, value is in what is achieved as a result of what you do.
If you’re measuring the cost to produce an increment of value, measuring cost per story first presumes a story is an increment of value. Secondly, it presumes more stories for the money equals more value. Are these assumptions true?
User Stories Aren’t Value
What you do is deliver user stories. User stories do not represent value. They presume to enable value, but value can only be known post-deployment.
Cost per story tells you how many things the team delivers for the money spent. It tells you what you did and at what cost. It’s a productivity measure.
It doesn’t tell you what was achieved as a result of what you did. It doesn’t tell you what achieving that outcome cost — which may not be a one-to-one correlation between story and business outcome.
Cost Per Story Cannot Be Compared Across Teams
Cost per story isn’t an outcome measure. It’s not even an output measure because there’s no common reference for story size and effort.
Think about it. You’ve heard over and over how velocity should never be compared across teams because story points are relative and uniquely sized based on how each team thinks about effort. The same effort could yield different story point sizes across teams. The same story point size across teams could result in different efforts.
If you aren’t supposed to compare story point sizing and team velocity across teams (and you shouldn’t be), why would you expect to be able to compare the cost per story across teams?
What’s the value of the metric when there isn’t a common definition of size and effort?
Stories Are Not Created Equal
Perhaps you concede that story point sizing varies and isn’t a valuable comparison across teams. What about stories as a unit of measure? One story is one story, right? Not necessarily.
Think about cost per mile, for example. A mile is a mile in every company. In every data center, one square foot is one square foot. These units of measure have a common definition and are the same everywhere.
What if you’re measuring the cost per adjudication for an application process? Given a specific application, it’s an organization-wide unit of measure, but each application process inside an organization will be different, much less across other organizations.
User stories aren’t comparable across distinct teams.
Within the same team, when the team learns to size all stories at about the same size, you can count stories without regard for story points. The slight variations in size will even out across the board. However, one team may create stories relative to about two days of work. Another team’s stories may frequently take about four days effort.
What if all your teams sized stories at about the same effort? Now you’re closer to an apples-to-apples comparison. Can you achieve this parity? Does it matter? Cost per story is an output measure, not a value measure.
Measure Impact to Outcomes Instead
Why do your software teams exist? Why do they develop and deploy new code? The code intends to provide a capability to its users so that the users can accomplish something of value.
The value is in what is achieved as a result of what you deliver.
Are you trying to reduce the time it takes to perform an action? First understand how long that action takes and then measure the impact your change makes to that metric. How much did it cost to achieve that efficiency gain?
Are you trying to reduce support calls by increasing self-help capabilities, and adoption? Understand your baseline and then measure the impact your change makes to that objective. How much did it cost to achieve that outcome?
Cost per story may be similar to cycle time in that it’s beneficial to get experiments out quicker with less cost and risk to test assumptions. The less you spend being wrong, the better. But ultimately it’s about what was achieved as a result of what you delivered.
Tie your performance management to the product or organizational strategic objectives. How are you helping achieve those targets? How much are you spending to reduce or increase some metric by 5%. What does that 5% change mean to your business? Was the investment worth it? Is it worth trying to move the metric another 5%?
Final Thoughts
What you do matters. Why you do it matters more. Your software team doesn’t (shouldn’t) exist merely to deliver user stories as cheaply as possible. They exist to help the organization meet its strategic objectives.
Effectively and efficiently delivering stories is meaningless if what you deliver doesn’t help meet the organization’s strategic objectives.
There’s a place for tracking a team’s operational efficiency for continuous improvement. Many of those metrics have no place in performance management systems. That’s a topic for another time.
You’re not the hired-help that should only care about the outputs you produce or system availability. You are an integral part of achieving the desired organizational outcomes. Be accountable to them. Measure your success to them. Show your value through your teams’ return on investment.
Measure what matters and brings about the desired outcomes.
Do you want to bounce some ideas around? Feel free to reach out and engage on the topic.
I write about strategy, business, leadership, product development, and Agile. Challenge what you know. Get short weekly insights in your inbox — sign up on this page.