What you are measuring is wrong, let me tell you why


There is one measurement that is commonly accepted when examining the effectiveness of delivery teams. Lead Time.

This is a definition from Wikipiedia.

A lead time is the latency between the initiation and execution of a process. For example, the lead time between the placement of an order and delivery of a new car from a manufacturer may be anywhere from 2 weeks to 6 months.

If software delivery team “LightningFast” is able to give to its customers product X in 1 week while software delivery team “SlowAndSteady” requires 4 weeks to deliver the same product we can state without error that LightningFast is better at software delivery than SlowAndSteady. In fact it is 4x better.

(For the purpose of this article let’s consider the quality of the product delivered by the 2 teams to be equal)

At this point we are considering the starting time to be when a development team has some form of requirements to start working on and finishing time when the bits are deployed to a production environment.

As a business owner you would want to hire team LightningFast and try to avoid SlowAndSteady, I agree with you.

Let’s imagine that team “LightningFast” works for “BestProduct.com” and “SlowAndSteady” works for “WeAreTheBest.com”


The CTO at WeAreTheBest.com would willingly spend a lot of money for either replacing SlowAndSteady with LightningFast or coaching SlowAndSteady to lower lead time and become more like LightningFast.

But the business world is not a software delivery speed contest, it is much bigger than that.

Follow me and you’ll discover that even if you have team LightningFast in your organisation you might have bigger problems to solve.

Let’s expand a little our concept and start looking upstream (left).

Team SlowAndSteady have a very effective way of transforming their roadmap into actionable requirements and are able to complete this specific task for product X in 2 days. Team LightningFast don’t have a lot of skills on this department and spend a lot of time upfront to create the requirements. In the specific, for product X it took them 4 weeks.

So let’s calculate lead time one more time.

LigntningFast => 1 week + 4 weeks = 5 weeks

SteadyAndSlow => 4 weeks + 2 days = 4.4 weeks

Okay, does this mean that SteadyAndSlow are more effective than LightningFast? It would seem so. The starting point for the calculation of the lead time changed completely the judgement we had built over the effectiveness of the delivery teams.

This seems interesting, what’s next?

Next is a different question. The question is what is the lead time that we should be measuring? What’s the lead time that is important to our business?

The answer, unsurprisingly, is the whole. From start to finish.

Now, let’s walk backwards to see where the start is.

Next bit I would add is the time that product X took before it got prioritised and given to the delivery team to work on. In many cases, companies have an Enterprise Portfolio from which the Products get selected from when prioritising.

WeAreTheBest.com (the home of SlowAndSteady) have mastered a very good prioritisation process and priorities are continuously assessed based on market conditions, customer feedback and active monitoring so when a product becomes the highest priority for the company the process signals it clearly and they can start creating a roadmap within 1 week.

BestProduct.com (the home of LightningFast)  are struggling with prioritisation, they are not aware of how their customers use their products and have no intention of using customer insights to make decisions on what to build. They rely on the CEO that is the smartest person in the company to decide what gets prioritised. The CEO got it wrong this time and should have started working on product X last year when  it was added to the enterprise portfolio. It took 1 year for the product to get picked up.

Ok. Now things are interesting.

BestProduct.com => 1 week + 4 weeks + 1 year = 1 year and 5 weeks

WeAreTheBest.com => 4 weeks + 2 days + 1 week = 5.4 weeks

Wow, more than a year difference between the 2 companies, this is incredible!

We could go on and on going back to the moment the idea appeared in that company for the first time and add the time lag between the idea appearing and the corresponding product materialising on the Enterprise Portfolio or even go back further and research when for the first time social conditions emerged a problem to be resolved for customers that will be eventually resolved by product X.

I have seen many enterprises and many delivery teams. Don’t take my word for it, look out there, delivery teams effectiveness is equated to how good they are at doing the Requirements to production transition.

This creates a need for improvement localised to that context. A lot of money is invested and spent to resolve the last bit of the ride, while monstrous inefficiencies slow down delivery in measures orders of magnitude bigger.

the narrow focus on a sub-system is called sub-optimisation and it is a very well known concept in systems thinking and lean but it seems to have flown over the heads of most of the agile community.

Still today, a lot of agile experts focus almost exclusively on technical aspects and are not concerned with resolving the real problems in the system.

Do you want to know more about this?

Do you want to know how to identify the real problems in your system?

Give me a shout and let’s have a chat!



One thought on “What you are measuring is wrong, let me tell you why

  1. I think all the team working on the software from BA to Deployment team should be named as delivery team. So they can work together better and then all the factors will be considered while calculating lead time or any such similar definition where one or other things are missed

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s