Ultimate guide to use metrics to distract your teams and destroy your company

I have had a lot of conversations, with agile people and not, around the topic of measuring success of an agile team. I have heard all sorts of metrics thrown around, from velocity, throughput, number of bugs or lack thereof, and so on.

The fact is that those metrics are completely useless, let me tell you why.

Imagine that your team in a period of 3 months has increased its velocity from 24 to 48. What does that mean? Some people will tell you they are 100% better or even 100% more successful!

I say that they are 100% better at delivering stories (assuming they didn’t game the metrics)

More than likely they work in an organisation that measure success based on the old Budget-Time-Scope paradigm.

Unfortunately in your search for speed you are sub-optimising your system and not achieving the real goal of your company.

A team that bases success on the Budget/Time/Scope triangle I call it BuildEverythingFastAndPray)

What is a successful team?

A team is successful if they help the organisation they serve be successful, regardless of how many story points they deliver.

Let me tell you what a successful team measures.

A successful team measures business outcomes. What are business outcomes? Let me give you some examples:

1) x% increase week to week on downloads of your mobile app
2) y% increase in signups month to month
3) z% reduction of customer support calls month to month

or any similar outcome depending on your context where x,y,z>0


Because an organisation that obtains those outcomes is normally successful.

Even more importantly, the team will continuously monitor how their actions affect the business outcome metrics they have set to achieve so that they might decide to:

  1. Stop writing that feature, we have obtained the result and any further bell and whistle wont give us ROI
  2. Do more of this, the metrics are going in the right direction but not as expected
  3. Stop doing this and do something else, this feature is not producing the results we were expecting
  4. Ah, look at what the customer is doing instead of doing what we thought he would do! Let’s help them do it in an easier way…



Build Measure Learn

Now compare this to delivering all the 1 zillion stories in the fixed scope at the velocity of 48 per sprint.

What’s a successful team for you?



Some quality metrics that helped my teams

Measuring_Tape_Inch+CMI’ve been asked the question “what are the best metrics to improve software quality?” (or similar) a million times, this blog post is a selfish time saver, you are probably reading this because you asked me a similar question and I sent you here.

Firstly, i am not a fan of metrics and I consider a good 99% of the recommended software quality metrics pure rubbish. Having said that there are a few metrics that have helped teams I worked with and these are the ones I will share.

Secondly, metrics should be used to drive change. I believe it is fundamental that the metric tracked is clearly associated to the reason why the metric is tracked so that people don’t focus on the number but on the benefit that observing the number will drive.

Good metric#1: In order to be able to re-factor without worrying about breaking what we have already built we decided to raise the unit test coverage to >95% and measure it. Builds would fail if the metric was not respected.

Good metric#2: In order to reduce code complexity, improve readability and make changes easier, we set a limit and measured the maximum size of each method (15 lines) and the cyclomatic complexity (don’t remember the number but I think it was <10). Builds would fail if the metric was not respected.

Good metric#3: In order to continuously deliver low complexity easily testable units of work and help with predictability we started measuring the full cycle time of user stories from inception to production with the goal of keeping it between 3 and 5 days. When we had user stories that took more than 5 days we retrospected and examined the reasons.

In the 3 cases above, the focus is on the goal, the number is what we think will drive the change and can always be changed.

If people don’t understand why they write unit tests, they will achieve unit test coverage without guaranteeing the ability to refactor, for example by writing fake tests that don’t have assertions. We should never decouple the metric from the reason we are measuring something.

These are the good metrics, for me. If you want to see some of the bad ones, have a look at this article I wrote some time ago on confrontational metrics and delivery teams that don’t give a damn about their customers.