The concept of success in software delivery has traditionally been associated with respecting cost (budget), scope and time. This concept is generally known as the project management triangle.
The decision on whether delivering the product and the size of the triangle depends also on another variable that is the projected revenue that the product will create for the company.
Agile and lean product delivery teams have pushed the boundaries and in the process questioned the validity of the approach.
The 3 main issues I see with the above are:
- Lack of learning: it assumes that there are no significant unknowns at the very beginning when scope, budget, time and projected revenue are set. Any deviation from the plan is seen as a failure rather than learning.
- Lack of customer focus: no consideration is given to customer feedback
- All products are the same: it does not separate contextually different products so date driven products that have a hard link to a delivery date are treated the same way as ones that are “simply urgent” but have no hard relationship with a specific date.
When we measure success based on the iron triangle and the projected revenue we risk of missing or straight out discouraging some extremely good behaviours.
Example #1: If we deliver a solution to the problem (in a form of a product) in half the time because we have helped identify the most valuable features, this doesn’t clearly stand out as a win against having spent within budget, delivered in time and to the full scope.
Example #2: If through customer testing and experimentation we realise early that the product is not going to deliver on promises and we decide to stop development, this is not recognised as being better than spending the full time delivering the product that eventually will fall short on expectations. Actually, the first behaviour is normally seen as a full blown failure while the second as a success.
Example #3: The same product could be delivered with great or terrible user experience for the customer but the measure above won’t spot any difference.
If we have learned something from the last 20 years of product development is that customer focus, active learning, discovery and data driven decisions have been key to successful products.
The iron triangle clearly does not encourage discovery and learning, what measure can we use that will make sure our teams feel empowered and will be rewarded for continuously learning about their customers and the solution they are delivering?
mysoftwarequality 
Thanks for this nice article. examples at the end are really very nice and sums up the article.
I do agree delivering with quality is not considered success but other parameters should also be in control to really call something a successful delivery.
The so-called ‘iron triangle’ (scope/time/budget) is a fallacy, well, let’s call it an excuse. The time/money budgets often are sufficient to deliver the right scope.
I don’t get it.
For most companies, there is one measurement – is the project profitable as it should be? Sure, the engineering is only a part of it, but that’s life – you can’t have a successful project without it being sold.
And yes – it is hard to measure (since what does it mean “as it should be”?).
Also, is the triangle posing criteria for success? Usually I see such triangles as “you can control two out of the three” (usually it’s “time\cost\whatever your goal is”). Criteria for success, besides the bottom line, can be customer satisfaction, valuable by-products (if we take your 2nd example, I think it will be fair to call both projects a failure, but if the case where the project was cancelled had a positive impact on the company reputation for being fair and professional – it could be deemed as a success).
For non profit organizations, the question is similar, but involves more difficult measurements – is the project achieving it’s goal, with the expected investment? If we create a game to teach programming (as there are so many these days), I will probably try to measure:
1) Does a player with no prior knowledge gains sufficient programming skills by the end of the game?
2) Is the game being played by enough people considering the marketing efforts?
3) Are the abandon rates (people that start to play and leave early) acceptable?
You can’t measure a project without knowing what are its intended goals.