Breaking people's balls since February 1970
September 15, 2014Posted by on
I’ve been asked the question “what are the best metrics to improve software quality?” (or similar) a million times, this blog post is a selfish time saver, you are probably reading this because you asked me a similar question and I sent you here.
Firstly, i am not a fan of metrics and I consider a good 99% of the recommended software quality metrics pure rubbish. Having said that there are a few metrics that have helped teams I worked with and these are the ones I will share.
Secondly, metrics should be used to drive change. I believe it is fundamental that the metric tracked is clearly associated to the reason why the metric is tracked so that people don’t focus on the number but on the benefit that observing the number will drive.
Good metric#1: In order to be able to re-factor without worrying about breaking what we have already built we decided to raise the unit test coverage to >95% and measure it. Builds would fail if the metric was not respected.
Good metric#2: In order to reduce code complexity, improve readability and make changes easier, we set a limit and measured the maximum size of each method (15 lines) and the cyclomatic complexity (don’t remember the number but I think it was <10). Builds would fail if the metric was not respected.
Good metric#3: In order to continuously deliver low complexity easily testable units of work and help with predictability we started measuring the full cycle time of user stories from inception to production with the goal of keeping it between 3 and 5 days. When we had user stories that took more than 5 days we retrospected and examined the reasons.
In the 3 cases above, the focus is on the goal, the number is what we think will drive the change and can always be changed.
If people don’t understand why they write unit tests, they will achieve unit test coverage without guaranteeing the ability to refactor, for example by writing fake tests that don’t have assertions. We should never decouple the metric from the reason we are measuring something.
These are the good metrics, for me. If you want to see some of the bad ones, have a look at this article I wrote some time ago on confrontational metrics and delivery teams that don’t give a damn about their customers.
September 4, 2014Posted by on
Every agile enthusiast will tell you how powerful a self-empowered cross-functional team can be. Once you have one, it brings complete team accountability from product idea to customer support, it naturally grows with continuous improvement, and finds self motivation in innovation and delivery of customer value.
It’s a beautiful and powerful concept; the practical implementation sometimes is not so beautiful and more often than not what you get, is a cross-dysfunctional team.
Let’s have a look at the cross-dysfunctional examples I have experienced.
Developer: “I am a developer I am not meant to test, the testers test!”
Tester: “I don’t need to know anything about how the product is designed, I only care about how the customers use it!”
Business Analyst: “I am not technical, I can’t help you guys!”
Developer: “It works in our environments, it’s operations responsibility to make it work in production”
Tester: ”Listen, it worked in UAT, it must be a configuration issue, or a missing firewall hole and nothing I could have spotted during testing…”
Customer: “Hello! Nothing works here…”
Developer: “The architect told me to do it like this”
Tester: “Feck it, let the Test manager deal with it”
Business Analyst: “I don’t think there is any value in this story, but the Product Owner wants it, so get on with it and develop it!”
Developer: “No point in doing retrospectives, things are always the same”
Tester: “We DON’T HAVE TIME to try new things!”
Business Analyst: “We do it like this, because that’s how we do things here, and that’s it!”
Developer: “My code works perfectly, it’s their system that doesn’t work, who cares”
Tester: “We have 100% coverage, all our tests pass and we have no bugs, if the developers of system X are idiots, there is nothing we can do about it”
Customer: “And still, nothing works here…”
Developer: “Testers are failed programmers, they shouldn’t be called engineers”
Tester: “Developers are only able to produce bugs, the world would be better with more testers and less developers”
Business Analysts: “I don‘t know why I bother talking to testers and developers, they are total idiots”
Do you recognise your team in one of the categories above? What have you done until now to help your team change? Little? Nothing? But you are still bitching about it, aren’t you?
Remember, you are very powerful and can become the change that you want to see.
July 29, 2014Posted by on
1. Asking Microsoft to do the keynote at an Agile conference is like asking Justin Bieber to conduct orchestral music. #fail
2. “Management is the Act of Serving and Supporting Everyone’s Natural Awesomeness” and Woody Zuill in person is a really awesome guy #mobprogramming
3. Moving from being a Test Manager to being a tester in an agile team is far from being a demotion #embracechange
4. Tim Ottinger can play guitar, knows a thing or two about pair programming and stand-up comedy #learnwithfun
5. Changing people’s job title can greatly affect the way they are perceived and accepted – thanks Martin Hynie! #usejobtitleslikeaweapon
6. The 3 amigos is an amazing concept and alligators are softer than you’d think #imissthemalready
July 15, 2014Posted by on
When I speak to people about how it is possible to continuously deliver customer value with near zero issues, I usually get laughed at.
After I tell them that there is nothing to laugh at, people start challenging me on how I integrate with other systems, how I manage defects, how I deal with changing requirements, how I manage regression issues, and how I cope with context switching and other similar issues.
Luckily enough I have empirical data and my answers are based on experience and not some book or model I read about somewhere, so after a while I manage to get people moving from laughing at me to at least starting to be curious.
It has to be said, people become curious but deep inside they still don’t believe me and still think I am a big fool.
How could I blame them? I would have done the same a few years back. I know that people need to have to prove it for themselves to be able to believe it, I have no problem with that.
While I manage to explain my way of dealing with defects, changing requirements, regression, and context switching, until now I haven’t been able to answer the biggest question of them all, the one that every conversation ends up with eventually: how do you deal with extremely complex systems that need to scale up?
I have been thinking about this for a while now and the more I think about it the more I become convinced that Complexity is the Excuse.
Complexity exists when we are not able to prioritise the value to deliver (we don’t know what we really want).
Complexity exist when we are not able to understand and describe the system we are building.
And finally, Complexity is a nice excuse for not doing our job properly as software engineers and having something to blame.
Reduce Complexity and stop taking excuses:
1) You want to deliver customer value, OK. You don’t need to deliver everything on day 1. Sit down with your business partners and identify the highest value added feature and focus on that one. If asked for bells and whistles, say, “we will do it later and only if we really need to”, chances are when you are finished doing the first feature you will have learned that you need something different anyway.
2) When deciding how to implement such feature, look for the simplest and cheapest solution, do NOT future proof. By future proofing you will be adding complexity and guess what? YAGNI. Measure the success of your feature and feel free to change direction, failure is learning.
3) Once you have identified the value to be delivered, make sure you break down its own complexity. If a user story or unit of work or whatever you call it has more than one happy path, then it is too complex, break it down into 2 or more units of work.
4) If you start working on something and you discover it is more complex than you had thought, then stop and break it down to less complex units, if you keep on going saying nothing, you will hide complexity and sooner or later you are bound to mess it up.
Scaling up is the wrong answer to the false complexity question.
Chances are you don’t need to scale up at all: Read 1 and 2 again and again, you will find out that you don’t need as many resources as you thought you would. Scaling up, most of the times, is the easiest, most expensive and laziest approach to fight complexity.
TL;DR: stop blaming complexity when you don’t understand what you are building
May 16, 2014Posted by on
I love going to tech conferences, I love the buzz, the learning, the engagement and the fun. There are a few things you can do to get the best value out of attending a tech conference, keep on reading.
1. Draw a plan and make sure you don’t stick to it
Every self respecting tech conference will have a public program listing all the available sessions, times, locations etc. Well before going you want to read it and identify the talks, workshops, open sessions you want to attend. That’s the easy part.
The measure of how successful your participation to the conference is, is directly proportional to the amount of unplanned sessions you attend. Why do I say that? When you were planning, you were reading a session’s abstract, maybe researching the work of a speaker, but deep inside you needed to trust your intuition believing that what will be at the session is what you expect. The great thing is that once you are at the conference you will be able to ask other people what they think about a speaker, about a specific subject, about other sessions and so on. Use people around you to challenge the assumptions you created while reading the session’s abstracts, discover what you didn’t know you didn’t know with the help of the other conference attendees. Be agile, discover, adapt your plan and act!
2. Ditch your work mates and discover the world
If you are at a tech conference, chances are you are with at least one or two colleagues. Let me ask you a question: do you think you are going to learn more if you stick to your mates and avoid talking to the rest of the people or if you open up and speak as much as possible to the other conference attendees? The answer seems obvious, but you will be amazed by how many little “same company groups” stick together through thick and thin and avoid contact with “the others”. Ask yourself, am I here to learn or to be shy and protected by my team mates? You know the answer, get out there, mix up with people from all over the world, share your experience and and learn from theirs, you will be surprised how many people do great things, yet in a different way from you.
3. Interact with the speakers, they are (mostly) decent human beings and they care
You have been following this guys’ blog posts for years, you’ve read all his books, you have attended 2 other talks he presented in the past and you are inspired by his ideas. Now he is there, having a coffee during a break between sessions and you are there with that “huge question” you have never been able to answer in your mind no matter how many times you reread his books. What do you do?
- Stare in awe and duck if he is looking at you
- Run away before he recognise you from your twitter account picture that has been re-tweeting all his tweets in the last 5 years
- Go there, introduce yourself and ask the “huge question”
Another obvious choice if you want to grow, but make no mistake, most conference attendees do either 1 or 2. I can guarantee that the speaker will be delighted you’re interested, will be flattered by the fact you are familiar with his work and will do everything possible to answer your question and make sure you can put your mind at rest. Here’s the deal, if you do 3 and the speaker treats you badly and embarrasses you because you asked a stupid question, I will buy you a beer.
4. Go to as many open sessions as you can
Be it a “pitch your idea” or a “lean coffee/beer” or a “games session” or any other open session where you can interact with others, please go go go go go! In these sessions you will find people that maybe had an intuition but their idea is still half baked and you can help them discover it, you will be asked to participate and interact with others, you will discover something new that you didn’t know existed and more importantly you will be able to interact with other people in an informal environment. Introduce yourself, ask for names, show interest, talk and listen, listen, listen.
5. Tweet, blog, talk about the interesting session you listened to and the things you learned
Be nice to others, if you learned something valuable, share it with the rest of us, we appreciate it. When tweeting or blogging add the handle of the speaker you are talking about allowing him to interact with you and make your tweet, blog richer and more interesting for everybody.
6. Relax and have the craic
Most conferences are hosted in fancy hotels that have beautiful bars with plenty of tasty fluid. Go to the bar and you will be surprised by the fact that a good 50% of the punters are speakers. For some of them speaking at a conference is their job, talking and interacting with people all day can be stressful, so some of us like to have a drink or two to unwind after all the talks are finished. Join in, bring your pint over to any group of people, speakers, attendees, it doesn’t matter and start talking to them, more than likely you will find out you have something in common with them, after all you are all there for one thing, learning.
December 31, 2013Posted by on
The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.
Here’s an excerpt:
The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 18,000 times in 2013. If it were a concert at Sydney Opera House, it would take about 7 sold-out performances for that many people to see it.
November 12, 2013Posted by on
When I joined PaddyPower in October 2012 I was asked to improve quality without affecting throughput. I studied the teams for a couple of months and I came up with this model based on Gojko Adzic’s Specification By Example and a white paper on ATDD from Elisabeth Hendrickson.
One year after, the bugs are a distant memory and cycle time has been almost halved, so I thought sharing the approach might be useful to somebody out there.
Here we go!
Acceptance Test Driven Development
Acceptance Test Driven Development is about people, communication, collaboration and delivering business value.
ATDD is a software development methodology based on enhanced communication among its actors.
ATDD uses high communications processes and tools to help developers write tests in a business language understandable by all actors. Such tests help developers focus on what to code. The tests can be automated and represent the blueprint of the application being developed. The tests live with the code never getting out of date. Any deviation between tests and application is communicated to the team in the form of a failing test. ATDD sits very well with many agile software development approaches including Scrum and Kanban and with XP engineering practices.
Activities Artifacts and Goal
Acceptance Test Driven Development can be described using 4 activities (Discuss, Distill, Develop, Demo) 4 artifacts (User Story, Examples, Tests, Working Software) and 1 goal (Business Value). Each activity takes as input an existing artifact and produces as output an evolved version of such artifact until the goal is reached as explained in the Business Value Evolution train picture below:
The starting artifact is a User story, during the Discuss phase such artifact is transformed in Examples, in the Distill phase we transform the Examples into Tests, Tests are transformed in to Working Software during the Develop phase and Working software demonstrates Business Value at the Demo phase.
For this process to be successful it is fundamental that none of the activities is performed in isolation (by a single actor) . As many actors as possible should participate to any of the activities, specific actors’ requirement for specific activities are described later in this document.
One Source Of Truth
The first 3 artifacts describe requirements, the fourth is the representation in software of such requirements and the last is the final business goal. The Tests will represent the one point of truth about what it is turned into software and business value. Once the Demo is completed the User story and the Examples can be disposed and the only source of truth will be in the Tests. This is due to the fact that in the transformation the team will have learned a lot more about the deliverable than what it is described in the user story or the Examples. The tests will be updated during all phases to quickly adapt to changes and new discoveries, it is not necessary to retrofit such changes into User Stories or Examples. Tests stay forever and describe the application.
It is imperative that the Tests are at all time visible to all the actors. This means for example that having them buried into source control is not an option because business people don’t use source control.
Actors on the Train
The picture below expands on the Business Value Evolution Train by showing which Actors participate in which activities.
Required Artifact: User Story – We need a business requirement to start from. This doesn’t necessarily need to be in the format of a user story, and it can be expressed in any format. What is needed is a business value to be delivered.
Required Actors: At least 3 members of the Development team should participate, ideally a mix of testers developers and business analysts. Either one between Product Owner or Business Analyst need to participate to this activity.
Format: Meeting with access to a whiteboard
How it works: The Business Analyst has previously developed the user story through his conversations with the Product Owner, he will be able to explain to the other actors the user story’s business value. He will also be able to explain the conditions of satisfaction. Shared understanding of goals will guarantee the real goal is attained and not a consequence of somebody’s assumption
The conditions of satisfactions will be translated into examples, for example if the user story is about giving free delivery for customers that buy 5 books or more the initial examples might be:
5 books free delivery
4 books paid delivery
Questions will be asked and other examples might come out, for example
5 books outside Ireland paid delivery (if the product owner decides to give free delivery only for home users)
4 books and 2 CDs paid delivery
5 books and 1 washing machine free book delivery and paid w/m delivery
and so on…
By the end of the meeting the examples very likely will describe more scenarios than we thought when reading the user story for the first time and by trying to create concrete examples, few questions over the applicability of rules will be raised. If all questions have not been answered and the help of the Product Owner is required, the Business Analyst will get the answers from the Product Owner and have a quick catch up with the other actors to define the last examples. Having the product owner at the Discuss activity can make it more effective as most of the questions will be answered during the activity.
If the questions have unlocked some large uncovered areas and cannot be addressed by the Product Owner, more analysis will be required before we can proceed with the next phase.
Outcome1: Examples – Notice as the examples cover all the aspects of the user story plus those aspects that were not covered in the user story. If we add a 2 liner with the business value to the examples we have an improved version of the user story. The original user story is now out of date and can be expended.
Outcome2: The team have a common understanding of the business value of the user story
Outcome3: The discuss activity might highlight that the user story is too big to be delivered, in this case the activity will produce a list of user stories and the examples for the first one that is taken into development.
Required Artifact: Examples
Required Actors: Two members of the development team need to participate. At least one Developer needs to participate, ideally the developer that will be designing the code for this item. The second member should possibly be a tester or a Business Analyst, if they are not available, a second developer would do.
Format: Pair programming
How it works: Now that we have the examples written down, we can transform them into tests in a format that works with our test automation framework. There are a variety of test automation frameworks that support defining the tests in advance of the implementation including Jbehave and Cucumber. Tests will be written using the Given When Then format. Tests will cover all the examples that were identified as result of the Discuss activity. Extra tests could be added based on the improved understanding of the business goal.
Outcome1: Tests – The Tests cover all the aspects of the examples plus those aspects that were not covered in examples that were uncovered while writing the tests. The tests will also contain the 2 liner with the business value contained in the examples. Again we have an improved version of the examples.
Outcome2: The tests will be written in English so that every actor is able to understand and give feedback. The examples are now out of date and can be expended. The Tests represent the blueprint (documentation) for what we will eventually deliver. The tests will be highly visible and easily accessible at any time.
Required Artifact: Tests
Required Actors: Two Developers need to participate.
Format: Pair programming or Single developer writing code + Code Review
How it works: When implementing the code, the developers are following a test-first approach, they execute the tests and watch them fail. They will write the minimum amount of code required to get the acceptance tests Green. Once the acceptance tests are green he will manually verify that everything hangs together and will call another Developer or a Tester to perform Exploratory Testing. Once exploratory testing is completed and any defects fixed the user story is done and working software is ready to be delivered. While coding the developer might identify scenarios that were not identified earlier and add tests for them. Such tests need to be added to the previous set and shared with the rest of the actors. If the new scenarios identified represent a large amount of work a decision might be made that pushes the new uncovered scenarios to a subsequent user story or we could decide to deliver them.
Outcome: Working software + more comprehensive tests
Required Artifact: Working Software
Required Actors: The Product Owner and 2 members of the development team. Possibly the developer that wrote the software and the other actor that performed the exploratory testing.
Format: Meeting with large monitor
How it works: Before organising a Demo the development team needs to be sure the user story adheres to the definition of done. One very good practice is to create a demo script in which the demo facilitator writes down the steps to follow in order to demonstrate the user story business value to the product owner.
The demo should be an occasion for the development team to be proud of what was delivered.
The product owner will be able to use the Tests to validate all the required functionality has been delivered. At the end of a successful Demo, the product owner will accept the original User Story through the business value demonstrated by running the tests.
Outcome1: Business value
1) Shared understanding
2) Front load issue resolution => reducing defects detected at exploratory testing
3) Tests, for their nature, are specific and unambiguous. Using tests as requirements, developers will create their code based on specific and unambiguous requirements.
4) Developers can avail of the specificity of the requirements and avoid over-engineering
5) The tests are not written in a technical language and can be reviewed/discussed/improved by anybody with some business domain knowledge. Everybody can participate in designing the application by collaborating in defining the tests.
6) The tests are the one source of truth for the application’s behaviours. The acceptance tests live with the code and are executed in CI, they will always be up to date unless the application diverges. The tests are the application documentation. With new functionality come new tests or old tests get adapted to fit the new behaviours.
7) Increased communication builds trust and alliances between the actors
8) Increases transparency; the tests content and results are shared, easily accessible and have high visibility among all actors.
Pleasant Side Effects:
1) Acceptance tests are automated and as such they grow into a comprehensive regression suite
2) Can help Impact analysis. If we want to assess the impact of a change on our application we can simulate that change and verify what pre-existing behaviours it affects by reading the acceptance test results.
3) In the not so uncommon scenario of a re-write, if the original application was written using ATDD it is now possible to recreate the old functionality that needs to be transferred by reusing the Acceptance Tests
Acceptance Tests add no value and can be counter productive when…
1) When we write Acceptance Tests for anything but new deliverables
2) When we write Acceptance Tests to create a regression suite of an existing application
3) When we confuse Acceptance Tests with the tool for writing Acceptance Tests
4) When we write Acceptance Tests that do not give fast feedback
5) When we write Acceptance Tests that are brittle and difficult to maintain
6) When we write Acceptance Tests that contain variables and parameters
7) When we write Acceptance Tests that contain unnecessary detail
8) When we write Acceptance Tests that do not focus on what the change is but on how to exercise such change
9) When we write Acceptance Tests that test the application using the UI when the change is not in the UI
10) When we write Acceptance Tests in isolation and we hide them in source control
11) When we write Acceptance Tests as workflows/scripts
Finally, if you want to learn how to write better acceptance tests have a look at http://mysoftwarequality.wordpress.com/2012/12/14/how-to-transform-bad-acceptance-tests-into-awesome-ones/
November 9, 2013Posted by on
I have been working with agile teams for a few years now and I have used a flavour or another of BDD almost since the very start of my journey. Like everybody I used it badly at the start, misunderstood it and made a mess of it for a good while. Unlike others I have no problems admitting it. Mistake after mistake, fuck up after fuck up I learned a lot of things and today I believe I have developed a decent approach that I will try to improve. Through these years of many failures and small successes one thing clearly emerged for me and it is I believe the true value of BDD.
To me BDD is about communication, collaboration and delivering business value.
The most important part of it? The discussions you have with your team just before you write the tests.
The inherited automation, the derived documentation, the shared knowledge, the reduction in gold plating are side effects and it’s nice to have them, but the real value is in the conversations. The conversations where the team explores new grounds, the conversation where if you want to add value, it doesn’t matter how senior or technical or business savvy you are, what it counts is how much you care about your customer.
November 6, 2013Posted by on
I thought that once I got away from the horrors of waterfall software development it would disappear forever. But no, the menacing SIGNOFF has made it into agile software development teams. I was at Agile Testing Days in Berlin last week and I had an incredible time talking and listening to passionate people. On the other hand, I was surprised to hear how many people still talk in term of SIGNOFFS.
Overheard in Berlin:
“The user stories need to be signed off before you can start working on them”
“…then after our signoff we can deliver the software…”
And so on
This is my definition of a signoff:
“A point in time where my responsibility ends, so for errors up to then you can blame me, after that it’s your fault and I don’t give a rats arse.”
That’s what it is, it is a Cover Your Ass activity. There is no other reason why you would want to signoff on something unless you are trying to cover your own ass and blame somebody else. The fans of the Cover Your Ass manifesto are also the ones that would sing off on anything.
What’s the value of a signoff to our customers?
0 – Zero – Nil
What’s the value to the development team?
Less than zero because signoffs cause finger pointing and blame games
So, why the hell do we do signoffs?
I don’t know.
One thing I know is that the agile manifesto (among other things) says:
Customer collaboration over Contract Negotiation
That in my head translates into
Shared activities over Signoffs
In my team we have started a new trend, it’s called “Signoff by High Five”, it works like this: You get 3 people (possibly the 3 amigos) and you get them working on a task, for example writing a user story. They work together and when
they are finished they go: are we happy? Yes! “High Five!”
At this point you don’t need to have a document with a checklist and a signature of somebody to blame, you simply need 3 people agreeing and one simple “High Five”.
We also do this when we do exploratory testing together, at a certain point we look at each other and we know there is nothing more that we can do to add value then it’s “High Five!”
Sometimes we do that at the end of a demo, the high five there means the story is accepted.
So what happens if there is a problem and a bug appears on software we had wrongly high fived? Not much, we just worry about repairing the issue as soon as possible and delivering the missing value to our customer, no need for arguing over who fucked it up and how we can punish him. What we do is deliver value, that’s what we care about.
November 2, 2013Posted by on
When I know that I don’t know something, it’s a really easy situation, I can study and research to remove the ignorance factor and eventually I will know that thing I didn’t know. This is called first order ignorance, I know what I don’t know.
The second order of ignorance is a bit more tricky, in fact it can be described with “I don’t know what I don’t know”. For example you are estimating a piece of work and what can happen is that your estimates cannot cater for what you don’t know that you don’t know. Being a bit moire specific, say you are estimating writing a new feature that allows your customers to buy tickets for the cinema. You probably know that you need customers, you need a movie, you need a movie theater and a date. What you might not know is that for instance some of the movie theaters are roofless and attendance might depend on whether the sky is full of stars or there is a full blown storm over the screen. Not knowing this you won’t initially estimate the work required for reimbursing the customers that stayed at home during the storm. You will more than likely find out about this either while you are working on the software thanks to an occasional “WTF? moment” or if you are unlucky when you get people knocking at your door because they want their money back. There are quite a lot of things that we can do to try to limit the negative impact of second order ignorance, like doing risk assessment, or breaking down the complexity in many small stories. This won’t guarantee you remove it but will help in many cases. One thing that we always need to keep in mind is that “we don’t know what we don’t know” so deep inside we know that estimates are only there as a placeholder for people that are obsessed with dates, but deep inside we know that they have no real value.
Then there is a third order of ignorance, and I would describe it as “I don’t know that I don’t know what I don’t know”, basically “false confidence”. This can be extremely dangerous because third level ignorant people make a lot of confident statements because they have no clue that there might be something they don’t know, but when it hits, like Homer once said “Forget it Marge, It’s Chinatown!”
If you want to see the original work on the 5 levels of ignorance see http://c2.com/cgi/wiki?OrdersOfIgnorance