I took my first steps in technology over 20 years ago. If we exclude the year off I took in 2006 I have always been employed in many different companies and I can say with certainty that I worked on a “shitload of products” and I use that specific term with purpose, read on to find out.
During all these years, being somebody that loves learning, I ended up working as system analyst, developer, tester, business analyst, manager, leader, coach, change agent, plus some other short term hats.
If I exclude the last 6-7 years where I had the skills and the ability to influence decisions I can go back and say for sure that the products that were initially envisioned, before any customer feedback was used, can be called a shitload of useless stuff and one or two good ideas that resolve real customer problems.
Very often the product envisioned was very similar to the product that we delivered.
Am I saying that in my first 13 years I worked I mainly produced waste?
Pretty much YES.
Another thing I remember in those early years was that I was never in a team where we could say, oh thank god we are busy but it’s not too bad, we can do our work, go home and have a balanced life. Invariantly there was pressure. We need all this by that date, come on! Work faster!
To me, it always felt like we were told by Dilbert’s boss that if we shove some more paper in the printer it will print faster. Also, when we were not busy, then managers will fill our capacity with a new shitload of useless products created for the purpose to make people sweat, very often no thought on the customer whatsoever.
Am I saying that trying to deliver fixed scope, fixed date shitloads of products creates big problems to the workers that build them?
YES, not even the pretty much is needed this time.
Another thing that I have noticed through the years is that without exception, companies older than 3 years have already built a shitload of products that are now impeding their ability to respond to change and survive. We will call this “shitload of legacy systems”.
This specific shitload is used as the excuse for not being able to change, as if saying “yes sure, we can’t compete with the market and we will die soon, but it’s not our fault it’s the fault of the legacy system (that by the way we built)
Am I saying that the shitload of products are also causing the slow death of the companies that created it in the first place?
Next time you start a product, think twice before rewarding people for the delivery of all the scope, in time.
I have been helping organisations deliver products that matter to their customer as soon as possible, I am not in the business of delivering projects or shitloads of products.
I am researching new ways of demonstrating THE VALUE in MONEYof the “non built shitload of products and features”, if you are interested, let’s do this together.
Previously on A Knowledge Worker’s Tale: A large bottleneck appeared in the exploratory test column of team X’s kanban board, but only Rick and Jane, the testers, seemed to give a damn about it working hard to unblock it. Gus saw the problem asked the team to take off their headphones, go to a room and get instructions from the testers on how to unblock the flow of work.
And the story continues…
Team Xers reappeared in their open space with renewed energy. Rick and Jane had explained what they needed to get their work unblocked and soon people started working in pairs. Rick and Peter were debugging together an issue that seemed difficult to track with the assistance of Fritz the business analyst that seemed positive he remembered something important that could help them. Jane was showing to Mike and Jimmy how, what she thought was the correct customer journey, was not what Zach the customer thought it was.
It was great to watch, a group of skilled professionals working in groups, using all their skills and knowledge to troubleshoot a complex problem.
As expected, the pairing and collaboration gave its fruits, defects were getting fixed and retested and the bottleneck was reducing fast. Flow was being reestablished thanks to the team swarming and focusing on the biggest problem in the system, it was great to observe.
Once things got back to normal Mike called the team to the board and said “Folks, it has taken only 3 hours to fix that bottleneck that had basically stopped our flow for two days, I think we have learned a very big lesson today and the lesson is that when we see a bottleneck, we all have to chip in and help unblock it!”. The other guys nodded with satisfied faces finally seeing their work getting to completion again. “Well done all!” Rick the tester added “without your help, we would still be where we were 3 hours ago, collaborating has been fantastic” and the lads gave themselves a small round of applause.
“Let’s go back to work!” said Mike.
“Hang on, hang on” intervened Gus, that had been lurking from distance. “It is great that you guys found a way to reestablish flow, I was delighted to see how you collaborated and forgot about your roles for the good of the customer, I say well done for that”. He continued “The fact is, if we go back to work without addressing the problem that created the bottleneck in the first place, it will be only a matter of time before it happens again, don’t you agree? Now that we have it fresh in mind, I suggest we try to understand the real problem that created the bottleneck, let’s do a quick retro, who’s with me?” The team nodded and followed Gus in the retrospective room.
“So WHY do you guys think the bottleneck formed at exploratory testing?” asked Gus. Immediately Peter jumped in “Because we need more testers!” to which some of the other team members nodded in unison. He added, “Rick and Jane are always super busy, they never have a minute free and at the same time they are always behind with work, the guys need help, we need more testers!”
“Adding more testers is a potential solution, but we haven’t identified the problem yet” said Gus. “I am going to ask a tester now, Jane, WHY do you think the bottleneck formed at exploratory testing?” Jane, slightly caught by surprise turns to Rick, mutters something, then says “Because exploratory testing was taking too long”. “Ok this sounds like a reasonable problem, and WHY was it taking too long?” added Gus. “Well, running tests would have been fast, but we found a lot of defects that needed fixing and that created a never ending back and forth with the developers and Zach” added Jane.
“Very good Jane, thank you, we are getting somewhere now” said Gus, and continued “so it appears we had too many defects in the stories you were testing, so WHY were we having so many defects?”. A long silence ensued.
Peter broke the silence and said “For my user story, it turns out I didn’t understand clearly what Zach (the customer) wanted, so I had to change it four times”. Jimmy added “Yes, the same for me, this feature is complex and the work me and Mike were doing had to be changed many times, every time Jane or Rick would find something wrong because they’d ask Zach and he changed his mind, yet again”
Gus smiled and went “Oh Oh Oh, I think we might be getting somewhere here folks, what do you think Mike?” Mike was deep in thought, head down. He stood up and said loudly “I get it! It looks like a communication problem early in the process presented itself as a visible problem in exploratory testing. It’s got nothing to do with testing!” He was truly excited by the revelation, then he added “The root cause of the problem is the bad communication between the team and the customer and the symptom shows up in exploratory testing.”
Gus said “That’s an excellent observation Mike, do you guys think that if you clarified your doubts with the customer before you started working you would have saved the time you have spent reworking?” Everybody in unison went “Yeah!”. Well, now we know what the real problem is, I’ll chat to you tomorrow morning at the stand-up and we’ll find a solution together, it’s getting late now. Great work everybody and well done for getting to the root of the problem with only four why’s, I normally need five…”
Gus stood up to leave but stopped mid-track and asked “Guys, so do we still think we need more testers?” Peter was first to answer “No Gus, you’re right that was a knee-jerk reaction, thinking about it ,we need more clarity, not testers”
Asking why enough times to get to the root cause of a problem is a great technique, thought Gus while leaving the office and heading to the pub.
A defect is anything that threatens the value of the product.
Before we start, let’s agree that:
we don’t want defects that threaten the value of our product
we want to give our customers as much value as possible at all times.
If you don’t agree with 1 and 2 then, don’t waste your time and stop reading now.
Testers are normally associated with finding defects. Some testers get very protective with the defects they find and some developers can be very defensive about the defects they wrote. Customers don’t like defects, developers don’t like defects, product managers don’t like defects, let’s be honest, nobody likes defects besides some testers.
Why would that be? The reason is that the focus of a lot of testers is on detecting defects, and that’s what they get paid for in a lot of organisations. If you are a tester and love your defects, you might find this article disturbing, if you decide to proceed, do so at your own peril.
Defects are waste
Let’s be clear from the start: defects are waste. Waste of time in designing defective products, waste of time in coding defective routines, waste of time in detecting them, waste of time in fixing them, waste of time in re-checking them. Even writing this sentence took a good while, now think how much time it takes you to produce, detect, fix, recheck defects.
Our industry has developed a defect coping mechanism that we call defect management. It is based on a workflow of detecting => fixing => retesting. Throughout the years it has become best practice (sic) to have defect management tools and to log and track defects. Defect management approaches are generally cumbersome, slow, costly and tend to annoy people no matter whether you are a tester that gets his defect rejected, you are a developer that gets a by design feature flagged as defect, or a product manager that needs to spend time prioritising, charting and trending waste.
Another dangerous characteristic of defects is that they can be easily counted and you will always find a pointy haired manager that decides he is going to shed light on the health of his product and on the efficiency of his team by counting and drawing colorful waste charts.
But, if we agree that defects are waste, why are we logging and tracking waste, creating waste charts seems even more ridiculous, wouldn’t it be easier to try to prevent them?
Oh, if only we could write the right thing first and reduce the number of defects we produce! I say we can, be patient and read on.
Software development teams have found many ways of being creative playing with defects, see some examples below.
Example 1:Reward waste
Some years back I was working on a business critical project in one of 5 scrum teams . Let me clarify first, that our scrum implementation was at best poor, we didn’t release every sprint and our definition of done was questionable.
Close to an important release, we found ourselves in a situation where we needed to fix a lot of defects before going into production. We had 2 weeks and our teams had collectively around 100 defects to go through. Our CTO was very supportive of the defect killing initiative and he was eager to deliver with zero defects. He put in place a plan that included free food all day and night and some pampering for the developers that needed to focus 100% on defect resolution. Then he decided to give a prize to the team that would fix the highest amount of defects.
I remember feeling frightened of the possible future consequences of this reward. I spoke to the CTO and told him that I would have liked more a prize for the team that produced the lowest amount of defects rather than the one that fixed the most. Our CTO was a smart guy and understood the value proposition of my objection, he changed his approach and spoke to the teams on how not introducing defects in the first place is much more efficient than fixing them after they have been coded. Soon after the release, we started applying an approach that focussed on preventing defects rather than fixating on detection. We never had the problem of fixing 100 bugs in 2 weeks again.
Example 2:Defects metrics
In my previous waterfall life, I remember when management introduced a performance metric directly linked to defects. Testers were to be judged on the Defect Detection Index calculated as (Number of Defects detected during testing / Total number of Defects detected including production)*100. An index lower than 90 would mean nobody in the test team would get a bonus. Developers were individually judged on the number of defects found in their code by the testers and business analysts were individually judged on the number of defects found by the testers in their requirements.
Welcome to the battlefield!
The bug prioritisation meetings were battles where development managers argued any bug was a missed requirement, product managers argued that every bug was a coding error or a tester misunderstanding and the test lead (me) was simply shouted at and criticised for allowing his testers to go beyond the requirements and make use of their intellectual functions outside a scripted validation routine.
Going to that meeting was a nightmare, people completely forgot about our customers and simply wanted to get their metrics right. The amount of time we wasted arguing and defending our bonuses was astonishing. Our customers were normally unhappy because instead of focusing on value delivery we focussed on playing with defects, what a bunch of losers we were!
Our customers were very unhappy.
Example 3:Defects as non-conformance to requirements
In the same environment as Example 2, testers, in order to keep their Defect Detection Index high used to raise large amounts of minor or non-significant “defects” that were in reality non-conformance to requirements. Funnily enough such non-conformances were generally improvements.
Testers didn’t care if they were requirements, code defects or even improvements, to them they were money, so they opened them. Improvements were filed as defects as they were in non-conformance to requirements. In most of the cases, these were considered to be low severity and hence low priority defects to make the testers happy and had to be filed, reviewed, prioritised and used in trends, metrics and other useless calculations.
This activity could easily take 30% of the tester time. Such defects would not only take testers’s time, but would also affect developers, product managers, business analysts and eventually clutter the defect management tool.
Waste that creates waste, exponentially, how wonderful.
Example 4:Defect charts, trends and other utter nonsense
Every week I had to prepare defect charts for management. These were extracted from our monstrous defect management tool and presented in brightly coloured useless charts. My manager got so excited at the prospect of producing useless information that she started a pet project to create charts that were more colourful than the ones I presented. She used 2 developers for 6 weeks to create this thing that was meant to wow the senior executives.
In the process of defining the requirements for wowing the big guys, she introduced a few new even more useless charts and consolidated it into an aggregating dashboard. She called it the product quality health dashboard, I secretly called it the dump.
Nobody gave a damn about the dashboard, nobody used the numbers for any reason, nobody cared that they could configure it, but my boss was extremely proud of it. A legend says that she got a big raise because of it. If you play with rubbish, then you will start measuring rubbish and eventually you will end up doing data analysis and showing a consolidated view of the rubbish you store in your code.
How can we avoid this?
1. Focus on defect prevention
Many development teams focus on delivering features fast with little consideration for defect prevention. The theory is that testers (whose time is sometimes less expensive than developers) will find the defects that will be fixed later. This approach represents a false economy; rework disrupts developers activities and harms the flow of value being delivered. There are many approaches available to development teams to reduce the amount of rework needed.
Do you want to prevent defects? You can try any combination of the below:
With BDD/ATDD/Specification By Example or other test first approach, delivery teams test product owners assumptions through conversations and are more likely to produce the right feature the first time.
The ability to have fast feedback loops also allows for early removal of defects, automated unit and integration tests can help developers quickly identify potential issues and remove them before they get embedded into a feature.
Tight collaboration between business and delivery teams helps teams be aligned with their real business goal and reduce the amount of unnecessary features. This means less code and as a consequence less defects. Because, your best piece of code is the one you won’t have to write.
Reducing complexity is very powerful in preventing defects, if we are able to break down a complex problem in many simple problems we are likely to reduce the amount of defects we introduce. Simple problems have simple solutions and simple solutions have less defects than complex ones.
Good coding standards like for example limiting the length of a method to a low number of lines, setting limits on cyclomatic complexity, applying good naming conventions to help readability also have a positive impact on the number of defects produced
Code reviews and pair programming greatly help reduce defects
Refactoring at all times also reduces defects in the long run
Moral of the story: If you don’t write defects, you will not have to fix them.
2. Fix defects immediately and burn defect management tools
If like me years back, you are getting tired of filing, categorising, discussing, reporting, ordering defects I have a very quick solution. Fix the defects as soon as you find them.
It is normal for a developer to fix a defect he finds in the code he is writing as soon as he finds it without having to log it, but as soon as the defect is found by a different individual (a tester for example) then apparently we need to start a strict logging process. Why? No idea really. People sometimes say: “if you don’t do root cause analysis you don’t know what you are doing, hence you need to file the defects”, but in reality nobody stops you from doing root cause analysis when you find the defect if you really want.
What I am suggesting is that whoever finds a bug walks to a developer responsible for the code and has a conversation. The consequence of that conversation (that in some cases can involve also a product owner) should be let’s fix it now or let’s forget about it forever.
Fixing it now, means normally that the developer is fresh on the specific code that needs to be fixed, surely fresher than in 4 weeks, when he won’t even remember he ever wrote that code. Fixing it now means that the issue is gone and we don’t have to worry about it any longer, our customer will be thankful.
Forgetting about it forever means that it is not an issue worth fixing, probably it doesn’t threaten the value of the project and the customer won’t care if we don’t fix it. Forgetting about it forever also means that we won’t carry a stinky dead fish in a defect management tool. We won’t have to waste time re-discussing the same dead fish forever in the future and our customers are happy we are not wasting time but working on new features. If you decide to fix it, I’d also recommend you write an automatic test for it, this will make sure that if the issue happens again you’ll know straight away.
I have encountered huge scepticism when suggesting to burn defect management tools and fix just in time. Only very few seem to think this is possible. As a matter of fact all my teams were able to do this for the last 6 years and nobody ever said, “I miss Jira and the beautiful bug charts”.
Obviously this approach is better suited for co located development teams, I haven’t tried it yet with a geographically distributed team, I suggest you give it a try and let me know how it goes.
Playing with defects waste index:
Epidemic: 90% – The only places that don’t file and manage defects I have ever encountered are the places where I have worked and have changed the process. In the last couple of years, I have heard of two other places where they do something similar but that’s just about it. The world seems to have a great time in wasting money filing, categorising, reporting, trending waste.
Damaging: 100% – Using defects for people appraisal is one of the worst practices I have ever experienced in my long career, the damage can be immense. The customer becomes irrelevant and people focus on gaming the system to their benefit. Logging and managing defects is extremely wasteful as well, it requires time, energy and can among other things, endanger relationships between testers and developers. Trending and deducting release dates from defect density is plain idiotic, when with a little attention to defect prevention defects would be so rare that trends would not exist.
Resistant: 90% – I had to leave one company because I dared doubt the defect management gospel and like an heretic I was virtually burned at the stake. In the second company I tried to remove defect management tools I was successful after 2 years of trying, quite resistant. The third one is the one where people were happy to experiment and as soon as they saw how much waste we were removing it quickly became the new rule. I have had numerous discussions with people on the subject and the general position is that defect management must be done through a tool and following a rigid process.
I have had a lot of conversations, with agile people and not, around the topic of measuring success of an agile team. I have heard all sorts of metrics thrown around, from velocity, throughput, number of bugs or lack thereof, and so on.
The fact is that those metrics are completely useless, let me tell you why.
Imagine that your team in a period of 3 months has increased its velocity from 24 to 48. What does that mean? Some people will tell you they are 100% better or even 100% more successful!
I say that they are 100% better at delivering stories (assuming they didn’t game the metrics)
More than likely they work in an organisation that measure success based on the old Budget-Time-Scope paradigm.
Unfortunately in your search for speed you are sub-optimising your system and not achieving the real goal of your company.
What is a successful team?
A team is successful if they help the organisation they serve be successful, regardless of how many story points they deliver.
Let me tell you what a successful team measures.
A successful team measures business outcomes. What are business outcomes? Let me give you some examples:
1) x% increase week to week on downloads of your mobile app
2) y% increase in signups month to month
3) z% reduction of customer support calls month to month
or any similar outcome depending on your context where x,y,z>0
Because an organisation that obtains those outcomes is normally successful.
Even more importantly, the team will continuously monitor how their actions affect the business outcome metrics they have set to achieve so that they might decide to:
Stop writing that feature, we have obtained the result and any further bell and whistle wont give us ROI
Do more of this, the metrics are going in the right direction but not as expected
Stop doing this and do something else, this feature is not producing the results we were expecting
Ah, look at what the customer is doing instead of doing what we thought he would do! Let’s help them do it in an easier way…
Now compare this to delivering all the 1 zillion stories in the fixed scope at the velocity of 48 per sprint.
If you don’t want to read the full article here’s the TLDR: Can we embed the agile values in the format of a beginners talk so that people will learn by breathing them rather than hearing about them?
I received quite a lot of encouragement from a lot of people. I love the agile/lean community, thank you folks, you are incredible!
I got some great suggestions from Patrick Steyaert that recommended looking into Lean Coffee and Fish Bowl.
Both formats are highly participative and pretty much agendaless and gave me a great point to start at.
My goals in priority order:
Do not bore an audience that for the first time hears about agile. Don’t push them away!
Identify a format that embodies the values I believe to be the most important in agile and make sure the attendees feel and recognise them while they are attending
Make sure people actively participate
To me the most important values in agile are: people, customer and responding to change.
I asked 3 fantastic practitioners to help me on the day. The 4 of us were “the agile product team” that was going to deliver the product (learning) to our customers (audience)
Thank you so much to Claudio Perrone, Andrea Baker and Lisa Hickey for accepting to help with less than 24 hour notice and no details whatsoever on what i needed them to do (isn’t this ability to respond to change? :-))
I told the audience that me and my team mates were going to give a product to them, they were our customers and as such they were extremely important.
To start we needed the customer help to understand what real value is to them.
We asked them to select with dot voting some agile topics from around 35 different agile topics (I took a subset including mainly basic concepts).
The customers immediately queued towards the table where the cards and the markers for voting were. We time boxed the activity to 5 minutes. Claudio, ever the lean man, immediately identified a bottleneck as the table was too small and only 2 to 3 people could vote at the same time.
The activity had to be extended to 6 minutes to allow everybody to vote.
The team took 6 topics with highest number of votes and put them on the wall in dot voting ranking order.
We started with the first topic.
It happened to be BDD: First thing, I asked the customers if they knew what it was. One of the people in the audience started giving us his take. When he finished, I spoke about it a bit, then my 3 team mates took turns in adding their perspective.
Responding to change
This lasted for 5 minutes when the timer went off and i asked the audience to tell us by using thumbs up or down whether they wanted to continue talking for 5 more minutes about BDD or if they wanted to move to the next topic.
People voted for sticking to BDD for 5 more minutes
After 5 more minutes we voted again and we went to the second topic.
We made sure that the team swapped activities, everybody took turns in talking about the topics, we alternated roles like time keeping and pulling the cards from the wall.
We wanted to show team collaboration and cross functional abilities.
I got loads of feedback from the people in the audience and the team.
Claudio suggested that when talking about topics, the first couple of sentences need to describe it in a easily understandable recipe format, this is true in particular because of the audience low level of agile knowledge.
Davide Lovetere an enterprise Architect among our customers gave me a lot of incredibly valuable feedback around some contradiction in terms he had noticed during execution.
Other customers said that they enjoyed the format and want to use it for some of the meetings they do in work (yay!)
Other customers said that they enjoyed it but it finished too early, we only had time to talk about 4 topics and they would have loved to touch more
I loved doing it, received valuable feedback to improve it and can’t wait for the next time!
In my last stint as a tester from October 2012 to Jan 2014, I helped my organisation at that time moving from delivering once every month, to delivering multiple times a day.
Let me first clarify that we didn’t move to multiple deliveries per day just for the fun of it, but because we needed it.
Your organisation might not yet know it needs this level of agility but more than likely it will at some stage in the future.
How did this transform the role of the testers within the organisation?
When I joined I found scrum teams that delivered either once a month or once every 2 months. The teams had 3 different defects management databases full with old and new defects. Testers were doing the following activities:
exploratory testing (~50-70&)
The batches were big, the exploratory sessions were long and found a lot of defects. The automation was not effective, as it was slow and unpredictable, its value was negative.
When I left
When I left, we were using kanban, delivering multiple times a day, defects were more or less a myth of the past, no defect management tool existed. Testers were doing the following activities:
Three amigos BDD sessions with customers and developers
Exploratory testing (1~5%) – never longer than 10 minutes per card, more often than not reporting no defects
Pairing with developers
Coaching developers on testing
Writing automation (0%)
Talking to the customer and the team
Improving the system
Designing the product with the team and the customer
Helping define what to monitor in production
Any other valuable activity the team needed them to do
As you can see the activities that before occupied 100% of testers time, now occupy from 1 to 5% of testers time.
Were testers busy before? Yes, absolutely
Were testers busy after? Yes, absolutely
Were testers complaining because they weren’t doing automation or enough exploratory testing? No, believe me. Most testers I worked with saw the new activities in the role as a learning activity and an opportunity to broaden their skills and become more valuable to any company.
If a tester didn’t want to adapt to the new reality and embrace the change and new ways of doing things, he would have been busy for 10 minutes a day (~2%) and he would have not been useful to the team.
Did we get there with the touch of a magic wand? No, the end stage was the result of many experiments. It was, back then, a good recipe for that context at that time (it is continuously changing)
So, tester, what’s your strategy for working in a company that releases multiple times a day?