mysoftwarequality

Breaking people's balls since February 1970

Little Tim and the messy house

The messy kitchen

The messy kitchen

A cute little boy, Tim, lives in a messy house.

In the morning Tim’s mum, Tina, spends an hour looking for rubbish in the house, when she finds some, she writes a note on a piece of paper where she describes the steps that she followed when she found it, and sticks the note in one of 5 different drawers. Each drawer is labelled “Severity 1″,  “Severity 2″ and so on down to “Severity 5″.

Tina and Tim’s uncle Bob, meet every evening to discuss the daily findings and after arguing for a good while they agree on how to file the notes written during the day into 5 folders with labels “Priority 1″, “Priority 2″ and so on up to “Priority 5″.

Tim’s father, Oleg, every morning picks the folder with label “Priority 1″ reads the notes Tina wrote, follows the steps, finds the rubbish and throws it in the bin. He then writes an extra note on the piece of paper saying that he has thrown the rubbish in the bin. If the Priority 1 folder is empty, Oleg picks the Priority 2 folder and follows the same process. Some times Oleg cannot find Tina’s rubbish even when following her written steps, in this case he adds a note saying “there is no rubbish there!”. Sometimes Tina takes it personally and Oleg sleeps in the spare room. Oleg barely ever opens the folders with Priority 3 to 5. Such folders are bursting with new and old notes from many years back.

Tina spends an hour a day rechecking the Priority folders to see if her husband has added his notes. When she finds one, she will follow her own steps to make sure that Oleg has removed the rubbish from where it was as he said he did. If he did it, she will shred the original note, if the rubbish is still there she will add a note at the bottom saying, “the rubbish is still there, please go and pick it up!”. She will spend some more time adding some extra information on how to find the piece of rubbish. Sometimes, while she is tracking some old rubbish she finds some new, in this case she creates another note and adds it to a drawer.

Each piece of rubbish was filed neatly

For each piece of rubbish, a report was filed neatly

From time to time uncle Bob calls around asking for rubbish reports and rubbish removal trends. In these occasions Tina and Oleg spend the night up counting and recounting, moving sorting and drawing before they send a detailed rubbish status report.

Strangely enough, no matter how hard Tina and Oleg work at identifying, filing, removing, reporting and trending rubbish, the house is always full of shit and uncle Bob is always angry. Tim’s parents are obsessed in finding new rubbish but they don’t pay much attention to family members dropping chewing gums on the floor, fish and chips wrapping paper in the socks drawer, beer cans in the washing machine and so on. After all Tina will find the rubbish and following their fool proof process they will remove it!

One day Tim calls her parents and Uncle and sits them down for a chat. He suggests to stop throwing rubbish on the floor and messing up the house so that they can reduce the amount of time spent finding, removing filing and trending the rubbish. He also suggests to get rid of the folders labelled Priority 3, 4 and 5 as nobody has done any work on them and after all the existence of a minuscule speck of dust on the bathroom floor is not going to make their life uncomfortable. He also suggests that Tina calls Oleg as soon as she finds some rubbish so that he can remove it straight away, without the need for adding notes.

Uncle Bob tells Tim that what he says is nonsense, because the family are following a best practice approach for rubbish management and in agreement with Tina and Oleg locks him up in a mental facility.

Everybody lived unhappy ever after.

Have I eventually gone bonkers and started talking nonsense?

No, I haven’t suddenly gone crazy. I am Tim and I want to change the world.

BDD is – BDD is not

socrates11_2074496c

Hang on, this is not the real… Ah, OK, it’s a joke :-)

 

“I’m the smartest man in Athens because I know that I know nothing.” —Socrates 470-399 BC

BDD stands for Behaviour Driven Development

What BDD is (for me)

1. Conversations

BDD is about conversations

The conversations help us understand what we are trying to build and identify the behaviours of our application

The conversations help us share the knowledge about what we are building

Through the conversations we deliberately discover the behaviour of what we are building and remove some of our first order ignorance

BDD uses continuous feedback for deliberate discovery

The discovery helps reduce the unknowns and deliver software that matters

2. Documenting the behaviour of an application

BDD scenarios (or tests) help document the behaviour of the application we are building as they document the outcome of the conversations

3. The tools

There are a number of tools that allow automating the execution of the scenarios.

4. Testing

BDD tests assumptions through conversations, no other relationship exists.

My conclusion:

“BDD is about conversations and collaboration to generate software that matters”

that means: the conversations generate the software that matters

“Wherever it makes sense, describing the the behaviours in business language through scenarios and automating them will help you produce fast feedback and maintain the application as it grows.”

that means: using scenarios and good engineering practices you can be more effective

If you don’t do point 1 (the conversations) you can produce as many scenarios as you want, automate and run them continuously in a server farm bigger than Google’s but you are not getting much value and in my humble opinion you are not doing BDD.

Liz Kheogh, whose contribution have strongly influenced the evolution of BDD puts it very simply saying:

22havingconversations0aismoreimportantthan0acapturingconversations0aismoreimportantthan0aautomatingc-default

What BDD is NOT (for me)

A recurring problem I have encountered with teams starting to use BDD is the emergence of fallacies where teams conflate the problem that BDD is trying to resolve with other concerns, in particular, tools, artefacts and testing.

I am going to come clean straight away, I have been culpable of this for a good while and learned on my own skin how mixing up concepts can be extremely dangerous. See some of my lessons learned below.

 #Fallacy #1: BDD is Testing and automation

This is a very common problem. Often it originates when somebody (usually a tester) hears or reads something about BDD and starts using Gherkin and BDD scenario style to write their tests. This has usually a honeymoon period in which benefits are reaped because tests are now written in business language hence readable/understandable by everybody. This seems to help communication, but in the long term it actually makes it worse because testers and developers start communicating through scenarios written in Gherkin and stop talking. I have personally done this many many years ago. Hands up I screwed up my team communication badly!

In some cases a decision is made to use BDD scenarios to replace automation tests. This creates a confusion around what scenario should be written for developing some behaviour (BDD) and what scenarios should be written for testing the application. Using all boundary, invalid, special scenarios for development is not optimal, we are not looking for bugs, we are building an application based on its behaviours.

Very often, testers will push for having the scenarios automated through the UI and run in an end to end full integration environment. This generally creates a large slow and non predictable automation suite that is not suited neither for BDD fast feedback loops and discovery nor for end to end integration tests.

When conflating BDD with testing we create unnecessary confusion and end up with things that are neither regression tests nor BDD scenarios and are unusable for their original purpose (development of software that matters).

Do you want to avoid all these problems? Separate BDD from testing. They are 2 solutions to 2 completely different problems. Use the appropriate tools for each domain. Live happy.

#Fallacy #2: BDD scenarios are a communication tool

In some shops I have seen business analysts and product owners that had heard or read of BDD decide that they were going to formalise the requirements into BDD scenarios. I have seen this approach, suggested by a few people that do BDD training, but it is a recipe for disaster.  The most important part of BDD is completely ignored and the PO will elegantly formalise his assumptions in BDD scenarios. Once a developer gets the BDD scenario, she will deliver the PO’s assumptions into elegant code.

Tests for their nature cannot be ambiguous, there is no need for questions or conversations if requirements are defined in the form of tests. This is brandished as a great advantage by some people, instead it is the death of deliberate discovery, and the birth of Assumption Driven Development(*).

Do you want to avoid this? Use 3 Amigos conversations as communication tool. After you’re done, you have all the information to formalise the findings of the conversations into BDD scenarios

#Fallacy #3: We use Cucumber we are doing BDD (Feel free to replace Cucumber with Jbehave, SpecFlow, et cetera)

This is a non sequitur.

Some developers get very excited by neat tools and the ones I mention above are quite cool. Using a tool and writing software through BDD scenarios in the absence of conversations is different from doing BDD. Again, in the absence of conversations, inevitably we end up doing Assumption Driven Development(*).

A similar non sequitur fallacy: “We use Jira, we are agile!”.

 I am looking forward to the day in which I will feel ashamed of what I wrote above, because that day I will have learned something.

(*)Assumption Driven Development (ADD) = A person single handedly builds a set of unquestionable assumptions about the behaviour of an application in the form of tests. Normally this approach fails late, when at exploratory testing, somebody questions the PO’s assumptions now built into code.

Playing Lean – A fun way to learn Lean Startup

Playing LeanYesterday, I was lucky to be in a group of people playing a great new educational game, Playing Lean.

Helped by Raomal and Tore we engaged in a fun and exciting strategy game to sell our product to more and more customers, starting with innovators, moving to early adopters and eventually conquering the masses.

To do that we had to run experiments and build features. While doing that we talked about validating problems, customers interviews, A/B testing and other very interesting Lean Startup topics. Our experiments were run in a Build Measure Learn cycle and customers feedback was used to build our features.

One other thing I liked was that the game rewarded the player that chose a lean approach to build his product, by delivering only the features necessary and in some cases less features meant more customers!

I can see a bright future for the game as a training platform for the Lean Startup methodology and if I could i would organise games in Paddy Power right now. I am looking forward to be a supporter at the new Kickstarter that Playing Lean are running soon and look forward to using the finished product in the future.

If you are interested, join the community @ http://community.playinglean.com/

Executable specifications – what would you do?

Can you help?

Executable specifications are used in BDD/Spec By Example as a mean to describe an application behaviour in business language. They are executable because they are linked to tests that exercise the behaviour of the application itself.

Let’s imagine we want to develop the solution to the FizzBuzz problem, Which one of the following solutions would you adopt and why? If you have a different better solution than the two I list, please add it!

1. Specific examples:

Given we are printing numbers using FizzBuzz
When the number is <number>
Then we will print <answer>

|number|answer|
|      1      |     1    |
|      3      | Fizz    |
|      5      | Buzz  |
|      6      |  Fizz   |
|      10    | Buzz   |
|      15    |FizzBuzz|

2. Generic Rules

Given we are printing numbers using FizzBuzz
When the number is a multiple of 3
Then we will print Fizz

Given we are printing numbers using FizzBuzz
When the number is a multiple of 5
Then we will print Buzz

Given we are printing numbers using FizzBuzz
When the number is a multiple of 3 and 5
Then we will print FizzBuzz

Given we are printing numbers using FizzBuzz
When the number is a not a multiple of either 3 or 5
Then we will print the number

Thin Vertical Slices of Value

I recently had a conversation with two colleagues where I was trying to explain the advantages of frequently delivering thin vertical slices of value, versus working on separated components and integrating at the end.

I have been familiar with this concept for so long, and generally take it for granted, but while explaining it, I found myself at a loss and only after a good while I was able to make my point. That conversation prompted me to write this article so that myself or anybody else can use it in the future for this purpose.

Let’s imagine a typical web application with a UI layer, a service layer and a database layer. Let’s consider the very common case scenario, where our developers are specialised and can either do UI or services or databases, no cross skills exist.

There are generally two approaches to deliver this application:

  1. Create component teams that define interfaces between the components and deliver each component in isolation
  2. Create a team that contains all the 3 skills required and deliver small increments of value, i.e. small deliverables that include the 3 layers and are releasable.

The first option, believe it or not is the generally preferred one by most developers and traditional software development managers. It gives the first ones the ability to use their main skill, and the second ones the illusion of efficiency as they think they are using the right skills for the right purpose.

Component Teams

Component Teams Model

Usually the dreams of component teams are shattered at integration time. Let’s imagine they deliver monthly sprints. Each team at the end of a sprint will believe they have their sprint done, because everything works in their context. What happens in reality is that at integration time, what seemed to be perfectly defined in the interfaces generally is proven incorrect. Things don’t work, teams defend their component design and implementation, blame other teams for the integration failures and nobody takes responsibility for the most important thing of them all: the customer can’t use the value.

By the second sprint, generally the teams decide that they need an “integration testing sprint” [sic] to fix the issues that the teams couldn’t prevent by using predefined interfaces.

This is generally the sign of the start of a slow death of the company ability to deliver customer value.

Time goes by and teams become more entrenched in defending their component and blaming the other components for any failure. Integration testing becomes lengthy and painful and the customer becomes less and less important than defending the status of our component team.

Have you been there? I have, and I bet you have too.

Components Integration Explosion

Components Integration Explosion

There is a solution.

The secret is to shift, from delivering a component, to identifying a thin slice of value that cuts through all components and the customer can use.

It seems easy, and once you get used to it, it actually is. The problem is the mind shift from system to value. People are used to delivering systems and not to resolving problems, if you want to identify thin slices of value to deliver you need to focus on resolving a problem for your customer abstracting yourself as much as you can from the underlying systems.

The first thing you need to do is to have teams that have all the skills required for the full stack of technology so that each individual team can deliver customer value independently. Like this:

Team Delivering Value

Team Delivering Value

Each user story will deliver a thin slice of value, developers will collaborate and pair instead of communicating through interfaces and the ownership will be on value being delivered instead of software components.

Working like this, the integration issues will appear immediately, as soon as we connect the layers with the first slice of value. This in turn will help us not to build too much software based on wrong assumptions. The first thin slice might be challenging and take some time, but it will contain a huge amount of learning that will be used to reduce the time to build the subsequent slices.

We can expand this same concept to product development.

Let’s imagine our company is an international online wholesale store where clothing shop owners can buy their stock. We now have an opportunity to expand our market by introducing jewellery to our products. The business owners are very excited because they believe that the new product could increase our revenue by 30% and they have identified the suppliers to get cheap stock.

The project Jewellery is starting! Everybody is excited.

After a short period of research we find out that customers living in the Democratic Republic of Regulationstan have some restrictions over the quantity of jewels that can be bought by an individual shop in a period of time. The amount varies depending on the jewels type (gold, silver, platinum). Also, other 10 countries don’t allow watches to be bought in bulk and to make things worse, in the country we live in, Paranoidland, regulation exists for jewels sellers to obtain and store Know Your Customer information for up to 10 years for jewels that contain diamonds.

We could design a complex system that is able to comply with all the limitations and the regulations, implement it and deliver it to be consumed by the whole world in 6 months. Or we could take one category of articles that don’t need to comply to our own country’s regulation and target a rich market that doesn’t impose regulation and deliver it in a week.

The choice will depend on so many context specific factors that I cannot possibly imagine, so I won’t make it.

What I know for sure is that, if we use the second option, there will be a massive difference in the output, the difference is that after one week, our product will receive feedback from its customers. With the customer information available we discover unknowns and making decisions on what to deliver next becomes much easier.

What’s this got to do with component teams and vertical slices teams you might say.

Everything.

By delivering a vertical slice of value in a short period of time, we are able to evaluate it meaningfully, and use the feedback to decide what and how to build in the next slice. More than likely the first slice of value will give us enough information that makes less important or even completely irrelevant some of the slices we were planning to build in the future.

By doing so, we are deliberately discovering the unknowns and using learning as our guide.

The feedback we receive after each thin slice is our compass, every time we release, we get new directions and make sure we are never too far off the path to success.

Do agile organisations need a Test Manager?

I have had this discussion a few times in the last few years, the question whether or not agile organisations need Test Managers keeps coming up. Now that the agile transformation wind is blowing stronger over traditional software testing departments, more and more Test Managers have their role challenged.

This blog post is not about the need of a Test Manager during a transition to agile, this blog post is about the need of a Test Manager within a mature agile organisation.

As software development professionals, the first thing that we should do when somebody is asking us to build a software product, is to ask “what problem are we trying to solve with this product?”.

Software products are financed and built for the same reason people are hired and paid, that is “to resolve a problem”. Software products are built to resolve a problem for users and people are employed to resolve a problem for organisations.

If we look at traditional software development organisation with phases, gates, siloed development and test departments, a Test Manager resolves the following problems:

1) Communication/Negotiation with other silos
2) Schedule/Resource allocation
3) Process improvement
4) Quality signoffs
5) Test strategy
6) Skill and Resource Development
7) People leadership

Now let’s think about a mature agile software development organisation.

Problem 1 disappears with the existence of a cross-functional team where people with all skills are sitting together and don’t need an intermediary to communicate.

Problem 2 ceases to exist because testing, in an agile team, is a continuous activity, there is no need to schedule anything. On resource allocation, testing is a shared activity and everybody in the team will help; the team itself will know if one or more testers are needed. Through retrospection the team will identify skills shortages. No need for somebody to call the shots from outside.

Problem 3 evaporates. Continuous improvement is a team activity, again, retrospectives will trigger changes, not an external entity.

Problem 4 gets sucked into a black hole and implodes, agile teams don’t need quality signoffs, quality is owned by the team, the team is accountable for it and a high five is all they need.

Problem 5 is not relevant. The test strategy is part of the development strategy and is defined by the team. Let’s remember that we are talking about a mature agile organisation where teams have the necessary skills.

Problem 6 can be resolved by a Test Guild or Test Community of Practice that don’t need a Test Manager for functioning, it simply needs people passionate about testing and software quality.

Problem 7 is still a problem we need to resolve. We need a people leader, so let’s solve the problem and hire a people leader then!

I was a Test Manager, I had a choice, fight the system and create problems that didn’t exist anymore so I could justify my role or embrace the new challenge, learn new skills and start resolving real problems for my organisation. I chose the latter and never looked back.

What team do you play for?

One team One goal

One team One goal

“If you had one wish for software testing this year what would it be? I found this interesting question in a forum on Linkedin a few days back and it made me think.

Many ideas came to mind, around better automation strategies, better understanding of one’s context, better collaboration and knowledge sharing, et cetera and then it hit me.

For 2015 I wish that testers would understand what team they play for. I wish it was the year when testers realise there is no us and them with developers and business analysts. I wish testers focused on delivering value over acting like the quality police. I wish that testers stopped cracking developer’s jokes. I wish that testers started looking at the whole and stop (sub) optimizing their process. I wish that testers stopped trying to resolve problems that don’t exists and embraced collaboration with developers and BA’s to prevent defects and save their companies a lot of money in the process.

You might say “hey! that’s a lot of wishes, not one!”. No it’s only one.

It’s I wish testers chose to play for the right team.

 

The Resource Utilisation Paradox – On why less is more

Aim for 100% resource utilization

Trad Mgmt: aim for 100% resource utilisation

Traditional management approaches have often put high resource utilisation levels at the top of managers agenda as a recipe for effective management, assuming that existing not fully utilised resources are waste. When managers do capacity planning, they focus on utilising all the resources they have to the max, and this approach has often been considered a universal best practice among managers.

For the purpose of this article a resource[1] is a person that works within a team on a product.

Now, let’s have a look at a system where 100% utilisation works well:

Think about an oil pipeline. It would be really wasteful to build a massive pipeline and just push a little quantity of oil through it. Engineers will calculate the optimal pressure and volume of oil that can be moved and the oil will fill the pipe in a constant stream. That’s what I refer to as 100% resource utilisation.

Based on a similar principle, management believe that they should utilise all their resources 100% so that they can push through a stream of value to be delivered to our customers. 

Oil Pipeline

Oil Pipeline

So, what’s the problem? You might say. It is easy to agree with that approach but the issue is that by applying the laws of fluid dynamics to people delivering value through software, we underestimate the impact of variability. Human beings introduce levels of complexity and variability that are of an order of magnitude higher to the ones found in an oil pipeline. Oil will flow in a pipe at a steady pace at 100% utilisation, on the other hand a system made of human beings delivering value through software will be affected by countless variables and will not work the same way at 100% resource utilisation.

So, for a better comparison with software development, let’s use a system that is slightly more similar to a software delivery team than an oil pipe is because it includes human variability. Let’s use traffic flow.

100% utilization = traffic jam

100% utilisation = traffic jam

Let’s now look at what happens when we try to get 100% utilisation, that in traffic terms means all lanes used with as close to constant as possible flow of vehicles. Human variability factors in this time include for example a driver that gets distracted and doesn’t move when there is space in front of him, or a car that breaks down, or a driver that tries to overtake in between lanes or even a driver that falls asleep at the wheel. The result is something like in the picture to the right.

What’s efficiency for a traffic system? I am simplifying but the throughput of cars would seem to me as a good metric. So let’s say that our measure of efficiency is C/h = number of cars that go through a section of the road in one hour. A traffic jam implies low throughput and high cycle time.

We have all been in a traffic jam and we should know that it is certainly not the most efficient way of using the capacity of the road. When you are in a queue and are not moving, you are not being efficient, you are wasting time and petrol. Starting and stopping is not efficient either as, when breaking, you waste your momentum you gained when accelerating, burning petrol and breaks.

Laess acthan 100% utilization = Highway flow

Less than 100% utilisation = Highway flow

Now look at the picture on the left and tell me, is that more efficient than the traffic jam? Will the C/h of the second picture be higher or lower than the one in the first one?

Believe it or not, the answer is higher.

But, we are not using all our resources! Look at that unused space between cars, surely this can’t be right!

Now look at the two pictures below and see if you can identify similarities with the traffic system.

Comparison 1

Comparison 1

Comparison 2

Comparison 2

 

This is a simple visual representation on how 2 similar systems can obtain higher throughput by limiting the work in progress (cars in transit). It is a counter intuitive idea, but limiting the amount of cars that can be on a road at the same time, can increase throughput. Similarly you can increase your software delivery throughput by limiting work in progress. For the moment I am going to leave it for your imagination to think and speculate whether this is true or not. In the next chapter of this blog post I will use mathematics to demonstrate it. Stay Tuned.

 [1]I generally don’t like referring to people as resources, but in this case I will make an exception because it makes the story I am about to tell you, easier to understand.

How about you stop lying and take responsibility for your actions?

Stop lying to yourself

Stop lying to yourself

Stop saying “I don’t have time to do X”

WRONG!

You have 24 hours per day as much as me and any other human being, animal, plant, rock on this planet.

Be honest with yourself and take responsibilities for your actions.

Say “I chose not to do X because I consider Y and Z more important”

On empowerment and the 2 types of downtime

Downtime

Downtime

In a recent LinkedIn discussion, the topics of testing downtime and what should testers do during it was examined. I read and reread the answers a few times and I became convinced that I have a completely different view of what downtime is and how it should be utilised.

The first thing that I noticed was that test managers and leads had a substantial list of items to ask testers to do during their downtime but none of them seemed interested in asking the testers themselves what they thought they should do when in such situation.

This made me reflect on different levels of testers empowerment and respective managers levels of trust. Asking a tester to do something during downtime is not a bad thing “tout court”, I can see some situations where it could be helpful. On the other hand fostering an environment where we assume people are going to do nothing if not asked to do something can be a self fulfilling prophesy.

I believe that by fostering and rewarding a culture of awareness and collaboration while assuming people’s good intent, leaders can help and motivate people to do things because they want to, not because they are asked to do them.

The second observation that made me reflect is the fact that most of the activities proposed were activities that generally testers that work with me do as part of their normal day to day job and not as a once off during downtime. Among things of this type, LinkedIn users suggested:

analyse coverage and issues root cause, learn automation, cross skill with subject matter experts, review of tests, document procedures on wikis, read a work related book, blog, watch a video, self placed training, knowledge transfer, and other “improving” activities.

This point made me think that maybe my downtime is different from other people’s downtime. We do most of the activities suggested in the thread (apart from some wasteful ones) as part of our normal work, so let me tell you what my 2 types of downtime are.

Good downtime that we need to encourage and Bad downtime that we need to fight.

Good downtime: good downtime is planned downtime that we introduce into our process by limiting resource utilization for example by setting WIP limits and allowing for slack. This downtime is designed so that people don’t burn out. During this time people can do whatever they want from searching funny memes on the web, to going for a walk in the park, or if they prefer, studying something they want to learn, there is no limit to what people can do during good downtime. In my experience this downtime generally is of the form of 1 to 2-3 hours with cadence that depends on the specific context and can be fine tuned depending on project and people needs.

Bad downtime

Bad downtime

Bad downtime: This is unexpected downtime due to us waiting for something to do because the flow of work is blocked somewhere else in the system. Say for example the build is broken and can’t be progressed for us to do exploratory testing or the business analyst is on holidays and there is shortage of new user stories coming through. This is bad downtime because it is affecting the flow of value towards our customer. In this case t-shaped testers can help a lot to fix the issue, in fact they are able to support the developer stuck with the broken build or help in the creation of new user stories. When issues like the above happen, using tools like Kanban can be extremely helpful, in fact you will be able to visualize the issue immediately in the form of a bottleneck. The next thing to do is for the team (including but not limited to the people in downtime) to swarm around the bottleneck, reduce it and restart the flow.

If you want to continuously improve your flow, swarming and resolving bottlenecks is necessary but not sufficient. It is important that you resolve the root cause of the downtime and related bottleneck. One effective way to expose bad downtime so that we can identify patterns and fine tune our process is the waste snake, extremely easy to set up and use, I’d strongly recommend it.

 

 

Follow

Get every new post delivered to your Inbox.

Join 94 other followers