The hidden dangers of Process Debt

Most of us involved in software development are familiar with the term “technical debt”.

As a quick reminder, it was introduced by Ward Cunningham to describe the phenomenon that occurs when we use code that is easy to implement in the short run instead of applying the best overall solution we have identified.

It is by definition a conscious decision to take a shortcut for short term gratification (like taking out a new credit card for a holiday) that in thew long term will cost us to spend extra time in development when improving the system (like paying interest on the credit card debt).

Until the capital is repaid in full we will always pay interest when adding features to the system as the debt creates obstacle to adding new code efficiently.

Ward suggests that when we are in a situation of rushing something out we need to be conscious that we will have to pay the capital in the future or the interests will cripple us.

Ward suggests refactoring as a solution to paying off technical debt.

I really like Ward’s vision and I want to expand the metaphor one level up.

Now if we change the word “software” in my description of technical debt above with “processe” we can define Process Debt.

Replacing we get:

Process Debt is the phenomenon that occurs when we use a processes that is easy to implement in the short run instead of applying the best overall solution we have identified.

Let’s look at one example

You notice that lately the system seems to be more unstable than usual. You know this because there are more calls from customer care and more defects get raised.

Option 1: You want to get to the bottom of the situation and believe that a root cause analysis with 5 whys could get you there. If you use this approach you will probably identify a change in your process to help you prevent some defects in the future.

Option 2: You implement a better policy for developers to select the defects to work on reducing the time the defects are in the unresolved queue while maintaining new features creation throughput relatively stable.

You know Option 1 is better in the long run because it will generate a change in the process to reduce the amount of rework. This will mean less time spent fixing and more time spent on new features that as a consequence means higher throughput and lower lead time for new features.

Option 1 requires some investment. You need to hold one or more root cause analysis sessions, identify the problem(s) experiment with solutions until you find a solution that mitigates your problem.

If you are under pressure, because the new product needs shipping and the old defects need fixing, you are likely to choose option 2.

By doing this you have introduced process debt

The capital of this debt is the lack of change in the process  you could have identified if using Option 1.

Oh, yes, I forgot, change is difficult.

The worst part of it is the interest on the debt you will pay forever until you pay the capital. This interest will appear in the form of.
1. bugs keep on coming, we seem to be fixing bugs all the time
2. new features get delayed because now the queue is in the new features to be delivered
3. the lead time of any new feature is expanded by a factor X
4. the customers continue on complaining because of your defects in production
5. your product owner starts being annoyed at the amount of work spent by the development team on defects
6. developers are tired of always fixing defects
7. …
n. need I say more?

This interest will be paid for the rest of your life if you don’t fix the problem.

Months after you are covered in defects, are stressed by your customers and change job.

Is this healthy? Certainly not.

Is there a cure? Oh yes, indeed.

Continuous improvement has the same effect on process that refactoring has on software. It repays the capital of your process debt.

With small changes in the form of experiments you will be able to clearly discuss the problem that the team is having and make small tweaks continuously.

My esteemed colleague and fervid innovator Claudio Perrone presents his continuous improvement model called PopcornFlow saying

If change is hard, make it continuous

I could not agree more.

By making process change a continuous activity, we enable behaviours that will stay with teams forever and we unleash the power of people’s creativity in improving their own way of working.

 

 

Advertisements

Test coach versus test specialist, impact on queues

I recently had a very interesting conversation with a group of skilled testers on whether or not there should always be a test specialist in a cross-functional team.

A lot of people say yes, my personal experience says not.

My personal experience tells me that the most effective and efficient approach uses a test coach that slowly makes himself scarce. My experience focuses on creating competencies for completing an activity (testing) in a collaborative context.

One aspect I am finding difficult to explain is the impact on queues of a test coach approach, I will use a series of Scenarios and pictures to facilitate my reasoning.

If you feel like substituting activity A = development and activity B = testing feel free, but this approach is activity agnostic in my experience as I have applied it to analysis, and development obtaining similar results.

Scenario 1 – Test specialist

In the presence of one specialist on Activity B and many specialists on activity A.
Problem: Long q
ueues form in Ready for Activity B (Case1) or worse specialist multitasks on many activities at the same time slowing down flow of work as per Little’s law (Case2)

Screen Shot 2017-05-29 at 10.40.47

Scenario 2 – Test coach

Stage 1: In the presence of one coach on Activity B and many specialists on activity A
Initially coach pairs on Activity B with people that do activity A. This way we obtain 2 benefits:

  1. Queue in “Waiting for Activity B” is reduced as one person normally performing activity A is busy pairing with coach on one activity B task
  2. By pairing on activity B feedback loops are shortened
  3. Person with activity A acquires new skills to perform activity B from pairing with coach
  4. Quality of activity B increases as it is a paired activity
  5. Flow improves because of 1 and 2

Screen Shot 2017-05-29 at 11.22.49

Stage 2: When cross-pollination of skills required for activity B starts to pay off, we have 2 benefits

  1. Normally some activity A person will show particular abilities with activity B, in this case this person can pair with another less skilled activity A person to perform activity B
  2. Queue in “Waiting for Activity B” is reduced as more people with activity A skills are performing activity B
  3. Flow of value improves lead time decreases
  4. More activity A people get skills on activity B

Screen Shot 2017-05-29 at 10.56.00

Stage 3: All activity A people are able to perform activity B
Activity B Coach can abandon the team to return only occasionally to check on progress. Benefits:

  1. Activity A and activity B can be performed by every member of the team
  2. the WIP limit can be changed to obtain maximum flow and eliminate the queue in Ready for Activity B.
  3. The flow of value is maximised
  4. The lead time is minimised

Screen Shot 2017-05-29 at 10.58.05

WARNING: I have applied this approach to help teams with testing for many years. It has worked in my context giving me massive improvements in throughput and reduction of lead time. This is not a recipe for every context, it might not work in your context, but before you say it won’t work, please run an experiment and see if it is possible.

This is not the only activity that a good test coach can help a team on, there are many shift left and shift right activities that will also reduce the dependency on activity B.

I have been told a million times “it will never work”, I never believed the people who told me and tried anyway, that’s why it worked.

Try for yourself, if it doesn’t work, you will have learned something anyway.

Is agile Alive? Dead? Misunderstood?

Lats Sunday, after reading multiple “Agile is Dead” articles, I posted this short update on linkedIn

screen-shot-2017-03-05-at-11-03-37

As you can see from the stats, in less than 7 days it has received a lot of attention (I am not an influencer and my updates generally do not receive such large feedback)

My contacts in linkedIn include a lot of Agile or Lean Coaches and as expected, initially the message got some positive comments. Soon after some agile detractors joined the conversation and made it much more interesting as generally feedback that comes from different perspectives enriches the conversation adding dimensions that sometimes cannot be expressed by a biased mind.

I noticed 3 interesting trends in the messages.

  1. When agile is not driven by technology, agile fails
  2. When agile is driven by technology and not the business stakeholders, agile fails
  3. Agile is only useful to deliver something nobody wants quickly

The first 2 are extremely interesting, in fact they say exactly the opposite thing but they both come to the same conclusion “agile is dead”. I read and reread those messages and then I saw it

If agile is driven by one part of the organisation, whichever it is, and trust is not built within the whole organisation, it will fail. Do it like this and agile is dead before you even start.

If you try to own something that will change your organisation and run with it, you better make sure you share your vision, your responsibilities and your success with the rest of the organisation. How do you expect people outside your little world to want to follow you in this difficult change if they don’t know, understand, own and help you change. Agile/lean transformations are not driven by a department, they are driven by the whole.

And the result might be that you even stop talking about departments and only talk about the whole.

Now on objection #3.

Agile is only useful to deliver something nobody wants quickly

I have seen this very often and honestly makes me sad. A lot of scrum implementations have a Product Owner that is seen as the heart of the product, the person that understands the vision of the product and that takes the responsibility to take the important decisions for the future of the product in regards to strategy, prioritization and so on.

If you look at it this way, you might think that the PO is a single point of failure, in fact what if he is not able to make good decisions, how about his bias, is he a dictator?

As an agile coach I make sure that any product owner that works with me will have the tools for making good decisions. He in fact will know how to manage flow using WIP limits, he will be aware and become proficient in UX techniques, he will learn how to monitor, gather and use feedback from his customers, he will understand the importance of small experiments, he will be aware of cost of delay and when prioritising his features and user stories will have access to many advanced prioritization techniques.

Being agile does not mean automatically ignoring lean startup, lean UX, research. No that is not being agile, that is being a scrum master after 2 days training.

 

Testers, what business are you in?

businesscat

Recently I read an inspirational story from Jesse Lyn Stoner’s excellent blog  I would like to share with you.

The owner of a window shades company, was asked by a business consultant “What business are you in?” He initially answered “We are in the window shade business”. Then he was asked “When someone walks into your store, why do they want a window shade? What are you really selling?” he had to pause and think before he saw it

“We’re in the light-control and privacy business! – not the window shade business.”

The implications of this discovery were that by understanding the company real purpose, he was able to introduce new products that his customers loved.

You can find the full story (here)

Reading this story reminds me of the struggle of the testing community to get buy in from business owners. Years and years of hearing complaints that business owners don’t understand the importance of testing, they don’t appreciate the hard work of testers, I have grown tired of it.

I believe it’s not the business owners not to understand testers, I believe it is testers not understanding the business they are in.

The majority of the testers I know believe that they are either in

“The business of finding bugs”

or

“The business of sourcing information to help make decisions”

The testers I want to work with are in “the business of delighting their customers”, that is what we all need to be in.

If we make our business “delighting our customers” believe me, we will find a lot of innovative ways to do it as testers.

Forget about defects, forget about information, focus on delighting your customers and

  1. you will find innovative ideas to improve the quality of your product
  2. the business owners will love your work – and most of all
  3. you will have a lot of fun in the process!

 

 

 

How a little man from Japan helped Team X focus on their customers

screen-shot-2016-10-28-at-08-26-32

Previously on “A knowledge worker’s tale“: Team X discovered that lack of clarity on what to build, was creating a backlog in their exploratory testing work queue. They realised that adding new testers would have only fixed the symptom of the problem but not addressed its root cause.

Team X met to talk about a possible solution to the fact that too many misunderstandings in the user stories were slowing down the work dramatically and driving the testers crazy.

When Gus walked in, they were all there, nobody was late, the team really wanted to resolve this problem.

Mike took charge and said “We need to ensure that the user stories contain all the details, we can’t be blamed for problems in the requirements”. All the others nodded in unison, the bloody user stories were the problem…

Peter added “Absolutely Mike, we need Fritz to sign off the user stories so that when we start, if there is a problem, we know who’s fault it is and we don’t get the blame every time, I am tired of this!” he added “we need to make sure we know who’s fault it is”.

Before Fritz could answer, Gus interjected “OK, let’s see. Am I right in saying that our goal is to deliver value to our customers? Right. How is identifying who to blame going to help our customers? How is blaming somebody going to help our customers at all?”

He continued “Customers will still get the product late because of the misunderstandings, I bet if you asked them, they will tell you that they don’t give a rats arse who’s fault it is. What they care about is to get a solution to their problems, through our software”

Then he went “Once there was a little man in Japan, many years ago. He said that every activity that does not benefit the final customer is waste and needs to be removed from the process. He went to define 7 categories of waste that can be found in manufacturing. This guy’s name was Taiichi Ohno, he revolutionised the motor industry and his learnings have been used to improve manufacturing all over the world, he worked at Toyota. I strongly believe his lessons can be used very much in every context, including software development, let’s not introduce waste, let’s focus on value add activities”

“It seems reasonable” said Mike “but how do we make sure the defects in our stories don’t get caught only at exploratory testing, we have seen that when that happens our flow gets to a standstill, we can’t allow that”

“Taiichi would be proud of you Mike” said Gus. “One of his 7 categories of waste was exactly what you described, ‘defects’.” he continued “Quality was fundamental in the Toyota production line, one tenth of a millimetre difference in a bolt could bring the line to standstill, Taiichi knew it and made sure the workers knew it too so that they could find new ways of avoiding it”

Gus stood up and went to the whiteboard, he wrote “Prevention over Detection”. Then he drew a circle that had 3 stages “1. Write a failing test” – “2. Write enough code for the test to pass” – “3. Refactor”.

He went “Ladies and gentlemen, this is TDD, the one most effective ways of preventing defects in your code I have found in my 20 years as an engineer. In one of it’s more modern evolutions it is called BDD. Through high collaboration and conversations it allows teams to deliver code that has a fraction of the defects of code written without it. If you like to know more, I can organise a one day workshop to talk about BDD and how it can help us prevent defects”

Peter said “I am a bit confused about all this stuff, little Japanese people building cars and tests that are written before code, it doesn’t make sense. But in all fairness Gus has been right before with even more nonsensical stuff like ‘do less to do more’, I think we could take a leap of faith and trust something good will come out of this, what do you think guys?”

The team nodded in silence, they were puzzled. Accepting something brand new and counterintuitive can be scary, but true agile teams are courageous in their decisions.

Gus smiled and left the room so that the guys could make a decision freely.

Will Gus’s nonsense help team X? Stay tuned and discover what happens next

 

 

 

 

Stop building a Shitload of products, you are killing your company

idreamoforganizations0awherepeopleareempowered0atodeliverproductsthat0amatterjoyfulorganisations0a28-defaultI took my first steps in technology over 20 years ago. If we exclude the year off I took in 2006 I have always been employed in many different companies and I can say with certainty that I worked on a “shitload of products” and I use that specific term with purpose, read on to find out.

During all these years, being somebody that loves learning, I ended up working as system analyst, developer, tester, business analyst, manager, leader, coach, change agent, plus some other short term hats.

If I exclude the last 6-7 years where I had the skills and the ability to influence decisions I can go back and say for sure that the products that were initially envisioned, before any customer feedback was used, can be called a shitload of useless stuff and one or two good ideas that resolve real customer problems.

Very often the product envisioned was very similar to the product that we delivered.

Am I saying that in my first 13 years I worked I mainly produced waste?

Pretty much YES.

Another thing I remember in those early years was that I was never in a team where we could say, oh thank god we are busy but it’s not too bad, we can do our work, go home and have a balanced life. Invariantly there was pressure. We need all this by that date, come on! Work faster!

To me, it always felt like we were told by Dilbert’s boss that if we shove some more paper in the printer it will print faster. Also, when we were not busy, then managers will fill our capacity with a new shitload of useless products created for the purpose to make people sweat, very often no thought on the customer whatsoever.

Am I saying that trying to deliver fixed scope, fixed date shitloads of products creates big problems to the workers that build them?

YES, not even the pretty much is needed this time.

Another thing that I have noticed through the years is that without exception, companies older than 3 years have already built a shitload of products that are now impeding their ability to respond to change and survive. We will call this “shitload of legacy systems”.

This specific shitload is used as the excuse for not being able to change, as if saying “yes sure, we can’t compete with the market and we will die soon, but it’s not our fault it’s the fault of the legacy system (that by the way we built)

Am I saying that the shitload of products are also causing the slow death of the companies that created it in the first place?

Yes

Next time you start a product, think twice before rewarding people for the delivery of all the scope, in time.

I have been helping organisations deliver products that matter to their customer as soon as possible, I am not in the business of delivering projects or shitloads of products.

I am researching new ways of demonstrating THE VALUE in MONEYof the “non built shitload of products and features”, if you are interested, let’s do this together.