The hidden dangers of Process Debt

Most of us involved in software development are familiar with the term “technical debt”.

As a quick reminder, it was introduced by Ward Cunningham to describe the phenomenon that occurs when we use code that is easy to implement in the short run instead of applying the best overall solution we have identified.

It is by definition a conscious decision to take a shortcut for short term gratification (like taking out a new credit card for a holiday) that in thew long term will cost us to spend extra time in development when improving the system (like paying interest on the credit card debt).

Until the capital is repaid in full we will always pay interest when adding features to the system as the debt creates obstacle to adding new code efficiently.

Ward suggests that when we are in a situation of rushing something out we need to be conscious that we will have to pay the capital in the future or the interests will cripple us.

Ward suggests refactoring as a solution to paying off technical debt.

I really like Ward’s vision and I want to expand the metaphor one level up.

Now if we change the word “software” in my description of technical debt above with “processe” we can define Process Debt.

Replacing we get:

Process Debt is the phenomenon that occurs when we use a processes that is easy to implement in the short run instead of applying the best overall solution we have identified.

Let’s look at one example

You notice that lately the system seems to be more unstable than usual. You know this because there are more calls from customer care and more defects get raised.

Option 1: You want to get to the bottom of the situation and believe that a root cause analysis with 5 whys could get you there. If you use this approach you will probably identify a change in your process to help you prevent some defects in the future.

Option 2: You implement a better policy for developers to select the defects to work on reducing the time the defects are in the unresolved queue while maintaining new features creation throughput relatively stable.

You know Option 1 is better in the long run because it will generate a change in the process to reduce the amount of rework. This will mean less time spent fixing and more time spent on new features that as a consequence means higher throughput and lower lead time for new features.

Option 1 requires some investment. You need to hold one or more root cause analysis sessions, identify the problem(s) experiment with solutions until you find a solution that mitigates your problem.

If you are under pressure, because the new product needs shipping and the old defects need fixing, you are likely to choose option 2.

By doing this you have introduced process debt

The capital of this debt is the lack of change in the process  you could have identified if using Option 1.

Oh, yes, I forgot, change is difficult.

The worst part of it is the interest on the debt you will pay forever until you pay the capital. This interest will appear in the form of.
1. bugs keep on coming, we seem to be fixing bugs all the time
2. new features get delayed because now the queue is in the new features to be delivered
3. the lead time of any new feature is expanded by a factor X
4. the customers continue on complaining because of your defects in production
5. your product owner starts being annoyed at the amount of work spent by the development team on defects
6. developers are tired of always fixing defects
7. …
n. need I say more?

This interest will be paid for the rest of your life if you don’t fix the problem.

Months after you are covered in defects, are stressed by your customers and change job.

Is this healthy? Certainly not.

Is there a cure? Oh yes, indeed.

Continuous improvement has the same effect on process that refactoring has on software. It repays the capital of your process debt.

With small changes in the form of experiments you will be able to clearly discuss the problem that the team is having and make small tweaks continuously.

My esteemed colleague and fervid innovator Claudio Perrone presents his continuous improvement model called PopcornFlow saying

If change is hard, make it continuous

I could not agree more.

By making process change a continuous activity, we enable behaviours that will stay with teams forever and we unleash the power of people’s creativity in improving their own way of working.

 

 

Advertisements

Is agile Alive? Dead? Misunderstood?

Lats Sunday, after reading multiple “Agile is Dead” articles, I posted this short update on linkedIn

screen-shot-2017-03-05-at-11-03-37

As you can see from the stats, in less than 7 days it has received a lot of attention (I am not an influencer and my updates generally do not receive such large feedback)

My contacts in linkedIn include a lot of Agile or Lean Coaches and as expected, initially the message got some positive comments. Soon after some agile detractors joined the conversation and made it much more interesting as generally feedback that comes from different perspectives enriches the conversation adding dimensions that sometimes cannot be expressed by a biased mind.

I noticed 3 interesting trends in the messages.

  1. When agile is not driven by technology, agile fails
  2. When agile is driven by technology and not the business stakeholders, agile fails
  3. Agile is only useful to deliver something nobody wants quickly

The first 2 are extremely interesting, in fact they say exactly the opposite thing but they both come to the same conclusion “agile is dead”. I read and reread those messages and then I saw it

If agile is driven by one part of the organisation, whichever it is, and trust is not built within the whole organisation, it will fail. Do it like this and agile is dead before you even start.

If you try to own something that will change your organisation and run with it, you better make sure you share your vision, your responsibilities and your success with the rest of the organisation. How do you expect people outside your little world to want to follow you in this difficult change if they don’t know, understand, own and help you change. Agile/lean transformations are not driven by a department, they are driven by the whole.

And the result might be that you even stop talking about departments and only talk about the whole.

Now on objection #3.

Agile is only useful to deliver something nobody wants quickly

I have seen this very often and honestly makes me sad. A lot of scrum implementations have a Product Owner that is seen as the heart of the product, the person that understands the vision of the product and that takes the responsibility to take the important decisions for the future of the product in regards to strategy, prioritization and so on.

If you look at it this way, you might think that the PO is a single point of failure, in fact what if he is not able to make good decisions, how about his bias, is he a dictator?

As an agile coach I make sure that any product owner that works with me will have the tools for making good decisions. He in fact will know how to manage flow using WIP limits, he will be aware and become proficient in UX techniques, he will learn how to monitor, gather and use feedback from his customers, he will understand the importance of small experiments, he will be aware of cost of delay and when prioritising his features and user stories will have access to many advanced prioritization techniques.

Being agile does not mean automatically ignoring lean startup, lean UX, research. No that is not being agile, that is being a scrum master after 2 days training.

 

A short report on my agile experimental talk

Last week I asked the community for help on designing an agile talk rather than a talk on agile .

If you don’t want to read the full article here’s the TLDR: Can we embed the agile values in the format of a beginners talk so that people will learn by breathing them rather than hearing about them?

I received quite a lot of encouragement from a lot of people. I love the agile/lean community, thank you folks, you are incredible!

I got some great suggestions from Patrick Steyaert that recommended looking into Lean Coffee and Fish Bowl.

Both formats are highly participative and pretty much agendaless and gave me a great point to start at.

My goals in priority order:

  1. Do not bore an audience that for the first time hears about agile. Don’t push them away!
  2. Identify a format that embodies the values I believe to be the most important in agile and make sure the attendees feel and recognise them while they are attending
  3. Make sure people actively participate
  4. Have fun

To me the most important values in agile are: people, customer and responding to change.

The People

I asked 3 fantastic practitioners to help me on the day. The 4 of us were “the agile product team” that was going to deliver the product (learning) to our customers (audience)

Thank you so much to Claudio Perrone, Andrea Baker and Lisa Hickey for accepting to help with less than 24 hour notice and no details whatsoever on what i needed them to do (isn’t this ability to respond to change? :-))

my teammates.png
My Fantastic Team Mates: from left to right Claudio, Andrea and Lisa

The Customers

I told the audience that me and my team mates were going to give a product to them, they were our customers and as such they were extremely important.

To start we needed the customer help to understand what real value is to them.

We asked them to select with dot voting some agile topics from around 35 different agile topics (I took a subset including mainly basic concepts).

The topics were taken from ‘s wonderful Agile Topics card deck that I printed and laminated (for the quite steep price of €50)

The customers immediately queued towards the table where the cards and the markers for voting were. We time boxed the activity to 5 minutes. Claudio, ever the lean man,  immediately identified a bottleneck as the table was too small and only 2 to 3 people could vote at the same time.

bottleneck.png
Customers queuing to dot vote (bottleneck)

The activity had to be extended to 6 minutes to allow everybody to vote.

The team took 6 topics with highest number of votes and put them on the wall in dot voting ranking order.

We started with the first topic.

It happened to be BDD: First thing, I asked the customers if they knew what it was. One of the people in the audience started giving us his take. When he finished, I spoke about it a bit, then my 3 team mates took turns in adding their perspective.

Responding to change

This lasted for 5 minutes when the timer went off and i asked the audience to tell us by using thumbs up or down whether they wanted to continue talking for 5 more minutes about BDD or if they wanted to move to the next topic.

People voted for sticking to BDD for 5 more minutes

After 5 more minutes we voted again and we went to the second topic.

We made sure that the team swapped activities, everybody took turns in talking about the topics, we alternated roles like time keeping and pulling the cards from the wall.

We wanted to show team collaboration and cross functional abilities.

the-topics
The topics selected by our customers

I got loads of feedback from the people in the audience and the team.

Claudio suggested that when talking about topics, the first couple of sentences need to describe it in a easily understandable recipe format, this is true in particular because of the audience low level of agile knowledge.

Davide Lovetere an enterprise Architect among our customers gave me a lot of incredibly valuable feedback around some contradiction in terms he had noticed during execution.

Other customers said that they enjoyed the format and want to use it for some of the meetings they do in work (yay!)

Other customers said that they enjoyed it but it finished too early, we only had time to talk about 4 topics and they would have loved to touch more

cufbzuuw8aazvyr
That’s me having a ball as usual when I speak at conferences 🙂

I loved doing it, received valuable feedback to improve it and can’t wait for the next time!

 

 

How about an agile talk instead of a talk about agile?

asleep

Here’s the thing, next week I am going to speak at the Agile Tour Dublin . My talk topic is in the beginner track and it is a Quick Introduction to Agile.

How do I keep people awake and deliver something that will be valuable?

I could start with a timeline of the events that brought us to the manifesto and then continue towards the different frameworks, mention the important people, talk about the values etc.

Well, I could, but maybe nobody would care.

I could talk about how agile requires a fundamental change in the mindset and discuss potential problems people might find once they embark into a change journey.

Maybe slightly better

Or I could focus on the values and principles in the manifesto and explain how they apply to what people do within organisations

But people might go back asleep, etc.

Fundamentally I think that no matter which of the 3 approaches I take, it might be something that people could get from reading a wikipedia page, not good enough for my customers!

So I thought, how about I do an agile talk instead of doing a talk about agile.

How about before the start I ask my customers (the audience) to pick from a set of topics they want to hear about. And how about I get the audience to give me feedback while I deliver so that I can react quickly and change the subject when it is not giving value any longer?

This would embody the close customer collaboration, the fast feedback and the ability to react to changing conditions.

Also, how about I ask the audience if they want me to speak for exactly 30 minutes and go through all my slides or if they believe that they can collaborate and give me feedback so that I might be finished 10 minutes earlier if the topics are complete or maybe even 5 minutes later if they still need some answers.

This would reflect the customer collaboration over contract negotiation

How about if I told them that I am not going to send them the slides by mail but I am available all day long to talk to each and any of them discussing the content of the talk.

This would sound like face to face collaboration over processes and tools

Wouldn’t this format also embody Working software over comprehensive documentation?

Do you think a format like this would be valuable?

Do you have any other suggestions to do an agile talk instead of a talk about agile?

Please help me by using the comments or mail me at augeva@gmail.com

 

 

 

 

 

Testers: what’s your strategy in a continuous delivery context?

smallbatchIn my last stint as a tester from October 2012 to Jan 2014, I helped my organisation at that time moving from delivering once every month, to delivering multiple times a day.

Let me first clarify that we didn’t move to multiple deliveries per day just for the fun of it, but because we needed it.

 

Your organisation might not yet know it needs this level of agility but more than likely it will at some stage in the future.

How did this transform the role of the testers within the organisation?

Massively

The start

When I joined I found scrum teams that delivered either once a month or once every 2 months. The teams had 3 different defects management databases full with old and new defects. Testers were doing the following activities:

  1. automation (~30-50%)
  2. exploratory testing (~50-70&)

The batches were big, the exploratory sessions were long and found a lot of defects. The automation was not effective, as it was slow and unpredictable, its value was negative.

When I left

When I left, we were using kanban, delivering multiple times a day, defects were more or less a myth of the past, no defect management tool existed. Testers were doing the following activities:

  1. Three amigos BDD sessions with customers and developers
  2. Exploratory testing (1~5%) – never longer than 10 minutes per card, more often than not reporting no defects
  3. Pairing with developers
  4. Coaching developers on testing
  5. Writing automation (0%)
  6. Talking to the customer and the team
  7. Improving the system
  8. Designing the product with the team and the customer
  9. Helping define what to monitor in production
  10. Any other valuable activity the team needed them to do

As you can see the activities that before occupied 100% of testers time, now occupy from 1 to 5% of testers time.

Were testers busy before? Yes, absolutely

Were testers busy after? Yes, absolutely

Were testers complaining because they weren’t doing automation or enough exploratory testing? No, believe me. Most testers I worked with saw the new activities in the role as a learning activity and an opportunity to broaden their skills and become more valuable to any company.

If a tester didn’t want to adapt to the new reality and embrace the change and new ways of doing things, he would have been busy for 10 minutes  a day (~2%) and he would have not been useful to the team.

Did we get there with the touch of a magic wand? No, the end stage was the result of many experiments. It was, back then, a good recipe for that context at that time (it is continuously changing)

So, tester, what’s your strategy for working in a company that releases multiple times a day?

 

The magic talking board

episode2newz
This is the second episode of the adventures of team X, if you haven’t already, read the first part of the story

Things looked slightly less grim now. We had visualised the work that was coming through and at least we could see how deep the hole we had dug for ourself was.

For a team of 8 (programmers testers and analysts) we had something like 60 work items in progress, it was gigantic, daunting and scary.

But that was huge progress, because now we knew it was there. Now that we could see it and show it, we could do something to stop it from happening again.

messy-deskLittle we knew that what we had created was not only a board, what we had created was a living and talking entity, you don’t believe me? Read on.

The morning after we built the board, Zach appeared, he was the most unreasonable customer in the Northern hemisphere, known for his famous saying “take this ticket, if you finish it last week you are late”. He walked hurriedly to the team area with his usual bunch of new unbelievably urgent tickets to work on.

While he was walking, I swear, I thought I heard the music from “The good the bad and the ugly” such was the tension in the air.

Jimmy, a developer, told him: “Hey Zach, yes, we will work on them but we need to finish these first” pointing at the board.

Zach wasn’t happy, squinted and said, “but this is urgent! I need it now!”.

Jimmy was relentless, looked at the cards that Zach had already in progress on the board and said “that’s fine Zach, no problem, have a look at these cards on the board, they are your 6 tickets already in progress, if your new two are more urgent then let’s replace them”.

Zach eyes widened with disbelief, looked at the other cards and said, “but, but, but… I, I, I… need, need , need… …well no, those are more urgent, work on these new 2 only once you are done with the old ones”, he said goodbye and went back to his office.

goodbadandugly
From Top to Bottom: Jimmy, Gus and Zach

Jimmy the developer had defeated Zach, the fastest ticket-slinger in the company!

The team X guys looked at each other, high fived Jimmy and noted that the board had told the customer what they had been trying to say for the last 2 years, i.e. “we are too busy we can’t take more work”, but for some magic reason now that the work was visible, that conversation was much easier and Zach was not shouting anymore, miracle!

This small event made the team determined to always make sure that their work was visible on the board, the board was a silent ally, they won’t forget to feed it!

If Zach had been brought to reason, what else could the board help them do?

Gus said, now the fun starts, let’s talk about setting a WIP limit…

Wanna know what happened next? Is the WIP limit going to make the team even more powerful? Stay tuned for the 3rd episode to find out what Jimmy and the team X lads get up to!

The new episode is out!