When I speak to people about how it is possible to continuously deliver customer value with near zero issues, I usually get laughed at.
After I tell them that there is nothing to laugh at, people start challenging me on how I integrate with other systems, how I manage defects, how I deal with changing requirements, how I manage regression issues, and how I cope with context switching and other similar issues.
Luckily enough I have empirical data and my answers are based on experience and not some book or model I read about somewhere, so after a while I manage to get people moving from laughing at me to at least starting to be curious.
It has to be said, people become curious but deep inside they still don’t believe me and still think I am a big fool.
How could I blame them? I would have done the same a few years back. I know that people need to have to prove it for themselves to be able to believe it, I have no problem with that.
While I manage to explain my way of dealing with defects, changing requirements, regression, and context switching, until now I haven’t been able to answer the biggest question of them all, the one that every conversation ends up with eventually: how do you deal with extremely complex systems that need to scale up?
I have been thinking about this for a while now and the more I think about it the more I become convinced that Complexity is the Excuse.
Complexity exists when we are not able to prioritise the value to deliver (we don’t know what we really want).
Complexity exist when we are not able to understand and describe the system we are building.
And finally, Complexity is a nice excuse for not doing our job properly as software engineers and having something to blame.
Reduce Complexity and stop taking excuses:
1) You want to deliver customer value, OK. You don’t need to deliver everything on day 1. Sit down with your business partners and identify the highest value added feature and focus on that one. If asked for bells and whistles, say, “we will do it later and only if we really need to”, chances are when you are finished doing the first feature you will have learned that you need something different anyway.
2) When deciding how to implement such feature, look for the simplest and cheapest solution, do NOT future proof. By future proofing you will be adding complexity and guess what? YAGNI. Measure the success of your feature and feel free to change direction, failure is learning.
3) Once you have identified the value to be delivered, make sure you break down its own complexity. If a user story or unit of work or whatever you call it has more than one happy path, then it is too complex, break it down into 2 or more units of work.
4) If you start working on something and you discover it is more complex than you had thought, then stop and break it down to less complex units, if you keep on going saying nothing, you will hide complexity and sooner or later you are bound to mess it up.
Scaling up is the wrong answer to the false complexity question.
Chances are you don’t need to scale up at all: Read 1 and 2 again and again, you will find out that you don’t need as many resources as you thought you would. Scaling up, most of the times, is the easiest, most expensive and laziest approach to fight complexity.
For doing 1) and 2) in a structured manner I strongly recommend an approach called Impact Mapping devised by Gojko Adzic, it works.
For doing 3) click here
For doing 4) use your head
TL;DR: stop blaming complexity when you don’t understand what you are building
mysoftwarequality 
Hey Augusto – ya big fool!
All good points and with my software tester hat on, the thinking has parallels directly to the testing challenges. Not breaking down the testing problem into the least complex solution is quite common. Ultimately, as you mention above for Dev, it’s the only way
You’ll often see overly elaborate test cases being written to cover any and all scenarios. It’s a bit of scope creep / future proofing. Test Plans that are way too verbose because the testing problem is not well understood. Even too much process and practice applied, breaking the YAGNI rule. Automation frameworks with too many bells and whistles. The list definitely goes on.
Whenever a tester or test manager dithers about, unable to define the testing approach or solution to be applied, it’s a sure sign the complexity of the problem is not being addressed and is ‘the excuse’! Get it simplified and stop using it as such, here here!!
Mark.
Thanks for your feedback Mark, you made me think about it more ans testers are probably the ones that use the complexity excuse more often. The consequence is that they believe they have to test everything no matter how small the change is. 🙂
Complexity is definitely the excuse, except for when it is not. 🙂
Your advice is spot on as far as application functionality. However, I disagree strongly that future-proofing at the architectural level is a bad idea. In fact, it would be completely irresponsible to deliver an application framework that cannot scale or is inflexible, since the cost of re-architecture is beyond the reach of many organizations.
One of the problems with Agile-based methods is that the focus on the need for speed can become a blind focus. There is still very much the need for careful thought about how applications are designed and built. Technology organizations that do not plan for the long haul do so at their own peril.
As always, the need is for balance.
Thanks for you feedback Darrell. I’d say you nailed it in the last sentence, the need is for balance. Instead of future proofing I should maybe said “gold plating” that is kind of future proofing at the code level rather than at the architecture level. At the same time experience has taught me that the best architecture normally emerges after trial and error, so expecting to nail the perfect architecture at the beginning is not always possible. I believe agility is also the ability to be able to rewrite/rearchitect something quickly and painlessly.
May I say that software architecture is complexity’s sister?
Most of the time there’s no need for a software architecture to be defined upfront, especially because you don’t know upfront what are your non functional requirements, you don’t know your functional ones!
Or, if they are known, they are: “fast, reliable, flexible, maintainable, scalable, resilient and simple…oh, I forgot I want it to be cheap!”
Now, reality is non functional requirements are unknown, as much, if not more, as functional ones. As a consequence you can’t draw an architecture upfront, but you can embrace change and understand the system value doesn’t reside in one architecture, but in it’s ability to change the architecture following the needs of the business.
In other words, if you can refactor frequently and with confidence you don’t need to define an architecture upfront, or at least you only need to define only the very basic and foundational stuff upfrunt and delay other decisions to a later stage when you and your business partners have a cleared picture of the goal.
My humble 0.02€ opinion
[…] Complexity is the Excusehttps://mysoftwarequality.wordpress.com/2014/07/15/complexity-is-the-excuse/~ Weekend Testing EU […]