Agile Testing Manifesto Anyone?


Let’s help new agile testers find the road to agility!

This is a call to all agile testers to contribute creating the agile testing manifesto.

Please use comments to let me know what you think of the proposed values and suggest new ones you think should be included.

————————————————————————————————————–

Agile Testing Manifesto [Draft]

————————————————————————————————————–

We are uncovering better ways of testing
software by doing it and helping others do it.
Through this work we have come to value:

Delivering value over criticizing implementation the product (*)

Influencing outcomes over telling others what’s right and what’s wrong imposing rules (**)

Preventing defects over detecting and logging them (**)

Listening to people over analysing quality metrics

Improving our trade craft over process good practices (**)

That is, while there is value in the items on
the right, we value the items on the left more.

(*) Changes proposed based on further reflection

(**) Changes proposed based on feedback

HELP US: If you are willing to help with the project, we have created a Google group where we are discussing improvements on each of the values and discussing the introduction of new ones. Using the blog comments for this purpose is becoming difficult due to number of comments. If you want to help and have a Google account, let augeva@gmail.com know and I will send you a private invite.

Advertisements

43 thoughts on “Agile Testing Manifesto Anyone?

  1. to me miss my preferred word: Responsibility: “the state or job of being in charge of someone or something and of making sure that what they do or what happens to them is right or satisfactory” . Macmillan dictionary.

    “Taking responsability over task assignment and label company ( i.e scrum master, senior tester and bla bla bla) ”

    For the rest , they are values that i promote even in my presentation in slideshare.

    • Thanks notesoftester, responsibility is certainly extremely important for an agile tester. An agile tester in my opinion is not only simply responsible for an outcome, but he is also responsible for influencing his team mates in order to get to that outcome. Could the concept of responsibility be included in the “Influencing outcomes over telling others what’s right and what’s wrong”?

      • to me it is the opposite .

        Through the responsibility you can get the outcome. The outcomes should not be influenced otherwise you don’t create value.

        In this case , The subject(team-mate..bla bla bla) is not forced to act by someone or something from outside.

        If each of us has as its purpose not the outcome, not the result but a value that creates an “good” then it is done.

        The “good” ,in itself as concept, contains objectivity and something of tangible.

        The act of building values depending only of subject.

        then the result is not a dictation of ignorance (your value creation), because the subject is also aware of the action that exists.

        If you influence the result I understand that you are able of to identify what is right and what is wrong and therefore would not need to create more value.

        There is a contradiction when you write about : value and influences .

        It is better , to me, to say:
        Influence people for obtaining excellence
        Discovering value over criticizing implementation

  2. I think its a great manifesto. I’ve been thinking about other values that we could add. Each time I circle back to the above list. As a agile tester I’d be proud to sign it.

    In my opinion ‘Taking responsibility’ does not fit well here, as I think this is something we do as a whole team and not as an individual.

    • @Marywalshe – I would argue that the word ‘not’ should be deleted, for the first instance, and replaced by ‘and’, in the second instance, in your last sentence so as to read “In my opinion ‘Taking responsibility’ does fit well here, as I think this is something we do as a whole team and and as an individual.”

      • @Johnwilson on reflection I think your right. The point I was trying to get across was that responsibility does not lie solely on a testers or team leads shoulders. It is responsibility on different levels. As a team and as an individual.

  3. Second read and more thought applied.
    Do I really value criticizing implementation, telling others what’s right and what’s wrong, detecting and logging, analysing quality metrics and applying best practice? Not really. I think along the lines below.
    Delivering value over delivering artifacts
    Influencing outcomes over influencing decisions
    Preventing defects over finding defects
    Listening to people over referring to documents
    Improving our trade over following our trade

    • Excellent points John, on first read I was 100% with your new approach, on further thought I am not so sure 100% :-). For a moment think about the original agile manifesto, do we really value comprehensive documentation? or contract negotiation? Maybe not, maybe just as much as “detecting and logging” and “analysing quality metrics”.

      Don’t get me wrong, I am still open to embrace your suggestions, I just would like to know what you think after rereading the real agile manifesto.

      • Third read and appreciate that “While there is value…” is different from the interpreted, by me, “there is value…”. Subtle, but important, difference. Thanks for pointing it out.

        Still feel uncomfortable though about some of the wording on the right hand side. Just seems to me to be far more negative than the agile manifesto ones.

        • Good stuff John, I agree that some of the stuff on the right is too negative. I particularly don’t really like “over telling others what’s right and what’s wrong” and they are my own words 🙂 How about “Influencing outcomes over building bridges”?

          • Gus
            First, thanks for starting this thought provoking discussion.
            Second, after yet more re-reading and thought I think there’s a possibility that I understand my previous discomfort about the items on the right. I feel that the items on the right are some of the negative traits that we see in the testing world. If we were to describe the behaviours of an agile tester then I feel there could be more positive contributions from some of the other posters.
            With acknowledgements duly given to those whose ideas I’ve used, examples, which may include duplications, could include things like:
            Taking responsibility for team and self over given responsibility
            Preventing defects over detecting them
            Using appropriate tools over using only one tool (This one definitely needs more work to polish. The point that I’m trying to get across is that an agile tester will use many tools whereas a non agile tester will try to make the one tool they have do it all)
            Achieving project, team, functional team and personal goals over achieving personal goals
            Thoughtful testing over scripted testing
            Actively identifying and removing obstacles over waiting for others to remove obstacles
            Timely reporting of progress over weekly reporting of progress
            Reporting when needed over reporting when told
            Good enough over total conformance
            Presents solutions over present problems
            Testing with others over testing alone

          • A lot of excellent points John, thank you for your contribution! On the first part, I am not sure they are negative traits. For instance criticizing the product is extremely important, same can be said for detecting bugs, what I am trying to express is that an agile tester values more delivering value and preventing defects. I have no problem with bug detection, it has an important place, I as an agile tester, focus on reducing the amount of detection required through prevention i.e. I value more prevention. They sound very much like comprehensive documentation and contract negotiation. I need documentation but I will reduce it to the minimum because I will have working software to show for. I hope I am explaining myself 
            Your other suggestions have a lot of value, I definitely think we should include the concept of responsibility that notesoftester also mentioned.
            Then we could take another approach altogether, we could limit the values to the first say or 4 to 5 and then expand through principles, just like the agile manifesto does.
            Lisa Crispin and Janet Gregory created the “ten principles for agile testers” in their Agile Testing book, we might merge our ideas with theirs, what do you think? http://agileinaflash.blogspot.ca/2009/03/ten-principles-for-agile-testers.html

  4. No, I don’t get it.

    You have set up strawmen in your manifesto: seriously, who would state that they now focus on criticizing implementation or telling others what’s right or wrong? How, or why, do you find values in those characterizations?

    Is this manifesto intended to show a better way of doing agile, or are you comparing against a generalized model of “testing”?

    I’m a Quality Assurance guy, so i’m all about the improvement of process and about learning from prior projects and past experience. There’s always an opportunity to improve. What I don’t get from your draft manifest — and I’m not saying that an agile manifesto is a bad idea! — is anything concrete. If I don’t waste any time on criticizing somebody’s implementation, how do I jump to providing value? How does somebody who does not have 14 years of experience with QA derive actionable value from that statement?

    I’d suggest that an Agile testing manifesto needs to provide a refined model for conceptualizing agile testing, and then needs to provide some kind of guidance for people to apply these new concepts.

    • Derek, thanks for your comment.
      I appreciate you don’t like the manifesto, and I welcome your criticism as an opportunity to improve it, are you willing to help?
      I agree with you that the sentence “telling others what’s right or wrong” is not good for the reason you mention and also something that John mentioned in an earlier message. I am happy to change it with what the contributors think is best or if we don’t find anything we can remove it all together, thanks for pointing it out!

      You also imply that nobody would say they find value in criticizing implementation, I would strongly disagree with that statement. Traditional test teams have very often focused their efforts on criticizing the product, a sign of this is that they often associate their value with the ability to detect defects.

      The goal of the manifesto is to show in a succinct way the main values of an agile tester to help agile testers, in the same way the agile manifesto helps agile teams. The obvious comparison is done with a non agile tester, same as it happens in the original agile manifesto.

      As per deriving actionable value from the statements, I don’t believe it is the goal of the manifesto, or at least not in this form. What I would expect a professional with 14 years of experience in QA to get from the manifesto, is maybe some personal meditation, a discussion subject with your colleagues. I am not trying to do anything special here Derek, don’t get me wrong.

      The manifesto is not meant to be a practical guide to becoming an agile tester, but it could be a quick thermometer to measure how agile you are at a certain point of your life.

  5. It is interesting…. but before I agree or disagree, it would be great if certain points are elaborated.

    Delivering value over criticizing implementation – Whats new ? This is how it should work, Agile or not Agile.

    Influencing outcomes over telling others what’s right and what’s wrong – How do you do it without telling what is right and what is wrong.

    Preventing defects over detecting and logging – How? I believe Agile has its limitations when it comes to preventing defects as it again leads to documentation and reviews. And, Agile cannot do away without detecting defects… May be you can avoid the defect logging part.

    Listening to people over analysing quality metrics – This seems to be a dangerous trend. People can be biased while the data is always unbiased.

    Improving our trade over applying best practices – Big business is always won by showcasing the best practices. If there are no best practices, what do you have to show to your prospective customer while trying to win a new business ?

    • Aditya, thanks for tour comment.

      On your first question, what’s new? The simple answer is nothing. I am not creating anything that is not out there already. Should I?

      On the influencing outcomes point, I can’t tell you how to do it because I don’t know your situation, but it is certainly possible, again nothing new here.
      For example agile testers should be able to influence developers in the way they test without having to tell them here is it how you do it, but in showing them the benefits of good testing. Maybe coaching them when they will ask for an opinion.

      I think that the focus of traditional teams on logging defects identifying them with value is one of the fundamental points that a tester needs work on in his search for agility. Defects are waste, we prevent waste we don’t detect it and categorize it. Maybe it could say “Preventing defects over detecting them”

      On the quality metrics, I see them as a dangerous trend in particular when utilised to measure testers performance. I will agree to disagree on this.

      On best practices. This is not a guide on how to win a new business. The concept of best practices also is static and clashes with continuous improvement that is what I mean when I say improving our trade.

      Please also keep in mind what the last sentence of the manifesto says: “That is, while there is value in the items on the right, we value the items on the left more.”

  6. Gus
    Please don’t take my earlier posts as ‘this is what should be done’, they were intended to be suggestions that others may wish to build upon.

    I like both your ideas about 4 or 5 and principles and using ten principles, from ‘Agile Testing’. (Short aside – Opened my copy directly at the 10 principles page) Not sure which I prefer though. Possibly a combination of both. I’d like to hear what others think, especially as I valued notesoftester, Dereck and Marywalshe’s comments.

  7. HI johnwilsonpcms ,

    I have a deep esteem of Augusto and this is my premise.
    In regard to manifesto I want to tell my opinion and before of all i’d like to apologize with everyone for my english because i’m not fluent as you are.

    [Augusto] Your English is more than fine Giuseppe!

    Point to point, i will try to explain my ideas and I propose you my personal vision of testing.
    I’m going to summarize this vision in five points that I presented this year to my client and big boss of my company.

    Delivering value over criticizing implementation the product (*)

    The tester doesn’t deliver value . Maybe he makes it actionable and concrete . Through the requirements and the refirements of these, the tester concretizes and stabilizes the value. I believe that verbs and adjectives have to be choiced with more attention before been proposed in a definition or a manifesto, such as in this case. For my personal experience, in a manifesto, it is necessary to see and analyze what you want put in this manifesto, starting from what you want achieve from the actual testing situation.

    [Augusto] I believe an agile tester delivers value as part of the team. Delivering value must be the first focus of any agile team member. Traditional testers very often don’t feel part of the delivery team because they limit their work to criticizing the product and don’t produce value directly, I believe that this is the fundamental difference, between a traditional and an agile tester.

    Influencing outcomes over telling others what’s right and what’s wrong imposing rules (**)

    Influencing outcomes, to me, isn’t acceptable both as a verb and as the “action of the verb”. It is a contraddiction with the first sentence . You want both “ deliver a value” and “influencing outcomes”.
    Over imposing rules, to me, is predicted beacuse you work in a company and of course you must accept its rules or general rules of testing.

    [Augusto] Why limit yourself to the rules of the company? Why can’t you discover new rules, better rules? What are the general rules of testing? Can we not improve them?

    Preventing defects over detecting and logging them (**)

    If the tester’s target is to achieve the best quality it is normal to prevent defects.
    Logging them has a sense.

    Listening to people over analysing quality metrics

    Here i don’t understand what augusto wants to state . Ok, I’m listening to you and ..after? I think that in this case it is better to talk about experience . I listen to you because you are talking about your experience. Metrics is redundant to me. Rather than them i prefer to talk about “horizons of meaning” or “reference points”

    [Augusto] Maybe this could be expressed better, what I mean to say is that to understand the state of the quality of the application I would rather listen to the product owner, to the customers and to my team mates rather than looking for the answer in defect trends, defect detection rates or other quality metrics.

    Improving our trade over applying best practices
    Why trade? ( significate of “trade” :
    trade treɪd noun [uncountable] BUSINESS the activities of buying and selling goods or services )

    I don’t like to use “trade” in a manifesto. Manifesto would be inspired from freedom’s principles whitin a circle of actions steered by the rules and less related to business area.

    [Augusto] How about craft? Improving our craft over applying best practices

    Below I would like to share my revision of “testing manifesto” according to my experience and my observations. Of course , my vision can be integrated with augusto’s one as intellectual – testing – team

    Responsability and cooperative over organizations and software
    Working Value over Preventing defects and working software
    Experience and Passion over Comprehensive documentation
    Customer Interaction over Automation Software
    Shared knowledge over Best practices

    [Augusto] There are definitely some good values, I want in particular borrow the concept of Responsibility and integrate it

    Giuseppe Torchia aka notesoftester

      • Augusto, I’m sorry but i’m lengthy and this is not a challenge for me but a deep analysis of agile manifesto . This is my premise.

        [Augusto] I believe an agile tester delivers value as part of the team. Delivering value must be the first focus of any agile team member. Traditional testers very often don’t feel part of the delivery team because they limit their work to criticizing the product and don’t produce value directly, I believe that this is the fundamental difference, between a traditional and an agile tester.

        [ Giuseppe ] I have a bit of problems with “to deliver” verb. Below, you can read different means of “to deliver” :
        Definition of deliver ( mcmillan english dictionary )
        1 [countable/uncountable] the process of bringing goods or letters to a place
        2 [uncountable] the process of providing a service
        3 [countable/uncountable] the process of giving birth to a baby
        4 [uncountable] computing the process of providing information through a computer
        5 [singular] the way in which someone speaks in public
        6 [singular] the way that someone throws a ball or makes a hit in a sport
        Now, Gus , Do you tell me what is your choice ”[1..6]” see above?

        I’m going to analyze my mental steps when if i reflect in terms of value
        I build : with requirements
        I discover : with observations , analisys and study
        I put : adding experience and solution
        I realize : with understanding and knwoledge
        but I don’t “deliver value”.
        To me, the value involves concepts as morality and ideal ; for these reasons i’ve adopted the word “responsability ” (http://en.wikipedia.org/wiki/Value_%28ethics%29)
        Augusto, I would like to know what is for you “value” and if you are reefering to http://en.wikipedia.org/wiki/Value_%28economics%29 .
        Do you “criticise” / Do you “criticize” ? (They have different and profound differences )
        Maybe there is a bit of confusion among value , product and testing.
        It is not important , in this phase , to compare the traditional and the modern because both they are valid also in agile. Here, now , in this moment we are in other prospective .
        I ask you :
        where does is “the frontier” for passing from to the past through the present and to project over the future ?

        [Augusto] Why limit yourself to the rules of the company? Why can’t you discover new rules, better rules? What are the general rules of testing? Can we not improve them?

        [ Giuseppe ] Of course , you can to find better rules but you have to define the “criterion for comparison” with others. Maybe , we are making a bit of confusion among limit,rules and improving.
        General rule of testing is simply to me “ no defects “ .
        The real target of testing is “ no defects “ and I would like to share with you the methodology concept :
        “A methodology is not specific methods or techniques, however, it must specify the various processes (and their inherent logic) that must be performed through the application of processes, methods and techniques. These processes constitute a kind of conceptual framework and operational”
        Now you try this link: http://en.wikipedia.org/wiki/Methodology

        [Augusto] How about craft? Improving our craft over applying best practices

        [Giuseppe] Augusto , isn’t a “terms” challenge 🙂 . To me, it’s a resivitation contain-semantic in agile testing manifesto. Best practices are a series of experiences on the same argument . Why do you tell of craft?
        Do you supply me another synonymous of craft ?
        (If you read the agile manifesto , the first part contains only the values and in the second part there are the tools for acting it.)

        In the week end, i will write my “ Responsability and cooperative over organizations and software “

        Pep’s

        • Excellent points Giuseppe, thanks.

          [ Giuseppe ] I have a bit of problems with “to deliver” verb. Below, you can read different means of “to deliver” :
          Definition of deliver ( mcmillan english dictionary )
          1 [countable/uncountable] the process of bringing goods or letters to a place
          2 [uncountable] the process of providing a service
          3 [countable/uncountable] the process of giving birth to a baby
          4 [uncountable] computing the process of providing information through a computer
          5 [singular] the way in which someone speaks in public
          6 [singular] the way that someone throws a ball or makes a hit in a sport
          Now, Gus , Do you tell me what is your choice ”[1..6]” see above?

          [Augusto] Giuseppe, the use of a word in a specific context can refer to more than one meaning. In this case I use the term “deliver” with the meaning [1] and [2]. In all fairness, I don’t want to get bogged down into a semantic battle, the goal of the agile testing manifesto is not “being semantically perfect” but “delivering value” to testers on the road to agility.

          [Giuseppe] I’m going to analyze my mental steps when if i reflect in terms of value
          I build : with requirements
          I discover : with observations , analisys and study
          I put : adding experience and solution
          I realize : with understanding and knwoledge
          but I don’t “deliver value”.

          To me, the value involves concepts as morality and ideal ; for these reasons i’ve adopted the word “responsability ” (http://en.wikipedia.org/wiki/Value_%28ethics%29)
          Augusto, I would like to know what is for you “value” and if you are reefering tohttp://en.wikipedia.org/wiki/Value_%28economics%29 .

          [Augusto] Giuseppe, agile teams deliver working software, working software gives value to its customers, this is my train of thought. This is why I say that an agile tester, being part of an agile team delivers value. I think we might be using 2 different meanings of the word “value”.

          [Giuseppe]
          Do you “criticise” / Do you “criticize” ? (They have different and profound differences )

          [Augusto] I criticise because I learned English in Dublin (Ireland), if I did it in San Francisco, I would probably criticize 🙂 As far as I know it is simply a spelling difference between British English and American English, but I might be wrong and happy to learn something new if I was.

          [Giuseppe]
          Maybe there is a bit of confusion among value , product and testing.
          It is not important , in this phase , to compare the traditional and the modern because both they are valid also in agile. Here, now , in this moment we are in other prospective .
          I ask you :
          where does is “the frontier” for passing from to the past through the present and to project over the future ?

          [Augusto] I don’t know, maybe we will discover it through this discussion. I’m actually not sure there is one only path to agile testing, i.e. old to agile, that’s another thing we can try to discover.

          [Giuseppe] Augusto , isn’t a “terms” challenge . To me, it’s a resivitation contain-semantic in agile testing manifesto. Best practices are a series of experiences on the same argument . Why do you tell of craft?
          Do you supply me another synonymous of craft ?
          (If you read the agile manifesto , the first part contains only the values and in the second part there are the tools for acting it.)

          [Augusto] Why improving one’s craft can’t be seen as a value?

          [Giuseppe]
          In the week end, i will write my “ Responsability and cooperative over organizations and software “

          [Augusto] I look forward to reading it

      • Giuseppe
        Your English is sometimes better than mine and your ideas are good.

        Gus
        I’ll certainly contact those folk who I think may be able to assist. (May take some time – pressure of work before my holiday)

  8. I think what you are attempting to do is fantastic. Extending this further are you looking at also doing the same for the agile principles? Maybe borrow from the context driven ones? http://context-driven-testing.com/

    The one thing I am not sure of is the use of the word ‘best’ practices, it is such a loaded term. Would it better to say good practices? Or could we say processes instead? The reasoning is that I do not value best practice at all and saying that we value it at all undermines, IMO, the point being made. Using processes instead to me means that I may see some value in some processes alongside placing more value in improving our craft

    • Hi John S., thanks a million for your encouragement!

      Adding the principles had crossed my mind and was mentioned earlier in the discussion, I like the ones used by Lisa Crispin and Janet Gregory in their Agile Testing book. I see it as a future step if we ever manage to agree on a set of values, for the moment they are somewhere in the backlog 🙂
      I am familiar with the context driven school and its principles, I value them and use them, but I am not sure they would help an agile tester as much as the ones used by Lisa and Janet.

      On the best practice concept, I don’t like them because I believe that if there was a best practice there would be no need for improvement while I am a kaizen guy. At the same time I recognise some of the benefits associated to having a set of basic rules in a confined context when we are beginners in such context.

      Process is a valid alternative, excellent suggestion, I am going to introduce the change to the draft and try to understand from the feedback if people like it.

      Thanks again, and please keep on contributing!

  9. At first look, my initial reactions to this manifesto are:

    1. What is the purpose of such a manifesto?
    2. The left over right items don’t seem to be related in the way that those are in the Agile manifesto.
    3. What are the principles that go with these values? Are these 4 things really the core values of agile testing?
    4. Testing does not deliver value. It can help protect it. Testing is more like insurance than an investment.
    5. Testing the implementation does not prevent defects.
    6. This assigns value to quality metrics and best practices. I’d have to struggle to find value in most things people think of as quality metrics. I don’t find any value that I value in best practices.

    • Ben, thanks a million for your contribution, see my answers below. Let us know what you think.

      1. What is the purpose of such a manifesto?
      [Augusto] The main goal is to help new agile testers find the road to agility. The secondary (selfish) goal, is to gain a better understanding of what agile testers think it is important for them. Basically I want to learn about agile testers and help them.

      2. The left over right items don’t seem to be related in the way that those are in the Agile manifesto.
      [Augusto] Can you give me an example here please?

      3. What are the principles that go with these values? Are these 4 things really the core values of agile testing?
      [Augusto] There are no principles yes, agile testers do just in time analysis, and we didn’t get there yet 🙂 but there has been talk of including Janet Gregory and Lisa Crispin’s principles. Are these 4 things really the core values of agile testing? I don’t know? I am trying to discover it as we speak. Help us discover them!

      4. Testing does not deliver value. It can help protect it. Testing is more like insurance than an investment.
      [Augusto] I disagree on this Ben. As an agile tester I deliver value. Each user story that my team delivers contain such value. Agile teams deliver value as a whole, testers are part of the agile team.

      5. Testing the implementation does not prevent defects.
      [Augusto] I agree Ben. Agile testers don’t limit themselves to testing the implementation. Agile testers do much more than that. On my current project I normally spend 80% of my time in preventing activities and 20% in detection mode.

      6. This assigns value to quality metrics and best practices. I’d have to struggle to find value in most things people think of as quality metrics. I don’t find any value that I value in best practices.
      [Augusto] See my answer to John S. on best practices. On quality metrics, they have their place and can be used with good results in specific contexts, wanting to use them at all time and trying to read what’s not there is not good. Also, look at the agile manifesto, do you really give much of a value to comprehensive documentation and contract negotiation?

  10. My very initial comments

    It is a great start and definitely praise worthy.

    1
    “Through this work we have come to value:

    Delivering value over criticizing the product

    If we read the entire sentence – there are two back to back Values.. with different meanings..
    I believe that starting the first point with delivering Value over criticizing the product .. needs a little change may be.. Criticism of product normally happens at the end when testers get to see the application. Normally as an Agile Tester even if I have to be a critique I would do that for the user stories .. and at that time it is not a Product.
    how about
    “Understanding and Delivering Voice of Customer over being a Critque”

    More comments I would write shortly

    Best Wishes
    Adi Garg

    • Thanks a million for your feedback and support Adi.
      If you are willing to help with the project, we have created a google group where we are discussing improvements on each of the values. It is an easier way to collaborate than the blog itself as the comments are many and have become unreadable here. If you are interested and have a google account I can send you an invite. Let me know.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s