Agile Testing

Two worlds
colliding!

Declan Whelan
dwhelan@dpwhelan.com
This presentation is licensed under a Creative Commons Attribution 2.5 License, which
means you can share and adapt it, including commercial and derivative works, as long as
you include attribution to Declan Whelan.
Scrum Overview

http://www.mountaingoatsoftware.com/scrum_figures
 Scrum Team
Product
Scrum Master Team
Owner
• Feature • Represents • Self organizing
definition management • Cross-
• Release dates • Removes functional
• Single decision obstacles • Estimates
point • Ensures Scrum • Tracks
• Accepts or process • Gets ‘er done
rejects work • Servant leader
• ROI
The Team
Agile Testing Conundrums
Conundrum #1

Developers Test Their Own Code

• Unit Testing by developers is given a high profile

Oxymoron?

• This is just marking your own homework

Axiom?

• Unit Tests are just the first of many quality strategies

Source: David Evans

© SQS Group Limited | Agile Quality Management - Oxymoron or Axiom? | 6
Conundrum #2

Developers and Testers are co-dependent

• Testers and developers are encouraged to work together

Oxymoron?

• Independence is fundamental to quality governance

Axiom?

• The whole team share the goal of creating quality software

Source: David Evans

© SQS Group Limited | Agile Quality Management - Oxymoron or Axiom? | 7
Conundrum #3

Developers know the acceptance tests

• Acceptance Tests are defined and agreed up-front

Oxymoron?

• They will only write code to make the tests pass!

Axiom?

• They won’t write code that makes the tests fail

Source: David Evans

© SQS Group Limited | Agile Quality Management - Oxymoron or Axiom? | 8
Conundrum #4

No Signed-Off Requirements

• Requirements seem to change with the weather

Oxymoron?

• I can’t possibly certify that the requirements have been met

Axiom?

• Each requirement is defined by its acceptance tests:

these are negotiated with and confirmed by the customer

Source: David Evans

© SQS Group Limited | Agile Quality Management - Oxymoron or Axiom? | 9
Conundrum #5

No Test Plan

• IEEE829 Test Plans are in short supply on agile projects

Oxymoron?

• Failing to plan is planning to fail! How will we know what to test?

Axiom?

• The Test Plan is implied in the product backlog:

everything we build, we test

Source: David Evans

© SQS Group Limited | Agile Quality Management - Oxymoron or Axiom? | 10
Conundrum #6

No Test Manager (or Test Management Tool)

• ‘Test Manager’ is not a role that exists in Agile literature

Oxymoron?

• So how can we possibly manage testing?

Axiom?

• Testers == team; tests == specifications;

...they don’t need separate management

Source: David Evans

© SQS Group Limited | Agile Quality Management - Oxymoron or Axiom? | 11
 What is Quality?

Jerry Weinberg:

“Value to some person(s).”

 Testing Schools of Thought
• How much has • Controlled process • Focus on QA vs.
been tested? • Formal rules testing
• Objectivity • Standards & • Process
• Branch of certification watchdog
CS/Math • Conformance focus • Risk driven
• QA signoff
Analytic Factory Quality

• Skilled mental activity

• People set context
Agile Helps
• Testing provides information
Set Context • Value driven

Context Driven

Source: http://www.qaiasia.com/bpw/innerpages/documents/Process_Improvement/Model_and_Framework/PI_007-shrinivas_kulkarni.pdf
 Quality via “Done, Done”

Designed Refactored
Coded Checked In
Documented
Unit Tested
CI Built
Acceptance Tested Explored
 Key Agile Testing Practices

Source: Elizabeth Hendrickson - http://testobsessed.com/wordpress/wp-content/uploads/2009/10/atqlblf-pnsqc2009.pdf

 Why Automate Tests?
• Provides safety net
• Supports rapid iterations
• Provides footholds to keep notching upward
• Provides rapid feedback
• Focuses effort on what is valuable
• Frees people to do their best work

Need to balance automation costs against delivered value

Don’t automate the
communication!
Use a Low-Tech Testing Dashboard

Check out
http://www.satisfice.com/presentations/dashboard.pdf
for ideas on building a low test testing dashboard.
 Brian Marick’s Agile Testing Matrix
Automate Customer Facing Manual

Functional Tests User Acceptance Tests

Customer Tests Exploratory Tests
Supports Development

Critiques Product
Story Tests/Examples Usability Tests
Q2 Q3

Q1 Q4

Unit Tests Performance Tests

Integration Tests Load Tests
Automate Technology Facing Tools
 Tester Activities
Product Owner Customer
Customer Facing
Collaboration Collaboration

Product Specifications UAT Design

Supports Development

Test Ideas Exploratory Testing

Critiques Product
Exploratory Testing Q2 Q3 Usability Testing
Q1 Q4
Automation Support Test Scripts
Unit Testing Testing
Test Analysis
Developer IT
Technology Facing
Collaboration Collaboration
 System Tests Unit Tests

Sprint 3 Sprint 4 Sprint 5

Writing stories
Acceptance tests
Exploratory Acceptance
Test ideas
Testing Tests
 Testing Pyramid
TCO - $

% automated work
Source: Patrick Wilson Welsh - http://patrickwilsonwelsh.com/?p=32
A Common Reality

Source: Patrick Wilson Welsh - http://patrickwilsonwelsh.com/?p=32

 What is a Bug?

“In an Agile context, I define a

bug as behavior in a “Done”
story that violates valid
expectations of the Product
Owner.”

Elizabeth Hendrickson
Source: http://testobsessed.com/2009/03/13/handling-bugs-in-an-agile-context/
 Handling Bugs

Find

Verify Agile Team Analyse

Fix
 Defect Tracking Systems

High performing agile

teams don’t need them!
 Defect Tracking Systems
• If we honour “Done, Done” we just fix them now!
• If a finding is deemed low priority and there are
others things of higher priority then add it to
product backlog
• If we are finding, fixing and defects quickly then
clearly our process works so why do we need a
system to tell us that?
• If a system is for CYA then the team has deeper
problems

Source: http://testobsessed.com/2009/03/13/handling-bugs-in-an-agile-context/
 Agile Testing Success Factors
Testers are part • Collective ownership
of the team • Sit together!

• Information providers not “Quality Police”

Agile testing • Focus on team goals & customer value
mindset • Rapid, high bandwidth feedback

• Automate tests wherever practical

Automate tests • Need rapid feedback

Look at the big • Balance against developer focus on

technical implementation
picture • Use agile test matrix as guide

Source: http://www.agiletester.ca/
 Agile Testing Success Factors
• Session-based testing
Foundation of • Test automation
critical practices • Informative workspace

• Collaborate with customers

Collaborate • Collaborate within team

• Team retrospectives
Continually
• Personal training: reading, blogs,
improve local QA groups, session debriefs

Source: http://www.agiletester.ca/
 Agile Testing Books
• Agile Testing: The Tester Role on an Agile
Project
Lisa Crispin, Janet Gregory

• Test Driven: Practical TDD and Acceptance

TDD for Java Developers
Lasse Koskela
 Web Blog Sites
• http://www.exampler.com (Brian Marick)
• http://testobsessed.com (Elizabeth Hendrickson)
• http://www.developsense.com (Michael Bolton)
• http://www.satisfice.com (James Bach)
• http://www.kohl.ca/blog (Jonathon Kohl)
• http://blogs.stpcollaborative.com/matt (Matt Heusser)
• http://www.agiletester.ca (Lisa Crispin, Janet Gregory)