Agile Test Strategy: Paul Gerrard

@paul_gerrard
Agile Test Strategy

Paul Gerrard
paul@gerrardconsulting.com
gerrardconsulting.com
Overview
What is Agile Test Strategy?
Project Profiling
(Test Strategy as) Agile Interventions
Test Automation
Whats Left?
Summary
Q&A
Intelligent Definition and Assurance Slide 2

What is Agile Test
Strategy?
Agile Strategy an oxymoron?

Agile Test Strategy
What do we mean by this?
1. AGILE Test Strategy how to create a test
strategy in an Agile way?
2. AGILE Test Strategy a test strategy for an
Agile project?
Well look at how we created an Agile

approach to strategy, but well spend more
time on strategy for an Agile project.

Google Agile Test Strategy
There are plenty of recipes out there
Most offer a selection of techniques but dont
provide much guidance on how to blend them
We need to know how to make choices, not
just know what choices exist
Strategy is a thought process, not a document

Although you might document the ideas for reuse, as
a reminder or for the record.

Agile governance
Governance is the act of governing. It relates
to decisions that define expectations,
grant power, or verify performance
Wikipedia
Define expectations DEFINITION of need
Grant power DELEGATION to a project
team
Verify performance ASSURANCE of solution.

Strategy helps you decide what to do
A. The strategy presents some decisions that can be
made ahead of time
B. Defines the process or method or information that
will allow decisions to be made (in project)
C. Sets out the principles (or process) to follow for
uncertain situations or unplanned events
In a structured/waterfall environment, most questions

answered off-the-shelf A-style, ready for anything
In an Agile environment might have some ready-made
policies but we manage scope and adapt (mostly C?)
Contexts of Test Strategy
Axioms Early Communication
Testing De-
Risks
Duplication
Test Opportunities
Goals
Strategy
Culture Automation
Contract User
Human involvement
resource Constraints
Skills
Artefacts
Environment Process
(lack of?) Timescales
Traditional v Agile test strategy
Traditional structured, goal/risk-driven
Identify stakeholders; what are their goals?
Product risk analysis
Allocate risks/goals to test stages
Formulate test stage definitions (entry/exit criteria,
environments, tools etc. etc.
Agile interventionist, consensus-driven
Project profiling to set the testing theme
Identify testing interventions (perhaps better,
contributions) in the Agile process
Test policy overlays the process; catches exceptions.

Project Profiling
Select a profile for your project first,

then choose the aspects of test
strategy that suite your project
Template-driven? Bah!
So this is just a template copy and edit process?
Wont you always end up with the same document?
Profiling doesnt need to be prescriptive

No need to write a document if you dont need to
But if company policy or common sense dictates certain
approaches, save yourself some time
Create a set of deeper, more detailed questions to be
answered (Pocketbook)
Profilers are really just checklists: heuristic guidelines
designed to help you make choices and trade-offs.

Incident Mgt.
Using a Project Profiler to Derive Project Plan Tools
a Test Strategy and Project Plan Story Guideline
(A government client example) Test

Environment
Project Profiler
Cerise Waterfall
Test Plan Test Strategy
Project Manager Items
Orange
Risk Profiler
SCRUM/Agile
Consultation
Product Test Strategy

Green
Risks
Assurance
Unknowns
The Project Profiler (with Test Assurance) helps

Blue Project Managers to:
Select a project style that fits (Waterfall or Agile)
Identify the product risks that need testing
Identify test activities to include in project plans
Business, Project Team
Carefully define the scope of the project
and Boards
Intelligent Definition and Assurance 12
Project profiling process
Task
1 Have the Information you need to hand
2 Project Profiler (section 3):
Select the statements that best match your project context. The Blue column indicates
that you need more information consult your stakeholders, team or relevant Board(s).
3 Generic Risk Profiler (section 4):
Consider the generic project risks which are significant for your project? Can testing
help?
4 Product Risk Profiler (Section 5):
Consider the functional and non-functional risks that most concern your stakeholders
should they be tested?
5 Actions and Test Activities (Section 6):
Consider the actions that prompt your next steps and the test activities that should be
incorporated into your project plan.
6 Create your Test Strategy from the Test Strategy Framework Template
7 Incorporate the activities from stage 5 and identified in 6 into your Project
Plan.
Project Profiler (part of section 3)
Project Aspect Cerise Orange Green Blue
Responsibility for Users will take Users will be responsible Users will take part in UAT Users are unwilling/unable
Acceptance responsibility for UAT; for UAT but have no test or witness tests at critical to take part in UAT;
they have UAT experience experience periods, and will review reluctant to make the
the outcome acceptance decision or not
known
Requirements (Sources New system replaces a Users want to collaborate Users put the onus of Inexperienced users who
of Knowledge) well-understood existing to jointly define requirements elicitation on are unable or unwilling to
system; users have clear requirements and meet the project; requirements collaborate with
vision of system goals and them incrementally or and the solution will evolve requirements gathering
prefer to document their iteratively
requirements up-front
Requirements Stability New system is a functional New system replaces an New system supports a New system supports a
replacement of an existing existing system with new business need; new business need;
system or a well-defined enhancements or an business process exists business process is not
process (requirements can established (but not but will change/evolve; yet known; users have no
be fixed early on) necessarily documented users have experience of experience or
process) requirements requirements
Visibility, Formality High visibility/risk to High visibility/risk to Relatively low business- Potentially, high visibility,
general public; formal business; formal progress risk; informal progress high risk project, uncertain
progress reporting reporting required; some reporting is acceptable; impact on the business
required at board level; defined deliverables, some partial solution may
fixed scope and deliverables will suffice,
deliverables; formal emerge/evolve; some incremental/iterative
approvals and sign-offs approvals and sign-offs delivery
External Dependencies More than one or new Single, known supplier In-house development, no Dependencies on external
external suppliers responsible for external dependencies suppliers, their
responsible for development (and supplier responsibilities or
development (and supplier testing) competence not yet known
testing)
Etc. Etc. Etc. Etc. Etc.

Project types - examples
Cerise Structured, waterfall style of project (and includes
COTS projects)
Orange Iterative/prototyping style of project using SCRUM in
a formal way and having dedicated resources for the
Business Analyst and Tester roles.
Green A project using SCRUM in a less formal way but not
having dedicated resources for the Business Analyst
and/or the Tester roles.
Blue Blue column statements describe where there is
insufficient information available to identify the style
of project and the recommendation must be that
some further investigation is required.
(Test Strategy as)
Agile Interventions
Im using Scrum/Sprint terminology,

but you dont have to of course
Interventions (government client example)
No. Activity When?
On the following 1 Story Challenge As stories are added to the

Product Backlog
slides, we highlight
2 Story Definition As stories are added to a
Sprint Backlog
3 Daily Stand-Up Once per day during the
8 interventions
These activities are repeated

Sprint
for each Sprint iteration

4 Story Refinement Occurs throughout the Sprint
Some are test

as new information emerges
5 Developer Testing Occurs throughout the Sprint
as the developer codes the
phases, but some 6 Integration (and
stories
During and at the end of
arent incremental
System) Testing
each sprint, including the
final sprint
7 System Testing At the end of each sprint,
including the final sprint
8 User Acceptance At the end of each sprint,
Testing including the final sprint
9 Non-functional Expected to take place on an
Testing and Pre- as-needs basis.
Production Testing

1. Story Challenge
Suggest what-ifs to
Project Level Test Activities
(This diagram shows three sprints, but there could be more or fewer)
challenge new stories
and define story
headlines Sprint Backlog Sprint Backlog Sprint Backlog
2. Story Definition
Introduce scenarios
to enhance the
Acceptance Criteria
Sprint 1 Sprint 2 Sprint 3

Developed Stories Developed Stories Developed Stories
New Code
Integration into 6. Integration Test 6. Integration Test 6. Integration Test

Existing Code base
Automated testing
Increasing
Scope of 7. System Test
Sys. Test
and UAT 8. User Test
Increasing Scope of Integration, Complete Tests after

System and Users Testing Final Sprint
1. Story Challenge
Suggest what-ifs to
and define story
2. Story Definition
Introduce scenarios
to enhance the
Acceptance Criteria

New Code

Existing Code base
Automated testing
Increasing
Sys. Test

1. Story Challenge
Suggest what-ifs to
and define story
2. Story Definition
Introduce scenarios
to enhance the
Acceptance Criteria

New Code

Existing Code base
Automated testing
Increasing
Sys. Test

1. Story Challenge
Suggest what-ifs to
and define story
2. Story Definition
Introduce scenarios
to enhance the
Acceptance Criteria

New Code

Existing Code base
Automated testing
Increasing
Scope of
7. System Test
Int. Sys.

1. Story Challenge
Suggest what-ifs to
and define story
2. Story Definition
Introduce scenarios
to enhance the
Acceptance Criteria

New Code

Existing Code base
Automated testing
Increasing
Scope of
7. System Test
Int. Sys.

Test Activities in the Sprint
3. Daily Stand-Up
Report anomalies found, 5) Developer Testing
Daily Scrum Private ad-hoc tests and
stories tested, amended,
Stand-Up build/run automated unit
created
Meeting 24 tests
Hours
4. Story Refinement 6) Integration/System Testing
Refine scenarios to Incorporate automated unit
enhance story definition, tests into the CI regime.
create system tests as On weekly basis and at end of
stories, as required Sprint, deploy to System test
environment and tester runs
2-4 Weeks system tests.
Backlog tasks
expanded
by team
Sprint Backlog
Potentially Shippable
Product backlog Product increment
As prioritised by Product Owner

3. Daily Stand-Up
created
Meeting 24 tests
Hours
Backlog tasks
expanded
by team
Sprint Backlog

3. Daily Stand-Up
created
Meeting 24 tests
Hours
Backlog tasks
expanded
by team
Sprint Backlog

3. Daily Stand-Up
created
Meeting 24 tests
Hours
Backlog tasks
expanded
by team
Sprint Backlog

4. Story Refinement (example definition)
Objectives To define acceptance criteria for all stories that are included in a Sprint as they are worked
on by development
To define scenarios that describe the tests and expected behaviours of the System
Improve understanding of the requirement and communicate anomalies to developers
To identify System Tests that exercise functionality of multiple stories that can be system
tested in this sprint
To assure the completeness for stories in the current Sprint
Whats being tested? Stories to be included in the current Sprint
Deliverables Refined story definitions with defined acceptance criteria and scenarios, where appropriate
System tests
Responsibilities (Orange) Tester challenges stories by suggesting potential scenarios, new stories, story merges and
splits; performs ad-hoc testing with/on behalf of developers; assures completeness of
stories.
Developers considers stories, evaluates impact on development
Product Owner or Analyst collates feedback and decisions on stories
Product Owner approves changes to stories, accepts completeness of stories
Scrum Master monitors progress; evaluates impact on resources and schedules
Responsibilities (Green) Not performed in Green projects
Baseline Story Guideline (reference 3)
Entry Criteria On commencement of the Sprint
Exit Criteria When all stories within a Sprint are considered complete
Queries, anomalies, discrepancies and inconsistencies have been eliminated or explained
System Tests appropriate to the Sprint have been defined
Definition of acceptance is agreed with Product Owner

Test Automation
Could you create an Agile Test

Strategy without automation?
Brian Maricks Testing quadrants

Test Automation Pyramid Lisa
Crispins version (Google others)
Pyramid reflects the
relative numbers of tests
Focus on
unit/component
Acceptance of Services
GUI are end-to-end
Manual checking the
exception?

Where do you automate?
3. Non-Technical
testers write scripts Stories/
Scenarios
Tools Experts write
interface Test Code 4. Programmers
write low level HTTP
GUI Test Framework GET/POST calls
through a driver that
simulates a browser
GUI Test Tool Unit Test Framework
2. Technical Testers
code scripts directly
Browser HTTP Driver 1. Programmers
write unit tests or
execute embedded
HTTP/S HTTP/S unit tests using a
unit test framework
to test components
Inter/Intranet
HTTP/S
Test Code
Web Server
App. Server Unit Test Framework

API
DB Server
Distributed testing
Use business stories and scenarios/acceptance
criteria to validate requirements
Reuse those stories to feed Acceptance-
Driven Development BDD/TDD process
Automated tests are an Anti-Regression tactic
Automated tests dont replicate manual tests;
think of them as a change trip-wire that
triggers an alarm, if tripped.

Deriving scenarios to test
To understand feature scope? Story Challenge
To get stakeholder to accept? Story Refinement
To validate the requirement? Story Definition
To estimate the work to build this feature?

Iteration Planning
To system test this feature? System Testing
To unit test this feature? Developer Testing
Scenarios are created to meet several goals

Whats Left?
Other aspects of test policy
Definitions (of done etc.)
Incident management
Test automation
Story format e.g. Gherkin
Environment request and management
Regression testing (at what levels)
Test deliverables and documentation.

The Three Amigos
Business Analyst
Liaises and manages stakeholders and their needs
Transforms business requirements into
specification (at multiple levels)
Developer
Scopes, designs, builds, tests and delivers features
Tester
Challenges the thinking on the project
Performs Assurance in the small.

The testers contribution to testing
_________ feature/story acceptance criteria
_________ the developers to unit test (auto)
_________ feature/story acceptance
_________ acceptance test
_________ service/UI level automation
Scope: from low-level detail to system integration
Liaison with integration testers and feedback
Fill in the blanks yourself; negotiate with your

team.
Summary
Close
Agile test strategy has its place and many aspects
of test can be pre-defined
Importantly, we use a principled approach to deal
with the unexpected
Project profiling can help
Testing as interventions, rather than test phases
The testing role is split at least three ways the
tester doesnt own testing think TESTMASTER
Test automation in the context of Specification by
Example, requirements validation, BDD, TDD.

@paul_gerrard
Agile Test Strategy
Paul Gerrard
paul@gerrardconsulting.com
gerrardconsulting.com

Agile Test Strategy: Paul Gerrard

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Agile Test Strategy: Paul Gerrard

Uploaded by

Copyright:

Available Formats

@paul_gerrard

Agile Test Strategy

Intelligent Definition and Assurance Slide 2

Agile Strategy an oxymoron?

Well look at how we created an Agile

Intelligent Definition and Assurance Slide 4

Strategy is a thought process, not a document

Intelligent Definition and Assurance Slide 5

Intelligent Definition and Assurance Slide 6

In a structured/waterfall environment, most questions

Intelligent Definition and Assurance Slide 9

Select a profile for your project first,

Profiling doesnt need to be prescriptive

Intelligent Definition and Assurance Slide 11

Using a Project Profiler to Derive Project Plan Tools

a Test Strategy and Project Plan Story Guideline

(A government client example) Test

Product Test Strategy

The Project Profiler (with Test Assurance) helps

Intelligent Definition and Assurance 14

Im using Scrum/Sprint terminology,

On the following 1 Story Challenge As stories are added to the

These activities are repeated

for each Sprint iteration

Some are test

Intelligent Definition and Assurance Slide 17

Sprint 1 Sprint 2 Sprint 3

Integration into 6. Integration Test 6. Integration Test 6. Integration Test

Increasing Scope of Integration, Complete Tests after

Sprint 1 Sprint 2 Sprint 3

Integration into 6. Integration Test 6. Integration Test 6. Integration Test

Increasing Scope of Integration, Complete Tests after

Sprint 1 Sprint 2 Sprint 3

Integration into 6. Integration Test 6. Integration Test 6. Integration Test

Increasing Scope of Integration, Complete Tests after

Sprint 1 Sprint 2 Sprint 3

Integration into 6. Integration Test 6. Integration Test 6. Integration Test

Increasing Scope of Integration, Complete Tests after

Sprint 1 Sprint 2 Sprint 3

Integration into 6. Integration Test 6. Integration Test 6. Integration Test

Increasing Scope of Integration, Complete Tests after

Intelligent Definition and Assurance 23

Intelligent Definition and Assurance 24

Intelligent Definition and Assurance 25

Intelligent Definition and Assurance 26

Intelligent Definition and Assurance 27

Could you create an Agile Test

Intelligent Definition and Assurance 29

Intelligent Definition and Assurance 30

App. Server Unit Test Framework

Intelligent Definition and Assurance 32

To get stakeholder to accept? Story Refinement

To validate the requirement? Story Definition

To estimate the work to build this feature?

To system test this feature? System Testing

To unit test this feature? Developer Testing

Scenarios are created to meet several goals

Intelligent Definition and Assurance Slide 35

Intelligent Definition and Assurance Slide 36

Fill in the blanks yourself; negotiate with your

Intelligent Definition and Assurance Slide 39

You might also like