EDS ISTQB TFC v2.6

03-23-05
Version 2.5
EDS ISTQB Testing Foundation Course
Presentation by Paul Weymouth, UK Testing ADU
Slide 1 EDS Internal

How to Navigate the Course
Click on the symbol to go to the next slide.
Click on the symbol to go to the previous slide.
Click on the symbol to go to the main Areas Covered slide.
Click on the symbol to move on to associated slides.
Click on the symbol to go back to the previous menu.
You may also see the following symbols displayed :-
EDS EDS specific information available used where an EDS exceptions exist.
If this appears top right, it means animation applies to the slide. It will also appear
bottom right after the final animation mouse click. For Tutors use only.
EDS ISTQB Testing Foundation Course Version 2.5

Slide 2 EDS
Other Symbols
Refers to the ISTQB Glossary Of Terms.

You will see this symbol appear alongside the
definition of a term found in this Glossary
These key terms are important to learn.
This is a Pearl of Wisdom. Typically a quote

from an external testing guru, that helps to re-
enforce what we have just learned. Youll find
a few of these dotted around the course.

Slide 3 EDS
Objectives
The primary objective of this course is to provide testing foundation

level training to staff involved in testing related activities (I.e.
developers, testers, managers)
It is anticipated that after taking this course staff will be in a position to
take and pass the external ISTQB Foundation exam in Software Testing

Slide 4 EDS
Areas covered
Fundamentals of testing
Testing Throughout the Software Lifecycle
Static Techniques
Test Design Techniques
Test Management
Tool Support for Testing
Index of key terms

Slide 5 EDS
03-23-05
Version 2.5
Fundamentals of Testing

Fundamentals of Testing
Areas Covered
Why Testing is Necessary

What is Testing
General Principles of Testing
Fundamental Test Process
The Psychology of Testing
Summary

Slide 7 EDS
Software Systems Some context

Software Systems are now part of our everyday life
They are used almost everywhere, for example in:
- Banking and Financial institutions
- Retail industry
- Central and Local Government
- Transport (e.g. Planes, Trains and Automobiles)
- Medicine (Hospitals, research centres)
- Home Entertainment
We have all experienced Software Systems failing!

Slide 8 EDS
Software Systems When things go wrong

Software System Failures can lead to:
Human Injury or Death
e.g. airplanes crashing
Technological disasters
e.g. Missile Systems malfunctioning
Legal action and associated costs
e.g. failure to meet contractual obligations
Loss of face for suppliers and/or their
customers;
e.g. mis-spelling company name on mail shots

Slide 9 EDS
Causes of Software Failure

A Human can make an Error (aka a Mistake)
An Error is A Human Action that produces an Incorrect Result
The Error can cause a Defect (aka a Fault or Bug)
A Defect is A flaw in a component or system that can cause the
component or system to fail to perform its required function
A Defect can be in the Software, System or in a Document
* ISTQB Standard Glossary of Terms

Slide 10 EDS
Errors, Defects and Failures

Defects occur because human beings are fallible
Also because of:
time pressure
complex code
complex infrastructure
changed technologies
and/or many system interactions

Slide 11 EDS

A Defect may result in a Failure
A Failure is a Deviation of the component or system from its

expected delivery, service or result
Failures can be caused by environmental conditions as well
E.g. radiation, magnetism, electronic fields
Pollution can cause faults in firmware or influence the execution of
software by changing hardware conditions.
* ISTQB Standard Glossary of Terms

Slide 12 EDS

a human action that produces
Error an incorrect result
Can manifest as
A flaw in a component or
Defect system that can cause the
component or system to fail to
perform its required function
May result in
Deviation of the component or

Failure system from its expected
delivery, service or result

Slide 13 EDS
The Role of Testing

Rigorous testing of systems and documentation can:
reduce the risk of problems occurring in an operational environment
contribute to the quality of the software system
How? By finding and correcting defects before the system is released
for operational use
Software testing may also be required to meet contractual or legal
requirements, or industry-specific standards

Slide 14 EDS
Testing and Quality

Testing can help us measure the Quality of software
Quality - The degree to which a component, system or process
meets specified requirements and/or user/customer needs and
expectations
This is measured in terms of defects found
Defects covering:
functional software requirements and characteristics
and non-functional software requirements and characteristics (e.g.
reliability, usability, efficiency, portability and maintainability)
Testing can give confidence in the Quality of the software if it finds
few or no defects

Slide 15 EDS
Testing and Quality

A properly designed test that passes, reduces the overall level of Risk
in a system
Risk A factor that could result in future negative consequences;
usually expressed as impact and likelihood
When testing does find defects, the Quality of the software system
increases when those defects are fixed
The Quality of systems can be improved through Lessons learned
from previous projects
Analysis of root causes of defects found in other projects can lead to
Process Improvement
Process Improvement can prevent those defects reoccurring
Which in turn, can improve the Quality of future systems
Testing should be integrated as one of the Quality assurance
activities

Slide 16 EDS
Testing Terminology
Testing Pearl of Wisdom
Quality is not intangible.

The purpose of testing is to make quality visible.
Testing is the measurement of software quality !"
Bill Hetzel 1988

Slide 17 EDS
Why exhaustive testing is impossible
Exhaustive testing of complex software applications:

requires enormous resources
is too expensive
takes too long
It is therefore impractical
Need an alternative that is pragmatic, affordable, timely and
provides results

Slide 18 EDS
Testing Terminology
In any form of testing it is impossible to achieve

total confidence.
The only exhaustive testing there is
is so much testing, that the tester is exhausted !"
Bill Hetzel 1988

Slide 19 EDS
Why dont we test everything ?

System has 20 screens
Average 4 menus / screen
Average 3 options / menu
Average of 10 fields / screen
2 types of input per field
Around 100 possible values
Approximate total for exhaustive testing

20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests
Test length = 1 sec then test duration = 17.7 days

Test length = 10 sec then test duration = 34 weeks
Test length = 1 min then test duration = 4 years
Test length = 10 mins then test duration = 40 years!

Slide 20 EDS
So, How Much Testing is Enough?

Deciding how much testing is enough should take
account of:
the level of Risk
project constraints such as time and budget
Risks should be evaluated at the Business Level,
Technological Level, Project Level and Testing Level
Risks are also used to decide where to start testing
and where more testing is needed
Risk considerations can include:
financial implication of software being released that is
un-tested (support costs / possible legal action)
software being delivered late to market
potential loss of Life (safety critical systems)
potential loss of face (may have financial implications
as well)

Slide 21 EDS

Risk analysis should be used to determine what to test in each
component and just as importantly what not to test
For example, an unacceptable risk would say we must test, an
acceptable one perhaps not to test
Testing is a risk-control activity that provides feedback to the
stakeholders
With this feedback the stakeholders can make informed decisions
about the release of the software (or system) being tested
More about Risks later in the Course

Slide 22 EDS
Why Testing Is Necessary

Exit criteria is used to determine when testing at any stage is
complete
The set of generic and specific conditions, agreed upon with the stakeholders,
for permitting a process to be officially completed
Exit criteria may be defined in terms of :-

Thoroughness i.e. coverage or requirements
cost or time constraints
percentage of tests run without incident
number of faults remaining

Slide 23 EDS
What is Testing?
A Definition (and a Misconception)

When asked, people often think that Testing only consists of running
tests, i.e. executing the software
Testing The process consisting of all life cycle activities, both static
and dynamic, concerned with planning, preparation and evaluation of
software products and related work products to determine that they
satisfy specified requirements, to demonstrate that they are fit for
purpose and to detect defects.
Test execution is only a part of testing, but not all of the testing
activities
Test activities exist before and after test execution

Slide 24 EDS
What is Testing?
A Definition (and a Misconception)

Activities such as:
Planning and control
Choosing test conditions
Designing test cases
Checking results
Evaluating completion criteria
Reporting on the testing process and system under test
Finalizing or closure (e.g. after a test phase has been completed)
Testing also includes reviewing of documents (including source code)
and static analysis

Slide 25 EDS
What is Testing?
Any activity that is undertaken with the

objective of helping us to evaluate or measure
an attribute of our software should be
considered a testing activity
Hetzel 1998

Slide 26 EDS
What is Testing?
Test Objectives
There are different test objectives:
To find defects
To gain confidence about the level of quality and to provide information
To prevent defects
Both dynamic testing and static testing can be used as a means for
achieving these objectives
They provide information in order to improve:
The system to be tested
The development and testing processes
Live operations (e.g. how long it takes for a process to run)

Slide 27 EDS
What is Testing?
Test Objectives
By designing tests early in the project life cycle it can help to prevent
defects from being introduced into code
Reviews of documents throughout the lifecycle (e.g. requirements and
design) also help to prevent defects appearing in the code. More
about this when we cover Static techniques

Slide 28 EDS
What is Testing?
Test Objectives cost of fault correction
Relative
Multiples

Slide 29 EDS
What is Testing?
Test Objectives
The Objectives of testing can vary depending on the stage of testing
being conducted. E.g.:
Development testing (e.g. component, integration and system testing) -

to cause as many failures as possible so that defects in the software are
identified and can be fixed
Acceptance testing - to confirm that the system works as expected, to
gain confidence that it has met the requirements
Maintenance testing - often includes testing that no new errors have been
introduced during development of the changes
During Operational testing - may be to assess system characteristics such
as reliability or availability

Slide 30 EDS
What is Testing?
Test Objectives
In some cases the main Objective of testing may be to assess the

quality of the software (with no intention of fixing defects), to give
information to stakeholders of the risk of releasing the system at a
given time
More on these test stages later in the course

Slide 31 EDS
What is Testing?
Testing is the process of executing a program or

system with the intent of finding errors
Myers 1979
"Testing is any activity aimed at evaluating an
attribute or capability of a program or system and
determining that it meets its required results "
Hetzel 1988

Slide 32 EDS
What is Testing?
Testing v Debugging
Debugging and testing are different:
Testing can show failures that are caused by
defects
Debugging identifies the cause of a defect,
repairs the code and checks that the defect
has been fixed correctly
Testing then ensures that the fix does
indeed resolve the failure
The responsibility for each activity is very
different, i.e.
Testers test
Developers debug

Slide 33 EDS
General Testing Principles
The Seven Key Principles
1. Testing shows presence of Defects

2. Exhaustive Testing is Impossible!
3. Early Testing
4. Defect Clustering
5. The Pesticide Paradox
6. Testing is Context Dependent
7. Absence of Errors Fallacy

Slide 34 EDS

1. Testing shows the presence of Defects
We test to find Faults (a.k.a Defects)

As we find more defects, the probability of undiscovered defects
remaining in a system reduces.
However Testing cannot prove that there are no defects present

Slide 35 EDS
Why Testing is necessary
Testing Pearls of Wisdom
The probability of the existence of more errors in a

section of a program is proportional to the number of
errors already found in that program
Do not plan a test effort under the tacit assumption
that no errors will be found
A good test is one that has a high probability of
detecting an as yet undiscovered error
A successful test is one that detects an as-yet
undiscovered error
Myers 2004
Slide 36 EDS

2. Exhaustive Testing is Impossible!
We have learned that we cannot test everything (i.e. all combinations

of inputs and pre-conditions).
That is we must Prioritise our testing effort using a Risk Based

Approach.

Slide 37 EDS

3. Early testing
Testing activities should start as

early as possible in the
development life cycle
These activities should be focused
on defined objectives outlined in
the Test Strategy
Remember from our Definition of
Testing, that Testing doesnt start
once the code has been written!

Slide 38 EDS

4. Defect Clustering
Defects are not evenly spread in a system

They are clustered
In other words, most defects found during
testing are usually confined to a small number of
modules
Similarly, most operational failures of a system
are usually confined to a small number of
modules
An important consideration in test prioritisation!

Slide 39 EDS

5. The Pesticide Paradox
Testing identifies bugs, and programmers respond to fix

them
As bugs are eliminated by the programmers, the software
improves
As software improves the effectiveness of previous tests
erodes
Therefore we must learn, create and use new tests based
on new techniques to catch new bugs
N.B It's called the "pesticide paradox" after the agricultural

phenomenon, where bugs such as the boll weevil build up tolerance
to pesticides, leaving you with the choice of ever-more powerful
pesticides followed by ever-more powerful bugs or an altogether
different approach. Beizer 1995

Slide 40 EDS

6. Testing is Context Dependent
Testing is done differently in different contexts

For example, safety-critical software is tested differently from an e-
commerce site
Whilst, Testing can be 50% of development costs, in NASA's Apollo
program it was 80% testing
3 to 10 failures per thousand lines of code (KLOC) typical for
commercial software
1 to 3 failures per KLOC typical for industrial software
0.01 failures per KLOC for NASA Shuttle code!
Also different industries impose different testing standards

Slide 41 EDS

7. Absence of Errors Fallacy
If we build a system and, in doing so, find and fix defects ....
It doesnt make it a good system
Even after defects have been resolved it may still be unusable and/or
does not fulfil the users needs and expectations

Slide 42 EDS
The five stages of the fundamental test process
Test Planning and Control
Test Analysis and Design
Test Implementation and

Execution
Evaluating Exit Criteria and

Reporting
Test Closure Activities

Slide 43 EDS

The process always starts with planning and ends with test closure
activities
Each phase may have to be executed a number of times in order to
fulfil exit or completion criteria
Although logically sequential, the activities in the process may
overlap or take place concurrently

Slide 44 EDS

Test Planning
Specifies how the test strategy and project Test Plan
A document describing the scope, approach, resources and schedule of
intended test activities
apply to the software under test
Principally:
verify the mission
define the Test objectives
Specify the Test Activities required to meet the mission and objectives

Slide 45 EDS

Test Planning (continued)
Major Tasks are :-
Identify the objectives of testing
Determine Scope
Determine the Test Approach
Determine the required test resources
Implement the test policy and/or the test strategy
Schedule test analysis and design tasks
Schedule test implementation, execution and evaluation
Determine the Exit Criteria
More on Test Planning in Test Management section

Slide 46 EDS

Test Control
The ongoing activity of comparing actual progress against the plan
Reporting status, including deviations from the plan
Taking actions necessary to meet the mission and objectives of the
project
Test Planning takes into account the feedback from monitoring and
control activities.
Major Tasks are :-
measure and analyse results
Monitor and document progress, test coverage and exit criteria
initiate corrective actions
make decisions
More on Test Control in Test Management section

Slide 47 EDS

General testing objectives are transformed into tangible Test
Conditions (An item or event of a component or system that could be verified by one
or more test cases, e.g. a function, transaction, feature, quality attribute, or structural
element)
and Test Designs (A document specifying the test conditions (coverage items) for a
test item, the detailed test approach and identifying the associated high level test cases).
Tests should be designed using the test design techniques selected

in the test planning activity
Major tasks are:
Review the test basis
From Analysis of test items, identify Test Conditions/Requirements and
required Test Data
Design the tests (note the detail, in the form of a Test Case, is
developed in the next stage)
Evaluate testability of the requirements and system
Design the test environment set-up
Identify any required infrastructure and tools

Slide 48 EDS
The act of designing tests is one of the most

effective error prevention mechanisms known
... The thought process that must take place to create
useful tests can discover and eliminate problems at
every stage of development.
Beizer 1983

Slide 49 EDS
Test Implementation and Execution

Test Conditions are transformed into Test Cases and Testware
The test environment is created
Major tasks are:
Create Test Cases
Develop and prioritise Test Cases
Create test data
Write test procedures
Preparing test harnesses
Write automated test scripts
Create test suites from the test cases for efficient test execution
Check the Environment
Verify that the test environment has been set up correctly

Slide 50 EDS
Each Test Case is specified in terms of :-

its objective
the initial state of the system
the input
the expected result.

Slide 51 EDS
Major tasks (continued):
Execute the Tests

Execute the Test Cases according to the planned sequence.
Log the outcome of test execution
Test execution records should uniquely identify the versions of the
software under test, test tools and Testware
It should be possible to establish that all testing has been carried out by
reference to the test records.

Slide 52 EDS

Major tasks (continued):
Check the Results

Compare actual results with expected results
Report discrepancies as Incidents
Analyse Incidents to establish Root cause
Repeat, as necessary, test activities as result of
action taken for each discrepancy
The test coverage levels achieved for those
measures specified as test completion criteria
should be recorded.

Slide 53 EDS
Thoroughly inspect the results of each test
Myers - 2004
Ref: Myers, The Art of Software Testing, J Wiley and Sons, 1979
Slide 54 EDS
Evaluating Exit Criteria and Reporting
Test execution is assessed against the objectives defined in Test

Planning
This should be done for each Test Level (i.e. test stage)
A group of test activities that are organized and managed together.
Major tasks are:
Check test logs against the exit criteria specified in Test Planning
If the exit criteria has not been met
Assess if more tests are needed
Assess which test activities may need to be repeated
Assess if the exit criteria specified should be changed
Produce a test summary report for stakeholders review

Slide 55 EDS
Collect data from completed test activities to consolidate experience,

Testware, facts and numbers
Major tasks are:
Check which planned deliverables have been delivered
Check that Incident reports status are up-to-date (e.g. Closed)
Ensure all Incident reports have associated change records
Record acceptance of the system
Finalise and archive Testware, the test environment and the test
infrastructure for later reuse
Handover Testware to the maintenance organization
Analyse lessons learned for future releases and projects, and for
process improvement.

Slide 56 EDS
5 Phases of the Fundamental Test Process
Fix test design and repeat

Fix component or test cases/scripts
and repeat
Test Test Test Evaluating Exit Test Closure

Planning Analysis Implementation Criteria and Activities
and and Design and Execution Reporting
Control
Fix test design and repeat
Fix component test plan and repeat

Slide 57 EDS
Psychology of Testing
The Testing Mindset
Historically testing was viewed as showing the system meets its

requirements
This has evolved to a stage where testing is performed with the
primary aim of finding faults rather than proving correctness. It is
perceived as a destructive process
Seeking to find failures (the right mindset) can be viewed as
criticism of the product and/or its author
But looking for failures is constructive!
Time can be saved
Risks reduced
Costs reduced
Skills improved

Slide 58 EDS
The Testing Mindset
It is important that the Objectives of testing are clearly understood

as humans will moderate their behaviour accordingly (however sub-
consciously):-
If testing is showing the system meets its requirements then I will just
produce tests that show this.
If testing is aimed at finding faults then I will be measured on this so I
will put effort into designing tests that are more likely to find faults.
The Testing mindset is different from a Developers

Slide 59 EDS
Testing is an extremely creative and

intellectually challenging task
Tests must be written for invalid and

unexpected, as well as valid and expected,
input conditions
Myers - 1979

Slide 60 EDS
Traits of Good Testers

A Tester needs:
good communication skills
good observation skills
people handling skills
Curiosity
patience
reliability
thoroughness
an inquisitive nature
attention to detail
creativity in terms of identifying likely faults
Experience
However as with most other disciplines an effective
test team will need a mix of skills so it is difficult to
generalise

Slide 61 EDS
Developer vs Tester Relationship
The relationship between a Developer and a Tester is

not normally an easy one because:-
testers point out problems with software
developers like to think their software is perfect
testers are perceived as delaying the project by
finding faults in the system
when the development slips testers normally have to
work long hours to test the product, which in turn can
cause resentment.
It is important that they work together
It is also important that they have mutual respect for
each other.
Collaboration is the right approach we work to a
common goal!
Communicate findings objectively, not subjectively

Slide 62 EDS
Independent testing
The right mindset could enable Developers to test the code
However, passing this responsibility to trained and professional
testing resources has many benefits (such as higher defect find
rates)
Authors tend to bring across assumptions they have made when
developing the software. They are less likely to write tests to show
faults in their own software (human nature)
With testing performed by independent testers, testing effort is
focused and not compromised by development effort and bias
It is generally believed that objective independent testing is more
effective

Slide 63 EDS
Independent testing
There are several levels of Independence (from Low to High)
Tests designed by the person(s) who wrote the software under
test
Tests designed by another person(s) (e.g. from the
development team).
Tests designed by a person(s) from a different organizational
group (e.g. an independent test team).
Tests designed by a person(s) from a different organization or
company (e.g. outsourcing to an in-house or external test
specialist organisation)

Slide 64 EDS
Fundamentals of Testing - Summary
We learned of the possible effects of Defects on people, on the

environment and on companies
We learned of the causes of a Defect and the results

Error/Defect/Failure remember these and alternative terms
We learned why testing was necessary how it improves Quality

and reduces Risk of failure
We learned how root cause analysis leads to process improvement
We learned why you cant test everything and when to stop testing
through Risk Analysis, Prioritisation and use of Exit Criteria

Slide 65 EDS
We learned about Testing:
its main Objectives, i.e. to find and prevent defects and to gain confidence in
system Quality
How we meet these Objectives
How they can vary by Test Level
That it is conducted before and after the code is delivered
What activities it comprises
That Debugging and Testing are different
We learned of the 7 Key Principles of Testing:
Testing shows presence of Defects

Exhaustive Testing is Impossible!
Early Testing
Defect Clustering
The Pesticide Paradox
Testing is Context Dependent
Absence of Errors Fallacy

Slide 66 EDS
We learned the Fundamental Test process:
The Stages:
Evaluating Exit Criteria and Reporting
And the main tasks for each stage of the process
And we learned about the Psychology of Testing:
What is the right testing mindset (looking for bugs!)

Why it is constructive not destructive!
The skills needed for a good tester
Tester and Developer relationship issues
The benefit and types of Independent testing

Slide 67 EDS
03-23-05
Version 2.5
Testing Throughout the Software Life-cycle

Testing Throughout the Software Lifecycle
Software Development Models

Test Levels
Test Types the Targets of Testing
Maintenance Testing
Summary

Slide 69 EDS
Software Development Models
The V-Model
Iterative Development Models
Testing within a Lifecycle Model

Slide 70 EDS
V-Model
User/Business Acceptance Test Acceptance
Requirements Plan Test
System Test System

System
Plan
Requirements Test
Integration Integration
Technical
Test Plan
Test
Specification
Unit Test Test

Development Program Unit
Plan
Levels Levels
Specification Test
Coding

Slide 71 EDS
V-Model
The benefits of the V Model include:
The testing phases are given the same level of management

attention and commitment as the corresponding development
phases
The outputs from the development phases are reviewed by the
testing team to ensure their testability
Verification and validation (and early test design) can be carried out
during the development of the software work products
The early planning and preliminary design of tests provides
additional review comments on the outputs from the development
phase

Slide 72 EDS
V-Model
The levels of development and testing shown in the model vary from
project to project
For example, there may additional test levels, such as System
Integration Testing, sitting between System Testing and Acceptance
Testing (more on these test levels later)
The work products coming out from any one development level may
be utilised in one or more test levels
For example, whilst the prime source for Acceptance testing is the
Business Requirement, the System Requirements (e.g. Use Cases)
may also be needed to support detailed test design

Slide 73 EDS
V Model
The V Model provides a powerful tool for managing and

controlling risk within the testing component of a project.
The process of bringing testing planning and design into the

development process as early as possible enables risks to
be identified and strategies for removing or mitigating them
to be put in place in a timely manner.
Watkins - 2001

Slide 74 EDS
Iterative Development
Establish Requirements
Design the System
Build the System
Test the System
Achieved with small developments Iterations and Increment within Iterations
As Increments are developed and tested the System grows and grows. Need for more testing with Regression Testing paramount
E.g. RAD, RUP and Agile development models
Agile development
aim is to deliver software early and often
Rapid production and time to market
Can handle (and anticipates) changing requirements throughout all development and test phases

Slide 75 EDS
Rapid Application Development
User
Requirements
Code
Acceptance
Test

Slide 76 EDS
Testing within a Lifecycle Model
Characteristics of good testing in any life cycle model:
A Test Level exists for every development Level

Each Test Level has specific objectives
Test analysis and design for each Test Level begins during corresponding development Level
Early and proactive involvement of testers in reviewing development deliverables benefits both parties
Test Levels should be adapted depending on Project nature. May be better to combine Test Levels, e.g. with COTS testing.

Slide 77 EDS
Test Levels
Component Testing
Integration Testing
System testing
Acceptance Testing

Slide 78 EDS
Component Testing
System Test System

System
Plan
Requirements Test
Technical
Test Plan
Test
Specification
Unit Test Test

Development Program Unit/Component
Plan
Levels Levels
Specification Test
Coding

Slide 79 EDS
Component Testing
Definition
Component A minimal software item that can be tested in isolation.

Component Testing The testing of individual software components.
Sometimes known as Unit Testing, Module Testing or Program
Testing
Component can be tested in isolation stubs/drivers may be
employed
Test cases derived from component specification (module/program
spec)
Functional and Non-Functional testing
Usually performed by the developer, with debugging tool
Quick and informal defect fixing

Slide 80 EDS
Component Testing
Definition
Test-First/Test-Driven approach create the tests to drive the

design and code construction!
Instead of creating a design to tell you how to structure your code,
you create a test that defines how a small part of the system
should function.
Three steps:
1. Design test that defines how you think a small part of the software
should behave (Incremental development).
2. Make the test run as easily and quickly as you can. Don't be concerned
about the design of code, just get it to work!
3. Clean up the code. Now that the code is working correctly, take a step
back and re-factor to remove any duplication or any other problems
that were introduced to get the test to run.
Russell Gold, Thomas Hammell and Tom Snyder - 2005

Slide 81 EDS
Integration Testing
Definition
Component Integration Testing
System Integration Testing

Slide 82 EDS
Integration Testing
Definition
Integration Testing - Testing performed to expose

defects in the interfaces and in the interactions between
integrated components or systems
Components may be code modules, operating systems,
hardware and even complete systems
There are 2 levels of Integration Testing

Slide 83 EDS
System Test System

System
Plan
Requirements Test
Technical
Test Plan
Test
Specification
Unit Test Test

Plan
Levels Levels
Test
Specification
Coding

Slide 84 EDS
Definition
component integration testing Testing performed to expose

defects in the interfaces and interaction between integrated
components
usually performed by the Developer, but could involve the test
team
usually formal (records of test design and execution are kept)
all individual components should be integration tested prior to
system testing

Slide 85 EDS
Test Planning
To consider - should the integration testing approach:
Start from top level components and work down?

Start from bottom level components and work up?
Use the big bang method?
Be based on functional groups?
Start on critical components first?
Be based on business sequencing? Maybe suit System Test needs.
Knowledge of the system architecture is important

The greater the scope of the integration approach the more difficult
it is to isolate defects
Non-Functional requirements testing may start here e.g. early
performance measures

Slide 86 EDS
Top-down testing
Component
under test
P
Q R
S T U V
Stubs

Slide 87 EDS
Top-down testing
Pros Cons
provides a limited working system stubs only provide limited
early in the design process simulations of lower level
depth first integration demonstrates components and could influence
end-to-end functions early in the spurious results
development process breadth first means that higher
early detection of design errors levels of the system must be
through early implementation of the artificially forced to generate
design structure output for test observations
early testing of major control or
decision points

Slide 88 EDS
Bottom-up testing
Component under test
P
P is the driver
for components
Q and R
Same for Q
and R
Q R driving their
components
S T U V
Slide 89 EDS
Bottom-up testing
Pros Cons
using drivers instead of upper unavailability of a demonstrable
level modules to simulate the system until late in the
environment for lower level development process
modules
late detection of system
necessary for critical, low level structure errors
system components
testing can be observed on the
components under test from an
early stage

Slide 90 EDS
Big Bang Approach
Main Menu Function 1 Function 2 Function 3
Component 1 Component 2 Component 3 Component 4 Component 5 Component 6
Not usually the preferred approach

Slide 91 EDS
Suggested Integration Testing Methodology
The following testing techniques are appropriate for Integration

Testing:
Functional Testing using Black Box Testing techniques against the
interfacing requirements for the component under test
Non-functional Testing (where appropriate, for performance or
reliability testing of the component interfaces, for example)

Slide 92 EDS
Well talk about System Integration Testing later.
For now, we should stick to the sequence of the Test lifecycle.
Which means System Testing next.

Slide 93 EDS
System Testing
Context
Definition
Functional Systems testing
Non-Functional Systems Testing
Good Practices for System Testing

Slide 94 EDS
System Testing
System Test System

System
Plan
Requirements Test
Technical
Test Plan
Test
Specification
Unit Test Test

Plan
Levels Levels
Test
Specification
Coding

Slide 95 EDS
System Testing
Definition
System Testing - process of testing an integrated system to

verify that it meets specified requirements
Concerned with the behaviour of the whole system, not with

the workings of individual components.
Carried out by the Test Team

Slide 96 EDS
Functional System Testing
Definition
Requirements-based functional testing
Business process functional testing

Slide 97 EDS
A Functional requirement is (per IEEE):
A requirement that specifies a function that a system or

system component must perform
A Requirement may exist as a text document and/or a

model

Slide 98 EDS
Requirements-based functional testing - Functionality
Accuracy Provision of right or agreed results or effects
Adhere to applicable standards, conventions, regulations

Compliance
or laws
Auditability Ability to provide adequate and accurate audit data
Suitability Presence and appropriateness of functions for specified

tasks
Security The functions (e.g. a firewall) relating to detection of

threats, such as viruses, from malicious outsiders

Slide 99 EDS
Requirements based testing
testing against requirements and specifications

test procedures and cases derived from:
detailed user requirements
system requirements functional specification
User documentation/instructions
high level System design

Slide 100 EDS
Version 2.5
Requirements-based functional testing - techniques
Starts by using the most appropriate black-box testing techniques

May support this with white-box techniques (e.g. menu structures,
web page navigation)
Risk based approach important

Slide 101 EDS
Version 2.5
Business Process based testing
test procedures and cases derived from:

expected user profiles
Business scenarios
use cases
testing should reflect the business environment and processes in
which the system will operate.
therefore, test cases should be based on real business processes.

Slide 102 EDS
Version 2.5
System testing is the most misunderstood and most

difficult testing process
Myers - 2004

Slide 103 EDS
Version 2.5
Non-functional System Testing
Definition
Non-functional requirements
Non-functional test types

Slide 104 EDS
Version 2.5
Definition
testing of those requirements that do not relate to functionality

Slide 105 EDS
Version 2.5
Emphasis on non-functional requirements:

Performance
Load
Data volumes
Storage
Recovery
Usability
Stress

Slide 106 EDS
Version 2.5
The non-functional aspects of a system are all the attributes other than
business functionality, and are as important as the functional aspects.
These include:
the look and feel and ease of use of the system
how quickly the system performs
how much the system can do for the user
It is also about:
how easy and quick the system is to install
how robust it is
how quickly the system can recover from a crash

Slide 107 EDS
Version 2.5
Non-functional test types

Reliability - The capability of software to maintain its level of
performance under stated conditions for a stated period of time. Is the
software product reliable?
Usability - Is the software product easy to use, learn and understand
from the users perspective?
Maintainability: The effort needed to make specified modifications. Is the
software product easy to maintain?
Efficiency: The relationship between the level of performance of the
software and the amount of resources used, under stated conditions. Does
the software product use the hardware, system software and other
resources efficiently? Is the number of resources required by the software
product during use affordable and acceptable?
Portability: The ability of software to be transferred from one environment
to another. Is the software product portable?
Interoperability - The ability to interact with specified systems

Slide 108 EDS
Version 2.5
System Testing
Good Practices for System testing
implement documented procedures for requirements analysis,

control and traceability
review deliverables to ensure feasible, testable requirements and

associated acceptance criteria
trace requirements to the design and tests which prove the

requirement has been met
test data, facilities and documentation must be sufficient to

demonstrate conformance with requirements
Test environment must closely mirror the target production system

Slide 109 EDS
Version 2.5
Context
Definition
Objectives
Interfaces to External Systems

Slide 110 EDS
Version 2.5

Plan System
Requirements Test
Integration
Testing
System Test System
System
Plan
Requirements Test
Technical
Test Plan
Test
Specification
Unit Test Test

Plan
Levels Levels
Test
Specification
Coding

Slide 111 EDS
Version 2.5
Definition
System Integration Testing is testing between the System and

Acceptance phases.
The System has already proven to be functionally correct, what
remains to be tested is how the system reacts to other systems
and/or organisations.

Slide 112 EDS
Version 2.5
Objectives of Systems Integration Testing

The objective of System Integration Testing is to provide confidence
that the system or application is able to interoperate successfully
with other specified software systems and does not have an adverse
affect on other systems that may also be present in the live
environment, or vice versa
It is possible that the testing tasks performed during System

Integration Testing may be combined with System Testing,
particularly if the system or application has little or no requirement
to interoperate with other systems
In terms of the V Model, Systems Integration Testing corresponds to

the Functional and Technical Specification phases of the software
development lifecycle

Slide 113 EDS
Version 2.5
Testing Interfaces to External Systems

Having completed Component integration testing and Systems testing, one
must execute the plan for system-to-system integration
Infrastructure may need to be transformed in order to feed to an external

system
Black Box testing techniques used

Slide 114 EDS
Version 2.5
Acceptance Testing
Context
Definition
User Acceptance Testing
Operational Acceptance Testing
Contract/Regulation acceptance testing
Alpha and Beta testing
Other Acceptance Test terms

Slide 115 EDS
Version 2.5
Acceptance Testing
System Test System

System
Plan
Requirements Test
Technical
Test Plan
Test
Specification
Unit Test Test

Plan
Levels Levels
Test
Specification
Coding

Slide 116 EDS
Version 2.5
Acceptance Testing
Definition
Acceptance testing: Formal testing with respect to user
needs, requirements, and business processes conducted to
determine whether or not a system satisfies the acceptance
criteria and to enable the user, customers or other
authorized entity to determine whether or not to accept the
system.

Slide 117 EDS
Version 2.5
Acceptance Testing
Definition
Usually the responsibility of the Customer/End user, though
other stakeholders may be involved. Customer may sub-
contract the Acceptance test to a third party
Goal is to establish confidence in the system/part-system
or specific non-functional characteristics (e.g. performance)
Usually for ensuring the system is ready for deployment into
production
May also occur at other stages, e.g.
Acceptance testing of a COTS product before System Testing
commences
Acceptance testing a components usability during Component
testing
Acceptance testing a new significant functional
enhancement/middleware release prior to deployment into
System Test environment.

Slide 118 EDS
Version 2.5
Acceptance Testing
User Acceptance Testing (UAT)

Usually the final stage of validation
conducted by or visible to the end user and customer
testing is based on the defined user requirements
Often uses the Thread Testing approach:
A testing technique used to test the business functionality or business
logic of the application in an end-to-end manner, in much the same way a
User or an operator might interact with the system during its normal use.
- Watkins 2001
This approach is also often used for Functional Systems Test -

The same Threads serve both test activities

Slide 119 EDS
Version 2.5
Acceptance Testing
Often use a big bang approach

black box testing techniques most commonly used
Regression testing to ensure changes have not regressed other areas of the
system

Slide 120 EDS
Version 2.5
Acceptance Testing
If love is like an extended software Q.A. suite, then true

love is like a final Acceptance Test one often has to be
willing to endure compromise, bug fixes and work-arounds;
otherwise, the software is never done
The Usenet Oracle

Slide 121 EDS
Version 2.5
Acceptance Testing
Operational Acceptance Testing (OAT)

The Acceptance of the system by those who have to administer it.
Features covered include:

testing of backup/restore
disaster recovery
user management
maintenance tasks
periodic checks of security vulnerabilities
The objective of OAT is to confirm that the Application Under Test (AUT)
meets its operational requirements, and to provide confidence that the
system works correctly and is usable before it is formally "handed over" to
the operation user. OAT is conducted by one or more Operations
Representatives with the assistance of the Test Team 1 Watkins 2001

Slide 122 EDS
Version 2.5
Acceptance Testing
Operational Acceptance Testing (OAT)
Employs a Black Box Approach for some activities

Also employs a Thread Testing approach Operations representatives
performing typical tasks that they would perform during their normal usage
of the system
Also addresses testing of System Documentation, such as Operations
manuals

Slide 123 EDS
Version 2.5
Acceptance Testing
Contract/Regulation Acceptance Testing
Contract Acceptance Testing - testing against the acceptance criteria defined

in the contract
final payment to the developer depends on contract acceptance testing being
successfully completed
acceptance criteria defined at contract time are often imprecise, poorly defined,
incomplete and out-of-step with subsequent changes to the application
Regulation Acceptance testing is performed against any regulations which
must be adhered to, such as governmental, legal or safety regulations

Slide 124 EDS
Version 2.5
Acceptance Testing
Alpha & Beta Testing

early testing of stable product by customers/users
feedback provided by alpha and beta testers
alpha tests performed at developers site by customer
beta tests conducted at the customer site by end user/customer
published reviews of beta release test results can make or break a product
(e.g. PC games)

Slide 125 EDS
Version 2.5
Acceptance Testing
Other Acceptance test terms

Factory Acceptance Testing (FAT)
Site Acceptance Testing (SAT)
Both address acceptance testing for systems that are tested before and
after being moved to a customers site

Slide 126 EDS
Version 2.5
Test Types The Targets of Testing
Definitions
Functional Testing
Non-Functional Testing
Structural Testing
Confirmation & Regression Testing

Slide 127 EDS
Version 2.5
Definitions
Target For Testing - A group of test activities aimed at verifying the

software system (or a part of a system) based on a specific reason.
Test type - A group of test activities aimed at testing a component or
system regarding one or more interrelated quality attributes. A test
type is focused on a specific test objective, e.g.
reliability test
usability test
Structure or architecture of the system/software
regression test
and may take place on one or more test levels or test phases.

Slide 128 EDS
Version 2.5
Definitions
A model of the software may be developed and/or used in structural

and functional testing
For example, in functional testing
a process flow model
a state transition model
a plain language specification
and for structural testing
a control flow model
a menu structure model

Slide 129 EDS
Version 2.5
Functional Testing
functional testing: Testing based on an analysis of the specification

of the functionality of a component or system.
Specification E.g. Requirements specification, Use Cases,

Functional specification or maybe undocumented.
Function what the system does
Functional test based on the Functions and features may be

applied at all Test levels (e.g. Component Test, System Test etc)
Considers the external (not internal) behaviour of the software. Black-

Box testing. What it does rather than how it does it. More on this later!

Slide 130 EDS
Version 2.5
non-functional testing: Testing the attributes of a component or

system that do not relate to functionality, e.g. reliability, efficiency,
usability, interoperability, maintainability and portability
May be performed at all Test levels (not just Non Functional Systems
Testing)
Measuring the characteristics of the system/software that can be

quantified on a varying scale- e.g. performance test scaling

Slide 131 EDS
Version 2.5
Structural Testing
Structural testing: Testing based on an analysis of the internal

structure of the component or system
Also known as White Box Testing or Glass Box Testing
May be performed at all Test levels but more commonly during
Component Test and Component Integration Test
Coverage measured as a % of items tested i.e. how much the
structure has been tested
May be based on the system Architecture e.g. a calling hierarchy
Need for use of Test Tools e.g. for testing coverage of Statements
and Decision in the code
More on White Box testing and Coverage later

Slide 132 EDS
Version 2.5
Confirmation (Re-Testing) and Regression Testing
Re-testing: Testing that runs test cases that failed the last time
they were run, in order to verify the success of corrective
actions
Regression Testing: Testing of a previously tested program

following modification to ensure that defects have not been
introduced or uncovered in unchanged areas of the software,
as a result of the changes made. It is performed when the
software or its environment is changed.

Slide 133 EDS
Version 2.5
Confirmation Testing (Re-Testing)
Whenever a fault is detected and fixed then the software should be

re-tested to show that the original fault has been fixed. This is
known as Re-Testing.
It is important that the test case is repeatable.
In order to support this the test identifier should be included on the
fault report.
It is important that the environment and test data used are as close
as possible as those used during the original test.

Slide 134 EDS
Version 2.5
Regression Testing
If the test is re-run and passes you cannot necessarily say the fault
has been resolved because ..
You also need to ensure that the modifications have not caused
unintended side-effects elsewhere and that the modified system still
meets its requirements Regression Testing
Regression testing should be carried out :-

when the system is stable and the system or the environment changes
when testing bug-fix releases as part of the maintenance phase
It should be applied at all Test Levels
It should be considered complete when agreed completion criteria
for regression testing have been met
Regression test suites evolve over time and given that they are run
frequently are ideal candidates for automation

Slide 135 EDS
Version 2.5
Regression Testing
Selecting suitable tests involves :-
knowledge of the bug fixes and how they affect the system
understanding the areas that have frequent faults
understanding which areas of the system have undergone the most
recent changes
understanding the areas of the system which are most critical to the
user
understanding the core features of the system which must function
correctly.
The effectiveness of a regression test suite can diminish over time
for a number of reasons :-
tests are added for short term goals but not removed
tests become redundant due to functionality changes
test suite is not updated when major functionality changes are
implemented
execution time becomes prohibitively high
maintenance of the test suite becomes prohibitively high.

Slide 136 EDS
Version 2.5
Regression Testing
Reduction in effectiveness can be countered by :-

maintaining cross references between system features and their
corresponding tests
monitoring the addition of tests to the suite
Periodic review and removal of redundant tests
review of the test suite when major enhancements are made to the
system
evaluation of the effectiveness of the test suite using metrics.

Slide 137 EDS
Version 2.5
Regression Testing
The probability of making an incorrect change is more than 50 %.

Much of this is due to overconfidence and ineffective or nonexistent
software change testing.
We change just a couple of statements and believe we have not affected
anything adversely.
We execute one case that tests the path that was changed and tell
ourselves that the change has been tested.
IS IT, THEN, ANY WONDER THAT WE EXPERIENCE SO MANY PROBLEMS?!

Hetzel 1998

Slide 138 EDS
Version 2.5
Maintenance Testing
What is Maintenance testing?
Objectives of Maintenance testing
Problems of Maintenance testing
Concerns of Maintenance testing
How can we test changes?

Slide 139 EDS
Version 2.5
Maintenance Testing
What is Maintenance Testing?

Maintenance testing: Testing the changes to an operational system or
the impact of a changed environment to an operational system
testing changes to a Live System
Triggered by, for example,
Modification
software upgrades
Operating system changes
system tuning
emergency fixes
software Retirement (may necessitate data archiving tests)
Migration
System migration (including operational tests of new environment plus
changed software)
database migration

Slide 140 EDS
Version 2.5
Maintenance Testing
Objectives of Maintenance Testing

Develop tests to detect problems prior to placing the change into
production
Correct problems identified in the live environment
Test the completeness of needed training material
Involve users in the testing of software changes

Slide 141 EDS
Version 2.5
Maintenance Testing
Problems of Maintenance testing
all that is available is the source code (usually with

poor internal documentation and no record of
testing) poor or missing specifications
program structure, global data structures, system
interfaces and performance and/or design
constraints are difficult to determine and frequently
misinterpreted
Baselined test plans and/or regression test packs
often not updated

Slide 142 EDS
Version 2.5
Maintenance Testing
Concerns of Maintenance testing

will the testing process be planned?
will testing results be recorded?
will new faults be introduced into the system?
will system problems be detected during testing?
how much regression testing is feasible?
will training be considered?

Slide 143 EDS
Version 2.5
Maintenance Testing
How can we test changes?

Maintenance testing involves testing what has been changed (i.e. Re-
Testing)
It also, importantly, utilises Impact Analysis as a method for determining
what Regression testing is required for the whole system
Traceability of Testware to source documents essential for effective impact
analysis (we cover this more in a later topic)
Scope of Maintenance tests based on Risk assessment including size of
change and size of system
Maintenance testing may involve one or more test levels and one or more
test types

Slide 144 EDS
Version 2.5
Testing throughout the Software Lifecycle
Summary
Firstly we looked at Software Development Models:
The V-Model identifying the stages of testing, their relationship to the
Development stages and the type of work products involved
Iterative Development Models, as used in RAD, RUP and Agile
developments
Also the characteristics that make for good testing in ANY life cycle model
And that development models must be adapted to the context of project
and product characteristics
Next we talked about the different Test Levels:

Component Testing testing of individual software components by the
development team
Integration Testing at Component level looking at different approaches
such as Top-Down, Bottom-up and Big Bang
System Testing
Functional System Testing requirements and business process based
Non-Functional System Testing testing the non-functional attributes
of a system
System (level) Integration Testing - testing that a system interoperates
with other systems or organisations
Slide 145 EDS
Version 2.5
Summary
And still under Test Levels .

Acceptance Testing, comprising:
Operational Acceptance Testing
Contract and Regulation testing
Alpha and Beta testing
Next we talked about Test Types

A group of testing activities aimed at testing one or more quality attributes
of the system, such as:
Functional Testing
Structural testing
Regression Testing
Re-testing (confirmation)
All Test Types can be performed at all test levels

Slide 146 EDS
Version 2.5
Summary
An finally we talked about Maintenance testing

Testing the changes (software and environmental) to an operational live
system
What the reasons (or triggers) are for Maintenance Testing
We looked at the objectives of Maintenance Testing
the problems that can be encountered during Maintenance testing
and what we should consider for our Maintenance testing approach such as
Impact Analysis and Regression testing

Slide 147 EDS
Version 2.5
03-23-05
Version 2.5
Static Techniques

Static Techniques
Static Testing a Definition

Reviews and the Test Process
Review Process
Static Analysis
Summary

Slide 149 EDS
Version 2.5
Static Techniques
Static Testing a Definition
Static testing techniques involve examination of the projects

documentation, software and other information about the software
products without executing them
Static Testing Includes both Reviews (e.g. of documentation) and
Static Analysis of code
Reviews, Static Analysis and dynamic testing have the same
objective identifying defects
Static Testing and Dynamic Testing are complementary
Each technique can find different types of defects effectively and
efficiently
dynamic testing: Testing that involves the execution of the

software of a component or system.

Slide 150 EDS
Version 2.5
Topics
Why Review
When and What to review
What do Reviews find
Benefits of reviews

Slide 151 EDS
Version 2.5
Why Review
Find defects early in the development Education and training of
lifecycle (more cost effective to fix) developers and other project staff
More than 60% of the errors in a Determine the root causes of
component can be detected using defects and measures for
informal inspections (1) preventing recurrence
Reduce the defects in delivered Test and improve standards and
systems procedures
Learning experience in standards and To assess the early stages of
techniques development for progress and
completeness
Team building and motivation
Improve the specification, design and
code
1) Fagan, M. E., Advances in Software Inspections, IEEE Transactions on Software

Engineering, 1987

Slide 152 EDS
Version 2.5
Why Review - The cost of errors
Relative
Multiples

Slide 153 EDS
Version 2.5
Why Review - Error/Fault Propagation
Requirement
Errors
Design
Errors
Coding
Errors
(Faults)
correct
correct
correct
Requirements Design Code

Slide 154 EDS
Version 2.5
Why Review - Where errors are introduced

Slide 155 EDS
Version 2.5
Why Review
Quality control has as its mission the detection and

elimination of defects in the product. Reviews are the front
line in this mission
John W Horch 2003

Slide 156 EDS
Version 2.5
When and What to Review?
A review could be done entirely as a manual activity

Or there is tool support available
The main manual activity is to examine a work product and make
comments about it.
Any software work product can be reviewed, including:
requirements and design specifications
source code listings
test plans, test cases, test scripts
user documentation
application administration and support material
Web pages
A software work product can be reviewed one or more times and

using one or more review types
Review everything as soon as possible
Reviews start well before Dynamic Test execution

Slide 157 EDS
Version 2.5
What do Reviews Find?
In contrast to dynamic testing, reviews find defects rather than failures
Typical defects that are easier to find in reviews than in dynamic testing
are:
deviations from standards
requirement defects
design defects
insufficient maintainability
incorrect interface specifications.

Slide 158 EDS
Version 2.5
Benefits
Detect faults as they are introduced i.e. early detection and correction
Reduce the risk of error/fault propagation
Detect defects that Dynamic test execution unlikely to find, e.g.
requirement spec defects
Shorten development timescales
Reduce fault levels in delivered software
Lower cost and shorten testing timeframes
Lower cost over the life of the software
Create development productivity improvements
Reliably evaluates progress and capability (1)
Educates and trains participants (1)
Improve communication between project teams
(1) The Complete Guide to Software Testing Bill Hetzel

Slide 159 EDS
Version 2.5
Review Process
Topics
Informal Reviews
Formal Review Types
Formal Review Process
Formal Review Roles and Responsibilities
Other Review Types
Key Success factors

Slide 160 EDS
Version 2.5
Review Process
Informal Review
No Formal Review Process employed
Desk checking, looking for possible problems
The author of material checking his or her own material, possibly
with one other peer (pair programming)
Possibly a technical lead reviewing design and code
Usually undocumented but useful, cheap and widely used
This technique may be applied in low risk situations
No metrics kept, review findings are not configured items
Weaknesses do not find as many faults as formal reviews

Slide 161 EDS
Version 2.5
Review Process
Formal Review Types Walkthroughs
A walkthrough is a review of authored material led by

the author and attended by a group of the author's
peers (typically 2 to 6 peers). Primary purpose is
education
The material is presented by the author to the peer
group, who focus on learning about the material,
improving it and recording defects
Peer group should include development, operation
representatives, target audience, etc.
Examples are Dry Runs or Scenario playing to validate
product
Sessions can be formal or informal
Review sessions often open-ended (not time-boxed)
Pre-meeting preparation often involved
Weaknesses do not find as many defects as technical
reviews and inspections

Slide 162 EDS
Version 2.5
Review Process
Formal Review Types Technical Reviews
May be performed as Peer Reviews without management participation
Preferably lead by a trained moderator (not the author)
Pre-Meeting preparation is required
Primary purpose is to:
Discuss
make decisions
evaluate alternatives
find defects
solve technical problems and check conformance to specifications and standards
Degree of formality varies

Reviewers bring a list of technical issues to the review
Optional use of checklists and a review report
During the meeting reviewers raise objections, ambiguities or inconsistencies
in design or technical aspects being discussed
Problems are clarified and documented - solutions are sought after the review
has concluded
Weaknesses do not find as many faults as Inspections

Slide 163 EDS
Version 2.5
Review Process
Formal Review Types Inspections
Formal, systematic reviews of material. Primary purpose is fault finding

and process improvement
Led by an independent trained moderator (but not the author)
Main purpose find defects
Attended by the author & the author's peers (usually 3 to 6) acting in
defined roles
Pre meeting preparation essential
Follow a strict format
stated entry criteria and exit criteria
seeking and recording defects
use standardised rules, check-lists and techniques
record metrics
Formal follow-up process
Optionally process improvement considerations part of review
Weaknesses expensive and time consuming but high defect yield!

Slide 164 EDS
Version 2.5
Planning
Kick-off
Review Overview optional
Preparation
Review Meeting
Rework
Follow-up
Repeat Review - optional

Slide 165 EDS
Version 2.5
Planning
Define Entry and Exit Criteria (for most

formal reviews)
Ensure that the volume of material to be
reviewed is appropriate
Identify roles, participants and establish a
time and place for the review

Slide 166 EDS
Version 2.5
Kick Off
Distribute the material to the participants

Explain objectives, process and material to be
reviewed
Obtain copies of the relevant review and report
templates
Create checklists of areas to cover and
distribute
checklists can make reviews more effective and
efficient
E.g. a checklist based on perspectives such as
user, maintainer, tester or operations
or a checklist of typical requirements problems
to focus on
Make sure entry criteria has been/will be met

Slide 167 EDS
Version 2.5
Review Overview (Optional)
Required for new or difficult material

Overviews:
educate the participants
allow participants to focus on technical content
describe where the material fits in the system and in the development process
focus on any complex functionality
highlight any changes and explain the need for these changes

Slide 168 EDS
Version 2.5
Preparation
Each participant reviews the material to:

learn about the material
note suspected defects
record questions
In some circumstances, depending on the
expertise of the participants, the moderator may
ask certain participants to concentrate on
particular aspects of the material during
preparation

Slide 169 EDS
Version 2.5
Review Meeting
The material is read to the participants by the
reader
Defects are raised by the participants and
recorded by the recorder
Participants may make decisions about
categorising and even handling the defects
though usually avoid solutionising
Deliverables may include meeting minutes
For Inspections - Pass or fail and repeat
review decisions are usually made by the
moderator
The preparation time and the actual time may
be recorded

Slide 170 EDS
Version 2.5
Rework
The author must resolve all defects found during the review by reworking
the material as recommended by the review report
Note, the cost of rework is NOT included in the cost of reviews

Slide 171 EDS
Version 2.5
Follow-up
Check the corrections to the material and account for all recorded defects
If necessary, schedule a repeat review for the corrected material
Inform management of the status of the corrected material
Add the defect data from the review to the project statistics database
enables process improvement!
Complete and sign the review report and forms (Inspections)
Ensure exit criteria met

Slide 172 EDS
Version 2.5
Repeat Review (optional)
If the material has been passed as is or if the rework is minor, no further

reviews are required
If a repeat review is required (e.g. if significant re-work was required) a
repeat review must be scheduled with the same participants to verify the
revised material

Slide 173 EDS
Version 2.5
Review Process
Formal Review Roles and Responsibilities
Manager decides on the execution of reviews, allocates time in project

schedules and determines if the review objectives have been
met
Moderator the person who leads, plans and runs the review
May mediate between the various points of view and is often the
person upon whom the success of the review rests
Author the writer or person with chief responsibility for the document(s)
to be reviewed
Reviewers Individuals with a specific technical or business background

(also called checkers or inspectors)
Identify and describe findings (e.g. defects)
Take part in any review meetings
Scribe/Recorder documents all the issues, problems and open points that were
identified during the meeting

Slide 174 EDS
Version 2.5
Review Process
Other Review Types
Some others not in the ISTQB Syllabus:
Management reviews reviews of plans, schedules, progress

Audits independent evaluation of conformance to standards, plans
and procedures
Post Implementation review of project approach (including test
approach)

Slide 175 EDS
Version 2.5
Review Process
Key Success Factors
Each review has a clear predefined objective
The right people are involved
Defects found are welcomed (Authors must leave their
ego at the door!)
Defects are expressed objectively no cornering the
author make it a positive experience!
Review techniques are applied suitable to the review
Checklists used if appropriate focus the review effort
Roles pre-defined avoid duplication and increase
effectiveness
Training is given in review techniques- especially for
Inspections
Management buy in schedule in review activities and
effort
There is an emphasis on learning and process
improvement

Slide 176 EDS
Version 2.5
Static Analysis
Topics
Definition
Description
Value
Types of Defects found
Use of Tools

Slide 177 EDS
Version 2.5
Static Analysis
Definition
static analysis: Analysis of software artifacts, e.g.

requirements or code, carried out without execution of
these software artifacts.

Slide 178 EDS
Version 2.5
Static Analysis
Description
Objective - to find defects in software source code and software
models
Static analysis is performed without actually executing the software

being examined by the tool whilst Dynamic testing does execute the
software code
Static analysis can locate defects that are hard to find in testing.
As with reviews, static analysis finds defects rather than failures
Static analysis tools analyze program code (e.g. using control flow
graphing and data flow analysis techniques), as well as generated
output such as HTML and XML.

Slide 179 EDS
Version 2.5
Static Analysis
Value
Early detection of defects prior to test execution
Early warning about suspicious aspects of the code or design
Calculation of metrics such as a high complexity measure
Identification of defects not easily found by dynamic testing
Detecting dependencies and inconsistencies in software models, such
as links
Improved maintainability of code and design
Prevention of defects, if lessons are learned in development
Cost effective problems found earlier are cheaper to fix
Static analysis is more effective and less expensive than dynamic
testing (1)
Static analysis is demonstrated to find 45% of expected errors before
testing actually starts

Slide 180 EDS
Version 2.5
Static Analysis
Value Complexity Analysis
Many complexity measures in common use, e.g.

Cyclomatic Complexity measures structural complexity
Halstead Complexity measures algorithmic complexity, measured by
counting operators and operands
Henry and Kafura metrics measures coupling between modules
(parameters, global variables, calls)
Cyclomatic Complexity is the most widely used. It identifies how

difficult the code is to understand, test and maintain. It also identifies
the risk associated with testing a program

Slide 181 EDS
Version 2.5
Static Analysis
Types of Defects found
Unreachable code Incorrect variable usage

Undeclared variables Syntax checking
Parameter type mismatches Violations of code standards
Uncalled functions and procedures Use of variables without first
Possible array bound violations defining them
Security Violations variables that are declared but
never used
Inconsistent interface between
modules and components Use of variables after they have
been killed

Slide 182 EDS
Version 2.5
Static Analysis
Use of Tools
Static analysis tools are typically used by developers
Used mainly before and during Component and Component Integration
testing
The tools check against predefined rules or programming standards
Also by designers during software modelling
Static analysis tools may produce a large number of warning messages
Hence the need to use the tools effectively (or cant see the wood for
the trees!)
Compilers may offer some support for static analysis, including the
calculation of metrics

Slide 183 EDS
Version 2.5
Static Techniques
Summary
We learned that Static Testing techniques involve examining
documentation and software products without executing them
That Static Testing covers both Reviews and Static Analysis
For Reviews we learned about:

the different types of products that can be reviewed and that we review as
early as we can
The Benefits of conducting Reviews
And the typical defects that reviews uncover
We learned about the Review process

The key characteristics of the 4 Review types, Informal, Walkthrough,
Technical and Inspection
The 6 main Stages that make up the Review process from Planning through
to Follow-up
The Roles and Responsibilities involved in that process
The Key Success Factors for Review what makes them work

Slide 184 EDS
Version 2.5
Static Techniques
Summary
And we learned about Static Analysis
Analysis of software artefacts (e.g. code/design) without executing them

The Value of Static Analysis what we gain by conducting it
The types of defects found in static analysis
And finally the use of tools such as a Compiler who uses them and when

Slide 185 EDS
Version 2.5
03-23-05
Version 2.5

Identifying Test Conditions and Designing Test Cases

Categories of Test Design Techniques
Specification based or Black Box Techniques
Structure based or White Box Techniques
Experience Based Techniques
Choosing Test Techniques
Summary

Slide 187 EDS
Version 2.5
Identifying Test Conditions and Designing
Test Cases
Developing test material can be split into two distinct stages:

Defining what needs to be tested
Defining how the system should be tested
This process can vary from organisation to organisation, can be very formal
or very informal with little documentation
The more formal, the more repeatable the tests, but it does depend on the
context of the testing being carried out
The process of identifying test conditions and designing tests consists of the
following steps:
Defining test conditions
Specifying test cases
Specifying test procedures
Developing a Test execution schedule

Slide 188 EDS
Version 2.5
Test Cases
Test Conditions
Test Conditions are binary statements which determine a systems fitness for purpose
Defines what must be tested
Sometimes referred to as a Test Item
Grouped by Test Object or system function/process
Test Requirement (Test basis ) documentation is analysed to determine the Test
Conditions
Test Conditions are then cross referenced to one or more test cases for execution
Not all Test Conditions are as important as others so each Test Condition is assigned a
risk
Test Conditions should be linked back to their source documents from which they are
derived. This helps for two reasons:
Impact Analysis
Traceability

Slide 189 EDS
Version 2.5
Test Cases
Test Cases
A Test Case defines how the system should be tested
They typically contain
Input values
Execution pre conditions
Expected results (output, changes in state etc)
Post conditions
Cross referenced test conditions
Remember expected results must be defined before execution
There can be many Test Cases developed to test a single Test Condition

Slide 190 EDS
Version 2.5
Expected Results
A necessary part of a test case is

a definition of the expected output
or result
Myers - 1979

Slide 191 EDS
Version 2.5
Test Cases
Test Procedures
The Test Procedures Specification specifies the sequence of actions for a
test, i.e. one or more Test Cases
It is also known as a Test Script
The Test Script can be manual or automated
Contents of a Test Procedure are:
Test procedure specification identifier
Purpose
Special requirements
Procedure steps

Slide 192 EDS
Version 2.5
Test Cases
Test Execution Schedule
The Test Procedure Specifications (i.e. Test Scripts) are subsequently
included in a Test Execution Schedule
This schedule defines the order in which the test scripts are executed, when
they are to be carried out and by whom
The Execution schedule will also need to take account of:
Regression Tests
Prioritisation
And technical and logical dependencies

Slide 193 EDS
Version 2.5
Test Conditions, Cases, Procedures and Schedule
What How How When
Test Test
Procedure Execution
Sourced Documentation
Specification Schedule
Manual Test
Test Script Test
Test Test Procedure
Cases
Condition Cases or Specifications
Automated
Priority Test Script

Slide 194 EDS
Version 2.5
Categories of Test Design Techniques
Definition of Black Box Testing

What is black box testing?
Definition of White Box Testing
What is white box testing?
Black, White and Experience based (comparison)
Use of Techniques and tools

Slide 195 EDS
Version 2.5
Black and White Box Testing
Definition of Black Box Testing
Black-box testing: Testing, either functional or non-functional,

without reference to the internal structure of the component or
system.

Slide 196 EDS
Version 2.5
What is Black Box Testing?
testing without knowing the internal workings of the code

WHAT a system does, rather than HOW it does it
typically used at System Test phase, although can be useful throughout the
test lifecycle
also known as specification based testing
applies for Functional and Non-Functional testing
Input Output
If Output = Expected result then pass

Slide 197 EDS
Version 2.5
Definition of White Box Testing
white-box testing: Testing based on an analysis of the

internal structure of the component or system.

Slide 198 EDS
Version 2.5
What is White Box Testing?
testing based upon the structure of the code

typically undertaken at Component and Component Integration
Test phases by development teams
also known as structural or glass box testing or structure based
testing
Input Output

Slide 199 EDS
Version 2.5
Black, White and Experienced based
Based on requirements
Black From the requirements, tests are created
(Specification Specification Models can be used for systematic test case design
Based) Techniques
Equivalence Partitioning
Boundary Value Analysis
Decision Tables
State Transition Testing
Use Case Testing
Based on code and the design of the system

White The tests provide the ability to derive the extent of coverage of the whole
(Structure application
Based)
Techniques
Statement coverage
Branch Coverage
Decision Coverage
Based on the knowledge of the tester

Experience
Using past experienced use & intuition to guess where errors may occur
(Black box)
Techniques
Error Guessing
Exploratory Testing

Slide 200 EDS
Version 2.5
Use of Techniques and Tools
techniques provide a systematic method of approaching

software testing
enable tests to be repeatable
provide a measure of software coverage
due to the scale and complexity of systems, tools are often
used to assist with testing especially white box testing

Slide 201 EDS
Version 2.5
Specification based or Black Box Techniques
Decision Table Testing
Use Case Testing
Other black box test techniques

Slide 202 EDS
Version 2.5
Black Box Test Techniques
aim is to treat groups of inputs as equivalent and to select one

representative input to test them all
Best shown in the following example.
If we wanted to test the following IF statement:

IF VALUE is between 1 and 100 (inclusive) (e.g. VALUE >=1 and
VALUE <= 100) THEN ..
We could put a range of numbers as shown in the next slide through
test cases

Slide 203 EDS
Version 2.5
IF Value >= 1 AND Value <= 100 THEN .
0 101
37 65 99
1 100
-1 19 53
48 87 1000
OUT OF OUT OF
RANGE IN RANGE RANGE

Slide 204 EDS
Version 2.5
the numbers fall into a partition where each would have the same, or
equivalent, result i.e. an Equivalence Partition (EP) or Equivalence Class
EP says that by testing just one value we have tested the partition
(typically a mid-point value is used). It assumes that:
1) if one value finds a bug, the others probably will too

2) if one doesn't find a bug, the others probably won't either

Slide 205 EDS
Version 2.5
in EP we must identify Valid Equivalence partitions and Invalid
Equivalence partitions where applicable (typically in range tests)
the Valid partition is bounded by the values 1 and 100
plus there are 2 Invalid partitions

Slide 206 EDS
Version 2.5
IF Value >= 1 AND Value <= 100 THEN .
0 101
37 65 99
1 100
-1 19 53
48 87 1000
VALID PARTITION
INVALID INVALID
PARTITION
EDS PARTITION
ISTQB Testing Foundation Course
Slide 207 EDS
Version 2.5
Time would be wasted by specifying test cases that covered a range of
values within each of the three partitions, unless the code was designed in
an unusual way
There are more effective techniques that can be used to find bugs in such
circumstances (such as code inspection)
EP can help reduce the number of tests from a list of all possible inputs to a
minimum set that would still test each partition

Slide 208 EDS
Version 2.5
If the tester chooses the right partitions, the testing will be accurate and
efficient
If the tester mistakenly thinks of two partitions as equivalent and they are
not, a test situation will be missed
Or on the other hand, if the tester thinks two objects are different and they
are not, the tests will be redundant
EP can be used for all Levels of Testing
EP is used to achieve good input and output coverage, knowing exhaustive

testing is often impossible
It can be applied to human input, input via interfaces to a system, or

interface parameters in integration testing

Slide 209 EDS
Version 2.5

Boundary Value Analysis (BVA) uses the same analysis of partitions as EP
and is usually used in conjunction with EP in test case design
As with EP, it can be used for all Test levels
BVA operates on the basis that experience shows us that errors are most
likely to exist at the boundaries between partitions and in doing so
incorporates a degree of negative testing into the test design
BVA Test cases are designed to exercise the software on and at either side
of boundary values

Slide 210 EDS
Version 2.5

For our previous example of Value >= 1and Value <= 100
Value = 1 Value = 100

(valid) (valid)
Value = 0 Value = 2 Value = 99 Value = 101
(invalid) (valid) (valid) (invalid)

Slide 211 EDS
Version 2.5
find the boundary and then test one value above and below it
ALWAYS results in two test cases per boundary for valid inputs and
three tests cases per boundary for all inputs
inputs should be in the smallest significant values for the boundary
(e.g. Boundary of a > 10.0 should result in test values of 10.0,
10.1 & 10.2)
only applicable for numeric (and date) fields

Slide 212 EDS
Version 2.5
Decision Table Testing
Table based technique where

Inputs to the system are recorded
Outputs to the system are defined
Inputs are usually defined in terms of actions which are Boolean (true or
false)
Outputs are recorded against each unique combination of inputs
Using the Decision Table the relationships between the inputs and the
possible outputs are mapped together
As with State Transition testing, an excellent tool to capture certain types of
system requirements and to document internal system design. As such can
be used for a number of test levels
Especially useful for complex business rules

Slide 213 EDS
Version 2.5
Decision Table Structure
Inputs / Actions Test 1 Test 2 Test 3

Input 1 T T F
Input 2 T T F
Input 3 T DONT CARE F
Input 4 T F T
Output / Response 1 Y Y N
Response Response 2 Y N Y
Response 3 N Y N
Each column of the table corresponds to a business rule that defines a unique
combination of conditions that result in the execution of the actions associated
with that rule
The strength of Decision Table testing is that it creates combinations of conditions

that might not otherwise have been exercised during testing

Slide 214 EDS
Version 2.5
Decision Table Example
Test 1 Test 2 Test 3
> 55 yrs old F T T
Smoker F T F
Exercises 3 times a week + T F T
History of Heart Attacks F T F
Insure Y N Y
Offer 10% Discount N N Y
Offer 30% Discount Y N N
What will be the out come of the following Scenarios?

Joe is a 22 year old non smoker who goes to the gym 4 times / week and has
no history of heart attacks in his family
Kevin is 62 year old non smoker who swims twice a week and plays tennis. He
has no history of heart attacks in his family

Slide 215 EDS
Version 2.5

State Transition Testing uses the following terms:
state diagram: A diagram that depicts the states that a component or system
can assume, and shows the events or circumstances that cause and/or result
from a change from one state to another. [IEEE 610]
state table: A grid showing the resulting transitions for each state combined
with each possible event, showing both valid and invalid transitions.
state transition: A transition between two states of a component or system.
state transition testing: A black box test design technique in which test cases
are designed to execute valid and invalid state transitions. Also known as N-
switch testing.
An excellent tool to capture certain types of system requirements and to
document internal system design. As such can be used for a number of test
levels
Often used in testing:
Screen dialogues
Web site transitions

Slide 216 EDS
Version 2.5
State Transition Diagram
State A Starting State
Transition Between
Event/Action etc start and end states
Event from outside

the system
State B Action triggered by
Event
End State

Slide 217 EDS
Version 2.5
State Transition Example Simplified Car Gears
Change Down/
Move Back
Neutral Reverse
Change Up/
Change Up/ Change Down/ Accelerate
Accelerate Decelerate
1st Gear
Change Up/ Change Down/
2nd Gear
Change Up/ Change Down/
3rd Gear

Slide 218 EDS
Version 2.5
State Transition - Switch Coverage
Switch Coverage is a method of determining the number tests based on the

number of hops between transitions. Some times known as Chow
0-Switch Coverage (1 hop) 1-Switch Coverage (2 hops)

RN RN1
NR RNR
N1 NRN
1N N12
12 N1N
21 1NR
23
32 Etc.
These hops can be used to determine the VALID tests

Slide 219 EDS
Version 2.5
State Transition - State Table
While Switch testing helps determine the valid tests, we also need to look
for the invalid tests.
Change Up Change Down
R Acc / Neutral Null

N Acc/1st Gear Dec / Reverse
1st Acc/ 2nd Gear Dec / Neutral
2nd Acc / 3rd gear Dec / 1st Gear
3rd Null Dec / 2nd Gear
Invalid tests are those identified by a null output in this case
Changing down from reverse

Changing up from 3rd

Slide 220 EDS
Version 2.5
State Transition Another example
for a Theatre Show reservation
Request Choose Reserve
Show Show Show
Options
Show
Show Options
Show selected Reservation
provided
Made
Pay for
Change Show
Mind/
Cancel
Return to
reservation
Options
Show
Reservation
Paid For
Cancelled Issue
Cancel reservation/
Reservation Issue Refund Ticket
Ticket
Cancel reservation
(return ticket)/Issue Received
Refund

Slide 221 EDS
Version 2.5
State Transition Another type of State Table
Current Event/ Event/ Next Event/ Next Event/ Next Event/ Next Event/ Next
State Next State State State State State State
A B C D E F
SS 1
1 2
2 1 3
3 4
4 ES
ES

Slide 222 EDS
Version 2.5
Use Case Testing
Use cases are a method of describing requirements

Structured approach to requirements design
Usually has one main flow (though may also have alternate flows)
Each use case is a process flow or scenario
Particularly useful in determining tests for (Functional) Systems Test and for
UAT
Also good at detecting defects in the integration of interfaces
The main parts of a Use Case are:

Actors users of the system
Pre conditions What are the starting requirements for the use case
Post conditions The state the system will end up in once completed

Slide 223 EDS
Version 2.5
Other Black Box Test Techniques
Syntax testing
test cases are prepared to exercise the rule governing the format of
data in a system (e.g. a Zip or Postal Code, a telephone number)
Random testing
test cases are selected, possibly using a pseudo-random generation

algorithm, to match an operational profile

Slide 224 EDS
Version 2.5
Structure Based or White Box Test Techniques
Statement Testing
Decision Testing
Assessing Completeness (Coverage)
Other White Box test techniques

Slide 225 EDS
Version 2.5
White Box Test Techniques
Statement Testing
aim is to display that all executable statements have been run at

least once
coverage measurement is:
number of statements executed

total number of statements

Slide 226 EDS
Version 2.5
Statement Testing
1 Control
Example 1 Flow Graph
2
1. Read vehicle
3
2. Read colour
4
3. If vehicle = Car Then
4. If colour = Red Then 5
5. Print Fast
6
6. End If
7. End If 7
Slide 227 EDS
Version 2.5
Statement Testing 1
Example 2 2
1. Read A 3
2. If A > 40 Then
4
3. A= A* 2
5
4. End If
5. If A > 100 Then 6
6. A = A 10
7
7. End If

Slide 228 EDS
Version 2.5
Statement Testing 1
Example 3
1. Read bread 2
2. Read filling
3. If bread = Roll Then 3/9
4. If filling = Tuna Then
4/6
5. Price = 1.50
6. Else 7 5
7. Price = 1.00 10
8. End If 8
9. Else
10. Price = 0.75 11
11. End If
Slide 229 EDS
Version 2.5
Decision Testing
Aim is to demonstrate that all Decisions have been run at least once
Coverage measurement is:
the number of Decisions executed

the total number of Decisions
Note that Decision testing is related to Branch testing

Decisions are lines of code where there are two or more alternative routes based on a
decision that has to be made, i.e. IF/THEN/ELSE
Branches are where there are one of two or more alternative routes from a line of code
but also include JUMPs and GO TOs
Decision and Branch Test coverage for a piece of code is often the same, but not
always
We are only interested in Decision coverage though.
Note DT is always >=ST

Slide 230 EDS
Version 2.5
Decision Testing 1
Example 1 2
1. Read vehicle 3
2. Read colour 4
3. If vehicle = Car Then
5
4. If colour = Red Then
5. Print Fast 6
6. End If
7
7. End If
Slide 231 EDS
Version 2.5
Decision Testing 1
Example 2 2
1. Read A
3
2. If A > 40 Then
4
3. A= A* 2
4. End If 5
5. If A > 100 Then
6
6. A = A 10
7. End If 7

Slide 232 EDS
Version 2.5
Decision Testing 1
Example 3
1. Read bread
2
2. Read filling
3. If bread = Roll Then 3/9
4/6
5. Price = 1.50
6. Else
7 5
7. Price = 1.00
10
8. End If 8
9. Else
10. Price = 0.75 11
11. End If
Slide 233 EDS
Version 2.5
Statement and Decision Testing
The pseudo-code can be expressed in Flow-chart format
1. Read A
2. If A > 40 Then
3. A= A* 2
4. End If
5. If A > 100 Then
6. A = A 10
7. End If

Slide 234 EDS
Version 2.5
1. Read bread
2. Read filling We already know:
Statement Coverage = 3
3. If bread = Roll Then
Decision Coverage = 3
Number of executable Statements = 7
5. Price = 1.50
Number of Branches = 4 (T and F of
6. Else each IF)
7. Price = 1.00
8. End If
9. Else
10. Price = 0.75
11. End If
Slide 235 EDS
Version 2.5
1. Read bread
2. Read filling Based on the following test set:
3. If bread = Roll Then Bread Filling
Roll Tuna
Sandwich Ham
5. Price = 1.50
6. Else What is the test Statement Coverage
and Decision Coverage?
7. Price = 1.00
8. End If
9. Else
10. Price = 0.75
11. End If
Slide 236 EDS
Version 2.5
Other White Box Test Techniques
Branch Condition testing

Branch Condition Testing requires that the True and False of each Boolean operand
is tested (Boolean Operands in this example: If A > 30 and B >= 5)
Branch Condition Combination testing
Branch Condition Combination Coverage would require all combinations of Boolean
operands to be evaluated
Modified Condition Decision testing
Modified Condition Decision Coverage requires test cases to show that each
Boolean operand can independently affect the outcome of the decision
Dataflow testing
Data flow testing aims to execute sub-paths from points where each variable in a
component is defined to points where it is referenced.
Linear Code Sequence And Jump (LCSAJ) testing
LCSAJ testing requires a model of the source code which identifies control flow
jumps (where control flow does not pass to a sequential statement).

Slide 237 EDS
Version 2.5
Definition
Error Guessing
Exploratory Testing

Slide 238 EDS
Version 2.5
Experienced Based Techniques
Definition
Uses the knowledge of people and the experience of past projects to

determine likely errors based on the knowledge
Testers
Users
Stakeholders
Experience based techniques are non structured and do not rely on
specification documents. This makes them unable to be measured in terms
of coverage

Slide 239 EDS
Version 2.5
Definition
Tests are derived from skill and intuition

Used to AUGMENT more systematic methods
Good to identify tests which may not be explicitly described in the
specifications
Most beneficial when applied after more formal measures
The success of these techniques is directly related to the knowledge and
skill of the testers
Does not provide any form of measurement or coverage

Slide 240 EDS
Version 2.5
Error Guessing
Using experience to postulate errors
Use Error Guessing to complement test design techniques
Use as a mopping up approach to supplement systematic

techniques
Can be useful to identify special tests not easily captured by
formal techniques, especially when applied after more formal
approaches
So dont use as a first choice technique!
Structured approach to error guessing
Create a list of all possible errors
Then create tests to attack these errors
Remember these defect attack lists are built on experience, previous

defects and from common knowledge as to why systems fail

Slide 241 EDS
Version 2.5
Error Guessing
Error Guessing tests may include
Enter 00000 or 99999 in to a field
Creating surnames with quotes in, such as ODonnell
Nulls in mandatory fields
Reserved characters ($%& for web systems)

Slide 242 EDS
Version 2.5
Exploratory testing
Exploratory testing is a concurrent process where
Test design, execution and logging happen simultaneously
Testing is often not recorded
Makes use of experience, heuristics and test patterns
Testing is based on a test charter that may include
Scope of the testing (in and out)
Why? - what is the focus of your testing
A brief description of how tests will be performed
Expected problems
Is carried out in time boxed intervals
More structured than Error guessing
Exploratory
Risk Analysis Charter Debriefing
Sessions
Notes

Slide 243 EDS
Version 2.5
Choosing test Techniques
How do you chose the right technique?

Type of system
Standards
Customer or contractual requirements
Level of risk
Type of risk
Testing objectives
Documentation available
Knowledge / skills of the testers
Time and budget
Development processes
Pick the right techniques for the right
situation

Slide 244 EDS
Version 2.5
Test Design Techniques - Summary
We learned about
Identifying Test Conditions
Designing Test Cases from the Test Conditions
Creating Test Procedure Specifications to sequence our Test Cases
Creating Test Execution schedules to define the order in which the test scripts
are executed, when they are to be carried out and by whom
The importance of traceability to requirements and specification of expected
results
We learned about the difference between Black and White Box testing
White Box (Structure-based) Testing is based upon the structure of the program
code
Black Box (Specification Based) Testing is without reference the internal working
of the program code
The reasons why both are useful

Slide 245 EDS
Version 2.5
Test Design Techniques - Summary
We learned about the different Black box techniques, mainly:

Decision Tables
Use Case Testing
Experience based Testing
We learned about the different White box techniques, mainly:
Statement Testing
Decision Testing
For all Black and White box techniques we learned why they are of use
and for which test levels they are typically applied

Slide 246 EDS
Version 2.5
03-23-05
Version 2.5
Test Management

Test Management
Test Organisation
Test Planning and Estimation
Test Progress Monitoring and Control
Configuration Management
Risk and Testing
Incident Management
Summary

Slide 248 EDS
Version 2.5
Test Organisation
Test Independence
Testing Roles Within the Team
The Test Leader
The Tester

Slide 249 EDS
Version 2.5
Test Independence
Levels of Independence
More effective for someone other than the developer to test the system
More impartial
No preconceived ideas about what requires testing
No bias or emotional attachment
Levels of Test Independence:

1. Independent testers within the development teams.
2. Independent test team or group within the organization, reporting
to project management or executive management.
3. Independent testers from the business organization, user
community and IT.
4. Independent test specialists for specific test targets such as
usability testers, security testers or certification testers
5. Independent testers outsourced or external to the organization.

Slide 250 EDS
Version 2.5
Test Independence
Most test projects require multiple levels of testing where some of the testing is
carried out by independent testers.
Acceptance
Testing
System Independent
Integration
Testing Testers
System Testing
Developers
Component Testing

Slide 251 EDS
Version 2.5
Test Independence
Independent testing benefits & drawbacks
Benefits
Independent testers see other and different defects, and are unbiased
An independent tester can verify assumptions people made during
specification and implementation of the system
Usually a Cost saving
Better skills = more effective testing and fewer defects getting into production
For Third Party test outsourcing, better to rent than to own
Drawbacks
Isolation from the development team (if treated as totally independent).
Independent testers may be the bottleneck as the last checkpoint
Developers lose a sense of responsibility for quality
Can be a greater cost need to consider viability
For Third Party test outsourcing, the project carries the risk

Slide 252 EDS
Version 2.5
Testing Roles within the Team
Independent testing who does what
Where possible some or all of the testing should be carried out by

independent testers
Developers may contribute at the lower levels (typically Component Testing
and maybe Component integration)
Specialist testers mainly play a role at Functional, Non Functional and
System integration test levels
The Business and System Administrators provide testing for Acceptance
testing (e.g. UAT, OAT)
The Test processes and rules are often defined by the Independent testers
but must be agreed by management

Slide 253 EDS
Version 2.5
The Test Leader
Role of the Test Leader
Also known as Test Manager or Test Coordinator

This role may also be done by
Project manager
QA manager
Development manager
In large projects this role may be split into a
Test Manager
Test (team) Leader
Key tasks of a leader
Test Planning
Test Monitoring
Test Control

Slide 254 EDS
Version 2.5
The Test Leader
The Test Leaders tasks

Strategy & Management
Write and review the test strategy
Coordinate the test strategy
Plan testing effort context, risks & approach
Proactive representation in other project activities ensure testing has correct focus
Ensure proper configuration management of Testware exists
Determine what should be automated
Select and implement the most appropriate tools for testing including necessary training
Management and definition of the test environmental requirements
Define the test schedule based on the delivery of code in to test
Monitor
Define, record and continually review the testing project metrics
Monitor test progress against the test schedule
Write the test summary reports
Control
Adapt testing effort based results and progress

Slide 255 EDS
Version 2.5
The Tester
The role of the tester
Make up the majority of the resources in the testing group.

May be specialists in a particular area
Automation
Performance
Usability
Security
Or alternatively may work more generally doing:
Test Analysis
Test Design
Test Execution
Test Environment management
Test Data management
Typically testers at the component level are developers
At the Acceptance level testers are typically business experts or users or operators
(for operational acceptance testing)

Slide 256 EDS
Version 2.5
The Tester
The testers tasks
Review and contribute to test plans.

Analyse, review and assess user requirements, specifications and models for
testability.
Create test specifications.
Set up the test environment (often coordinating with system administration and
network management).
Prepare and acquire test data.
Implement tests on all test levels, execute and log the tests, evaluate the results and
document the deviations from expected results.
Use test administration or management tools and test monitoring tools as required.
Automate tests (may be supported by a developer or a test automation expert).
Measure performance of components and systems (if applicable).
Review tests developed by others.

Slide 257 EDS
Version 2.5
Test Planning and Estimation
Test Planning
Test Planning Activities
Exit Criteria
Test Estimation
Test Approaches

Slide 258 EDS
Version 2.5
Test Planning
Test strategies and test plans
All projects require a set of plans and strategies which define how the
testing will be conducted.
There are number of levels at which these are defined:
Test Policy Defines how the organisation

will conduct testing
Master Test Defines how the project will

Plan/Test Strategy conduct testing
Defines how each level of

Functional System Integ UAT
Test Plan Test Plan Test Plan testing will be conducted

Slide 259 EDS
Version 2.5
Test Planning
Test planning factors
Factors which affect test planning

The organisations test policy
Scope of the testing being performed
Testing objectives
Project Risks e.g. business, technical, people
Constraints e.g. business imposed, financial, contractual etc
Criticality (e.g. system/component level)
Testability
Availability of resources
Test plans are continuously refined
As more information becomes available
As new risks arise or others are mitigated
Not set in concrete, but changes must be carefully managed

Slide 260 EDS
Version 2.5
Test Planning
Test Plan Contents (IEEE 829)
1. Test Plan Identifier
2. Introduction
3. Test Items
4. Features to be Tested
5. Features not to be Tested
6. Approach
7. Item Pass/Fail Criteria
8. Suspension Criteria and Resumption Requirements
9. Test Deliverables
10. Testing Tasks
11. Environmental Needs
12. Staffing and Training Needs
13. Responsibilities
14. Schedule
15. Planning Risks and Contingencies
16. Approvals

Slide 261 EDS
Version 2.5
Test Planning Activities
Defining a test plan
Approach - Defining the overall approach of testing (the test strategy), including the
definition of the test levels and entry and exit criteria.
Integrating and coordinating the testing activities into the software life cycle
activities: acquisition, supply, development, operation and maintenance.
Making decisions about:
what to test
who i.e. what roles will perform the test activities
when and how the test activities should be done and when they should be stopped (exit
criteria see next slides)
how the test results will be evaluated
Assigning resources for the different tasks defined.
Testware definition- Defining the amount, level of detail, structure and templates
for the test documentation.
Selecting metrics for monitoring and controlling test preparation and execution,
defect resolution and risk issues.
Process - Setting the level of detail for test procedures in order to provide enough
information to support reproducible test preparation and execution.

Slide 262 EDS
Version 2.5
Exit Criteria
How do we know when to stop testing?
The business
Run out of tells you it went
time? live last night!
Boss says
Run out of stop?
budget?
All defects have

been fixed?
When our exit criteria
have been met?

Slide 263 EDS
Version 2.5
Exit Criteria
What are exit criteria?
Purpose of exit criteria is to define when we STOP testing either at the:

End of all testing i.e. product Go Live
End of phase of testing (e.g. hand over from System Test to UAT)
Exit Criteria typically measures:
Thoroughness measures, such as coverage of requirements or of code or risk
coverage
Estimates of defect density or reliability measures. (e.g. how many defects
open by category)
Cost.
Residual Risks, such as defects not fixed or lack of test coverage in certain
areas.
Schedules - such as those based on time to market.

Slide 264 EDS
Version 2.5
Test Estimation
Test Estimation - Why
estimates provide predictions of effort and cost for conduct of testing

activities that support :
establishment of budgets
procurement of resources
production of a schedule for use in tracking and control
by definition estimates are not exact and good estimates are ones that
provide predictions within realistic tolerance limits

Slide 265 EDS
Version 2.5
Test Estimation
Test Estimation - How
Estimation based on metrics from similar or previous projects or

typical values
Use metrics such as
Length of time in preparation
Length of time in execution
Defect turn around time
Down time and delays
However it is important that you compare metrics from similar sized
projects.
Size of project is also required
Function point analysis
LoC (lines of code)

Slide 266 EDS
Version 2.5
Test Estimation
Test Estimation - How
Estimating tasks by the owner of these tasks or by experts

Experts may use formal metrics
Or may be more gut feel and experience based

Slide 267 EDS
Version 2.5
Test Estimation
Estimation - Factors
The testing effort can depend on:
Characteristics of the product:
the quality of the specification
the size of the product
the complexity of the problem domain
Characteristics of the development process:
the stability of the organization
tools used
test process
skills of the people involved
time pressure
The outcome of testing
the number of defects
the amount of rework required.
Once estimation is complete the resource requirements and projects
schedules can be finalised
Slide 268 EDS
Version 2.5
Test Estimation
Test estimation is hard to do perfectly, but not terribly hard

to do well
Black - 2002

Slide 269 EDS
Version 2.5
Test Approaches
One method of classifying the way testing is done is by looking at when the
bulk of testing is carried out
Preventative Reactive
Test Design Test Design

done here done here
Requirements Delivered Code Delivered Go Live

Slide 270 EDS
Version 2.5
Test Approaches
More Approaches
Analytical approaches, such as risk-based testing where testing is directed to
areas of greatest risk.
Model-based approaches, such as stochastic testing using statistical information
about failure rates (such as reliability growth models) or usage (such as operational
profiles).
Methodical approaches, such as failure based (including error guessing and fault-
attacks), check-list based, and quality characteristic based.
Process- or standard-compliant approaches, such as those specified by industry-
specific standards or the various agile methodologies.
Dynamic and heuristic approaches, such as exploratory testing where testing is
more reactive to events than pre-planned, and where execution and evaluation are
concurrent tasks.
Consultative approaches, such as those where test coverage is driven primarily by
the advice and guidance of technology and/or business domain experts outside the
test team.
Regression-averse approaches, such as those that include reuse of existing test
material, extensive automation of functional regression tests and standard test suites.

Slide 271 EDS
Version 2.5
Test Approaches
Approaches Combined
A preventative, Risk & Process based approach would start

testing early and use a standard industry approach such as the V
model using the defined risks as a basis for testing
or
A reactive, heuristic approach - would start after the code has
been delivered and use unplanned testing techniques such as
exploratory testing

Slide 272 EDS
Version 2.5
Test Approaches
Selection of Approaches - Choosing the right approach
The selection of a test approach should consider the context, including:

Risk of failure of the project, hazards to the product and risks of product failure
to humans, the environment and the company
Skills and experience of the people in the proposed techniques, tools and
methods
The objective of the testing endeavour and the mission of the testing team
Regulatory aspects, such as external and internal regulations for the
development process
The nature of the product and the business

Slide 273 EDS
Version 2.5
Test Progress Monitoring and Control
Test Progress Monitoring

Test Reporting
Reporting and Interpreting Test metrics
Test Control

Slide 274 EDS
Version 2.5
Test Monitoring - Why
Need to know the status of the testing project at any given point in time
Need to provide visibility on the status of testing to other stake holders
Need to be able to measure your testing against your defined exit criteria
Need to be able to assess progress against
Planned schedule
Measure how you are tracking against your defined budget

Slide 275 EDS
Version 2.5
Test Monitoring - How
Typical Metrics include

Test preparation - Percentage of work done in test case preparation (or
percentage of planned test cases prepared).
Test environment preparation - Percentage of work done in test environment
preparation.
Test case execution (e.g. number of test cases run/not run, and test cases
passed/failed).
Defect statistics (e.g. defect density, defects found and fixed, failure rate, and
retest results).
Test coverage -e.g. % of requirements tested, risks or code coverage.
Dates of test milestones monitoring the test schedule
Testing costs, including the cost compared to the benefit of finding the next
defect or to run the next test.
Subjective confidence of testers in the product.

Slide 276 EDS
Version 2.5
Test Reporting
Test Reporting - What
Summarizes the testing carried out

What happened during testing, for example:
Were the dates met?
Was the Test Plan adhered to and if not, what
changes were made and why
What problems were encountered and how were
these addressed
Analysed metrics from testing, for example:

Defects remaining unresolved along with
associated risks and resolution plans
Number of tests run, not run, passed, failed etc

Slide 277 EDS
Version 2.5
Test Reporting
Test Reporting - What
Typical Contents are (IEEE 829)

Test summary report identifier;
Summary
Variances;
Comprehensive assessment;
Summary of results;
Evaluation;
Summary of activities;
Approvals.

Slide 278 EDS
Version 2.5
Reporting and Interpreting Metrics
Test Reporting - How
Metrics should be collected during and at the end of a test level in order to
assess:
The adequacy of the test objectives for that test level.
The adequacy of the test approaches taken.
The effectiveness of the testing with respect to its objectives.
Metrics are a valuable input into process improvement

Slide 279 EDS
Version 2.5
Interpreting Metrics testing progress
On the face of it things look good
Can we go live?
What else would we need to know?
However
What are the risks of the parts that
have failed
Does this chart account for all the
testing scheduled is there more to
come?

Slide 280 EDS
Version 2.5
Interpreting Metrics Defect reporting
Gap between = number of

Incidents still outstanding
No. Defects
Time

Slide 281 EDS
Version 2.5
Testing is a measurement of software quality.
Bill Hetzel 1988

Slide 282 EDS
Version 2.5
Test Control
Test Control - Why

Test control is the response to Test Monitoring and Test Reporting that
allows us to be IN CONTROL of the project
Projects do go awry and issues occur. We need Monitoring and Reporting to
know whether they are likely to go, or have already gone, awry.
The process of Control is the corrective actions required to put a testing
effort (project) back on track.
For Example;
Re-prioritize tests when an identified risk occurs (e.g. software delivered late).
Change the test schedule due to availability of a test environment.
Set an entry criterion requiring fixes to have been retested by a developer
before accepting them into a build.

Slide 283 EDS
Version 2.5
Configuration Management
Symptoms of Poor CM
Configuration Items and their control

Slide 284 EDS
Version 2.5
Symptoms of poor CM
Cant match source and object code

Cant identify source code changes made for a particular version of software
Simultaneous changes to source code by more than one programmer
(changes lost)
Old bugs previously fixed reappear
Incorrect (e.g. out of date) versions of documents or other objects are
used
You have no control over the composition of the test environment
(hardware and software items)

Slide 285 EDS
Version 2.5
The purpose of Configuration Management is to establish and maintain the

integrity of the products (components, data and documentation) of the
software or system through the project and product life cycle.
Items such as:

plans, specifications (requirements, design)
data dictionaries and cross-references
software modules (purchased or developed)
user and maintenance documentation
support software (libraries, databases, compilers, debuggers)
test Tools (software, discs, manuals etc)

Slide 286 EDS
Version 2.5
And just as importantly all items of Testware, such as:

Test Plans, Test Cases, Test Data, Conditions, Procedures, test environment
infrastructure components (hardware and operating software), test tools
(purchased and bespoke) etc
These must be:
Uniquely Identified
Version controlled
Tracked for changes
Related to each other
Related to other development items (functional specifications etc)
Referencing to other documents allows for traceability
The configuration management procedures and infrastructure (tools) should
be chosen, documented and implemented during the Test Planning stage.

Slide 287 EDS
Version 2.5
Risk and Testing
Testing and Risk

Measuring Risk
Project Risks
Product Risks
Risk Based Testing in practice

Slide 288 EDS
Version 2.5
Testing and Risk
Risk based testing
Testing is an exercise in risk reduction

Exhaustive testing is impossible
As testers we are under constant time pressure
Cannot reduce the risk of software failing to 0!
Risk can be classified as two distinct types
Project
Product

Slide 289 EDS
Version 2.5
Project Risk
What is Project risk?
Project Risk: A risk related to management and control of the (test) project.
Supplier Issues
Contractual Issues
Third party goes in liquidation or fails to deliver
Organisational Issues
Skills and Staff shortages
Training and support issues
Communication/Political Issues, e.g. between testers and other project teams
Technical
No or poor requirements
Quality of the design or code
Architectural solution under question

Slide 290 EDS
Version 2.5
Product Risk
What is Product risk?
Product risk: A risk directly related to the test object.

Buggy software delivered
Software does not meet its requirements
Software delivered may harm the organisation, data or individual
Poor software characteristics
Functionality
Security
Reliability
Usability
Performance
Product risks are a risk to the success of the project

Slide 291 EDS
Version 2.5
Risk Based Testing in Practice
Issue 1- We only have time to do half the testing we planned

If we have defined the risks of test items which we wish to test then we can focus on
highest risk items first and deliver an application which fit for purpose
Issue 2 we have 500 passes and 3 failures. Can we go live?

Again if we know the risk of the test items which make up 500 passes and 3 failures
we can then make a decision based on the potential risk of failure of the items left
out standing or which have failed
Risks can help decide where we should start testing or where we may need to
more testing
Risks also help us analyse our current state and through test monitoring we can
determine if a system is ready to be implemented
Risks can also drive the number of test levels and determine the techniques to use

Slide 292 EDS
Version 2.5
Prioritise so that when you stop testing you

have made the best of the time you have
available.
Weymouth
2006

Slide 293 EDS
Version 2.5
Testing can support the identification of new risks identified during test
planning
Testing is used to reduce the risk of an adverse effect occurring, or to
reduce its impact
Testing provides feedback about the residual risk- through measuring the
effectiveness of critical defect removal and contingency plans
In analysing, recording and managing the Product and Project risks the Test
Manager is following well defined project management principles
In a risk-based approach we can not only determine our test prioritisation
but also the test techniques to use

Slide 294 EDS
Version 2.5
Measuring Risk
RISK = Likelihood * Impact

High
High Impact and Likelihood
Needs to be addressed ASAP
Likelihood
Low Impact and Likelihood

Needs no action
Low
Low Impact High

Slide 295 EDS
Version 2.5
Functional Risk Matrix - MoSCoW

Likelihood Likelihood
Could Test Must Test

EP/BVA
Statement Coverage 70% Decision Tables
Branch coverage
Pair inspection
C A
Wont Test Should Test
Informal Test specification Formal Test Specification
Error Guessing Statement Coverage
100%
D B
Impact Impact

Slide 296 EDS
Version 2.5
Incident Management
Definition
Basic principles
Benefits
Attributes of an Incident
Tracking and Analysis
Writing Good Incident Reports

Slide 297 EDS
Version 2.5
Definition
Incident: Any event occurring that requires investigation
Or more simply put
Any discrepancy between actual and expected results
=
expected actual!
Slide 298 EDS
Version 2.5
Basic Principals
All incidents should be raised

Incident should be tracked from discovery through classification and
correction to confirmation
A formal incident management process and workflow is required to
effectively manage incidents
Incidents may be raised at ANY point in the lifecycle
Requirements review
Coding
Testing
Incidents may be raised AGAINST any product in the lifecycle
Source Requirements
System Designs
Source Code
Test Material
Help or user guides

Slide 299 EDS
Version 2.5
Benefits
Incidents:
Provide developers and other parties with feedback about the problem to enable
identification, isolation and correction as necessary
Provide test leaders a means of tracking the quality of the system under test and
the progress of the testing
Provide ideas for test process improvement

Slide 300 EDS
Version 2.5
Incident logging Key attributes
Date of issue, issuing organization, author, approvals and status

Scope, Severity* and Priority* of the incident
References, including the identity of the test case specification that
revealed the problem
*severity: The degree of impact that a defect has on the development or

operation of a component or system.
*priority: The level of (business) importance assigned to an item, e.g. defect.

Slide 301 EDS
Version 2.5
Incident logging Other attributes

Expected and actual results
Date the incident was discovered
Identification or configuration item of the software or system
Software or system life cycle process in which the incident was observed
Description of the anomaly to enable resolution
Degree of impact on stakeholder(s) interests
Status of the incident (e.g. open, deferred, duplicate, waiting to be fixed,
fixed awaiting confirmation test or closed)
Conclusions and recommendations
Global issues, such as other areas that may be affected by a change
resulting from the incident
Change history, e.g. of actions taken by project team members

Slide 302 EDS
Version 2.5
Tracking and analysis Incident Status examples
raised and logged resolved (closed)

prioritised and assigned invalid incident report (closed)
under investigation irreproducible problem (closed)
fixed by developers (or technical enhancement (to be actioned next
writers) upgrade)
tested
on hold (deferred pending some other
released to live environment action)

Slide 303 EDS
Version 2.5
Tracking and analysis - steps
categorise incidents by type

assign an appropriate priority according to incident severity
report incidents to the appropriate management level
determine incident causes and propose corrective measures
ensure timely corrective action by reviewing incidents and tracking their clearance
re- testing and reviews after changes have been made to the software (refer
regression testing)
review corrective measures to determine their effectiveness
analyse deficiency trends to prevent non-conforming software
N.B. Incident Review Boards often used to to oversee the reporting, analysis and
timely resolution of incidents during the course of a project. Members from all
interested parties, Project Manager, Test Manager, Lead developer, Customer, CM
Manager etc

Slide 304 EDS
Version 2.5
What makes a good incident report
The most important part of an Incident report is the failure description

It will provide the developer with all the information they need to repeat the
incident and then fix it
Badly written reports
Slow the project
Provide bad metrics
Delay fixes
Write in short sentences which are to the point accurate, complete and
concise
Contents of a failure description
Summary
Steps to reproduce
Isolation

Slide 305 EDS
Version 2.5
Contents of a failure description
Summary
one or two lines which provide an overview of the incident and its severity and
impact. Be sharp and to the point.
Steps to reproduce
provide the exact steps that were undertaken to create the incident. Be as
concise as possible, but make sure you add EVERY step
get these steps right makes it easier for the developers to reproduce. it
reduces the it works on my machine phenomenon
Isolation
is the incident repeatable, what particular factors effect the ability to reproduce
the incident, For example I observed the error on the following platforms IE5,
NS7

Slide 306 EDS
Version 2.5
Bad Example
Summary
There were a number or errors on the add customer screen
Steps to reproduce
1. Opened the add customer screen
2. Entered a new customer
3. Pressed add
4. Got error message
Isolation
I tried on a few different branches and it worked on most of them

Slide 307 EDS
Version 2.5
Good Example
Summary
Error cannot find object (see attached screen shot) message was displayed when
trying to add a new customer to the system using screen ADD_CUST.
Steps to reproduce
1.Opened the add customer screen using the menu
2.Entered a new customer (details are attached in spreadsheet)
3.Selected customer as corporate
4.Added to branch Littleton
5.Pressed add
6.Got error message cannot find object
Isolation
I tried on a few different branches and the system worked without issue when the
branch was classified as open. The Incident only occurred when the branch was
classified as closed

Slide 308 EDS
Version 2.5
Test Management - Summary
We learned about the Test organization
The importance of independent testing
The types of Independent test organisations
The benefits and drawbacks of independent testing within an organization
The different team members to be considered for the creation of a test team
The tasks of typical Test Leader and Tester
We learned about Test planning and estimation

The different levels and objectives of test planning
The purpose and content of the Test Plan and what we need to consider
when writing the Test Plan
Difference between different estimation approaches: the metrics-based
approach and the expert-based approach
Test preparation and execution tasks that need planning
The need for, and examples of, adequate exit criteria for specific test levels
and groups of test cases

Slide 309 EDS
Version 2.5
We learned about Test progress monitoring and control

Why we need to Monitor testing and how, i.e. what metrics we
gather
Understanding and interpreting the metrics for Test Reporting, plus
the contents of a Test Report
Why we need Test Control
We learned about Configuration Management

Why we need CM for testing and how it supports testing
Typical Configuration items, including Testware

Slide 310 EDS
Version 2.5
We learned about Risks and Testing

What a Risk is
The two main types of Risk - Project and Product
That risks are determined by likelihood (of happening) and impact
(harm resulting if it does happen)
How we should employ a Risk Based Testing approach benefits
Testing and the Project
And finally we learned about Incident Management

The definitions of Incidents and Defects
The Benefits of Incident Management
The Attributes of an Incident
How we should track and Analyse Incidents
And what makes a Good Incident Report

Slide 311 EDS
Version 2.5
03-23-05
Version 2.5

Types of Test tool

Effective Use of Tools Benefits and Risks
Introducing a Tool into an Organisation
Summary

Slide 313 EDS
Version 2.5
Types of Test Tools
Test Tool Classification

Tool Support for Management of Testing
Tool Support for Static Testing
Tool Support for Test Specification
Tool Support for Execution and Logging
Tool Support for Performance and Monitoring
Tool Support for Specific Application Areas
Tool Support using other tools

Slide 314 EDS
Version 2.5
Types of Test Tool
Test Tool Classification (1)

Tools are classified according to the test activities they support
(e.g. test management, static testing, etc.)
Some tools may support multiple activities while others just one
Tool vendors may provide a suite of (hopefully) integrated tools
for the testing lifecycle. (Major vendors include Mercury,
Compuware, Borland, etc.)
Tools can be any piece of purchased or locally developed software
Tools can be tremendously helpful in improving the efficiency and
quality of testing
but can also be a waste of time and effort

Slide 315 EDS
Version 2.5
Types of Test Tool
Test Tool Classification (2)

Some tools may be intrusive i.e. they affect the test outcome
itself this is the Probe Effect
For example, some tools that measure the performance of a piece of

code may insert code to count or time instructions though very
detailed information can be obtained, the timings will be affected
simply because of the extra code
Some tools offer greater support to developers rather than testers
e.g., Static Analysers are more commonly used by developers
Some tools offer greater support for different parts of the software
development lifecycle (e.g. Unit Testing, System Testing)
Computer Aided Software Test Tools (CAST Tools) is a common
term used in testing circles

Slide 316 EDS
Version 2.5
Types of Test Tool
Just like any carpenter, a tester needs a toolbox

full of tools to get the job done But
...like a good carpenter, a good tester is not
defined by his tools, but by his skill at using
them
Scott Loveland et al 2005

Slide 317 EDS
Version 2.5
Types of Test Tool
Tool Support for Management of Testing (1)

Management tools can be applied across the entire software development
lifecycle:
during unit testing, acceptance testing, etc.
and by all project staff - by managers, developers, testers, etc.

Slide 318 EDS
Version 2.5
Types of Test Tool

Test Management Tools:
Help Manage tests and activities
Provide management and control of test documentation, test cases,
specifications and results
Support scheduling of tests, logging results, recording test environments, etc.
Interface to other tools
Link to test automation tools to run tests automatically
Link to defect tracking tools to allow cross referencing to incidents/defects
Link to requirements management tools to cross reference tests and
requirements
Support Version control
Allow test cases, logs etc. to be baselined, checked-out, modified, retrieved
and controlled (often via another linked tool)

Slide 319 EDS
Version 2.5
Types of Test Tool

Test Management Tools - continued
Allow Traceability
Allow traceability to be recorded between test and requirements
Allow traceability to associated defects/incidents
Support Logging
Allow recording of results within the tool
Have results linked to useful information (who, when, on what hardware, etc.)
Provide Analysis
Produce analysis reports on testing progress and test coverage
Produce reports on tests passed and failed
Produce reports on incidents raised
Tools differ in their ease of use, accuracy of the information and portability of
results for distribution

Slide 320 EDS
Version 2.5
Types of Test Tool

Requirements Management Tools
Primary purpose is to store requirements (as simple text, as Use Cases,
formal language, etc.)
Generally provide a means of recording the priority of each requirement
To support testing, such tools can:
Verify and validate requirements (consistency checking, missing links to other
requirements)
Allow tests cases to be linked to the requirements
Test Teams should review requirements early in the lifecycle to check for
consistency, testability and to allow test cases to be constructed

Slide 321 EDS
Version 2.5
Types of Test Tool

Incident Management Tools
Also known as defect tracking tools

Record defects/problems/incidents/anomalies with virtually any aspect of
the project
E.g. operational problems, testing failures, documentation errors.
Fortunately, it is not common to use them for reporting problems with
people
Widely used by all members of a project team particularly testers

They support the prioritisation of incidents (e.g. High for incidents that
must be fixed in 4 hours through to Low for incidents that may never get
fixed.)

Slide 322 EDS
Version 2.5
Types of Test Tool

Incident Management Tools continued
Allow incidents and resulting actions to be assigned to project members.

E.g. to Investigate, to test, to cost, etc.
Record current status and history of an incident. E.g. New, or Rejected, or
Resolved, or Waiting for test, etc.
Some generate analysis reports on trends in incident creation, resolution,
effort recorded, etc.
Most can be tailored to specific project needs e.g. to define project specific
states or priorities, to specify the layout of information, to construct
predefined reports, etc.

Slide 323 EDS
Version 2.5
Types of Test Tool

Configuration Management Tools
Allow software baselines to be kept and managed (See notes for
background information)
Not strictly testing tools, but are heavily used by test teams to:
obtain specific version of software (code, documents, etc.)
maintain their own test material
Traceability between testware and software can be achieved by applying
build/release labels to corresponding items
Useful (often essential) when managing different version of a system for
different target environments or different operational releases

Slide 324 EDS
Version 2.5
Types of Test Tool
Tool Support for Static Testing (1)

Review Process support tools
During a review:
Documents are issued, people are nominated, roles are assigned
Templates and guidelines are distributed
Feedback is produced, times are recorded, people are nagged etc.
Tools support these activities by storing the documents, providing
communication mechanisms, recording metrics, issue reminders, etc.
Some support on-line reviews where people are geographically dispersed.
(Within EDS, eRoom is used to support online reviews)
Many companies (including EDS) use tailored Spreadsheets to record
comments and generate review metrics

Slide 325 EDS
Version 2.5
Types of Test Tool

Static analysis Tools
Static analysis tools provide information about the quality of the software
by examining the code but not executing the code.
Static Testing can often be carried out early in the development lifecycle
thus defects are detected sooner and (generally) more cheaply
Static analysis tools usually give objective measurements of various
characteristics of the software, such as the complexity measure
They are used by developers, testers and quality assurance personnel
Code defects detected include unused code, unused variables, coding
standard violations, etc.
Vendors include McCabe, Segue and many more.

Slide 326 EDS
Version 2.5
Types of Test Tool

Modelling tools
Projects may have analysis models, design models, data models, state
transition models, etc.
These models of the system are checked for defects
For example, a state transition model may have states with no entry point or
exit point, etc.
Tools will require the model to be in a format they can read - Perhaps as a
formal language or as part of the tool itself
Some tools actually produce test cases from the models
These tools allow testing to take place early - defects are identified sooner

Slide 327 EDS
Version 2.5
Types of Test Tool
Tool Support for Test Specification (1)

Test Design Tools
Test design tools generate some or all of the following automatically
Test inputs (i.e. the test data) this would be data from external system or user
Test cases
Expected results i.e. may use a test oracle
They use requirements, analysis models, data models, state models, code
Tools will require the model to be in a format they can read - Perhaps as a
formal language or as part of the tool itself
The tools either have built in or configurable test generation criteria
They may look for input boundary conditions, state transitions, paths
through the system, etc.

Slide 328 EDS
Version 2.5
Types of Test Tool

Test Design Tools - continued
Their value is from:
Ensuring thoroughness of test coverage by generating independent tests and
data
The time saved in test generation
Some Test design tools also generate test stubs and templates from models
and code

Slide 329 EDS
Version 2.5
Types of Test Tool

Test data preparation tools
Enable data to be selected from existing databases, files or captured
transmissions
Enable data to be created, generated, manipulated and edited for use in
tests
The most sophisticated tools can deal with a range of file and database
formats
The application under test is often used to Drive the data into databases,
etc.
They have the benefit of generating anonymous data for data protection
and security purposes.
i.e. They can generate data in the right format, with the right bounds, but isnt
actually live/real data

Slide 330 EDS
Version 2.5
Types of Test Tool
Tool Support for Execution and logging (1)

Test execution tools
Provide capture, replay and comparison facilities
Tools simulate mouse movement, button clicks and keyboard inputs
Can recognise GUI objects and states for later comparison windows, fields,
buttons, bit maps, etc.
Test scripts are normally captured in a programmable script language (VB, C,
Java, etc.)
Additional code can be added to the script for data manipulations,
verifications, iterations, etc.
When executed, tests run automatically, following the recorded/ programmed
script using the defined input data
Compare results with the predefined expected values and record the logs in
the tool (i.e. they use a Comparator).
A test Comparator may use a test oracle , especially if it is automated.

Slide 331 EDS
Version 2.5
Types of Test Tool

Test execution tools - continued
Generating automated scripts can be time consuming, challenging and fun
They are best used for tests which will be repeated many times (these tools
are most often used to automate regression tests)
Using the Capture-replay facilities of such tools, testers can record their
actions which can be very useful when failures occur. The complete history
of the user actions can be replayed and studied.
Can be used to replicate Users, i.e. multi-user testing.

Slide 332 EDS
Version 2.5
Types of Test Tool

Test harness/unit test framework tools
test harnesses, drivers and stubs simulate other parts of the system or
other connecting systems (or people)
They are used to:
Simulate other components that are not yet available so testing can continue
Simulate other components to allow detailed, controllable testing of a specific
component
Simulate events that may be difficult (even dangerous) to produce with the
whole system (e.g. database failures or nuclear meltdown)
Commercially available tools exist, but custom-written programs also fall
into this category
Simulators also come under this category

Slide 333 EDS
Version 2.5
Types of Test Tool

Test harness/unit test framework tools - continued
Test harnesses inject data into a component and capture outputs from the
component
Comparisons with previous runs and pre-defined expected outcomes can be
performed
Some frameworks are designed for developers for unit/component testing
-these are unit test frameworks (e.g. JUnit, HTTPUnit)

Slide 334 EDS
Version 2.5
Types of Test Tool

Test comparators
Comparison tools are used to detect differences between actual results and
expected results
Standalone comparison tools normally deal with a range of file or database
formats
Test execution tools usually have built-in comparators that deal with
character screens, GUI objects or bitmap images
These tools often have filtering or masking capabilities - they can ignore
rows or columns of data or areas on screens (e.g. volatile date fields)

Slide 335 EDS
Version 2.5
Types of Test Tool

Coverage measurement tools
These tools are dynamic test tools and allow test coverage to be monitored
Most commercial tools are designed for code coverage i.e. how much of the
code (statements, branches) has been executed
Provide objective measures of test coverage when tests are executed
Some tools add extra code to the source code (called instrumentation) to
allow detailed information to be obtained but recall the Probe Effect
After execution, the log file is analysed and coverage statistics generated
Statistics are provided on the most common coverage measures such as
statement or branch coverage

Slide 336 EDS
Version 2.5
Types of Test Tool

Security tools
With the rise of the internet, the need for preventing malicious attacks on
systems has increased
Tools have emerged which test a systems vulnerabilities to attacks
Virus scanners monitor for unexpected arrival of damaging code (via email,
embedded in web pages or documents, etc.)
Denial of service attacks
Tools examine code and attempt to cause a failure by injecting rogue data.
They may look for weaknesses in code (in HTML, .NET, etc.) deliberately trying
to damage the service
Penetration testing trying to break in. Tools will look for security
loopholes, monitor traffic, or password weaknesses, etc. and attempt to
gain access to machines or services

Slide 337 EDS
Version 2.5
Types of Test Tool
Tool Support for Performance & monitoring (1)

Dynamic Analysis tools
These are used to find defects that are apparent only
when the software is actually running (see also Static
Analysis Tools)
These tools are most commonly used to: -
identify unassigned pointers
monitor the allocation, use and de-allocation of memory to
flag memory leaks
These tools are generally used by developers

Slide 338 EDS
Version 2.5
Types of Test Tool

Performance testing tools
Performance test tools have two main facilities - load generation
measurement and test transaction measurement
Load generation can simulate either multiple users or high volumes of input
data
Load generation is done either by driving the application using its user
interface or by test drivers
Records of the numbers of transactions executed are logged
Response times are taken and logged for selected transactions
Performance testing tools normally provide reports based on test logs and
graphs of load against response times
Tools in this category are also called load test tools or stress test tools

Slide 339 EDS
Version 2.5
Types of Test Tool

Monitoring tools
These tools are not strictly test tools, but enable failures or potential
failures to be detected
Monitoring tools monitor the systems memory, CPU usage, file spaces,
network traffic, disk I/O, current processes running, database optimisation
(e.g. monitoring embedded into products such as Oracle) etc.
They can be tuned to report potential failures e.g. file space 80% full,
process XYZ should always be running, etc.

Slide 340 EDS
Version 2.5
Types of Test Tool
Tool Support for Specific Application areas

Tools exist which are specialised for a specific task, environment or
application
For example:
Harnesses, monitoring tools, etc for embedded systems where their
specialised hardware and application requires specialised tools
hyperlink testing tools (Spiders) are used to check that no broken hyperlinks are
present on a web site
Tools for a specific operating system (Windows, UNIX, etc.)
Tools for a specific language (JAVA, C++, .Net, etc)
Performance tools specifically for WEB based user interfaces

Slide 341 EDS
Version 2.5
Types of Test Tool
Tool Support using other tools

Many other tools are used to support the test effort.
For example:
Word processing tools widely used for producing test plans, test reports, to
record test cases, etc.
Spreadsheets also widely used for test cases, cross-referencing, analysis of
data, reporting, etc.
Database interrogation tools (e.g. SQL, Toad, Raptor) database interrogation is
performed to validate the contents of tables. Simple SQL statements can be
used or more sophisticated tools (Toad, Raptor, etc. which allow the novice to
interrogate a database)
Operating system utilities and scripts to monitor resources, generate data, run
tests
Code debugging tools to step through code, monitor variables, etc.

Slide 342 EDS
Version 2.5
Effective Use of Tools benefits and risks
Potential Benefits and risks all tools (1)

Some tools bring quick and easy benefit to the test effort, but
Some tools require considerable investment in time and thought
to realise their benefits
Dont just look at the adverts or listen to the salesman

Slide 343 EDS
Version 2.5

Potential Benefits of using tools
Repetitive work reduced Relieve the boredom
re-running the same test many times can be extremely boring and
error prone
Tools can reduce the time we spend on repetitive tasks
Consistency and repeatability Errare Humanum Est (to Err is
Human)
Test identified by a tool or run by a tool can be more repeatable and
consistent. Tools do what they are told
Objective assessment tools have no axe to grind
they are objective with their assessments and results
Ease of access to information
tools can give easy access to test information stored and present it in
multiple formats for distribution
Automated tests can be run out of hours
Slide 344 EDS
Version 2.5

Risks of using tools - every silver lining has a cloud
Unrealistic expectation - Dont just look at the adverts or listen to
the salesman. Tools may be rich in features but are they useful
to your test effort?
Underestimating how much effort/money is required to purchase,
install, get trained and simply get started with the tool
Underestimating how much effort is required to gain experience so
that benefits can be realised
Underestimating the effort to incorporate into the test process
Underestimating the effort required to maintain the assets
(automated scripts may need to change for every change in the
data or user interface)
Over-reliance not considering other tests that need to be found
or test that need to be run

Slide 345 EDS
Version 2.5
Selection of the tool affects the effectiveness and

efficiency of testing. .
The technique for hammering a nail into a piece of
wood is well understood
if the wrong hammer is selected the entire
process can be inefficient.
William E. Perry 2000

Slide 346 EDS
Version 2.5
Special Considerations for some tools (1)

Test Execution tools
Scripts for such tools are generated in a scripting language
(usually VB, C, Java)
Scripts can be very time-consuming to produce and difficult for
non programmers to maintain - which could delay significant
testing while creating automated tests
Scripts do exactly what they are told and nothing more. A
simple pop up from another application on the desktop could halt
the script
Two approaches are used to assist with script generation:
Data-driven approach script is produced to read test data from
(typically) a spreadsheet. Testers run tests with different data by
manipulating the spreadsheet and not the script
Key-word driven approach scripts are produced using building
blocks each given a key-word. E.g. Logon, Add_User,
Delete_User. Testers create tests by picking the blocks they need

Slide 347 EDS
Version 2.5

Performance Testing Tools
Requires specialists to design, execute and analyse the results
Often requires detailed understanding of how the system is
constructed (databases, networks, etc.)
Often requires detailed understanding of expected system usage
profiles
Need to be re-run when system tuning takes place

Slide 348 EDS
Version 2.5

Static Analysis Tools
Code Static Analysers examine code and reports on defects and
bad practice
Are best used at the start of coding to ensure problems are
removed quickly
But
Many systems contain automatically generated code, which often
causes static analysers to generate many reports
Applying these tools to legacy code can produce a flood of
unhelpful messages
Get the tool to filter out unwanted messages and so target the
required code

Slide 349 EDS
Version 2.5

Test Management Tools
Need to interface with other tools. E.g to import and export
information from word processors and spreadsheets
Must help your test process not hinder progress
Must ensure the information they hold is readily available for
viewing and distribution
Must produce reports easily

Slide 350 EDS
Version 2.5
Principles to follow (1)

Organisational Maturity
Tools should help the projects test effort and should fit into the culture
Spending time introducing automated execution tools for rapid
reaction or prototype projects may be of no value
Evaluation of product
Produce weighted Acceptance Criteria
Determine test tool options, prices and conditions
Produce a Return on Investment (ROI)
Investigate the market to see what products are available
Produce a Short List of suppliers
Conduct evaluation tests against application(s) with at least two
suppliers and score results

Slide 351 EDS
Version 2.5

Proof-of-concept before rolling out the tool across the
organisation, test it out on a manageable, resourced, pilot
project.
If it fails the impact will be minimised
If it succeeds greater experience will be available and organisational
roll out can be better supported
Evaluate the vendor
Discuss contractual issues company stability, licensing and
maintenance costs, usage restrictions
Evaluate training courses, technical support and consultancy
Review the tools development roadmap is it going in the right
direction for you
Internal Coaching
Identify staff to be trained
Who will provide internal support for the tools installation and usage

Slide 352 EDS
Version 2.5

Other questions to ask:
What protocols, languages, networks, hardware are being used now
and in future?
What are the budget and timescale for delivery?
What is the problem to be solved?
Are there alternative solutions other than tool purchase?
Are there any specific tool features and characteristics?
Do you need a project specific tool, or is this for the organisation?
But avoid too many tools of the same type
- tools are not just for Christmas

Slide 353 EDS
Version 2.5
Identifying, investigating, and evaluating tools

are important responsibilities of a test leader
But its important to recognize how much time
and effort is spent doing so.
Scott Loveland et al 2005

Slide 354 EDS
Version 2.5
Objectives of the Pilot project

Learn about the tool
No training course or brochure will give enough information about how
the tool actually works in your environment.
It is vital to learn about the tool before rolling it out for wider use
Is the tool a good fit?
Will the organisations processes accommodate this tool?
What organisational processes need to be updated?
Can it be integrated with other tools sets for improved effectiveness?
Generate a best practice guide
create guides and work instructions in the use of the tools
Benefits?
understand and measure as objectively as possible the benefits to be
gained
will it be worth the effort?

Slide 355 EDS
Version 2.5
Success factors for tool roll-out (1)

Incremental roll out
Avoids a sudden increase in demands for extra support and training
Allows experience to be gained and passed on ensuring a more
successful roll out
If problems are encountered, there is less impact if the roll out is
delayed or aborted
Adapt processes
Processes may benefit from changes because of the tool
For example, when and how regression testing is done (with
automation), how test cases are developed and recorded, etc.
Training
To avoid tools becoming shelfware new users will need local training
and support
Some users will be easily deterred by new tools

Slide 356 EDS
Version 2.5

Produce Guidelines
Tools may come with extensive user guides but it is useful to
supplement them with local conventions, FAQs, standards, etc.
make the guidelines easily accessible
Learn lessons and monitor usage
As tool usage increases capture users experience for sharing and
process improvement. Consider:
Web based forums
Email distribution groups
Discussion groups
Nominated tool authorities to approve changes
Regular assessment of benefits vs costs

Slide 357 EDS
Version 2.5

Other success factors:
Maintain close relationship with tool vendor to understand their
future plans, to get quick access to information
Monitor test tool market for new or improved products
Ongoing research into how to better utilise test tools
Use consultants to resolve short term or difficult issues where there is
no gain from wasting resource time on complex one-off problems

Slide 358 EDS
Version 2.5
EDS Application Engineering Toolkit (AET)
The Applications Engineering Tools team evaluates, recommends and pilots

standard Enterprise Application Development Life Cycle tools, utilities and
frameworks to enable the delivery of service offerings within the global
applications engineering organization
AET Objectives are to: -
Improve applications development sustainable productivity and reduce rework
through the deployment of integrated toolkit and framework recommendations
Decrease operating cost and improve functionality for dollar spent through
implementation of application software and database product replacement
recommendations
http://www.gsms-ea.eds.com/Tools/index.html

Slide 359 EDS
Version 2.5
Collateral on Tools
External Sites (non supplier based)

QA Forums - http://qaforums.com/
StickyMinds - http://www.stickyminds.com/
Automation Junkies - http://www.automationjunkies.com/
Software Test & Performance - http://www.stpmag.com/

Slide 360 EDS
Version 2.5
Summary (1)
Firstly, we looked at types of test tool and how they are classified

Slide 361 EDS
Version 2.5
Summary (2)
Test management Requirements management
Test management tools
Incident management Configuration management
Static analysis Review process support
Static testing tools
Modelling
Test specification tools Test design Test data preparation
Test execution Test harnesses

Test execution and logging
Test comparators Coverage measurement
tools
Security
Dynamic analysis Performance/load/stress
Performance, monitoring tools
Monitoring
Embedded systems Web testing
Application specific tools
Language specific
Word Processors Operating system utilities
Other tools
Spreadsheets SQL, Code debugging

Slide 362 EDS
Version 2.5
Summary (3)
Then we looked at the effective use of tools:
Benefits of tools:
Save time, thorough testing, reduce repetitive tasks, etc.
And the risks:
Underestimating time and effort, Over-reliance, etc.
Special considerations for:
Test execution tools, performance tools, static analysis tools and
test management tools
And finally, we looked at how to introduce a tool:
Main principles (evaluation, assessments, etc.)
Pilot project objectives
Success factors

Slide 363 EDS
Version 2.5
Index
The following slides provides a quick reference guide to some of the key terms used in the course, to aid revision. Each term is hyperlinked
and has slide numbers plus associated ISQTB Syllabus section numbers. Remember to use the Glossary of terms for exact definitions.
Term Slide(s) Topic Syllabus

reference
Acceptance Testing 116 Testing Throughout the S/W Lifecycle 2.2.4
Black Box Testing 197 Test Design Techniques 4.2
Boundary Value Analysis 211 Test Design Techniques 4.3.2
Component Integration Testing 85 Testing Throughout the S/W Lifecycle 2.2.2
Component Testing 80 Testing Throughout the S/W Lifecycle 2.2.1
Configuration Management 282 Test Management 5.4
Decision Tables 214 Test Design Techniques 4.3.3
Decision Testing 229 Test Design Techniques 4.4.2
Equivalence Partitioning 204 Test Design Techniques 4.3.1
Error 10 Fundamentals of Testing 1.1.2
Error Guessing 239 Test Design Techniques 4.5
Exit criteria 261 Test Management 5.2.3

Slide 364 EDS
Version 2.5
Index
Term Slide(s) Topic Syllabus reference
Exploratory Testing 241 Test Design Techniques 4.5
Failure 12 Fundamentals of Testing 1.1.2
Fault 10 Fundamentals of Testing 1.1.2
Formal Review Process 166 Static Techniques 3.2.1
Functional Testing 131 Testing Throughout the S/W Lifecycle 2.3.1
Fundamental Test process 44 Fundamentals of Testing 1.4
General Principles of Testing 34 Fundamentals of Testing 1.3
Incident Management 295 Test Management 5.6
Independent Testing 248 Test Management 5.1.1
Informal Review 162 Static Techniques 3.2.3
Inspection 165 Static Techniques 3.2.3
Maintenance Testing 140 Testing Throughout the S/W Lifecycle 2.4
Non- Functional Testing 132 Testing Throughout the S/W Lifecycle 2.3.2
Product Risk 289 Test Management 5.5.2
Project Risk 288 Testing Throughout the S/W Lifecycle 5.5.1
Regression Testing 136 Testing Throughout the S/W Lifecycle 2.3.4
Re-testing (Confirmation) 135 Testing Throughout the S/W Lifecycle 2.3.4

Slide 365 EDS
Version 2.5
Index
Term Slide(s) Topic Syllabus
reference
Risk Based Testing 287 Static Techniques 5.5
State Transition Testing 217 Test Design Techniques 4.3.4
Statement Testing 225 Test Design Techniques 4.4.1
Static analysis 178 Static Testing 3.3
Structural Testing 133 Testing Throughout the S/W Lifecycle 2.3.3
System Integration Testing 111 Testing Throughout the S/W Lifecycle 2.2.2
System Testing 95 Testing Throughout the S/W Lifecycle 2.2.3
Technical Review 164 Static Techniques 3.2.3
Test Case 191 Test Design Techniques 4.1
Test Condition 190 Test Design Techniques 4.1
Test Estimation 263 Test Management 5.2.4
Test Execution Schedule 194 Test Design Techniques 4.1
Test Monitoring/Control 272 Test Management 5.3.1
Test Objectives 27 Test Management 1.2
Test Planning 256 Test Management 5.2.1
Test Procedure 193 Test Design Techniques 4.1

Slide 366 EDS
Version 2.5
Index
Term Slide(s) Topic Syllabus reference
Test Tool types 312 Test Tools 6.1
Use Case Testing 222 Test Design Techniques 4.3.5
V-model 72 Testing Throughout the S/W Lifecycle 2.1.1

Walkthrough 163 Static Techniques 3.2.3
White Box Testing 199 Test Design Techniques 4.2

Slide 367 EDS
Version 2.5
Statement of Confidentiality
The information contained in this document is proprietary to

EDS. EDS submits this document with the understanding that it
will be held in strict confidence and will not be disclosed,
duplicated or used, in whole or in part, for any purpose other
than the evaluation of EDS qualifications, without the prior
written consent of EDS.
EDS is a registered mark and the EDS logo is a trademark of Electronic Data Systems
Corporation.
EDS is an equal opportunity employer and values the diversity of its people.
Copyright 2006 Electronic Data Systems Corporation. All rights reserved.

Slide 368 EDS
Version 2.5
30
Version
Sept 2005
2.5
Presentation and Course owner Paul Weymouth, UKIA Testing ADU Paul.Weymouth@eds.com
Presentation and Course contributors:
Paul Weymouth, Testing Architect UKIA Testing ADU
Mark Otter, Senior Test Consultant Australia South ADU
Dave Broughton, Testing Architect UKIA Testing ADU
EDS and the EDS logo are registered trademarks of Electronic Data Systems Corporation. EDS is an equal opportunity employer
and values the diversity of its people. 2005 Electronic Data Systems Corporation. All rights reserved.

EDS ISTQB TFC v2.6

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EDS ISTQB TFC v2.6

Uploaded by

Copyright:

Available Formats

03-23-05

EDS ISTQB Testing Foundation Course

Presentation by Paul Weymouth, UK Testing ADU

Slide 1 EDS Internal

Click on the symbol to go to the next slide.

Click on the symbol to go to the previous slide.

Click on the symbol to go to the main Areas Covered slide.

Click on the symbol to move on to associated slides.

Click on the symbol to go back to the previous menu.

You may also see the following symbols displayed :-

EDS ISTQB Testing Foundation Course Version 2.5

Refers to the ISTQB Glossary Of Terms.

This is a Pearl of Wisdom. Typically a quote

EDS ISTQB Testing Foundation Course Version 2.5

The primary objective of this course is to provide testing foundation

EDS ISTQB Testing Foundation Course Version 2.5

Index of key terms

EDS ISTQB Testing Foundation Course Version 2.5

Slide 6 EDS Internal

Why Testing is Necessary

EDS ISTQB Testing Foundation Course Version 2.5

Software Systems Some context

EDS ISTQB Testing Foundation Course Version 2.5

Software Systems When things go wrong

EDS ISTQB Testing Foundation Course Version 2.5

Causes of Software Failure

* ISTQB Standard Glossary of Terms

EDS ISTQB Testing Foundation Course Version 2.5

Errors, Defects and Failures

EDS ISTQB Testing Foundation Course Version 2.5

Errors, Defects and Failures

A Failure is a Deviation of the component or system from its

* ISTQB Standard Glossary of Terms

EDS ISTQB Testing Foundation Course Version 2.5

Errors, Defects and Failures

Deviation of the component or

EDS ISTQB Testing Foundation Course Version 2.5

The Role of Testing

EDS ISTQB Testing Foundation Course Version 2.5

Testing and Quality

EDS ISTQB Testing Foundation Course Version 2.5

Testing and Quality

EDS ISTQB Testing Foundation Course Version 2.5

Testing Pearl of Wisdom

Quality is not intangible.

Bill Hetzel 1988

EDS ISTQB Testing Foundation Course Version 2.5

Why exhaustive testing is impossible

Exhaustive testing of complex software applications:

EDS ISTQB Testing Foundation Course Version 2.5

Testing Pearl of Wisdom

In any form of testing it is impossible to achieve

Bill Hetzel 1988

EDS ISTQB Testing Foundation Course Version 2.5

Why dont we test everything ?

Approximate total for exhaustive testing

Test length = 1 sec then test duration = 17.7 days

EDS ISTQB Testing Foundation Course Version 2.5

So, How Much Testing is Enough?

EDS ISTQB Testing Foundation Course Version 2.5

So, How Much Testing is Enough?

EDS ISTQB Testing Foundation Course Version 2.5

So, How Much Testing is Enough?

Exit criteria may be defined in terms of :-