An

Introduction
to Testing
Life Cycle
Process
 Contents

 Testing Life Cycle Overview

 Test Strategy
 Test Planning
 Test Case Design
 Test Execution
 Testing Types
 Testing Tools Overview
Testing Life Cycle Overview
 Testing Life Cycle Overview
 Test Strategy
 Test Planning
 Test Case Design
 Test Execution
 Testing Types
 Testing Tools Overview
 Testing Life Cycle Overview

•Normally, testing of any Large Systems will be in TWO

parts. The functional verification and validation against
the Requirement Spec and Performance evaluation
against the indicated requirements

Testing activity is involved right from the beginning of the

project  Testing
Life Cycle
V&V Process model to successfully deliver a project on Overview
 Test
time Strategy
 Test

This model is followed for a complete project life cycle Planning

 Test Case

model Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Testing Life Cycle - V&V
Process Model
Business Acceptance Tests
requirements, Master Test Strategy Release &
Solution Certification
Architecture

Performance
Project Planning Integrated Testing,
Test Planning
Solution Business Cycle
testing

HLD/LLD for each Strategy for  Testing

application individual Life Cycle
Application Incremental
involved applications, Test Overview
Integration Integration Testing
case design  Test
Interface design
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
Solution System Testing
 Testing
Mapping/ (Application level
Types
Development/ Testing)
 Testing
Customization Tools
Overview
 Testing Life Cycle - V&V
Process Concept
The "V &V " concept relates the build components of
development to the test components that occur during
that build phase.
Testing related activities during Requirement phase

Creation and finalization of testing templates  Testing

Life Cycle
Creation of over-all Test Plan and Test Strategy Overview
 Test
Capturing Acceptance criteria and preparation of Acceptance Strategy
 Test
Test Plan Planning
 Test Case
Capturing Performance criteria of the software requirements Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Life Cycle - V&V
Process Concept …
• Testing activities in System Testing phase
– System test is done for validating the product with respect to client
requirements
– Testing can be in multiple rounds
– Defect found during system test should be logged into Defect
Tracking System for the purpose of tracking.
– Test logs and defects are captured and maintained.  Testing
– Review of all the test documents Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Testing Life Cycle - V&V
Process Concept…
Testing activities in Design phase
Develop Test cases to ensure that product is on par with
Requirement Specification document.
Verify Test Cases & test scripts by peer reviews.
Preparation of traceability matrix from system requirements

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Life Cycle - V&V
Process Concept …
• Testing activities in Integration Testing Phase
– This testing is conducted in parallel with integration of various
applications (or components)
– Testing the product with its external and internal interfaces
without using drivers and stubs.
– Incremental approach while integrating the interfaces.

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Life Cycle - V&V
Process Concept …
• Performance Testing – This is done to validate the
performance criteria of the product / application.
This is non-functional testing
• Business Cycle testing – this refers to end to end
testing of real life-like business scenarios.
• Testing activities during Release phase
– Acceptance testing is conducted at the customer location.  Testing
– Resolves all defects reported by the customer during Life Cycle
Overview
Acceptance Testing  Test
Strategy
– Conduct Root Cause Analysis (RCA) for those defects  Test
Planning
reported by customer during acceptance testing  Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Life Cycle - Team
Structure
• An effective testing team includes a
mixture of members who has
– Testing expertise
– Tools expertise
– Database expertise
– Domain/Technology expertise  Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Life Cycle - Team
Structure (Contd…)
• The testing team must be properly structured,
with defined roles and responsibilities that allow
the testers to perform their functions with
minimal overlap.
• There should not be any uncertainty regarding
which team member should perform which
duties.  Testing
Life Cycle
Overview
• The test manager will be facilitating any  Test
Strategy

resources required for the testing team.  Test

Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Testing Life Cycle - Roles and
Responsibilities
• Clear Communication protocol should be
defined with in the testing team to ensure
proper understanding of roles and
responsibilities.

• The roles chart should contain both on-site  Testing

and off-shore team members. Life Cycle

Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Testing Life Cycle - Roles and
Responsibilities
• Test Manager
– Single point contact between onsite and
offshore team
– Prepare the project plan
– Test Management
– Test Planning
 Testing
– Interact with onsite lead, Client QA manager Life Cycle
Overview
 Test
– Team management Strategy
 Test

– Work allocation to the team Planning

 Test Case
Designing
– Test coverage analysis  Test
Execution
 Testing
Types
 Testing
Tools
Overview
Testing Life Cycle - Roles and
Responsibilities
• Test Manager cont..
– Co-ordination with onsite for issue resolution.
– Monitoring the deliverables
– Verify readiness of the product for release through release
review
– Obtain customer acceptance on the deliverables
– Performing risk analysis when required  Testing
Life Cycle
Overview
– Reviews and status reporting  Test
Strategy
– Authorize intermediate deliverables and patch releases to  Test
Planning
customer.  Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Testing Life Cycle - Roles and
Responsibilities
• Test Lead
– Resolves technical issues for the product group
– Provides direction to the team members
– Performs activities for the respective product group
– Review and Approve of Test Plan / Test cases
– Review Test Script / Code
– Approve completion of Integration testing  Testing
Life Cycle
– Conduct System / Regression tests Overview
 Test
– Ensure tests are conducted as per plan Strategy
 Test
– Reports status to the Offshore Test Manager Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Life Cycle - Roles and
Responsibilities
• Test Engineer
– Development of Test cases and Scripts
– Test Execution
– Result capturing and analysing
– Defect Reporting and Status reporting
 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Life Cycle - Term
Definitions
• Testing: It is a process of executing software in a
controlled manner, in order to answer the question
“Does the software behave as specified?”
• Verification: Says “Are we building the product
right?” All QC activities throughout the life cycle that
ensure that interim deliverables meet their input
specification.
• Validation: Says “Are we building the right product?”
The test phase of the life cycle which assures that the  Testing
Life Cycle
end product meets the user’s needs. Overview
 Test
• Test Case: “An input operation and the corresponding Strategy
 Test
expected output” in order to test a small unit of work. Planning
 Test Case
• Test Script: “A logical group of test cases which, when Designing
 Test
taken together, test a particular function or unit of a Execution
 Testing
system” Types
 Testing
Tools
Overview
 Testing Life Cycle - Term
Definitions (Contd.)
• Test Bed: “A group of test scripts which, when taken
together, test all functions of an entire system”
• Audit: An inspection/assessment activity that verifies
compliance with plans, policies and procedures
• Baseline: A quantitative measure of the current level of
performance
• Bug: A catch all term for all software defects or errors
• Certification: Acceptance of a software by an
authorized agent or its validity been demonstrated to an  Testing
Life Cycle
authorized agent Overview
 Test
• Cost of Quality (COQ): Money spent above and beyond Strategy
 Test
expected production costs, this involves prevention, Planning

appraisal, internal & external failure costs

 Test Case
Designing
 Test
• Cyclomatic Complexity: The number of decision Execution
 Testing
statements plus one. Types
 Testing
Tools
Overview
 Testing Life Cycle - Term
Definitions (Contd.)
• Defect: It is “A deviation from specification or
standard” or “Any thing that causes customer
dissatisfaction”.
• Desk Check: Verification technique conducted by the
author of the artifact to verify the completeness
• Dynamic Assertion: A dynamic analysis technique that
inserts into the program code assertions about the
relationship between program variables
• Force field Analysis: A group technique used to  Testing

identify both driving and restraining forces that Life Cycle

Overview
influence a current situation  Test
Strategy

• Inspection: A formal assessment of a work product  Test

Planning
conducted by one or more qualified independent  Test Case
Designing
reviewers to detect defects, violation of development  Test
Execution
standards, etc. Inspection identifies defects but does  Testing

not attempt to correct them. Types

 Testing
Tools
Overview
 Testing Life Cycle - Term
Definitions (Contd.)
• Instrumentation: The insertion of the additional code
into a program to collect information about a
program behavior during program execution.
• Life Cycle Testing: The process of verifying the
consistency, completeness and correctness of software
at each phases of the development life cycle.
• Phase Containment: Method of control put in place
within each stage of the development process to  Testing
Life Cycle
promote error identification and resolution so that Overview
 Test

defects are not propagated downstream to subsequent Strategy

 Test

stages of the development process

Planning
 Test Case
Designing
• Productivity: The ratio of the output of a process to  Test
Execution

the input  Testing

Types
 Testing
Tools
Overview
 Testing Life Cycle - Term
Definitions (Contd.)
• Quality Assurance: The set of activities (including
facilitation, training, measurement, and analysis)
needed to provide adequate confidence that process are
established continuously improved to produce
products that meet specifications and are fit for use.
• Quality Control: The process by which product quality
is compared and detected w.r.t requirements and other
relevant specifications, focus is in detection and
removal  Testing
Life Cycle
• Test Driver: A program that directs the execution of Overview
 Test
another program against a collection of test data sets Strategy
 Test
• Test item: A software item that is an object of testing Planning
 Test Case
Designing
• User: The customer that actually uses the product  Test
Execution
received  Testing
Types
 Testing
Tools
Overview
 Test Strategy
 Testing Life Cycle Overview
 Test Strategy
 Test Planning
 Test Case Designing
 Test Execution
 Testing Types
 Testing Tools Overview
 Test Strategy
• Test strategy is statement of overall approach of
testing to meet the business and test objectives.
• It is a plan level document and has to be prepared
in the requirement stage of the project.
• It identifies the methods, techniques and tools to be
used for testing .
• It can be a project or an organization specific.
• Developing a test strategy which effectively meets  Testing
Life Cycle
Overview
the needs of the organization/project is critical to  Test
Strategy
the success of the software development  Test
Planning

• An effective strategy has to meet the project and  Test Case

Designing
 Test
business objectives Execution
 Testing
• Defining the strategy upfront before the actual Types
 Testing

testing helps in planning the test activities Tools

Overview
 Elements of Test Strategy
• A test strategy will typically cover the following
aspects
– Definition of test objective
– Strategy to meet the specified objective
– Overall testing approach
– Test Environment
– Test Automation requirements
– Metric Plan
– Risk Identification, Mitigation and Contingency plan  Testing
Life Cycle
– Details of Tools usage Overview
 Test

– Specific Document templates used in testing Strategy

 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Strategy - Project/Business
Objectives
• All the business and test objectives are
identified and test strategy shall be defined
in order to meet those objectives.
• As far as possible all the test objectives to
be met are defined in quantitative terms in
the test strategy.
• Examples of test objectives can be like 90  Testing
Life Cycle
% of code coverage, finding at least a Overview
 Test

minimum of 100 defects etc. Strategy

 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Strategy - Test Approach
• Test approach will be based on the objectives
set for testing
• Test approach will detail the way the testing to
be carried out
• Types of testing to be done viz Unit,
Integration and system testing
• The method of testing viz Black–box, White-  Testing
Life Cycle

box etc., Overview

 Test
Strategy

• Details of any automated testing to be done  Test

Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Strategy - Test Environment
• All the Hardware and Software requirements
for carrying out testing shall be identified in
detail.
• Any specific tools required for testing will also
be identified
• If the testing is going to be done remotely, then
it has to be considered during estimation  Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Strategy - Metric Plan
• Metric plan will contain the measures that will
indicate the effectiveness of testing .
• All the metrics to be tracked for testing like
defect density ,residual defect density ,code
coverage etc are identified in test strategy .
• The objectives set for testing are also measured
in the metric plan.  Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Note: More details are covered in Test Execution section Overview
 Test Strategy - Risks
• Risk analysis should carried out for testing phase
• The risk identification will be accomplished by
identifying causes-and-effects or effects-and-causes
• The identified Risks are classified into to Internal and
External Risks. The internal risks are things that the
test team can control or influence. The external risks
are things beyond the control or influence of the test
team
• Once Risks are identified and classified, the following  Testing

activities will be carried out Life Cycle

Overview

– Identify the probability of occurrence  Test

Strategy

– Identify the impact areas – if the risk were to occur  Test

Planning
– Risk mitigation plan – how avoid this risk?  Test Case
Designing
– Risk contingency plan – if the risk were to occur what do we  Test
Execution
do?  Testing
Types
 Testing
Tools
Overview
 Test Strategy - Tools Usage
• Tools required in the project and rationale to use
that particular tool(s) should be documented.
• Automation requirements should be identified
and analyzed
– Which phases need to be automated
– How much of testing needs to be automated
• Based on the automation requirements, tools  Testing
Life Cycle
strategy should be prepared. Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Strategy - Templates
• Any specific templates to be used for testing apart
from the normal templates based on client or project
requirements are identified in the test strategy.
• In some case client provides the templates and we’ll
use that.

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
Sample Strategy
 Testing
doc Tools
Overview
 Test Strategy - Acceptance criteria
• Acceptance test plan can be prepared either by
us or by the customer, as agreed.
• Customer will be involved in preparation, review
and approval of acceptance test cases
• This plan specifies the criteria for client
acceptance of the final tested product including
Features/functionalities to be tested and  Testing
Life Cycle
traceability information. Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Strategy - Summary
• Test strategy is a high level document which
states in detail way testing is to be carried out.
• It is done in line to meet the test and business
objectives
• Tools ,Methods ,and techniques to be used are
stated in the test strategy.
 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning
 Testing Life Cycle Overview
 Test Strategy
 Test Planning
 Test Case Designing
 Test Execution
 Testing Types
 Testing Tools Overview
 Test Planning - Documenting
assumptions
• If any assumptions are there, it needs to be
documented to avoid any confusions
– Testers has understanding of V&V-process model or
any SDLC model
– Testers are aware of test phases of the project
– A good understanding of the Domain
– Standard naming conventions are followed in
documentation  Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning -Traceability
• Requirements tracing is the process of
documenting the links between the user
requirements for the system you are building and
the work products developed to implement and
verify those requirements.
• Work products include software requirements,
design specifications, software code, test plans and
other artifacts of the systems development process
 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
Traceability Matrix  Testing
Types
 Testing
Tools
Overview
 Test Planning -Traceability
• Traceability is documenting of
– the dependencies and logical links between individual
requirements and other system elements
– traceability to greatly facilitate change impact
analysis.
– traceability is that it can help identify the
propagation of change that can result when a specific
requirement is deleted or modified.  Testing
Life Cycle
– specific requirements mapped into tasks in a work- Overview
 Test
breakdown structure, you know those tasks that will Strategy
 Test
be affected if and when a requirement is changed or Planning
 Test Case
deleted. Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning - Schedule/
Milestones
• Test plan captures the details about different
milestones of the project and the schedule
details.
• Define the schedule for each event for offshore
and onsite. Events may include familiarization,
training, test case preparation, test scripting,
test execution etc.,  Testing
Life Cycle
• Completion criteria for each milestone should Overview
 Test

be identified and documented Strategy

 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning - Resources
• Staffing:
– Check what types of personnel should be
considered for inclusion on the test team
– Check for the skills of the tester
– Ensure a good ratio of testers to developers.
• Steps to form the best team:
– Identify potential test team members  Testing
Life Cycle
– Recruit test team members and develop Overview
 Test

tentative test assignments Strategy

 Test
Planning
– Define individual work assignments  Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning - Test run plan
• The execution order of test Release Plan 4

cases depends on many 1400

Cumulative TCs
1200

factors and is shown as 1000

Executed
800

below: 600
400
200
0
7/4 7/5 7/6 7/7 7/8 7/9 7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17
4-Jul 5-Jul 6-Jul 9-Jul 10-Jul 11-Jul 12-Jul 13-Jul 16-Jul 17-Jul
Plan 106 257 575 669 767 877 983 1065 1119 1119

Impacted Actual 297 483 632 737 855 939 1015 1079 1140 1147

# of Days
Functionality
 Testing
Life Cycle
Overview
Unstable Execution
TC priority Test
 Test
functionality Order Strategy
Execution  Test
Planning
Run Plan  Test Case
Customer Execution Designing
Priority Complexity rate  Test
Execution
 Testing
Types
Functionality TC Inter- Execution  Testing
Tools
Dependency dependency Order Overview
 Test Planning - Change
management
• Version control - Versioning test cases before
and after release to customer
• Change control - Authority to approve and
prioritize changes
• Configuration auditing - Ensure changes made
properly
• Reporting
 Testing
• Addition/Updating of test cases/plans because of Life Cycle
Overview

bugs which are identified by the customer and  Test

Strategy

that are not available in test plans  Test

Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
• Deadlines (release/testing deadlines etc.) are
met and quality of the product is as per the
agreed norms.
• Test cases completed with certain percentage
passed
• Test budget depleted
• Coverage of code/functionality/requirements  Testing
Life Cycle

reaches specified point Overview

 Test
Strategy

• Beta or Alpha testing period ends  Test

Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
Determination of Test Completion criteria
• Most popular method of establishing a test
phase completion criteria is by detecting a pre-
defined number of bugs or when a pre-defined
time elapses
• E.g.: Unit testing of a unit is not deemed
complete unless at least 5 errors are detected  Testing
Life Cycle
Overview
• Testing completion criteria could be detection  Test
Strategy
and fixing of 30 bugs, or elapsed time of 30  Test
Planning

days, whichever is later.  Test Case

Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
Estimation of number of errors
• How to determine the number of errors to be
detected?
– An estimate of total number of errors in the
program.
– An estimate of what percentage of these errors can
be feasibly found through testing.  Testing
Life Cycle
– Estimates of what fraction of errors originated in a Overview
 Test
particular phase, during what testing phases these Strategy

are likely to be detected.  Test

Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
• Methods of estimation
– Rough estimate of total number of errors can be obtained
by previous experience on similar programs.
– Error-seeding techniques – involves intention introduction
(seeding) of error in the code. After some period of testing
time, the product of the number on non-seeded errors
found during testing and the ratio of total seeded errors to
the number of seeded errors detected during testing, gives
an estimate of the number of non-seeded errors remaining  Testing
in the program. Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
• Estimation of errors in a particular phase - Example
– Existing data reveals that in large programs, 40% of errors
are coding and logic-design mistakes and the rest are
generated in earlier phases.
– Example to illustrate estimation of total number of errors –
– We have a 10,000 statement program, and the number of
errors remaining in the program after code review is 5 per
100 statements. Our objective is to find 98% of coding and
logic design errors and 95% design errors. Therefore-  Testing
Life Cycle
• Total number of estimated errors = 500 Overview
 Test
• Coding and Logic design errors = 200 (40% of 500) Strategy
 Test
• Design errors = 300. Planning
 Test Case
• We need to find at least: Designing
 Test
• 196 Coding and Logic Design errors Execution
 Testing
• 285 Design errors. Types
 Testing
Tools
Overview
Test Planning – Suspension and
Resumption criteria
Estimation of error distribution in different phases
Coding & Logic Design Errors
Design errors

Unit Test 65% 0%

Module 30% 60%  Testing

Life Cycle
Integration Test Overview
 Test
Strategy
 Test
Planning
System Test 3% 35%  Test Case
Designing
 Test
Execution
 Testing
Total 98% 95% Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
Establishment of completion criteria
• Exit Criteria for Unit testing would be detection and
correction of 130 errors (65% of estimated 130
coding and design logic errors)
• Exit Criteria for Module Integration testing would
be detection and correction of 240 errors (30% of
200 plus 60% of 300) or elapse of one month; which
ever is later.  Testing
Life Cycle
Overview
• Exit Criteria for System Testing is when 111 errors  Test
Strategy
are detected or when 2 months elapses; which ever is  Test
Planning
later.  Test Case
Designing
 Test
Assumption: We have established one month for Integration Execution

Testing and 2 months for System Testing.

 Testing
Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
• Resolution in case of over estimation of defects
• In the previous example, let us say that 240
errors could not be detected even after one
month of integration testing is completed. Then
the project manager should take the following
action
– With the help of the appropriate tool, he/she should
analyze the test cases and decide whether  Testing
Life Cycle
• There are inadequate Test cases, or Overview
 Test
• Excellent Test cases, but lack of errors to detect Strategy
 Test
Planning
 Test Case
Designing
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning – Suspension and
Resumption criteria
• Determination of completion criteria through a graphical representation
• A graph is plotted using number of errors found per unit time during a
testing phase. By examining the shape of the curve, one can determine
whether to continue the test phase or end it.
• Assume a program is being Integration tested, and the number of errors
per day is plotted. Lets say on the 7h day the graph looks as follows:

 Testing
Life Cycle
Overview
Number of errors

AT the end of 7th day, even  Test

Strategy
if the number of errors  Test
Planning
detected has crossed the  Test Case
criteria mark, it would be Designing
 Test
wise to continue Integration Execution
Testing. This is because,  Testing
Types
the error rate is still very  Testing
Tools
No. of days 7
high. Overview
 Test Planning – Suspension and
Resumption criteria
• Determination of completion criteria through a graphical
representation
• On the hand if the graph looks as shown, then this shows that
error rate has decreased, and that testing can be stopped.
Number of errors

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Designing
No. of days 7  Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Planning – Document
templates
• Typical Test Plan
– Test Plan identifier
– Introduction
– Test items
– Features to be tested
testplan template
– Features not to be tested
– Approach
– Testing strategy
– Item pass/fail criteria  Testing
Life Cycle
– Suspension criteria and resumption criteria Overview
 Test
– Test deliverable Strategy
 Test
– Automation scope Planning
 Test Case
– Testing tasks Designing
 Test
– Effort estimation Execution
 Testing
– Environmental needs Types
 Testing
– Roles and Responsibilities Tools
Overview
 Test Case Design
 Testing Life Cycle Overview
 Test Strategy
 Test Planning
 Test Case Design
 Test Execution
 Testing Types
 Testing Tools Overview
 Test Case Design (Contd…)
Development Of Test Objectives:
• At very First stage of testing , you need to set the
goals and objectives of what you need to achieve.
• This Simplifies the limits or borders of testing
process.
• Test Objectives will be contain the list of tasks you
are going to do and the achievements.  Testing
Life Cycle
Overview
Test Case Writing:  Test
Strategy
 Test
• A specific set of steps and data along with expected Planning
 Test Case

results for a particular test objective. A test case Design

 Test
Execution
should only test one limited subset of a feature or  Testing
Types
functionality.  Testing
Tools
Overview
 Test Case Design (Contd…)
A typical test case sheet contains these fields…
• Test Case ID: It is unique number given to test case in
order to be identified.
• Test description: The description if test case you are
going to test.
• Revision history: Each test case has to have its revision
history in order to know when and by whom it is  Testing
Life Cycle

created or modified. Overview

 Test
Strategy

• Function to be tested: The name of function to be  Test

Planning
 Test Case
tested. Design
 Test

• Priority/Build/time: The priority of the Test Case,

Execution
 Testing
Types
Build version of the test case and the Minimum time-  Testing
Tools
Overview
to-execute the test case is also mentioned.
 Test Case Design (Contd…)
• Environment: It tells in which environment you are
testing.
• Test Setup: Anything you need to set up outside of
your application for example printers, network etc.,
• Test Execution: It is detailed description of every step
of execution.
• Expected Results: The description of what you expect  Testing
Life Cycle

the function to do. Overview

 Test
Strategy

• Actual Results: Pass /Failed.  Test

Planning
 Test Case

If pass – What actually happens when you run the test Design
 Test
Execution
If failed - Description of what you've observed.  Testing
Types
 Testing
Tools
Overview
 Test Case Design (Contd…)
Characteristics of a Good Test:
• Tests are likely to catch bugs
• No redundancy
• Not too simple or too complex.

TIP: Test case is going to be complex if you have  Testing

more than one expected results. Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Case Design (Contd…)
Test Data Inputs:
• Before testing, the tester should plan what
kind of data he is giving for test.
• Give data inputs as functional, boundary,
stress, performance, usability values etc.

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Case Design (Contd…)
Test Scripts:
• A test script is the executable form of a test.
• It defines the set of actions to carry out in order
to conduct a test and it defines the expected
outcomes and results that are used to identify any
deviance in the actual behavior of the program
from the logical behavior in the script (errors  Testing
Life Cycle
during the course of that test). Overview
 Test
Strategy
• In essence it is a program written for a human  Test
Planning

computer (tester) to execute.  Test Case

Design
 Test
Execution
 Testing
Types
Tes tCas e Template Tes tCas e Status  Testing
Template Tools
Overview
 Test Execution
 Testing Life Cycle Overview
 Test Strategy
 Test Planning
 Test Case Design
 Test Execution
 Testing Types
 Testing Tools Overview
 Test Execution
• Test execution is culmination of testing
activities which involves executing the
planned test cases and conducting of the
tests.
• Test execution phase broadly involves
execution and reporting.
 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - Activities
• Test execution consists of following activities
to be performed
– Creation of test setup or Test bed
– Execution of test cases on the setup
– Test Methodology used
– Collection of Metrics
– Defect Tracking and Reporting
 Testing

– Regression Testing Life Cycle

Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Test Execution Cycle

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - Testing
• Only exhaustive testing can show a program that it is
defect free ,but exhaustive testing is impossible
• Tests should exercise the systems capabilities fully
• Testing typical situations is more important than
boundary value conditions

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution -Test Data and
Test Cases
• Test Cases: Inputs to test the system and predicted
output from the system are detailed, further will
contain the expected result and test topology
• Test Data: Inputs which are to exercise the system
under test.

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Test Execution -Types of Testing
and Methodologies
• There are three major types of testing and
methods used are based on the testing type
– Unit testing
– Integration testing
– System testing

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - Unit Testing
• Unit testing is typically done as white box testing.
• Unit testing is done at module level.
• Derivation of Test cases are based on the program
structure .Knowledge of the program is used to
identify the test cases .
• Objective is to exercise all parts of the program .
• Test coverage measurements are done to verify that  Testing
all program parts are exercised Life Cycle
Overview

• Drivers are used to feed the data to the program to

 Test
Strategy
 Test
test it, wherever required. Planning
 Test Case

• Test stubs are used to stub for the functions which Design
 Test
Execution
are outside the module  Testing
Types
 Testing
Tools
Overview
 Test Execution - Integration
Testing
• Tests partial systems composed of integrated
components
• Integration testing is black box testing
• Main difficulty in integration testing is
localizing errors
• There two approaches to integration testing
– Top-down approach  Testing
Life Cycle
Overview

– Bottom - up approach  Test

Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Integration testing- Top Down
approach
• Start from the top level module and integrate
individual components and stub the rest of the
system

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Integration testing - Bottom –
Up approach
• Integrate individual components from lower
level module until whole system is integrated .

 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - System
Testing
• System testing is typically “Black Box “ testing
where the system is taken as black –box and
tested based only on its functionality
• Test cases are based on the system specification
• It is critical part of the testing process as the
system is verified as a whole
 Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - Regression
Testing
• Regression testing is done after fixing all the
defects.
• It is re-testing after fixing the bugs found
during testing and also when there is change in
the test environment.
• The impacted portions of the code after fixing
the bug is re-tested.  Testing
Life Cycle

• Automation of testing helps in faster regression Overview

 Test
Strategy
testing.  Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - Defect
•
Tracking and Reporting
All the defects/bugs found during testing is
logged and they are classified based on the
severity and priority.
Test report
• They are assigned to developers for fixing. template
• Each defect found is logged and tracked to
closure.
• Test summary report shall be prepared
containing the details like numbers of test cases  Testing
Life Cycle
Overview
passed/failed ,number of cycles of testing.  Test
Strategy

• Generally test defect logging and tracking tools  Test

Planning

are used for this purpose.  Test Case

Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - Start / Stop
Criteria
• Testing is done in rounds. Software passes
though several rounds before it gets released.
• Testing is stopped when there are many
major bugs and critical system errors, when
further testing is not feasible
• Testing is stopped when the system becomes
un stable during test.
• When those critical errors are fixed and
system is stable, the next round of testing is  Testing
Life Cycle

started . Overview
 Test
Strategy
• The test cases are executed all over again in  Test
Planning
each round.  Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution - Metrics
• Metrics are important to collect for any project as it
helps in monitoring the health of project /product
/application under test.
• Metrics are collected during testing depending on the
objective of testing.
• There are different kinds of metrics which can be
used in testing.
– For example LOC (Lines of Code) is a metric for Unit
testing. i.e., Code related metrics comes in the category of  Testing
Life Cycle
Unit testing. Overview
 Test
• Test case related metrics can be used for all kinds of Strategy
 Test
testing types, since for every type of test, we will Planning
 Test Case
write test cases. Design
 Test
• Normally source of errors/defects is requirements, Execution
 Testing
design or Implementation. Testing can not be the Types
 Testing
source of errors/defects. Tools
Overview
 Test Execution - Metrics
• Typical testing metrics
– No of LOC tested per day
– No of test cases executed per day
– Number of test cases failed
– Number of bugs reported
– Number of rejects of bugs reported
– Number of persons working on the test plan during
the month
– Total Effort spent on test execution in person hours  Testing
Life Cycle
– Number of simulations made during the month Overview
 Test
– Number of out of norms per month Strategy
 Test
– Average time taken to find a problem categorized by Planning
priority  Test Case
Design
– Defect density  Test
Execution
– Review efficiency  Testing
Types
 Testing
Tools
Overview
 Test Execution - Metrics
• Having a complete Metrics data helps in
building the “Trustworthy, Capable, Reliable
and Predictable” Product or Application.
– Productivity Increase
– Field error rate Decrease
– Customer Overhead reduces
– Improvement release on release
– On time delivery  Testing
Life Cycle
– Risks in Program identified and proactively intimated Overview
 Test
– Decrease in Cost of Quality Strategy
 Test
– Decrease in Re-work Planning
 Test Case
– Helps in identifying the area which needs focus in the Design
 Test
project/organization for improvement Execution
 Testing
– Helps in arriving at better estimates for the project Types
 Testing
Tools
Overview
 Test Execution – Good test
engineer?
• What makes a good test engineer?
– A good test engineer has a 'test to break' attitude.
– An ability to take the point of view of the customer
– A strong desire for quality, and an attention to detail.
– Tact and diplomacy for maintaining a cooperative relationship with
developers.
– An ability to communicate with both technical (developers) and non-
technical (customers, management) people.
– Previous software development experience can be helpful, gives the
tester an appreciation from the developers' point of view, and reduce
the learning curve in automated test tool programming.  Testing
Life Cycle
– Judgment skills are needed to assess high-risk areas of an application Overview
on which to focus testing efforts when time is limited.  Test
Strategy
– Notice little things that others miss/ignore (See symptom not bug)  Test
– Good testers use files, Databases and all the other accoutrements of Planning
 Test Case
an organized mind Design
– Testers are fundamentally honest and incorruptible  Test
Execution
– Last but not the least is “Patience”  Testing
Types
 Testing
Tools
Overview
Test Execution – Testing Rules?
• Software testing 10 rules?
– Test early and test often.
– Integrate the application development and testing life
cycles.
– Formalize a testing methodology; you'll test everything the
same way and you'll get uniform results.
– Develop a comprehensive test plan; it forms the basis for
the testing methodology.
– Use both static and dynamic testing.
 Testing
– Define your expected results. Life Cycle
Overview
– Understand the business reason behind the application.  Test
Strategy
You'll write a better application and better testing scripts.  Test
– Use multiple levels and types of testing (regression, Planning
 Test Case
systems, integration, stress and load). Design
 Test
– Review and inspect the work, it will lower costs. Execution
 Testing
– Don't let your programmers check their own work; they'll Types
 Testing
miss their own errors. Tools
Source: SQAtester.com Overview
 Test Execution - Summary
• Test execution consists of preparing of test
bed/setup ,execution of planned test cases
,verification of results ,Test tracking and
reporting and metrics collection.

• Apart from that ,it involves applying different

test methods like black box to system testing  Testing
Life Cycle
,white box at unit testing level and top –down and Overview
 Test

bottom –up in case of integration testing . Strategy

 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Test Execution – Summary
Contd..
• Testing will involve several rounds based on the
stability and bugs found on the software and
start /stop criteria identified .
• Regression testing will be carried out when ever
the new bugs are found and fixed in the tested
software to verify the impacted items.
• Test execution will involve preparation of test  Testing
Life Cycle
reports ,collection of different metrics and Overview
 Test

meeting the test objectives . Strategy

 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Tools Overview
 Testing Life Cycle Overview
 Test Strategy
 Test Planning
 Test Case Design
 Test Execution
 Testing Types
 Testing Tools Overview
 Testing Tools Overview
(contd…)
• Testing tools are used to automate the testing
process.
• These tools will have features like
– Record and play back
– Test Script generation
– Virtual users
And so on…  Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Tools Overview
(contd…)
• Different vendors released different testing
tools in the market. Some of them are

Functional / GUI Testing Tools:

• Win Runner
• Rational Robot  Testing

• Visual Test Life Cycle

Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Tools Overview
(contd…)
Performance Testing:
• Rational Quantify
• True time

Web Testing:
• Silk Test  Testing
Life Cycle
• Astra Quick Test Overview
 Test
Strategy
• eTest Suite  Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
 Testing Tools Overview
(contd…)
Java Testing Tools:
• Jtest
• JProbe
• QStudio
• Optimizeit
• JStyle  Testing
Life Cycle
Overview
 Test
Strategy
 Test
Planning
 Test Case
Design
 Test
Execution
 Testing
Types
 Testing
Tools
Overview
Appendix 1: Example for test life
cycle
• To understand the different stages of testing, consider an
example in which you need to test a Webpage, say
HRWEB->resume part.
• To test this Webpage following activities needs to be done
– Prepare Test Strategy for testing HRWEB->Resume section
– Prepare Test Plan for testing HRWEB->Resume section
– Develop Test Cases for testing HRWEB->Resume section
– Execute the test cases and report the results, including bug
reporting, re-test and tracking.
Appendix 1: Example for test life
cycle cont..
• Test strategy to test HRWEB->Resume page includes overview of
– Document the test requirements
– User Interface testing
– Test for associated links
– Error Handling and Exceptions
– Functionality testing
– Boundary conditions
– Test for Load, Performance, Security issues
– Deciding of tools and test set-up to test

• Test Planning to test HRWEB->Resume includes

– Resume page and all associated links will be tested
– Specify how it will be tested - Test Cases
– Identify resources for testing and finalize timeline
– Specify Hardware, Software configuration requirement
Appendix 1: Example for test life
cycle cont..
• Test Case Development
– Several Test Cases will be generated as per the test strategy and plan
for User Interface, Functionality, Error Handling of Resume Section in
HRWEB
– Integration and System Test cases will be generation to test for the
whole application including Resume Section.
• Test report:
– Test Cases will be executed and Test Results will be recorded and
tracked.
– After bugs are fixed, the system is tested again to verify the same
– The whole system is also tested to verify that new bugs have not been
introduced.
Appendix 1: Example for test life
cycle cont..
Sample User Interface Test Case for Resume Section
• Perform the necessary steps to view the Resume Section on
TEDWEB
• Verify that the menu TEDWEB Menu displays at the top and
the Resume Menu displays at the left of the screen
• Verify the presence of the following buttons - Add Resume,
Modify Resume, Add project, Modify Project etc..
Sample Functionality Test Case for Resume Section
• Click on ‘Add Resume’ link
• Verify a new screen appears to the right that allows the user
to add his resume details
• Put the cursor in Educational Qualifications field and verify it
is enabled for user input
• Put the cursor in Summary field and verify it is enabled for
user input.
Appendix 1: Example for test life
cycle cont..
Sample Error Handling Test Case for Resume Section
• Click on ‘Add Resume’ when the resume already exists.
Verify the message displayed is ‘Profile Already Exists’
• Click on ‘Modify Resume’ when no resume has been added.
Verify that a proper error message is displayed that no
resume exists.
 Appendix 2: Testing Standards
• IEEE Standard for Software Test Documentation
(ANSI/IEEE Standard 829-1983). This is a summary of the
ANSI/IEEE Standard 829-1983.
• It describes a test plan as: “A document describing the
scope, approach, resources, and schedule of intended testing
activities. It identifies test items, the features to be tested,
the testing tasks, who will do each task, and any risks
requiring contingency planning.”

IEEE testplan
template
Appendix 3: Additional readings
• Please go through the following documents, which are
available in Interops KM Page.
– Inter101_002-Intro Software Testing
– Inter101_003-Test Plan & Test Case
– Inter101_004-SDLC&TLC
– Inter101_005-Unit & Integration Testing
– Inter101_006-Functional & Regression Testing
– Inter101_007-General Testing Types
– Inter101_008-Test Report Automation Tools
– Inter101_009-Review & Walk thru
– Inter101_010-Reliability terminology