Software Testing: © 2013 Quest Global, Inc. Confidential 1

Introduction
Software Testing Computer programs are designed and developed by human beings and hence are prone to errors.
Unchecked Software, they can lead to a lot of problems, including social implications. Testing the software becomes an essential part of the software development lifecycle. Carrying out the testing activities for projects has to be practiced with proper planning and must be implemented correctly.
2013 QuEST Global, Inc. Confidential
Basic Questions on Testing
Why to test? Testing becomes absolutely essential to make sure the software works properly and does the work that it is meant to perform. Any working product which forms part of the software application has to be tested. Both data and programs must be tested. How often to test? When a program (source code) is modified or newly developed, it has to be tested. Who tests? Programmer, Tester and Customer.
Definition: Software Testing

Software Testing is the process used to measure the quality of developed computer software. Software Testing means Finding an Errors in Software. Software Testing is stated as a process of Validating and Verifying that a Software program/application/product: meets the requirements that guided its design and development. works as expected. can be implemented with the same characteristics. satisfies the needs of stakeholders.
TYPES OF TESTING
MANUAL TESTING Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user. Testing that part of software testing that requires human input, analysis, or evaluation.
TYPES OF TESTING
AUTOMATION TESTING Testing application with the help of tools. Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions . The Test Scripts are reusable, the process ensures time Saving, and mainly done when we have monotonous or repetitive task.
Topics Covered
Software Problems and Objectives of Testing Quality Principles Software Development Life Cycle Software Life Cycle Models Testing Methodologies Software Testing Life Cycle Defect / Bug Life Cycle Test Automation Testing Certifications
Most Common Software Problems And Objectives of Testing

Common Software Problems Incorrect Coding / Implementation of business rules. Confusing and misleading data. Incorrect File handling.
Objectives of Testing Executing a Program with the intent of finding an Error. To check if the System meets Requirements. A good test case is one that has a probability of finding an as yet undiscovered Error. A good test should neither be too Simple nor too Complex.
Objectives of a Software Tester
To understand Application Requirements well. Finding bugs as early as possible and make they get fixed. Study the functionality in detail to find where the bugs are likely to occur. Create the Test cases in such a way that testing is done to uncover the hidden bugs. Ensuring that the Software is Usable and Reliable.
QUALITY PRINCIPLES
Quality the most important factor affecting an Organizations Long-term performance. Quality the way to achieve improved productivity and competitiveness in an Organization. Quality saves. It does not cost. Quality is the solution to the problem, not a problem.
Quality Assurance Vs. Quality Control
Quality Assurance (QA) A planned and systematic set of activities necessary to provide confidence that requirements are properly established, products and services conform to specified requirements. QA is the responsibility for entire team. Quality Control (QC) The process by which product quality is compared with applicable standards; and action taken when non-conformance is detected. QC is the responsibility for Testers.
10
Verification and Validation
Verification process confirms that the Software meets its Technical Specifications. It involves Reviews and meeting to evaluate documents , plans, code requirements.
Validation process confirms that the Software meets the Business requirements. It involves actual testing and takes place after Verifications are completed. Verification and Validation process continue in a Cycle till the Software becomes defects free.
11
Software Development Life Cycle (SDLC)
SDLC Software Development Life Cycle is step by step procedure to develop a software application. The Different phases of SDLC are : Requirement Collection Feasibility Study Design Coding Testing Installation Maintenance
12
Different Stages in SDLC
Process of SDLC phases

REQUIREMENTS
Business Analyst (collects Requirement)
ANALYSIS
DESIGN
Finance Dept, HR, BA, Architect (Cost, Time, Resources, Project Doable or Not) H.L.D (Architect) L.L.D (Sr. Developer)
CODING TESTING IMPLEMENTATION MAINTENANCE
Developer
Test Engineer (Application is stable or not)
Installation Engineer
13
REQUIREMENTS: Requirements are set of functionalities and constraints that the end-user expects from the system. The requirements are gathered from the end-user by consultation . REQUIREMENT SPECIFICATION CRS [Customer requirement specification] SRS [Software requirement specification]
14
CRS This document will describe in detail , about what is expected out of the software product from the user's perspective. The wordings of this document will be in the same tone that of a user . SRS This document will describe in detail about what is expected out of the software product from the user's perspective. The wordings of this document will be in the same tone that of a developers which means software language.
15
ANALYSES: The requirements are analyzed for their validity and the possibility of incorporating the requirements in the system to be developed are studied. Finally, a Requirement Specification document is created which serves the purpose of guideline for the next phase of the model.
DESIGN: TWO TYPES
HLD: [High Level Design] Architecture of the project. LLD: [Low Level Design] Designing the each & every model of project in detail.
16
CODING: Converting the pseudo code into a programming language in the specified platform. TESTING: Verifying the functionalities of an project against requirements. INSTALLATION: Application has been installed at the clients place.
MAINTENANCE: It generally refers to warranty period given by the company to the client for the application.
17
Software Life Cycle Models
WATERFALL MODEL SPIRAL MODEL PROTOTYPE MODEL V-PROCESS MODEL AGILE MODEL
18
V V Model
CRS
Customer
SRS
Test Engineer
H.L.D
Test Engineer
L.L.D
Developer
Verification (Review Process)
Coding
Validation (Testing)
19
V-Shaped Steps
Project and Requirements Planning: Allocate resources. Product Requirements and Specification Analysis: Complete specification of the software system. Architecture or High-Level Design: Defines how software functions fulfill the design. Detailed Design: Develop algorithms for each architectural component. Production, operation and maintenance: Provide for enhancement and corrections. System and acceptance testing: Check the entire software system in its environment. Integration and Testing: Check that modules interconnect correctly. Unit testing: Check that each module acts as expected. Coding: Transform algorithms into function specification.
20
Testing Methodologies
Types Of Testing
White Box Testing (Unit Testing)
Black Box Testing (Test Engineer)
Grey Box Testing
Unit Testing: It is small program which is written by the developers that in tern test the main program.
21
Testing Methodologies
White Box Testing It is the type of testing done by the developers where they test each and every line of code written by them before handing over the product to the test engineer. Black Box Testing It is type of testing done by the Test Engineer where they check if the application is working according to the requirements document.
White Box testing Generally it is done by the Test Engineer who has knowledge Of both coding as well as testing.
22
Types of White Box Testing
Path Testing In this type developers test for the complete flow of the programs or path of the programs & total number of paths in the program . Conditional Testing In this type developers check if all the conditional statements written in the program are executed for both true as well as false conditions. Loop Testing (Memory and Response Time) In this type developers check if all the looping conditions written in the program are executed for the complete number of cycles with these types developers also test the memory and response time w. r. t code.
23
How to do Testing and its Life cycle

Testing Objectives: Testing is a process of executing software with the intent of finding errors Good testing has a high probability of finding as-yetundiscovered errors
Successful testing discovers unknown errors

If did not find any errors, need to ask whether our testing approach is good Goal of testing: given some code, uncover as many errors are possible
24
How to do Testing and its Life cycle

Major Testing Methods: Unit Testing Sanity Testing Functional Testing Integration Testing UAT Testing System (End to End ) Testing Retesting Regression Testing Globalization Testing
25
Unit and Sanity Testing
Unit Testing Is a level of the Software Testing process where individual units/components of a Software/System are Tested. A unit is the smallest testable part of Software. It covers testing on a specific part of code perform by developers . Sanity Testing /Smoke Testing Testing basic functionality of an application Or Critical features of an application. Its very basic level Testing done by QA after fixing some bug.
26
Black Box Testing
Test the functionality of the application against requirement specification. Types of black box testing Functional Testing Integration Testing System Testing
Requirement
Coding White Box Testing
Functional Testing
Integration Testing
System Testing
27
Black Box Testing
Functional Testing Testing each and every component of an application of an application and each and every module independently is called as functional testing. For functional testing we require: Requirements, Application Points to remember while doing functional testing. Testing application with positive inputs first. Then we should do negative testing. Dont do Over testing /Exhaustive testing. Do not assume requirements by oneself, No assumptions. Under testing, if two components are same.
28
Black Box Testing
Integration Testing Testing the data flow or integration from one module to another module is Integration Testing. Points to remember while doing Integration testing: Complete the functional testing on all the features first. As an T.E, one should have complete knowledge of the modules and data flow between modules. Do not do the Integration testing randomly, whereas pick each and every module and complete the data flow from that module to all other modules wherever the data flow is available.
29
Black Box Testing
System Testing Testing the end to end flow of an application is called as system testing. In system testing the test environment is similar to product environment, similar w. r. t software, hardware, data. To begin our testing one should have Required documents. Application should be installed on to the test server. Test URL should be given.
30
UAT and Exploratory Testing
User Acceptance Testing (UAT) The client will test it, in their place, in a near-real-time or simulated environment. AT is a type of testing done by engineers sitting in it division of customer place. Exploratory Testing Checking application functionalities without document, Explore the application. Test Design and Test Execution at the same time.
31
Difference of Re test and Regression Testing
Re-test Testing only certain part of an application again and not considering how it will effect in the other part or in the whole application. Testing the bug again once the code is fixed to ensure that the particular functionality is working fine. Regression Testing Testing the application after a change in a module or part of the application for testing that is the code change will affect rest of the application. Testing some other test cases/functionality also to ensure that the above fix havent introduce any problem to existing code/functionality which was working fine earlier.
2013 QuEST Global, Inc. Confidential 32
Globalization Testing
Globalization Testing the applications for different languages is called as globalization testing. Globalization Testing is validating a specific Application or Product that can support multiple languages or that can be accessed by different Locales around the Globes.
Two major Components Internationalization & Localization. Internationalization (I18N) : Testing the application for multiple languages using property files. This type of testing is done for multiple languages using suffix and prefix. Localization (L10N) Testing the application as per country standard is called localization(L10N) testing e.g.. pin code, std code , currency.
33
Software Testing Life Cycle (STLC)

Software Development and Software Testing Life cycles go parallel
34
Test Plans
Test Plan - Derives future activities of Testing process. A Test Plan should be Overall coverage of Test to be conducted useful and Understandable by the people Inside and Outside the Test group. 1. 2. 3. 4. 5. 6. 7. Test Plan Consists of following attributes. Objective 8.Risk Scope 9.Mitigation/ Contingency Plan Test Approach 10.Enviornment(H/W, S/W) Test Methodology 11.Automation tools Roles & Responsibility 12.Effort Estimation Schedule 13.Entry & Exit Criteria Assumption 14.Test Deliverables
35
Test Cases / Use cases
Test Case is a Collection of Test steps also call it as a Design Case.

Test Case Template should contains 3 parts : 1. Header 2. Body 3. Footer
Use Case can be a basis for deriving the tests for a System. UML (Unified Modeling Language)is the facilities for drawing the Use case diagrams. It is used during the Analysis phase of Project, they separate the System into Actors and Use Cases.
36
Test Case Template
37
Test Case Design Techniques

Involves designing the Test cases (Inputs/Outputs) used to test the System.
The goal of Test case design is to create a set of tests that are effective in Validation and Defect Testing.
Design Approaches
Error Guessing In this approach the test engineer keeps on guessing the values according to his understanding
Equivalence Partitioning If input is range of values then derive the test case for one valid and two invalid data. If the input is set of values then derive the test case for one valid and two invalid inputs.
Testing Approaches
White-box testing
Control-flow-based testing
Data-flow-based testing
Black-box testing
Equivalence partitioning
39
Control-flow-based Testing
From the source code, create a graph describing the flow of control Called the control flow graph The graph is created (extracted from the source code) manually or automatically. Design test cases to cover certain elements of this graph: Nodes, edges, paths
s:=0; d:=0;
1 2 3 4 5 6 7
while(x<y) {
x:=x+3; y:=y+2; if x + y < 100
s:=s+x+y
else d:=d+x-y }
8
40
Elements of a control Flow Graph
Three kinds of nodes
Statement nodes: single-entry-single-exit sequence of statements Predicate(decision) nodes: conditions for branching Auxiliary nodes: (optional) for easier understanding (e.g. merge points for IF) Edges: possible flow of control
IF-THEN, IF-THEN-ELSE, SWITCH
41
IF-THEN, IF-THEN-ELSE, SWITCH
If (c) then
If (c) then
switch (c) case 1: . case 2: join point
join point
else
join point
42
EXAMPLE
switch(position) case CASHIER: if(empl_yrs > 5) bonus := 1; else bonus := 0.7; break; case MANAGER: bonus := 1.5; if (retiring_soon) bonus := 1.2 * bonus break; case . endswitch
43
Mapping for Loops
While (c) { . }
Note: other loops (e.g. FOR,DO-WHILE, ) are mapped similarly
Mini-assignment: figure out how this is done
44
Statement Coverage
Given the control flow graph, define a coverage target and write test cases to achieve it Traditional target: statement coverage Test cases that cover all nodes Code that has not been executed during testing is more likely to contain errors Often this is the low-probability code
45
EXAMPLE
Same example as before: two test
cases Path 1-2-exit Path 1-2-3-4-5-7-8-2-3-4-5-7-8-2exit Problem: The false branch of 4 is never taken dont have 100% branch coverage
2
3 4
T 5
F 6
7 8
Branch Coverage
Target: write test cases that cover all branches of predicate nodes
True and false branches of each IF The two branches corresponding to the condition of a loop All alternatives in a SWITCH In modern languages, branch coverage implies statement coverage
47
Branch Coverage
Statement coverage does not imply branch coverage

Example: if (c) then s; By executing only with c=true, we will achieve statement coverage, but not branch coverage
Motivation: experience shows that many errors occur in decision making

Plus, it subsumes statement coverage
48
EXAMPLE
Same example as before: two test cases Path 1-2-exit Path 1-2-3-4-5-7-8-2-3-4-5-7-82-exit
2
3 4
Problem: the false branch of 4 is never taken dont have 100% branch coverage
T 5
F 6
7 8
Achieving Branch Coverage

Branch coverage: a necessary minimum Pick a set of start-to-end paths that cover all branches, and write test cases to execute these paths Basic strategy Add a new path that covers at least one edge that is not covered by the current paths
Sometimes the set of paths chosen with this strategy is called the basis set
50
Data-flow-based Testing
Test connections between variable definitions (write) and variable uses (read) Variation of the control flow graph A node represents a single statement, not a single-entry-single-exit chain of statements Set DEF(n) contains variables that are defined at node n (i.e., they are written) Set USE(n): variables that are read
51
EXAMPLE
Assume y is already initialized

1 s:=0 2 x:=0
3 4 5 6 7 9 10 8
3 while(x<y) { 4 x:=x+3; 5 y:=y+2; 6 If(x+y<10) 7 s:=s+x+y; else 8 s:=s+x-y; }
52
Black-box Testing
Unlike white-box testing: dont use any knowledge about the internals of the code Test cases are designed based on specifications
Example: search for a value in an array

Post condition: return value is the index of some occurrence of the value, or -1 if the value does not occur in the array
53
Equivalence Partitioning
Consider input/output domains and partition them into equivalence classes For different values from the same class, the software should behave equivalently Test values from each class Example for input range 2..5: less than 2, between 2 and 5 and greater than 5 Testing with values from different classes is more likely to find errors than testing with values from the same class
54
Equivalence Classes
Examples Input x in range [a..b]: three classes x<a, a<=x<=b, b<x
Boolean: classes true and false

Some classes may represent invalid input Choosing test values
Choose a typical value in the middle of the class(es) that represent valid input
Choose values at the boundaries of classes E.g. for [a..b], use a-1, a, a+1, b-1, b, b+1
55
Example
Spec says that the code accepts between 4 and 24 inputs; each is a 3digit integer One partition: number of inputs Classes x<4, 4<=x<=24, 24<x Chosen values: 3,4,5,14,23,24,25
Another partition: integer values

Classes x<100, 100<=x<=999, 999<x Chosen values: 99,100,101,500,998,999,1000
56
Another Example
Similarly for the output: exercise boundary values Spec: the output is between 3 and 6 integers, each in the range 10002500 Try to design inputs that produce 3 outputs with value 1000 3 outputs with value 2500 6 outputs with value 1000 6 outputs with value 2500
57
Example: Searching
Search for a value in an array. Return: index of some occurrence of the value, or -1 if the value does not occur One partition: size of the array. Programmer errors are often made for size 1: a separate equivalence class. Classes: empty array, array with one element, array with many elements. Another partition: location of the value, first element, last element, middle element, not found.
58
Array empty [7]
Value 5 7
Output -1 0
[7]
[1,6,4,7,2] [1,6,4,7,2] [1,6,4,7,2] [1,6,4,7,2]
2
1 4 2 3
-1
0 2 4 -1
59
Defect / Bug Life Cycle
What is Defect ? A Defect is a variance from a desired product attribute. Two Categories of Defects are Variance from Product Specification. Variance from Customer / User expectations. Why Defects occur ? Wrong Implementation Missing Implementation Extra Implementation
Defect / Bug Tracking Tools Bugzilla Rational Clear Quest Quality Center(QC) etc.
60
Defect / Bug Classification
Based on Priority
Defect Priority Critical High Guidelines Must be fixed immediately. Serious effect on Testing. Must be fixed before testing has completed.
Medium Low
Fixed if possible before application implemented in production. Fixed if time available.
61
Defect / Bug Classification
Based on Defect Severity

Defect Severity Show Stopper Major Guidelines Almost the entire test activities suspended due to fault. System un-usable The test Activities for a module is been suspended. Major module of system unstable. The test script is suspended but other testing process can be continued. Individual functions affected and individual users prevented from completing tasks. Minor fault or Cosmetic Error that has no impact on testing schedule. Users can continue working. Action All testing stopped until fixed All testing for this module is stopped A single test stopped until fixed
Minor
Trivial Error/Cosmetic
No testing stopped
62
Defect Status
Defect Life cycle/ Bug Life cycle comprises of all the defect status changes it would under go once a new defect is logged and till the defect is closed or cancelled.
Defect Life Cycle
63
Defects Status Explanation
NEW defect is one of ASSIGNED, DROPPED and DEFERRED. ASSIGNED/OPEN : Test /Development/Project lead studies the NEW defect and if it is found to be valid, it is assigned to a member of the Development Team. The assigned Developers responsibility is now to fix the defect and have it COMPLETED. DEFERRED : If a valid NEW or ASSIGNED defect is decided to be fixed in upcoming releases instead of the current release it is DEFERRED. This defect is ASSIGNED when the time comes. PROPPED/REJECTED : Test/Development/Project Leads approaches the NEW defect and if it is found to be invalid, it is DROPPED/REJECTED .
DROPPED/REJECTED : Test/Development/Project Leads approaches the NEW defect and if it is found to be invalid, it is DROPPED/REJECTED . COMPLETED/FIXED/RESOLVED/TEST : Developer fixes the defect that is ASSIGNED to him or her. Now, the fixed defect needs to be verified by the Test Team and the Development Team assigns the defect back to the Test Team. A COMPLETED defect is either CLOSED, if fine , or REASSIGNED, if still not fine.
65
If a Developer cannot fix a defect , some Organizations may offer the following statuses : Wont Fix/Cant Fix : The Developer will not or cannot fix the defect due to some reason Cant Reproduce : The Developers is unable to reproduce the defect. Need more Information : The Developer needs more information on the defect from the Tester.
REASSIGNED / REOPENED : If the Tester finds that the fixed defect is in fact not fixed or only partially fixed, it is reassigned to the Developer who fixed it. A REASSIGNED defect needs to be COMPLETED again. CLOSED/VERIFIED : If the Tester /Test Lead finds that the defect is indeed fixed and is no more of any concern, it is CLOSED/VERIFIED.
66
Severity Vs. Priority
Severity Factor that shows how bad the defect is and the impact it has on the Product. Severity Levels :
Critical Major / High Average / Medium Minor / Low Cosmetic defects
Priority Based on input from Users regarding which defects are most important to them, and to be fixed first.
67
Test Automation
Automation Is the process of writing a computer program to do Testing. Once tests have been automated, they cab be run quickly and repeatedly. Doing any task using a Tool without any Manual intervention . Automation increases Speed of Test Execution. Automation helps increase Test Coverage. Ex: AutoCAD. Various Automation tools available in market are Why Automation ? Faster Reduces Human efforts Reliable and Consistency Reusable
68
Test Automation
Some Test Automation Tools Tool Name Produced By
HP Quick Test Professional

IBM Rational Functional Testes Silk Test Selenium Robot Framework Sikuli
HP Software Division
IBM Rational Borland Open Source Open Source Open Source
69
Software Testing Certification
IEEE Standards (Institute Of Electrical and Electronics Engineers) designed an entire set of standards for Software and to be followed by the Testers. IEEE Standard for Software Quality Assurance Plan IEEE Standard for Software Unit Testing IEEE Standard for Software Management IEEE Standard for Software Verification and Validation IEEE Standard for Software Quality Metrics Methodology
70
Other Standards
ISO International Organization for Standards Six Sigma Zero Defect Orientation NIST National Institute of Standards and Technology ISTQB International Software Testing Qualification Board
71
Conclusion
Software process are the activities involved in producing and evolving a Software System. Testing can show the presence of faults in a System ; it cannot prove there are no remaining faults. General activities are specification, design ,implementation, Validation and evolution. Developers are responsible for Unit Testing , System Testing is the responsibility of a Testing Team. Test Automation reduces Testing costs by supporting the test process with range of Software Tools.
72

Software Testing: © 2013 Quest Global, Inc. Confidential 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Software Testing: © 2013 Quest Global, Inc. Confidential 1

Uploaded by

Copyright:

Available Formats

Introduction

2013 QuEST Global, Inc. Confidential

Basic Questions on Testing

2013 QuEST Global, Inc. Confidential

Definition: Software Testing

2013 QuEST Global, Inc. Confidential

2013 QuEST Global, Inc. Confidential

2013 QuEST Global, Inc. Confidential

2013 QuEST Global, Inc. Confidential

Most Common Software Problems And Objectives of Testing

2013 QuEST Global, Inc. Confidential

Objectives of a Software Tester

2013 QuEST Global, Inc. Confidential

2013 QuEST Global, Inc. Confidential

Quality Assurance Vs. Quality Control

2013 QuEST Global, Inc. Confidential

Verification and Validation

2013 QuEST Global, Inc. Confidential

Software Development Life Cycle (SDLC)

2013 QuEST Global, Inc. Confidential

Different Stages in SDLC

Process of SDLC phases

CODING TESTING IMPLEMENTATION MAINTENANCE

Test Engineer (Application is stable or not)

2013 QuEST Global, Inc. Confidential

Different Stages in SDLC

2013 QuEST Global, Inc. Confidential

Different Stages in SDLC

2013 QuEST Global, Inc. Confidential

Different Stages in SDLC

DESIGN: TWO TYPES

2013 QuEST Global, Inc. Confidential

Different Stages in SDLC

2013 QuEST Global, Inc. Confidential

Software Life Cycle Models

2013 QuEST Global, Inc. Confidential

Verification (Review Process)

2013 QuEST Global, Inc. Confidential

2013 QuEST Global, Inc. Confidential

White Box Testing (Unit Testing)

Black Box Testing (Test Engineer)

Grey Box Testing

2013 QuEST Global, Inc. Confidential

2013 QuEST Global, Inc. Confidential

Types of White Box Testing

2013 QuEST Global, Inc. Confidential

How to do Testing and its Life cycle

Successful testing discovers unknown errors

2013 QuEST Global, Inc. Confidential

How to do Testing and its Life cycle

2013 QuEST Global, Inc. Confidential

Unit and Sanity Testing

2013 QuEST Global, Inc. Confidential

Black Box Testing

Coding White Box Testing

2013 QuEST Global, Inc. Confidential

Black Box Testing

2013 QuEST Global, Inc. Confidential

Black Box Testing

Black Box Testing

2013 QuEST Global, Inc. Confidential

UAT and Exploratory Testing

2013 QuEST Global, Inc. Confidential