Software Testing

Software Testing can be defined as: Testing is an activity that helps in finding out bugs/defects/errors in
a software system under development, in order to provide a bug free and reliable system/solution to the
customer.

Software Testing is the process of executing a program or system with the intent of finding errors. Or, it
involves any activity aimed at evaluating an attribute or capability of a program or system and
determining that it meets its required results. Software is not unlike other physical processes where
inputs are received and outputs are produced. Where software differs is in the manner in which it fails.
Most physical systems fail in a fixed (and reasonably small) set of ways. By contrast, software can fail
in many bizarre ways. Detecting all of the different failure modes for software is generally infeasible

Software Testing
Software testing is the process used to help identify the correctness, completeness, security and quality
of developed computer software.

The IEEE definition for testing is:

“The process of exercising or evaluating a system by manual or automatic means to verify that it
satisfies specified requirements or to identify differences between expected and actual results”.

The definition of Myers says,

“Testing is the process of executing a program with the intent of finding errors”.

Testing is an activity aimed at evaluating an attribute or capability of a program or system to determine

that it meets its required results. It is a process of evaluating a system or system component by manual
or automated means to verify that it satisfies specified requirements or to identify differences between
expected and actual results.

Testing is used to determine the status of the product during and after the build or do component of the
process. The role of testing changes as the type of process used to build the product changes.

What is the goal of testing?

The main goal of testing is to check if the system meets the user requirements and check for the systems
reliability.

Why software testing?

• To produce quality product.

• To reduce the failure cost and maintenance cost.
New to Testing?

Software Testing is the process of executing a program or system with the intent of finding errors.
Although crucial to software quality and widely deployed by programmers and testers, software testing
still remains an art, due to limited understanding of the principles of software. The difficulty in software
testing stems from the complexity of software: we can not completely test a program with moderate
complexity. Testing is more than just debugging. The purpose of testing can be quality assurance,
verification and validation, or reliability estimation

Testing can be used as a generic metric as well. Correctness testing and reliability testing are two major
areas of testing. Software testing is a trade-off between budget, time and quality.
Unlike most physical systems, most of the defects in software are design errors, not manufacturing
defects. Software does not suffer from corrosion, wear-and-tear -- generally it will not change until
upgrades, or until obsolescence. So once the software is shipped, the design defects -- or bugs -- will be
buried in and remain latent until activation. Regardless of the limitations, testing is an integral part in
software development. It is broadly deployed in every phase in the software development cycle.
Typically, more than 50% percent of the development time is spent in testing. Testing is usually
performed for the following purposes:

To improve quality
For Verification & Validation (V&V)
For reliability estimation

Software testing can be very costly. Automation is a good way to cut down time and cost. Software
testing tools and techniques usually suffer from a lack of generic applicability and scalability. The
reason is straight-forward. In order to automate the process, we have to have some ways to generate
oracles from the specification, and generate test cases to test the target software against the oracles to
decide their correctness. Today we still don't have a full-scale system that has achieved this goal. In
general, significant amount of human intervention is still needed in testing. The degree of automation
remains at the automated test script level. Testing is potentially endless. We can not test till all the
defects are unearthed and removed -- it is simply impossible. At some point, we have to stop testing and
ship the software. The question is when. Realistically, testing is a trade-off between budget, time and
quality. It is driven by profit models. The pessimistic and unfortunately most often used approach is to
stop testing whenever some or any of the allocated resources -- time, budget, or test cases -- are
exhausted. The optimistic stopping rule is to stop testing when either reliability meets the requirement,
or the benefit from continuing testing cannot justify the testing cost. This will usually require the use of
reliability models to evaluate and predict reliability of the software under test. Each evaluation requires
repeated running of the following cycle: failure data gathering -- modelling -- prediction. This method
does not fit well for ultra-dependable systems, however, because the real field failure data will take too
long to accumulate.
SDLC

The software development life cycle (SDLC) is the entire process of formal, logical steps taken to
develop a software product. Software development is the process of developing software through
successive phases in an orderly way. This process includes not only the actual writing of code but also
the preparation of requirements and objectives, the design of what is to be coded, and confirmation that
what is developed has met objectives.

The software development life cycle (SDLC) is a conceptual model used in project management that
describes the stages involved in an information system development project, from an initial feasibility
study through maintenance of the completed application.

Typical phases of software development:

1) Requirement Analysis

2) Software design

3) Development or Coding

4) Testing

5) Maintenance

Requirement Analysis:

The new system requirements are defined. The requirements of a desired software product are
extracted. Based the business scenario the SRS (Software Requirement Specification) document is
prepared in this phase. The purpose of this document is to specify the functional requirements of the
software that will be produced by the SOS project group. The specifications are intended to guide the
group through the development process.

Design:

Plans are laid out concerning the physical construction, hardware, operating systems, programming,
communications, and security issues for the software. Design phase is concerned with making sure the
software system will meet the requirements of the product. It should also ensure that the future
requirements will also be addressed..

Development (or) Coding:

Programs are developed by the developers with the help of the design. The design is reduced to code by
the software engineers.

Testing:

Testing is evaluating the software to check for the user requirements. Here the software is evaluated
with intent of finding defects.

Maintenance:
Once the new system is up and running for awhile, it should be exhaustively evaluated. Maintenance
must be kept up rigorously at all times. Users of the system should be kept up-to-date concerning the
latest modifications and procedures.

Software development life cycle models:

The waterfall model: This is the best known and oldest model, with a linear and sequential method that
has goals for each development phase. In waterfall model there are no iterative or overlapping steps.
They state requirements, analyze them, design a solution approach, develop code, test, deploy, and
maintain. After each step is finished, the process proceeds to the next step. One drawback of the
waterfall is that it is static about user requirements. It does not allow for much revision.

Rapid application development (RAD): The problem with previous methodology was that applications
took so long to build that requirements had changed before the system was complete, often resulting in
unusable systems. This model is based on the concept that better products can be developed more
quickly by: using groups to gather system requirements, prototyping and reiterative testing of designs
and less formality of team communications such as reviews.

Advantages:

Increased speed of development and increased quality.

Disadvantages:

Reduced Scalability and reduced features.

The prototyping model: In this model, a prototype is built, tested, and then reworked as necessary until
an acceptable prototype is finally achieved from which the complete system or product can now be
developed.

The spiral model: This model of development combines the features of the prototyping model and the
waterfall model. Each phase starts with a design goal and ends with the client (who may be internal)
reviewing the progress thus far. Analysis and engineering efforts are applied at each phase of the
project, with an eye toward the end goal of the project. The spiral model is favored for large, expensive,
and complicated projects.
Testing Projects - Introduction

Testing is not done only after the code is written. For effective testing, it should be considered as an
ongoing process that is involved in software design, programming style, and debugging.

What to Test?

Test all boundaries

Check upper & lower bounds of ALL buffers.
Ensure that a loop iterates for the given number of times.
Recursive functions:
– Consider all possible states of the data.
– Be sure about terminal condition.
Ensure that all conditional statements branch in the right direction

Test pre and post conditions

Assertions can be used to verify the necessary pre-conditions or post-conditions.
Pre and post conditions are simply assumptions about the state of required data in critical portions of
the code.
Using assertions is just another way to test some of these assumptions.

• Also, Test the return values from all system calls and library functions. These values are
returned for a reason. They indicate problems and error states that need to be handled.
Deal with all errors and anomalies.
Check the consistency of data and the stability of the program wherever possible.

How to Test?
• Test simple parts first
Write a small portion of code (or a function) and test it before coding anything else.
This will reduce the amount of testing and debugging that you need when you finish writing the
software.
Testing and debugging should go hand-in-hand and be a continuous process intertwined with
writing the code.
Test the small portions first and gradually build your code step-by-step, testing and debugging
at each and every step.

“Black Box” testing

– Here you are not concerned about the details of what a function or piece of code does.
– You should be concerned simply with whether it produces the correct results.
– Give the function controlled data and test the return values. Since you know what values you should
receive you can test whether the function works with varying data.
– This is the type of testing you are doing when you use test harnesses.
Test interfaces between functions
– Each function provides an interface to another function and hides the details of what it does.
– This is one area that is susceptible to bugs because incorrect use may cause incorrect results.
– Black box testing would be appropriate to use here.
– Test all return values from functions and be sure to handle (and/or report) error conditions in a sane
manner.
– Be sure that each function has the ability to test for applicable error conditions.

Test incrementally
– Write a function and test it before writing another or using it in another function.
– For larger functions, write a small portion of functional code and test it.
– When a function relies on other functions, combine the smaller functions into a larger function and
test as a whole functional unit.
– Check for and handle error states.
– Black Box testing, again, would be appropriate.

Testing Process

In a software development process, there are five different phases. They are

-Requirement analysis
-Design
-Development (or) Coding
-Testing
-Maintenance.

Here the testing comes in fourth phase. But the actual testing process begins during the first phase itself
i.e. testing process begins during the Requirement analysis phase itself.

The steps in the testing process are as follows.

Requirement analysis:
Testing should begin in the requirements phase of the software life cycle (SDLC). The actual
requirement should be understand clearly with the help of Requirement Specification documents,
Functional

Specification documents, Design Specification documents, Use case Documents etc.

During the requirement analysis the following should be considered.

-Are the definitions and descriptions of the required capabilities precise?
-Is there clear delineation between the system and its environment?
-Can the requirements be realized in practice?
-Can the requirements be tested effectively?
Test Planning:
During this phase Test Strategy, Test Plan, Test Bed will be created.
A test plan is a systematic approach in testing a system or software.
The plan should identify:
-Which aspects of the system should be tested?
-Criteria for success.
-The methods and techniques to be used.
-Personnel responsible for the testing.
-Different Test phase and Test Methodologies
-Manual and Automation Testing
-Defect Mgmt, Configuration Mgmt, Risk Mgmt. Etc
-Evaluation & identification – Test, Defect tracking tools

Test Environment Setup:

During this phase the required environment will be setup will be done. The following should also be
taken in account.
• Network connectivity’s
• All the Software/ tools Installation and configuration
• Coordination with Vendors and others

4. Test Design:

During this phase

Test Scenarios will be identified.
Test Cases will be prepared.
Test data and Test scripts prepared.
Test case reviews will be conducted.

5. Test Automation:

In this phase the requirement for the automation will be identified. The tools that are to be used will be
identified. Designing framework, scripting, script integration, Review and approval will be undertaken
in this phase.

6. Test Execution and Defect Tracking:

Testers execute the software based on the plans and tests and report any errors found to the
development team. In this phase

• Test cases will be executed.

• Test Scripts will be tested.
• Test Results will be analyzed.
• Raised the defects and tracking for its closure.
7. Test Reports:

Once testing is completed, testers generate metrics and make final reports on their test effort and
whether or not the software tested is ready for release.

• Test summary reports will be prepared

• Test Metrics and process Improvements made
• Build release
• Receiving acceptance