Software Testing

Chapter 19

Get Out the Test Plan

At the time testing is ready to begin, almost
all of our testing decisions and information
should be located in our test plan, which has
been growing throughout the project.
The test plan shows our decisions and
approaches for the project based on the
requirements, specification and design.

What Should Testing

Accomplish?
Testing does not and never will prove that
the system is defect free!
Instead, testing increases our confidence
that we built software that behaves as
specified.
Many testing techniques are closely
associated with statistics that indicate a
level of confidence.

Unit Testing vs. System Testing

Unit testing focuses on individual units of the
system.
Sowhat is a unit? A unit can be essentially what
ever we want: a subsystem, a group of
subsystems, a module (compilation unit, or file),
or even a specific segment of code.
System testing looks at the behavior of the entire
system as if it were monolithic .

Unit testing
Unit testing focuses on the inputs/outputs of a
unit.
Specifically, do we get the outputs we expect from
a unit for a given set of inputs.
Testers will also look at trying to fool the unit
by trying obtuse combinations of inputs.
Also, the tester will make sure that the unit
handles errors (erroneous inputs) the way it is
supposed to!

Test harness and stubs

Test harnesses are systems built to test systems.
That is, harnesses are computer programs that
supply inputs to another program, then read the
outputs and evaluate the response.
Sometimes, testing requires us to replace existing
routines with stubs to capture the output of a
particular unit (or system) before it goes to an
output device.

Subsystem testing
Subsystem testing almost always requires
instrumentation, such as a harness, to supply
inputs and intercept outputs.
Subsystem testing on process control systems can
be somewhat difficult: calls to mechanical control
subsystems must be intercepted to prevent damage
(or catastrophe). How do we test reactor control
software without activating a reactor?

Testing abstract data types

ADTs have the potential for us to define
environments in which the ADT can be
tested in isolation.
This proves useful for unit testing, but does
not provide for system or subsystem testing.
In those cases, testers usually resort to the
usual approaches, such as harnesses.

Testing object-oriented code

Likewise, object-oriented code has the
ability to have objects tested in isolation.
In many instances, objects can (and should)
have test methods (or member functions)
built into the object.
Subsystem and system testing, again,
usually is performed with the assistance of a
harness.

Sequences of inputs to
subsystems
Often, inputs must be sequenced to achieve a desired
state. When testing an array object that can
dynamically expand its capacity, testers must force the
capacity expansion to test it. This requires a series of
inputs to be transacted before the actual test begins.
It is not unusual for defects (bugs) to manifest
themselves only after a particular set of inputs has been
supplied to the system/subsystem/unit. It is important
to document what happened to the system before the
failure!

System testing
System testing can often be thought of in
the same manner of unit testing, with the
system as a single unit.
System testing is sensitive to the same
issues that unit testing are: input sequences
and irreversible actions.

Inspection vs. testing

Cleanroom engineering suggests that with
sufficient inspection and engineering diligence,
testing becomes a redundant almost needless
activity.
Experiments in very closely controlled
environments have indicated that there is very
much to be gained from inspection, but it will not,
in Joes opinion, be a satisfactory substitute for
testing.