Software Testing

(09CS 491)

Unit II White Box Testing (Chapter 3)

Testing can prove presence of defects and not their absence!
Software Testing Slide 1

Objectives
l

To review the V Model and set the context for different types of tests

l l l

To focus on white box testing To distinguish between different types of white box testing To place special emphasis on

Code reviews / inspections

Methods of code coverage and relationship to quality Code complexity

Software Testing

Slide 2

Bending the Waterfall

Overall Business Requirements Software Requirements Acceptance Testing

System Testing

High-Level Design

Integration Testing

Low-Level Design

Component Testing

Development

Unit Testing

Testing

Software Testing

Slide 3

V Model
Overall Business Requirements Acceptance Test Design Acceptance Test Execution

Software Requirements

System Test Design

System Test Execution

High-Level Design

Integration Test Design

Integration Test Execution

Low-Level Design

Component Test Design

Component Test Execution

Development

Unit Test Design

Unit Test Execution

Software Testing

Slide 4

Principles of the V Model

l l l

Do an early test design. Involve the right people in the design of the right type of tests.

Make the tests reflect closely the steps on the left-hand side of
the V.

Unit testing

White box testing

Black box testing

Software Testing

Slide 5

White Box Testing

l

Requires access to code

Looks at the program code and performs testing by

mapping program code to functionality

Software Testing

Slide 6

White Box Testing

White-box testing is a method of testing software that tests internal structures or workings of an application. White box testing is also known as clear, open, structural, and glass box testing. White Box Testing method is applicable to the following levels of software testing:
Unit Testing System Testing : For testing paths within a unit Integration Testing : For testing paths between units : For testing paths between subsystems
Software Testing Slide 7

Types of White Box Testing

Software Testing

Slide 8

Why White Box Testing?

l

The program code truly represents what the program actually does and not just what it is intended to do!

It minimizes delay between defect injection and defect detection (i.e., does not postpone detection to be done by someone else).

Can catch obvious programming errors that do not necessarily map to common user scenarios (e.g., divide by zero).
Software Testing Slide 9

Static Testing
l l

Involves only the source code and not the executable or binaries
Does not involve executing the program on a machine but rather humans going through it or the usage of specialized tools

Some of the things tested by static testing:

Whether the code works according to the functional requirement Whether the code has been written in accordance with the design

developed earlier in the project life cycle

Whether the code follows all applicable standards Whether the code for any functionality has been missed out Whether the code handles errors properly
Software Testing Slide 10

Static Testing by Humans

l l l

Humans can find errors that computers cant. Multiple humans can provide multiple perspectives.

A human evaluation can compare the code against the

specifications more thoroughly.

It can detect multiple defects at one go.

It reduces downstream, inline pressure.

Software Testing

Slide 11

Static Testing Types

l

Different types
Desk checking of the code Code walkthrough

Code review
Code inspection

l
l l l

QA vs QC argument Increasing the involvement of people More variety of perspectives Increasing formalism Increasing the likelihood of identifying more complex defects
Software Testing Slide 12

Desk Checking
l

Author informally checks the code against the specifications and

corrects defects found.

No structured method or formalism is required to ensure completeness.

l l l

No log or checklist is maintained. It relies only on the thoroughness of the author. It may suffice for obvious programming errors but may not be effective for incomplete / misunderstood requirements.

Software Testing

Slide 13

Advantages and Disadvantages of Desk Checking

l

Advantages
The programmer knows the code and programming language well and hence is best suited to read the program Less scheduling and logistics overheads Reduces delay in defect detection and correction

Disadvantages

Person dependent, not scalable or reproducible

Tunnel visioning of developers Developers prefer writing new code and dont like testing

Software Testing

Slide 14

Code Walkthrough
l l l

Group oriented (as against desk checking) Brings in multiple perspectives

Multiple specific roles (author, moderator, inspector, etc.),

as discussed in Fagan Inspection

Software Testing

Slide 15

Formal Inspection / Fagan Inspection

l l

Group-oriented activity Highly formal and structured

l
l

Has specific roles

Requires thorough preparation

Software Testing

Slide 16

Roles in a formal inspection

l

Author Author of the work product Makes available the required material to the reviewers Fixes defects that are reported
Moderator Controls the meeting(s) Inspectors (reviewers) Prepare by reading the required documents Take part in the meeting(s) and report defects Scribe Takes down notes during the meeting Assigned in advance by turns Can participate to review to the extent possible Documents the minutes and circulates them to participants
Software Testing Slide 17

Process in a Fagan Inspection

Typical documents circulated: Program code

Design / program specifications

SRS (if needed) Any applicable standards (e.g., coding standards) Any necessary checklists (e.g., code review checklist)

Software Testing

Slide 18

Meetings in a Fagan Inspection

l

Preliminary meeting (optional) Author explains his / her perspective Makes available the necessary documents Highlights concern areas, if any, for which review comments are sought

Defect Logging Meeting All come prepared! Moderator goes through the code sequentially Each reviewer comes up with comments Comments / defects categorized as defect / observation, major / minor, systemic / mis-execution

Scribe documents all the findings and circulates them

Software Testing Slide 19

Meetings in a Fagan Inspection

l

Follow-up meeting (optional)

Author fixes defects If required, a follow-up meeting is called to verify completeness of fixes

Software Testing

Slide 20

Advantages and Disadvantages of Fagan Inspection

l

Advantages
Thorough, when prepared well Brings in multiple perspectives

Has been found to be very effective

Disadvantages
Logistically difficult Time consuming May not be possible to exhaustively go through the entire code

Software Testing

Slide 21 Slide : 21 White Box Testing

Combinations of various methods

l

Prioritizing various programs High priority / important codes subject to formal

inspection, medium ones for walkthrough and low ones

for desk checking

Deciding between 100% coverage and partial coverage

Software Testing

Slide 22

Static Analysis Tools

Are used to identify errors such as:
l

Whether there are unreachable codes (usage of goto statements sometimes creates this situation; there could be other reasons too)

l l l l l

Variables declared but not used Mismatch in definition and assignment of values to variables Illegal or error-prone type-casting of variables Use of non-portable or architecture-dependent programming constructs Memory allocated but not having corresponding statements for freeing up memory

l l

For calculation of cyclomatic complexity As an extension of compilers (lint, compiler flag driven checking)
Software Testing Slide 23

Use of a Code Review Checklist

l

The code review checklist aids in organizational learning by indicating what to look for.

It should be kept current as we learn. A discussion of a sample checklist is given on Page 54.

Software Testing

Slide 24

Structural Testing
l
l

Done by running the executable on the machine

Entails running the product against some predefined test cases and comparing the results against the expected results

Designing test cases and test data to exercise certain portions of

code

Types of structural testing

Unit / code functional testing

Code coverage testing Code complexity testing
Software Testing Slide 25

Unit / Code Functional Testing

l
l l l l l

Initial quick checks by developer

Removes obvious errors Done before more expensive checks Building debug versions Running through an IDE Debugging or testing? (Who cares?!)

Software Testing

Slide 26

Code Coverage Testing

l
l l

Exercises different parts of code

Maps parts of code to required functionality Finds out percentage of code covered by testing by adopting a technique called instrumentation of code.

Instrumentation rebuilds the product, linking it with libraries provided by the tool vendors.

l l

Instrumented code can monitor what parts of code is covered Can help identify critical portions of code, executed often

Software Testing

Slide 27

Types of Code Coverage

l

Statement coverage Path coverage

Condition coverage
Function coverage

Software Testing

Slide 28

Programming Constructs
l

Sequential instructions Two-way decision statements (if then else)

Multi-way decision statements (switch statements)

Loops, like while/do, for, repeat/until, etc

Software Testing

Slide 29

Testing sequential instructions

l

Generate test data to make the program enter the sequential block, to make it go through the entire block (Is this valid?)

Asynchronous exceptions
Multiple entry points, in non-structured programming

Statement coverage as a metric: # of statements exercised / Total # of statements

Software Testing

Slide 30

Testing IF THEN ELSE

l l

Have data to test the THEN part

Have data to test the ELSE part
Total = 0; /* set total to zero */

if (code == M) {
stmt1; stmt2;

}
else percent = value/Total*100; /* divide by zero */

Software Testing

Slide 31

Path Coverage
l l

Statement Coverage may not indicate true coverage Path Coverage provides better representation
Previous example: Having data go through the THEN part and ELSE part gives only 50% coverage, irrespective of number of statements in each path

l l

More detailed example of path coverage in next slide

Path coverage = # of paths exercised / total # of paths in the program

Software Testing

Slide 32

Path Coverage.. Example

Fig: flowchart for a date validation routine

Software Testing

Slide 33

Condition Coverage
l l

Further refinement of path coverage Makes sure each constituent condition in a Boolean expression

is covered
l
l

Protects against compiler optimizations

Condition coverage = # of conditions covered / # of conditions in the program

Software Testing

Slide 34

Function Coverage
l
l

Finds how many functions are covered by test cases

Higher level of abstraction; hence possible to achieve 100% coverage

l
l l l l

More logical than the other types of coverage

Can be derived from RTM Easy to prioritize as they are based on requirements Leads more naturally to black box tests Can also be a natural predecessor to performance testing

Software Testing

Slide 35

Other Uses of Code Coverage Methods

l l

Performance analysis and optimization Resource usage analysis

l
l l

Checking of critical sections

Identifying memory leaks Dynamically generated code (e.g., dynamic generation of URLs and security checking)

Software Testing

Slide 36

Code Complexity Testing

l

Which of the paths are independent? (If paths are not independent, we may be able to minimize the number of tests.)

Is there an upper bound on the number of tests to be executed to ensure that all the statements have been executed at least once?

Cyclomatic complexity provides answers to some of these

questions.

Software Testing

Slide 37

Intuitive Meaning of Cyclomatic Complexity

l

Provides an indication of how many independent paths are there in a program

l l

What gives birth to new paths?

How can one count the new paths that are born?

Software Testing

Slide 38

Steps in Determining Cyclomatic Complexity

l l

Start with a conventional flow chart. Identify the decision points.

Convert complex predicates to simple predicates (see

next slide).

Software Testing

Slide 39

Software Testing

Slide 40

Steps in Determining Cyclomatic Complexity.. contd

l
l

If there are loops, convert the loop conditions to simple predicates.

Combine all the sequential statements into a single node in the flow graph.

When a set of sequential statements are followed by a simple predicate,

combine all the sequential statements and the predicate check into one node and have two edges emanating out of this node. Nodes with two

emanating edges are called predicate nodes.

l

Make sure that all edges terminate in some node, adding a node to represent all the statements at the end of the program.

Software Testing

Slide 41

Conventional Flow Chart

Flow Diagram for Calculating Complexity

Software Testing

Slide 42

Formal and Intuitive Interpretation of Cyclomatic Complexity

l

Formal definitions
CC = # of predicate nodes + 1 CC = E N + 2 (E = edges, N = nodes)

Intuitive definitions

When a condition is introduced, adds a new path

See the next two slides

Software Testing

Slide 43

Case (i): A hypothetical program with no decision node

# of independent paths = 1 # of nodes, N = 2; # of edges, E = 1; Cyclomatic complexity = E N + 2 = 1 # of predicate nodes, P = 0; Cyclomatic complexity = P + 1 = 1

End

Software Testing

Slide 44

Case (ii): Adding one decision node

# of independent paths = 2 # of nodes, N = 4; # of edges, E = 4; Cyclomatic complexity = E N + 2 = 2 # of predicate nodes, P = 1; Cyclomatic complexity = P + 1 = 2 End

Software Testing

Slide 45

Independent Paths and Basis Sets

l

An independent path is a path in the flow graph that has not been traversed before in other paths.

A set of independent paths that cover all the edges is

called a basis set.

Find the basis set and write the test cases to execute all the paths in the basis set.

Software Testing

Slide 46

Meaning and Interpretation of Cyclomatic Complexity

Complexity What it means Well-written code, testability is high, cost / effort to maintain is low Moderately complex, testability is medium, cost / effort to maintain is medium Very complex, testability is low, cost / effort to maintain is high Not testable, any amount of money / effort to maintain may not be enough
Software Testing Slide 47

1-10

10-20

20-40

>40

Challenges in White Box Testing

l

Developers not finding time for testing They develop blind spots for their own defects

Fully covered code may not correspond to realistic

scenarios

Software Testing

Slide 48

Objectives (Recap)
l

To review the V Model and set the context for different types of
tests

l l l

To focus on white box testing To distinguish between different types of white box testing To place special emphasis on
Code reviews / inspections Methods of code coverage and relationship to quality Code complexity

Software Testing

Slide 49

Appendix
EXAMPLES

Software Testing

Slide 50

White-box Testing Example

FindMean(float Mean, FILE ScoreFile) { SumOfScores = 0.0; NumberOfScores = 0; Mean = 0; Read(Scor eFile, Score); /*Read in and sum the scores*/ while (! EOF(ScoreFile) { if ( Score > 0.0 ) { SumOfScores = SumOfScores + Score; NumberOfScores++; } Read(ScoreFile, Score); } /* Compute the mean and print the result */ if (NumberOfScores > 0 ) { Mean = SumOfScores/NumberOfScores; printf("The mean score is %f \n", Mean); } else printf("No scores found in file\n"); }
Software Testing Slide 51

White-box Testing Example: Determining the Paths

FindMean (FILE ScoreFile) { float SumOfScores = 0.0; int NumberOfScores = 0; float Mean=0.0; float Score; Read(ScoreFile, Score); 2 while (! EOF(ScoreFile) { 3 if (Score > 0.0 ) { SumOfScores = SumOfScores + Score; NumberOfScores++; 5} Read(ScoreFile, Score); 7 } /* Compute the mean and print the result */ if (NumberOfScores > 0) { Mean = SumOfScores / NumberOfScores; printf( The mean score is %f\n, Mean); } else printf (No scores found in file\n); 1

8 9

}
Software Testing Slide 52

Constructing the Logic Flow Diagram

Start 1 F T 3 T 4 6 7 T 8 Exit
Software Testing Slide 53

F 5

F 9

Finding the Test Cases

Start
1 a (Covered by any data) 2 b (Data set must contain at least one value) (Positive score) d c 4 (Data set must f be empty) 3 e (Negative score) 5 h (Reached if either f or g e is reached) j (Total score > 0.0)

(Total score < 0.0) 8

9 k
Exit

Software Testing

Slide 54

Self reading

Unit-testing Heuristics
1. Create unit tests as soon as object design is completed: Black-box test: Test the use cases & functional model White-box test: Test the dynamic model Data-structure test: Test the object model 2. Develop the test cases Goal: Find the minimal number of test cases to cover as many paths as possible 3. Cross-check the test cases to eliminate duplicates Don't waste your time! 4. Desk check your source code Reduces testing time 5. Create a test harness Test drivers and test stubs are needed for integration testing 6. Describe the test oracle Often the result of the first successfully executed test 7. Execute the test cases Dont forget regression testing Re-execute test cases every time a change is made.
Big cost -> what should be done?

8. Compare the results of the test with the test oracle Automate as much as possible
Slide 55

Software Testing

THANKS!

Software Testing

Slide 56