Software Quality Engineering

Lecture 1
Introduction to Software Quality Engineering

Muhammad Fawad
Lecturer
COMSATS Institute of Information Technology

1 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Course Introduction (Marks Distribution) …
Criteria %
Sessional # 1 10%

Sessional # 2 15%

Quizzes 5%

Assignments 5%

Presentations/Projects 15%

Terminal Exam 50%

2 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Course Introduction (Books) …
• Course Book:
“Software Quality Engineering: Testing, Quality
Assurance, and Quantifiable Improvement”, by Jeff Tian
available in soft form

• Reference Book:
“Software Quality Assurance From Theory to Implementation
by Daniel Galin
available in soft form

3 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 General Class Rules

• Attendance: Greater than 80% attendance REQUIRED!

• Disturbance in class: ‘Nothing’ which ‘disturbs’ will be tolerated!

• Mobile Phone in Class: You will be marked absent if your mobile

phone rings.

4 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Plagiarism/Cheating
University has ‘zero tolerance’ policy for cheating!

Can earn you a punishment from getting zero in the cheated stuff to awarding an F
grade in the whole course to expulsion from university based on the severity of the act.

In this course, a student found guilty of cheating in an assignment/quiz/project will be

awarded ZERO in that exam.

A student marked as a cheater twice in assignments will get ZERO in all assignments.

Further attempt to cheating will be referred to the DC.

Exam cheating cases go to DC which generally awards a minimum of an F grade in the

whole course if proven guilty.

5 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Time Management …
• Manage your time well while on campus, during travel
and at home! Utilize the maximum NOW instead of
getting to full throttle near exams without much benefit!

• Identify your weaknesses and strengths and work

accordingly.

• It is recommended that:
– Be attentive in the class.
– In the first possible free time, do a ‘few problems’ related to
the topic(s) covered in class.
– If you are able to do it, it means you got the point; if not, at
least you know what you do not know. Ask the teacher in the
next class.
– At the weekend, sum up all the week’s study.

6 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 What we will cover?

• General understanding of Software Quality

• Software Quality Process
• Software Quality Models
• Software Testing Techniques
• Defect Prevention and Process improvement
• Automatic Testing
• Related tools and techniques

7 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Software Engineering

• A little about your software engineering background?

8 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 OK……

• Lets start our proper lecture now….

9 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Why Quality??
• Software is a major component of computer
systems (about 80% of the cost) – used for
– Communication (e.g. phone system, email system).
– Health monitoring.
– Transportation (e.g. automobile, aeronautics).
– Economic exchanges (e.g. e-commerce).
– Entertainment.
– etc.
• Software defects are extremely costly in term of
– money
– reputation
– loss of life

10 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Why Quality???
• NASA Mariner 1 , Venus probe
(period instead of comma in FORTRAN DO-Loop, 1962)
– Cost: $18.5 million
• Ariane 5, Explosion
(data conversion of a too large number, 1996)

• Patriot-Scud
(rounding error, 1991)

• Mars Climate Orbiter, Loss

(Mixture of pounds and kilograms, 1999)

• London Ambulance (1992).

11 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Why Quality?
• Internet viruses and worms
– Software Vulnerable
– Data Protection

• Usability problems

12 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Why Quality?
• Monetary impact of poor software quality (Standish
group - 1995)
• 175,000 software projects/year - Average Cost per
project
– Large companies - $US 2,322,000
– Medium companies - $US 1,331,000
– Small companies - $US 434,000
• 31.1% of projects cancelled before completed.
– cost $81 billion

13 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Why Quality?
• 52.7% of projects exceed their budget - costing 189% of
original estimates.
– cost $59 billion
• 16.2% of software projects completed on-time and on-
budget (9% for larger companies).
• Large companies - delivered systems have
approximately only 42% of originally-proposed features
and functions.
• 78.4% of smaller companies projects get deployed with
at least 74.2% of their original features and functions.

14 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 What is Software Quality?

• Conformance to requirements or in narrowest sense

software quality is.
– Lack of bugs.
– Low defect rate (# of defects/size unit)
– High reliability (number of failures per n hours of operation).
• However, there are many perspectives and expectation
that answer it..

15 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Perspectives

• Five major views

– Transcendental view: seen/not-defined.

– User view: fitness for purpose.
– Manufacturing view: conform to specs.
– Product view: inherent characteristics.
– Value-based view: willing to pay.

16 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 People’s roles and responsibilities
and expectation
• Two broad groups: Consumer and Producers
– Consumer (External)
• Customers and users
• generalized user: other hw/sw/system/etc.
• “good enough" for the price
• fit-for-use, doing the “right things"
• conformance, doing “things right"

17 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Producers Quality expectations

– Producers (Internal)
• Fulfil their contractual commitments to produce product conforming to
product specification.
• “good enough" for the cost
• functionality & correctness via V&V
• cost: managers
• service related: maintainability
• interfacing units: interoperability

18 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 People: Consumers vs producers
• Quality expectations by consumers

• to be satisfied by producers through software quality engineering

(SQE)

19 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Expectations
• Does what it is supposed to do (A valid system)
– E.g., An online reservation system

• Does the things correctly (A verified system)

– Needs to make reservations within a pre-specified time.

20 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Meeting Quality Expectations
• Difficulties in achieving good quality:
– size: MLOC products common
– complexity
– environmental stress/constraints
– flexibility/adaptability expected
• Other difficulties/factors:
– product type
– cost and market conditions
• no silver bullet", but...
– SQE (software quality engineering) helps

21 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 SQE as an Answer

• Major SQE activities:

– Testing: remove defect
& ensure quality
– Other QA alternatives
to testing
– How do you know:
analysis & modeling

22 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Quality Assurance
• Managing quality is an essential and difficult challenge. Quality is
essential because it brings value.

• As important as quality is to business, managing to achieve it is

extraordinarily difficult. Managing quality is as challenging as trying
to manage life itself.

23 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Software Testing

• Plays the central role

– By running the software system or executing its prescribed
functions, testers can determine if the observed system behavior
conforms to its specifications or requirements.
– If error found, debugging and design modification carried out.
– Helps reduce the number of defects.
• So if we don’t find any errors in testing and it takes up
like 50% of resources, then what the point????

24 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Other Quality Assurance Activities
• Inspection,
• Formal verification,
• Defect prevention,
• Fault tolerance.

25 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Software Quality Engineering
• All testing and other quality assurance activities needs to be
properly managed.

• QA goals set early in the product development and monitored to

achieve these goals

26 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Quality Frameworks
• To accommodate the different quality views, many frameworks have
been defined.

• One of the most influential one is ISO-9126.

27 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 ISO 9126 quality characteristics

Six top level quality characteristics

– Functionality: what is needed?
– Reliability: function correctly.
– Usability: effort to use.
– Eficiency: resource needed.
– Maintainability: correct/improve/adapt.
– Portability: one environment to another.

28 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 ISO-9126

29 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Other Quality Frameworks

• Adaptation of ISO-9126:
• customized for companies
– e.g., IBM's CUPRIMDS.

• adapted to application domains

– reliability, usability, security for Web

30 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Other quality frameworks/mega-models
(McCall model)

31 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 ISO 9000

• ISO 9000 is a set of international standards that can be used in the

development of a quality management system in all industries.

• ISO 9001 is the most general of these standards that applies to

organizations concerned with the quality process in products design,
development, and maintenance

• ISO 9000-3 is a supporting document, which interprets ISO 9000 for

software development.

32 COMSATS Institute of Information Technology, Attock Muhammad Fawad

Some of the requirements in ISO 9000-3
• Develop and document software test plans.
• Develop methods to test whether the software meets the
customer's requirements.
• Perform software validation and acceptance tests.
• Maintain records of the test results.
• Control how software bugs are investigated and resolved.
• Prove that the product is ready before it is released.
• Develop procedures to control the software's release process.
• Identify and define what quality information should be collected.
• Use statistical techniques to analyze the software development
process.
• Use statistical techniques to evaluate product quality

33 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 CMM

34 COMSATS Institute of Information Technology, Attock Muhammad Fawad

35 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Focus on Correctness 1/2

• Among the software quality attributes, some deal

directly with the functional correctness
– Demonstrated by presence/absence of problems
• Other attributes deal with usability portability etc
• Correctness is typically related to several quality
characteristics or sub-characteristics in quality
frameworks described before
– For example, in ISO-9126 it is related to both functionality and
reliability

36
36 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Focus on Correctness 2/2

• Correctness is typically the most important aspect of

quality for situations where daily life or business depends
on the software
• Even in other cases, correctness remains a fundamental
part of the users’ expectations
• Therefore, we adopt the correctness-centered view of
quality throughout this course

37
37 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Correctness & Defects: Definitions

• Failure: external behavior

– Deviation from expected/specified behavior
• Fault: internal characteristics - cause for failures
– An incorrect step, process, or data definition in a computer
program
• Error: refers to a missing or incorrect human action such
as human misconceptions, misunderstandings, etc
resulting in certain fault(s) being injected into a software
– during design, coding and data entry

38
38 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Correctness & Defects: Definitions

• Defect:
– Generally refers to some problem, either with external behavior or
with internal characteristics
– error/fault/failure are collectively referred to as defects
• Bug/debug:
– not good terms, avoid
– Instead use defect detection & removal

39
39 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Defect and Quality

40 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Error, Fault, Failure: Relation

• The software system as represented by its artifacts is

depicted in the middle box.
• The artifacts include
– mainly software code
– designs,
– specifications, requirement documents, etc.
• The faults scattered among these artifacts are depicted
as circled entities within the middle box

41
41 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Error, Fault, Failure: Relation

• The input to the software development activities, depicted

in the left box, include
– conceptual models and information,
– developers with certain knowledge and experience,
– reusable software components, etc.
• Various error sources are also depicted as circled entities
within this left box
• Right most box shows behavior patterns
– Subset of these behavior patterns or results can be classified as
failures when they deviate

42
42 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Error, Fault, Failure: Relation

• A causal relation exists among the three aspects of

defects:
– errors → faults → failures
• However, this relationship is not necessarily 1-to-1:
– a single error may cause many faults
– a single fault may cause many failures in repeated executions
– conversely, the same failure may be caused by several faults
– the same fault may be there due to different errors

43
43 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Nine Causes of Software Errors
(Galin)
• Faulty requirements definition.
• Client-developer communication failures.
• Deliberate deviations from software requirements.
• Logical design errors.
• Coding errors.
• Non-compliance with documentation and coding
instructions.
• Shortcomings of the testing process.
• User interface and procedure errors.
• Documentation errors.

44 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Faulty requirements definition
• Erroneous definition of requirements.
• Absence of vital requirements.
• Incomplete definition of requirements
• Inclusion of unnecessary requirements

45 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Client–developer communication
failures
• Misunderstanding of the client’s instructions as stated in the
requirement document.
• Misunderstanding of the client’s requirements changes
presented to thedeveloper in written form during the
development period.
• Misunderstanding of the client’s requirements changes
presented orally to the developer during the development
period.
• Misunderstanding of the client’s responses to the design
problems presented by the developer.
• Lack of attention to client messages referring to requirements
changes
• and to client responses to questions raised by the developer on
the part of the developer
46 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 Deliberate deviations from software
requirements
• The developer reuses software modules taken
from an earlier project without sufficient analysis
of the changes and adaptations needed to
correctly fulfill all the new requirements.
• Due to time or budget pressures, the developer
decides to omit part of the required functions in
an attempt to cope with these pressures.
• Developer-initiated, unapproved mprovements
to the software, introduced without the client’s
approval

47 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Logical design errors
• Definitions that represent software requirements by means of erroneous
algorithms.
• Process definitions that contain sequencing errors.
• Omission of required software system states.
• Omission of definitions concerning reactions to illegal operation of the
software system.

48 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Coding errors
• misunderstanding the design documentation,
• linguistic errors in the programming languages,
• errors in the application of CASE and other development tools,
• errors in data selection

49 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Non-compliance with documentation
and coding instructions
• Can lead to much confusion

50 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Shortcomings of the testing
process
• Incomplete test plans leave untreated portions of
the software or the application functions and
states of the system.
• Failures to document and report detected errors
and faults.
• Failure to promptly correct detected software
faults as a result of inappropriate indications of
the reasons for the fault.
• Incomplete correction of detected errors due to
negligence or time pressures.

51 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Procedure errors
• Procedures direct the user with respect to the activities required at
each step of the process.
• So not following proper procedure can lead to error

52 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Documentation errors

• Omission of software functions.

• Errors in the explanations and instructions given
to users, resulting in “dead ends” or incorrect
applications.
• Listing of non-existing software functions, that is,
functions planned in the early stages of
development but later dropped, and functions
that were active in previous versions of the
software but cancelled in the current version.

53 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 When are defects introduced?

• The majority of defects are introduced in earlier phases.

– Requirements are the top factor in a project’s success
or failure.

Phase % of defects
Requirements 56
Design 27
Code 7
Other 10

54 COMSATS Institute of Information Technology, Attock Muhammad Fawad

 Cost of fixing defects
• Relative cost of fixing defects
– benchmark: cost at requirements phase = 1

Phase found Cost ratio

Requirements 1
Design 3–5
Coding 10
Unit / integration testing 15 - 40
System / acceptance 30 – 70
testing
Production 40 - 1000
55 COMSATS Institute of Information Technology, Attock Muhammad Fawad
 What is software quality
Assurance?

56 COMSATS Institute of Information Technology, Attock Muhammad Fawad