Software Quality

Assurance

SEN-269 : Software Engineering

Tazeen Muzammil
 What is Quality
• The term 'quality' is often used in a vague,
blurred way. If someone talks about 'working on
quality', they may simply mean activities
designed to improve the organization and its
services. 
• Quality is about:
– Knowing what you want to do and how you want to
do it. Learning from what you do.
– Using what you learn to develop your organization
and its services.
– Seeking to achieve continuous improvement.
– Satisfying your stakeholders - those different people
and groups with an interest in your organization.
 Why Quality Matters?
Increasing complexity of software product
Increasing demand for software products
Increasing lifetime of software products
Increasing use of "Safety Critical" or
"Mission Critical" software

3
 Software Quality
Software Quality is defined as Conformance to:
1. explicitly stated functional and performance
requirements,
2. explicitly documented development standards,
3. and implicit characteristics that are expected
of all professionally developed software.

In the context of software engineering,

software quality measures how well software
is designed (quality of design), and how well
the software conforms to that design (quality
of conformance).

4
What is Quality Control?
Quality Control is the process of developing systems that ensures
that products or services are designed and produced to meet or
exceed customer requirements, and expectations.
Key concept of quality control:
Is to ensure that the products, services, or processes provided
meet specific requirements and are dependable, satisfactory,
and fiscally sound.
Implementation approaches:
- Fully automated
- Entirely manual
- Combination of automated tools and human interactions
 What is Software Quality
Assurance?
Software Quality Assurance (SQA) is defined as a planned and
systematic approach to the evaluation of the quality of the
software product standards, processes, and procedures.

SQA includes the process of assuring that standards and

procedures are established and are followed throughout the
software acquisition life cycle.

Goal --> provide management with the necessary data about

product quality.
--> gain the insight and confidence of product quality
 Software Quality
Assurance
The most popular tool used to determine quality
assurance is the Shewhart Cycle, developed by
Dr. W. Edwards Deming.
This cycle for quality assurance consists of four
steps: Plan, Do, Check, and Act. These steps are
commonly abbreviated as PDCA.
ISO 9001:2000 was developed in line with the
long-used Shewhart cycle of process or project
management; Plan, Do, Check, Act, referred to in
ISO 9004 as Plan, Implement, Check, Review.

7
 Plan, Do, Check, Act
(PDCA)

Plan; adhering to the company mission and vision, the management

sets objectives based on information gathered through the quality
system (customer feedback, management reports, etc.)
Do; the product is produced or the service delivered in accordance
with the customers requirements.
Check; was it done right? Obtain information from the customer and
the organization to measure and monitor performance.
Act; “freeze” successful activities and improve less successful ones.
This step is the key to continual improvement.

8
9
10
11
12
13
14
Quality Consulting

15
 What is Software Quality
Engineering?
Software Quality Engineering is composed of two primary activities
Process level quality
1. Process level quality is normally
called quality assurance.
2. Process level quality establishes
the techniques, procedures, and
tools that help promote,
encourage, facilitate, and create
a software development
environment in which efficient,
optimized, acceptable, and as
fault-free as possible software
code is produced.
Product level quality
1. Product oriented quality is
normally called testing
2. Product level quality
focuses on ensuring that
the software delivered is
as error-free as possible,
functionally sound, and
meets or exceeds the real
user’s needs. Testing is
normally done for finding
errors in order to get rid
of them.
 Cost of Quality
Cost of quality includes all costs incurred in the pursuit of quality or
in performing quality related activities
Quality cost includes:
Prevention cost:
quality planning
formal technical reviews
testing equipment
Training
Appraisal cost: Include activities to gain insight into product
condition the “first time through” each process.
in-process inspection
equipment calibration and maintenance
testing

17
 Cost of Quality
Failure cost:
internal failure cost: Internal failure cost are incurred when we
detect a defect in our product prior to shipment
rework
Repair
failure mode analysis
external failure cost: External failure costs are is associated with
defects found after the product has been shipped to customer.
complaint resolution
product return and replacement
help line support
warranty work
The relative cost to find and repair a defect increases dramatically as we
go from prevention to detection to internal failure to external failure cost.
(See fig: Relative cost of correcting an error)

18
 SQA Activities
Prepare SQA plan for the project.
The plan identifies:
Participate in the development of the project's software process
•Evaluation to be performed.
description.
The software team selects a process. The SQA group reviews
Review software engineering activities to verify compliance with
•Audits and reviews to be performed.
the
the process description
defined software for compliance with organizational
process.
•Standards that are applicable to the project.
policy,
The
Audit SQAinternal
groupsoftware
designated software standards,
identifies,
workdocuments,
products externally
to and imposed
tracks
verify deviations
compliance
Standards
with those
from (E.g. ISO-9001),
defined
the process as part
and and
of the
verifies other
software
that parts of have
process.
corrections
•Procedures for error reporting and tracking. the software
been
Ensure that
project plan.any deviations in software or work products are
made.
•Documents to be produced by the SQA group.
The SQA group
documented identifies,
and handled documents,
according and tracksprocedure.
to a documented deviations
from
Recordthe process and
of verifies that corrections
•Amount of feedback provided to the software
any evidence noncompliance and reportshave
thembeen
to
management.
Made; and periodically reports the result of its work to the
project team.
project manager.
Deviation may be encountered in the project plan, process
description, applicable standards, or technical work products.

Non-compliance items are tracked until they are resolved.

19
 Software Reviews
Objective: find errors early before they become defects.
Excellent ROI: empirical data show that design activities
introduce 50-65 % of errors, but formal reviews can
detect up-to 75 % of those.
Reviews are made to
Uncover errors.
Ensure that software meets requirements and standards.
Ensure that software and the process is manageable.
Using formal technical reviews (walkthroughs or
inspections) is an effective means for improving software
quality.

20
 Review Roles
Presenter (designer/producer).
Coordinator (not person who hires/fires).
Recorder
records events of meeting
builds paper trail
Reviewers
maintenance oracle (vision)
standards bearer
user representative
others

21
Reviews (Walkthroughs)
In the review process, the producers present a
product (part of the software, e.g. a code module
or a design model) which is then reviewed by
reviewers.

Records should be kept about the reviews. This

includes lists of issues discovered.

A follow-up procedure must be in place to correct

issues.

22
 Defect Amplification
Models

Defect amplification models can be used to

illustrate the importance of detecting defects
in an early stage. They are a simple
mathematical model for describing how
defects are propagated through the various
steps in the SD process.

23
Defect amplification,
no review

Defect amplification,
Reviews conducted

24
 Defect Causes
IES: Incomplete and erroneous specification. Example: customer
"forgets" an interface to an existing system.
MCC: Misinterpretation of customer communication. Example:
Customer means one thing, analyst understands something
else.
IDS: Intentional derivation from specification. Example: Analyst
or programmer things he knows better.
VPS: Violation of Programming Standards. Example: Tools are
used which rely on naming patterns (e.g. bean tools), but the
code does not adhere to those patterns.
EDR: Error in Data Representation. Example: wrong container
types are used, e.g. containers which support duplicates, but
this is not permitted in the business logic.
ICI: Inconsistant Component Interface. Example: component
interface does not support all parameters needed to customize
the component.

25
 Defect Causes
EDL: Error in design logic. Example: wrong algorithm
implemented.
IET: Incomplete and erroneous testing. Example: there are
errors in the test cases itself, and the code is correct w.r.t.
the incorrect test cases.
IID: Inaccurate and incomplete documentation. Example:
the meaning of an API method is incorrectly (or not)
documented, and programmers in a team using this
component pass the wrong parameter to this component.
PLT: Error in PL translation of design. Example: classes
design to be abstract are implemented as concrete classes
and instantiated.
HCI: Ambiguous or inconsistent human computer interface.
Example: different meaning of the same key stroke or
menu function in different contexts.
MIS miscellaneous.

26
Formal Technical Reviews
Involves 3 to 5 people (including reviewers)
Advance preparation (no more than 2 hours
per person) required
Duration of review meeting should be less
than 2 hours
Focus of review is on a discrete work product
Review leader organizes the review meeting
at the producer's request.

27
Formal Technical Reviews
Reviewers ask questions that enable the
producer to discover his or her own error (the
product is under review not the producer)
Producer of the work product walks the
reviewers through the product
Recorder writes down any significant issues
raised during the review
Reviewers decide to accept or reject the work
product and whether to require additional
reviews of product or not.

28
 Why do peer reviews?
To improve quality.
Catches 80% of all errors if done
properly.
Catches both coding errors and
design errors.
Enforce the spirit of any
organization standards.
Training and insurance.
29
 Review Guidelines
Keep it short (< 30 minutes).
Don’t schedule two in a row.
Don’t review product fragments.
Use standards to avoid style
disagreements.
Let the coordinator run the
meeting and maintain order.

30
 SQA Plan
Management section
describes the place of SQA in the structure of
the organization
Documentation section
describes each work product produced as part
of the software process
Standards, practices, and conventions
section
lists all applicable standards/practices applied
during the software process and any metrics to
be collected as part of the software
engineering work

31
 SQA Plan
Reviews and audits section
provides an overview of the approach
used in the reviews and audits to be
conducted during the project
Test section
references the test plan and
procedure document and defines test
record keeping requirements

32
 SQA Plan
Problem reporting and corrective action
section
defines procedures for reporting, tracking,
and resolving errors or defects, identifies
organizational responsibilities for these
activities
Other
tools, SQA methods, change control, record
keeping, training, and risk management

33