Defect Prevention: A General Framework and Its Application
Meng Li He Xiaoyuan
Neusoft Group Ltd. Neusoft Group Ltd.
Shenyang, LN, 110003 Shenyang, LN, 110003
P.R.C P.R.C
rnennl@,neusoft.corn Hexy@,neusoft.cont
Abstract
Defect prevention in CMM and Causal Analysis
and Resolution in CMMI are focused on identzjjing the
root cause of defects and preventing defects from
recurring. Actions are expected at a project level as
well as organization level. This paper provides a
general framework of defect prevention activities
which consists of organization structure, defect
definition, defect prevention process and quality
culture establishment. Implementation of defect
prevention results in rapid and sustained improvement
in sofhvare product quality which is evident from an
example in Neusoft Group, where defect density in post
release phase decreased from 0.85defects/KLOC in
2000 to O.Idefects/KL.OCin 2005.
1. Introduction
Everybody likes Defect prevention as nobody likes
defects, especially those defects that are found by our
customers. We can stop defects from reaching our
customers by catching them before we deliver them to
our customer. Defect prevention requires a very
systematic approach based on clear understanding of
our own capabilities related to processes, tools and
staff skills.
Defect prevention in the CMM and Causal
Analysis and Resolution in the CMMI are focused on
identifying the root cause of defects and preventing
defects from recumng. Actions are expected at a
project level as well as at the organization level. This
paper is based on a real life example, having built a
process database, successfUlly implemented defect
prevention process to meet the customer requirements
of delivered quality. While implementing the defect
prevention process, the organization paid specific
attention to the following aspects as organization
Proceedings of the Sixth International Conference on Quality Software (QSIC'06)
0-7695-2718-3/06 $20.00 © 2006
Ashok Sontakke
Nihilent Technologies Pvt. Ltd.
Pune, 411001,
India,
a.sontakke@nihiIent.com
structure, defect type definition and building quality
culture.
The paper starts with a brief introduction to the
organization with a specific emphasis on the software
process improvement activities before the start of
defect prevention activities. It describes the
organization structure used for the defect prevention
activity, emphasizing the need to build the domain
knowledge and use it for defect prevention activities. It
brings out the issues that need to be addressed in
defining the defect types .The activities camed out in
defining the defect types are explained and different
types of defect types and their descriptions are
presented. The paper also describes the defect
Prevention process, the role of organization to support
the defect Prevention process and the actual results
achieved.
2. Software process improvement
background of the example
This example has over 10 years experience in
developing embedded software for automobile
industry. The customers required that the software
product quality must reach the world-class standards,
while ensuring on time delivery. So over the years, the
division had to establish the activities addressing
improvement of product quality and shorter
development cycle time, as a business goal.
The division adopted IS09001 standard at the
beginning, and from 1999 started to incorporate CMM
continuous improvement thoughts and Level213
requirements. It focused on the data collection and
analysis of software process, and built the
organization-level process database based on project
data and started using the data in project planning and
process tracking. During this period, the division also
focused on making the staff conscious about quality
and importance of process improvement and guided
 the staff to take an active part in controlling and The example is focused on a very special domain
improving the product quality. The staff volunteered to of automobile system development. Most of the
form quality control teams to address the problems and defects are due to the lack of domain skills and
shortcoming in products on their own initiative. knowledge, especially hardware chips and hardware
However, this approach was not proactive and it was testing environments. To improve the quality of the
not focused on the root cause analysis and preventive product and to implement defect prevention process
actions and as a result, similar quality problems kffectively, it was necessary to address the
continued to occur. In order to meet the customer accumulation of skills and knowledge. The division
quality goal for delivered quality for 02defect/KLOC selected a matrix structure to manage its projects in
by 2004, the division decided to implement the defect Figure 1.
prevention process, making the actions proactive rather Project's defect prevention group is responsible for
than reactive. project-level defect prevention activities and this group
consists of project manager, module leaders and
3. Construct organization structure for experienced developers, generally 3-4 persons. It is
mainly responsible for defect data collection and
defect prevention activities causal analysis. Based on organization's recommended
defect prevention proposals, the group decides the
appropriate prevention measures and action plans after
1
I
Defeet Prevention Group
ofproject 1
i~ I considering the project's features, and then tracks and
I I I summarizes the results of these measures during the
development process.
~r~anizatlon
Defeet I Organization-level defect prevention activities are
Preventioq managed by organizational defect prevention group,
which consists of Software Engineering Process
Technical Leader Groups, technical director and technical team leader,
I "'
generally consisting of 5-6 persons. The group is
mainly responsible for gathering the analysis results of
project's defect prevention activities, analyzing and
- -

identifying the common causes and making proposals

and action plans. Each technical team, as shown in
Module 2 I Figure 1, implements the plan. The technical team of
each module is responsible for the prevention of
common defects related to this module, and design
Module n team is responsible for the prevention of defects
... injected by design methodsfdesign template. The
I Module n Leader
organization-level defect prevention tracks and
summarizes the results of action plans and preventive

i / /~
measures on a periodic basis.
Organization-level defect prevention activities
focus on eliminating the common defects in several
projects. Its purpose is to improve capability of the
~ t e m ~ s i ~ / a s &view
i g n
Design organization. The preventive actions always result in
the modification of Organization Standard Software
Process. Thus the organization-level defect prevention
activities are the main source of continuous process
improvement and they support development of stable
process.

.
Figure1 Construct defect prevention groups both
organization-level and project-level

Proceedings of the Sixth International Conference on Quality Software (QSIC'06)

0-7695-2718-3/06 $20.00 © 2006
 4. Implement defect prevention activities Interface;
Function;
4.1. Preparation Build.package/merge;
Assignment;
Documentation;
For an organization starting the defect prevention
Checking;
activities for the first time, some basic work needs be
Algorithm;
finished before project-level defect prevention
Timing/Serialization.
activities can start. For example:
Third, it mapped the defects in defect prevention
Analyze historical data.
database to these defect types to see if the coverage for
Define types of defects.
the defects generated in the development of the
Establish organization-level defect prevention
software is adequate. If a group of defects had no
checklist.
proper defect type, then it added a new defect type. To
While this work is being done, special attention
make sure that the developers use the defect types
needs to be given to the product features and staffs
correctly, the relationship between the defect types and
maturity level while analyzing the historical defect
corresponding activities was given.
data. Otherwise, organization's defect prevention
After completing the two steps given earlier, the
activities will be confronted with following issues:
organization defect prevention group released the
Organization's defined defect types are too
organization defect type standard for use by the
abstract, and it is difficult for project members to
projects. Some new defect types related to
master or understand.
requirements and hardware were added after further
Organization's defined defect types are not
understanding of the embedded software projects and
adequate; some defects can not be categorized.
understanding the customer requirements.
Organization's defined defect prevention
Here is a sample of defect type definition. In
checklist only gives the common requirements
Table 1, there are 10 categories of defects defined by
instead of requirements for specific and
organization defect prevention group. After they
frequently occurring errors. It does not interest
released it to project teams, there were another two
project members.
categories were added by developers. There were four
If the above issues occur, the defect prevention
categories of defects that would be detected and
activities suffer because of resentment from project
recorded with great attention. They are Feature error,
members. In such a situation, even if defect prevention
Feature omission, Interface error and Function error.
activities are implemented, project members can't
The phases to detect defects are: Requirement Analysis
focus on them properly and they lead to a failure.
(RA), Preliminary Design (PD), Detailed Design (DD),
and Coding (COD).
4.2. Definition of defect type

First, the organization defect prevention group

collected the defects generated by the past projects,
and then selected some of the defects to build a defect
prevention database. The factors that were considered
during the defect selection were
Typicality, the defects are found in most of the
projects.
Severity, the defects have resulted in serious
impact to the product quality.
Reoccurrence, the defects occur many times.
Defects detected by the customer.
Second, the organization defect prevention group
used the standard defect types such as Orthogonal
defect Classification (ODC) as an initial classification.
ODC involves tracking and analyzing many attributes
of defects. These attributes includes type of defect,
detecting activity, and inserting activity. Eight types of
defects are defined within ODC:

Proceedings of the Sixth International Conference on Quality Software (QSIC'06)

0-7695-2718-3/06 $20.00 © 2006
 Table 1. Typical defect definition in embedded system
development
43. Defect prevention process
To start with, the defect prevention policy was
documented and communicated to the staff .The defect
prevention process was defined and documented and
the staff was trained on the process with a special
emphasis on understanding the defect types, root cause
analysis and individual roles and responsibilities.
The Project defect prevention groups meet at the
beginning of a phase to understand changes made to
Organization Standard Software Process defect
prevention checklist and also to address project
Proceedings of the Sixth International Conference on Quality Software (QSIC'06)
0-7695-2718-3/06 $20.00 © 2006
specific needs. The defect data is captured using an
automated tool. The Project defect prevention groups
also meet at the end of a phase, to cany out root cause
analysis and communicate results to the Organizational
defect prevention group. The defect distribution across
defect types is presented in a Histogram and Pareto
analysis is carried out to compare against expected
defect distribution, based on the past project defect
distribution. Corrective and preventive actions are
discussed and a record is kept. The project defect
prevention group calls for a special meeting, during a
phase, if the phase defect density is likely to be outside
the phase defect density target, and decides actions
required.
The Organizational defect prevention group
meets once a month to review the results obtained at
the project level. It makes suggestions for updating the
defect prevention check lists in the Organization
Standard Software Process and identifies requirements
for process training for defect prevention
activities .The defect prevention results are
communicated to organizational Software Process
Improvement project team, which coordinates
activities for Technology Change, Process Change and
defect prevention to achieve specified Software
Process Improvement goals to meet business needs.
Defect prevention groups are able to find the
most important types of defects based on the Pareto
Analysis and then they focus on preventing these
defect types. Once these kinds of defect types are
controlled, further analysis is done to prioritize defect
types for prevention.
4.4. Support for defect prevention
To ensure that the activities of the defect
prevention are implemented continuously and
effectively, necessary environment must be provided
by organization.
The first requirement is to create suitable
circumstances for the implementation of defect
prevention. Here the circumstances relate to
establishing policies and providing resources. They are
the necessary preconditions to ensure smooth
implementation of defect prevention. To encourage the
developers to participate actively in the
implementation of defect prevention, a series of
policies were defined such as awarding the best defect
prevention groups and the best individuals with a prize,
rewarding a project's contribution to defect prevention
activities while doing the project performance
evaluation, and periodic publishing of the progress and
results of defect prevention activities in the meetings
and in the division's newsletter .To provide continuous
 and effective support it is important that the senior defects in 2001 and 50% in 2002. For Feature Error,
management makes firm commitments by providing defect number decreased from 405 to 150 and defect
the resources needed by defect prevention activities, density decreased from 6.17 to 2.65. For Feature
and getting actively involved in the development of Omission, defect number decreased from 353 to 68
action plan and in monitoring the defect prevention and defect density decreased from 5.38 to 1.19. The
results. results in the following table show that the defect
The second requirement is to provide adequate density for the four defect types decreased by 57%,
training. Any new process institutionalization will go 78%, 86% and 75%, respectively.
through four stages. They are awareness,
understanding, implementation, and active Table 2. Defect density and defect distribution in
participation. Therefore, three types of defect development phase
prevention training were provided in the division.
Training course: the course focuses on the defect
prevention concepts, defect prevention process
and the definition of the organization level defect
types. The course is compulsory for the
developers as everybody should have the basic
knowledge on defect prevention.
Initial coaching: when a project defect prevention
group implements defect prevention activities for
the first time, they must do it under the guidance
of the organizational defect prevention group
members. This is providing help in using the
organization defect prevention checklist to
generate a project defect prevention checklist and
in executing the causal analysis activity.
On job training: as needed, the organizational
defect prevention group members provide error
consulting to the project team to help them
resolve any specific issues related to the defect 5.2. Product quality in post release phase
prevention implementation.
From 2001, the division achieved good business
5. Overall results results in product quality through defect prevention
and some other Software Process Improvement
The benefits of defect prevention implementation activities like using new tools. The data in Figure 2
in an organization can be seen in three areas, shows that the product quality improved rapidly and
improvement in the product quality, delivery capability significantly. The defects found in post release phase
and changes in the developer's attitude towards work. decreased from 0.85 defects/KLOC in 2000 to
0.1 defects/KLOC in 2005, greatly exceeding the
5.1. Product quality in development phase customer requirement 0.2 defectsKLOC.

Based on the analysis of the historical defect data, Defect density

the defect prevention activities focused on the
following four main defect types: Feature Error,
Feature Omission, Function Error and Interface Error.
The actions defined by the defect prevention group
dealt with template revision, implementation
techniques and training.
The defect type, number of defects, defect density
and percentage of total defects in 2001 and 2002 are
illustrated in Table 2. The average project size is
65.57KLOC in 2001 and 56.86KLOC in 2002. The Figure 2. Defect density in post release phase
four main defect types cover approximately 70%

Proceedings of the Sixth International Conference on Quality Software (QSIC'06)

0-7695-2718-3/06 $20.00 © 2006
 5.3. Product delivery on time
Another result from defect prevention activities is
the average delivery delay days in Version 0.99 as
shown in Figure 3. The delay days is reduced from
36.7 days in 2000 to 3 days in 2005, almost 12 times
decreased.
Ver0.99 Delivery Delay Days
[l]Paulk M.,et al., Capability Maturi&~Model, Addison -
Wesley, Reading, MA, 1994.
[ZIChrissis M.B., et al., CMMI Guidelinesfor Process
Integration and Product Improvement, Addison-Wesley,
Reading, MA, 2003
[3]Chillarge R., et al., "Orthogonal defect classification",
IEEE Transactions on Software Engineering (November
1992)

Figure 3. Delivery delay days of Version 0.99

5.4. Changes in the attitude towards work

Though improvement in product quality is

important, the culture of defect prevention and change
in the developer's attitude towards work may be even
more important. With defect prevention
implementation, the developers have acquired a new
way to improve their working skills. Now, whenever
they make an error in their work, they think of why it
happened and then do some changes to avoid it in the
future.

6. Summary and conclusions

Implementation of defect prevention practices

result in rapid and sustained improvement in the
product quality as is evident from the example. To get
such results, organization should fully consider the
product characteristics, make defect prevention
activities responsibilities for each working-unit and
demonstrate a firm senior management commitment by
creating a suitable environment.
In adoption of CMM, CMMI and IS09001, this
paper introduced the empirical studies on defect
prevention activities in a software company. It may
help the academic researchers to identify important
new areas of focus. Defect prevention needs to be
approached more rigorously and objectively than it
often has been in practice. Academic researchers can
help to fill the gap by studying the principles and
evaluating the results from industrial feedback.

References

Proceedings of the Sixth International Conference on Quality Software (QSIC'06)

0-7695-2718-3/06 $20.00 © 2006