05 SQA Overview

Course Notes Set 5:
Software Quality Assurance
Computer Science and Software Engineering

Auburn University
Auburn University
Computer Science and Software Engineering COMP 6710 Course Notes Slide 5-0
What is Software Quality?
• Simplistically, quality is an attribute of

software that implies the software
meets its specification
• This definition is too simple for ensuring
quality in software systems
– Software specifications are often incomplete
or ambiguous
– Some quality attributes are difficult to
specify
– Tension exists between some quality
attributes, e.g. efficiency vs. reliability
Auburn University
Software Quality Attributes
• Safety • Modularity
• Security • Complexity
• Reliability • Portability
• Resilience • Usability
• Robustness • Reusability
• Understandability • Efficiency
• Testability • Learnability
• Adaptability
Auburn University
Software Quality
• Conformance to explicitly stated functional and
performance requirements, explicitly documented
development standards, and implicit characteristics that
are expected of all professionally developed software
– Software requirements are the foundation from which
quality is measured.
• Lack of conformance to requirements is lack of quality.
– Specified standards define a set of development criteria
that guide the manner in which software is engineered.
• If the criteria are not met, lack of quality will almost surely
result.
– There is a set of implicit requirements that often goes
unmentioned.
• If software conforms to its explicit requirements but fails to
meet its implicit requirements, software quality is suspect.
[Adapted from Pressman 4th Ed]
Auburn University
Software Quality Assurance
• To ensure quality in a software product, an organization must
have a three-prong approach to quality management:
– Organization-wide policies, procedures and standards must be
established.
– Project-specific policies, procedures and standards must be
tailored from the organization-wide templates.
– Quality must be controlled; that is, the organization must ensure
that the appropriate procedures are followed for each project
• Standards exist to help an organization draft an appropriate
software quality assurance plan.
– ISO 9000-3
– ANSI/IEEE standards
• External entities can be contracted to verify that an
organization is standard-compliant.
Auburn University
A Software Quality Plan
ISO 9000
model
Organization
quality plan
Project A Project B Project C

quality plan quality plan quality plan
[Adapted from Sommerville 5th Ed]
Auburn University
SQA Activities
• Applying technical methods
– To help the analyst achieve a high quality specification and a high quality design
• Conducting formal technical reviews
– A stylized meeting conducted by technical staff with the sole purpose of
uncovering quality problems
• Testing Software
– A series of test case design methods that help ensure effective error detection
• Enforcing standards
• Controlling change
– Applied during software development and maintenance
• Measurement
– Track software quality and asses the ability of methodological and procedural
changes to improve software quality
• Record keeping and reporting
– Provide procedures for the collection and dissemination of SQA information
Auburn University
Advantages of SQA
• Software will have fewer latent defects,

resulting in reduced effort and time
spent during testing and maintenance
• Higher reliability will result in greater
customer satisfaction
• Maintenance costs can be reduced
• Overall life cycle cost of software is
reduced
Auburn University
Disadvantages of SQA
• It is difficult to institute in small

organizations, where available
resources to perform necessary
activities are not available
• It represents cultural change - and
change is never easy
• It requires the expenditure of dollars
that would not otherwise be explicitly
budgeted to software engineering or
QA
Auburn University
Quality Reviews
• The fundamental method of validating the quality of a
product or a process.
• Applied during and/or at the end of each life cycle phase
– Point out needed improvements in the product of a single
person or team
– Confirm those parts of a product in which improvement is
either not desired or not needed
– Achieve technical work of more uniform, or at least more
predictable, quality than what can be achieved without
reviews, in order to make technical work more
manageable
• Quality reviews can have different intents:
– review for defect removal
– review for progress assessment
– review for consistency and conformance
Auburn University
Requirements
Quality Reviews
Analysis Specification
Review
1x
Design Design
Review
3-6x
Code Code
Review
10x Test
Testing Review
15-70x Customer
Maintenance Feedback
40-1000x
Auburn University
Cost Impact of Software
Defects
Errors from
Previous
Steps
Errors Passed Through Percent Efficiency
Amplified Errors 1:X for error
Newly Generated Errors detection
Errors
Passed to
Next Step
Auburn University
Defect Amplification and
Removal
Preliminary Design
0
10
0 0% Detailed Design
10 6
6
4 37
4x1.5 0% Code/Unit Testing
25 10
10
27 94
37 27x3 20%
25
116
To integration
testing...
Auburn University
Defect Amplification (cont’d)
94
Integration Testing
94
94
0 47
0 50% Validation Testing
0 47
47
0 24
94 0 50% System Testing
0 24
24
0 12
47 0 50%
0
24
Latent Errors
Auburn University
Review Checklist for Systems
Engineering
• Are major functions defined in a bounded and
unambiguous fashion?
• Are interfaces between system elements defined?
• Are performance bounds established for the system as a
whole and for each element?
• Are design constraints established for each element?
• Has the best alternative been selected?
• Is the solution technologically feasible?
• Has a mechanism for system validation and verification
been established?
• Is there consistency among all system elements?
Auburn University
[Adapted from Behforooz and Hudson]
Review Checklist for
Software Project Planning
• Is the software scope unambiguously defined and
bounded?
• Is terminology clear?
• Are resources adequate for the scope?
• Are resources readily available?
• Are tasks properly defined and sequenced?
• Is the basis for cost estimation reasonable? Has it been
developed using two different sources?
• Have historical productivity and quality data been used?
• Have differences in estimates been reconciled?
• Are pre-established budgets and deadlines realistic?
• Is the schedule consistent?
Auburn University
Software Requirements
Analysis
• Is the information domain analysis complete,
consistent, and accurate?
• Is problem partitioning complete?
• Are external and internal interfaces properly defined?
• Are all requirements traceable to the system level?
• Is prototyping conducted for the customer?
• Is performance achievable with constraints imposed by
other system elements?
• Are requirements consistent with schedule, resources,
and budget?
• Are validation criteria complete?
Auburn University
Software Design
(Preliminary Design Review)
• Are software requirements reflected in the
software architecture?
• Is effective modularity achieved? Are modules
functionally independent?
• Is program architecture factored?
• Are interfaces defined for modules and
external system elements?
• Is data structure consistent with software
requirements?
• Has maintainability been considered?
Auburn University
Software Design
(Design Walkthrough)
• Does the algorithm accomplish the desired function?
• Is the algorithm logically correct?
• Is the interface consistent with architectural design?
• Is logical complexity reasonable?
• Have error handling and “antibugging” been specified?
• Is local data structure properly defined?
• Are structured programming constructs used throughout?
• Is design detail amenable to the implementation language?
• Which are used: operating system or language dependent
features?
• Is compound or inverse logic used?
• Has maintainability been considered?
Auburn University
Review Checklist for Coding
• Is the design properly translated into code? (The
results of the procedural design should be available at
this review)
• Are there misspellings or typos?
• Has proper use of language conventions been made?
• Is there compliance with coding standards for language
style, comments, module prologue?
• Are incorrect or ambiguous comments present?
• Are typing and data declaration proper?
• Are physical constraints correct?
• Have all items on the design walkthrough checklist been
reapplied (as required)?
Auburn University
Software Testing (Test Plan)
• Have major test phases been properly identified and
sequenced?
• Has traceability to validation criteria/requirements been
established as part of software requirements analysis?
• Are major functions demonstrated early?
• Is the test plan consistent with the overall project plan?
• Has a test schedule been explicitly defined?
• Are test resources and tools identified and available?
• Has a test recordkeeping mechanism been established?
• Have test drivers and stubs been identified, and has
work to develop them been scheduled?
• Has stress testing for software been specified?
Auburn University
Software Testing
(Test Procedure)
• Have both white and black box tests been
specified?
• Have all independent logic paths been tested?
• Have test cases been identified and listed with
expected results?
• Is error handling to be tested?
• Are boundary values to be tested?
• Are timing and performance to be tested?
• Has acceptable variation from expected results
been specified?
Auburn University
Maintenance
• Have side effects associated with change been
considered?
• Has the request for change been documented,
evaluated, and approved?
• Has the change, once made, been documented
and reported to interested parties?
• Have appropriate FTRs been conducted?
• Has a final acceptance review been conducted
to assure that all software has been properly
updated, tested, and replaced?
Auburn University
Formal Technical Review
(FTR)
• Software quality assurance activity that is performed by
software engineering practitioners
– Uncover errors in function, logic, or implementation for
any representation of the software
– Verify that the software under review meets its
requirements
– Assure that the software has been represented according
to predefined standards
– Achieve software that is developed in a uniform manner
– Make projects more manageable
• FTR is actually a class of reviews
– Walkthroughs
– Inspections
– Round-robin reviews
– Other small group technical assessments of the software
Auburn University
The Review Meeting
• Constraints
– Between 3 and 5 people (typically) are involved
– Advance preparation should occur, but should involve no
more that 2 hours of work for each person
– Duration should be less than two hours
• Components
– Product - A component of software to be reviewed
– Producer - The individual who developed the product
– Review leader - Appointed by the project leader; evaluates
the product for readiness, generates copies of product
materials, and distributes them to 2 or 3 reviewers
– Reviewers - Spend between 1 and 2 hours reviewing the
product, making notes, and otherwise becoming familiar
with the work
– Recorder - The individual who records (in writing) all
important issues raised during the review
Auburn University
Review Reporting and
Recordkeeping
• Review Summary Report
– What was reviewed?
– Who reviewed it?
– What were the findings and conclusions?
• Review Issues List
– Identify the problem areas within the
product
– Serve as an action item checklist that
guides the producer as corrections are
made
Auburn University
Guidelines for FTR
• Review the product, not the producer
• Set an agenda and maintain it
• Limit debate and rebuttal
• Enunciate the problem areas, but don’t attempt to solve
every problem that is noted
• Take written notes
• Limit the number of participants and insist upon
advance preparation
• Develop a checklist for each product that is likely to be
reviewed
• Allocate resources and time schedules for FTRs
• Conduct meaningful training for all reviewers
• Review your earlier reviews (if any)
Auburn University
Reviewer’s Preparation
• Be sure that you understand the context of

the material
• Skim all product material to understand the
location and the format of information
• Read the product material and annotate a
hardcopy
• Pose your written comments as questions
• Avoid issues of style
• Inform the review leader if you cannot prepare
Auburn University
Results of the Review
Meeting
• All attendees of the FTR must make a decision
– Accept the product without further modification
– Reject the product due to severe errors (and perform
another review after corrections have been made)
– Accept the product provisionally (minor corrections
are needed, but no further reviews are required)
• A sign-off is completed, indicating
participation and concurrence with the review
team’s findings
Auburn University
Software Reliability
• Probability of failure-free operation for a

specified time in a specified environment.
• This could mean very different things for
different systems and different users.
• Informally, reliability is a measure of the
users’ perception of how well the software
provides the services they need.
– Not an objective measure
– Must be based on an operational profile
– Must consider that there are widely varying
consequences for different errors
Auburn University
IO Mapping
Subset of inputs
Input Set causing erroneous
outputs
Software
Output Set
Erroneous
outputs
Auburn University
Software Faults and Failures
• A failure corresponds to erroneous/unexpected runtime
behavior observed by a user.
• A fault is a static software characteristic that can cause a
failure to occur.
• The presence of a fault doesn’t necessarily imply the
occurrence of a failure.
Input Set
User A Erroneous
Inputs Inputs
User B User C
Inputs Inputs
Auburn University
Reliability Improvements
• Software reliability improves when faults

which are present in the most frequently used
portions of the software are removed.
• A removal of X% of faults doesn’t necessarily
mean an X% improvement in reliability.
• In a study by Mills et al. in 1987 removing
60% of faults resulted in a 3% improvement
in reliability.
• Removing faults with the most serious
consequences is the primary objective.
Auburn University

05 SQA Overview

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

05 SQA Overview

Uploaded by

Copyright:

Available Formats

Course Notes Set 5:

Software Quality Assurance

Computer Science and Software Engineering

• Simplistically, quality is an attribute of

[Adapted from Pressman 4th Ed]

Project A Project B Project C

[Adapted from Sommerville 5th Ed]

• Software will have fewer latent defects,

• It is difficult to institute in small

Errors Passed Through Percent Efficiency

Amplified Errors 1:X for error

Newly Generated Errors detection

[Adapted from Pressman 4th Ed]

• Be sure that you understand the context of

• Probability of failure-free operation for a

[Adapted from Sommerville 5th Ed]

• Software reliability improves when faults

You might also like