Verification and Validation Presentation (4!6!09)

Verification & Validation
Am I building the thing right?

Am I building the right thing?
by John W. Brand
4/6/2009 IIBA - Pittsburgh Chapter 1
Agenda
Define the terms

Discuss useful techniques
Outline risks

What’s the difference?

Interchangeable?
Verification and Validation have been used

interchangeably by many people.
However, they are distinctly different.

According to the IEEE
Verification is, “The process of evaluating a

system or component to determine whether
the products of a given development phase
satisfy the conditions imposed at the start of
that phase.“
Validation is, "The process of evaluating a
system or component during or at the end of
the development process to determine
whether it satisfies specified requirements."

According to Karl Wiegers
Verification determines whether the product

of a development activity meets the
requirements established for it.
Validation assesses whether a product
actually satisfies the customer needs.
Wiegers, Karl E., 2003, Software Requirements, 2nd Ed.

According to IIBA (1.6)
Verification ensures that requirements are

defined clearly enough to allow solution
design and implementation to begin.
Validation ensures that the stated
requirements correctly and fully implement
the business requirements.
IIBA, 2006, BABOK, Release 1.6

Requirements verification involves

evaluating requirements to verify that they
meet quality specifications.
Requirements validation ensures that all
requirements support the delivery of value to
the business, fulfill its goals and objectives,
and or meet a stakeholder need.
IIBA, 2008, BABOK, Release 2.0 (draft)

Verification
Ensure the requirements have been defined

correctly.
Determine that the requirements analysis has
been correctly performed.
Determine the requirements provide all the
information needed to develop the solution.
Determine that the requirements are ready
for formal review and validation by the
customer and users.

Why Verify?
Requirements that seem fine when you read

them in the requirements document might turn
out to have problems when developers try to
work with them.

Validation
Determines whether the requirements:

correctly align to the needs of the customer;
are accurate;
have the appropriate level of detail.

Differences
Requirements verification focuses on

quality:
Completeness,
Correctness,
Usability.
Requirements validation focuses on:
Supporting business goals/objectives,
Aligning with business goals/objectives,
Meeting stakeholder needs.

Differences
Requirements verification is building the

thing right.
Requirements validation is building the right
thing.

How do we do it?

Expected Outcome
Documented requirements that are:

unique,
well written,
unambiguous.
They are “good quality” requirements.

Good Quality Characteristics
Complete Prioritized
Consistent Traceable
Correct Unambiguous /
Does not say “how” Understandable
Feasible Testable
Mandatory/Necessary

Verification

Verification
Karl Wiegers has stated, “If a requirement

isn’t verifiable, determining whether it was
correctly implemented becomes a matter of
opinion, not objective analysis.”

Verification Process
Select 1 or more techniques.

Identify participants.
Execute the selected technique(s).
Update the requirements documentation.
Gottesdiener, Ellen, 2005, The Software Requirements Memory Jogger

Verification Techniques
Reviews
Desk Check
Pass Around
Walkthrough
Inspection
Checklists
Conceptual Testing

Verification - Reviews

Reviews
The problem is that people interpret

requirements in different ways.
To prevent this, you need to address 2
different issues:
Do we all interpret the requirement the same
way?
Are the requirements complete?
Cook, David A, 2002, “Verification and Validation of Requirements”, Presentation to USAF/STSC

Desk Check
An informal process.
A single colleague looks over the document
and models.
Returns comments to the requirements
author.

Pass Around
Several colleagues look over the document
and models.
The colleagues work concurrently and
independently.
Returns comments to the requirements
author.

Walkthrough
A group of colleagues are assembled in a
meeting.
Typically, the requirements author leads the
discussion.
The requirements document is reviewed with
the assembled group and comments are
solicited.

Inspection
May be an informal or a formal process.

Informal – requirements are inspected after
each JAD / Focus Group.
Formal – the entire requirements document is
inspected after it has been written.

Inspection
Small teams are assembled to inspect the

requirements:
Business Analyst
Developer
Tester
Include testers to make sure the
requirements are verifiable and can serve as
the basis for System Testing.
Team members inspect the document prior to
a formal meeting.

Inspection
Each requirement is read aloud in a paraphrased

form.
Potential defects and issues are pointed out by
meeting participants.
Defects and issues are formally recorded and
presented to the requirements author.
Meeting Roles:
Author
Moderator
Reader (an inspector)
Recorder
Inspectors

Checklists
Author’s Pre-Review Checklist

Requirements Checklist
Pre-Signoff Checklist

The standard template was used for all requirements

documentation.
The document is written in business language.
All requirements have been prioritized.
The documents were spell checked.
The document layout is neat and orderly.
Line numbering is active prior to printing.
All model components and lines are labeled.
Continuity among all models is intact:
Elements mentioned in 1 model are mentioned in the others;
Components referenced in multiple models use consistent terms.
All open issues are marked as TBD.

All implied requirements (performance, security) are stated.

All information to be displayed to the user are listed in the
requirements.
All triggers and outcomes have been accounted for.
All requirements are within Project Scope.
Future issues (upgrades, planned migration, long term use) have
been addressed in the requirements.

Organization and Completeness
Are all internal cross-references to other requirements correct?
Are all requirements written at a consistent and appropriate level of
detail?
Do the requirements provide an adequate basis for design?
Is the implementation priority of each requirement included?
Are all external hardware, software, and communication interfaces
defined?
Are algorithms intrinsic to the functional requirements defined?
Does the SRS include all the known customer or system needs?
Is any necessary information missing from a requirement? If so, is it
identified as a TBD?
Is the expected behavior documented for all anticipated error
conditions?

Correctness
Do any requirements conflict with or duplicate other requirements?
Is each requirement written in clear, concise, and unambiguous

language?
Is each requirement verifiable by testing, demonstration, review, or
analysis?
Is each requirement in scope for the project?
Is each requirement free from content and grammatical errors?
Can all the requirements be implemented within known constraints?
Are all specified error messages unique and meaningful?

Quality Attributes
Are all performance objectives properly specified?
Are all security and safety considerations properly specified?
Are other pertinent quality attribute goals explicitly documented and

quantified, with the acceptable trade-offs specified?
Traceability
Is each requirement uniquely and correctly identified?
Is each software functional requirement traced to a high-level

requirement?

Special Issues
Are all requirements actually requirements, not design or
implementation solutions?
Are the time-critical functions identified and their timing criteria
specified?
Have internationalization issues been adequately addressed?

Are the requirements complete? Do they cover all of the new
features needed in this release?
Are all the requirements SMART (specific, measurable, achievable,
relevant, time-bound)?
Are all requirements actual requirements, not design or
implementation solutions?
Are there any conflicts between some of the requirements?
Are there duplicate requirements?
Have all the requirements regarding the external hardware and
software and minimum system requirements been defined?
Have all the issues regarding internationalization been properly
addressed?
Have all non-functional requirements (quality attributes) been
properly specified?
Gantthead.com, 2007, Requirements Verification Checklist

Are there any requirements that specify how errors and/or

anticipated failure paths should be handled?
Are the non-requirements clearly identified? Does the document
clearly indicate which proposed requirements were dropped during
the review process and are not part of this release?
Are the requirements uniquely identified for traceability purposes?
Are the requirements prioritized?
Are all the internal and external references accurate?
Is the document free from grammatical errors?
Have all the TBDs been addressed?
Gantthead.com, 2007, Requirements Verification Checklist

Verification - Conceptual
Testing

Conceptual Testing
According to Boris Beizer, “The simple act of

designing test cases will reveal many problems
with the requirements even if you don’t execute
the tests on an operational system.”
Beizer, Boris, 1990, Software Testing Techniques, 2nd Ed.

Conceptual Testing
Karl Wiegers states, “Testing and requirements

have a synergistic relationship because they
represent complementary views of the system.”

V Model
Acceptance Testing is associated with Requirements Development.
The V model is a
software development
model that is based on
the relationships between Verification Validation
each phase of the
development life cycle as
described in a typical
Waterfall model of
and its associated phase
of testing.

Conceptual Testing
Plan your testing activities and begin

developing preliminary test cases during the
corresponding development phase.
Conceptual (implementation-independent)
test cases based on the requirements will
reveal errors, ambiguities and omissions in
your requirements and analysis models.

Conceptual Testing
You can begin deriving conceptual test cases from:

Use Cases
Requirements:
Business
User
Functional
Non-Functional
You can use these test cases to evaluate textural
requirements, analysis models and prototypes.

Conceptual Testing
Writing test cases from the requirements:

reveals ambiguities,
reveals vagueness,
verifies correctness.

Conceptual Testing
Writing black box test cases crystallizes your

vision of how the system should behave
under certain conditions.
Vague and ambiguous requirements will jump
out at you because you won’t be able to
describe the expected system response.
These conceptual (abstract) test cases are
independent of implementation.

Validation

Validation
Validation typically occurs twice:

During requirements development, but after
verification,
After coding.
The best way to validate requirements is to
involve customers in the process.
End-users are the most effective.
Cook, David A, 2002, “Verification and Validation of Requirements”, Presentation to USAF/STSC

Validation
Look for these documentation principles:

Requirements must be realistic,
Functionality must be defined,
Behavior must be represented.
Hass, Kathleen, et.al., 2008, Getting It Right: Business Requirement Analysis Tools and Techniques

Validation Process
Select 1 or more techniques.

Identify participants.
Execute the selected technique(s).
Update the requirements documentation.

Validation Techniques
Reviews
Walkthrough
Inspection
Checklists
Conceptual Testing
Acceptance Criteria
Model Walkthrough
Prototyping

Validation - Reviews

Walkthrough
A group of colleagues are assembled in a
meeting.
The requirements document is reviewed with
the assembled group and comments are
solicited.
Typically, the requirements author leads the
discussion.

Inspection
May be an informal or a formal process.

Informal – requirements are inspected after
each JAD / Focus Group.
Formal – the entire requirements document is
inspected after it has been written.

Inspection
Small teams are assembled to inspect the

requirements:
Customer
Subject Matter Experts
Developer
Include end-users to make sure the
requirements are verifiable and can serve as
the basis for User Acceptance Testing.
Team members inspect the document prior to
a formal meeting.

Inspection
Each requirement is read aloud in a paraphrased

form.
Potential defects and issues are pointed out by
meeting participants.
Defects and issues are formally recorded and
presented to the requirements author.
Meeting Roles:
Author
Moderator
Reader (an inspector)
Recorder
Inspectors

Checklists
Customer developed.
Custom to the project.
Lists of features and functions.

Checklists – General Characteristics
All conditions under which a requirement applies are stated.
Each requirement must accurately describe the functionality to
be built.
The requirement can be tested to ensure that it meets measures
of success.
The requirement is traceable to a goal stated in a project
initiating document, such as:
Project Charter
Business Case
Project Scope
The requirement is necessary to enable the solution to meet the

business goals and objectives.
The requirement is traceable back to a specific customer or user.
The requirement, as stated, has only one interpretation.
Hass, Kathleen, et.al., 2008, Getting It Right: Business Requirement Analysis Tools and Techniques

Validation – Conceptual
Testing

Conceptual Testing
Karl Wiegers states, “Testing and requirements

have a synergistic relationship because they
represent complementary views of the system.”

V Model
Acceptance Testing is associated with Requirements Development.
The V model is a
model that is based on
the relationships between Verification Validation
each phase of the
development life cycle as
described in a typical
Waterfall model of
and its associated phase
of testing.

Conceptual Testing
You can begin deriving conceptual test cases

from:
Use Cases
Requirements:
Business
User
Functional
You can use these test cases to evaluate
textural requirements, analysis models and
prototypes.

Acceptance Criteria
“Acceptance criteria – and hence acceptance testing

– should evaluate whether the product satisfies its
documented requirements and whether it is fit for
use in the intended operating environment.”
“It’s a shift in perspective from the requirements
elicitation question of ‘What do you need to do with
the system?’ to ‘How would you judge whether the
system satisfies your needs?’.”
“If the customer can’t express how they would
evaluate the system’s satisfaction or a particular
requirement, that requirement is not stated
sufficiently clearly.”

Acceptance Criteria
“Define the conditions for accepting the

system.”
“Guides users to more explicitly describe how
they expect the software to work.”
Identifies:
Functionality (functional requirements),
Quality attributes (non-functional requirements),
Expected Results from a defined set of inputs.
“How you will judge whether the system satisfies
your needs.”

Acceptance Criteria
User involvement is critical.

Criteria are often based on:
the user’s ability to accomplish specific tasks and,
the system’s ability to meet certain quality
attributes.
Every criterion should have 1 or more
Acceptance Test Cases.

Acceptance Testing
Sources:
Use Cases,
Requirements Analysis Models:
Use Case Models,
Business Process Models,
Data Flow Models.
Focus on anticipated usage scenarios.
Consider the:
most commonly used Use Cases (normal flow),
most important Use Cases.
De-emphasizes:
Alternate Flows,
Exception Condition Handling.

Model Walkthrough
Uses test simulations to step through

analysis models.
Simulates system operation without actually
testing code.
Why?
Find missing steps,
Find missing data,
Find missing business rules.

Model Walkthrough
Identify and create test cases.

Select the analysis model(s) to validate.
Trace the test cases through the model(s) in
a step-by-step manner.
Correct the requirements model(s), as
necessary.

Validation – Prototyping

Prototyping - What
A mock-up of a new system.

Brings Use Cases to life.
Closes gaps in understanding of
requirements.
Helps the stakeholders arrive at a shared
understanding of the system’s requirements.

Prototyping - What
Assesses the feasibility of quality attributes.

Detects unnecessary functionality.
Detects missing steps.
Exercises interfaces.
Reveals missing, erroneous and infeasible
requirements.

Prototyping – How To
Determine which requirements to validate

using a prototype.
Develop the prototype.
Evaluate the prototype.

Validation Techniques - Summary
Reviews
Walkthrough
Inspection
Checklists
Conceptual Testing
Acceptance Criteria
Model Walkthrough
Prototyping

Validation
Writing test cases from the requirements:

reveals vagueness,
Executing prototypes:
reveals vagueness,

Validation
Writing black box test cases crystallizes your

vision of how the system should behave
under certain conditions.
Vague and ambiguous requirements will jump
out at you because you won’t be able to
describe the expected system response.
These conceptual (abstract) test cases are
independent of implementation.

Risks

Risks
“Studies have shown that it can cost

approximately 100 times more to correct a
customer-reported requirement defect than to
correct an error found during requirements
development.”
“Any measures you can take to detect errors
in the requirements specifications will save
you substantial time and money.”

Summary

Verification ensures that requirements are

defined clearly enough to allow solution
design and implementation to begin.
Validation ensures that the stated
requirements correctly and fully implement
the business requirements.

Differences
Requirements verification is building the

thing right.
Requirements validation is building the right
thing.

Questions?

References
Cook, David A., 2002, “Verification & Validation of Requirements”, Software Technology
Support Center. Online at www.sstc-
online.org/Proceedings/2002/SpkrPDFS/ThrTracs/p961.pdf (Accessed December 2008).
Gantthead.com, 2007, “Requirements Verification Checklist”. Online at
www.gantthead.com/checklists/Requirements-Verification-Checklist.html (Accessed
December 2008).
Gottesdiener, Ellen, 2005, The Software Requirements Memory Jogger. Salem, NH:
GOAL/QPC.
Hass, Kathleen B., Don J. Wessels, and Kevin Brennan, 2008, Getting It Right: Business
Requirement Analysis Tools and Techniques. Vienna, VA: Management Concepts, Inc.
International Institute of Business Analysis, 2006, A Guide to the Business Analysis Body
of Knowledge, Version 1.6. Toronto, ON. International Institute of Business Analysis.
International Institute of Business Analysis, 2008, The Guide to the Business Analysis
Body of Knowledge, Version 2.0 – Draft for Public Review. Toronto, ON. International
Institute of Business Analysis.
Wahab, Sammy, “Can I Have My Requirements and Test Them Too?”, Business Analyst
Times. Online at www.batimes.com/content/view/321/1/ (Accessed December 2008).
Wiegers, Karl E., 2003, Software Requirements, Second Edition. Redmond, WA: Microsoft
Press.

Verification and Validation Presentation (4!6!09)

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Verification and Validation Presentation (4!6!09)

Uploaded by

Copyright:

Available Formats

Verification & Validation

Am I building the thing right?

Define the terms

4/6/2009 IIBA - Pittsburgh Chapter 2

4/6/2009 IIBA - Pittsburgh Chapter 3

Verification and Validation have been used

4/6/2009 IIBA - Pittsburgh Chapter 4

Verification is, “The process of evaluating a

4/6/2009 IIBA - Pittsburgh Chapter 5

Verification determines whether the product

Wiegers, Karl E., 2003, Software Requirements, 2nd Ed.

4/6/2009 IIBA - Pittsburgh Chapter 6

Verification ensures that requirements are

IIBA, 2006, BABOK, Release 1.6

4/6/2009 IIBA - Pittsburgh Chapter 7

Requirements verification involves

IIBA, 2008, BABOK, Release 2.0 (draft)

4/6/2009 IIBA - Pittsburgh Chapter 8

Ensure the requirements have been defined

4/6/2009 IIBA - Pittsburgh Chapter 9

Requirements that seem fine when you read

Wiegers, Karl E., 2003, Software Requirements, 2nd Ed.

4/6/2009 IIBA - Pittsburgh Chapter 10

Determines whether the requirements:

have the appropriate level of detail.

IIBA, 2008, BABOK, Release 2.0 (draft)

4/6/2009 IIBA - Pittsburgh Chapter 11

Requirements verification focuses on

4/6/2009 IIBA - Pittsburgh Chapter 12

Requirements verification is building the

4/6/2009 IIBA - Pittsburgh Chapter 13

4/6/2009 IIBA - Pittsburgh Chapter 14

Documented requirements that are:

They are “good quality” requirements.

IIBA, 2006, BABOK, Release 1.6

4/6/2009 IIBA - Pittsburgh Chapter 15

IIBA, 2006, BABOK, Release 1.6

4/6/2009 IIBA - Pittsburgh Chapter 16

4/6/2009 IIBA - Pittsburgh Chapter 17

Karl Wiegers has stated, “If a requirement

4/6/2009 IIBA - Pittsburgh Chapter 18

Select 1 or more techniques.

Gottesdiener, Ellen, 2005, The Software Requirements Memory Jogger

4/6/2009 IIBA - Pittsburgh Chapter 19

4/6/2009 IIBA - Pittsburgh Chapter 20

4/6/2009 IIBA - Pittsburgh Chapter 21

The problem is that people interpret

Cook, David A, 2002, “Verification and Validation of Requirements”, Presentation to USAF/STSC

4/6/2009 IIBA - Pittsburgh Chapter 22

Wiegers, Karl E., 2003, Software Requirements, 2nd Ed.

4/6/2009 IIBA - Pittsburgh Chapter 23

Wiegers, Karl E., 2003, Software Requirements, 2nd Ed.

4/6/2009 IIBA - Pittsburgh Chapter 24

4/6/2009 IIBA - Pittsburgh Chapter 25

May be an informal or a formal process.

Wiegers, Karl E., 2003, Software Requirements, 2nd Ed.

4/6/2009 IIBA - Pittsburgh Chapter 26

Small teams are assembled to inspect the

4/6/2009 IIBA - Pittsburgh Chapter 27

Each requirement is read aloud in a paraphrased

4/6/2009 IIBA - Pittsburgh Chapter 28

Author’s Pre-Review Checklist

4/6/2009 IIBA - Pittsburgh Chapter 29