Applying Earned Value

Management to Project
Formulation
A Survey of Policy and
Strategy for Execution
Sam Padgett
 Agenda
Š The Question
Š Why Do We Care?
Š Policy
Š Common Sense
Š Terminology
Š Conclusions
Š Is it Practical (Feasible)?
Š If so, Is it Practicable (Benefit > Cost)?
Š What are the Relevant Considerations or ?
 The Question
Š “Should We Be Doing EVM In Project
Formulation?”
Š What Do Typical Project-centric Entities (Agencies
And Professional Associations) Require In Terms
Of Policy? (Did Not Survey Individual Companies)
Š What Structures Can Serve As A Viable Framework
For Formulation-phase Baselines?
Š What Considerations Or Attributes Make A
Formulation Effort Amenable To EVM?
What Do We Mean By “Formulation”?

Š Generally, That Project Effort Leading Up To PDR

That Precedes “Implementation”
Š Defined In NPR 7120.5d

QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
 What Do We Mean By “Formulation”?

Pre-Phase A Phase A Phase B Phase C Phase D Phase E

NASA Advanced Preliminary Definition Design Develop- Operations
Studies Analysis ment

Formulation Implementation

*INCOSE SE Handbook, versions 2b and 3

What Does The Survey Of
Policy Say?
Š Sources Consulted
Š OMB (Circular A-11)
Š DoD
ŠŠ Directive
Directive 5000.1
5000.1
ŠŠ Instruction
Instruction 5000.2
5000.2
ŠŠ Defense
Defense Acquisition
Acquisition Guidebook
Guidebook
Š NASA – NPR 7120.5
Š DOE
Š PMI
Š INCOSE
Š SEI
 Policy: OMB Circular A-11
Š Yes, limited EVM
Š EVM for prototypes during planning phase
Š Development, Modernization, Enhancement (DME)
Š “EVMS is required for those parts of the investment where
developmental effort is required. This includes prototypes
and tests to select the most cost effective alternative during
the Planning Phase, the work during the Acquisition Phase,
and any developmental, modification or upgrade work done
during the Operational/Steady State Phase.”
Š “300.7 What must I report on exhibit 300 and when?
…Where prototypes are acquired as part of the planning
process, the prototypes must be reported as full
acquisitions.”
 Policy: NDIA
PMSC EVMS Application Guide*
ŠŠ Yes
Yes but,
but,
ŠŠ Somewhat
Somewhat ambiguous
ambiguous
ŠŠ “…planning
“…planning the the Earned
Earned Value
Value
Management
Management (EVM) (EVM)
application
application through all
through all phases
phases
of
of the
the acquisition
acquisition life
life cycle”
cycle”
ŠŠ 4.1.2
4.1.2 …For
…For example,
example,
work
work done during a planning
done during a planning
phase
phase toto develop
develop aa prototype
prototype oror
test
test article
article lends
lends itself
itself to
to
performance management
performance management
using
using EVM.
EVM.
ŠŠ 4.3.3
4.3.3 Pre-Systems
Pre-Systems Acquisition
Acquisition
–
Planning – Does not address
Planning Does not address
application
application of of EVM
EVM in in this
this
program
program phase
phase butbut rather
rather
describes
describes the process of
the process of
establishing,
establishing, evaluating
evaluating andand
approving
approving thethe system.
system.
*3/07 Working Release
 Policy: DoD
Š Defense Acquisition Guidebook
Š No Requirement
Š No Exclusion
Š Application is no longer related to appropriation.
Š “The requirement for EVM applies to cost or incentive
contracts, subcontracts, intra-government work
agreements, and other agreements that meet the dollar
thresholds prescribed in USD(AT&L) policy
memorandum dated March 7, 2005”
Š EVM Policy Letter 3-7-05 Doesn’t Discuss Application
With Respect To Phase, Only To Agreement Types
(Contract, Subcontracts, Intra-government Work
Agreements, Etc.)
 Policy: DoD
Š Defense Acquisition Guidebook
Š Somewhat Ambiguous As To Phase: “The
program manager should implement EVM on
applicable contracts within acquisition,
upgrade, modification, or materiel
maintenance programs,…”

QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
 Policy: NASA
Š Program And Project Management
Requirements (NPR 7120.5D)
Š Requires Application Of EVM Only To
“Implementation Phase” Project Work
Š Not Required Prior To Key Decision Point C
(Roughly PDR) Or For Operations
 Policy: No Mention
Š EIA-748-B
Š The Federal Acquisition Certification For Program And
Project Managers (FAC-P/PM) April 25, 2007
Š Dod - Integrating-se-acquisition-
contracts_guide_121106.Doc
Š Understanding And Leveraging A Supplier’s CMMI ®
Efforts: A Guidebook For Acquirers CMMI Guidebook
For Acquirers Team March 2007 TECHNICAL
REPORT CMU/SEI-2007-TR-004 ESC-TR-2007-004
Summary of Policy Regarding EVM in
Formulation
Š OMB - Yes, If It’s DME
Š Implementers
Š Somewhat Ambiguous in Writings
Š Usually Neither Discourages Nor Requires
Š May Well Differ From Facts On The Ground
 Why Do We Care?
Š The Formulation of Large Projects May
Š Cost Much
Š Take Several Years
Š Have Complex Internal Interdependencies
Š If the Completion Criteria for Formulation are Clear,
Progress Assessment May Not Be Optional
Š A Viable Preliminary Design
Š A Complete Risk Register
Š A Date Certain
If We ’ve Really Thought Hard About It
We’ve
And Decided We Really Need EVM …
EVM…
 What Do We Use For A
WBS?
Š In Implementation Phase It’s Straightforward;
Š Product-oriented Prime Mission Equipment Branch
Š Common Elements For Each PME Node
Š But In Formulation Phase It May Not Be So
Straightforward
Š There May Be Multiple Competing Architectures
Until The Winning Concept Is Chosen
Š The Prime Mission Equipment Branch Is Only
Specified As The Culmination Of Formulation
Š During Formulation, There May Not Be A Single
PME Tree On Which To “Hang” Common Elements
 What Course To Choose?
Š The Simplest Approach
Š Assume Formulation Effort Is All Part Of An Undifferentiated
PM&SE Node At The Project Level
Š Not Well Suited To Quantifying Progress
Š Another Approach
Š Create A WBS Peculiar To The Task At Hand
Š Based On The Systems Engineering Products That Dominate
Formulation
Š Provides For The Application Of Technical Processes To
Concrete Products
 But What About Earlier Work?

Š Many Programs Spend Lots Of Money Before Development

Begins
ŠŠ Concept
Concept Exploration
Exploration And
And Refinement
Refinement
ŠŠ Low-trl
Low-trl Technology
Technology Evaluation
Evaluation And
And Development
Development
Š It May Be Internal To The Customer (Not Yet On Contract)
Š There’s No Accepted WBS For This Kind Of Work
ŠŠ Not
Not Addressed
Addressed By
By MIL-HDBK-881A
MIL-HDBK-881A
ŠŠ The
The Architecture
Architecture Of
Of The
The Potential
Potential System
System IsIs Purely
Purely
Academic/Imaginary
Academic/Imaginary -- During
During Such
Such Formulation
Formulation Activities,
Activities, ItIt Is
Is
Just
Just A
A Node
Node In
In The
The Cost
Cost Estimate
Estimate -- IF
IF THE
THE SYSTEM
SYSTEM WERE
WERE TO TO
EVER
EVER BEBE BUILT!
BUILT!
Š But - Decision-makers Frequently Still Want To Apply
Schedule And Cost Targets To Manage The Studies
 An Example:
Formulation-phase End Product
Definitions
Concept Refinement Phase

Defense Acquisition Guidebook, Fig 4.3.1.3.1

 Most Results Captured In:
The Analysis Of Alternatives (AoA)
ŠŠ Introduction
Introduction ŠŠ Effectiveness
Effectiveness Analysis
Analysis
ŠŠ Background
Background ŠŠ Effectiveness
Effectiveness Methodology
Methodology
ŠŠ Purpose
Purpose ŠŠ Models,
Models, Simulations, And
Simulations, And Data
Data
ŠŠ Scope
Scope ŠŠ Effectiveness
Effectiveness Sensitivity
Sensitivity Analysis
Analysis
ŠŠ Ground
Ground Rules
Rules ŠŠ Cost
Cost Analysis
Analysis
ŠŠ Scenarios
Scenarios ŠŠ Lifecycle
Lifecycle Cost
Cost Methodology
Methodology
ŠŠ Threats
Threats ŠŠ Models
Models And
And Data
Data
ŠŠ Environment
Environment ŠŠ Cost
Cost Sensitivity
Sensitivity And/Or
And/Or Risk
Risk Analysis
Analysis
ŠŠ Constraints
Constraints And
And Assumptions
Assumptions ŠŠ Cost-effectiveness
Cost-effectiveness Comparisons
Comparisons
ŠŠ Alternatives
Alternatives ŠŠ Cost-effectiveness
Cost-effectiveness Methodology
Methodology
ŠŠ Description
Description Of
Of Alternatives
Alternatives ŠŠ Displays
Displays Or Presentation Formats
Or Presentation Formats
ŠŠ Nonviable Alternatives
Nonviable Alternatives ŠŠ Criteria For Screening Alternatives
Criteria For Screening Alternatives
ŠŠ Operations
Operations Concepts
Concepts ŠŠ Organization
Organization And
And Management
Management
ŠŠ Support
Support Concepts
Concepts ŠŠ Study
Study Team/Organization
Team/Organization
ŠŠ Determination
Determination Of
Of Effectiveness
Effectiveness Measures
Measures ŠŠ Aoa
Aoa Review
Review Process
Process
ŠŠ Mission
Mission Tasks
Tasks ŠŠ Schedule
Schedule
ŠŠ Measures
Measures Of
Of Effectiveness
Effectiveness
ŠŠ Measures
Measures Of
Of Performance
Performance

•Briefings
•Final Report Defense Acquisition Guidebook, 3.3.1
 Completing Work Definitions
Š Technical Processes Should Be Technical Processes
Applied To Each End Product In Requirements
The WBS Hierarchy Development
Š Those Processes Require That Logical Analysis
Tasks Be Performed Specifically Design Solution
For Each WBS Element Implementation
Š Perform Requirements Allocation Integration
Š Prepare Subsystem Spec Verification
Š Perform Component Testing Validation
Š Issue Purchase Order Transition
Š The Process Applied To Each (Defense Acq. Guidebook, 4.2.2)
End Product Element (WBS)
Represents The Work Definition
 Candidates For
Formulation-phase Common
Element Definitions
Š The Draft Acquisition Plan
Š The Draft Systems Engineering Plan
Š Others
Š Will Make Up The Equivalent Of The
Common Elements Task Definitions For
Concept Refinement
 In Total,
Š It’s good practice to “…derive schedule and
cost baselines from normal systems
engineering products.
Š That will likely improve the quality of the
PMB and
Š increase acceptance and reduce
reluctance by the systems engineering
community”
Š Normal systems engineering products can
provide useful structures for assessing
physical progress
 If You Decide You Should Do
EVM in Formulation…
Š What considerations or attributes make it
worthwhile?
Š What Considerations make it “doable”?
 What Considerations Or Attributes
Make Formulation-phase EVM Worth
Considering?
Š Period Of Performance – ~Authorization To PDR –
Š More Than One Year?
Š Hard Date For Pdr (Implementation Authorization)
Š Complexity
Š Multiple Business Entities
Š Multiple Geographic Locations
Š Unfamiliar Cultures
Š Multiple Immature Technologies
Š Visibility To Important Stakeholders
Š …
 What Considerations Make It “Doable”?

Š A Well-defined Life Cycle

Š A Well-defined Process
Š An Existing EVMS
Š …
 Note On Simplification
Š This Example Focused On Policy And
Guidance Of The Department Of Defense
Š That Is An Environment That’s Familiar To Many
Of Us
Š Significant Useful Material Exists
Š It’s Not Materially Different From Policy And
Guidance Of Other Major System Acquirers
(Commercial Entities, DOE, NASA, Etc.)
 Sources Consulted
Š EIA-632, Processes for Engineering a System, 1999
Š ISO-15288, Systems Engineering - System Life Cycle Processes, 2002
Š INCOSE SE HB, 2004 and 2006
Š DODD 5000.1, The Defense Acquisition System
Š DODI 5000.2, Operation of the Defense Acquisition System
Š Defense Acquisition Guidebook, 2004
Š Fundamentals of Systems Engineering, DAU, 2001
Š MIL-HDBK-881A, 2005
Š MIL-STD-499A, Engineering Management, 1974
Questions: