Systems Engineering Execution - Boeing

Strengthening Systems Engineering

Execution at Boeing …..
And Meeting Customer Needs

Dev A. Banerjee
Division Director – Systems and Flight Engineering (SAFE)
Boeing St. Louis

6th Annual Systems Engineering Conference

October 23, 2003

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 1 / October 22, 2010

 Systems Engineering Execution - Boeing

Outline

• Assessment of USAF for better SE

– Incentivizing Contractors (Excerpts from Dr. Sambur guidance)
– Mapping of USAF SE Elements to PMBP, CMMI and SEPM
– Joint DoD/Industry Suggested Actions

• Strengthening SE Execution at Boeing

– Establish Management Commitment
– Provide Infrastructure
– Monitor and Support Execution

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 2 / October 22, 2010

 Systems Engineering Execution - Boeing

Incentivizing Contractors for Better SE

Excerpts from Dr. Sambur Memo

• “An immediate transformation imperative for all our programs is to focus

more attention on the application of Systems Engineering (SE) principles
and practices throughout the system life cycle.”
• “A more robust SE environment can only be achieved through joint
cooperative efforts with our contractors.” “ SAF/AQ and all other MDAs will
not sign out any future ASPs that lack the necessary and sufficient
attention to SE.”
• “Additionally, we are identifying ways to improve SE throughout the
acquisition process, including workforce issues such as education and
training,; tools such as policies, instructions, and guidance; and institutions
such as the proposed Center of Excellence for SE under the Air Force
Institute of Technology (AFIT).”
• “---the Air Force leadership will take a strong and renewed interest in how our
acquisition community addresses SE disciplines and practices.”
Marvin Sambur
Assistant Secretary of the Air Force (Acquisition)
Policy Memo, dated April 09, 2003

USAF
USAFCommitted
Committedto
toRobust
RobustSE
SEPractice
Practicefor
forFuture
FutureAcquisitions
Acquisitions
\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 3 / October 22, 2010
 Systems Engineering Execution - Boeing
Mapping of Air Force SE Elements to PMBP, CMMI, and SEPM – existing pieces of our SE Infrastructure
Sambur SE Elements (1) Program Management CMMI Process Areas CMMI Process Area
- Examples - Best Practices to SEPM Section

Systems Requirements Document Requirements Management Requirements Management 2.2

System(s) Requirements Development Requirements Development 1.2, 1.3, 1.4
Conduct and thoroughness of reviews Use of Independent Reviews Verification 1.2, 1.5, 1.6
Validation 1.5, 1.6
Test and Evaluation Master Plan (TEMP)
Provides fully integrated engineering solutions Systems Integration (Proposed) Product Integration 1.5, 2.6

Effective SE interface control and management Interface Definition / Management (Proposed)

Trades Studies (Proposed) Decision Analysis and Resolution 1.4

Affordability Technical Solution 1.2, 1.3, 1.4, 1.5, 1.6

System Safety Plan System Safety

Risk Management Plan Risk Management Risk Management 1.4, 2.5

Configuration Management Plan Baseline Management Configuration Management 1.6, 2.6, 2.7

Systems Engineering Management Plan Organization Organizational Process Focus 3.1

Organizational Process Performance
Organizational Innovation and Deployment 3.3
Integrated Project Management (IPPD) 2.1, 2.2, 2.3
Integrated Teaming 2.1
Life cycle management w/production and retrofit strategic plans
Integrated Master Plan Integrated Planning Project Planning 2.1, 2.2, 2.6
Integrated Master Schedule Integrated Schedules

Earned Value Management Project Monitoring and Control 2.2, 2.5

Source Selection Plan Supplier Management Supplier Agreement Management 2.4

Integrated Supplier Management

System Maturity Matrix Technical Performance Measurement Measurement and Analysis 2.2

Closed Loop Corrective Action Causal Analysis and Resolution

Contractor Performance Assessment Rating System Management Information System Quantitative Project Management 2.1

Help Needed

All Program Management Best Practices Apply Organizational Process Definition 3.1
Organizational Training 3.2

Program Quality Plan Process and Product Quality Assurance 2.8

Logistics Management Plan Provide Support Organizational Environment for Integration 3.4

Customer informed of configuration changes and impacts Customer Communication/Contact Plan General Goals and Processes

Program Management Best Practice Suggested new Program Management Best Practice

Proposed new PMBP awaiting approval Quality and Mission Assurance Best Practice
(1) Source Memo:
"Incentivizing Contractors for Better Systems Engineering", dated 6 Jan 2003
from Dr. Marvin Sambur, Assistant Secretary of the Air Force for Acquisition

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 4 / October 22, 2010

 Systems Engineering Execution - Boeing

Suggested Actions from the Joint DoD / Industry

Meeting on SE – 11 April 2003

• SE needs to be considered across the Program Life Cycle

• Develop case studies on SE, with both good and bad program
examples
• Make use of disciplined assessment process to monitor rate and level
of SE implementation
• Recommend minimum or normal set of SE products or deliverables
• Create a consistent set of SE measures/metrics
• Define an appeal process and take corrective action if a program
deviates from the minimum level of SE needed for sound engineering
practice

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 5 / October 22, 2010

 Systems Engineering Execution - Boeing

Strengthening Systems Engineering Execution at Boeing

I. Establish
Management
Commitment
• Command Media II. Provide
• VP Engineering guidance Infrastructure
to Chief Engineers
• EIB & PM Council
direction to Bus • Comprehensive SE process III. Monitor
• Goals/Metrics assets (processes, tools, and Support
templates, etc)
• Training, SMEs Execution
• Management tools
• SE HILT activity across sites • Oversight by SE HILT
- SE review of proposals
and programs
• Disciplined assessment process
Corrective
Action • Metrics:
- Monitoring Rate
- Compliance Rate
\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 6 / October 22, 2010
 Systems Engineering Execution - Boeing

I. Establish Management Commitment

• Command Media
– “Systems Engineering Process Requirements”- in-work
– Systems Engineering Process Documents
– Systems Engineering Process Manual

• Leadership Endorsement / Direction

– VP Engineering advisory to site Chief Engineers emphasizing commitment to SE
– PM Council endorsement

• Goal
– Initiate effort to define goal/metric addressing SE execution

Leadership
LeadershipAdvisories
Advisoriesand
andGoals
Goalswill
willReinforce
ReinforceExpectations
Expectationsfor
for
SE
SEImplementation
Implementationacross
acrossBoeing
Boeing

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 7 / October 22, 2010

 Systems Engineering Execution - Boeing

II. Systems Engineering Infrastructure

• SE Process Assets, Management Tools

– The SEPM outlines detailed requirements for effective SE process
– CMMI to provide detailed set of process assets (process documentation, templates, etc)
– Program Management Best Practices (PMBPs) address some key aspects of systems engineering
– Gaps in Process Assets (e.g. handbooks in specialty areas) to be filled by SE Horizontal Integration
Leadership Team (HILT)

• Training, SMEs
– Considerable training in SE process exists throughout Boeing
– The SE HILT to consolidate SE Training
– CMMI compliant process assets in development

SE
SEHILT
HILTLeadership,
Leadership,coupled
coupledwith
withSite
SiteCMMI
CMMIactivities,
activities,
will
willensure
ensureaacomprehensive
comprehensiveSE SEinfrastructure
infrastructure

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 8 / October 22, 2010

 Systems Engineering Execution - Boeing

Boeing Systems Engineering HILT / Process Action Teams (PATs)

Huntington &
Anaheim Canoga Park El Segundo Florida Houston Seal Beach Huntsville Long Beach Mesa / Philly Puget Sound St. Louis Wichita

Systems
Engineering
HILT

Process Action Teams (PATs)

Network
Systems Human Operations System of Survivability /
Centric Info.
Engineering Affordability R,M&T Factors Safety Analysis Systems Vulnerability
& Comm.

Strengthen/Integrate
Strengthen/IntegrateSE
SEAcross
AcrossBoeing
Boeing

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 9 / October 22, 2010

 Systems Engineering Execution - Boeing
Available
SE Training • Info & Comm • SE Processes • UCLA Candidates
Arch • SoSE Process Ext

;
for Network

s
SE S t Teams
s

; CE
alist

Lds; s; IPT
Centric-Info • Networks • DoD Sys Acq • UC
& Comm • Info Process Irvine

yone

P Ms
peci
(Sample) Operations • Sys Arch Devel • AFCEA

E
ec

Sr. S
• Adv Tech • U of

Ever

Proj
ni s-
in /
r a ai n

ng
g

ai es
Source of Courses KS

ng
in
lT m

ni
Tr c
Certificate

d ro
ca D o

ai
Training

te P

Tr
Level Basic Inter Adv
ni al

la al

(Processes & Tools)

al
Training
ch rn

rn
Re ter
Te nte

te

# of Courses
In

Basic 4 6 7+

Ex
I

Advanced
Network Centric - Info & Comm Intermediate 2 4 4

Intermediate
SE Analysis & Control Advanced 1 2
By SE Discipline

Affordability
Training Topic

Formal Edu
R, M&T Basic - Graduate
- Workshops
Human System Interface
- Cross Trng
System Safety
SE Leader
Ops Analysis i ng
n l
System of Systems (SoS) r ai ve Exper (yrs) 4-8 8-13 13-18
T Le
Survivability/ Vulnerability
SE Training Database Typical Guidance
• SE Specialists and Technical Fellows more in-depth training in a selected domain.
• IPT Leaders and Program Managers more general training across the SE disciplines.
\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 10 / October 22, 2010
 Available Network Centric Training
Network
Centric (Sample)
- Communications Training
Topics
Network
Centric
Program: - Communications Training Progr
Internal Process
Related
- Training
Architectures Networks Information Operations SE Process
 Analog vs. Digital Comm  Internet  Information assurance Requirements Management
Risk Management
 Comm Spectrum  Interoperability  Information Warfare Trade Study Methodology
 SATCOM  Performance  C4ISR
System- of-Systems Systems Engineering Process
 Wireless Comm/  Protocols/controls  Information Protection Detailed System- of - Systems Process
Terrestrial/ Cellular Systems OO/UML Process
 Processing  Processing Techniques OO Modeling in UML
 Laser Com Techniques
 Advanced/evolving OO Analysis and Design
 Low Probability of  Vulnerabilities technologies SoSE Tool Kit
Intercept  Network protection  Information/data The DoD Systems Acquisition Process
 Evolving Technologies management System Architecture Development
 Anti- jam Techniques High Level Architecture for Simulation
DII - COE
 Performance C4ISR Architecture Framework Principles and Applications
In-house training plans focused in 2 areas:
domain/technology training and process training
Network Centric Communications Training Progra
External Training
Network
Centric
- Communications Training Program:
Universities / Associations / Gov. Labs / Agencies / Commercial Companies
Internal Domain / Technology
Related
- Training UCLACommunications;
Extension –Short Course Programs
Network Technology; Space Mission Analysis
Satellite Communications Payloads & Systems Design
Information & Communication Architectures High Frequency Satellite Communications
Satellite Communications Systems UC Irvine– Certificate Program
Open Architectures
Phased Array Antennas Communications & Network Engineering (18 units required)
Laser Communications Advanced Communications Systems Engineering (18 units required)
Software Architecture Armed Forces Communications & Electronics Association (AFCEA) Website courses:
RF Subsystem Design http://www.afcea.org/education/index.htm
Networks Aviation & Electronics Schools of America– individual short courses (3- 6 days)
Network - Centric Warfare Fiber Optics
Web Services Systems for RF Telecommunications
Network Technologies and Practices
Information Operations Defense Information Systems Agency (DISA) offers COE course
Government Information Technology Thrust University of Kansas -the Information and Telecommunication Technology Center:
Modulation Techniques Lawrence Livermore National Laboratory
Working with C4ISR Software/Commercial Companies offering related technology/software training:
Fundamentals of Digital Signal Processing Robelle, Sun Microsystems, Borland USA, Object Management Group, etc.
Time and Frequency Reference Concepts Industry Seminars
Advanced Technology Courses Cisco - Networking
Advanced Satellite Communications Payloads Alta Vista- - Search/MetaData
Satellite Network Architectures Topia - Intelligent Agents
Advanced
.
Fiber Optics Entrust - Security
 Systems Engineering Execution - Boeing

III. Monitor and Support Execution on Proposals /

Programs
1. Sites provide resources to execute SE and monitor execution.
2. SE HILT provide ‘integration mechanism across Boeing’
• Two Key Metrics to be monitored:
- “Monitoring Rate” Metric - “Implementation Level” Metric
• SE Evaluation Templates required for program reviews:
- proposal coverage - kickoff planning - program in-process reviews
3. SE HILT analyze metrics with sites; Jointly conduct root cause
analyses and develop Corrective Action Plans
4. SE HILT monitor implementation & results of corrective actions

••Effective
Effectivemonitoring
monitoring will
willstrengthen
strengthenthe
theexecution
execution
••Root
Rootcause
causeanalysis
analysis of
ofSystems
SystemsEngineering…
Engineering…
••Implementation
Implementationof ofcorrective
corrective consistent
consistentwith
withcustomer
customerdesires.
desires.
action
action

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 12 / October 22, 2010

 Systems Engineering Execution - Boeing

Conduct In-Progress Reviews on Programs

RFP
System Life Cycle

Pre-Concept /
Concept Preliminary Detail
Requirements Production Support
Definition Definition Definition
Development

Proposal Program SRR PDR CDR

Start Start

Ensure
EnsureSE
SEIntegrity
IntegrityThrough
Through Tailored
Tailored Reviews
Reviews
during
during Major
MajorProgram
ProgramMilestones
Milestones

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 13 / October 22, 2010

 Systems Engineering Execution - Boeing
The Metrics:
- “Monitoring Rate” Metric: Was the SE Process
Monitoring Activity conducted?
- Measure instances as % of opportunities.
Example: At each site, measure the number of proposals
reviewed for SE compliance at Gate C as a percentage of
the total number of Gate C proposals reviewed by the site.

- “Implementation Level” Metric: When the SE Process

Monitoring Activity was conducted, what was the level of
execution implementation?

Example: For each proposal reviewed, evaluate the level

of coverage of SE process execution.
\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 14 / October 22, 2010
 Systems Engineering Execution - Boeing

Boeing
Boeing is
is Committed
Committed to
to SE
SE
Execution
Execution across
across the
the Enterprise
Enterprise

\MUNRO\BANERJEE\PRESENTATIONS\SE_NDIA_23OCT03 15 / October 22, 2010