Quantitative Project Management

Agenda
• Metrics and Measurement
• Quantitative Project Management Cycle
• Objectives and Metrics 2014-15
• How To Capture Metrics
• Process Health Index as a Metric
• Risk RPN
“We cannot control what we
cannot measure”
 Metrics and Measurements
What to Measure? Costs &
Profitabilit
Project Constraints and success factors that need to y
be measured, monitored and controlled –
Quality
• Schedule
Schedule
• Scope
• Quality Measureme
• Cost nt Indexes
• Risks
• Resources Operation
al risks Scope

Resource
Why Measure?
 To provide us with objective data on quality and process performance
 Help us track progress and improvements over recurring releases
 Provide early insight into performance and enable early mitigations

When to Measure and how often?

As early as possible and as often as possible

Managing a Project without metric is like sailing a ship without a compass!

You will have no idea where you are going…..

 Key Metrics - Implementation and
Support
METRICS
Process Performance Measure Product Quality Measures

Implementation Support Implementation Support

• Schedule Variance • Mean Time To Respond • Defect Density • Defect Density

• Effort Variance • Mean Time to Resolve • Defect Removal • Defect Removal
• Requirement Volatility • % Defects Reproduced Efficiency Efficiency
• Review Efficiency • Effort Variance • Defect Rate • Defect Rate
• Review Effectiveness • Review Efficiency • Post Release Defects • Post Release Defects
• COQ • Review Effectiveness • Defect Distribution • Defect Distribution
• Variance Plan Vs Replan • CSAT • Defect Age • Defect Age
• CSAT

Metrics List with all available definitions is published in SunQMS

 Q&A
• How early can we identify project metrics?

• Let’s look at Metrics Definitions in the following scenarios –

1. I need to closely monitor my Schedule

2. I need to closely monitor my Product Quality

3. I need to closely monitor the efficiency of my Review Process

4. I need to closely monitor the most defect prone phase in my

Delivery Lifecycle
Managing Projects Quantitatively

Quantitative Project Management

is a mature Project Management Practice where
the project’s quality and performance objectives
are achieved based on Quantitative Techniques
Managing Projects Quantitatively

What?
Usage of defined Quantitative Techniques to manage a project and achieve its
established quality and process performance objectives

How?
Define clear measurable objectives, monitor the project against the objectives and
take timely actions to control deviations in expectations

Why?
• Objective and proactive decision making based on real time data enabling timely
corrective actions
• Standard definition across projects ensuring a rollup to map to review organizational
Quality and Process Performance

Who?
Project Manager
Establishing Objectives – Organization to Projects
Organization Objectives Client Objectives
Project Objectives

Define Business Goals

Define Project Metrics

Define Organization Metrics

Define Targets for Project Metrics

Define Targets for Org Metric

Organization

Project
Monitor Project using selected Metrics

Analyze Metrics across projects to measure

organizational performance
Report Metrics (Performance Results)

Analyze Performance Results periodically to know

Perform RCA
where we are

Plan and Implement Improvement Actions Take Actions

Repeat Cycle Report Metrics (Performance Results)

Objectives and Metrics – 2014-15

Business Aligned Goals Performance Improvement Plan

Objectives Processes Metrics Targets Measures

Reduce Project • Effort Variance EV <= 10%

Schedule Management • Schedule
Variance Variance SV < =5%
• Initial Vs Replan
SV I Vs R <= 5 %

Reduce Requirement • Review RE >= 95%

Requirement Management Effectiveness
Rapid Volatility • Requirement -
Delivery Volatility

Unit /Product • DRE DRE >= 90%

Testing • Defect
Reduce Identification
Defect Solution Testing Efficiency* -
Density
(Delivered) Non-Functional -
Testing

* Q2 onwards
Objectives and Metrics – 2014-15

Business Aligned Goals Performance Improvement Plan

Objectives Processes Metrics Targets Measures

Reduce TAT Defect • Review Efficiency -

for Defects Management • Review
(Internal) Effectiveness RE >= 95%

Reduce COQ Reviews • COQ COQ < 28%

• QA Health Index
• Trg Metrics QAI > 90%
Testing
QA -
Profitability Training
Unit /Product • DRE DRE >= 90%
Testing • Defect
Identification
Reduce Defect Solution Testing -
Efficiency*
Density
• Rework %*
(Delivered)
Non-Functional -
Testing

CSAT CSAT • CSAT Index

* Q2 onwards
Objectives and Metrics – 2014-15

Business Aligned Goals Processes Performance Improvement Plan

Objectives Metrics Targets Measures

Reduce Estimation • Peak Team Size Vs -

Variation to Baseline Team
Resource Size *
Planning

Project • Initial Vs Replan I Vs R <=5%

Management (Product)
• MTTR (Product)
Resource -
Utilization

Estimation • Effort Distribution -

Project Level Vs
Reduce Effort Baseline
Project -
Variance • Productivity *
Management

* Q2 onwards
Objectives and Metrics 2014-15

Metrics Organization Goal* Past Performance

Effort Variance <=10% 28.19%
Schedule Variance <=5% 37.60%

Review Effectiveness >=95% New Metric

Initial Vs Replan SV <=5% New Metric

Defect Removal Efficiency >=90% New Metric

COQ 28% 30%
Testing Efficiency TBD New Metric
SLA Compliance TBD
CSAT Score 80% - Long standing partnership
75% - Service Levels Good/Excellent

* Organization Goals are the Defined Organizational objectives as defined in the Strategic Annual Plan
 Q&A
• The Organizational Goal for Schedule Variance is <= 5%. However
your client doesn’t have any specifications for SV but wants a FTR
delivery. What will your project goal for SV be in this scenario?

• In the past 3 projects you have executed for ‘My Bank’, Quality has
been very unpredictable and numerous issues have been identified
during the SIT/UAT phases which have led to recurring release
cycles.

• Starting with a new project, you set clear Quality Goals of a DRE of
90%. However, since the processes remain the same you still see
unpredictability in achieving the goals.
What needs to be immediately done to ensure the goal is met?
Metrics Collection and Reporting
How to Capture Metrics
Metrics Metrics Applicable New / Existing Existing Data Source

Effort Variance Implementation / Support Existing Whizible - Project Module

Schedule Variance Implementation Existing Whizible - Project Module

Variance Plan Vs Replan Implementation New Whizible - Project Revision History

Requirements Volatility Implementation New Whizible - CR’s >200 PD, URS

Defect Removal Efficiency (DRE) Implementation / Support New Whizible (External) + Microbes (Internal)

Cost of Quality (COQ) Implementation / Support Existing Whizible

Review Effectiveness Implementation / Support Existing Whizible - Review Module

Review Efficiency Implementation / Support New Whizible - Review Module, Review Form

QA health index Implementation New SQA Score card

SLA Compliance Support Existing Whizible - SLA Management

CSAT index Implementation / Support New Survey

Defect identification efficiency (DIE) Implementation / Support New Whizible (External)+Microbes(Internal)

Rework % Implementation / Support New Whizible – Task Type – Rework / MPP

Peak team size Vs Baseline team size Implementation / Support New PMO

Productivity Implementation / Support New PMO

 Metrics Planning, Collection, M&C and
Reporting Steps
• Metrics are planned by PM during Project Planning
Metrics Planning

• Project Goals are set as per organizational and Project’s objectives

• Processes are strengthened further if the need be (in order to meet
critical to project goals)

• Project is monitored at milestones to identify early schedule or quality

Metrics Monitoring

deviations if any
• Early Deviations are reported and necessary actions taken to ensure no
impact on consecutive milestones

• Data for all Metrics are submitted through the Project Dashboard
Metrics Data and
Reports

• Reporting Frequency – Monthly

• Corrective Actions – at milestones

• Organizational Performance Baselines – Bi-Annual by QA

Perf
Org
SQA Scorecard
Process Health Index – SQA Score Card

What • Process Health Index provides an objective assessment of the process health, Quality Gates
and the key Project milestones

Why • Helps in identifying process deviations and risks and thus taking proactive measures in timely
corrective actions to ensure a healthy and defect free release and is as per plan

Who • SQA with the support from Project Manager

When • SQA Score Card executed Fortnightly or trigger based

Publish • Consolidated Health Index published Monthly starting April 2014

Process Health Index – Summary and Key Findings

Key Findings
• Pre initiation and knowledge transfer at
the time of project hand over needs
improvement
• Unavailability of the integrated plan
• Lack of project M&C
• Gap identification at later stage of
delivery
• Regular Beta version of code released
with limited testing
• Ineffective unit testing and
compromised solution testing and
testing knowledge gap
• Release Quality not streamlined
 Q&A
• The last project you led for ‘Your Bank’ closed
with a 25% Schedule Variance. The next phase
of the project is about to initiate and is
required to have 0% Schedule Variance.
As a Project Manager how will you plan and
monitor the project objectives to ensure Client
Delight?
Thank you