Module 4

Software Metrics

SQAM Course, Proficience, IISc 1

When you can measure what you are
speaking about and express it in
numbers, you know something about it;
but when you cannot measure, when
you cannot express it in numbers, your
knowledge is of a meager and
unsatisfactory kind; it may be the
beginning of knowledge, but you have
scarcely, in your thoughts, advanced to
the stage of science – Lord Kelvin

You cannot control what you cannot

measure - DeMarco
SQAM Course, Proficience, IISc 2
 Measurement
 Economic measurements determine price
and pay increases
 Measurements in radar systems enable us
to detect air craft
 Measurements enable doctors to diagnose
specific illnesses
 Measurements in atmospheric systems
basis for weather prediction

SQAM Course, Proficience, IISc 3

 Measurement
A process by which numbers or
symbols are assigned to attributes of
entities in the real world in such a
way as to describe them according to
clearly defined rules.

SQAM Course, Proficience, IISc 4

 Measurement helps..
 Understand about software products,
processes and services
 Evaluate against the established
standards and goals
 Control resources and processes
 Predicts the attributes of software in
future

SQAM Course, Proficience, IISc 5

Information, Data & Metrics

SQAM Course, Proficience, IISc 6

Measurement at different
levels

SQAM Course, Proficience, IISc 7

Use of Measurement at different levels

SQAM Course, Proficience, IISc 8

 Measurement
Characteristics
 Simple and robust
 Be a part of the process
 Repeatable
 Easy to measure
 Non threatening
 Non evaluative

SQAM Course, Proficience, IISc 9

 Steps to design a metric
 Identify metrics customers
 Target goals
 Ask questions
 Select metrics
 Standardize definitions
 Choose a measurement function
 Establish a measurement method
 Define a decision criteria
 Define reporting mechanisms
 Determine additional qualifiers

SQAM Course, Proficience, IISc 10

 Goal Question Metric

Goal 1 Goal 2

Q1 Q2 Q3 Q4

M1 M2 M3 M4 M5 M6 M7 M8

SQAM Course, Proficience, IISc 11

 GQM - Example
Goal: Reduce maintenance costs by 50% within a year
Questions:
- How much do we spend each month?
- What is the fraction of cost on maintaining each
application?
- How much money do we spend on corrective, adaptive and
preventive maintenance?
Metrics:
- Total Budget on maintenance
- Hours spent on for maintenance for each application
- Hours spent on for every type of maintenance and cost
involved
One more example..
SQAM Course, Proficience, IISc 12
 Class work..
 Improve Schedule Estimation
accuracy by 10% within 6 months
 Reduce system testing defects for
the next 3 development projects
 Reduce time to close a defect by
40% within 3 months

SQAM Course, Proficience, IISc 13

 Metrics Examples
Product size: Kloc, FP, Object classes, Screens,
Interface points, etc.
Effort and Schedule: Time sheets
Cost: Cost of Quality, Rework
Reviews: Preparation rate, Review Rate, Defect
Density
Planning: Effort variance, schedule variance,
Productivity
Requirements:Requirements Volatility
Defects: Severity, Defect Density, Defect Removal
Effectiveness, Defect Distribution, etc
Maintenance: Turn around time, Backlog
Management Index
SQAM Course, Proficience, IISc 14
Software Measurement Sins
 Lack of management commitment – Educate!
 Measuring too much, too soon – Start simple and balanced
 Measuring too little, too late – Measure the aspects
together!
 Measuring the wrong things – Right audience!
 Imprecise metrics definitions – Standardize!
 Use the data for evaluating individuals – Never!
 Use metrics to motivate, than understand – Focus on results
 Collecting data that is not used – Inform or remove!
 Lack of communication and training – Handbook, website,
short meet
 Misinterpreting metrics data – Do not over react!

SQAM Course, Proficience, IISc 15

 Establishing a measurement program

Problems to be tackled are

•         lack of management commitment
•         imprecise or inappropriate objectives
•         unclear linkage to business and management
goals
•         differing management and staff expectations
•         assessment criteria not defined
•         lack of feedback
•         concerns about misuse of information
•         cultural change needed
•         willingness to continue in the longer term.

SQAM Course, Proficience, IISc 16

Establishing a measurement program-
Step1
1. Appoint an executive sponsor – committed,
dynamic, strong motivator, sets hard and
realistic targets
2. Establish scope and goals – Examples
•         identify where changes can improve
productivity
•         assess the effectiveness of reviews and
testing
•         determine the effort spent on rework
•         identify trends in defects

SQAM Course, Proficience, IISc 17

Establishing a measurement program-Step 3

3. Define Key Performance Indicators, measures, models –

Productivity
Function Points per person month
Quality
percentage effort spent on rework
 percentage effort spent on reviews
 effectiveness of reviews
 effectiveness of systems testing
 average effort to correct defects found during development
 percentage bad fixes during development
Estimates
Estimated vs. actual effort (variance)
 Estimated vs. actual timescale (variance)

SQAM Course, Proficience, IISc 18

Establishing a measurement program-Step 4

4. Establish a data collection Infrastructure

•         who collects data?
•         when is it collected?
•         where is it collected from?
•         how is it collected and validated?
•         who analyses and presents the results?

SQAM Course, Proficience, IISc 19

Establishing a measurement program-Step 5

•  5. Explain the Program:

      Workshop for managers
Workshop for project teams and
developers
Training for staff in measurement and
analysis
Questions to be answered:
•         Do teams understand the purpose of the
program?
•         Do teams support the program?
•    Do teams know what is expected of them?
SQAM Course, Proficience, IISc 20
Establishing a measurement program-Step 6

6. Devise success criteria – Periodic review

•        Is the right information being generated at
the
right time?
•       What is the quality of the information?
•         Have baselines and targets been set for each
KPI?
•         Have we got quantifiable information to aid
decision making?
•        Has information been acted upon?
•        Is it understood when to act on the
information? SQAM Course, Proficience, IISc 21
Establishing a measurement program-Step 7

7. Set baselines and targets for KPI

•   Identify when projects provide data and when data is
assessed
•   Establish annual (or more frequent) baselines and targets,
derived from quantifiable information
•   Assess performance against baselines and targets
•   Use data generated in first 3-6 months to set initial
baselines
•   Base initial targets on achievements of known
improvement programs
•   Review and revise baselines and targets when measured
information becomes available

SQAM Course, Proficience, IISc 22

Establishing a measurement program-Step 8

8. Develop a Feedback mechanism

• Use effective methods of presenting and
reporting results
• Dash boards – Green, Yellow, Red

SQAM Course, Proficience, IISc 23

 Metrics for SME
 Keep it simple
 Customize to local needs
 Be sure that it adds value
 Schedule
 Effort
 Defect Removal Effectiveness

SQAM Course, Proficience, IISc 24

 Cost of Software Metrics
 For small groups, initial cost is 3-8%
then drops to less than 1% after
familiarity
 In major organizations, initial cost is
relatively high then it drops to less
than 5%.

Source: Software Metrics, Fenton & Fleeger

SQAM Course, Proficience, IISc 25
 References
 Software Metrics, Fenton and
Pfleeger, Second Edition
 Metrics and Models in Software
Quality Engineering, S.Kan
 Karl Wagers,
www.processimpact.com
 www.spc.ca, Creating a metrics
program – Step by step guide,
Available for free download
SQAM Course, Proficience, IISc 26