Professional Documents
Culture Documents
Sree Yellayi
SCAMPI Lead Appraiser, SEI Authorized CMMI Instructor, Certified Scrum Master
Siemens, Greater New York City Area
sree.yellayi@siemens.com
sreeramamurthy@yahoo.com
Abstract
Software industry is one of the most empirical industries due to its high dependence on technology and people. Each
software company adopts a specific development methodology, and furthermore seeks to build a system for its process
improvement by adopting one or more Software Process Improvement (SPI) models. CMMI is a process framework
which is widely adopted by software and systems development companies, while Scrum is one of the more
recent project management agile method whose adoption is growing rapidly. CMMI is basically a process
improvement framework which provides a set of processes for software and system development management, Scrum
can be thought of as an iterative project management framework for development activities, CMMI has a wider scope
and different aims to those of Scrum and covers production support, maintenance, product implementation and
application transition type projects as well.
Scrum and other agile methods have clearly appeared in 2001, this paper shows how to implement the Measurements
and Analysis (M&A) process area of CMMI model and clarify how M&A can be achieved in the agile (Scrum)
organization.
Most of the promising objectives of any software development method or process are delivering working software on
time, quality and budget. Software Engineering Institute (SEI) has paid a lot of efforts to mostly satisfy these
objectives in its process improvement models, and lately CMMI version 1.2 has been released.
Nevertheless, the world becomes convinced with adding another objective, which is delivering a business value to the
customer. Along the years, software engineers have proposed several methodologies. Agile is the most known and
latest proposed development method that can achieve this objective beside the other mentioned objectives.
Our objective in an agile environment is not to do software measurement. We must learn to build reliable software
measurement process based on valid software measurement tools. If we try to do too much too soon, we will likely
fail.
Introduction
Software industry is one of the most empirical industries due to its high dependence on technology and people. Each
software company adopts a specific development methodology, and furthermore seeks to build a system for its process
improvement by adopting one or more Software Process Improvement (SPI) models. CMMI is a process framework
which is widely adopted by software and systems development companies, while Scrum is one of the more recent
project management agile method whose adoption is growing rapidly. CMMI is basically a process improvement
framework which provides a set of processes for software and system development management, Scrum can be
2009
CMMI® 9th Technology Conference and User Group | www.ndia.org/meetings/0110
thought of as an iterative project management framework for development activities, CMMI has a wider scope and
different aims to those of Scrum and covers production support, maintenance, product implementation and application
transition type projects as well [1]. In 1996, Kent Beck joined the Chrysler C3 payroll project. It was in this context
that the full set of XP practices matured, with some collaboration by Ron Jeffries and inspiration from earlier 1980s
work at Tektronix with Ward Cunningham. XP went on to garner significant public attention because of its emphasis
on communication, simplicity, and testing, its sustainable developer-oriented practices, and its interesting name. [2]
Scrum and other agile methods have clearly appeared in 2001, Alan MacCormack reported a study of key success
factors in recent projects; first among these was adopting an IID life cycle: Now there is proof that the evolutionary
approach to software development results in a speedier process and higher-quality products. […] The iterative process
is best captured in the evolutionary delivery model proposed by Tom Gilb [3]. Scrum appeared when a group of 17
process experts—representing DSDM (Dynamic Systems Development Method), XP (extreme Programming) ,
Scrum, FDD (Feature Driven Development), and others— who are interested in promoting modern and simple IID
(Iterative and Incremental Development) methods and principles, they met in Utah to discuss common ground. From
this meeting came the Agile Alliance (www.agilealliance.org) and the now popular catch phrase “agile methods,” all
of which apply IID. And in 2002, Alistair Cockburn, one of the participants, published the first book under the new
appellation [4]. Moreover, prominent software-engineering thought leaders from each succeeding decade supported
IID practices, and many large projects used them successfully [5]. Agile is all about value individuals and interactions
over processes and tools, working software over comprehensive documentation, customer collaboration over contract
negotiation, and responding to change over following a plan [15]. This paper shows how to implement the
Measurements and Analysis (M&A) process area which is an important Process Area (PA) in CMMI model; the
organization cannot be appraised without satisfying this PA, and clarifying how M&A can be achieved in the agile
(Scrum) organization.
Basically, software engineering measurement is not a resource issue; it is a commitment issue. The bottom line is that
all the usable measures from practicing Scrum on a project could be used to address the practices of M&A process
area. In fact, the alignment of these measures to the information needs is very visible due to the very nature of “value-
based” focus of the scrum method. In our research, we found that there need not be any additional measures that are
required to be “invented” to fulfill the M&A goals from CMMI model. Usage of existing ones is enough and we will
share those mappings and arguments with you for your application and subsequent extension to other process areas in
the model.
Agile CMMI
Why Agile CMMI
Most of the promising objectives of any software development method or process are delivering working
software on time, quality and budget. Software Engineering Institute (SEI) has paid a lot of efforts to mostly
satisfy these objectives in its process improvement models, and lately CMMI version 1.2 has been released.
Nevertheless, the world becomes convinced with adding another objective, which was somehow neglected
more than highly considered, which is delivering a business value to the customer. Along the years, software
engineers have proposed several methodologies (see figure 1),
1965
Years
Yes! [6]
Waterfall (Royce): Requirements,
1970 Design, Implementation, Verification
and maintentance
Feasible together?
RUP (Rational)
(Figure-1: History of Software Development Methodologies)
1999
Agile (Kent Beck)
Incremental, user driven, customer interaction
2009
Agile is the most known and latest proposed development method that can achieve this objective beside the
2009
CMMI® 9th Technology Conference and User Group | www.ndia.org/meetings/0110
Implementation of M&A
M&A Procedure Flow
2009
CMMI® 9th Technology Conference and User Group | www.ndia.org/meetings/0110
Our objective in an agile environment is not to do software measurement. We must learn to build reliable
software measurement process based on valid software measurement tools. If we try to do too much too
soon, we will likely fail. Basically, software engineering measurement is not a resource issue; it is a
commitment issue. We are going to build a measurement process that will generate great volumes of data.
These data must be converted to information that can be used in the software development decision-making
process. The principle is a simple one. Even a small amount of measurement data that can be converted to
useful information is better than large volumes of measurement data that have little or no information value.
We must begin with simple tools and focus most of our attention on the measurement and management
processes. We must remember that we learned how to measure distances in the first grade with a very crude
ruler. Our teachers did not give us micrometers to learn measurement. Ultimately, we learned that our rulers
could be used to quantify size attributes of the objects in our environment. These rulers, in fact, had utility
beyond their obvious use as bludgeons that we could use on our classmates. [10] In order to monitor and evaluate
process performance we must consider views of different stakeholders that take part in the process. The best performance is
achieved when the objectives of all stakeholders are satisfied. Kueng [11] defines process performance as “the degree of
stakeholder satisfaction”.
2009
CMMI® 9th Technology Conference and User Group | www.ndia.org/meetings/0110
The objectives are subjective and should be defined from the organization; however, here are some
examples of what objectives vs. stakeholders can be established (table 1):
Objectives Stakeholder
• Timely information on project information performance
and progress
• Product Quality Improvement Progress IT Management
• Process Quality Improvement Progress
• Efficient and effective resolutions for the impediments Project Scrum Master and
• Process Compliance Quality Assurance
To illustrate the meaning of who could most likely be the fore mentioned stakeholders:
IT Management: General Manager, Project Manager who are concerned with the traditional aspects of
software development from the perspective of time, cost, and quality.
Project Team Members: Developers, Testers, Team Leader, and Technical Writers
Project Quality Assurance & Coach: Concerned with facilitating the use of SCRUM and CMMI, creating
conditions for smooth running of the development process, hence the main goal is to provide an efficient
impediments resolution.
Specification of Measures
The measures are obtained based on the defined objectives, by thinking of each objective and all related
links that negatively or positively affect it.
According to CMMI, measures may be either “base” or “derived”. While data for base measures are
obtained by direct measurement, data for derived measures come from other data, typically by combining
two or more base measures. Derived measures serve as performance indicators showing the achievement of
particular goals. Originally, Scrum had only one base measure: the estimate of the amount of work
remaining that needs to be done in order to complete a Product Backlog item or a task in the Sprint Backlog
(SB). At the task level, this measure is collected every day for each task in the Sprint Backlog separately.
[12]
Fortunately, the daily Scrum meetings give an outstanding help to achieve Measurements & Analysis,
especially the data collection.
Here are some proposed measures as a practical example:
Objective Measures
• Timely information • Remaining work from Sprint S
on project • Remaining work on day d for each task in the Sprint Backlog
information (SB)
performance and • Each task progress for day d in the SB
progress
2009
CMMI® 9th Technology Conference and User Group | www.ndia.org/meetings/0110
• Roles and • The number of completed tasks for each team member (for
Responsibilities each Sprint Backlog)
Commitment
• Process • Number of Non-Compliances (NCs) reported by Quality
Compliance Assurance per Sprint S.
• Number of resolved NCs per Sprint S.
It’s not recommended to propose a mapping between the measures and its collection points in a general
proposal, to allow more flexibility to different environments and practices.
2009
CMMI® 9th Technology Conference and User Group | www.ndia.org/meetings/0110
The collection points should be defined in the process and can be customized per project (if needed).
Moreover, the use of automated tools is much recommended after gaining the main concept of Agile and
deeply knowing the goals of CMMI.
Measurements Analysis
Derived measures are derived from the base measures, such derived measures support the decision
makers in different management levels that serve for analyzing software process performance in
comparison to target values set by software development organization.
As practical examples:
The objective “Timely Information on Project Progress” can be analyzed using the following indicators:
- Work Effectiveness
It’s the ratio between the decrement of remaining work and done work.
The decrement of remaining work between days d1 and d2 of Sprint should be equal or greater than the done
work in the same interval, meaning that the best value for this indicator is 1 or more; however, the values
significantly greater than 1 may be a sign of poor planning.
WS d
WE d = (Eq. 1)
WP d
Where WEd denotes the Work Effectiveness of a specific day d in a Sprint, WP d denotes the planned
work that should be accomplished in the beginning of the day d, WS d denotes the work spent in the
day d. The calculation of Works can be in terms of tasks’ number or the average percentages of
tasks’ completion.
However, burn-up or burn-down charts can do this job in an easier visualized way, and it can be done
weekly or every 2 days.
And to calculate the Schedule Performance Index using the work remaining and work spent measures we
assume that the amount of tasks that must be accomplished at a certain point in the Sprint is proportional to
the time elapsed from the beginning of the Sprint. The work remaining and work spent measures allow a
precise definition of the Earning Value equation EVd , j for each task j in the Sprint Backlog on the day d of a
Sprint. It can be computed as a ratio between the amount of work already spent and all the work required
(spent and remaining) to accomplish the task:
d −1
∑WS
i =1
i, j
EVd , j = d −1 (Eq. 2)
∑WS
i =1
i, j + WRd , j
WSi , j denotes the amount of work spent for task j on day i, i=1,2,…,d-1, and WRd , j denotes the amount of
work remaining for task j on day d.
By using the earning value, we can get the SPI by the following formula:
2009
CMMI® 9th Technology Conference and User Group | www.ndia.org/meetings/0110
∑ EV
j =1
d, j WRinitial , j
SL
SPI d = n
• (Eq. 3)
DE
∑WR
j =1
initial , j
Where WRinitial , j denotes the initial estimate of the work remaining for task j, SL the Sprint Length in the unit
of days, and DE the number of days elapsed.
Summary
In fact, there is no practice in scrum like the expected practices of Measurements and Analysis (MA).
Nevertheless, in scrum you have the opportunities to get the measures that implement MA easily.
And to summarize the specific practices (table3) and generic practices (table4) of MA process area [16]:
SP 1.4 Specify how measurement • The Scrum process does describe the purpose and use
data will be analyzed and the Sprint and Release burndown charts.
reported. • Define the analysis procedure clearly and the mechanism
of reporting.
SP 2.1 Obtain specified • Daily Scrum meeting where Sprint and Release
measurement data. burndown/burnup data are collected.
SP 2.2 Analyze and interpret • Daily Scrum meeting where Sprint and Release
measurement data. burndown data are analyzed.
SP 2.3 Manage and store • This can be declared/defined in Team Kickoff or
measurement data, Chartering of the project
measurement specifications,
and analysis results.
SP 2.4 Report results of • Daily Scrum meeting where Sprint and Release
measurement and analysis burndown charts are reviewed.
activities to all relevant
stakeholders.
Table 3: Summary of MA specific practices in Scrum
GP 2.3 Provide adequate resources • The resources and schedule time allocated to perform
for performing the Scrum planning, monitoring and requirements activities
measurements and analysis (for example: the internal and external chartering that
process shows the environment, roles and resources)
GP 2.4 Assign responsibility and • The resource roles allocated to perform the activities in
authority for performing the process documents and in the chartering as well (i.e.
process, developing the work define who is responsible for measurement planning,
products, and providing the collection… and all activities)
services of the MA
GP 2.5 Train the people performing • Scrum team training for the aspects of Scrum that relate
or supporting the MA to measurements & analysis. (i.e. training materials,
process as needed …etc)
GP 2.7 Identify and involve the • The list of Scrum team members, their roles and
relevant stakeholders of the allocations in the chartering (the beginning of the
measurement and analysis project)
process as planned. • Report to the relevant stakeholders with the results of
measurement analysis.
GP 2.8 Monitor and control the • Scrum master follows the plan & schedule of
measurement and analysis measurements, and data collection points.
process against the plan for • Scrum master assures that the measurements are
performing the process and collected and analyzed according to the plan & schedule.
take appropriate corrective
action.
GP2.10 Review the activities, • The senior manager has a visibility into scrum teams’
status, and results of the work and progress by burnup or burndown charts.
measurement and • The senior manager can be reported with the reports of
analysis process with measurement & analysis results.
• The actions that are taken by the senior manager
higher level management
according to the reported measurement & analysis
and resolve issues. results.
(other generic goals for
level 4 and 5)
Table 4: Summary of MA generic practices in Scrum
Conclusion
The bottom line is that all the usable measures from practicing Scrum on a project could be used to address
the practices of M&A process area. In fact, the alignment of these measures to the information needs is very
visible due to the very nature of “value-based” focus of the scrum method. In our research, we found that
there need not be any additional measures that are required to be “invented” to fulfill the M&A goals from
CMMI model. Usage of existing ones is enough.
References
[1] Scrum and CMMI: A High level assessment of compatibility, Srinivas Chillara and Pete Deemer
[3] A. MacCormack, “Product-Development Practices That Work,” MIT Sloan Management Rev., vol. 42,
no. 2, 2001
[5] Craig Larman, Victor R. Basili , Iterative and Incremental Development: A brief History
[6] Hillel Glazer, Jeff Dalton, David Anderson, Mike Konrad, Sandy Shrum, CMMI or Agile: Why Not
Embrace Both, Nov 2008
[7] CMMI Product Team, CMMI for Acquisition Ver 1.2, Nov 2007
[8] Juran, J.M., Juran on Quality by Design: The New Steps for Planning Quality into Goods and Services,
Free Press, New York, 1992
[9] Deming, WE., Out of the Crisis, MIT Press, Cambridge, MA, 2000
[10] Munson, J.C. and Coulter, N.S., "An Investigation of Program Specification Complexity," Proceedings
of the ACM Southeastern Regional Conference, April 1988
[11] P. Kueng, “Process performance measurement system: a tool to support process-based organizations,”
Total Quality Management, 2000
[12] J. Sutherland et al., “Scrum and CMMI Level 5: The Magic Potion for Code Warriors,” in Proc.
AGILE 2007
[16] Neil Potter and Mary Sakry, Process Group Newsletter Volume 16 No 2, March 2009