CMMI Update V1.2 and Beyond...

CMMI Update V1.
2 and Beyond
Mike Phillips Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 August 1, 2007
2007 Carnegie Mellon University
CMMI Transition Status As reported to the SEI as of 6-30-07 -1

Training Introduction to CMMI 70,791 Intermediate CMMI 2,549 Introduction to CMMI Instructor 504 SCAMPI A Lead Appraiser 731 SCAMPI B&C-Only Team Lead 33 Understanding CMMI High Maturity Practices 120 Authorized Introduction to CMMI V1.2 Instructors 400 SCAMPI V1.2 Lead Appraisers 417 SCAMPI B&C V1.2Team Leads 20
CMMI Update V1.2 and Beyond... Phillips, August 1, 2007
Transition from V1.1 to V1.2 Status As reported to the SEI as of 3-31-07 -2

Introduction to CMMI Students Registered for Upgrade Training Upgrades Complete Lead Appraisers and Instructors Registered for Upgrade training Upgrade Complete Authorized v1.2 Lead Appraisers Certified High Maturity Lead Appraisers 637 452 387 54 2,387 1,695

CMMI Adoption, Web Views

421K views/month in Q4 2006; over 24K views on 27 Sep 2006 Most downloaded files in Q4 2006

CMMI-DEV, V1.2 CMMI V1.2 Overview Presentation Extreme Programming (XP), Six Sigma, & CMMI: How They Can Work Together CMMI V1.2 Model Changes Presentation

Intro to the CMM and CMMI Attendees (Cumulative)
6-30-07
Number of SCAMPI v1.1/v1.2 Class A Number of SCAMPI v1.1/v1.2 Class Appraisals (Conducted by Quarter) A Appraisals Conducted by Quarter
300 275 250 225 200 175 150 125 100 75 50 25 0
2/ 02 3/ 02 4/ 02 1/ 03 2/ 03 3/ 03 4/ 03 1/ 04 2/ 04 3/ 04 4/ 04
Reported as of 30 June 2007
1/ 05
2/ 05
3/ 05
4/ 05
1/ 06
2/ 06
3/ 06
Organization Size
Based on the total number of employees within the area of the organization that was appraised
1001 to 2000 4.8% 501 to 1000 7.4%
2000+ 2.5% 25 or fewer 10.8%
301 to 500 9.6% 26 to 50 13.5%
2 1 to 2 0 + 0 00 3 .5 4 %
201 to 300 10.2% 101 to 200 19.6%
1 to 1 0 0 4 .9 5 %
51 to 75 12.9%
76 to 100 8.8%
1 6 Based on 80
organizations reporting size data

Countries where Appraisals have been Performed and Reported to the SEI
Argentina Australia Austria Bahrain Belarus Belgium Braz il Canada Chile China Colombia Cz ech Republic Denmark Dominican Republic gypt E Finland France Germany Hong KongIndia IndonesiaIreland Israel Italy Japan Korea, Republic of Latvia Malaysia Mauritius Mexico Morocco Netherlands New Zealand Pakistan Peru Philippines Portugal Russia SingaporeSlovakia South Africa Spain Sweden Switzerland Taiwan Thailand Turkey United Kingdom United StatesVietnam
Red country name: New additions with this reporting

Number of Appraisals and Maturity Levels Reported to the SEI by Country

C o u n try M a tu r ity M a tu ri ty M a tu ri ty M a tu ri ty M a tu rity N u m b e r o f L e v e l 1 L e v e l 2 L e v e l 3 L e v e l 4 L e ve l 5 A p p ra isa l s R e p o rte dR e p o rte d R e p o rte d R e p o r te d R e p o rte dC o u n try No Y es few e r few e r few e r few e r No No No Y es few e r few e r few e r few e r No few e r Y es Y es No few e r few e r few e r Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es No Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es M a tu rity M a tu rity M a tu rity M a tu rity M a tu rity N um be r of L e ve l 1 Le ve l 2 Le ve l 3 L e ve l 4 Le ve l 5 A p p ra isa lsR e p o r te dR e p o rte dR e p o rte dR e p o rte dR e p o r te d Y es fe w e r No fe w e r No fe w e r fe w e r fe w e r fe w e r fe w e r No fe w e r fe w e r fe w e r fe w e r fe w e r No fe w e r fe w e r No fe w e r fe w e r Y es Y es fe w e r Y es Y es Y es Y es Y es Y es No Y es Y es Y es No Y es Y es Y es No Y es Y es Y es No Y es Y es Y es Y es Y es Y es Y es No Y es Y es Y es Y es Y es A rge n tin a 19 A u s tra lia 23 A u s tria 10 or B a h ra in 10 or B e la ru s 10 or B e lg iu m 10 or B raz il 48 Canada 26 C h ile 15 C h in a 240 C o lo m b ia 10 or C z e c h R ep u b lic 1 0 o r D e n m a rk 10 or D o m in ic a n R e p u blic r 10 o E gy pt 17 F in lan d 10 or F ra n c e 75 G e rm a n y 35 H o n g K o ng 10 In d ia 204 In d on e s ia 10 or Ire lan d 10 or Is ra el 10 Ita ly 10 or Ja pa n 172 K ore a , R e pu b lic 8 f 7O L a tvia 1 0 or M a lay s ia 19 M a u rit iu s 1 0 or M e x ic o 15 M o ro c c o 1 0 or N eth e rla n d s 1 0 or N ew Ze a la n d 1 0 or P ak is ta n 1 0 or P eru 1 0 or P hilipp in e s 16 P ortu g a l 1 0 or R us s ia 1 0 or S in g a p o re 1 0 or S lo va k ia 1 0 or S ou th A fric a 1 0 or S pa in 31 S w e de n 1 0 or S w itz e rlan d 1 0 or Ta iw a n 46 Th a ila n d 1 0 or Tu rk ey 1 0 or U nite d K in g d o m 8 4 U nite d S tate s 7 1 8 V ie t N a m 1 0 or

Maturity Profile by All Reporting USA and Non-USA Organizations

100% 90% 80% 70% 60% USA: 100 % = 590 Non-USA: 100 % = 1122
% of Organizations
50% 40% 30% 20% 10% 0% Not Given

93
218
346
439 189
Initial
Managed
Defined
Quantitatively Managed
Optimizing
199 70
60
11
62
16
1590 Based on 12 2
USA organizations and
Non-USA organizations

10
Combined Appraisal Opportunities
Current ISO 9001

ISO 9001 IA
SCAMPI A & ISO 9001

Rating letter & or certificate with scope indicating in accordance with Level X
(Combined ISO Surveillance using Cat C appraisal)
Visit Report
continues to demonstrate compliance with ISO 9001:2000
Rating letter indicating level achieved
no behaviours inconsistent with operating at level X
Current CMMI
SCAMPI A
SCAMPI A
(Cat C appraisal)

11
Performance Results Summary

Improvements Median # of data points 21 19 16 18 6 14 Low High
Cost Schedule Productivity Quality
20% 37% 67% 50%
3% 2% 11% 29% -4% 2:1
87% 90% 255% 132% 55% 27.7 : 1
Customer Satisfaction 14% Return on Investment 4.8 : 1
N = 25, as of 15 December 2005 Organizations with results expressed as change over time

12
CMMI Books
A Guide to the CMMI: Second Edition CMMI: A Framework CMMI Assessments CMMI Distilled: Second Edition CMMI SCAMPI Distilled CMMI Survival Guide CMMI: Un Itinraire Flch: Second Edition De kleine CMMI Interpreting the CMMI Making Process Improvement Work Practical Insight into CMMI Real Process Improvement Using the CMMI Systematic Process Improvement Using ISO 9001:2000 and CMMI

13
How About SEI Publications?

Technical notes and special reports:
Interpreting CMMI:

for Operational Organizations for COTS Based Systems for Service Organizations for Business Development Team Software Process (TSP) Earned Value Management Product Line Practices Six Sigma
Using CMMI with:

Supplementing CMMI for Safety Critical Development (+Safe) Demonstrating the Impact and Benefits of CMMI (and Web pages http://www.sei.cmu.edu/cmmi/results)

14
CMMI V1.2 and Beyond...
CMMI Today
Version 1.1 CMMI Product Suite was released January 2002.

CMMI Web site visits average over 20,000/day Over 66,000 people have been trained Over 1800 class A appraisals have been reported to the SEI
Annual rate now over 800/year
Now we want to continuously improve

16
Major Themes
Reduce complexity & size Increase coverage Increase confidence in appraisal results

17
CMMI Model Combinations

V 1.1
Supplier Sourcing Integrated Product and Process Development
V 1.2
IPPD
Organizational Goal (OPD) Project Goal (IPM)
SE Related Examples SW Related Examples SW Related Examples
SE Related Examples
HW Related Examples
CMMI Core
CMMI Core (now includes SS)

18
IPPD Changes
V1.1
t r oppu S ssec o P r s AP tgM s AP
V1.2
SG1 SG2
SG1
I EO
SG2
SG1 SG2 SG3 SG4 SG1
SG1 SG2 SG3
MP I
MP I
DP O
SG2 = Enable IPPD principles
SG3 = Apply IPPD principles
T I e ga na M ce o P t j r
SG2

19
SCAMPI A Changes for V1.2

Method implementation clarifications

interviews in virtual organizations practice characterization rules organizational unit sampling options
Appraisal Disclosure Statement (ADS) improvements

reduce redundancy with other appraisal documents improve usability for sponsor and government require sponsors signature on the ADS require all team members to show agreement on findings both V1,1 and V1.2 ADS will reflect these as of Nov 2006
Lead Appraisers must be certified to lead high maturity appraisals Maturity level and capability level shelf life 3 years, given 1 year of V1.2 availability
20
Published Appraisal Results

21
CMMI V1.2 Schedules

Version 1.2 CMMI Product Suite release Update material available Upgrade course available on-line First Lead Appraiser face to face First high maturity oral exam V1.2 ADS required for all SCAMPIs Last V1.1 Intro training First expiration of V1.1 appraisals (3-year validity) Last V1.1 appraisal August 25, 2006 August 25, 2006 August 25, 2006 October, 2006 October, 2006 November, 2006 December, 2006 August 25, 2007 August 31, 2007

22
Beyond V1.2
Improved architecture will allow post-V1.2 expansion.
Extensions of the life cycle (Services, Outsourcing/Acquisition) could expand use of a common organizational framework:
allows coverage of more of the enterprise or potential partnering organizations adapts model features to fit non-developmental efforts (e.g., CMMI Services, CMMI Acquisition)

23
3 Complementary Constellations
CMMI-Services provides guidance for those providing services within organizations and to external customers
CMMI-Dev provides guidance for measuring, monitoring and managing development processes
CMMI-SVC
16 Core Process Areas, common to all CMMI-ACQ provides guidance to enable informed and decisive acquisition leadership
CMMI-DEV
CMMI-ACQ

24
Acquirer/Supplier Mismatch
High
Mismatch
mature acquirer mentors low maturity supplier outcome not predictable
Matched
acquirer and supplier are both high maturity highest probability of success
Acquirer
Disaster
Low
no discipline no process no product
Mismatch
immature acquirer mature supplier Customer encourages short cuts.
Technical & Management Skill
Low
Supplier
High
25
Initial CMMI-ACQ Key Acquisition Processes

Solicitation & Supplier Agreement Development
Agreement Management
Core Acquisition Validation Project Management Processes Acquisition Requirements Development Acquisition Verification Acquisition Technical Management
*based on initial CMMI-ACQ model developed by General Motors/SEI

26
Acquisition Improvement is Needed.

Acquirers cannot ensure that mature processes are applied to their programs
High
Acquirers need more internal process focus

Mismatch
outcome not predictable
XYZ Corp.
Division A Division B Division N
mature acquirer mentors low maturity supplier
acquirer and supplier are both high maturity
Matched
PMO
highest probability of success
Low
Project 1
MLs usually apply HERE based upon appraisals of THESE
Project 2
Project 3
Project 4
Project 5
Project 6
but your project is HERE or HERE
Project 7
Project x
Project y
Project z
Disaster
no discipline no process
less mature acquirer derails mature supplier; encourages short cuts
Mismatch
no product Low
supplier compromises processes
Contractor
High
Technical & Management Skill

27
Guidebook Concept
Provide a process toolbox for the acquirer

Include practical guidance on how to recognize the real practitioners Encourage the use of capability and maturity profiles vice "single level" approach Improve acquisition organizations' understanding of the meaning of high maturity (levels 4 and 5) and equivalent staging Include multiple tools and guidance that may be used throughout the acquisition lifecycle

28
Planned Sequence of Models
CMMI-SVC
CMMI V1.1
CMMI-DEV V1.2
GM IT Sourcing CMMI-AM
CMMI-ACQ
SA-CMM

29
Questions to Guide Discussion

Do we need something different or additional to define High Maturity (i.e. CMMI Level 4 & 5)? How can we "slim down" the CMMI models while still preserving integrity? Can we likewise "slim down" the Appraisal method? Can we eliminate the Staged representation? Is the CMMI v1.2 Constellation Strategy the right approach? Can we identify "next-generation" process improvement methodology? Can CMMI be harmonized with other continuous process improvement efforts? Can repeatability, consistency and overall model and appraisal methodology be improved? Are there "breakthrough" concepts that we can apply to overall process improvement?
30
Can CMMI be harmonized with other standards and continuous process improvement efforts?
Agree that harmonization should be a goal, but should not slow progress too much Harmonization efforts take time
(This may be the only formal harmonization effort) Currently,15288 being harmonized with 12207 (ongoing several years). There is a move to say either or. Recent work in this area to come out soon. Standards Process Improvement Methods 9001, 14000 (environmental standard), AS 9100, FAA Standard (Aviation Critical Safety Items), 15288, 12207, 15504, ITIL, COBIT, Sarbanes-Oxley, 632 (Systems Engineering), 1220, Malcolm Baldridge, Six Sigma [not all standards here are at the same level of abstraction], PM BOK and OPM3
Are there standards we want to focus on?


31
How can we "slim down" the CMMI models while still preserving integrity?
How can we make this more user friendly? Can we slim down for small projects? Can the model have some scalability according to various factors (e.g., project size, PoP, organization size)? How do extensions fit in with the model? Consider options for packaging (remove redundant stuff or repackage better) Consider fundamental, intermediate and advanced volumes Consider architectural views for appropriate for the different using communities Consider streamlining the generic practices and look at measures for institutionalization
Consider folding the GPs into PAs [note risk of losing integrity]

32
Do we need something different or additional to define High Maturity? -1

Focus on the best practices, not focus on the high maturity aspects Consider combining level 4 and 5 into one level because of their close tie 4 and 5 are not adequately elaborated for implementation so these may needs more detail to drive proper behavior

For example, if we tie level 4 and 5 to business objectives this may need to be a practice If there are additional requirements for the model these can be turned into practices. E.g., High maturity body of knowledge and high maturity training Risk: adds more to the model

33
Do we need something different or additional to define High Maturity? -2

Consider redistributing practices across the levels to even out effort and expectation Atlas study items that impact the model and results Consider maturity levels within PAs (e.g., project management PAs for each level) Consider better interfacing approaches with other methodologies (e.g., six sigma for high maturity)

34
How can repeatability, coverage (scope) and consistency of the model be improved?
Coverage (Scope)
Areas for consideration
Operations, Support, Transition to operations, Deployment, Disposal, Pre-project, Proposal, sustainment, transition to production, production/manufacturing, training Better coverage of maintenance and technical reviews Safety, security, dependability, systems assurance, environmental Strategic planning, enterprise management, enterprise architectures, enterprise investment strategies, finance Work force management and development IR&D, Advanced technology, advanced technology test bed or lab environment Small settings Product lines Business practices Information management (both enterprise and project) Systems of systems and their acquisition

35
What are the "next-generation" process improvement methodology? Are there "breakthrough" concepts that we can apply to overall process improvement?
Consider how CMMI could interface with other process improvement methodologies (e.g. Lean, six sigma, PMBOK, theory of constraints, next generation IDEAL) Integration of how people use the various methodologies (same list as above)
Agile techniques (extreme programming), TSP/PSP Consider optimizing measurements
When everyone is level 5, then what?
Consider an emphasis on process performance effectiveness and efficiency, (e.g., effectiveness 6 sigma, efficiency LEAN) How do we measure program health?
Need for leading indicators

36
What representation should we have (e.g., Staged, Continuous)? -1

Is the question really level-mania? (root cause?)

Level-mania is about doing the minimal work to achieve a level ignoring what you did to achieve the grade Levels are not bad, but we need to get the integrity of the level back up
Provides a roadmap for projects to break PI into pieces Guidance for where improvement is needed
Risk: has 5 been around so long that getting rid of it will have unintended consequences?
Maybe we have a 5 level model that only really has 3. Redesign the model to address this

37
What representation should we have (e.g., Staged, Continuous)? -2

How can we make two representations fit the same model? If the model is expanded to handle additional scope, then we may need to consider changes in the way appraisal results are presented due to sponsor driven time constraints [packaging]

How do we slim at the same time as providing better understanding and cover all the things that are need? As you expand scope, do you need to abstract concepts versus mega model? [Scope, slimming] Need a vision and plan for model evolution
Consider a fixed size approach and looking at ways to present the remaining information
Consider pulling out OPF and coupling it with levels 4 and 5 Dont need to cover everything in the same model (packaging)
38
Is the CMMI v1.2 Constellation Strategy the right approach?

Alternative approach: Start with a CMMI Model Framework (CMF) and add where you need to expand scope (+ concept) Are there differentiators for constellations? Instead of creating constellations, encourage projects to do what makes sense with respect to what they are doing using the parent model

Consider looking at lifecycle and what is needed at each phase Can the unsophisticated tailor the parent model for their perspective?
How multiple constellations can be used in an organization for improvement and appraisal?

39
How can we "slim down" the CMMI appraisals while still preserving integrity? How do we eliminate non-value add in appraisal and appraisal preparation? How do we make appraisals more efficient and effective?
Consider making the appraisal be focused on goals Add an appendix for application practices Lead Appraiser and the Appraisal Team should have enough experience to review company implementations Make some assumptions that some processes are in place (e.g., assume project planning has happened, but dont look at PP specifically unless you see something out of place in PMC; similarly, could start with IPM for a level 3, or QPM for a level 4)
Need more guidance on where and how you might be able to do this
More official or formal mechanisms for sampling coverage
Consider cost implications

40
Can we identify "next-generation" appraisal methodology? Are there "breakthrough" concepts that we can apply to overall appraisals? Delta appraisal, continuous, incremental, using measures to judge satisfaction, leading indicators, process performance measures, program health (versus process health), 15504 (SPICE), EIA 732 (percentage of practices performed, effectiveness of generic attributes) Data reuse from previous appraisals Appraisal by parts

Example, OSP separate from projects CMF separate from model components unique to constellations Certify processes instead of model (e.g., EV or SEMP)
Sponsor commits to ongoing process improvement

41
How do we improve the trust and believability in the results of the appraisals?
Process enactment tools can help with approval (workflow management) Need to define consistent process for OSP tailoring approval Requiring the appraisal be redone every three years will help with believability (already in place)

Consider notion of visits or interim steps (like ISO surveillance audits) Six month assessments focus on correlation between results and performance (process reviews)
Doesnt interrupt the program and not as expensive
What if you could extend the life of your appraisal if you did interim review? What is in it for the organization to provide incentive for the use of reviews?
Would the customer accept the results of the interim review?

42
How do we improve the trust and believability in the results of the appraisals?

What should the results of a bad review be? Should you lose your level? Could you use the delta appraisal here? How are organization changes that may impact the process capability tracked?
Consider adding a practice to the model for these changes
Would interim reviews impact the capability profile
Might red flag the program or organization
What happens if you miss by one practice in the full appraisal?
Do I get rewarded if I fix it? Consider delta approach
Consider a more formal mechanism to track changes that may impact the process capability or level

43
Next Steps
Send us your ideas

Form available on-line from SEI Submit like a Change Request
Open Discussion.

44

CMMI Update V1.2 and Beyond...

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CMMI Update V1.2 and Beyond...

Uploaded by

Copyright:

Available Formats

CMMI Update V1.

2007 Carnegie Mellon University

CMMI Transition Status As reported to the SEI as of 6-30-07 -1

Transition from V1.1 to V1.2 Status As reported to the SEI as of 3-31-07 -2

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

CMMI Adoption, Web Views

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

Intro to the CMM and CMMI Attendees (Cumulative)

Reported as of 30 June 2007

2007 Carnegie Mellon University

1001 to 2000 4.8% 501 to 1000 7.4%

2000+ 2.5% 25 or fewer 10.8%

301 to 500 9.6% 26 to 50 13.5%

organizations reporting size data

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

Red country name: New additions with this reporting

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

Number of Appraisals and Maturity Levels Reported to the SEI by Country

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

Maturity Profile by All Reporting USA and Non-USA Organizations

50% 40% 30% 20% 10% 0% Not Given

USA organizations and

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

Combined Appraisal Opportunities

Current ISO 9001

SCAMPI A & ISO 9001

(Combined ISO Surveillance using Cat C appraisal)

continues to demonstrate compliance with ISO 9001:2000

Rating letter indicating level achieved

no behaviours inconsistent with operating at level X

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

Performance Results Summary

Cost Schedule Productivity Quality

20% 37% 67% 50%

3% 2% 11% 29% -4% 2:1

87% 90% 255% 132% 55% 27.7 : 1

Customer Satisfaction 14% Return on Investment 4.8 : 1

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

How About SEI Publications?

Using CMMI with:

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

CMMI V1.2 and Beyond...

2007 Carnegie Mellon University

Annual rate now over 800/year

Now we want to continuously improve

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

CMMI Model Combinations

Organizational Goal (OPD) Project Goal (IPM)

SE Related Examples SW Related Examples SW Related Examples

CMMI Core (now includes SS)

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

SG1 SG2 SG3 SG4 SG1

SG1 SG2 SG3

SG2 = Enable IPPD principles

SG3 = Apply IPPD principles

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

SCAMPI A Changes for V1.2

Appraisal Disclosure Statement (ADS) improvements

Published Appraisal Results

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007

CMMI V1.2 Schedules

CMMI Update V1.2 and Beyond... Phillips, August 1, 2007