A CMMI Maturity Level 2 Assessment of RUP

Level: Introductory Markus Grundmann, Consultant, IT Process Optimization 15 Dec 2005

http://www-128.ibm.com/developerworks/rational/library/dec05/grundmann/

From The Rational Edge: This article examines how well the IBM Rational Unified Process (now available as part of the new IBM Rational Method Composer) can help software development organizations achieve a higher process maturity level. It assesses RUP coverage in teams of process areas defined by Level 2 of the Capability Maturity Model Integration (CMMI) and identifies areas to supplement in RUP in order to compensate for the gaps. More dW content related to: requirements for achieving maturity level The rapid pace of modern software development has created some serious challenges for development teams:

Requirements invariably change over the lifecycle of a project. You can't get the model right on paper the first time, as you can when building a bridge, for example. State-of-the-art projects cannot rely on proven technologies.

These problems have had a severe impact on project success rates,1 as documented by the Standish Group in their Chaos Report. As Figure 1 shows, although these rates rose by 75 percent between 1994 and 2000, still only 28 percent of all projects were delivered as planned in 2000.

Figure 1: Software development project success rates from the Standish Chaos Report 2001 The IBM Rational Unified Process, or RUP, is rooted in a set of best practices that address the challenges outlined above. It helps software development

organizations to determine which roles can best perform certain activities and defines a set of artifacts (i.e., deliverables) that support these activities. In addition, it models the interactions and dependencies between these elements, thereby bringing greater transparency to the project. Complementing RUP, with its execution-oriented approach and detailed guidelines, are process improvement frameworks such as ISO 9000:2000, the eSourcing Capability Model for Service Providers (eSCM), and the Capability Maturity Model Integration (CMMI), which operate on a more generic level. They focus on structuring an organization to achieve process-driven quality and leave the details of how to execute the process up to each organization. This article examines RUP's capability to help organizations achieve a high level of process quality, using the Capability Maturity Model Integration for Software (CMMI-SW 1.1) for the assessment.

The CMMI
The CMMI helps organizations sharpen their competitive edge by providing better predictability and greater efficiency, ultimately leading to lower costs and more satisfied customers. Data gathered during CMMI process improvement initiatives at companies such as Siemens, JP Morgan, Chase, and Lockheed Martin tells us that supporting a major process improvement initiative requires between 2 percent and 10 percent of the engineering power involved in a project. But the benefits of implementing CMMI more than compensate for this effort, as a study2 published by the Software Engineering Institute (SEI) of Carnegie Mellon University shows. The median distribution of the improvement percentages is as follows:

20 37 67 50 14

percent percent percent percent percent

cost reduction more likely to be on schedule productivity increase fewer defects higher customer satisfaction

Of course, a CMMI implementation entails additional engineering and processrelated costs. But again, the study reveals that the positive impact clearly overweighs these costs: Implementations typically deliver a return on investment of almost 5:1. Using the CMMI to assess RUP gives organizations a means to compare RUP with other processes and process models to see where it might need augmentation when used as a standalone process. Managers can also use this assessment to see where they might use RUP productively to supplement their organization's current processes.

CMMI Maturity Level 2 appraisal

CMMI offers two architectural representations: "continuous" and "staged." The continuous version groups the CMMI process areas by four process area categories:

Process Management Project Management Engineering Support

For this appraisal, I will use the staged version, as it groups the process areas by maturity level, and they are, in part, the focus of this article. The maturity levels are defined as follows: 1. Initial: The process is usually ad hoc and chaotic. This leads to poor predictability, and the process is not repeatable and reactive. 2. Managed: The process is defined for the project but mostly reactive. The project team does obtain commitment from stakeholders, and the status of work is visible to management. 3. Defined: The process is tailored to the organization's process assets. The relationships are better understood, output is measured with a greater level of detail, the process is managed proactively, and the outcomes are qualitatively predictable. 4. Quantitatively managed: The process is controlled and measured by statistical data, and outcomes are therefore quantitatively predictable. 5. Optimizing: The process addresses the causes of process variation and adapts itself. It includes identifying success factors and contributing them to the organization's process assets. A maturity level is defined as a certain set of process areas; each process area combines a group of goals that are achieved through practices. This assessment will look at RUP as an aid to achieving Level 2. The requirements defined at this level incorporate the most important project management areas and are a prerequisite to obtaining higher maturity levels. The assessment will identify strengths and weaknesses and show areas of improvement for operating at CMMI Level 2. The method I will use is a Class A appraisal method developed by the SEI called the Standard CMMI Appraisal Method for Process Improvement, or SCAMPI. Class A methods implement the highest level of requirements and generate ratings that may be used for process comparisons. Following SCAMPI, a "real-world" method, would normally require interviews with the individuals who execute the process and review work results. This is not possible when assessing RUP simply as a product, and assessing a single instance of a RUP implementation would open the door for distortion, because organizations typically adapt RUP to their organizational and project-specific needs. Therefore, in viewing the results of this assessment, we must imagine a neutral environment in which RUP has not been tailored to a particular organization and has been executed in a "classic" configuration. The CMMI Level 2 consists of the following process areas:

Requirements Management Project Planning

Project Monitoring and Control Supplier Agreement Management Measurement and Analysis Process and Product Quality Assurance Configuration Management

Each process area contains one or more goals, which you can meet by implementing that process area's practices. A practice, the smallest unit in CMMI, represents a goal requirement. According to SCAMPI, a practice can be implemented in four degrees: 1. Fully implemented: Elements addressing this practice are present, and there are no substantial weaknesses. 2. Largely implemented: Elements addressing this practice are present, but there are some weaknesses. 3. Partially implemented: Elements primarily addressing this practice are absent, but elements that cover part of this practice as a side-effect are present; there are weaknesses with regard to this method. 4. Not implemented: The practice requirements are not met. SCAMPI defines a practice as fulfilled if it is at least "largely implemented." The corresponding goal is achieved if all practices are fulfilled, and the process area is covered if all goals are met. You achieve a maturity level by meeting all of its process area goals and generic goals. For example, a generic goal in Level 2 is:

Institutionalize a managed process.

Generic goals apply to several process areas and are therefore treated separately. But achieving them is mandatory.

CMMI Maturity Level 2 appraisal of RUP

Tables 1-6 show RUP coverage for six of the CMMI Maturity Level 2 process areas. Table 1: RUP coverage for the Requirements Management process area

Table 2: RUP coverage for the Project Planning process area

Table 3: RUP coverage for the Project Monitoring and Control process area

Table 4: RUP coverage for the Measurement and Analysis process area

Table 5: RUP coverage for the Process and Product Quality Assurance

process area

Table 6: RUP coverage for the Configuration Management process area

Two process areas that can supplement RUP

In addition to the process areas discussed above, we should note that RUP does not provide sufficient coverage of the Supplier Agreement process area and the generic goal Institutionalize a Managed Process. Organizations using RUP can use CMMI requirements to supplement RUP in these areas. Supplier Agreement Management RUP provides no implementation of the Supplier Agreement Management process area. Because of its focus on software development, RUP assumes that the organization will develop the entire product. This is a weak spot because relying on proven solutions may be a better alternative than custom developing everything; you can integrate commercial-off-the-shelf (COTS) products or components and still create a new product that delivers value to the customer. Often, this can also help lower development costs and shorten delivery time. Table 7 shows the practices under the CMMI Supplier Agreement Management process area; organizations using RUP can use this to supplement RUP guidelines.

Table 7: RUP coverage for the Supplier Agreement Management process

area

Institutionalize a Managed Process RUP also does not consider staff training because it focuses on the project rather than on broader organizational needs. Although it includes an activity involving staffing, it merely states that roles should be staffed with people who have the required skill-set, experience, and capabilities. Table 8 shows the practices under the CMMI Institute a Managed Process process area. Again, organizations using RUP can use this to supplement RUP guidelines. Table 8: RUP coverage for the Institutionalize a Managed Process process area

Conclusion
Although RUP covers 97 percent of the CMMI requirements for Maturity Level 2, it cannot actually perform at this level when implemented without any supplementary practices. That is because it does not include two Level 2 process areas, which cover essential best practices for project implementation. Of course, a project that uses RUP might fail for reasons having nothing to do with either of these two process areas, but supplementing RUP in the areas identified by the CMMI appraisal represents another step organizations can take to help prevent failure. Note that I am not urging managers to order the creation of dozens of documents and the execution of needless activities just to achieve CMMI

compliance. The goal of a project is ultimately to satisfy a client's needs, and frameworks such as CMMI are designed to help you achieve this. Execution of activities and the creation of artifacts should only happen in a real-world scenario when they are needed by the project. Fortunately, RUP is built upon an extensible architecture that allows you to use plug-ins to adapt the process to your needs. A good variety of existing plug-ins already do address some of the needs I have identified above. The COTS plug-in, for example, could be used for implementing the identified lack of supplier agreement management. You can also implement additions through a set of tools recently issued for the IBM Rational Method Composer solution. These allow tool-driven customization of the RUP and support plug-in development. You can use these tools for developing plug-ins based on this assessment, for other needs you identify, or to integrate RUP into an existing organizational process that covers the topics. Organizations might want to implement additions as separate plug-ins so they can later select process components as plug-ins out of their organizational process assets for addressing specific needs (e.g., what skills are required, whether COTS is an option, etc.) and the level of formality required for a specific project. Although RUP does require supplementation to achieve Maturity Level 2, I would still recommend it to organizations that strive for CMMI compliancy. Its open architecture allows you to reuse existing solutions, to address gaps in coverage, and to merge RUP relatively easily into your existing corporate processes. As a process model attribute, this openness is a critical factor because it allows seamless integration, which is a great advantage when communicating change processes and process improvement. RUP has the tools necessary for integration of subprocesses through plug-ins and customization already built in, and that is a huge advantage for any organization striving for the best development process.

Notes
1 2

Successful projects = those that are delivered on time and within budget. http://www.sei.cmu.edu/cmmi/results.html

About the author

As a consultant for IT process optimization, Markus Grundmann has worked at IBM Research and Development on grid computing and at Pinnacle Systems in the fields of object-oriented development and architectures. He completed an M.S. in business information science while working at IBM Rational on process integration. He can be reached via his Website at www.MarkusGrundmann.de