PMI Virtual Library 2011 Jane Ann Pighin

Benchmark for Success!

By Jane Ann Pighin, PMP

ue to continual changes in technology and conditions in the marketplace, IT services should be benchmarked every 18 to 24 months. This will encourage competitiveness and identify improvement opportunities for both insourced and outsourced services. Additionally, benchmarking is a tool used to show executives and customers that IT services are cost effective and well managed. The term benchmark can mean different things. There are three types of benchmarks, which are generally used by IT (Table 1). The competitive assessment benchmark is the most commonly used and will be the focus of this article. Competitive Assessment Overview Competitive assessments assess the prices paid for externally provided IT services and/or the cost of internally provided services against marketplace peers that offer similar services and service levels. Competitive assessments identify strengths and actionable improvement opportunities for cost, quality, and productivity. An independent company, benchmarker, specializing in benchmarking, analyzes collected data and

compares them with a reference group, which is typically made up of five or six companies, similar in size, complexity, and quality for the benchmarked service. Selecting a benchmarker who will treat all parties fairly and objectively and assure meaningful results is critical; otherwise, the results will not be taken seriously and resulting actions will not be taken. Market and internal business conditions, supplier contract re-competition dates, and contract expiration dates are factors when determining the timing of a benchmark. A re-competition can ensure market competitiveness at a certain point in time; however, other mechanisms (e.g., a competitive assessment) should be used to ensure competitiveness over the life of the contract. Project Structure A benchmark should be managed as a project. A WBS (work breakdown structure) and schedule should be created and a project manager assigned to provide task, schedule, and risk oversight. The benchmarking process has five basic steps: 1. Preplanning: Identify a dedicated and autonomous core team that can provide oversight and ensure schedule

Benchmark Type Best Practice Sharing Macro Level Analysis Competitive Assessment

Description Peer-to-peer exchanges, leadership boards, and community of practice teams A portfolio level analysis (e.g., comparing company IT spend as a percentage of revenues with those of other industries and companies) A granular review of internal services, plus a market price assessment of those services provided by external suppliers to ensure competitiveness. Benchmarks identify strengths and actionable improvement opportunities for cost, quality, and productivity.

Table 1: The three types of benchmarks.

adherence. Determine the benchmark scope (e.g., what is included and what is excluded) and reporting views (comparison breakdowns). Create data collection templates. Identify key milestones and create a project plan. Benchmarking can be complicated and time consuming. It is important to break down the work into logical, manageable parts; thus, phasing benchmarks by technology or service group will encourage success. The service provider should be involved in the benchmark from the beginning. Executive support is critical to show workload prioritization for team members. Benchmark dates should be negotiated with service providers. Outsourced services are analyzed separately from retained services because of the different cost elements involved. An externally sourced service should have a benchmark clause in the contract at the unit price level. A benchmark clause is necessary to ensure that marketplace assessments are performed against externally sourced services. If the benchmark results show that defined marketplace targets were not met, the price must be adjusted retroactively. Some suppliers embrace benchmarking as an opportunity to demonstrate value. Other suppliers may be skeptical that a like comparison is possible and try to discredit the process and make the benchmark clause non-binding; they may argue that it is too expensive and time consuming, isnt a fair comparison, and may harm your relationship with them. A good benchmarker, however, is an expert at ensuring a fair and valid comparison by using a robust process with normalized data, thereby mitigating these concerns. The project manager must obtain agreement from team members and management on the benchmark scope. A matrix can be used to document what service areas are included and excluded in the benchmark study. Upfront consideration should be given to ability, time, and difficulty to include or exclude certain data. For example, if classified areas are to be excluded, can you reasonably break out its data from the total organization? 2. Data Collection: Collect labor and non-labor costs, volume, and service level agreement (SLA) data on predefined templates. A sub team of data collectors who can devote adequate time during the Data Collection step (approximately 1 to 1 1/2 months) must be selected. This sub team should include subject matter experts from the service provider, finance, and the sourcing organization. It is important that team members, as well as

management stakeholders, understand the benchmarking process and methodology and commit to the schedule. The benchmarkers terminology must also be fully understood; this will improve process efficiency and reduce the potential for error due to misunderstanding of data collection questions. The benchmarker must also fully understand the customers environment and take all relevant factors into account; this will lead to the selection of a well-matched reference group. The Data Collection step is the most time-intensive and critical step for the project manager. Without accurate and representative data, the benchmarker will not be able to adequately perform a marketplace assessment. Frequent statusing is necessary to ensure that data collection focals can meet the schedule. The project manager will also serve as a liaison between the benchmarker and service provider. It is important that questions from the data collectors are documented and quickly answered. In this step, the benchmark scope is sometimes adjusted if certain question areas are too difficult to answer. 3. Data Validation: Validate that the collected data is reasonable and without obvious errors. All team members and stakeholders must believe in the datas accuracy and integrity. During this phase, clarification of collected data is made. After data is gathered and submitted, the benchmarker will validate the data through a series of iterative reviews with the internal and/or external service provider. Anomalies in the data are sought, which may indicate an error in either the understanding of the question or a mistake in keying in the data. This is the first step in making sure that the data is organized and accurately recorded in a fashion that will be comparable with peers when a reference group is selected in the next step. Data must be comparable in size, scope, and complexity. The project manager must ensure quick resolution of any anomalies. 4. Analysis: A like for like reference group is selected and used to compare with the collected data. Often, the reference group companies are in a different industry. What is important is that the IT service performed by the selected companies is similar to the IT service being benchmarked in size, scope, and complexity. The benchmarker uses mathematical models for comparison of the benchmarked service to the reference group.

PMI Virtual Library | www.PMI.org | 2011 Jane Ann Pighin 2

It is imperative that the benchmarker has access to robust data for comparable companies. In addition to cost comparisons, the benchmarker may identify contractual shortcomings, technological inefficiencies, and operational/process inefficiencies based on best practices and lessons from selected peer companies. The project manager and core team have little to do during this step. The benchmarker may ask questions that were not discovered during the Data Validation step; however, few questions should arise at this stage. The benchmarker should provide the project manager with a weekly status report to ensure schedule adherence. 5. Findings: Benchmark findings showing measurements based on the key performance indicators (KPIs) of cost, quality, and productivity are captured in a report. Specific recommendations for improvements are included, along with estimated cost savings. Benchmark findings and recommendations must be clear and in an actionable format. Benchmarking is successful whether it demonstrates a need for cost reductions or validates competitiveness of current pricing. The project manager must ensure that all findings and recommendations are well understood by the stakeholders. The project manager also needs to work with the benchmarker to make sure the findings are properly communicated. Although the results cannot be altered, the way the message is delivered can be. The service provider will believe that the benchmark results are accurate and fair only if the benchmark process was objective and conducted with transparency, consistency, and data integrity; otherwise, the relationship with a supplier and/or trust of internal employees may be impacted. Recommendations can include: price reductions incorporated into a contract/agreement, contract re-competitions, operational best practices, and technology enhancements.

It is important that careful review and analysis be given to each of them in order to determine which should be implemented. A feasibility study can help determine the next steps. Once the benchmark has been completed, an action plan needs to be put in place and followed over the upcoming months and years so that savings are realized and recommendations are implemented. Summary The cost of a competitive assessment for one IT service varies depending on the scope, number of views, and complexity, although it can cost several hundreds of thousand of dollars. So, why spend the money and invest the time in benchmarking? If the service provider understands and trusts in the benchmarking process, the potential benefits should significantly outweigh the costs. Savings reaped from implementing the benchmarkers recommendations are often ten times or more than the cost of the benchmark. Additionally, the service provider gains facts and data about the competitiveness of its IT service, which will help run its business. Even if the service is competitive, a benchmarker should provide recommendations for improvement. As long as benchmark findings are thoughtfully considered and the agreed-on next steps are followed, benchmark success will be achieved!

About the Author Jane Ann Pighin, PMP, is an IT sourcing specialist at The Boeing Company in Seattle, Washington, USA. In this role, she helps develop and implement sourcing strategies for IT infrastructure services; she also leads competitive assessment benchmarks, which have produced significant savings over the years. Prior to this role, she was a senior IT project manager at Boeing, supporting major enterprise infrastructure deployments. In addition to her PMP, she is a Standford Certified Project Manager and has a Masters Certificate in Project Management from Stevens Institute of Technology.

PMI Virtual Library | www.PMI.org | 2011 Jane Ann Pighin 3