2011

SUPPLIER SPOTLIGHT

Year Founded: 2007 Website: www.utest.com uTest community: 41,200 Headquarters: Southborough, Massachusetts Leadership: Doron Reuveni, CEO & CoFounder Ann Brady, Chief Financial Officer Matt Johnston, Chief Marketing Officer Fumi Matsumoto, Chief Technology Officer Roy Solomon, VP Product Management & Co-Founder Reuven Fein-Barsegian, VP Engineering John Montgomery, VP Project Delivery Marc Weinstein, VP Global sales Exhibit 1 Geographic distribution of testing resources
Source: Everest Group

Overview
In our times of quick information dissemination through various instant consumption channels such as social networks, blogs, and interest groups; an organization, especially in the B2C segment, can hardly afford to have defects in its user facing applications. Every organizations brand image and business are directly tied to the quality of its offering. Moreover, given that organizations today work at a global stage, yet with a local flavor, the need for a distributed testing environment that can leverage testers from multiple regions, languages, skills, and acumen is increasing. Increasingly, testing service providers are leveraging their global sourcing model to deliver cost effective testing services to their clients. However, a quick analysis of their sourcing strategy demonstrates a significant focus on selected geographies. Moreover, the execution of testing in a traditional testing set-up revolves around testing lab where the testing resources perform test execution on their standard infrastructure, dedicated to testing.
Others Eastern Europe (1%) Latin America North America EMEA (excluding Eastern Europe)
8% 3% 3% 13% 14% 21% 80% 3% 12% 4% 4% 14% 2% 11% 4%

China (1%) 1%

India

58% 43%

Overall

MNCs

Tier-1 Offshore

However, in real life, an application rarely gets used in isolation. Therefore for certain applications, especially large user base applications, this kind of testing may not suffice due to its limitation in mirroring the real life application usage. This is where a company such as uTest brings its crowdsourcing innovation to provide testing services. The company is a market place for testers who provide their services to uTest to execute a testing project that uTest has sourced from its clients. The inherent flux in a crowdsourcing model, where various testers possessing different skills, languages, testing infrastructure, testing acumen, and approach, provides the needed in-the-wild testing of end user applications.

Everest Research InstItute EGR-2011-X-W-XXXX

www.everestresearchinstitute.com

SUPPLIER SPOTLIGHT

Company Profile
uTest was founded in 2007 and is headquartered at Southborough, Massachusetts. The company is a leading software testing service provider and caters to clients across industries such as retail, hi-tech, media & entertainment, telecom, financial services, manufacturing, and healthcare. The company's most important investors include Scale Venture Partners, Longworth Venture Partners, Egan-Managed Capital, Mesco Ltd., and Massachusetts Technology Development Corporation. Crowdsourcing in testing Traditionally, developers had only two options viz. in-house QA or buying outsourced testing services. uTest offers an alternative approach to testing via crowdsourcing for the buyers. Crowdsourcing enables the company to offer 'in-the-wild' testing across OSs, locations, languages, and industries. By using a crowdsourcing Software-as-a-Service model for providing testing services, uTest has been able to produce an innovative and cost efficient solution. The company has created a worldwide community of software testers who facilitate coverage for most environments (OS, browswer, firewall, handset, carrier network, expertise, etc.) and demographics (age, sex, location, etc.) The crowdsourcing model provides uTest with a set of distinct advantages: Wide testing coverage: uTest's pool of 40,000+ testers offer a range of skills/expertise to meet the QA needs of companies Reduced time-to-market: Custom bug-reports from the uTest community can be received in 48 hours, thereby, bringing down the time-to-market Size and growth Exhibit 2 uTest revenue growth The company grew at a CAGR of 532 percent over 2008 to 2010.
uTest Growth in Revenues Revenue Index = 100 for FY 2008
12,000

Source: uTest

4,000

1,000
100

FY 2008

FY 2009

FY 2010

FY 2011 (E)

It has a network of about 41,200 members in its testing community across multiple geographies. Its testing community is expected to grow by approximately 1,000 members per month.
Everest Research InstItute EGR-2011-X-W-XXXX

www.everestresearchinstitute.com

SUPPLIER SPOTLIGHT

Key Services The company's testing services can be classified under: Functional testing Testing across OS & browser, ISP, firewall, and antivirus Usability Testing Survey-based usability and UAT Load testing Live load, synthetic load, hybrid load, and performance testing Security testing Focus on black box security testing Localization testing Localization validation across locations and languages The company provides testing services for the following: Web Applications: The company tests web applications across multiple platforms, environments, and locations. Mobile Phone Applications: The company tests mobile applications across carriers, handset language, and locations. Applications covered include iPhone apps, Android apps, Blackberry apps, Windows apps, and iPad apps Gaming Applications: uTest validates gaming applications by testing across multiple OS, browsers, plug-ins, hardware platforms, and environments. Desktop Applications: uTest provides testing and validation services for desktop software companies Key Market Segments Unsurprisingly North America accounts for 80 percent of uTests revenues, being the largest IT service and technology market. However, the interesting aspect is uTest generates significant revenue from enterprises that are not into core technology. Normally for newer companies, especially with very different business models, hi-tech industry is the predominant revenue generator given its propensity to experiment, shorter time-to-market, frequent application releases, and significant importance for testing.

Everest Research InstItute EGR-2011-X-W-XXXX

www.everestresearchinstitute.com

SUPPLIER SPOTLIGHT

Exhibit 3 Market segments by geography and industry

Distribution of revenues by client geography Percentage

Others EMEA (excluding UK
10% 10%

Distribution of revenues by client vertical Percentage

Healthcare/ Pharma Manufacturing 5% 5% Financial 10% services Telecom
10%

CPG/retail
25%

Source: uTest

80%

North America

Media & entertainment

20%

25%

Hi-tech

However, for uTest even CPG/retail, media, and telecom are big segments. Moreover, other traditional outsourcers such as financial services and manufacturing, also contribute reasonable revenues. uTest Community Given the crowdsourcing nature of its business, the contributing testers are spread across the globe,though, a majority of the testers come from India and the U.S. The company does not leverage cost arbitrage model and the pay-outs to the testers (of similar skills) are same across the globe, unlike traditional providers. Exhibit 4 Geographical segmentation of contributing testers
Source: uTest uTest community by geography Percentage 100%=41,200

ROW

24%
China Austraila 1% 2% Canada 2% Ukraine 2% 3% Israel 4% Russian Federation 6% UK

India

29%

27%
U.S.

Everest Research InstItute EGR-2011-X-W-XXXX

www.everestresearchinstitute.com

SUPPLIER SPOTLIGHT

uTest divides its testers under gold, silver, bronze, and general badges. Each tester under similar badge receives the same payout irrespective of the geographical location of the tester. Large part of uTest community works during its rest hours as shown below. Most of their contributing testers are full time employees at some other place and therefore work for uTest after their office hours. Apart from the monetary incentive, this also shows the dedication and commitment of these testers towards their profession. Exhibit 5 Segmentation of contributing testers
Source: uTest uTest community by employment status Percentage 100%=41,200 uTest community by educational qualification Percentage 100%=41,200

Employed part time Others

6%2% 8%

Unemployed & not looking

(2%) Some high Associate degree 2% school High school diploma 3%

7%

Work from 10% home

15%

40%

Employed full-time

Some college 9%
47%

Bachelors degree

Contract work

32%
18%

Graduate degree

Currently job-seeking

Concluding remarks Though there appear to be challenges for crowdsourcing model to grow, such as difficulties to leverage it in business and domain heavy applications, buyers lack of knowledge of this model, their intent to interact with fewer providers, and scaling up of the crowd model, its benefits cannot be neglected, wherever the model is applicable. As application testing acquires more importance due to shorter application development cycle, reduced budgets (do right the first time), constantly evolving technology dynamics, impact of social networking on brand, need for localization, global usage of applications, and a wide crowdsourced testing resource network will be an asset. However, buyers are not enthused about uTest just because it leverages the crowdsourcing model. They expect their work to be executed efficiently, cost effectively, in a timely manner, and with quality, business model is the providers prerogative. uTest stands to benefit from new paradigms given the head start and testing resources it possess. Though it needs to educate the buyers, develop a comprehensive suite of offering, maintain its differentiators, and execute its vision effectively.

Everest Research InstItute EGR-2011-X-W-XXXX

www.everestresearchinstitute.com

SUPPLIER SPOTLIGHT

Case Study
Client overview

Microsoft was founded in 1975. The company develops and markets software, services, and hardware. The company's software products and services include: Operating systems for PCs, servers, and intelligent devices Business solutions applications High-performance computing applications Software development tools Video games Microsoft is headquartered at Redmond, Washington, U.S. and has more than 100 offices globally.

Engagement overview
Shie Erlich, Testing Manager, Microsoft on: Increased testing coverage "We wanted to test across every possible angle, which is impossible if you're relying solely on your in-house team. We needed exposure in areas like South America and China, and uTest enabled us to achieve this with relative ease" On crowdsourced software testing "It was really quite impressive to watch bugs and feedback come in from all over the world. The theory works".

Client situation and objectives: The company wanted to test procedures for its free anti-virus software 'Microsoft Security Essentials. The company wanted to have an expanded testing coverage as it acknowledged the fact that the testing procedures in place were not sufficient on their own. Microsoft chose uTest for expanded testing coverage in January 2009. Scope of the Testing Project: Number of Testers: >100 Geographic scope: >10 countries Testing type: Exploratory App type: Desktop OS: Multiple versions of Windows

Rationale for selecting uTest

The Microsoft Security Essentials team needed to validate the product's use in countries such as Brazil, China, India, Russia, the U.S. and Israel. In essence, the team wanted to understand how the software performed outside the lab environment. For instance, Chinese users are not dependent on a single anti-virus program, thereby, creating a need for Microsoft to verify if its product functions concurrently with other such applications. Through uTest, the Microsoft Security Essentials team was able to add such testers who had the specific OS, browsers, and plug-ins required. With the help of a team from uTest comprising of a project manager and QA professionals, the Security Essentials team was able to launch an app on an aggressive schedule (The free anti-virus was launched in July 2009). Further, post completing 5 distinct and different test cycles, the team from Microsoft was able to achieve a greater degree of testing coverage. The uTest community helped Microsoft to review the software from a tester's point-of-view. The early reviews of Microsoft Security Essentials were impressive. As per reviews, users favor this product as there are no serious bugs/usability issues.

Key benefits and results

Everest Research InstItute EGR-2011-X-W-XXXX

www.everestresearchinstitute.com

SUPPLIER SPOTLIGHT

About Everest
Everest Group is an advisor to business leaders on next generation global services with a worldwide reputation for helping Global 1000 firms dramatically improve their performance by optimizing their back- and middleoffice business services. With a fact-based approach driving outcomes, Everest Group counsels organizations with complex challenges related to the use and delivery of global services in their pursuits to balance short-term needs with long-term goals. Through its practical consulting, original research, and industry resource services, Everest Group helps clients maximize value from delivery strategies, talent and sourcing models, technologies, and management approaches. Established in 1991, Everest Group serves users of global services, providers of services, country organizations, and private equity firms, in six continents across all industry categories. For more information, please visit http://www.everestgrp.com and http://www.everestresearchinstitute.com.

Everest Research InstItute EGR-2011-X-W-XXXX

www.everestresearchinstitute.com