Transforming Mobile

Handset Testing
How Accenture’s new ROSA Testing Framework delivers
High Performance by reducing costs, enhancing quality
and protecting brand reputations.
Mobile handset manufacturers and network operators are under severe pressure
to rethink their approaches to product testing. As mobile communications
technology becomes increasingly sophisticated and complex, current testing
strategies leave them vulnerable to spiraling costs, diminishing product quality
and threats to their brand reputations.
What’s driving these challenges? With
mobile technology maturing and opening
up to third-party developers, handset
platforms are incorporating a new
generation of applications. No longer
simply a “talk and text” device, this
intelligent companion is now enabling
fully-fledged entertainment capabilities,
financial services and enterprise
mobility.
Given the growing list of specifications
associated with these mobile devices,
the demands on product testing are
rapidly increasing as well. To achieve
high performance, manufacturers,
carriers and other stakeholders
are challenged to produce high-
quality products that meet the rising
expectations of their customers without
missing critical timelines or allowing
testing costs to explode. Accenture
research confirms this trend as
application testing costs are likely to rise
from 2 percent of overall testing costs
to as much as 30 percent under present
circumstances.
2
To smartly and strategically address
the challenges around cost, quality
and reputation, companies will need
to explore new ways of managing and
executing product testing. Existing
testing approaches tend to be
fragmented, costly and labor intensive.
Recognizing these challenges, Accenture
has developed a comprehensive new
testing framework and approach, which
is known as ROSA*. This holistic and
systematic strategy is designed to help
product development and validation
organizations cut overall testing costs
by 30 to 50 percent and improve time to
market by as much as 20 percent while
increasing the volume of release testing
by as much as 25 percent.
Such gains will soon be critical to
success. If companies are to generate
profitable and sustained growth in
today’s hypercompetitive mobile arena,
they’ll need to ensure their products
cost-effectively meet the increasing
rigors and expectations of this
demanding global market.
Market Drivers: Three
Key Trends in Mobile
Communications
The communications world is rapidly
beginning to embrace fourth generation
(4G) technologies, while opening up
new mobile device innovations through
open source applications and platform
foundations such as Symbian and
Android. This trend is expected to
offer users a wider range of advanced
services while achieving greater network
capacity. What are the market drivers
encouraging handset manufacturers and
network operators to rethink how they
are approaching product testing?
Industry challenges in testing today’s
devices are linked to three key trends:
unprecedented application growth,
global product launches and the
uncompromising demands of customers.
Unprecedented Application Growth.
Convergence leads to applications
that can no longer be tested in a
simple and easily controlled network
environment. Applications, for instance,
*Testing by Remote access, Offshore testing, Simulated
network testing, Automation
are now enabled by connectivity to
financial services, content providers,
points of sale and location based
services. They also impose new security
requirements. All these capabilities and
specifications add complexity to the test
environment—raising the bar associated
with assuring and validating necessary
performance levels.
The application could be a gaming
device using Java and 3D hardware
accelerators. It could be a health service
that integrates content from various
sources, provides useful tips on wellness,
analyzes the user’s health profile and
measures key health indicators via
Bluetooth. It could be a Web 2.0 service
enabling social networking and access
to enterprise-based applications. While
the possibilities are staggering, the
challenge of cost-effectively testing
these proliferating applications to ensure
they perform to spec are eclipsing
the existing test capabilities of most
companies.
Global Product Launches.
Whereas manufacturers may have had
the luxury of introducing new mobile
handsets region by region in the past,
they are now expected to launch them
worldwide—mandating a global launch
strategy. Considering that managing the
economics and efficiencies of a global
product launch is very different from
what has been experienced in the past,
most manufacturers and operators view
this as a new challenge.
What does it mean for manufacturers?
Network configurations and behavior
are not necessarily similar across
geographies. Local operators launch
services that are relevant and unique to
the local market. This not only includes
language support features but also
services that are used by operators as
differentiators. They may not be able
to identify many performance failures
until after the launch has occurred and
the product hits the market. What does
it mean for operators? Their reputations
are on the line as they roll out services
that are expected to meet certain
performance levels.
Unfortunately, manufacturers and
operators will have far less time and
opportunity to test their complex
offerings—region by region—in an era of
global product launches. If they remain
wedded to existing testing approaches,
their vulnerabilities will quickly become
evident to their customers.
The Uncompromising Demands
of Customers.
Unsurprisingly, customer expectations
remain exceedingly high in terms of
their experience with these products.
Indeed, user experience has become
a key driver in the success of product
launches. As underlying technologies
and platforms are commoditized,
manufacturers begin to focus on user
experience and user interfaces to
differentiate their handsets.
The integration of user interfaces on
multiple platforms further increases
testing challenges. As handsets grow to
support business-to-consumer, business-
to-business and business-to-employee
applications, users expect performance
to match what they’ve experienced with
their laptops and personal computers.
They expect typical applications like
Skype, YouTube, Adobe Flash to be
available at similar performance levels
on their devices. They also want the
ability to download media content
and perform actions on content such
3
as editing, sorting and sending. And,
finally, they expect their handhelds to
have an intuitive design that is highly
user friendly. Clearly, performance
expectations are immense—and growing.
As convergence, innovation and market
pressures continue to drive new device
specifications and higher customer
expectations, testing strategies must
inevitably be transformed.
In previous years, testing for mobile
communications devices was far more
deterministic. What needed to be
tested was a lot more well defined.
The complexity of testing was most
often attributed to communication
technologies that essentially were
standards driven.
In the sweep of a decade, the industry
has shifted from voice quality as a
differentiator to an era defined by
the introduction of exotic third-party
applications and content. In this new
environment, application testing is
becoming an enormous challenge and,
for some, a barrier to continued growth.
4
As open-source platforms invite a wider
set of application developers into the
market, standardization has become
extremely difficult. Device testing is no
longer a deterministic endeavor. Indeed,
application testing costs grew from 3
percent of overall testing costs in 2004
to ~20 percent of overall costs in 2008.
This number is expected to increase over
the next two years to more than 30
percent of overall costs.
Smart and strategic testing is
now essential if companies are to
reduce these costs, deliver expected
quality levels and defend their
brand reputations in an unforgiving
market. These are the issues that both
manufacturers and network operators
must now confront.
Business Challenges:
What Manufacturers and
Operators Now Face
Handset manufacturers and network
operators both face significant testing
hurdles in today’s fast-moving market
environment. However, their specific
testing challenges differ slightly.
Testing Challenges Facing
Manufacturers:
Handset manufactures spend millions
of dollars testing a device from
development to product launch. This
testing comes in two phases:
1. Primary testing (or ”hygiene”
testing) involves taking a handset
through various certifications that
are typically performed or provided
by accredited test houses. This
includes tests related to Global
Certification Forum (GCF) and
Personal Communication Service Type
Certification Review Board (PTCRB),
Federal Communication Commission
(FCC) marking, electromagnetic
compatibility testing (EMC), Radio
Frequency (RF) Performance,
Bluetooth® and WiFi, and Microwave
Communications. While primary
testing is demanding, the cost, quality
and schedules for such testing are
relatively predictable. They bring in
low to moderate challenges with
growing device complexity.
2. Secondary testing (or field testing)
is the key area of concern for
manufacturers. Device complexity
comes from various factors including:
• Media-rich content demanding high
data throughput that is reliable and
sustained seamlessly when the mobile
user roams globally.
• Interoperability with contemporary
devices over short range
communication technologies.
• Concurrency of application interfaces/
interactions with multiple servers and
subsystems, adding a large number of
test cases to a test suite.
• Testing in multiple locations before
launch.
While handset manufacturers have built
test strategies for traditional testing
needs, they rely largely on location-
bound testing, moving test teams to
testing locations (in various geographies)
and coordinating between locations
for extended periods of time on defect
resolution and product stability.
Alternatively, they may rely on manual
methods that render them incapable
of running automated test scripts to
quickly and efficiently run through
testing scenarios.
Testing Challenges Facing Operators.
While operators may spend less
than manufacturers on testing, their
investments are nevertheless extensive.
An operator typically is involved in the
device testing process in three phases:
1. Early engagement testing—In this
stage, network infrastructure for
advanced features is beginning
to stabilize. New applications and
services that would potentially be
key differentiators for carriers are
extensively tested and proven.
2. Prelaunch device and service
testing—As the operator gets nearer
to launch, product validation teams
are forced into “abridged” testing or
“risk-based” testing. This approach
potentially is the cause for most
of the postlaunch defects. User-
experience testing is conducted to
determine the perception of the
service on the device by the target
demographics. Any mismatch between
user expectations and the quality
of service must be identified at this
point.
3. Postlaunch service testing—Operators
continue to test multiple devices that
deliver their services to the end user
for sustained performance and service
quality.
While some large operators perform
a significant part of testing in labs
with a captive network and other
infrastructure, simulation and
automation tools, they continue to
face challenges related to the quality
of testing, meeting product launch
schedules and the resultant cost
implications.
Most manufacturers and operators
today remain vulnerable to rising costs
and expanding risks because they lack
a cohesive test strategy. Rather, they
have tended to put in place approaches
that address their testing challenges
in fragmented ways. They may use
remote testing services in some cases.
They may offshore testing to service
providers in certain instances, again
promising limited savings or geographic
testing coverage. They may use limited
simulation capabilities. They may use
automated systems to handle test cases
more productively.
But they tend not to use all of these
approaches or apply them in any
systematic and integrated fashion.
Nor do they have the global testing
capabilities necessary to productively
apply these solutions across geographies
as global product launches draw near.
In the absence of new strategies that
redefine how testing is managed, it is
increasingly clear that manufacturers
and operators can only hope to meet
market expectations through linear
growth in their testing operations.
They would need to hire more testing
personnel, deploy them more widely
and ask more of them than has ever
been asked before. Not surprisingly,
testing costs would be expected to rise
tremendously as companies scramble to
meet the service levels their customers
expect.
Should they fail to meet these service
levels, they run the risk of undermining
their brand reputations—a deadly
possibility in today’s hypercompetitive
markets. Alternatively, they may miss
crucial deadlines and allow market
windows to close—another potentially
fatal blow in markets that often revolve
around seasonal product releases.
To avoid these consequences,
manufacturers and operators must
now rethink their strategies for product
testing. They’ll need to find new and
integrated testing approaches that can
address their looming cost, quality and
brand reputation risks.
New Methodology for
Mobile Handset Testing:
Accenture ROSA Framework
High-performance businesses in the
mobile communications field are now
embracing a new approach to product
testing—one that offers a comprehensive
and systematic way of reducing costs,
raising quality and bolstering brand
reputations. It’s called the Accenture
ROSA Framework.
Drawing on the perspectives of
strategy leaders, senior management
and function heads in mobile
communications, Accenture Product
Software Solutions devised this
transformative framework for handset
testing in today’s increasingly complex
and demanding marketplace.
The ROSA Framework is a combination
of four approaches:
• Remote access
• Offshore testing
• Simulated network testing
• Automation
While none of these individual
approaches is new to most companies
in the sector, manufacturers and
operators now have an opportunity to
achieve their cost and quality objectives
by applying these approaches in an
integrated and holistic fashion. This
transformative approach dynamically
and selectively applies each of these
four elements to optimize investments in
terminal testing.
This approach represents the difference
between a suboptimal point solution
and an optimized solution that is fully
integrated. By capitalizing on this
systematic strategy, Accenture estimates
(see figure 1) that product development
and validation organizations can:
• Cut overall testing costs by 40 to 50
percent over a three year period.
• Improve time to market by as much as
20 percent.
• Increase the volume of release testing
by as much as 25 percent.
Note: Estimated cost reductions are
derived from Accenture analysis of
industry and will vary largely based
on the extent of remote, offshore and
simulation and automation that are part
of the current strategy of a client.
5
 Figure 1: ROSA Framework Drives Significant Cost Reduction
ost Reduction
t Reduction Ha
Hand
15-25% 35-45% 40-50%
35-45% 40-50% Estimated Quality
5-25% Estimated
Down Down
Cost
Down Down Down
Reduction Estimated Quality andand
own Estimated Cost Reduction Productivity Benefits
Productivity Benefits
15-25% 35-45% 40-50% Estimated Quality and
A A B B
15-25% Down
Down 40-50%
35-45% Down TestTest
Estimated Casesand
Quality
Productivity Benefits
Cases
Down Down Down Productivity Benefits AM
Ma
A
First
rst Present Test Cases
Year
ear Present
Cost Test Cases
Cost
Second First
Second
Year
Year First
Year
Br
Brow
Third
Third
Year
Year Second Functions
Year
Second
Year
Functions
Year Third
Third
Year Device
Functions Device M
Man
Year Functions

Volume
Volume Time
Time Fea
Featu

Volume Time
Volume Time
Cost Reduction
st Reduction from
from 15-50%
15-50%
r period
period
Graduate Cost Reduction from 15-50%
Graduate
over 3 yearCost Reduction from 15-50%
period Device
Device Feature
Feature NoN/W ToF ToF
NoN/W TfR TfR ToS ToS
over 3 year period Active
Active Idle Idle Screen
Screen 20%20% 20%20% 50%50% 10%10%
Analog
Analog TV TV 90%90% 10%10%
Applications
Applications 10%10%Device
30%Feature
30% 40%40% NoN/W
20% 20% ToF TfR ToS
Browser
Browser Active
Device
10%10% Idle
10% Screen30%30%
Feature
10% 20%
NoN/W
50% 50% 20%
ToF 50%
TfR 10%
ToS
Camera
Camera Analog
Active
50%50% 10% TV10%
Idle Screen10%10% 20%
30% 30% 90%
20% 10%
50% 10% 20%
20%
Camera
Camera Applications
Analog
30%30% 30% TV30% 30%30% 10%
10% 10% 30%
90% 40%
10% 20%
e to
to acess
acess and
and test
test device
device fromfrom Content
Content Download
Download 10%10%Browser
Applications
30%30% 40%40% 20% 10%
10%
20% 10%
30% 30%
40% 50%
20%
eocation
location Data Synchronization 60%Camera
Browser 10% 10%10% 20% 50%
10%
20% 10%
10% 10%
30% 30%
50% 20%
TfR Data Synchronization 60% 10%
mom Remote)
Remote) TfR
Possible
Digital
Digital
to acess and test device from Digital TV
Rights
Rights Management
Management 10%10%Camera
Camera 20%20% 60%60% 10% 30%
50%
10% 30%
10% 30%
10% 10%
30% 20%
Digital TV 10%10%Content
Camera 70%Download
70% 20%20% 0% 10%
30%
0% 30%
30% 40%
30% 20%
10%
Possible location
Remote to acess and test device from DualDual Data Synchronization 60% 10% 10% 20%
Consider
Remote
(Test the
from value
location
Remote) of the following four •
Keyboard
Keyboard W
 ith minimum
40%40%Content10% effort,
Download
10% 10%test scripts
10% 40% 10%
40%
can
30% 40%
of defects
20%
can be identified. The
W DualDual
ScreenScreen
be ported
60%60%Digital
Data5%
from
Rights
5% Management
Synchronization
one
25%25% 10%
device
10%
to
60%
10% 20%
10%
another,
60%
10% 10%
20% 35%
35%
elements in the
(Test from ROSA framework:
Remote) DualDual
SIMSIM 0% 0%Digital
Digital
30% TV
Rights
30% Management
60%60% 10%10%
10%
10% 70%
20% 20%
60% advantages
0%
10% of such testing include:
lone Testing
ne Testing Email related which
functions helps confirm
Dual
DigitalKeyboard
TV10% 30% overall
30% 60% advantage
40%
10%
60% 10%
70% 10%
20% 40%
0%
Email related functions 10%
twork
work Required)
Required)
Remote Testing. Formats over time. • Easy
40% debugging environment for 35%the
NoN/W File File
Formats 70%70%Dual
Dual Screen
Keyboard 20%20% 10% 60%
40%
10% 5%
10% 25%
10% 10%

Through
NoN/W this method,
Standalone Testing devices are testedFlashFlash
RSS RSS Dual
Dual30%SIM
Screen
30% 30%30% 40% 0%
60%
40% 30%
5% 60%
25% product
10%
10% development team to35% localize,
HTTPHTTP Offshore10% Testing.
10%Email
Dual10% related
SIM 10% functions
10%10% 70% 0%70% 10%
30% 30%
60% 60%
analyze
10%
and fix defects as well as
withinStandalone
(No aNetwork Testing
particular Required)
geography under a Language
LanguageA large part of
25%25%File
EmailaFormats
10% handset
related
10% functionstest40%
25%25% 70%40% 10% 20%
30% 10%
60% 20% 20% resolution.
sted
ed in in
(No Network Required)
communications
controleed/Simulated
controleed/Simulated carrier. However, testsLocalLocal Connectivity
Connectivity 70%70%Flash RSS10% 10%10% 10%
File Formats
10% 70%
10% 30% 30%
20%
verify
40%
10%
the defect
specification does not require a live 30%• M70%echanism to reproduce a defect
nment are conducted from a remote location Location
ment Location Based
Based Service 10%10%
Service HTTP
Flash20% RSS20% 60%60% 10% 10%10% 10%
30% 10% 40%

n Simulation) TbS
Simulation) MailMail network. These HTTPare
Language
20%20% network
20%20% independent
25%
10%
60% 60% 10%
10% 25%
10% 40%
observed
70%
in the field that can20%
by accessing
TbS Testedthe
Best keys of the device MediaMedia
in controleed/Simulated pLayer
pLayer 70%70%Local
LanguageConnectivity
10% 10%10% 10% 70%
25%
10% 10%
10% 10%
25% 10%
40% 20%
tests that10% can be10% run on the device otherwise not be reproduced in the
Best
the Tested
overenvironment
Internet in controleed/Simulated
and monitoring the Messaging
Messaging 10%Location
Local20% Based Service
Connectivity
20% 30%30% 40%10%
70%
40% 20%
10% 60%
10% 10%
10%
environment
(Test on Simulation) Navigate
Navigate manually.25%
condition
condition This
25% type
Mail
Location
10%10% of Service
Based test
15%15%can be
50% moved
10%
50% 20%
20% 20%
60% real
10%network environment.
60%
responses on the device screen of the NewsNews
(Test on Simulation) Casting
to
Casting locations where
10%10%Media
Mail60%
tester
pLayer
60% availability
30%30% 0% 70%0% and
10%
20% 10%
20% 10%
• Software
60%
25%stability
25% ensured by building
tester’s computer. The advantages of Povisioning Povisioning 20%20%Messaging
Media 30% 30% 50%50%
pLayer 10%
70% 20%
10% 30%
10% 40%
10%
nonField
Fieldinevitable
inevitable RadioRadio costs are low. Navigate
90%90%
Messaging condition
10%10% 25%
10% 10%
20% 15%
30% a test library and continuously
50%
40%
n Field) this approach are multifold:
Field) RingtonesHowever, 50%
Ringtones
offshore
50%News
Navigate solutions can
Casting
condition 25%also 60%
10% 10% 30%
15% enhancing
0%
50% it with test cases that
ToF 25%
• A Testing
tester can testinevitable
RSS RSS
Reader
a large number of Settings Reader Povisioning
News30% 30% 40%40% 30%
Casting 20%
10%
30% 30%
60% 50%
30%
verify
0%
defect resolutions.
ToF on Field Settings be defined 90%in
90% terms of having10%
Radio
Povisioning access
20%
10% to
90%
30% 10%
50% 25%
device
Testing
(Test features
on on Fieldin
Field) multiple locations Screaming
inevitable Screamingfield testers within Ringtones
Radio 10%10% any40% given
40% 50%geographic
50%50% 90% 10% Automation.
(Test on Field) without traveling to Subscreen
simultaneously Subscreen 50%50%RSS Reader
Ringtones 20%20% 30% 50%
30% 30% 40%
It is 30%
essential to fully leverage
area. Service providers
Settings
with 10%
a90%
global 10%
the location. Virtual teams across theTest Editors
Test Editors 60%60%
RSS 10%
Reader
10% 10% 30% 40% 30%
automation when designing a test
Utilities network of
Utilities talent
30%30% 30%and
Screaming
Settings 30% 20%testing experience
20% 20%90%
20% 10% 40% 50%
10%
Accenture Global Delivery Network VideoVideo 40% framework for handset testing.
Telephony
may have theSubscreen
Telephony
ability to conduct
20%20% 10%10% 70%
Screaming 50%
tests10%
70%
in
10% 20% 30%
50%
test devices and report results, Test Editors
Subscreen 60%
50% 20% 10%
30%
Whatever the means of testing,
helping to reduce costs significantly. various geographical
Utilities
Test Editors locations,30%sparing
60% 30%
10% 20% 20%
10%
the manufacturer
Video or carrier the30%costs20%
Telephony
Utilities of
30% 20% automation
10% 70%
20% dramatically increases the
• Scripts are created easily by playback
flying experts
Video Telephony
to the field location
20%
and
10%
quality
70%
of testing results. It enables
of a user scenario that is performed
setting up relevant field tests. testers to run thousands of test
once on the device. Quality and
consistency of testing can be cases unmanned. It enables testers to
Simulated Network Testing. execute a test library, allowing them
confirmed as a result. This method of testing is used for
• Test execution can be automated by to conclude that a defect resolution
new feature testing when network has taken care of all possible scenarios
a single command that runs a test
infrastructure is not yet ready for in which a defect can potentially
library at a predetermined time and
logs the results in a log file. Compared network tests. Using a simulated occur. Such results are impossible to
to manual testing, the volume of tests network, features can be tested. The achieve through manual testing. Testing
that can be executed in a given time method provides a network condition automation, in other words, is a key
is far larger. under which a reasonable number factor in confirming product stability.

6
7
Process Transformation
In a world of proliferating features,
there is no sensible and cost-effective
way to manage the testing challenge
using conventional, feature-driven
testing approaches. Testing teams
would merely grow in a linear fashion,
adding significant costs while slowing
down the testing operation.
To confirm success with the ROSA
approach, companies must begin to
organize around skill sets and test
categories instead of product features.
In order to redesign the testing process
(see figure 2), it is first necessary to
classify a handset test specification into
the following categories:
Test from Remote (TfR): All tests that
can be performed by remote access.
Feature or functionality tests can all be
tested by remote access while mobility
tests and tests that verify performance
quality may not be possible by remote.
No Network Required (NoN/W): All
tests that can be performed standalone
without needing to connect to a
network. Most tests for the phonebook,
media player and similar features fall
under this category.
T est by Simulation (TbS): All tests
that can be performed in a simulated
environment. Most features that can be
tested over an ideal network condition
and do not require complex interfaces to
various services fall under this category.
Tester on Field (ToF): All tests that
must be performed in the field. This
assumes the tests must be performed by
a tester physically operating the device
in a real network condition.
All test cases and test plans can be
coordinated using these test categories
or methods—an option that radically
simplifies testing approaches for
manufacturers and operators.
Organizing for Change
Typically, product-testing teams are
organized by function or feature,
working across multiple handset models.
The ongoing addition of features and
functions merely increases the number
of teams and resources necessary to
meet certain performance benchmarks.
Companies, however, can enable a
significant increase in efficiencies
and cost optimization by reorganizing
teams based on the ROSA framework.
By organizing around ROSA testing
categories, they gain the ability to
group all testing activities within testing
categories (see figure 3).
Reorganizing teams in this fashion helps
drive testing productivity by enabling
the focused training and professional
development of teams. For instance:
• When no network is required,
standalone test teams require
very little knowledge of device
internals. Training can be focused
on relevant features, which enables
the standardization of techniques
for coverage, failure and concurrent
behavior.
• Remote testing requires training on
capabilities in scripting, automation,
defect identification and reporting.
Under this approach, tool licensing
can be optimized and knowledge of
multiple testing tools can be imparted
efficiently, which may require higher-
degree technical skills as a tester that
can be developed in a conventional
manner.

Figure 2: ROSA Framework Drives Savings in Key Testing Categories

Device Feature NoN/W ToF TfR TbS

Active Idle Screen 20% 20% 50% 10%
Analog TV 90% 10%
Applications 10% 30% 40% 20%
Browser 10% 10% 30% 50%
Camera 50% 10% 10% 30%
Bluetooth 30% 30% 30% 10% 20%
TfR Content Download 10% 30% 40% 20%
Possible to access and test device from Data Synchronization 60% 10% 10% 20%
Remote location Digital Rights Management 10% 20% 60% 10%
(Test from Remote) Digital TV 10% 70% 20% 0%
Dual Keyboard 40% 10% 10% 40%
Dual Screen 60% 5% 25% 10%
NoN/W Dual SIM 0% 30% 60% 10% 35%
Standalone Testing Email related functions 10% 30% 60%
(No Network Required) File Formats 70% 20% 10%
Flash RSS 30% 30% 40%
HTTP 10% 10% 10% 70%
TbS Language 25% 10% 25% 40% 20%
Best Tested in controlled/Simulated Local Connectivity 70% 10% 10% 10%
environment Location Based Service 10% 20% 60% 10%
(Test by Simulation) Mail 20% 20% 60%
Media Player 70% 10% 10% 10%
Messaging 10% 20% 30% 40%
Navigate condition 25% 10% 15% 50%
ToF News Casting 10% 60% 30% 0% 25%
Testing on Field inevitable Provisioning 20% 30% 50%
(Test on Field) Radio 90% 10%
Ringtones 50%
RSS Reader 30% 40% 30%
Settings 90% 10%
Screaming 10% 40% 50%
Subscreen 50% 20% 30%
Test Editors 60% 10% 10%
Utilities 30% 30% 20% 20%
Video Telephony 20% 10% 70%

Figures are representative of the categorization as an average of the various device test specifications across organizations that
Accenture has worked with.

8
 • Simulated network testing would
require a strong understanding of
network behavior as well as the
ability to use low-level scripting
languages, work with USB and other
connectivities on the device, and
automate testing. Team members
must have the experience and skills to
debug logs on the network side and
reproduce user scenarios observed in
the field to enhance defect resolution
significantly.
• Field testing will require skills in
network maps, managing devices
and simulations. Testers must
report defects in an intelligent and
informative way to developers at the
home office and reproduce typical
scenarios that are critical but not
repeatable. Such skills can be built
through a structured approach, and
knowledge transfer can be managed
efficiently through appropriate
training and project management.
While these principles of organizational
and process redesign can be applied
within a product development and
validation organization, manufacturers
and network operators are increasingly
looking to outside specialists for several
key reasons.
Figure 3: Organizing Around testing Catergories
Handsets
A B
Mail
Browser
Functions
Device Management
Time Feature n
20%
35%
20%
First, specialist service providers can
rationalize their investments in tools,
knowledge and experience across a wide
customer base. Second, their focused
commitment to such solutions enables
them to continually develop new skills
and experience—remaining on the
leading edge of testing solutions. And,
finally, they can offer global testing
coverage by remotely testing products
and by deploying personnel in regions
worldwide.
Accenture’s Assets and
Experience
Accenture offers an example of the
types of capabilities and expertise that
must now be deployed to deliver high
performance in terms of handset testing.
T est Libraries. Accenture has device
test libraries with more than 6,000
test scenarios based on experience
with carrier test specifications, GCF
field trials guidelines and in-house test
development.
C D
Handsets
S imulator Labs. The firm’s Third
Generation (3G) Network Simulator
enables testing in a controlled network
environment. Early detection of
defects, feature testing and release test
automation are all handled through
these labs.
Global Remote Test Labs. Accenture
hosts remote test labs across its Global
Delivery Network and client locations,
which enables benchmarking and helps
to optimize device testing in terms of
time and cost.
Solution Enablers. Accenture has made
an array of strategic investments in
its partnerships and alliances, which
provides know-how on commercial
test infrastructures and knowledge of
standards and test specifications.
Such capabilities—which manufacturers
and operators should expect from their
testing solution providers—represent
the foundations of an advanced testing
infrastructure. By deploying and
leveraging such assets, companies can
cost effectively master the complexities
and challenges posed by the new
generations of mobile devices.
Test Teams
NoN/W ToF TfR TbS
Model 1
Model 2
Model 3
Model 4
9
Conclusion: Holding on to
the Tiger’s Tail
There’s a famous Indian fable that
illustrates the challenges handset
manufacturers and network operators
now face. It’s called The Tiger’s Tale.
Imagine you are in the forest and you
come across a bright and beautiful
strand of rope in the bushes. You reach
out to grab it, but quickly realize you’ve
grabbed the tail of a tiger. You dare not
let go lest you be eaten by this ferocious
animal. And yet, you know you are in for
a wild ride.
So it goes with the companies
confronted with testing today’s mobile
handsets. Expectations are greater
than ever. Costs threaten to spiral
out of control. Quality is in danger of
diminishing. And should you lose your
grip, consumers will turn on you like an
angry tiger.
10
You simply cannot let go. You must The demands on handset manufacturers
provide more features, functions and and network operators are clearly
capabilities without allowing price higher than ever. They’ll need to
to rise excessively or quality to fall rethink product testing and ensure
below expectations. If you fail, you will they have the right partnerships if
be eaten. It’s simply critical to meet they are to reduce costs, maintain high
consumer expectations and deliver on quality levels and defend their brand
quality benchmarks. reputations. Indeed, they’ll need to take
transformative steps if they are to meet
Handset manufacturers and network
today’s extensive demands and achieve
operators seeking a testing solution
high performance.
provider that can help them address the
rigors and demands of today’s mobile
device market should apply several
criteria. To achieve high performance,
look for a solution provider with:
• Long-standing relationships with
manufacturers and carriers.
• Deep industry experience in handset
testing.
• E xtensive knowledge of device
and service testing methods and
processes.
• Global delivery capabilities.
• End-to-end testing framework to
deliver cost effective, high-quality
test solutions.
So it goes with the companies confronted with testing today’s mobile handsets.
Expectations are greater than ever. Costs threaten to spiral out of control.
Quality is in danger of diminishing. And should you lose your grip, consumers
will turn on you like an angry tiger.

11
About Accenture
Accenture is a global management
consulting, technology services
and outsourcing company, with
more than 176,000 people serving
clients in more than 120 countries.
Combining unparalleled experience,
comprehensive capabilities across
all industries and business functions,
and extensive research on the world’s
most successful companies, Accenture
collaborates with clients to help them
become high-performance businesses
and governments. The company
generated net revenues of US$21.58
billion for the fiscal year ended Aug.
31, 2009. Its home page is
www.accenture.com.
Contact us
For more information on achieving
high performance through innovation
and testing go to www.accenture.com/
productsoftware or contact your local
representative about how Accenture
can deliver high performance through
product software and testing solutions.
Please contact:
Risto Valtakari
Executive Director
Product Software Solutions
Tel: +358 205725 508
Contributed by
Venugopal Ramakrishnan
Accenture Product Software Solutions

Copyright © 2010 Accenture

All rights reserved.

Accenture, its logo, and

High Performance Delivered
are trademarks of Accenture.