Future of Testing Impact of Mobile Devices

Somenath Nag 24th March, 2012

Copyright 2011 Calsoft Labs

Future of Testing Impact of Mobile Devices

Over last couple of years, mobile devices have shown a phenomenal growth, at the same time PC industry is on a declining path. Due to this, we are experiencing a paradigm shift on how applications are built, tested, and used by the end users, and has a potential to create a disruption in the traditional way of software development and QA. Though it started with consumers, enterprises are also embracing mobility more and more, especially after the stupendous success of iPad. At the same time enterprises are also facing challenges in the area of provisioning, data management, device management, and security. Mobile devices are also used differently compared to traditional computing Somenath Nag platforms. Due to this mobility devices and platforms throw up a new kind of challenges to testing fraternity. Calsoft Labs, with its Director - Business unique competence and experience of working with leading Development & Marketing, Software and Hardware companies, has been in the forefront of Calsoft Labs mobility. Calsoft Labs Mobility & Testing practices have been working together for some time to build methodologies, processes, http://in.linkedin.com/in/somenathnag and frameworks to address the challenges arising because of the above mentioned scenarios and to create a new paradigm in application and product testing.

Copyright 2011 Calsoft Labs

Content

How is Mobility Changing the World

Market Opportunity

Challenges in Testing Mobility Applications Addressing the Challenges

Copyright 2011 Calsoft Labs

Can we think of a day in our life without cellphone?

COMMUNICATION

UTILITIES/ TOOLS

PRODUCTIVITY

ENTERTAINMENT

Copyright 2011 Calsoft Labs

Same is true for enterprises also

Execs

Line of Business Managers

Task and Business Users

Consumers and Ecosystem

Mobile E-Commerce Mobile Marketing Self-Service

Copyright 2011 Calsoft Labs

Enterprises are adopting mobility faster than ever

Numbers and types of devices are changing dramatically 1 billion smartphones and 1.2 billion mobile workers by 2013 Large enterprises expect to triple their smartphone user base by 2015
Source: Gartner, IDC

Mobility is driving the consumerization of IT 46% of large enterprises supporting personally-owned devices2 Billions of downloads from Apple App Store

Increasing demand for enterprise applications 20% of mobile workers are getting business apps from app stores today 50% of organizations plan to deploy mobile apps by 2012

Source: Gartner, IDC

Copyright 2011 Calsoft Labs

When every industry is worried about recession, mobility is growing at a supersonic speed

585,000+ Apps 25,000,000,000+ Downloads

400,000+ Apps 10,000,000,000+ Downloads

Copyright 2011 Calsoft Labs

The unique challenges in testing mobile applications

Diversity in Device Management Application Usage UX Design Network consideration

Application & Device security Rapid Application Testing

Copyright 2011 Calsoft Labs

Challenge 1 - Diversity in Device Management

Different application platforms
Each platform has its own standards, application interface, Programming languages Needs to test applications on each platform for each features Different certification process for different platforms Increases time, cost, and effort

Device factors
No standardization of device & form factors New innovations creating more complexities Following is a list of the factors that can affect your application behavior:
Screen size Android OS versions CPU

Copyright 2011 Calsoft Labs

Challenge 2 - Paradigm Shift in Application Usage

PC Application

Mobile Application

Indoor use
Application dependent Data entry through key-board

Anytime, Anywhere
Context Sensitive Complex data entry process (key-

pad, touch, voice, etc.)

Low or zero interruptions High interruptions

Low environmental input

High environmental input

(Camera, voice, etc.)

Information Processing through

Information processing through

normal funnel

an inverted funnel

Copyright 2011 Calsoft Labs

Challenge 3 Evolving Usage Patterns and innovations are creating new testing challenges
No established mobile software usability culture different for different platforms Direct usage of usability principles used in PC based applications Impact of smaller form factors on usability Handling interruptions Interaction with other native and third party applications (camera, voice, music players) Complexity created by gestures, multi-touch and screen rotation
Copyright 2011 Calsoft Labs

Challenge 4 - Network Considerations Unlike the network landscape of the PC environment, the network landscape of a mobile device may have gateways Applications should be tested for their compatibility with WAP-enabled as well as HTTP-enabled devices Unpredictable network latency leading to inconsistent data transfer speeds. Gateways in a wireless network may act as data optimizers that may result in decreased performance for heavy traffic

Copyright 2011 Calsoft Labs

Challenge 5 - Application and Device Security Threat Model

Courtesy: OWASP Project

Copyright 2011 Calsoft Labs

Courtesy: OWASP Project

Challenge 5 - Application and Device Security Risks that Need to be Tested

Top 10 Security Risks

Insecure Data Storage

Improper Session Handling

Weak Server Side Controls

Security Decisions Via Untrusted Inputs

Insufficient Transport Layer Protection

Side Channel Data Leakage

Client Side Injection

Broken Cryptography

Poor Authorization and Authentication

Courtesy: OWASP Project

Sensitive Information Disclosure

14
Copyright 2011 Calsoft Labs

Courtesy: OWASP Project

Challenge 6 Rapid Application Testing

In order to deliver the benefits of faster time to market, RAD environments are used for mobile application development. Since the time taken for development is reduced by the introduction of RAD tools, builds will be available for testing much earlier RAD methodology imposes an indirect pressure on testing teams to reduce the testing cycle time without compromising quality and coverage This pressure becomes more when applications are targeted for multiple platforms simultaneously
Copyright 2011 Calsoft Labs

A Mobility Testing Strategy Framework for addressing these challenges

Network Landscape Device Landscape Usage Scenarios Performance Benchmark Application Category

Security Consideration (WOASP)

Automation Consideration

Test Strategy

P
O

Data Centric Apps Device Capability based Apps

Test Execution
Use a Weighted Device Platform Matrix method to identify the most critical hardware/ platform combination to test Check the end-to-end functional flow in all possible platforms at least once. Conduct performance testing, GUI testing, and compatibility testing using actual devices. Measure performance only in realistic conditions of wireless traffic and user load. Select the right automation test tool
Copyright 2011 Calsoft Labs

Suggested Guideline for Tool Selection Support for ALL object identification methods Secure solution Ability to test devices taken out of a retail store - No Jailbreak/Rooting Support for all device functionalities such as gestures, security alerts, wake and reboot Plugs-in to existing test environment Same test should run on different devices

Copyright 2011 Calsoft Labs

Q&A

Thank You
Copyright 2011 Calsoft Labs

About Calsoft Labs & Alten

Calsoft Labs

ALTEN SA

Wholly owned subsidiary of ALTEN

Established in 1988

Outsourced Product Design arm of the ALTEN Group

Pioneer in Product Engineering Services

European Leader in Engineering and Technology Consulting services

14800 employees worldwide, >90% engineers and consultants Located in 16 countries Revenue of 1 Billion+ for FY 2011

Over 300 satisfied customers

Over 1000 products developed and tested

Service Offerings
Embedded Product Engineering Service Software Product Engineering Service Independent Verification & Validation Service Design Engineering Service

Industry Focus
Telecom Healthcare Education Enterprise Software Automotive Media & Entertainment Semiconductor Aeronautics / space Financial

People
15000+ people 65% of engineering team with 5 years+ experience 27% Master Degree holders ALTEN is one of Europe's most dynamic employers, hiring almost 2,000 new people each year.

Locations
Located in 16 countries 10 Delivery centers Design & Development Centers in Bangalore, Mysore & Chennai US Delivery Centre in Boston, MA

Our Partners

Copyright 2011 Calsoft Labs

Calsoft Labs Testing Practice

200+ dedicated test professionals Track record

Partner

QA solutions throughout the SDLC based on TMMi framework

Reusable repository of test cases, scenarios, and validation libraries
Mobility Testing
Field testing and Interoperability Carrier validation test Protocol and Conformance test Device Management Testing Mobile App testing Usability Testing

Service Offerings
Test Strategy Consulting Embedded Product Testing Software Product Testing Test Automation On-Demand Testing Compliance testing Security Testing

Industry Focus
Telecom Healthcare Education Enterprise Software Automotive Media & Entertainment Semiconductor Aeronautics / space Financial

Thought Leadership
OWASP Compliance Accelerator for OWASP Compliance Home-made Open Source Automation Framework

Copyright 2011 Calsoft Labs