Professional Documents
Culture Documents
Mobile Automation
Justin O’Mahony
Automation Engineer
Niall Conroy
Senior Automation Engineer
Roger Ashwood
Head of Automation Service
ERIC MURPHY
Automation Engineer
eric.murphy@sqs.com
JUSTIN O’MAHONY
Automation Engineer
justin.omahony@sqs.com
NIALL CONROY
Senior Automation Engineer
niall.conroy@sqs.com
ROGER ASHWOOD
Head of Automation Service, Ireland
roger.ashwood@sqs.com
Contents
Management summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Solution components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Solution benefits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Open Source Mobile Automation 5
Management summary
Given the increasing pressure on companies to before shipping the app for live release, so testing,
deliver apps to market in short times, the testing of in effect, is not executed against the app that is
these apps is becoming a huge financial overhead. released.
Updates to mobile operating systems and greater
competition between device manufacturers to This paper focuses on a mobile automation solution
produce the latest new devices is taking its toll on that was delivered for a large client with a require-
mobile testing as there is a more frequent need for ment to deliver a native mobile app for both the iOS
regression testing across an ever-expanding range and Android platforms going to market at the same
of devices. time. The solution implemented was based on open
source tools such as Selenium [1] and Appium [2]. It
Many of the mobile automation solutions that exist enabled tests to be written that could be run across
currently are expensive and generally require tests multiple devices simultaneously, reducing time to
to be written separately across the platforms being market and man hours of manual test effort, and
tested. This increases potential duplication and main- increased testing coverage across devices that would
tenance issues between automation solutions for otherwise not be tested due to time constraints. The
Android and iOS. Additionally, most tools require solution proves that it is possible to build a robust
the app to be instrumented, involving the injection mobile automation solution in which the same set of
of specific configurations into an app in order to scripts can be run across both platforms and also
integrate with the automation tool in use. This is not in which instrumentation is not required.
ideal as the injected configuration must be removed
Introduction
The increasing trend among companies to move This paper reviews the current state of the mobile
to agile practices enabling faster time to market, app market, delves into the many challenges of
has resulted in automation becoming a key part of mobile automation and then looks at how these
the testing process when building mobile apps. The challenges were addressed by the choice of auto-
relative immaturity of the market and proliferation of mation techniques and tools used in the solution
devices and platforms results in specific challenges. presented.
Open Source Mobile Automation 6
1,500 k
iOS App Store
1,250 k Google Play
750 k
500 k
250 k
0
2010 2011 2012 2013 2014
◦ The device selection process for manual and ◦ Developing the solution in such a way that it
automated testing handles cross-platform functionality
◦ The use of emulators versus physical devices ◦ Version control process of the app and auto-
mation solutions
◦ The use of cloud-based services to test
against real devices or building in-house ◦ Continuous integration and efficient scheduling
capability of test runs
Component Description
Selenium Selenium is by far the most popular open source web testing tool available, with a vast
number of contributors, both commercially and individually.
Page • The framework used a Page Factory design pattern. iOS Calendar
Factory This is a useful feature because discrepancies in the
app across both platforms can be handled by the
page logic, which allows for the same element to be
identified across platforms, even if it has a different
Android Calendar
locator in each platform.
• The diagram on the right side shows how similar
objects are set out very differently across iOS and
Android.
Component Description
Dashing & • Reports are created using the ReportNG tool; these are then emailed to the relevant
ReportNG project stakeholders after each run.
• Results are also posted to a central dashboard using Dashing, a Ruby-based plat-
form which displays nightly run results, as well as other information about the run.
The solution developed is designed in such a way that elements in the app were labelled so that they
that it combined all the above components to allow could be identified by the automation tools. Com-
for a situation where a test can be written once and munication with the test team was vital to decide
tested across different devices and platforms. what tests should and would be automated.
The app was developed in an agile environment. Figure 2 shows the path of an automated test run
This ensured that communication with all stake- from end to end and how it integrates with the
holders, most importantly developers and testers, client’s continuous integration process.
was paramount. Developers assisted by ensuring
Build Tool
Test Framework
3. Framework fetches the
latest version of the apps
GUI Automation Tool
References
[1] http://www.seleniumhq.org/
[2] http://appium.io/
[4] LiveMint (2015). Flipkart moves towards becoming app-only platform. http://www.livemint.com/
Industry/J9VeQxowSOlHU8ZMUParUL/Flipkart-moves-towards-becoming-apponly-platform.html
Open Source Mobile Automation 13
© SQS Software Quality Systems AG, Cologne All liability of the contributors, the editors,
2015. All rights, in particular the rights to the editorial office or the publisher for any
distribution, duplication, translation, reprint possible inaccuracies and their consequences
and reproduction by photomechanical or is expressly excluded.
similar means, by photocopy, microfilm or
other electronic processes, as well as the The common names, trade names, goods
storage in data processing systems, even in descriptions etc. mentioned in this publication
the form of extracts, are reserved to SQS may be registered brands or trademarks, even
Software Quality Systems AG. if this is not specifically stated, and as such
may be subject to statutory provisions.
Irrespective of the care taken in preparing the
text, graphics and programming sequences, no SQS Software Quality Systems AG
responsibility is taken for the correctness of the Phone: +49 2203 9154-0
information in this publication. Fax: +49 2203 9154-55
info@sqs.com | www.sqs.com