You are on page 1of 18

IKEA IT

Test Automation Service

User guideline
Revision: 1.3
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

Table of contents
1 INTRODUCTION .................................................................................................... 3
1.1 Purpose ................................................................................................. 3
1.2 Intended audience .................................................................................. 3
1.3 Definitions, Acronyms, and Abbreviations .................................................. 3
1.3.1 Definitions.......................................................................................... 3
1.3.2 Acronyms and Abbreviations ................................................................ 3
1.4 References ............................................................................................ 3
1.5 Open issues ........................................................................................... 3
2 TEST AUTOMATION .............................................................................................. 4
2.1 General information ................................................................................ 4
2.1.1 Purpose of test automation .................................................................. 4
2.1.2 Benefits of test automation .................................................................. 4
2.1.3 Objectives - What to automate? ............................................................ 4
2.2 TestCenter - Test Automation Service ....................................................... 4
2.2.1 Test Automation Request ..................................................................... 5
2.2.2 Test Automation Process ...................................................................... 5
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

2.3 Preparation before a test automation assignment ....................................... 6


2.3.1 Well documented test cases ................................................................. 6
2.3.2 Test cases executed manually with passed result .................................... 6
2.3.3 ALM project ........................................................................................ 7
2.3.4 Application access and demonstration .................................................... 7
2.3.5 Support ............................................................................................. 7
2.3.6 Test Data ........................................................................................... 7
2.3.7 Return of Investment (ROI) input.......................................................... 8
2.4 Good to know ........................................................................................ 8
3 SETUP COMPUTER AND USERS.......................................................................... 9
3.1 Setup computer where you should run QTP scripts ..................................... 9
3.1.1 Ordering QTP profiles .......................................................................... 9
3.2 Setting up ALM on a user computer .......................................................... 9
3.3 Create a ALM user .................................................................................. 9
3.4 Run tests directly from QTP (User access to QTP) ....................................... 9
3.4.1 Ordering QTP application access........................................................... 10
4 HOW TO UPDATE AND ADD TEST DATA ...........................................................11
4.1 Test Data stored in Excel ........................................................................ 11
4.1.1 How to update encrypted password ...................................................... 11
4.2 Test data in customized ALM fields .......................................................... 11
4.2.1 Display customized fields in the execution grid: ..................................... 12
5 TEST EXECUTION ................................................................................................14
5.1 ALM ..................................................................................................... 14
5.2 Schedule an execution ........................................................................... 14
6 EXECUTION RESULTS.........................................................................................16
6.1 Open logs ............................................................................................. 16
6.2 Navigating in the logs ............................................................................ 16
6.2.1 Analyze log files ................................................................................. 16
6.2.2 Examples .......................................................................................... 17
REVISION HISTORY ...................................................................................................18

Author(s) : Peter Johansson [PEJO] Page 2 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

1 Introduction
1.1 Purpose
This document describes what the Test Automation Service at TestCenter delivers and
how the service and its deliveries should be used. It also describes what an initiative
needs to prepare and think about when writing test cases that should be automated by
the Test Automation Service.

1.2 Intended audience


Target group is Test Managers, Test Specialists or other test personnel at IKEA that is
responsible for test execution and test case design and wants to use the Test
Automation Service.

1.3 Definitions, Acronyms, and Abbreviations


1.3.1 Definitions
Term Description
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

Test script Script created by QTP (or other automation tool) that is used to
run an automated manual test case. The test script is the
automated implementation of a manual test case.

1.3.2 Acronyms and Abbreviations


Term Short form for Description
QTP QuickTest Test Automation tool from HP used by the
Professional test Automation Service for development of
automated scripts and used by users for
execution of scripts.
ALM Application Test management tool from HP used at IKEA.
lifecycle Also used by the Test Automation Service for
management storing QTP test scripts and other associated
resources.

1.4 References
[1] <exchange this with a reference name>
<define where to find reference>

1.5 Open issues


None

Author(s) : Peter Johansson [PEJO] Page 3 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

2 Test Automation
2.1 General information
2.1.1 Purpose of test automation
To reduce the time and resources required for regression testing, thereby
speeding up the test execution and increasing the overall coverage of
regression testing.

2.1.2 Benefits of test automation


 Save time (to do more testing)
o Time and resources required for regression testing is reduced in both
development initiatives (projects) and the maintenance effort (running
services)
o Gives you time to test what’s important!
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

 Improve test coverage (and thereby quality)


o Repeating tests with various input data combinations improves the overall
coverage of regression testing.
o Testing not only correct values but also incorrect values and verifying
expected error messages etc.
 Remove tedious and repetitive tasks
 Possible to automate preparation and restoration of test data
 Monitor application or test environment state on a daily/weekly/monthly basis

2.1.3 Objectives - What to automate?


You can get the most benefits out of the automated testing effort by automating:
 Smoke Test
 BCS – Business Critical Scenarios
 Regression Tests
 Tests running on several different hardware or software platforms and
configurations
 Tests that take a lot of effort and time when doing manual testing
 Tests that are repeated with multiple data sets as input

2.2 TestCenter - Test Automation Service


The Test Automation Service at TestCenter delivers:
 Feasibility Studies: TestCenter investigates the possibility and benefits of using
Test Automation. (Time limited to maximum 40 hours)

 Assignments: Delivery of automated scripts (BCS, Smoke test, regression test


etc), ROI (Return of Investment) calculation and handover of scripts with
education in script usage and managing the Test Data for the scripts.

 Scheduled executions: Possibility for the users to setup schedule executions of


scripts once, daily, or weekly.

Author(s) : Peter Johansson [PEJO] Page 4 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

 Support and script maintenance: The Test Automation service owns and
takes full responsibility for maintenance/update of delivered scripts including
support in test script usage and test data handling.

 Tools & Framework: External vendors that want to use test automation can
use our framework and tools. (But will not receive any education or support on
tool usage)

 Script Reviews: Review of scripts delivered by external vendors. Evaluation of


scripts from a maintainability and reusability perspective.
The Test Automation service is a fully self sustained service. No budget is needed from
individual projects.
The service do not provide QTP education We only educate the users in how to use
the scripts we deliver and handling of test data, i.e. how to run the scripts from ALM,
read the QTP log files, how to schedule an execution and how to change/update the test
data. (If you need QTP education you need to go outside IKEA)

2.2.1 Test Automation Request


Test Automation Service – User guideline Revision: 1.1

The Test Automation Service can be requested through a mail to the TA service
Full generic template Revision: 4.1 Date 2009-05-11

coordinator (olof.ernstsson2@ikea.com) or with a TPR (Test Period Request) sent to


Test Desk.

2.2.2 Test Automation Process

Qualification phase
During this phase, a service request (TPR) is sent to the Test Desk (or through a mail to
the Test Automation Service coordinator); TestCenter decides if a Feasibility study is
needed based on previous experiences of the application/system.
An Initial Feasibility Meeting is held and a Pre-Qualifying Checklist is filled in together
with the project; all requested information is sent to TestCenter; the readiness of the
project for the Test Automation Service is then assessed.

Feasibility study
During the feasibility study phase the application is proven to work with an automation
tool; an automated Test Script is produced and a Feasibility Report containing a ROI
Calculation and a recommendations for an automation assignment is delivered to the
project.

Assignment planning
During this phase, the Test Case that should be automated should be defined and
completed. Execution result must be passed and each step and verification point in the
Test Case must be documented on a detailed level. Test Data is defined and provided by
the project. TestCenter reviews and approves the Test Cases and verifies that the
requirements for an assignment are fulfilled. An Assignment Description is created by
TestCenter, reviewed and approved by all stakeholders.

Script Development
During the Script Development phase, all Test Scripts, and Test Scenarios are created
and tested. The project provides necessary Test Data, application upgrades and support
to investigate and resolve any issues that come up and block the script development.
During this phase there is at least one handover meeting where the scripts are
delivered, reviewed and approved and the users receives education in how to run the
scripts, setup scheduled executions and read the QTP log files.

Author(s) : Peter Johansson [PEJO] Page 5 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

Reporting
During the assignment reporting phase, a Customer Satisfaction Survey is filled in by
the project. TestCenter delivers an Assignment Report, documents lessons learned and
investigates if the Service can be improved.

2.3 Preparation before a test automation assignment


Here are the Test Automation Service requirements that need to be fulfilled and some
recommendations on what you need to consider before starting an automation
assignment.

2.3.1 Well documented test cases


When you write test cases you should think about how the test should be executed
(which steps) and what to verify (expected results) so that the purpose of the test case
is fulfilled. The test case steps needs to be written on a detailed level so that the
automation specialist can understand how to execute it and what to verify in each test
step. The manual test cases should be stored in ALM and contain required pre- and post
conditions that needs to be fulfilled for each test case.
The pre-condition should specify in which state the application should be in, required
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

test data that should exist in the system and what actions that needs to be done before
starting the test case, i.e. “A user with role YY has logged on to the system and the
main screen is displayed. At least one new order exist” and post condition “Return to
main screen”
WHY: We do not know how your system/application works and therefore we need step
by step instructions on what to do and what to verify.
It is not clear enough to have a step that says “Create an order in system Z” or “Verify
that the order was sent to system X”. You need to specify how to create an order and
how and what to verify in system X:
“Fill in info_x in field_1, info_y in field_2 … and click button ‘Create order’, write down
the order number”
“Open System_X and navigate to ‘Show placed orders’ under Orders menu and verify
that the order number from system Z is displayed in list ‘New orders’ with the correct
information info_x, info_y…”
If test data is needed as input to the test case and to compare the expected results it
needs to be available, specified where it can be found or how to generate it. (See
[2.3.6])
How we work: Before we start an assignment we review the test cases and run them
manually to understand how to run it and what to verify. (If there are unclear steps or
verification points we need to contact the project for clarification.) Then we design and
create automated scripts that are based reusability, flexibility and maintainability.
The test cases are broken down into reusable components, such as Login, CreateUser,
VerifyXXXXX etc. which are easy to update if something changes in the GUI or test case.
If we find similar actions (like creating different kind of users) we try to build one
component that can handle both kinds of actions based on different input parameters.

2.3.2 Test cases executed manually with passed result


Before handing over a test case to the Test Automation Service they shall have been
executed manually by you with a passed result!
WHY: If a test case is not correctly described or can’t be executed in the system due to
defects or not implemented functionality it is very difficult to automate the test case.
Author(s) : Peter Johansson [PEJO] Page 6 of 18
Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

2.3.3 ALM project


You need to have a ALM project to be able to store and execute the automated scripts
delivered.
WHY: All automated scripts and associated resources are stored in ALM.
The scripts are placed together with the manual test cases (tab script) in the Test Plan
and all associated resources (Function libraries, reusable actions (components), test
data tables, object repositories etc) are stored in Test Resources.
IKEA Inside page that describing how to order a new ALM project through iDesk (SRM)

2.3.4 Application access and demonstration


The automation specialist needs to get access (user name and password) to the
required application/system/database in a test environment where we can develop the
scripts. If your test cases require actions to be made in other external systems then we
also need access to these.
It is always useful to have a demonstration of the application, including execution of
manual test case and explaining the purpose of the application, before starting an
assignment (or feasibility study) so that the automation specialist gets a look and feel
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

for the application.

2.3.5 Support
The automation specialist needs a contact person from the project that can answer
questions about the usage of the application/system, test cases and test data. Normally
this is a minor issue and does not require much time from the project resource, but if
the test cases are unclear or the application is not stable enough more time might be
needed.
WHY: If issues with the test cases or application usage are found, the time schedule for
the assignment is put at risk if we don’t get help to solve it quickly.

2.3.6 Test Data


The project needs to define and provide test data that can be used by the different test
scripts.
One question that needs to be sorted out is if the automated scripts can use the same
test data over and over again or if the test data is destroyed after each execution? Do
we need to create test data before or restore it after execution?
Maybe the Test Automation Service can create a QTP script that selects/creates the
required test data before or restores it after an execution?
QTP can handle Test data in different ways
 By default all test data is stored in an Excel file (Each QTP script contains an
Excel file for storing test data). This Excel file can be stored in ALM and easily
updated by the users of the scripts. (See [Test Data stored in Excel])
 QTP can connect to a database and select and/or create/restore test data:
Note: Database access and all required SQL statements needs to be provided by
the project.
o Select relevant available test data with an SELECT statement provided by
the project.
o Create/Insert relevant test data that can be used by the scripts
o Restore changed test data after script execution
Author(s) : Peter Johansson [PEJO] Page 7 of 18
Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

 Stored in other types of files such as XML, CVS etc


 Created from scripts executed on other external systems
 Input for each execution from customized fields in ALM (See: [Test data in
customized ALM fields])
You can combine these different methods and keep some test data in an
Excel file and get other parameters/data that is frequently changed (like
test environment URL etc) from ALM fields or get test data extracted from a
database.

2.3.7 Return of Investment (ROI) input


The project needs to provide input for TestCenter to calculate an estimation of the
Return of Investment for IKEA for a test automation assignment.
The project should provide an estimation of the following ROI input:
 Number of tests to automate
 Test cycles per year (How many times will theses test cases be executed per
year, including all releases, patches, reruns)
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

 Time to execute all tests that should be automated manually (man hours)
 Application Lifetime (year)

2.4 Good to know


With QTP you can automate the following actions
- Measure and verify transaction times (time to do a search, save, create etc)
- Send and receive payloads to Business Services (BS) on EBC’s
- Connect and interact with databases. SELECT, INSERT, DELETE etc (requires
access rights)
- Interact with terminal emulators like Putty
- Preparation of test data and system for testing
Other:
- For each verification QTP does, it includes a screenshot of the GUI in the log file
- You can get the test results in html-format (i.e. published on a Wiki page)
- CAPTCHA objects needs to have a static answer. (A CAPTCHA is a program that
protects websites against bots by generating tests that humans can pass but
current computer programs cannot.)
- The automation tool, QuickTest Professional (QTP), needs to be installed on the
same server as the application client. QTP can NOT work on a Citrix published
image of the application. (See [Setup computer where you should run QTP ])

Author(s) : Peter Johansson [PEJO] Page 8 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

3 Setup computer and users


3.1 Setup computer where you should run QTP scripts
QTP and ALM needs to be installed on the computer where you should run the QTP
scripts and to setup this you need to install a profile containing the required ICC3
modules.
On the computer where you want to execute test you need to order profile
“App HP QTP”, if it is a client or “App HP QTP_srv”, if it is a server.

3.1.1 Ordering QTP profiles


For IKEA.com: Order the profile thru ICC3 Software Catalogue (ISWC)
1. Navigate to module HP_QTP_11:
https://iswc.ikea.com/Details.aspx?id=10074
2. Select “Add to cart”
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

3. In the popup “Add to cart” select ‘App HP QTP’ and click OK


Note: The specified License cost and Yearly cost is not put on the ordering user. IKEA
owns these licenses and the figures are only displayed to show what each installation of
the application costs IKEA. We have a limited number of concurrent licenses so please,
do not use QTP without contacting the Test Automation Service at TestCenter.
For IKEADT.com (PTE, CTE & PPE environment)
Create an Rfc in iDesk for installing the server profile ‘App HP QTP srv’
or the client profile ‘App HP QTP’ on your client/server.

3.2 Setting up ALM on a user computer


To be able to read QTP execution reports (log files) and update test data stored in
Excel files in the Test Resources module in ALM on your local computer you will need to
install a QTP-addin to ALM (“HP_QTPAddinForQualityCenter_11.EN”).
This module is only needed if you don’t have the full QTP profile on your computer.
1. Open https://iswc.ikea.com/Details.aspx?id=10075
2. Order it without any profile. Select “No profile (install module only)” when
you are asked to select profile.

3.3 Create a ALM user


All users that should execute QTP scripts needs to have a user in ALM. To order a ALM
users send in a “Add user to ALM”-request via the Service Request Management module
in iDesk.
See guide on IKEA Inside for this here

3.4 Run tests directly from QTP (User access to QTP)


If a user, that doesn’t have QTP installed on his own computer, needs to start and
execute tests directly from QTP on a machine that has QTP installed, he needs to be
given access to the QTP application. By default he will not be able to see and start QTP

Author(s) : Peter Johansson [PEJO] Page 9 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

on the start menu on a computer that has QTP installed. (He hasn’t got the Application
Access to QTP).
If you have a user that needs access to QTP on a machine you need to grant the user
Application Access: MercuryQTP.
Note: This access is only needed when a user needs to be able to start and
execute tests from QTP and not from ALM. (Executing QTP tests from ALM does
not require QTP application access!)

3.4.1 Ordering QTP application access


 In IKEA.com send a request to your local helpdesk to grant specified IKEA user(s)
Application Access: MercuryQTP
 In IKEADT.com create an Incident (type request) in iDesk to grant specified
IKEADT user(s) Application Access: MercuryQTP and assign it to ACTWINTEL.
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

Author(s) : Peter Johansson [PEJO] Page 10 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

4 How to update and add Test Data


4.1 Test Data stored in Excel
1. Go to ALM and log in to your project
2. Change to “Test Resources” in the navigation bar on the left
3. In the resource tree go down to:
Resources -> Subject -> Projects -> [Project name] -> Data Table
4. Click on the Test Data (XLS) file you want to edit
5. Change to tab “Resource Viewer” on the right side
Note: If you don’t have “Resource Viewer”, can’t click it or if it’s empty you
need to order “HP_QTPAddinForQualityCenter_11.EN” from the ICC3 software
catalogue. Select it without a profile (see 1.2)
6. Click “Download” and select a local folder to save the excel to
Note: Before you are familiar with how it’s done make sure you save a copy
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

of your local file before you upload and overwrite the old file
7. Go to local folder and edit your locally saved copy and save.
8. Press “Upload File” and select your locally saved file. (IMPORTANT)

4.1.1 How to update encrypted password


Only valid if your QTP script use this feature!
If your password have changed you need to update the excel file with a new password
and to get the encrypted password follow these steps
1. Go to ALM
2. Go to Test Lab
3. Go to Root/automatic test set/Password
4. Press Run Test Set
5. First input your new password and press OK
6. Copy encrypted password and press OK
7. Paste the encrypted password in your Excel file

4.2 Test data in customized ALM fields


In some ALM projects custom fields have been added to be able to easily update test
data that changes often. These fields can be located under the Details tab in Test Set or
in the Execution grid.
Information about how to update test data and use customized fields should
be described in a document in the test data folder in Test Resources:
Test Resources -> Subject -> Projects -> [Project name] -> Data Table)

Author(s) : Peter Johansson [PEJO] Page 11 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

Example on custom fields in the tab Details:


Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

In the Execution grid:

4.2.1 Display customized fields in the execution grid:


1. Go to “Test Lab” and navigate to your Test Set
2. Select “Select Columns”

Author(s) : Peter Johansson [PEJO] Page 12 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

3. Add the specified custom columns:


Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

4. You then click on desired cell for each row (test case) and change value.
5. Important: Change to another Test Set and back (So ALM saves changes).
6. Run the Test Set. Your updated test data should now be used.

Author(s) : Peter Johansson [PEJO] Page 13 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

5 Test execution
To start execution you have two options to choice from. Either you can use ALM to start
your execution or you can use our “SchedulePlanner” (see 5.2 Schedule an execution)
tool which use ALM as well but add extra features. When execution is done you can view
results from ALM.

5.1 ALM
The benefit of using ALM to start execution is that you will see the execution and can
stop it if something goes wrong.
1. Login using remote desktop to the computer that you want to execute the
tests on. Open remote desktop (Start -> All Programs -> Accessories ->
Remote Desktop Connection).
2. If it is an IKEA.com computer, login using your normal IKEA.com login or
if it is a computer in IKEADT use your IKEADT user.
3. Open a browser and navigate to ALM
Test Automation Service – User guideline Revision: 1.1

http://iwww.alm.ikeadt.com/qcbin/start_a.htm
Full generic template Revision: 4.1 Date 2009-05-11

4. Go to Test Lab
5. Navigate to the Test Set you want to execute
1. To run all tests in the Test Set, click “Run Test Set”
2. To run selected tests:
1. Select the Tests you want to run by clicking on them and
holding the “CTRL”-key
2. Run selected tests by clicking “Run”
6. In the “Automatic runner” window, make sure “Run All Tests Locally” is
checked.
7. Click “Run All”
8. The execution starts
IMPORTANT: Remember to keep the remote desktop window open
during the entire execution. Don’t close or minimize the window as
that will tell Windows to stop updating the screen which might make
execution fail!

5.2 Schedule an execution


The “SchedulePlanner” tool was developed to solve one of the main drawbacks with
ALM, namely nightly executions. When using ALM you can only run executions while you
have your own computer up and running which limits execution to working hours.
Note: This tool also works perfectly fine for instant executions.
Scheduling an execution serves two main purposes
A)You don’t need to keep your own computer on during the entire execution. You
just start and then you are removed from the execution events.
B)You can schedule execution to start whenever you want and also have repeated
executions.
Author(s) : Peter Johansson [PEJO] Page 14 of 18
Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

1. Go to \\itsehbg-nt0007.ikea.com\Common\TEST_INSIDE\Test
Automation\Tools\SchedulePlanner and download SchedulePlanner.exe (Also
have a look at the manual)
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

2. On General tab fill in all fields. (Tip1: Use “Select” to select test set more
easily. You can also select several test sets to be executed after each other.
Tip2: You can send email to several emails by separating them with ;
(semicolon).)
3. On Schedule tab fill in time for execution (If you want to execute immediately
set at least 2 minutes in the future.
4. In RDP tab. If the Host you specified on the General tab is in IKEA.com then
you need to uncheck “Use default login” and change domain to
“IKEA.COM” and input your own login. (This is because default user is not
allowed to login to your IKEA.com computer)
5. Click on “Schedule”
When you see the success message in the status bar everything is setup and you can
close down the tool and your computer. When execution is completed you will get an
email depending on your settings.

Author(s) : Peter Johansson [PEJO] Page 15 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

6 Execution results
6.1 Open logs
1. In your ALM project, navigate to your Test Set
2. Select test your want to read report for
3. Click on Launch report

(Note: if this button isn’t visible click on this button in the bottom right
corner of the window)

6.2 Navigating in the logs


Test Automation Service – User guideline Revision: 1.1

When you first open up the report you will get an overview of passed/failed/warnings. If
Full generic template Revision: 4.1 Date 2009-05-11

test has failed you can fast forward to next failed by pressing Find Next (Alt+N) in the
toolbar.
Normally when an error has happened you will on the next step see a screenshot. If you
get “Run error” that means that an unanticipated event has happened. This can for
example be that a popup dialog has appeared which blocks execution. When this
happens the rest of the test will most likely fail and to see what actually went wrong you
might need to scroll down in the log to the next screenshot and see what state the
application is in.

6.2.1 Analyze log files


A step in the log can have a couple of statuses and clicking on a step will give you a a
more detailed description on what has been done

icon description

This is a verification that something worked as


expected

This is a verification that failed, click on the step to


see expected and actual result

A warning could occur for a number of different


reasons, either generated by QTP or put there by
developer to highlight something. Click on the step
to see what caused it

A done step is used to give information on what has


or should be done

Information step, often gives information on what

Author(s) : Peter Johansson [PEJO] Page 16 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

should be done

If logging for this hasn’t been turned off there will


be a row like this for each action performed

When QTP can’t find an object it will use a


technique called smart identification to try and
guess which object should be used and when that
happen this icon will be put in the log. This is a
good indication that script should be updated to
uniquely identify each object

When there is a script error of any kind this error


will be displayed. It is not uncommon that several
Run error will follow so solving the first might fix
all. When looking at the detailed description of the
run error you will see what line number caused it
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

Indicates a recovery scenario has kicked in. If


errors occur after this it could be caused by the
recovery scenario closing something that shouldn’t
be closed

6.2.2 Examples

Run error

Sometimes you can get run errors that are cause by errors in the application that hasn’t
been predicted and there for no error handling is done. It could be lack of test data that
prevents a selection, popup that blocks the script etc. In this example it happens to be
just that when a picture should be selected there is an error dialog (caused by
connection to picture share is down). Since error message isn’t closed it because several
Run errors in the log, if option to save screenshot on error is turned on you will see a
screenshot on the next step in the log that might help identify what the problem is.

Author(s) : Peter Johansson [PEJO] Page 17 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc
Test Automation Service Revision: 1.3
User guideline Revision date: 2013-08-28

Revision history
Describe the changes made and the reason for the change between the revisions.
Revision Date Description of changes Author
1.0 2010-11-13 Document approved TA Team
1.1 2011-09-02 Updated with general information about Peter Johansson
automation, service requirements and
preparation.
1.2 2013-02-05 Updated contact information and some Olof Ernstsson
links
1.3 2013-08-28 Updated links and changed to ALM Olof Ernstsson
Test Automation Service – User guideline Revision: 1.1
Full generic template Revision: 4.1 Date 2009-05-11

Author(s) : Peter Johansson [PEJO] Page 18 of 18


Internal Approver: TA Team
File name: Test Automation Service - User Guideline 1.3.doc

You might also like