CONFIDENTIAL

QTP BDD Keyword Framework

Date:

March 2014

Version:

v01

Authors:

Richard Ecott

v01 R.Ecott

Page 1/37

27/12/2016

CONFIDENTIAL

Document History
Version
v0.1
V0.2

Status/Change
New Document
With BDD examples

Date

Reason for Distribution

February 2014

Initial Draft

March 2014

For review

APPROVED AT:
APPROVED BY:

[Sponsor]

Signature:

Date:

Distribution List:
Name

v01 R.Ecott

Role

Page 2/37

Title

27/12/2016

CONFIDENTIAL

Table of Contents

1.

Introduction...................................................................................................5
1.
2.
3.

2.

Keyword Framework.......................................................................................6
1.
2.
3.
4.
5.
6.

3.

Current Batch Manager Process..................................................................14

Running Folders Test Packs and Feature Sets................................................14
Example Test Pack Feature Set List.............................................................14

Test Results..................................................................................................15
1.
2.
3.
4.
5.

9.

Checking Screen data is as Expected...........................................................13

Checking Screen Windows and Objects Exist................................................13
Checking Database data is as Expected via SQL............................................13

Running Groups of Tests via Batch Manager.................................................14

1.
2.
3.

8.

Generic Keyword Functions........................................................................10

Utility Keyword Functions...........................................................................10
Randomisation of Test Scenarios.................................................................11
Randomisation of Medical Conditions...........................................................11
Bespoke Keywords for New Features...........................................................12
Advanced Conditional Steps.......................................................................12

Writing Keyword Check Steps.......................................................................13

1.
2.
3.

7.

Output Data...............................................................................................9
Output Parameters and Excel Calculations......................................................9

Writing Keyword Tests..................................................................................10

1.
2.
3.
4.
5.
6.

6.

Parameters for Keyword Functions................................................................8

Parameters from [Data] Sheets....................................................................8
Parameter Data from [Screen].....................................................................8
Parameter To/From [Environment] Variables..................................................8

Keyword Output Data & Calculations..............................................................9

1.
2.

5.

QTP Keyword Framework.............................................................................6

QTP Keyword Test.......................................................................................6
The Hybrid Approach...................................................................................6
System Libraries & Object Repositories..........................................................6
The Keyword Test Excel Spread Sheet...........................................................7
The Keyword [TestSteps] Sheet....................................................................7

Keyword Input Parameters.............................................................................8

1.
2.
3.
4.

4.

Keyword Frameworks..................................................................................5
Background...............................................................................................5
Purpose of Document..................................................................................5

Output Data Sheet xls...............................................................................15

Test Execution Log Results.........................................................................15
Active Progress Log...................................................................................15
Detailed Step & Check Results....................................................................15
Test Pack Summary Results........................................................................15

Debugging Keyword Framework Scripts.......................................................16

1.
QTP Debugging of the Keyword Framework..................................................16
v01 R.Ecott
Page 3/37
27/12/2016

CONFIDENTIAL

10.

BDD Gherkin Feature Principles....................................................................17

1.
2.

11.

Writing BDD Feature Scenario Tests.............................................................18

1.
2.
3.
4.
5.
6.
7.

12.

VISIT KW_Open_WebPortal.....................................................................20
ENTER KW_Enter_WebEdit......................................................................20
SELECT KW_Enter_WebList.....................................................................20
CLICK KW_Click_WebObj.......................................................................20
CHECK Equals KW_Check_ActualVsExpected.............................................20
CHECK Exists KW_Check_Web_Object_Exists............................................20
CHECK SQL KW_Check_SQLvsExpected....................................................21
SAVE/STORE KW_StoreData....................................................................21
WITH existing Functions & Actions............................................................21
FUNCTIONS & ACTIONS............................................................................21

Setting up a BDD Project Folder and Running a Script..................................22

1.
2.
3.
4.
5.
6.

14.

BDD Features...........................................................................................18
BDD Scenario Steps..................................................................................18
Given and Existing Business Functions.........................................................18
When an Action is performed......................................................................19
Then the Expected Results are Checked.......................................................19
Screen field Names...................................................................................19
Running the tests from Batch Manager........................................................19

Example BDD Step Language.......................................................................20

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

13.

Features of the Gherkin Language...............................................................17

Scenarios Given, When, Then..................................................................17

Setting up a BDD Project Folder..................................................................22

Creating a QTP keyword test for the new Project Folder.................................22
Template BDD keyword tests and FEATURESETs............................................23
FEATURESET Summary Results...................................................................23
Example BDD Feature xls Script..................................................................24
Test Results from a BDD Feature Run..........................................................25

Future Framework Developments.................................................................26

1.
2.
3.

Current work Outstanding..........................................................................26

Future Integration with QAC.......................................................................26
Future Enhancements................................................................................26

Appendix A.

Validation of BDD Test Steps........................................................27

Appendix B.

QAC Test Steps Converted to BDD Steps.......................................28

v01 R.Ecott

Page 4/37

27/12/2016

CONFIDENTIAL

1. Introduction
1

Keyword Frameworks

Keyword frameworks are not restricted to one style of scripting or technology. The test
steps can be written into columns of Excel spread sheets or can be Comma or Line Feed
delimited in notepad txt files, or can be in QAC/Quality Centre Applications. They can
combine Functions with Parameters or can use specific business language syntax.
Once a format is agreed and adopted, the Keyword test scripts will be fit for manual testing
and Automation, regardless of which coding Engine is used e.g. C#, Java, QTP.

Background

Most QTP Automation is built using a Data-Driven frameworks that allows large volumes of
data to be processed through a repeatable set of Actions in test scripts that are built and
maintained by Automation Engineers.
An alternate approach is to use a Keyword driven framework that allows the test steps to
be written and run automatically by Business and Test Analysts using a generic set of
Keywords and Functions. This approach requires a lot less support from the automation
team and allows the non-technical testers to create new automated tests.
To enable integration with data driven tests, a hybrid framework that combines the use of
large data sheets of test cases and Keyword test steps has been piloted in QTP as an
alternative and more productive way of using Automation. All code has been written into
Functions that can be re-used in either the original Action scripts, or the newer Keyword
framework.

Purpose of Document

The purpose of this document is to explain how the new automation framework has been
built, how tests should be written, how those tests are run and what test results are
provided. This is detailed in sections 2-9 of this report.
It will also describe how the Keyword framework has been enhanced to work with
Behaviour Driven Development BDD, and how Test Steps should be written using the
Cucumber / Gherkin language. This is documented from section 10 onwards.

v01 R.Ecott

Page 5/37

27/12/2016

CONFIDENTIAL

v01 R.Ecott

Page 6/37

27/12/2016

CONFIDENTIAL

2. Keyword Framework
4

QTP Keyword Framework

To run a Keyword QTP test there are 2 function libraries that need to be associated to the
test. The first is the KW_QTP_Framework library that will load the xls TestStep sheet and
any system Object Repositories/Function Libraries. The second is the KW_FunctionLib that
contains the Controlling Function to run the test steps, and also has Generic Keyword
functions that interpret the test steps e.g. Navigate, Enter-Text and Check-Results.

QTP Keyword Test

All keyword tests are identical in QTP terms, and are made up of 3 function calls:

KW_Initialise_Test

KW_Process_Keywords

KW_Cleanup_Test

The keywords that are written in the TestSteps datasheet define what the test will do.

The Hybrid Approach

This framework will allow existing Excel data sheets of Quote creation data to be used in
conjunction with TestSteps also written in Excel data sheets. The QTP principle of test
Iterations is still used if required, and the Keyword TestSteps will be executed for each row
in the Global datasheet (if there are any). This enables a Quote to be easily created as a
first step in the test using existing Actions, and then subsequent steps can check the newly
created quote and if required take further actions like clicking menu links, entering data into
edit fields and clicking Submit buttons. All of these steps are then repeated for each
Row(Iteration) of data in the Global datasheet.
This process has been designed to fit in with the existing Initalise_Test & Test_Cleanup
functions, so any additional Medical datasheets are automatically loaded, and the Summary
& Detailed outputs are also still produced.

v01 R.Ecott

Page 7/37

27/12/2016

CONFIDENTIAL

System Libraries & Object Repositories

A Keyword test will need the existing function libraries associated to it eg: Initialise,
Cleanup, Error_Handling, HTML_Logging & Varrious_Functions. In addition to this the new
framework will attempt to load Libraries depending on the System under test. So the first 3
letters of the test xls name will be used to determine the system under test, and then load
any Libs and Object Repositories that also start with the same 3 letters. As an example, an
xls Test Sheet named ILL_ICE001_BDD_Ceding_Annuity_Calcs.xls will load the
IllustratePlus_iFrame.tsr & IllustratePlus_Shared.tsr ORs at run-time from the
Q:\Framework\Object_Repositorys folder as they start with the 3 letters Ill.
Similarly, an xls Test Sheet named INV_TRD011_Var_Security_Buy_Sells will load the
function library Investpro.qfl from the Q:\Framework\App_Function_Libs folder.

v01 R.Ecott

Page 8/37

27/12/2016

CONFIDENTIAL

The Keyword Test Excel Spread Sheet

The test xls Spread sheet should be in the format of Excel 97 2003, as the newer formats
are not supported by QTP. It is good practice to use a prescribed format for the Name of
your tests. I would suggest that the 1st 3 Characters represent the system under test ie Ill
for Illustrations. Then a project id and number can be used e.g. ICE001 or MU2001,
followed by a description of the feature being tested. The addition of BDD can also indicate
the style of the test.
Eg:

INV_TRD011_Var_Security_Buy_Sells.xls
ILL_ICE001_BDD_Ceding_Annuity_Calcs.xls

A QTP test will be created for each new project under Q:\Framework\Scripts\ BDD_Features
and the test xlss should be stored in a Project \ Drop Folder as is currently the practice:
Eg:

\Scripts\BDD_Features\InvestPro\InvestPro_2014_02_Spring_Release\Drop Folder

The test xls will contain 3 data sheets [Global] which can contain Quote creation data,
[Output] which has a standard set of result columns, and [TestSteps] containing a
description of the test and the detailed steps to be run by QTP.

The Keyword [TestSteps] Sheet

The first 3 rows under the titles of the TestStep sheets are for documentation only and
should not contain any Test Steps.
Comments

Step Type

Keyword

Data Params

'Test Name : ICE001_Ceeding_Annuity_Calcs

'Description : Create 8 100k policies from data on the Global sheet
'Author

: Created by RE - 12/11/2013

The test steps are entered from row 4 of the Excel TestSteps data sheet in 4 columns. The
first column is a Comment to help the script be more understandable. Column2 is the Step
Type and will mostly be set to Function, but can be occasionally used to indicate that you
want to run an Action, allowing historic code to be re-used. Column 3 defines the Keyword
which is the name of the Function that is to be run, and function parameters can be entered
in the 4th column.
Comments

Step Type

Keyword

Data Params

'Create backdated Policy

Function

Create_Quote

'Receive Payment NPC

Action

Policy_Activation

"*Ceeding_Amt"

'Click the Activation link

Function

KW_Click_WebLink

"Activation Info"

End Test
v01 R.Ecott

Page 9/37

27/12/2016

CONFIDENTIAL

3. Keyword Input Parameters

10

Parameters for Keyword Functions

To give flexibility to the Keyword functions you can pass variable Data Parameters into the
functions from the 4th column in the sheet. These parameters should be wrapped with
Quote marks, and multiple values can be provided if separated by commas. When this is
interpreted by QTP the two columns will be concatenated and the function will be called e.g.
KW_ClientInfo Mr., John, Smith.
11

Parameters from [Data] Sheets

If the test relies on a quote being created then data from the [Global] sheet may need to be
passed into the function, and this is achieved by preceding the parameter with an asterisk,
Eg: *FirstName. The * tells QTP to get the data from a datasheet which will be enclosed
by square brackets [ ]. When these brackets are not provided, the [Global] sheet is used
as a default. This example would get data from the FirstName column in the Global sheet.
Other datasheets can also be used e.g. *[Cancer]Med_Type or [Output]RefNo
12

Parameter Data from [Screen]

Using a similar syntax as the above for Data, we can access any field on the screen. This
can be used for scraping values from the screen and storing them to a datasheet, or can be
used when Checking that the expected information is displayed.
Keyword

Data Parameters

KW_StoreData

*[Output]QuoteNum , *[Screen]Quote Num2:2

KW_Check_ActualVsExpected

*[Screen]ReQuoteAnnuity[1], =, 5973.27

The first of these two examples shows how to grab a value from a Table on the screen,
where you want data at Row2 : Column2. The second example shows how to grab data from
the second occurrence of the ReQuoteAnnuity Web Element specified with [index1].
13

Parameter To/From [Environment] Variables

In some tests, data from one step is required to be used in a later step. There are two ways
to achieve this, one is to store the data to the [Output] datasheet, but another is to store
data to an [Environment] variable. Then it can be used as an input to a later keyword.
Keyword

Data Parameters

KW_StoreData

*[Environment]SavedQuote, *[Screen]Quote

KW_Click_WebLink

*[Environment]SavedQuote or %SavedQuote

v01 R.Ecott

Page 10/37

27/12/2016

CONFIDENTIAL

4. Keyword Output Data & Calculations

14

Output Data

The Keyword functions that access data from the screen will also automatically write that
value to the TestSteps data sheet in the 6th column named Out. This value can then be used
in later steps in the test similar to saving data to Environment variables.

15

Output Parameters and Excel Calculations

There are two further columns in the TestSteps data sheet that can be used for customising
your test steps with Excel calculations. Column5 Working Prams and the aforementioned
Column 6 Out. Using formulas, you can perform calculations on columns E5 & F5 to give
the result as an input parameter Data Param in column D5.
Example 1. Use a previously collected Quote number to link to the Quote;

Comments

Step Type

Keyword

Data Params

'New Quote

Function

Create_Quote

*Name,
*Pension

Link to
Quote

Function

KW_Click_WebLink

= & F5&

Working
Param

Out
AH00092016

Example 2. Use Excels Today() function to check some dates on the screen;

Comments

'New

Quote

'Enter
todays
Date
Check
Guarante
e Date

Step Type

Keyword

Data Params

Working
Param

Function

Create_Quote

Function

KW_Enter_WebEdit

ToDay, =E6

=Today()

Function

KW_Check_ActExp

[Screen]Newd,=,E7

=Today()+4

Ou
t

Example 3. A more complex calculation to use a previously collected Annuity value and
divide it by 12 to get the Monthly value;
="""" & IF(LEN(F13)>0,ROUND(VALUE(SUBSTITUTE(F13," ",""))/12,2),"Quoted Annuity / 12") & """"

v01 R.Ecott

Page 11/37

27/12/2016

CONFIDENTIAL

v01 R.Ecott

Page 12/37

27/12/2016

CONFIDENTIAL

5. Writing Keyword Tests

16

Generic Keyword Functions

Several generic functions have been written to form the basis of the framework and allow
simple Steps to be achieved without the need to write visual basic QTP code. Additionally,
Web Objects do not need to be stored in the Object Repository as descriptive programming
technics are used to id objects via their name (Spy can be used to find the object name).
KW_Generic Functions
KW_Open_WebPortal
KW_Enter_WebEdit
KW_Enter_WebList
KW_Click_WebButton
KW_Click_WebLink
KW_Click_WebImage
KW_Click_WebRadio
KW_Click_WebCheckBox
KW_Check_ActualVsExpected
KW_Check_SQLvsExpected
KW_Check_WebObject
17

Description & Parameters

Start IE if not already open and navigate to URL
Param1: URL
Enter data into an Edit field [with index if needed]
Param1: FieldName[index], Param2: Data
Select data in a selection List
Param1: ListName[index], Param2: Data
Click on a Web Button ie Submit
Param1: ButtonName[index]
Click on a Web Link
Param1: LinkName[index]
Click on a Web Image
Param1: ImageName[index]
Select data in a Radio Button Group
Param1: RadioGroupName[index], Param2: Data
Set data in a CheckBox On/Off
Param1: CheckBoxName[index], Param2: Data
Check Actual vs Expected and Log results
Params: p_Actual, p_Operator, p_Expected
Checks SQL statement via GrabValue v expected
Params: p_SQL, p_Operator, p_Expected
Checks a Browser/Page/Object Exists/Enabled
Params: ObjType, ObjName, Exists

Utility Keyword Functions

There are a few other functions that are used to interact with the xls Data Sheets to Get
and Store data. These functions can also be used in the TestSteps, as an example you may
want to record the newly created quote number to the [Output] sheet, so use
KW_StoreData [Output]RefNo, *[Screen]QuoteNo
KW_Utility Functions
KW_StoreData
KW_GetData
KW_Get_WebElement
KW_Get_WebTable

v01 R.Ecott

Description & Parameters

Save data to a Data Table[Sheet] Column
Param1: [Sheet]Column-Name, Param2: Data
Get data from a Data Table[Sheet] Column
Param1: [Sheet]Column-Name
Get data from an Element of the screen
Param1: FieldName[index], Param2: Output-To
Get data from an Table on the screen
P1: TableName[index], P2: Row, P3:Col P4: Output-To

Page 13/37

27/12/2016

CONFIDENTIAL

18

Randomisation of Test Scenarios

If your tests require Quotes to be created, rather than specifying all 90 input columns of
data required, you can use the randomized test case feature. Simply enter the text
Randomized into the FirstName field on the [Global] sheet and a random mix of
Product / Charge Type and Lifestyle will be generated during the Quote creation run.
Additionally you can specify if you want a Single/Joint case, a Smoker or just about any
attribute of a Quote by putting that text into the LastName field. Here are some examples:
Title

FirstName

LastName

Dr

Randomized

Heavy-Smoker

Rev

Randomized-Joint

With-Profits-RPI

Mr

Randomized-OMO

Single-Fee-Monthly

Mrs

Randomized-Smoker

IVP-TCF

Specifying the title Mr will give a Male life, Mrs - Female, and Rev/Dr will get a random sex.

19

Randomisation of Medical Conditions

If your tests require Medical conditions on the Quote, then these can also be specified by
adding the condition name to the FirstName or LastName fields on the [Global] sheet.
Title

FirstName

LastName

Dr

Randomized-Single

OMO-Cancer

Mr

Randomized-Diabetes

Joint-With-Profits-Blood

Mrs

Randomized-Joint

IVP-Heart-Cancer

Then the RandomiseCase function will extract the medical condition and set the
appropriate Control-Column at the end of the [Global] sheet to a Random selection.
If the test case is for a single life Applicant/Random will be used. If the test case is for
joint lives then one of four values will be used:
"Annuitant/Random", "Annuitant/RandomAll", "Both/Random", "Spouse/Random"

Blood_Pressur
e

Cholesterol

Diabetes

Heart_Conditi
on

Cancer
Annuitant/Random

Both/Random

Spouse/Random
Both/Random

v01 R.Ecott

Page 14/37

Spouse/Random

27/12/2016

CONFIDENTIAL

20

Bespoke Keywords for New Features

When new systems are specified with new screens or complex processes, the generic
keywords may not be sufficient or appropriate. This does not stop Keyword tests from being
written before delivery of the system takes place, as Place-Holder keywords can be used. If
a new screen has 20 input fields, a Submit to click and then a value to check, this would
result in 22 steps in the xls TestSteps, and there could be 3 or more screens which would
result in a long complex set of steps. So an alternate approach is to request a bespoke
Function be built by the Automation Engineering team to perform multiple actions.
In This example, the tester will write the test steps using a generic keyword to Login to the
Portal, then some bespoke keywords for AN (ANewCust) to fill in the specific screens for
Client and Pension are needed. Then some more generic keywords for navigation and
checking are used. The next step will be to populate the [Global] data sheet with some
appropriate data, and then the xls test sheet can be sent to the Automation team for
validation and the building of any Bespoke functions.

Ste
p

Keyword

'Login to A Portal

Functio
n

KW_Login_Portal

'Create a new Client

Functio
n

AN_Enter_Client

'Add the Pension

Details

Functio
n

AN_Enter_Pension

'Go to the results

screens
'Check results
screen has been
displayed

Functio
n

KW_Click_WebLink

"Quote Results"

Functio
n

KW_Check_WebObject

"Browser, Quote Results", Exists

Comments

Data Parameters

'Test Name :
AVL_QUT001_New_Qu
ote
Description: Get a
Quote
'Author : Created by
RE - 14/11/2013

21

"https://www.ANew.com/annuity/security.a
spx", "TestUser", "*Password"
"*Deal_Type", "*Advice_Type",
"*Title", "*FirstName", "*LastName",
"*DOB", "*Marital_Status",
"*Joint_Single", "*Partner_Title",
"*Partner_FirstName", "*Part
"*Purchase_Amount", "*TFC_Amount",
"*Deal_Type", "*Fee_Method",
"*Percent" , "*Amount"

Advanced Conditional Steps

It is possible to conditionally execute steps by using If to check an Environment variable,

and then use EndIf to finish condition processing.

Comments
Start If

v01 R.Ecott

Ste
p

Keyword

Data
Parameters
EnvNeedResults

If

Page 15/37

Assignment
Yes

27/12/2016

CONFIDENTIAL

do a function
End the If

v01 R.Ecott

Functio
n

KW_Click_WebLink

"Quote Results"

EndIf

Page 16/37

27/12/2016

CONFIDENTIAL

6. Writing Keyword Check Steps

22

Checking Screen data is as Expected

When you have test steps that perform an action you will want to verify that the action has
been executed correctly. We have a Generic Keyword KW_Check_ActualVsExpected that
will check the Actual results are either = Expected results or >, <, Not Equal, Contains or
Within. This keyword requires 3 parameters: Actual_Results, Operator,
Expected_Results;

Keyword

Data Parameters

KW_Check_ActualVsExpected

*[Output]Exp_ELEM, EQUALS, 25.99

KW_Check_ActualVsExpected

*[Screen]ELEMale, =,*[Global]Exp_ELEM

KW_Check_ActualVsExpected

*[Screen]ReQuoteAnnuity[1], >, 5973.27

KW_Check_ActualVsExpected

*[Screen]Message, =, Quote completed OK

23

Checking Screen Windows and Objects Exist

A further Keyword KW_Check_WebObject allows us to check that Web Browsers, Pages

and Objects exist on the Screen, or dont exist, or are enabled and visible. If a TolerancePC
field is in the Global sheet, any value there-in will be applied during the check;

Keyword

Data Parameters

KW_Check_WebObject

Browser, PDF, Exists

KW_Check_WebObject

Page, Results, DoesNotExist

code TBC

KW_Check_WebObject

Object, Purchase Amt, IsEnabled

code TBC

KW_Check_WebObject

Object, Purchase Amt, IsNotVisible code TBC

24

Checking Database data is as Expected via SQL

More complex checks can be made against databases by providing the exact SQL and the
expected results to the KW_Check_SQLvsExpected keyword;

Keyword

Data Parameters

KW_Check_SQLvsExpected

Select Amount From xyz, EQUALS, 25.99

KW_Check_ SQLvsExpected

"SELECT Transaction_No FROM dbo.All_Transactions where

Securities_Fund_Deals_ID =F5 ", =,*[Screen]ID no

v01 R.Ecott

Page 17/37

27/12/2016

CONFIDENTIAL

All of the above Check Keywords output detailed Pass/Fail information to the Output Test
Execution Results, which is explained later in this document.

7. Running Groups of Tests via Batch Manager

25

Current Batch Manager Process

The current batch manager is used to select the xls data sheet to run against the QTP
script. This will continue, but has been enhanced to allow multiple data sheets to be run.

26

Running Folders Test Packs and Feature Sets

A new concept has been introduced that will allow several test data sheets to be run one
after the other ie a TESTPACK. As usual in the Batch Manager, navigate to the test script
that is to run, then from the drop down list of data sheets select your newly created
TESTPACK.xls list. Alternately you can put the text FOLDER into the Ref field, and all of
tests in the test drop folder will be run.

27

Example Test Pack Feature Set List

Execut
e

Feature_File

Yes

INV_TRD001_Bond_Buy_Sells

No

INV_TRD002_Bond_Amends

No

INV_TRD003_Bond_Deletes
INV_TRD011_Var_Security_Buy_
Sells
INV_TRD012_Var_Security_Amen

Yes
No

v01 R.Ecott

Feature_Path

Q:\Framework\Scripts\Ad_Hoc\KeywordTest\Drop
Folder
Q:\Framework\Scripts\Ad_Hoc\KeywordTest\Drop
Folder
Q:\Framework\Scripts\Ad_Hoc\KeywordTest\Drop
Folder
Q:\Framework\Scripts\InvestPro_201402_Spring_Rele
ase\Drop
Q:\Framework\Scripts\InvestPro_201402_Spring_Rele

Page 18/37

27/12/2016

CONFIDENTIAL

No

ds
INV_TRD013_Var_Security_Delet
es
INV_TRD021_Currency_Buy_Sell
s

No

INV_TRD022_Currency_Amends

No

INV_TRD023_Currency_Deletes

No

v01 R.Ecott

ase\Dropr
Q:\Framework\Scripts\InvestPro_201402_Spring_Rele
ase\Drop
Q:\Framework\Scripts\InvestPro_Regression_Pack\Dr
op Folder
Q:\Framework\Scripts\InvestPro_Regression_Pack\Dr
op Folder
Q:\Framework\Scripts\InvestPro_Regression_Pack\Dr
op Folder

Page 19/37

27/12/2016

CONFIDENTIAL

8. Test Results
28

Output Data Sheet xls

When the QTP test completes, it formats and writes out the [Output] sheet, the [Medical]
conditions sheets and if required the [Global] sheet. This results file will be stored in the
Q:\Framework\Output folder. If the QTP test does not reach the end of its run for some
reason, only the [Output] data sheet will be contained in the result file, but this does
contain all the critical output like captured Quote and ELE numbers. The name of the result
file includes the Test Sheet name, the run date and the rows that were run. An additional
backup is written to the C:\temp\QTP-EndRun-Backup.xls file.
29

Test Execution Log Results

For each test, there is an Execution Result html file showing Pass/Fail information for each
test Iteration row. There is also a summary at the end of this report that gives the total
run information and Start/End/Total timings. The results file will be stored in the
Q:\Framework\Logs folder.
30

Active Progress Log

To check progress of active tests, view the RunLog.html file, also in the \Logs folder.
31

Detailed Step & Check Results

When the KW_Check_ActualVsExpected keyword is used, additional check pass/fail details

will be written to the above Execution Result html file. This information is initially hidden,
but can be made visible for each iteration in turn by clicking on the text in the Notes
column. Check pass/fail counts are also included in the detail and in the final summary.

32

Test Pack Summary Results

After running a Pack or Folder of tests a final QTP script will be run to build a Summary
report: Q:\Framework\Scripts\Ad_Hoc\FeatureSet_Summary_Report. The statistics for the
run will be available in the Q:\Framework\Results folder

v01 R.Ecott

Page 20/37

27/12/2016

CONFIDENTIAL

9. Debugging Keyword Framework Scripts

33

QTP Debugging of the Keyword Framework

First put a break point at line 13, then run the test.

Then F11 into the next Library, and put a break point at line 109.

From this point onwards when you run the script it will break at every Step, allowing you to
either click run to go on to the next Step, or F11 into the function that is being called by the
Step.
Adding the text COMMENT to the step Comment_Step_Description column will by-pass
running a particular step.
v01 R.Ecott

Page 21/37

27/12/2016

CONFIDENTIAL

For large tests you can put Debug into column 4 of the Step that you want to break at, and
this will pause the test when it get to that step.

v01 R.Ecott

Page 22/37

27/12/2016

CONFIDENTIAL

10.
34

BDD Gherkin Feature Principles

Features of the Gherkin Language

For testing the behaviour of your application, there is a descriptive special language called
Gherkin. Gherkin is a Business Readable, Domain Specific Language created especially for
behaviour descriptions. It gives you the ability to remove logic details from behaviour tests.
It is also a line-oriented language that uses indentation to define structure. Line endings
terminate statements (called steps) Finally, most lines in Gherkin start with a special
keyword : Feature, Scenario, Given, When, Then, And, But.
Gherkin serves two purposes: serving as your projects documentation and automated tests.
The Feature describes what is being tested at a high level.
Feature: Some terse yet descriptive text of what is desired
In order to realize a named business value
As an explicit system actor
I want to gain some beneficial outcome which furthers the goal
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
And yet another action
Then some testable outcome is achieved
And something else we can check happens too
Extracted from: http://docs.behat.org/guides/1.gherkin.html#givens

35

Scenarios Given, When, Then

The Scenario gives a specific description of a test case of which there can be multiples in a
Feature. For each Scenario you have the Pre-Requisite - Given, the Steps- When, and the
expected results Then.
Feature: Serve coffee
In order to earn money
Customers should be able to
buy coffee at all times
Scenario: Buy last coffee
Given there are 1 coffees left in the machine
And I have deposited 1 dollar
When I press the coffee button
Then I should be served a coffee

v01 R.Ecott

Page 23/37

27/12/2016

CONFIDENTIAL

11.
36

Writing BDD Feature Scenario Tests

BDD Features

Like our Keyword tests, the first 3 rows of the BDD tests are for documentation, and can be
made up of the Feature ie the name of the test file, the Scenario and the Author. The
Scenario description can also include a summary of any rows from the Global datasheet.

37

BDD Scenario Steps

BDD Scenario steps are very similar to our Keyword steps but are made up of descriptive
verbs & words as opposed to Function names and Parameters. If there is a Space present in
the keyword then we know that we are using BDD language, and we need to Parse it
through a translator, converting it to Function-Speak that QTP understands.
The BDD step is interpreted by the KW_Convert_BDD_To_Keyword function and it will
remove any spurious words like I, the, and, number, field, value, list and then identify the
first word as the Keyword. Some of the more common keywords are visit as in visit a web
url, enter as in enter into a field and click as in click the link or click the button. Any
remaining words will be interpreted as parameters.
The step differentiators Given, When, Then, And But do not actually mean anything to QTP
they are there to make the test scrip easier to understand.
38

Given and Existing Business Functions

The first steps of a test may be pre-requisite setup steps and these are referred to as
Given steps. When our test is using the [Global] sheet for Quote creation there are a set of
existing QTP Functions and Actions that can be used:
"a new illistration quote is created"
"login and load a quote"
"login and load a policy"
"quote application is completed"
"quote is activated"
"quote monnies are applied"
"policy is activated"

=
=
=
=
=
=
=

"FUNCTION:Create_Illistration_Quote"
"FUNCTION:Quote_Login_And_Load"
"FUNCTION:Policy_Login_And_Load"
"ACTION:Policy_Application"
"ACTION:Policy_Activation"
"ACTION:Apply_Monies"
"ACTION:Activation_Info"

Additionally, if you want to use the Randomized test case feature, you can append with
to the statement and then append any specific case words e.g.
Given a new illistration quote is created with Mr Randomized Smoker Cancer OMO
Other examples of Given pre-requisite steps are:
Given I visit https:\xyz.com
and I login and load a quote with [Output]Reqoute number
v01 R.Ecott
Page 24/37
27/12/2016

CONFIDENTIAL

v01 R.Ecott

Page 25/37

27/12/2016

CONFIDENTIAL

39

When an Action is performed

When steps represent actions like entering text into fields on the screen or clicking links and
buttons. These can be performed by using the generic KW-Functions documented earlier.
But for BDD instead of using the function name we use the key component of the function
name e.g. Click for KW_Click_WebLink. Then we add the unique description for the link on
the screen which could be called Show Results, so we have click the show results, then we
add the type of click that we want, link / button or image e.g.
When I click the show results link
and I click the save button
Other actions can be entering values into fields and selecting choices from lists e.g.
When I enter USER1 into the screen username field
and I enter [Global]LastName into the screen surname field
and I select POUND to EURO from the screen conversion[1] list
Using the square brackets for a screen object allows us to handle multiple occurrences of
identically named fields. As in the above example we would select from the 2nd list on the
screen named conversion as we have specified the index 1. To select from the first list we
could use [0] but as 0 is the default value for index we actually dont need it.

40

Then the Expected Results are Checked

Once the action is performed, Then we will probably want to Check the outcome, and this
can be checking a value on screen, checking a web Page is displayed or checking a database
field value via SQL e.g.
Then I check that the Annuity ref no on the screen should equal AH0200102
and I check that the Quote Results Successful text exists on the screen

41

Screen field Names

Some screen fields are in tables or have multiple similar descriptions and can be tricky to
hook onto. So as a Test Analyst you can just use an appropriate name for the field and ask
the Automation team to map that field for you. There is a keyword function that performs
these mappings KW_Convert_BDD_ScreenName_To_ObjID so your appropriate field
name will be added to this function and the correct behind the scenes Object ID will be
used. An example would be that a screen field name requoted number gets converted to
messyxyzTableelementrow2col7 as it is in the html.
v01 R.Ecott

Page 26/37

27/12/2016

CONFIDENTIAL

42

Running the tests from Batch Manager

The BDD xls tests can be run from the batch manager individually, or like the keyword tests
they can be grouped in a FEATURESET list as documented in Section 27.

v01 R.Ecott

Page 27/37

27/12/2016

CONFIDENTIAL

12.
43

Example BDD Step Language

VISIT KW_Open_WebPortal

Given I visit https:\\Google.com

and I visit url https:\\Google.com
and I navigate to https:\\Google.com web page
Then I go to the web page https:\\Google.com

44

ENTER KW_Enter_WebEdit

When I enter USER1 into the screen field Username

and I enter *[Global]Password into the Password field
and I enter *[Cancer]Med_Type into Medication_Name[1]
and I enter High St as Address_Line_1

45

SELECT KW_Enter_WebList

When I select Mrs from the title list

and I select item Pounds from the screen conversion list
and I select *[Global]Currency in the conversion list

46

CLICK KW_Click_WebObj

When I click the Login button

and I click the Activation Info link if it exists
and I click the XYZ-Logo[2] image
and I click Enhanced from the Prod_Type radiobutton
and I click the Open2:3 table item (not developed yet)

47

CHECK Equals KW_Check_ActualVsExpected

Then I check that the re-quoted date on the screen should equal 12/04/2014
and I check that *[Output]Exp_ELEM equals 25.99
and I check [Screen]ELEM is greater than *[Output]Exp_ELEM
and I check the Search text exists on the screen

48

CHECK Exists KW_Check_Web_Object_Exists

Then I check that the pdf window appears

and I check that the Medical page exists on the screen
v01 R.Ecott

Page 28/37

27/12/2016

CONFIDENTIAL

and I check the Re-Quote[2] image appears

49

CHECK SQL KW_Check_SQLvsExpected

Then I check that the Select amount from xyz SQL equals 25.99
and I check Select next-num from abc SQL not equal [Screen]AH_num

50

SAVE/STORE KW_StoreData

Then I save the re-quote number from the screen to the [Output]ReQ_Num field

51

WITH existing Functions & Actions

When writing steps to use the existing Actions and Functions, you may want to provide
varying parameters. This is achieved by appending with to the statement and then
appending the required words eg
Given a new illistration quote is created with Mr Randomized Smoker Cancer OMO
and I login and load a quote with the [Output]Reqoute number
and I login and load a quote with value AH092894836
and the quote is activated with the value [Global]Ceeding_Amt
and Login Portal with *UserName and *Password

10.

FUNCTIONS & ACTIONS

and login portal with *UserName and *Password

In this example step we are attempting to use an existing login function. The framework will
make 3 attempts to call the function. Firstly it will try login_portal, then secondly it will try
upper-case first letters, Login_Portal. If neither of these work then it will append the first
2/3 letters of the Test.xls name and try again e.g. XYZ_Login_Portal.

v01 R.Ecott

Page 29/37

27/12/2016

CONFIDENTIAL

13.
52

Setting up a BDD Project Folder and Running a Script

Setting up a BDD Project Folder

When a new project is started, one of the first things to do is to set up the Test Scripts
folders and save the QTP Keyword test with the same name as the new folder. Once this has
been done, then the folder will be available to the Batch Manager for running your tests.
So create a new folder under the Q:\Framework\Scripts\BDD_Features folder with the name
of your project. In this example I have made the folder using the System name:
Illustrations, the Year&Month 2014_03, a description of the project: MU2_Regression e.g.
Illustrations_2014_03_MU2_Regression. Also make a sub folder named Drop Folder.

53

Creating a QTP keyword test for the new Project Folder

Now load the template Keyword QTP test Q:\Framework\Scripts\Ad_Hoc\KeywordTest, and

Save_As into the new folder with the same name as the new folder.

v01 R.Ecott

Page 30/37

27/12/2016

CONFIDENTIAL

54

Template BDD keyword tests and FEATURESETs

Now you can copy in some existing BDD feature xls files to the Drop Folder ready for
running. The Batch Manager can run the entire Folder of xls sheets, or as described in
section 7. you can use a FEATURESET.xls that details which tests to run. There is a template
for this file in Q:\Framework\Scripts\BDD_Features.

55

FEATURESET Summary Results

After running a Set or Folder of tests, a final QTP script will be run to build a Summary
report in the Q:\Framework\Results\2014_TestResults folder containing statistics for the
run. Every time a run takes place a new summary html file will be build summarising all of
the Unique test results files in the Folder. The name of the summary result file contains the
DDHH & MM, so no files are overwritten and a full audit history is available of all runs.

v01 R.Ecott

Page 31/37

27/12/2016

CONFIDENTIAL

56

Example BDD Feature xls Script

This example of a BDD test script shows navigation to a web site, data entry into an edit
box, selection from a list and Checking of expected results. It also shows the concept of
multiple Scenarios in one Feature.

Comment_Step_
Step Keyword_Operation
Desc
'Feature : AAA_GOO001_BDD_Google
'Scenario : Euro Converter has correct rates
'Author : Created by RE - 04/02/2014 .
'1st test case

Scenari
o

'go to Google

Given

'look for a Euro

Converter

and

I enter "Euro Converter" into the screen q field

'goto a converter

and

I click the "Euros To Pounds Converter and Pound To

Euro Conv" link

'enter an amount

When

'Select the conversion

required

and

'Check the rates

Then

'Check the rates

and

'2nd case
'enter an amount

'Check the rates

EndTest

v01 R.Ecott

Converting 1000 gets > 1200

I visit "https://www.google.co.uk/"

I enter "1000" into the screen f_amount field

I select "POUND to EURO" from the screen f_type list
I check that the outputbox on the screen should be
greater than 1200
I check that the outputbox on the screen should be
less than 1300

Scenari
o
When

Converting 2000 > 2300

I enter "2000" into the screen f_amount field

and

I select "EURO to POUND" from the screen f_type list

and

I select "POUND to EURO" from the screen f_type list

I check that the outputbox on the screen should be
greater than 2300

Then

EndT
est

EndTest

Page 32/37

27/12/2016

CONFIDENTIAL

57

Test Results from a BDD Feature Run

This is the Summary of the BDD script run results.

After clicking on the Notes data, the results are expanded to show the detailed steps.

v01 R.Ecott

Page 33/37

27/12/2016

CONFIDENTIAL

14.
58

Future Framework Developments

Current work Outstanding

Complete the Checking function to cover Enabled/Visible/Not-Equal/Not-Exist.

Test the TESTPACK / FOLDER running from the Batch Manager.
Build a Summary report that shows the results of a TESTPACK / FEATURESET run.

59

Future Integration with QAC

Select the test steps from the SmartBear QAC application held in the dbo_TestCases table,
and run them automatically. Connection to the QAC database is available via:User-id = SmartbearRO
Server = HQSQL2008V0003\VIRTUALCENTER
Database = SoftwarePlanner
Options to run individual tests or complete Folders should be explored.

60

Future Enhancements

Development of Smoke & Regression packs of tests.

Nightly Continuous Integration testing with a RAG status Dashboard.

v01 R.Ecott

Page 34/37

27/12/2016

CONFIDENTIAL

v01 R.Ecott

Page 35/37

27/12/2016

CONFIDENTIAL

Appendix A.

Validation of BDD Test Steps

Once the test steps have been written, the QTP Automation team can validate them against existing functions, and identify any new
functions that are required for the test. Simply uncomment this line in the QTP test 'Environment("VALIDATE_BDD") = "True".

v01 R.Ecott

Page 36/37

27/12/2016

CONFIDENTIAL

Appendix B.

v01 R.Ecott

QAC Test Steps Converted to BDD Steps

Page 37/37

27/12/2016