SmarTest Framework Manual

Version 1.5.3

2010 QA Tools Holger Penka Auf der Beune 85 - 64839 Mnster

1 - Overview....................................................................................................................................................4 2 - Installation................................................................................................................................................. 4 2.1 - The project structure..........................................................................................................................5 2.2 - Customising the configuration files ................................................................................................... 5 2.2.1 - server_plf.ini................................................................................................................................................ 6 2.2.2 - <ComputerName>.ini.................................................................................................................................. 7 2.3 - server_plf.conf................................................................................................................................. 10 2.4 - Starting the Server........................................................................................................................... 11 3 - Organisation of the Table Structure......................................................................................................... 12 3.1 - Suite................................................................................................................................................ 12 3.1.1 - Column 1 - Executing................................................................................................................................ 12 3.1.2 - Column 2 - Process chain...........................................................................................................................13 3.1.3 - Column 3 - test cases................................................................................................................................. 13 3.1.4 - Column 4 - host name................................................................................................................................ 13 3.1.5 - Column 5 - group name............................................................................................................................. 13 3.1.6 - Column 6 - module name...........................................................................................................................13 3.1.7 - Column 7 - comment..................................................................................................................................13 3.2 - PCC................................................................................................................................................. 13 3.3 - DLG................................................................................................................................................. 13 3.4 - General............................................................................................................................................14 4 - Data Management................................................................................................................................... 14 4.1 - Using The Macro............................................................................................................................. 14 5 - Special Functions.................................................................................................................................... 15 5.1 - Overview of all supported functions................................................................................................. 15 5.1.1 - <<SET_STATIC_VAR>>.......................................................................................................................... 15 5.1.2 - <<GET_STATIC_VAR>>..........................................................................................................................15 5.1.3 - <<SET_DYNAMIC_VAR>>.................................................................................................................... 15 5.1.4 - <<GET_DYNAMIC_VAR>>....................................................................................................................16 5.1.5 - <<TODAY>>............................................................................................................................................. 16 5.1.6 - <<EXEC>>................................................................................................................................................ 16 5.1.7 - <<ACTION>>........................................................................................................................................... 16 5.1.8 - <<MENU>>...............................................................................................................................................16 5.1.9 - <<CHK>>.................................................................................................................................................. 16 5.1.10 - <<SET>>................................................................................................................................................. 17 5.1.11 - <<TRANS>>............................................................................................................................................17 5.1.12 - <<INI>>................................................................................................................................................... 17 5.1.13 - <<COMMENT>>.................................................................................................................................... 17 5.1.14 - <<getVal>>.............................................................................................................................................. 17 5.1.15 - <<PCC>>................................................................................................................................................. 17 5.1.16 - <<DEFAULT>>....................................................................................................................................... 18 5.1.17 - <<EMPTY>>........................................................................................................................................... 18 5.1.18 - <<MODE>>.............................................................................................................................................18 5.1.19 - <<CLEAR>>........................................................................................................................................... 18 6 - HP WinRunner.........................................................................................................................................18 6.1 - Installation....................................................................................................................................... 18 6.1.1 - Set the Preferences..................................................................................................................................... 18 6.1.2 - The Gui Map.............................................................................................................................................. 19

Page 2/29

6.1.3 - The custom libraries...................................................................................................................................19 7 - HP QuickTest Pro.................................................................................................................................... 20 7.1 - Installation....................................................................................................................................... 20 7.1.1 - Set the Preferences..................................................................................................................................... 20 The Gui Map........................................................................................................................................................... 20 7.1.2 - The custom libraries...................................................................................................................................20 8 - Borland Silk............................................................................................................................................. 21 8.1 - Installation....................................................................................................................................... 21 8.2 - The Gui Map.................................................................................................................................... 21 8.3 - The custom libraries........................................................................................................................ 21 8.3.1 - cust_lib....................................................................................................................................................... 21 8.3.2 - cust_gui...................................................................................................................................................... 21 8.3.3 - cust_callback.............................................................................................................................................. 21 9 - IBM Rational Functional Tester................................................................................................................ 22 9.1 - Installation....................................................................................................................................... 22 Open a new Test project in IBM rational Functional Tester. Locate that project in the folder...............................22 <MyProject>\project\rft.......................................................................................................................................... 22 9.2 - GUI Map.......................................................................................................................................... 22 9.3 - The custom libraries........................................................................................................................ 23 9.3.1 - CustKickoff................................................................................................................................................ 23 9.3.2 - CustDialog................................................................................................................................................. 23 9.3.3 - CustCallback.............................................................................................................................................. 23 9.3.4 - CustLib.......................................................................................................................................................23 9.3.5 - Custom....................................................................................................................................................... 23 10 - Reporting............................................................................................................................................... 23 10.1 - Database Reports..........................................................................................................................23 10.1.1 - Web Front End......................................................................................................................................... 23 10.1.2 - Delete Test Cases..................................................................................................................................... 26 10.2 - Log Database................................................................................................................................ 26 10.2.1 - Database Model........................................................................................................................................26

Page 3/29

1 - Overview
The SmarTest framework is created for supporting data driven tests. The main benefit is the seperation of the test data and the test case workflow from the auto test tool script code. Generic functions reduce the effort of writing script code. These functions are written in the tool specific programming language and handles the standard object methods like enter a text into a text field or selecting an item from a list box. For the most dialogs it is not necessary to write any line of code. Text files containing all the test data, the workflow - and the logic of the testcase. These text files drive the tests, not the script code. The framework itself can not do anything without the data files. The SmarTest framework loads, analyse and compiles the test data stored in these tab separated text files. During this also a consistence check will be performed. The interface to the data is placed in the test tool. This part communicates with the Smartest Server and drives the test case execution. The key features are: Tests are mainly independent of the test tool The test data drives the tests Reports are stored in a centralised data base Less scripting effort Support for different test tools. Currently supported: HP Mercury WinRunner HP QuickTest Pro Borland SilkTest IBM Rational Functional Tester Open Interface to adapt other test tools Support for multi language testing in different encodings, also multibyte encodings are supported

2 - Installation
Open a command box and call the program install.exe as follows: install.exe -p <path and name of the new project directory> Example: install.exe -p c:\MyProject Then the installation program starts and unpacks all necessary files into the new project structure. During the installation the MySQL database for logging will be installed on an existing MySQL server. This requires an user id which has rights for creating users and databases i.e. the user root. In addition the SQL for creation are also part of the delivery. So a MySQL administartor could install it if a root user is unknown. The framework needs also the tool ImageMagick. The version 6.0.7-Q16 is part of the package and could be installed during the general installation process. The installation is a pre-condition for correct functionality of the framework. The ImagMagick is needed to take screenshots during the test case execution.

Page 4/29

2.1 - The project structure

The following figure shows a project with the name 'MyProject'.

Description of the particular directories 1. MyProject The name of the project. This name was passed to the install script as a parameter. 2. Plf: This directory contains all unchangeable files. The SmarTest Server are also located here. 3. Project: This is the project directory. All modifiable data is located here. Among that the created test data stored as well as the scripts of the framework which are adaptable to each project. 4. Data: This is the data directory. 5. Qtp, rft, silk, wr: Contains the project related scripts for the supported test tools. Currently that is HP QuickTest Pro, IBM Rational Functional Tester, Borland SilkTest and HP WinRunnner.

2.2 - Customising the configuration files

The framework will be installed with a set of configuration files. The parameters needs to be changed to the project requirements. In total three configuration files needs to be customised.

Page 5/29

2.2.1 - server_plf.ini
<ProjectName>/plf/cfg/server_plf.ini

#-------------------------------------------# Pathes #-------------------------------------------[PATH_DATA] # The path to the directory containing all your ini files PATH_INI=C:\MyProject\project\data\ini #-------------------------------------------# Loging Database #-------------------------------------------[LOG_DATABASE] # The name of the MySQL database DB_NAME=log # The host where the database is located DB_HOST=localhost # The port where the DB listen on DB_PORT=3306 # The user of the DB DB_USER=framework # The password for the DB user DB_USER_PASSWD=framework

This file defines the settings of the SmarTest server. The configuration contains two section. The first section defines the path to the configuration files. The second section contains the connection information to the MySQL database which is used for logging test run results.

2.2.1.1 - Section PATH _DATA

The PATH_DATA section contains the information where to find the project related configuration files.
Parameter Description The default is set to the <installation path>\data\ini.

PATH_INI

2.2.1.2 - Section LOG_DATABASE

The LOG_DATABASE section contains the connection information to the MySQL database. The information, comments, screenshots and results collected during the test execution will be stored at this place.
DB_NAME DB_HOST Parameter The default is set to log. No default value is set. Description

Page 6/29

DB_PORT DB_USER DB_USER_PASSWD

Parameter

Description The default is set to 3306. No default value is set. The default is set to framework.

2.2.2 - <ComputerName>.ini
<ProjectIniDirectory>/<ComputerName>.ini

[ETC] # Executes a command after the server is started. It will # started in a new shell so the server will not wait until # the command is executed. POST_EXEC = [ATTACHEMENT] # Should the screenshot be deleted from the directory after it is imported into the DB DEL_ATTACH_SCREEN_SHOT = Y # Should a attachement be deleted from the directory after it is imported into the DB DEL_ATTACH = N

#-------------------------------------------# SUITE #-------------------------------------------[PATH_DATA] # The path and name of the suite file FILE_SUITE=C:\MyProject\project\data\suite\demo_bank.suite

################################################################ # If your data directory has the following structure you do not # need to set the rest of the path vartiables. The Path will # be calculated with the location of your suite file. # If one or more pathes are changed, you need to set the # corresponding vars. # base/suite/suiteFile.suite # pcc/<all pcc files> # dlg/<all dlg files> # ini/<ini files> # translation/<all translation files> # <dialogs.special> ################################################################ # The path to the directory containing all your pcc files PATH_PCC=

Page 7/29

# The path to the directory containing all your dlg files PATH_DLG=

# The path to the directory containing all your translation files PATH_TRANS= # The file containing all the names of the special dialogs FILE_SPECIAL_DLG= # The path to a project tmp directory PATH_TMP=C:\MyProject/project/data/tmp # The path to a project tmp directory PATH_SCREENSHOT=C:\MyProject/project/data/tmp #-------------------------------------------# ENCODING #-------------------------------------------[ENCODING] TRANSLATION = ISO-8859-1 #TRANSLATION = utf8

#-------------------------------------------# LANGUAGE #-------------------------------------------[LANGUAGE] Language_App = EN Language_Os = EN

#-------------------------------------------# DEBUG LOG #-------------------------------------------[LOG] # The name of the configuration file for the debug log FILE_LOG_CONF=C:\MyProject\project\data\ini\server_plf.conf #-------------------------------------------# RUNTIME FLAGS #-------------------------------------------[FLAG] # The name of the configuration file for the debug log STOP_ON_ERROR = No #-------------------------------------------# SETTINGS FOR TESTTOOLS #--------------------------------------------

Page 8/29

[TESTTOOL] QTP_XML_REPO =C:\MyProject/project/data/gui/demo_bank.xml #################################################################### # After this you can add your own ini entries #################################################################### [MyProject_DATA] startURL=http://www.qatools.de/login.php This file defines the project specific settings for the PLF Server. There are six framework specific sections. They providing information about specific behaviour, the project paths, the multi language support (encoding and languages) and the configuration file for the additional local logging. After this six sections the project specific sections and parameters could be added.

2.2.2.1 - Section ETC

The ETC section contains the information about an optional command to be executed after compiling the test data.
POST_EXEC Parameter No default value set. Description

2.2.2.2 - Section ATTACHMENT

The ATTACHMENT sections contains the information about the handling of attachements like screenshots or text files on the local drive.
Parameter DEL_ATTACH_SCREEN_SHOT DEL_ATTACH The default is set to Y. The default is set to Y. Description

2.2.2.3 - Section PATH_DATA

The PATH_DATA section contains the information about all the paths to the project specific configuration files, data files and temporary data folder.
FILE_SUITE PATH_PCC PATH_DLG PATH_TRANS FILE_SPECIAL_DLG PATH_TMP PATH_SCREENSHOT Parameter No default value will be set. No default value will be set. No default value will be set. No default value will be set. No default value will be set. No default value will be set. No default value will be set. Description

2.2.2.4 - Section ENCODING

The ENCODING section contains the information about the encoding to be used.
TRANSLATION Parameter Description The default is set to ISO-8859-1.

2.2.2.5 - Section LANGUAGE

The LANGUAGE section contains the information about the used languages for the application under test and the operation system of the test client.
Language_App Language_OS Parameter The default is set to EN. The default is set to EN. Description

Page 9/29

2.2.2.6 - Section LOG

The LOG section contains the information about the path and name of the server configuration file.
FILE_LOG_CONF Parameter Description The default is set to server_log.conf.

2.2.2.7 - Section FLAG

The FLAG section containing flags to control the behaviour.
STOP_ON_ERROR Parameter The default is set to Yes. Description

2.2.2.8 - Section TESTTOOL

The FLAG section containing flags to control the behaviour.
QTP_XML_REPO Parameter Description The path to the XML gui file. Only necessary if HP QuickTest Pro is used.

2.3 - server_plf.conf
<ProjectIniDirectory>/server_plf.conf

log4perl.rootLogger=DEBUG, A1 log4perl.appender.A1=Log::Dispatch::File log4perl.appender.A1.filename=server_plf.log #log4perl.appender.A1.mode=append log4perl.appender.A1.mode=write log4perl.appender.A1.layout=Log::Log4perl::Layout::PatternLayout log4perl.appender.A1.layout.ConversionPattern=%d %p> %F{1}:%L %M - %m%n

This file defines the debug log for the SmarTest server. For the debug information log4perl (http://log4perl.sourceforge.net/) is used. In this file only two settings should be changed: the level of logging and the name of the log file. The default value for the logging level is set to DEBUG and the file name is set to server_plf.log. For the logging level there are 5 different levels: DEBUG INFO WARN ERROR

FATAL For normal use the logging level should be set to WARN - all warnings, errors and fatal errors will be logged. Only if there are problems the DEBUG level should be switched on as it slows down the compiling of the test data extremely. For more information please visit the log4perl webpage.

Page 10/29

2.4 - Starting the Server

The server is started using a command shell. Change to the directory where the start_server.exe is installed. Type in start_server_plf. The server is ready if the message Start the PLFServer on port '6666' is displayed.

Once started the PLF Server shows the port it is 'listening' to as well as the total number of loaded testcases. Collecting and managing test data

Page 11/29

3 - Organisation of the Table Structure

The following figure shows the relations between the particular data table types suite, pcc and dlg.

3.1 - Suite
A suite defines a set of test cases to be executed.The suite file has seven columns. The first column indicates whether the test case is to be executed.

3.1.1 - Column 1 - Executing

Three alternatives are possible. 1. A 'y' or 'yes' indicates that the test case will be executed. 2. A computer name indicates that the test case will be executed if the suite is executed on this computer. With the aid of computer names the task of building up a test computer farm is simplified. 3. Any other character indicates that the test case will not be executed.

Page 12/29

3.1.2 - Column 2 - Process chain

A reference to a process chain (a file with the extension '.pcc' in the directory 'pcc') is entered in this column. In this example the entry 'smoke' refers to a file 'pcc/smoke.pcc' (3).

3.1.3 - Column 3 - test cases

1. A range is specified by a '-'. e.g. 'Test1-3' points at the test cases 'Test1, Test2 and Test3'. All these test cases have to be available. If Test2 does not exist, an error message will be displayed. Important: The number in the range have no leading zero. 2. An enumeration is specified. e.g. 'Test1, Test3, Test6-8'. An enumeration may contain a range again.

3.1.4 - Column 4 - host name

The hostname on which this test should be executed. If empty, every host is accepted (optional).

3.1.5 - Column 5 - group name

The name of the group this test belongs to (optional).

3.1.6 - Column 6 - module name

The name of the module this testcase belongs to (optional).

3.1.7 - Column 7 - comment

A user defined comment. This comment will be added to the test case header (optional).

3.2 - PCC
The PCC files are defining the way through the application. The first column defines a unique ID for each row in the PCC file. The name of this column is always HEADER. The appropriate row from the suite (or in special cases also from a PCC file) is called via this ID. The column headings are either special functions (identified by the signature <<nameOfTheSpecialFunction>> e.g. <<EXEC>>) or refer to DLG files. In this case the column heading is the same as the name of the DLG file without the extension '.dlg'. The dialogs and special functions defined in the columns of a process chain are a superset of the dialogs and special functions used during a test case. If a cell is empty it will be skipped. This supports different ways through the application under test.

3.3 - DLG
The DLG files are defining the inputs in a window or a page of the application (either real or logical). The definition of a page or a window depends on the testtool used and on the recording of the Gui-Map, ObjectMap or WindowsDeclaration. In the normal case one DLG file exists for each defined window in the testtool. The first column defines a unique ID for each cell in the DLG file. The appropriate row in the PCC file is called via this ID. The name of this column is always HEADER. The column headings are the logical names of the Gui-Objects as recorded by the test tool. The order when actions on the Gui-objects are executed is also specified by the order of the columns. Columns may contain special functions again as in the PCC files. An overview of possible special functions can be found below.

Page 13/29

3.4 - General
The term ObjectName will often appear in the following overview. It is the logical name of an object in the GuiMap of the test tool. Winrunner uses the term GUI-Map, Silk uses WindowsDeclaration. When special functions are used in the table head, they apply to the whole column, whereas used in the cell only, they apply only to that single data set. When a function is used in the table head the first parameter is generally the name of the Gui object, simply because the information which object is related has to be preserved. The value used is located in the table cell. Special functions can not be nested.

4 - Data Management
For an easy management of the test data an Excel macro supports the text based handling of the data. This macro opens the tab seperated files in a way that Excel will not automatically reformatting the data. It supports also the hierarchy of the three file types suite, pcc and dlg. Open the smarTest.xla file once in MS Excel. Enable the macros. The first time you open this macro a request for configuration will appear. Click on Yes to edit the project data path

Select the data directory of your project and click on OK. A new tool box should appear.

Settings Details Open Save Close file Close all files

Icon

Explanation Open the "Project Data Path" Open the file related to the selected cell in the data file (more details below). Opens the File Open dialoag to open a data file Save the current data file as a tab separated file Closes the current data file Closes all open files

4.1 - Using The Macro

The key feature of the macro is the "Details" function. It opens the file based on the name in the header line e.g. a pcc, and select the related test case or dataset.

Page 14/29

5 - Special Functions
5.1 - Overview of all supported functions

5.1.1 - <<SET_STATIC_VAR>>
This function allows the storage of values in variables inside the Framework. The validity of these variables ends with the current testcase. This function is only evaluated at compile time. Example of use: An application is tested that requires a login with a user ID. This ID is visible within the application. Now it is to be checked if the Login ID is displayed correctly. <<SET_STATIC_VAR>> is used In the part where the login is realised. <<GET_STATIC_VAR>> is used in the part of the examination.
Table head Table cell Appearance Yes Yes Allowed Syntax <<SET_STATIC_VAR>>:ObjectName:VarNa me <<SET_STATIC_VAR>>:Varname:Value

5.1.2 - <<GET_STATIC_VAR>>
Fetches a static variable's value for further usage. If no value was previously assigned to the variable, a compile error will occur.
Table head Table cell Appearance Yes Yes Allowed Syntax <<GET_STATIC_VAR>>:ObjectName <<GET_STATIC_VAR>>:Varname

5.1.3 - <<SET_DYNAMIC_VAR>>
This function allows the storage of values of the application in variables inside the Framework. The validity of the variables ends with the current testcase. This function is only evaluated at runtime. Example of use:

Page 15/29

An application is tested where data sets are recorded that obtain a dynamically generated ID. This ID is required to locate the data set later again. This ID can be stored with <<SET_DYNAMIC_VAR>> and retrieved with <<GET_DANYMIC_VAR>> later again.
Table head Table cell Appearance Yes Yes Allowed Syntax <<SET_DYNAMIC_VAR>>:ObjectName <<SET_DYNAMIC_VAR>>:Varname

5.1.4 - <<GET_DYNAMIC_VAR>>
Fetches a dynamic variable's value for further usage. If no value was previously assigned to the variable, a runtime error will occur.
Table head Table cell Appearance Yes Yes Allowed Syntax <<GET_DYNAMIC_VAR>>:ObjectName <<GET_DYNAMIC_VAR>>:Varname

5.1.5 - <<SET_PERSIST_VAR>>
This function allows the storage of values of the application in variables inside the Framework. The values will be stored in an INI file, so the values are persistent This function is only evaluated at runtime. Example of use: An application is tested where data sets are recorded that obtain a dynamically generated ID. This ID is required to locate the data set later again. This ID can be stored with <<SET_PERSIST_VAR>> and retrieved with <<GET_PERSIST_VAR>> later again.
Table head Table cell Appearance Yes Yes Allowed Syntax <<SET_PERSIST_VAR>>:objectName:secti on[:fileName [:encoding]] <<SET_PRSIST_VAR>>:section:key[:fileNa me [:encoding]]

5.1.6 - <<GET_PERSIST_VAR>>
Fetches a dynamic variable's value for further usage. If no value was previously assigned to the variable, a runtime error will occur.
Appearance Allowed Syntax <<GET_PRSIST_VAR>>:objectName:sectio n[:fileName [:encoding]] <<GET_PRSIST_VAR>>:section:key[:fileNa me [:encoding]]

Table head Table cell

Yes Yes

5.1.7 - <<TODAY>>
Sets a date value depending on the current client date. If the function is defined in the table head, the format is written to the table cell.
Table head Table cell Appearance Yes Yes Allowed Syntax <<TODAY>>:ObjectName <<TODAY>>[:[-|+]d[:[-|+]m[:[-|+]y[:format]]]]

The first parameter after the key word TODAY could be filled with several special semantics. To calculate a date three business days in the futture you should use the follwoing syntax: <<TODAY>>:+3 business days:0:0 More information about such special key words could be find at http://search.cpan.org/~sbeck/Date-Manip5.55/lib/Date/Manip.pod.

Page 16/29

5.1.8 - <<EXEC>>
This functions allows the execution of user defined code. Arbitrary functions may be written in the respective language of your tool and called via the framework. The content of the table cell is passed to the function as a parameter.
Table head Table cell Appearance Yes No Allowed Syntax <<EXEC>>:ObjectName

5.1.9 - <<ACTION>>
This funcion simplifies the invocation of buttons, hyperlinks or other clickable objects. When a dialog/window contains different buttons, the framework's philosophy would require a table column for each button. To activate a certain button a 'yes' should have been written to the corresponding cell. This function was introduced to simplify this. A single column with the keyword <<ACTION>> exists and the object's logical name is written to the cell.
Table head Table cell Appearance Yes No Allowed <<ACTION>> Syntax

5.1.10 - <<MENU>>
This function is the equivalent to <<ACTION>> but for menus. The menu path is written to the cell.
Table head Table cell Appearance Yes No Allowed <<MENU>> Syntax

5.1.11 - <<CHK>>
Sets the mode actions are to be executed in. If the value is located in a cell, the mode will be applied to this cell only. If the mode is located in the table head, it will be applied to the whole column.
Table head Table cell Appearance Yes Yes Allowed Syntax <<CHK>>:ObjectName <<CHK>>:CellValue

5.1.12 - <<SET>>
Sets the mode actions are to be executed in. If the value is located in a cell, the mode will be applied to this cell only. If the mode is located in the table head, it will be applied to the whole column.
Table head Table cell Appearance Yes Yes Allowed Syntax <<SET>>:ObjectName <<SET>>:CellValue

5.1.13 - <<TRANS>>
Translates a key value depending on the current language. Example of use: An application is tested that is available for different languages. All tests are to be executed successively for each language. The current language is set in the ini-file. Furthermore a listbox is assumed that offers the choice of a country. First a translation table for the values of this listbox is needed
DE ES key Deutschland Spanien DE Germany Spain EN

Table head Table cell

Appearance

Yes Yes

Allowed

Syntax <<TRANS>>:objectName:fileName [: [keyColumn [:resultColumn [:encoding]]]] <<TRANS>>:keyValue:fileName[: [keyColumn [:resultColumn [:encoding]]]]

Page 17/29

5.1.14 - <<INI>>
Loads a value from an INI file. If no filename is given, the computer name will be assumed. If no encoding is given, the value from the Ini file will be taken.
Table head Table cell Appearance Yes Yes Allowed Syntax <<INI>>:objectName:section[:fileName [:encoding]] <<INI>>:section:key[:fileName [:encoding]]

5.1.15 - <<COMMENT>>
With <<COMMENT>> comments may be written to the Log.
Table head Table cell Appearance Yes No Allowed <<COMMENT>> Syntax

5.1.16 - <<getVal>>
A user defined function may be called with <<getVal>>. The return value is then used as input value for the framework.
Table head Table cell Appearance No Yes Allowed Syntax <<getVal>>:functionName

5.1.17 - <<PCC>>
This function allows additional PCC files to be invoked from a PCC file. Thus long testcases may be separated into short and clear parts. This invocation is only allowed in the PCC File. The cell's values contain the row's ID in the invoked PCC file.
Table head Table cell Appearance Yes No Allowed Syntax <<PCC>>:NameOfThePCCFileWithoutExten sion

5.1.18 - <<DEFAULT>>
The key word <<DEFAULT>> is use to log the current value of an GUI - object. Based on the assumption that the current value was not changed directly during the test the current value should be the default valu set by the application under test. To verify such a value you have use the CHK mode and define the expected value.
Table head Table cell Appearance No Yes Allowed <<DEFAULT>> Syntax

5.1.19 - <<EMPTY>>
Using <<EMPTY>> to check that the object e.g. a text field has no value.

5.1.20 - <<MODE>>
The keyword <<MODE>> changes the current mode of the test case during execution. If no mode is explicitly set the default mode SET is used. Each test case starts with the mode SET. The new mode is valid until it is changed or the test case ends.
Table head Table cell Appearance Yes No Allowed <<MODE>> Syntax

Page 18/29

5.1.21 - <<CLEAR>>
This special value causes the test tool to set an empty value as far it is possible. As an example the text in an input field would be deleted.
Table head Table cell Appearance No Yes Allowed <<CLEAR>> Syntax

The Test Tool Part

6 - HP WinRunner
6.1 - Installation

6.1.1 - Set the Preferences

6.1.1.1 - The startup script

The startup script sets initial global variables.This script needs to be set in in the WinRunner run option as the startup script.

6.1.1.2 - The run mode

When you run your tests unattended you need to switch the run mode to 'batch', otherwise WinRunner pops up a dialog each time an unexpected behaviour occurs. This will pause the test. An unwatched test execution would not be possible.

6.1.2 - The Gui Map

The gui map maps the real gui objects of the application under test to the test tool. The logical names defined in the gui map were used in the framework to access these objects. When defining names for windows or objects in these windows a few rules need to be considered. The name must not contain any blanks or special chars. After you have renamed the object it should not be surrounded with double quotes. For the naming of gui objects it is useful to consider naming conventions. The following name prefixes were used by the author:
Button Check box Radio Button List Combo box Text field Static text Hyperlink Element "cmd" "chk" "rdb" "lst" "lst" "txt" "stc" "lnk" Prefix

These prefixes are not important for running the framework, but very handy when working with the DLG files.

6.1.3 - The custom libraries

The framework provides some custom libraries to provide extension point for project dependent extensions.

Page 19/29

6.1.3.1 - cust_gui
This library has only one function 'plf_load_all_gui' which will be called before the test starts. Here you could load your gui maps.

6.1.3.2 - cust_dlg
If you have dialogs which could not be handled by the framework and you decided to make a 'special dialog' out of it, you should define this function in this library.

6.1.3.3 - cust_callback
This library contains some callback functions which will be called by the framework. If you add your code here you could react on these 'events'.

6.1.3.4 - cust_kickoff
This library has only one function 'plf_customer_kickoff' which will be called after the main 'kickoff' before the test starts. Here you could define your own global variables and reload your own libraries.

Page 20/29

7 - HP QuickTest Pro
7.1 - Installation

7.1.1 - Set the Preferences

7.1.1.1 - The kickoff script

To use the framework just open the kickoff test located at plf/lib_qtp. All necessary scripts are linked to that test.

The Gui Map

The window declaration maps the real gui objects of the application under test to the test tool. The logical names defined in the window declaration were used in the framework to access these objects. When defining names for windows or objects in these windows a few rules need to be considered. The name must not contain any blanks or special chars. After you have renamed the object it should not be surrounded with double quotes. The GUI map have to be exported to an xml file which should be stored in the folder <MyProject>\project\data\gui . The path to that xml file is defined in the client ini in section[Testtool].

7.1.2 - The custom libraries

The framework provides some custom libraries to provide extension point for project dependent extensions. The custom libraries are located in the projectfolder in the directory qtp.

7.1.2.1 - cust_lib
This library contains all the project related functions, e.g. basestate or handling of special dialogs.

7.1.2.2 - cust_callback
This library contains some callback functions which will be called by the framework. If you add your code here you could react on these 'events'.

7.1.2.3 - cust_kickoff
This library has only one function 'plf_customer_kickoff' which will be called after the main 'kickoff' before the test starts.

Page 21/29

8 - Borland Silk
8.1 - Installation
The startup script sets initial global variables. The startup.inc library needs to be added to the startup libraries in the options.

8.2 - The Gui Map

The window declaration maps the real gui objects of the application under test to the test tool. The logical names defined in the window declaration were used in the framework to access these objects. When defining names for windows or objects in these windows a few rules need to be considered. The name must not contain any blanks or special chars. After you have renamed the object it should not be surrounded with double quotes. For the naming of gui objects it is useful to consider naming conventions. The following name prefixes were used by the author:
Button Check box Radio Button List Combo box Text field Static text Hyperlink Element "cmd" "chk" "rdb" "lst" "lst" "txt" "stc" "lnk" Prefix

These prefixes are not important for running the framework, but very handy when working with the DLG files. The Borland Silk interface is realised by a new method for each GUI object like text fields or combo boxes. Each GUI object used by the frame work need a methode called PLF. Silk only calls the method of the current object during test execution and passing the current mode and the parameters. The structure of this method is always as shown in the example below. There are two parameter passed to the methode. First parameter is always the mode which could be SET (PLF_MODE_SET) , GET (PLF_MODE_GET) or CHK (PLF_MODE_CHK). The second parameter contains all the parameter passed from the executor.

8.3 - The custom libraries

The framework provides some custom libraries to provide extension point for project dependent extensions.

8.3.1 - cust_lib
This library contains all project related functions.

8.3.2 - cust_gui
If you have dialogs which could not be handled by the framework and you decided to make a 'special dialog' out of it, you should define this function in this library.

8.3.3 - cust_callback
This library contains some callback functions which will be called by the framework. If you add your code here you could react on these 'events'.

Page 22/29

9 - IBM Rational Functional Tester

9.1 - Installation
Open a new Test project in IBM rational Functional Tester. Locate that project in the folder <MyProject>\project\rft After finishing the creation open an Explorer. Copy the content of the folder <MyProject>\plf\lib_rft into the above folder. After copying that switch to the test tool and update the super helper class. To do this right click on you project and select the Properties entry. Then enter the new super helper class 'qatools.smt.xde.Dialog'. After the update change to the standard Java view and update the project. Switch back to the Functional Tester view. Open the kickoff script, which is the initial starting script. The strucure of your project should now look like the following:

9.2 - GUI Map

For each dialog it is necessary to create an empty Test Script. The gui object of the dialog needs to be added to the test script. The definition of the dialog or the web page itself have to be added and called win.

Page 23/29

9.3 - The custom libraries

The framework provides some custom classes to provide extension point for project dependent extensions. The classes are located in the package qatools.smt.xde.cust.

9.3.1 - CustKickoff
This class contains one method called init. This method will be called when the test execution begins.

9.3.2 - CustDialog
If you have dialogs which could not be handled by the framework and you decided to make a 'special dialog' out of it, you should define this function in this library.

9.3.3 - CustCallback
This class contains some callback functions which will be called by the framework. If you add your code here you could react on these 'events'.

9.3.4 - CustLib
This class contains all project related functions.

9.3.5 - Custom
This class should not be changed at all. It is part of the framework and the base for all the other Cust classes.

10 - Reporting
There are several kinds of logs/reports written by the framework and/or the test tool. This section will explain all these logs/reports in detail.

10.1 - Database Reports

During execution the single actions will be logged in a MySQL database. The next section will explain the organisation and the meaning of the log database in detail. For analysing the test runs a php based web frontend is part of that framework. There is an easy to use filter screen to select the test cases of interest. Details of one selected test case will be shown in a second window.

10.1.1 - Web Front End

The web frontend provides an easy to use analysis of the test results. It is written in PHP and requires an enabled session module. The files need to copy into a directors of the web server, e.g. htdocs.

Page 24/29

10.1.1.1 - Installation and Configuration

To use the web frontend just copy the folder web_frontend into the web server directory. It requires PHP 5.0 or newer and an activated session support. Before using the web frontend the database information needs to be updated. The values could be set in the top of the file smartest.inc.

10.1.1.2 - Main page

The main site shows an overview about all executed test cases diverted by client, suite, start time, pcc and test case name. The list boxes supports multi selection. The text fields above could be used for regular expressions.

After every selection the content of the list boxes will be reloaded. If a regular expression is inserted the reload will be done once the field is left. A simple crieria for a test case would be test. Now all test cases containg test will be selected. Scrolling in the list box the selection will be visible. The same could be done e.g. for the execution time using the date 2007-06-27 as a criteria. Now all test cases executed on this day will be selected. The button Show Runs triggers the load of all test cases hit the search criteria. The bunber of displayed test cases is limited by 1,000. By a click on the button Reset Selection the current selection will be reseted and the main page will be reloaded wihout any preselection.

Page 25/29

10.1.1.3 - Test Case Overview

The list of test cases displays a short overview containing the number of test cases in total, the number of passed, failed and not completed test cases. The percentage of passed and failed test cases based on the total number of complete executed test cases. As seen in the example below 100% of the complete executed test cases are passed.

10.1.1.4 - Test Case Details

In the overview list a click on the test case name opens a new browser window and displays the details of the selected test case. The header shows a short statistic.

Page 26/29

If a test case has the status pass a green square symbolizes a passed test case. A red square symbolizes a failed test case. Clicking on that square the position of the error will be scrolled into view.

10.1.2 - Delete Test Cases

In the test case overview check boxes are displayed. Selecting the top check box all test cases will be selected. Selecting a check box in a header of a run the complete run will be selected. It is also possible to select single test cases. Click on the button Delete selected a new screen will be displayed showing all selected test cases for deletion. After confirming by clicking on Delete Selected the test case results will be deleted. Clicking on Cancel the main page will be displayed. The previous selection is removed.

10.2 - Log Database

MySQL 4.0 or higher is used as log database. It can be installed on any computer. In any case the use of a separate computer for the database is recommended.

10.2.1 - Database Model

Page 27/29

10.2.1.1 - Suite
id name client begin end Field Integer Text Text DateTime DateTime Type Description Unique ID for each data set The name of the suite file The name of the client computer the test was executed on The time (timestamp) the test was started The time (timestamp) the test was finished

If a suite is used in a distributed environment, a separate suite record will be generated for each client computer, even if the same suite was used on each computer.

10.2.1.2 - Info
id suite_id key_name value Field Integer Integer Text Text Type Description Unique ID for each data set Foreign key and the ID of this suite An arbitrary key name The value corresponding to the key_name

This table contains additional information about the environment the test was executed in, e.g. the version of Internet Explorer if available or the version of the operating system. Such an entry would look similar to this: OS -> Windows 2000 SP2. In this case 'OS' would be the 'key_name' and 'Windows 2000 SP2' the 'value'. The framework provides funktions/methods to insert own values here at runtime. E.g. an own fonction/method could be written that reads the current version number from the application to be tested and inserts it here. Details may be found in the section below the respective test tool in use.

10.2.1.3 - Testcase
id suite_id file_name dataset_name obj_id begin end status Field Integer Integer Text Text Text DateTime DateTime Varchar Type Description Unique ID for each data set Foreign key and the ID of this suite The name of the PCC file The name of the record inside the PCC file A unique Id for each test object that also reflects the hierarchy. (0001.0003) This would be the third testcase of this suite. The time (timestamp) the test was started The time (timestamp) the test was finished PASS | WARNING | ERROR The status of this testcase

The testcase is the real testcase.

10.2.1.4 - Dialog
id suite_id file_name dataset_name obj_id Field Integer Integer Text Text Text Type Description Unique ID for each data set Foreign key and the ID of this suite The name of the PCC file The name of the record inside the PCC file A unique Id for each test object that also reflects the hierarchy. (0001.0003.0009) This would be the ninth dialog in the third testcase of this suite The time (timestamp) the test was started The time (timestamp) the test was finished PASS | WARNING | ERROR | RUN The status of this dialog

begin end status

DateTime DateTime Varchar

In the standard case a dialog corresponds to a dialog or a web site in the application to be tested. With the start of a testcase all dialogs of the testcase are inserted into the database. The status is RUN in this case. The status is not reset until the dialog has finished or an error occurs while executing it.

Page 28/29

10.2.1.5 - Action
id suite_id element method method_value obj_id Field Integer Integer Text Text Text DateTime Type Description Unique ID for each data set Foreign key and the ID of this suite The name of the Gui element. The method/function to be called A value for the method/function to be executed. A unique Id for each test object that also reflects the hierarchy. (0001.0003.0009.0004) This would be the fourth action in the ninth dialog in the third testcase of this suite. The timestamp this action was last updated PASS | WARNING | ERROR | RUN The status of this action.

timestamp status

Timestamp Varchar

An action corresponds to an action the test tool performs. Not each of the fields element, method and method_value is always filled. With the start of a dialog all actions of the dialog are inserted into the database. The status is RUN in this case. The status is not reset until the action has finished or an error occurs while performing it. An action may be very large

10.2.1.6 - Step_Log
id suite_id description obj_id Field Integer Integer Text DateTime Type Description Unique ID for each data set Foreign key and the ID of the suite The text of this log entry A unique ID for each test object that also reflects the hierachy. (0001.0003.0009.0004) This would be a log entry for the fourth action in the ninth dialog in the third testcase of this suite (Thus different step_logs with the same obj_id may exist.) The timestamp this log entry was written PASS | INFO | WARNING | ERROR The status of this log

timestamp status

Timestamp Varchar

This table's entries are finally giving hints what really happened during a test. Why did the respective action fail, could the Gui object not be found, was the dialog no longer available or whatever. Furthermore the possibility to write your own log entries is given. Details about that may be found in the section: below the respective test tool in use.

10.2.1.7 - Attachment
id stp_log_id type size name comment content Field Integer Integer Varchar Integer Varchar Varchar LongBlob Type Description Unique ID for each data set Foreign key and the ID of the step_log entry The mime type of the attachement The size in bytes The filename if available An arbitrary description The attachement itself

Any attachment may be inserted.

Page 29/29