Test Automation Using SilkTest

Test Automation Rules

Product feature understanding, test planning, test case documentation, test bed setup, defect tracking, progression tests are all manual Test automation success depends on test cases Every manual step has commands in test script If it is not worth, do NOT automate
2

Tools - Comparison
WinRunner, QTP are from Mercury Interactive USD 1bn company and market leader; Robot from Rational IBM - USD 100bn company Silk from Segue USD 50mn company Script here is object oriented and close to QTP script. Winrunner TSL is like C and Robot SQA Basic is like Visual Basic GUI to create verification points are better in WinRunner, QTP and Robot Silk is more towards moving scripts on MS Windows and Unix; whereas Robot or WinRunner scripts cannot be moved to other OS as those tools do not support Rational does not provide GUI specification in a separate file; WinRunner, QTP and Silk provide that You need to buy Java Plugin for WinRunner; Silk and Robot come with Java plugin Record, replay, verification method, data driven tests, database connectivity, bitmap verification are all same at concept level and differ only at script syntax level Wizards are less in SilkTest, compared to other tools Cost-wise, QTP and WinRunner are most expensive; Silk stands in between WinRunner and Rational Robot

Architecture
Project .vtp (for maintenance) Testplan .pln (organizing the tests) Test Frame* .inc (AUT Window details) Scripts .t (test scripts) Data driven tests .g.t (scripts with many data) Include file .inc Suite .s (for batch runs) Result .res
5

Test Automation Principle

Make the tool learn about AUT GUI Record steps based on mouse and key events Organize steps into test case Organize test cases into test plans Run test cases Batch different sets and run Analyze results Do NOT try to code application logic into testing code Whenever possible, generate code thru the tool and not hand-code Read STUSER Help file (user guide for SilkTest)
6

Test Process
Manage requirements using SilkCentral Test Manager Manage test plans and cases using Silk test Associate test plans/cases to requirements using test manager Declare GUI declarations using SilkTest Develop scripts for atomic activities first so that they can be reused later Develop scripts using SilkTest and run them Track defects using Defect Tracker
7

Test Frames
Before test case recording, do this step This creates an .inc file that has GUI declarations Identifies GUI controls as classes Label or caption is the best way to identify objects; if not available, object index is used (available tags are caption, prior text, index, window id, coordinates) Class and tag are the key elements in declarations Record window declarations is the way to store test frames When objects are not identified properly, use different tags to uniquely identify the objects. This will vary from application to application If object X,Y coordinates are to be used, declare constants and then use those constants for further declarations To declare new objects within a window, it is better to use AnyWin or CustomWin classes. Use ~ActiveApp to specify the coordinates
8

Test Frames
Along with window declarations, we can declare methods that are specific to that window and objects The method declarations within a window object is similar to object oriented programming Uses can use these method declarations as wrapper classes for executing a series of steps that are recorded To the extent possible, do not have any hard coded values in the GUI Window methods While recording new test cases, ensure that you update window declarations as well; but check the frame.inc file, not to have duplicate declarations for same window Default is frame.inc; but we can have any number of include files Unless the window declarations are in scope, the test script will not recognize the controls

10

Recording
Recording will automatically generate statements on every user action To capture the mouse click coordinates and keyboard events, we can use low level enabler for every control in recording options We can record the base state as well as test steps Soon after recording, ensure the test frame declarations, for any changes or duplicates Recording can be done for window declarations or test case or actions Every recorded script must start from a known base state and must return back to the same known base state BaseState and DefaultBaseState can also be specified using recording (appState is a function that ensures the pre-condition state for a test case)

11

12

13

Test Cases
Each test case must be independent Initialize (pre-requisite setup), run and verify, cleanup One test case must have One purpose Establish application state for every test case. Record the base state Test Case contains commands for every test step. Record the steps and edit if needed Establish verification points for every test case Record the clean up steps as well Sequencing of test cases is very important when it comes to automation These are also functions, with the type as testcase We can use parameters for testcase as well A testcase is also a function by itself, but it can have base states

14

4Test Elements
Variables INTEGER, STRING, BOOLEAN, ANYTYPE, REAL, LIST Variables do have scope public or private Const is for constants Regular arithmetic and string operations hold good 2 types of for statements Foreach and While loops are also available IF, select-case and SWITCH-case are available for branching Do-except is for exception handling Exception functions will be useful for tracing exception number and details Test steps will have window.object.method or window.object.property style of commands User defined functions with in, out, inout parameters Functions can return values as well Use built-in functions for list and array manipulations Print and Printf commands are used for sending output to results file

15

Built-in Functions, Classes

For debugging purpose, we can use print function. It helps in documentation as well as to track the test run status, at any given point of time Refer to General Core Classes section in SilkTest Help (PARTNER file) AnyWin, MainWin, CheckBox, ComboBOx, DialogBox, ListBox, Menu, PushButton, RadioButton, StaticText, TextField, StatusBar are the most used classes Refer to Functions by Programming Task section in SilkTest Help (PARTNER file) Datatype commands, Database commands, List and string Manipulations, Window Information commands are the most used ones System related functions can be used to carry out any DOS commands to set the test environment (Sys_Execute, Sys_Kill) String functions such as ltrim, rtrim, etc are used extensively to manipulate test data; arithmetic functions are not used often User defined functions must be called from main function
16

17

18

AnyWin Properties & Methods

CaptureBitmap ClearTrap Click DoubleClick Exists GenerateDecl GetAppId GetArrayProperty GetBitmapCRC GetCaption GetChildren GetClass GetContents GetEverything GetHandle GetHelpText GetIDGetIndex GetInputLanguage GetManyProperties GetName GetNativeClass GetParent GetProperty GetPropertyList GetRect GetTag InvokeMethods IsActive IsArrayProperty IsDefined IsEnabled IsOfClass IsVisible InvokeJava MenuSelect MoveMouse MultiClic

19

AnyWin Properties & Methods

MultiClick PopupSelect PressKeys PressMouse ReleaseKeys ReleaseMouse ScrollIntoView SetArrayProperty SetInputLanguage SetProperty SetTrap TypeKeys VerifyActive VerifyBitmap VerifyEnabled VerifyEverything VerifyText VerifyProperties WaitBitmap bActive AppId sCaption lwChildren Class bEnabled bExists sID iIndex sName wParent Rect hWnd WndTag

20

Method examples
Find.OK.Click Find.MatchCase.Check Find.FindText.SetText(abc) Notepad.CloseWindows Find.Up.VerifyValue(TRUE) Fonts.FontName.verifyContents() Notepad.Editor.SelectAll() // this is a custom method to select all contents. This needs to be declared in the windows declarations User defined functions can have in, out, inout and optional parameters ClipBoard functions are used to manipulate clipboard text Cursor class methods can be used to manipulate coordinates Property prefixes b for boolean, I for int, ls for list, s for string

21

Get, Set, Verify for Controls

For every object, Get methods are used to obtain attribute values into a variable. After getting to a variable, we need to manipulate For every object, Set methods are used to set attribute values on the screen. This will work only when the object cooperates with SilkTest operations For every object, Verify methods are used to check the attribute values implicit Get and an if-then-else condition. It must have an expected value as argument Whatever properties that can be used for get statement, can be used for set and verify statements Verify methods evaluate to pass or fail status Any other custom methods can be used, if they are already declared in window declarations Most of the properties and methods are inherited from AnyWin to all other GUI classes Dlls can be loaded using dll declaration and the functions can be invoked from 4Test Capture and Compare bitmap commands are used for image checks
22

Utility Elements
Array functions are used to manipulate arrays (ArraySize, ArrayFind) List functions are used to manipulate lists (ListSort, ListAppend) DateStr, TimeStr, Sleep, StartTimer, StopTimer are used for timer functions FileOpen, FileClose, FileWriteLine, FileReadLine are used for file operations Wild cards - ? And *; MatchStr command is used for expressions Multitags enable an object to be identified with many labels Registry Operations are available not used widely (Reg_CreateKey) LogError, LogWarning, Print to write to results file Timer commands to create, start, stop, destroy and get the interval Date format and functions to manipulate date fields DLLs can be loaded and then functions can be executed; but this needs to be kept at minimum To get script, environment and browser related information, we can use the respective built-in functions (GetArgs, GetBrowserType) Date related functions are available; but it is advised not to use inside script to manipulate dates
23

Database Test
Database tests follow the verification of GUI verifications Database commands start with DB_ Connect to DB (DSN needed) Execute Query (give SELECT) Fetch the records and compare with baseline Finish DB operation (release resources) Disconnect after every database operation Do not keep the database connections for a long time in 4Test scripts
24

Verification
Object properties are the key elements Control state, value, label, text, count, size, location are the most common properties Feed the expected result to the verify command Bitmaps can be verified using CaptureBitmap and VerifyBitmap Use Bitmap tool to capture baseline image Is* methods evaluate to a boolean value (IsActive, IsEnabled) LinkTester checks the links in web applications
25

Data Driven Tests

Same script, variety of data, less coding Use Data Driven work flow for a standard recorded test case Create a test database and configure DSN Replace hard coded recorded values to data driven variables (using find/replace option) When you run the test, it runs for all rows of data in the data set A new test case with DD_ prefix will be created
26

27

Run Tests
One can run a test case or test plan or test suite Suite contains many test cases in it Using distributed run mode, one can run scripts in another machine that has Silk Agent We can run in run mode or debug mode Debug mode enables us to step, step into, step out, declare breakpoints etc All other hosts run the Agent Spawn, Parallel, Rendezvous are used in distributed testing but unless required, distributed testing is not widely used One must connect and disconnect to target machines
28

Traceability
Requirements to test traceability is thru Test Manager Test case to tester traceability is again part of test manager We need to define naming conventions by ourselves for test plans, test cases, requirements Suggested guideline is to name the test case with the test case ID itself and have the test case description as comments Use base states for capturing test pre-requisites Version control is integrated with PVCS for check-in, check-out and labeling features
29

Recovery Scenarios
Recovery system enables the test to come to the Base State Record application state is used to specify the recovery scenarios Exception handler itself is script recovery feature StopRunning() method bypasses all recovery settings
30

Test Extensions
These are used to help SilkTest to identify new objects and their properties Silktest can automatically enable extensions for open applications Java, .Net, AOL Browser, Netscape Navigator, Internet Explorer require extensions For browsers, enabling DOM or Virtual Object mode would be configured to identify objects

31

32