Professional Documents
Culture Documents
All Segue documentation and training materials are copyrighted, and all rights are reserved. Neither
the documentation nor any software that accompanies it may be reproduced, translated, or reduced
to any electronic or printed form without the prior consent of Segue Software, Inc., except as
authorized in the terms of a valid license agreement.
Copyright 1998 Segue Software, Inc.
All Rights Reserved. Printed July, 1998. Printed in the United States.
Trademarks
LiveQuality, LiveQuality BackOffice, LiveQuality Delivery, LiveQuality FrontOffice, LiveQuality
InterBusiness, LiveQuality Producer, LiveQuality Realizer, LiveQuality WebSite, QualityWare,
QualityWorks, QA Partner, QA Organizer, QA DBTester, QA Radar, QA Performer, Segue,
SilkPerformer, SilkTest, Universal Testing Architecture, and 4Test are trademarks or registered
trademarks of Segue Software, Inc.
All other system and product names are either trademarks or registered trademarks of their
respective companies.
U.S. Patent No. 5,600,789.
Preface xi
Where this manual fits in xi
Typographical conventions xiii
iv User’s Guide
Viewing the debugging transcript 177
Viewing the call stack 178
Viewing other script elements 178
Evaluating expressions 179
Running, resetting, and terminating execution 180
Tips on debugging 180
User’s Guide v
Measuring testplan completion 256
Dividing a testplan into a master plan and subplans 257
Editing a master plan in a multi-user environment 260
vi User’s Guide
Defining new class properties 308
Defining new verification properties 308
Defining new attributes 311
Defining new classes 313
Index 547
User’s Guide ix
x User’s Guide
Preface
The goal of this manual is to provide you with the task-oriented information
you need to accomplish your day-to-day activities using SilkTest™ and
QA Organizer™. This manual describes how to use SilkTest on Windows
platforms.
Manual Description
Tutorial Step-by-step procedures that explain how to build
and run tests and testplans, and how to interpret
the results. We recommend strongly that you go
through the tutorial before using the rest of the
documentation.
User’s Guide How to use SilkTest and QA Organizer, including
details on creating and running tests, debugging
tests, interpreting results, generalizing tests,
handling errors, cross-platform testing,
customizing SilkTest, GUI-specific tips and tools,
and an alphabetical reference of menu
commands.
Note This manual is also available
online: Select Help/Online User’s Guide
(assuming you installed the online
manual when installing SilkTest).
User’s Guide xi
PREFACE
Where this manual fits in
Manual Description
Testing Web Applications Provides a complete description of how to test
with SilkTest business applications on the Web. Provides a
hands-on QuickStart to get you up and running
quickly.
Online Help SilkTest also provides extensive online Help containing the following
information and more:
• Complete documentation for all 4Test classes, functions, statements, and
declarations
• What is new in this release
• Detailed description of the Segue® QA methodology
• Complete documentation for all installed extensions, which provide
support for third-party development environments
• Complete documentation for all installed versions of GO!
For more information about the Help contents, select Help Topics from the
Help menu and look at the topic “About this Help.”
For information about what is new in this release, select Help Topics from the
Help menu and look at the topic “What’s new.”
Typographical conventions
Variable prefixes Variables are specified in Hungarian notation, that is, prefixed with an
abbreviation for the data type. The data types and abbreviations are as
follows: anytype (a), agentoption (ao), boolean (b), browsertype (bt),
dataclass (dc), datatype (dt), guitype (gt), handle (h), integer (i), number (n),
point (p), real (r), rect (re), semaphore (se), string (s), string or integer (si),
sbrange (sr), textpos (tp), textrange (tr), window (w), winclass (wc), winstate
(ws) and wndtag (wt). Any of these types can be a list, such as list of string
(ls) or list of window (lw).
Chapter Page
Chapter 1, “Overview” 17
Chapter 2, “Creating Testplans” 49
Chapter 3, “Enabling Extensions for Applications Under Test” 69
Chapter 4, “Recording a Test Frame” 75
Chapter 5, “Designing and Recording Testcases” 101
Chapter 6, “Running Tests and Interpreting Results” 141
Chapter 7, “Using the Debugger” 171
User’s Guide 15
16 User’s Guide
1
rC
et
p
a
h Overview
1
This chapter gives you the big picture, so that you can quickly begin creating
powerful reusable tests in the Segue® 4Test language—tests that
dramatically decrease your risk and increase the return on your QA
investment. You’ll not only be able to do more testing with fewer staff in the
same amount of time, but by reusing tests, you’ll maintain a single standard
of quality for your application over time—across releases, platforms, and
networks.
This chapter covers the following topics:
Topic Page
The automated testing process 17
Getting started 19
The anatomy of a basic testcase 26
The built-in recovery system 27
SilkTest architecture 27
The 4Test language 29
The basic tools 34
Online information 38
Where to go from here 43
User’s Guide 17
1 OVERVIEW
The automated testing process
Group description
Test descriptions
QA Organizer
statements
Recording a test frame Next, you record a test frame, which contains descriptions, called window
declarations, of each of the GUI objects in your application. A window
declaration specifies a logical, cross-platform name for a GUI object, called
the identifier, and maps the identifier to the object’s actual name, called the
tag. In addition, the declaration indicates the type of the object, called its
class.
Creating testcases The 4Test commands in a testcase collectively perform three distinct actions:
1 Drive the application to the state to be tested.
2 Verify the state (this is the heart of the testcase).
3 Return the application to its original state.
18 User’s Guide
1 OVERVIEW
Getting started
Getting started
If you are a first-time user and want to get up and running quickly, you may
want to begin with the following:
• QuickStart Wizard
• SilkTest Web Quick Start
User’s Guide 19
1 OVERVIEW
Getting started
As prompted, you simply open your application and open the various
windows and dialogs that you want to test in the application. The wizard
automatically records all the declarations in a file called frame.inc. You
don’t have to do any coding.
3 Record testcases.
You name the testcase and provide a description for the testplan, then
simply record the testcase. Again, you don’t have to do any coding. The
wizard automatically saves the testcase in a script (.t) file with the same
name as the testplan.
4 Run testcases.
Example The following scenario illustrates how you might get started testing the Text
Editor application that ships with SilkTest.
Since the Text Editor is a standard C application, you don’t need to have any
extensions (which provide support for testing applications created with third-
party development environments) enabled before invoking the wizard.
To learn how to disable support for a particular extension, look in the SilkTest
online help for that extension.
Now you are ready to use the QuickStart Wizard with the Text Editor.
20 User’s Guide
1 OVERVIEW
Getting started
Now you will name a new testplan, which will organize and manage your
tests.
2 Click Next.
The following panel displays:
User’s Guide 21
1 OVERVIEW
Getting started
4 To create a new test frame, leave New Test Frame selected and click
Next.
At this point, the wizard lists all the open (running and not minimized)
applications. If Text Editor is not open, you can open it now (it is in the
directory where you installed SilkTest). After you open the Text Editor,
click on the QuickStart Wizard title bar to see Text Editor added to the
list of applications.
22 User’s Guide
1 OVERVIEW
Getting started
8 Now you simply open a document window and open all the dialogs that
you want to test in the Text Editor. When you place the mouse pointer on
a window or dialog, the wizard records all the declarations that SilkTest
needs in a file called frame.inc in the same directory as your testplan. For
example, when you capture the Find dialog, you see this:
9 When you have finished capturing the windows and dialogs in Text
Editor, click Return to Wizard in the Capturing New Windows dialog.
Now that you have created your test frame, you are ready to create a
testcase.
10 Click Next twice. You see the following panel.
11 Name the test FindBox and enter the description “Verify controls in Find
dialog.” Click Next.
Your test is now being recorded, as indicated by the Record Status
window on your screen.
User’s Guide 23
1 OVERVIEW
Getting started
12 Now go to Text Editor, select Search/Find to open the Find dialog, place
your mouse pointer over the dialog’s title bar, and press Ctrl+Alt to
verify its state. The Verify Window dialog displays. Click OK to verify
all properties for the dialog. Close the Find dialog (to return to your base
state), then click Done in the Record Status window.
You return to the Wizard and are asked to confirm that the test is what
you want.
13 Click Next.
14 Run the test by clicking the Run Test Button.
15 The wizard reports the results. You can move the wizard to the side and
look at the results file that is created whenever you run a test.
16 In the wizard, click Next to save your testcase.
The testcase is saved in a script (.t) file with the same name as the
testplan (in this case, edit.t).
17 Click Close to close the wizard.
You see a window containing the results file from the test you just ran. In
another window is the testplan:
What has been SilkTest has recorded all the needed information for you in the testplan:
recorded in the
testplan • The first line records the script file containing the testcase.
• The second section (under Header:) contains information that the wizard
uses to find files. Click the plus sign to the left of “header” to see the
information.
Warning The wizard maintains the information in the header
section. You should not make changes in this section.
• The last section lists the testcase you just recorded.
Using the wizard from Notice the five large buttons at the top of the testplan. Those are wizard
a testplan buttons. You can click them anytime to have the wizard step you through one
of the testing processes.
24 User’s Guide
1 OVERVIEW
Getting started
Disable Wizard
Run one testcase
Run all testcases in plan
Record a testcase
Identify new windows
Whenever you invoke a wizard function by clicking one of its buttons, the
wizard tells SilkTest which files to use for this particular testplan (it does this
by automatically populating the Runtime Options dialog, which is described
in “Options/Runtime...” on page 484).
Clicking Disable Wizard removes the buttons from the testplan. You can
reactivate the wizard from the Options menu.
How the wizard sets When testing Web applications, the wizard sets the application state to
application states DefaultBaseState, thus activating the recovery system. When testing non-
Web applications, the wizard sets the application state to none, thus
deactivating the recovery system. For more information about application
states and the recovery system, see “The built-in recovery system” on
page 27.
Where to go next Once you have recorded one or more testcases using the wizard, you will
probably want to go to your testplan and use the outline editor to manage it.
Also, you might eventually want to use the outline editor to edit your
testcases to make them more powerful and general.
Before exploring the outline editor and the other tools, you should read “The
anatomy of a basic testcase” on page 26 to get a good conceptual
understanding of testcases.
User’s Guide 25
1 OVERVIEW
The anatomy of a basic testcase
Name of testcase
Testcase keyword
Object-oriented
commands
Testcase keyword
Object-oriented
commands
The testcase keyword Each automated test begins with the testcase
keyword. If you have worked with computer languages before, you can
consider a testcase a special kind of function, one with built-in error recovery
and results reporting capabilities.
The testcase name Immediately after the testcase keyword is the name of
the testcase. The testcase is a name of your choice that indicates the testing
task being performed.
26 User’s Guide
1 OVERVIEW
SilkTest architecture
SilkTest architecture
Normal use of an application consists of a person manipulating a keyboard
and mouse to initiate application operations. The person is said to be
interacting with the GUI (Graphical User Interface). During SilkTest testing,
SilkTest interacts with the GUI to submit operations to the application
automatically. Thus SilkTest can simulate the actions of a person who is
exercising all the capabilities of an application and verifying the results of
each operation. The simulated user (SilkTest) is said to be driving the
application. The application under test reacts to the simulated user exactly as
it would react to a human user.
SilkTest consists of two distinct software components that execute in separate
processes:
• The SilkTest host software
• The 4Test Agent software
User’s Guide 27
1 OVERVIEW
SilkTest architecture
SilkTest host software The SilkTest host software is the program you use to develop, edit, compile,
run, and debug your 4Test scripts and testplans. This manual refers to the
system that runs this program as the host machine or the SilkTest machine.
The Agent The 4Test Agent is the software process that translates the commands in your
4Test scripts into GUI-specific commands. In other words, it is the Agent that
actually drives and monitors the application you are testing. One Agent can
run locally on the host machine. In a networked environment, any number of
Agents can run on remote machines. This manual refers to the systems that
run remote Agents as target machines.
In a client/server environment, SilkTest drives the client application by means
of an Agent process running on each application’s machine. The application
then drives the server just as it always does. SilkTest is also capable of
driving the GUI belonging to a server or of directly driving a server database
by running scripts that submit SQL statements to the database. These
methods of directly manipulating the server application are intended to
support testing in which the client application drives the server.
Note For more information on client/server testing, see Part V,
“Testing Client/Server Applications”.
The Agent menu If the Agent has been started, an Agent button appears in the taskbar. Press
the right mouse button over the Agent button to display the Agent menu. The
menu contains the standard menu items for a Window, plus the following:
28 User’s Guide
1 OVERVIEW
The 4Test language
Object-oriented features
Classes and methods Classes are the core of object-oriented languages. For each kind of GUI
object, there is an associated class that defines the actions, called methods,
that can be performed on all objects of that type. For example, the
PushButton class defines the methods that can be performed on all the
pushbuttons in your application.
Inheritance Classes are organized in a hierarchy. The reason for this is that classes are
related to each other, sharing characteristics common to their parent classes.
For example, the parent class Control defines all the characteristics common
to all kinds of controls: check boxes, text fields, pushbuttons, and so on. In
User’s Guide 29
1 OVERVIEW
The 4Test language
this way, each class does not need to define all the methods it needs; the class
can just inherit the existing definition from its ancestor classes.
For example, one action you can perform on objects of class DialogBox is the
GetDefaultButton method. This method returns the identifier of the default
pushbutton, which is the pushbutton that is pressed when the Return or Enter
key is pressed. You could not use this method with objects that were not
dialog boxes, such as menus or text fields.
When you record testcases, the proper 4Test methods for each of your
manual actions are recorded automatically for you, so it really is transparent
to you which class the recorded methods belong to. However, if you decide to
write or augment a testcase by hand, you can look up the class and the
methods it supports in the online Help or Library Browser.
The class hierarchy The following illustration shows the hierarchy of the core classes of GUI
objects recognized by SilkTest.
30 User’s Guide
1 OVERVIEW
The 4Test language
For information about all these classes, see the online Help.
Note Extensions to SilkTest provide additional classes that are
specific to particular development environments. For example, the
Java extension includes classes that are specific to GUI objects in
Java applications. For information about these classes, see the online
Help for the extension (assuming you have installed the extension).
User’s Guide 31
1 OVERVIEW
The 4Test language
Properties A property is a characteristic of an object that you can access directly. You
typically verify a GUI object’s properties in a testcase. You will learn more
about that in Chapter 5, “Designing and Recording Testcases”.
Statements
By using 4Test flow-of-control statements, you can add logic and robustness
to a recorded testcase. The following table summarizes the statements.
For complete information about all 4Test statements, see the online Help.
32 User’s Guide
1 OVERVIEW
The 4Test language
Example The following 4Test code shows how to use the if statement to
conditionally execute a method. The code picks the Close menu item from
the File menu of the Text Editor application, which causes a message box to
appear if there is unsaved work. The if statement tests for the existence of the
message box.
TextEditor.File.Close.Pick ()
if MessageBox.Exists () // if the message box exists
MessageBox.No.Click () // then dismiss it
User-defined types You can also create new data types, including enumerated types and records.
For example:
type FILE is LIST OF STRING
type COLOR is enum
red
green
Adding variables to a In the following testcase a record data type is defined that contains each of
testcase the data values the testcase needs. The methods in the testcase process fields
from the record instead of literal data values.
User’s Guide 33
1 OVERVIEW
The basic tools
Built-in functions
4Test contains a function library to handle the most common programming
tasks. The following table summarizes the functions by category:
Application state List manipulation Startup
Array manipulation Numeric operations String manipulation
Char/string conversion Random values System calls
Data type manipulation Results file operations Timers, time/date info
Distributed processing Script information Window information
Exception handling Semaphore operations
File manipulation Set manipulation
For complete information about all 4Test functions, see the online Help.
34 User’s Guide
1 OVERVIEW
The basic tools
Expanded testcase
Notice how the statements that make up the body of the expanded testcase are
indented. The outline editor uses indentation to differentiate levels.
A minus icon ( ) indicates that a level is fully expanded. Similarly, a plus
icon ( ) indicates that a level is collapsed and has hidden detail. Click on the
icons to expand and collapse the outline, to show the desired level of detail.
Outlining commands As you work in the outline editor, you can use menu, keyboard, or tool bar
commands to change indentation levels. The following table summarizes the
commands:
Line continuation Use Shift+Enter to force a soft line break within a long line of code. The
character outline editor considers such lines as one statement.
Using an ASCII text Typically, you write scripts and testplans using the built-in outline editor,
editor which is optimized for managing scripts and testplans.
User’s Guide 35
1 OVERVIEW
The basic tools
However, you can also use your favorite ASCII text editor to write and edit
scripts and testplans, then open them in the outline editor for execution. You
can even make changes in the outline editor, save the changes, then reopen
the modified file in your ASCII editor for continued editing.
You can use these scripts and testplans the same way you use files created in
the outline editor: Select File/Open to open them. You can make and save
changes in the outline editor, then reopen the file in your ASCII editor for
further revision.
Using Microsoft Word You can also use Microsoft Word to develop testplans. For more information,
see “Developing your outline in Microsoft Word” on page 57.
36 User’s Guide
1 OVERVIEW
The basic tools
Summary
statistics
Collapse or
expand results
Click on
box icon
The debugger
SilkTest comes with a powerful set of debugging tools to help you find
problems with your scripts. Using these debugging tools, you can step
through a script a line at a time and stop at specified breakpoints, as well as
examine local and global variables and enter expressions to evaluate.
User’s Guide 37
1 OVERVIEW
Online information
The following figure shows a testcase in the debugger with a breakpoint set.
Icon in margin
indicates line
currently executing
Icon in margin
indicates
breakpoint
Online information
You also have the following sources of online information:
• The Library Browser
• Online Help
• Online User’s Guide
38 User’s Guide
1 OVERVIEW
Online information
Looking at methods 4Test classes have methods and properties. When you select the Methods or
and properties Properties tab in the Library Browser, you see a list of all the built-in and
user-defined classes in hierarchical form.
User’s Guide 39
1 OVERVIEW
Online information
Online Help
Select Help/Help Topics to invoke online Help, which provides a wealth of
information, including the comprehensive reference for all 4Test language
constructs. For information about what is in online Help, read the About This
Help topic.
40 User’s Guide
1 OVERVIEW
Online information
Full-text search
Bookmarks
Zoom
User’s Guide 41
1 OVERVIEW
Online information
As you can see, you can perform a number of functions using the menu bar,
toolbar, bookmarks, and buttons on the bottom of the window. We have
highlighted some key controls. The toolbar also provides tooltips that identify
the tool icons.
For more information about how to use Acrobat Reader, look at Adobe’s
online documentation.
Browsing from Contents and Index You can click on the Contents and
Index bookmarks to view the Table of Contents and Index as they appear in
the printed book. Click on any contents or index topic to view the page that
describes that topic.
Print pages of interest Select File/Print from the Reader menu bar to print
the current page, a range of pages, or the entire Online User’s Guide.
42 User’s Guide
1 OVERVIEW
Where to go from here
Viewing graphics An optimal zoom factor for viewing graphics online is 200%. To enter this
zoom factor, click the button, then select 200 from the drop-down
list or type in the value.
Troubleshooting If you select Help/Online User’s Guide, but cannot access the book, go back
to the SilkTest Setup to make sure you install both the online User’s Guide
and Acrobat Reader 3.0. If you are missing either component, you will not be
able to bring up the online book.
Chapter Page
Chapter 2, “Creating Testplans” 49
How to work in the QA Organizer environment, use
QA Organizer keywords, and structure a testplan.
Chapter 3, “Enabling Extensions for Applications Under Test” 69
When and why to enable extensions for applications on
target and host machines.
Chapter 4, “Recording a Test Frame” 75
How to record, understand the structure of, and modify
window declarations.
Chapter 5, “Designing and Recording Testcases” 101
How to design and record testcases that verify application
correctness.
Chapter 6, “Running Tests and Interpreting Results” 141
How to run testcases and interpret test results.
Chapter 7, “Using the Debugger” 171
How to use the built-in debugger to uncover problems in
your scripts and in your application.
User’s Guide 43
1 OVERVIEW
Where to go from here
44 User’s Guide
1 OVERVIEW
Where to go from here
User’s Guide 45
1 OVERVIEW
Where to go from here
46 User’s Guide
1 OVERVIEW
Where to go from here
User’s Guide 47
1 OVERVIEW
Where to go from here
48 User’s Guide
2
rC
et
p
a
h Creating Testplans
2
If you are using QA Organizer, the first step in creating automated tests is to
create a testplan. (If you are not using QA Organizer, the first step is creating
a test frame, which is described in the next chapter.)
A QA Organizer testplan consists of two distinct parts: an outline that
describes the test requirements, and statements that connect the outline to the
4Test scripts and testcases that implement the test requirements.
What you will learn This chapter contains the following sections:
Topic Page
Starting a new testplan 49
Understanding the outline model 50
Structuring an outline 52
Using a template of the GUI hierarchy as an outline 60
Linking the testplan to scripts and testcases 61
Documenting manual tests in the testplan 67
User’s Guide 49
2 CREATING TESTPLANS
Understanding the outline model
A user enters the character(s) to search for in the Find What text field, checks
the Case Sensitive check box to consider case, and selects either the Up or
Down radio button to indicate the direction of the search.
Here is what a partial outline of the testing requirements might look like:
I. Find dialog
A. Case-sensitive
1. Forward search
a. Search for a character
b. Search for a word
2. Backward search
a. Search for a character
50 User’s Guide
2 CREATING TESTPLANS
Understanding the outline model
Find dialog
User’s Guide 51
2 CREATING TESTPLANS
Structuring an outline
Structuring an outline
Because there are many ways to organize information, you can structure a
QA Organizer outline using as few or as many levels of detail as you feel are
necessary. To demonstrate the variety of organizational options, this section
presents a series of sample plans, each of which adds successive levels of
description to the outline.
Note that for completeness, each of the plans also shows the script and
testcase statements that link the descriptions to the 4Test scripts and testcases
that implement the test requirements, though how to establish these links is
not described until “Linking the testplan to scripts and testcases” on page 61.
52 User’s Guide
2 CREATING TESTPLANS
Structuring an outline
Using a list structure At its simplest, an outline is a hierarchy with just a single level of detail. In
other words, it is a list of test descriptions, with no group descriptions. For
example:
Test descriptions
QA Organizer
statements
As the figure shows, each test is fully described by a single line, which is
followed by the script and testcase that implement the test.
You may find this style of plan useful in the beginning stages of testplan
design, when you are brainstorming the list of test requirements, without
regard for the way in which the test requirements are related. It is also useful
if you are creating an ad hoc testplan that runs a set of unrelated 4Test scripts
and testcases.
Using a hierarchical The following testplan has a single level of group description, preceding the
structure level that contains each of the test descriptions. The group description
indicates that all the tests are for the Find dialog.
User’s Guide 53
2 CREATING TESTPLANS
Structuring an outline
Group description
Test descriptions
QA Organizer
statements
When working with large testplans, collapsing and expanding testplan detail
makes it easy to see as much or as little of the testplan as you need.
The next testplan adds a second level of group description which indicates
whether or not the tests in the group are case sensitive.
54 User’s Guide
2 CREATING TESTPLANS
Structuring an outline
Group descriptions
for case sensitivity
The final testplan in the series adds a third level of group descriptions which
indicate whether the tests in the group search in the forward or backward
direction.
As each of these sample testplans shows, the goal when writing testplans is to
create a top-down outline that describes all of the test requirements, from the
most general (as in Find dialog) to the most specific (as in Find dialog, case-
sensitive, forward, character search).
User’s Guide 55
2 CREATING TESTPLANS
Structuring an outline
How to indent and To enter the group and test descriptions that constitute the outline, you use
change levels menu, keyboard, or toolbar commands to change the level as you are typing
the descriptions. The following table summarizes the commands:
In the Editor Item list box at the left of the dialog, select the outline editor
item to change. Then apply a color to the item by selecting a pushbutton from
the list of predefined colors.
56 User’s Guide
2 CREATING TESTPLANS
Structuring an outline
You can create a new color to apply by selecting the red, green, and blue
values that compose the color, as shown in the preceding illustration.
Adding comments that You can add comments to your testplans that will display in the results when
display in the results you run your tests. Comments are a handy way to annotate your tests to make
it easier to interpret your results.
To add a comment, include the following statement in your plan:
comment: Your comment text
User’s Guide 57
2 CREATING TESTPLANS
Structuring an outline
Setting up for the Once you have a testplan as a Word outline, you can convert it into a testplan.
conversions To do this, you use a Word macro that is provided in a template that is
shipped with SilkTest. First, you use Word’s Macro Organizer to make the
macro available to all your Word documents.
58 User’s Guide
2 CREATING TESTPLANS
Structuring an outline
4 Click the Copy button to copy the WordToQAP macro to your global
template (normal.dot).
5 Close the dialog.
The WordToQAP macro is now available to all your documents.
6 Close qapmacro.dot. You don’t need this file anymore since the macro
has been copied into your normal.dot file.
Converting an outline Procedure To convert a Word outline into a testplan:
into a testplan
1 Open your outline in Word.
2 Select Tools/Macro.
3 Select WordToQAP and click Run.
4 The macro’s Testplan Filename dialog appears.
5 Specify the full pathname of the testplan file you want to create. You
must include the.pln extension. (Don’t press Enter to dismiss the dialog.
If you do, you will see an error dialog saying that it could not create a
file.)
6 Click OK.
7 The macro converts the Word outline into a testplan. You see a
confirmation dialog.
You can now open the testplan and continue working on it.
User’s Guide 59
2 CREATING TESTPLANS
Using a template of the GUI hierarchy as an outline
(Note that the conversion is one direction only. You cannot use the macro to
take a testplan and convert it into a Word document.)
Characters that do A small number of characters may not convert properly. If this occurs you
not convert will have to make the corrections yourself by editing the testplan after the
conversion. For example, the quote (") character may not convert properly in
certain fonts.
2 Select each of the GUI objects that are related to the application features
you want to test (because this is a multi-select list box, the objects do not
have to be contiguous).
For each selected object, two lines of descriptive text are inserted into the
testplan.
60 User’s Guide
2 CREATING TESTPLANS
Linking the testplan to scripts and testcases
For example, here is the template QA Organizer inserts for the Find
dialog of the Text Editor application:
Tests for DialogBox Find
Tests for StaticText FindWhatText
(Insert tests here)
Tests for TextField FindWhat
(Insert tests here)
Tests for CheckBox CaseSensitive
(Insert tests here)
Tests for StaticText DirectionText
(Insert tests here)
Tests for PushButton FindNext
(Insert tests here)
Tests for PushButton Cancel
(Insert tests here)
Tests for RadioList Direction
(Insert tests here)
User’s Guide 61
2 CREATING TESTPLANS
Linking the testplan to scripts and testcases
You can also insert a testcase statement at the group level, although doing so
is only appropriate when the testcase is data-driven, meaning that it receives
test data from the plan (otherwise the same testcase would be called several
times with no difference in outcome). See “Using data-driven testcases” on
page 189 and Chapter 10, “Adding Data to a Testplan” for more details.
Three ways to link a There are three ways to associate a testplan with a 4Test script and testcase:
testplan with a script
or testcase • Use the Testplan Detail dialog to automate the process
• Enter the script and testcase statements manually
• Record the testcase from within the testplan
The benefits and details of the first two methods are described in the
following sections. The third method is described in “Recording from within
a QA Organizer testplan” on page 136.
How to link using the The Testplan Detail dialog automates the process of linking to scripts and
Testplan Detail dialog testcases. It lets you browse directories and select script and testcase names,
and it enters the correct QA Organizer syntax into the plan for you.
62 User’s Guide
2 CREATING TESTPLANS
Linking the testplan to scripts and testcases
2 Select Testplan/Detail.
QA Organizer invokes the Testplan Detail dialog, with the Test
Execution tab showing:
The multi-line list box at the top of the dialog displays the line in the
testplan that the cursor was on when the dialog was invoked, as well as
its ancestor lines. The black arrow icon indicates the current line. The
current line appears in black and white, and the preceding lines appear in
blue.
3 If you know the names of the script and testcase, enter them in the Script
and Testcase fields, respectively, and skip to step 8.
4 If you are unsure of the script name, click the Scripts pushbutton to the
right of the Script field to browse for the script file.
QA Organizer invokes the Testplan Detail - Script dialog:
User’s Guide 63
2 CREATING TESTPLANS
Linking the testplan to scripts and testcases
6 Click the Testcase pushbutton to the right of the Testcase field, to browse
for the testcase name:
QA Organizer invokes the Testplan Detail - Testcase dialog:
This dialog shows the names of the testcases that are contained in the
selected script. Testcases are listed alphabetically, not in the order in
which they occur in the script.
64 User’s Guide
2 CREATING TESTPLANS
Linking the testplan to scripts and testcases
8 Click OK.
The script and testcase statements are entered in the plan. For example:
User’s Guide 65
2 CREATING TESTPLANS
Linking the testplan to scripts and testcases
For example, to insert the cursor correctly to link the group description
Find dialog:
2 Enter the script and/or testcase statements using the following syntax,
substituting actual names for myscript and MyTestcase:
script: myscript.t
testcase: MyTestcase
For example, to link to the find.t script:
If you enter a statement correctly, it appears in red, the default color used for
QA Organizer statements. If not, it will either appear in blue, indicating the
line is being interpreted as a test description, or black, indicating it is being
interpreted as a group description.
66 User’s Guide
2 CREATING TESTPLANS
Documenting manual tests in the testplan
For example, the following testplan indicates that the case-insensitive tests
are manual:
Tracking manual tests You can describe the state of each of your manual tests in a dialog in SilkTest.
This information is used in reports.
User’s Guide 67
2 CREATING TESTPLANS
Documenting manual tests in the testplan
To Do this
Mark the test complete Select the Complete radio button. “Complete”
means that a test has been defined. A manual
test marked here as Complete will be tabulated
as complete in Completion reports.
Indicate whether the test Select the Has Been Run radio button, select
passed or failed Passed or Failed, specify when the test was run,
and, optionally, specify the machine. To specify
when the test was run, use this syntax:
YYYY-MM-DD HH:MM:SS
(Hours, minutes, and seconds are optional.) For
example, enter 1996-10-01 to indicate the test
was run Oct 1, 1996.
Manual tests marked as Passed or Failed will be
tabulated as such in Pass/Fail reports, as long as
you have also specified the time they were run.
(A test marked Has Been Run is also considered
complete in Completion reports.)
Add any comments you Fill in the Comments text field.
want about the test
68 User’s Guide
3
rC
et
p
a
h Enabling Extensions for
3
Introduction SilkTest provides extensions that you can install for testing applications that
use non-standard controls in specific development and browser environments.
If you plan to test applications in environments that require extensions, you
might need to enable the appropriate extensions for each application before
you begin testing.
This chapter presents an overview of why and when to enable extensions. To
learn the procedures for enabling or disabling a specific extension, please
refer to the online Help for that extension.
What you will learn This chapter contains the following sections:
Topic Page
How extensions work with applications 69
When to enable extensions 70
General process for enabling extensions 73
User’s Guide 69
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST
When to enable extensions
If you plan to use extensions for testing in specific environments, you must
tell SilkTest which extensions must be loaded for each application under
test—regardless of whether the application will run locally or on remote
machines. You do this by making sure extensions are enabled on your host
machine and on each target machine before you record or run tests.
Once extensions are enabled for applications under test, SilkTest can send
each Agent the list of extensions to be loaded. Then, the Agent dynamically
loads the appropriate extensions for testing each application.
70 User’s Guide
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST
When to enable extensions
Extensions you must You must enable extensions manually for applications that do not run in the
enable manually extension’s standard runtime environments, as in these situations.
• You are testing PowerBuilder and Visual Basic applications, which are
not invoked in standard runtime environments.
• You developed applications in Forté, Java, Oracle Developer/2000, or
Blue Express as stand-alone applications, which you invoke outside their
standard runtime environments.
• You explicitly change the executable name of a Forté, Java, or Oracle
Developer/2000 application.
• Your application uses controls that require additional extensions. For
example, if your Forté application contains ActiveX controls that you
want to test, you must manually enable the ActiveX extension for this
application.
The Extension Enabler The Extension Enabler is the utility that allows you to enable or disable
extensions on your target machines. All information that you enter in the
Extension Enabler is stored in the file extend.ini and allows the Agent to
recognize the non-standard controls you want to test on target machines.
If the default settings satisfy your testing requirements, you do not need to
run the Extension Enabler. Otherwise, you need to invoke this utility to
modify your settings—even if you are testing locally (that is, your target and
host are the same machine).
If you will run your applications on a local or remote Windows target
machine, you should invoke the Extension Enabler from the Windows Start
menu by selecting Start/Programs/<Silk program folder>/Extension Enabler.
To invoke the Extension Enabler on a remote non-Windows target machine,
run extinst.exe, located in the directory on the target machine where you
installed the SilkTest Agent.
For more information about how to use the Extension Enabler, see the online
Help for the extensions you wish to enable for testing non-standard controls
in your application.
User’s Guide 71
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST
When to enable extensions
72 User’s Guide
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST
General process for enabling extensions
User’s Guide 73
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST
General process for enabling extensions
74 User’s Guide
4
rC
et
p
a
h Recording a Test Frame
4
Introduction Before you begin recording testcases, you first record a test frame for your
application. The test frame is the backbone that supports your testcases and
scripts. It is a file that contains all the information about your application’s
GUI objects that SilkTest needs when you record testcases. This information
minimally consists of a declaration for each GUI object, but can also include
any data that you want to associate with each GUI object, as well as any new
classes and methods that you want to define.
What you will learn This chapter contains the following sections:
Topic Page
Why window declarations make your tests robust 76
Preparing to record window declarations 76
How to record window declarations 79
Window declaration syntax and structure 83
Modifying identifiers and tags 93
Mapping custom classes to standard classes 95
Specifying how a dialog is invoked 97
Defining a method for a GUI object 97
Defining a data structure for an object 99
Modifying the message box declaration 100
Related topics See Chapter 13, “Porting Tests to Other GUIs”, to learn about the ways you
modify the test frame when porting your testcases to other GUIs.
See Chapter 14, “Supporting Internationalized Applications”, to learn about
internationalizing the test frame.
User’s Guide 75
4 RECORDING A TEST FRAME
Why window declarations make your tests robust
76 User’s Guide
4 RECORDING A TEST FRAME
Preparing to record window declarations
2 Enable support for the browsers you want to use, if they aren’t already
enabled. In the Primary Extension field for a browser, choose Enabled.
3 Select any other extensions that the browser needs for testing such as
ActiveX or fault trapping. We recommend that you do not turn on fault
trapping until you really need it.
4 Click OK to close the Extension Enabler dialog.
Enabling browser You enable support for browsers on the host machine using the Extensions
support on a host dialog. Be advised that there is overhead associated with having more than
machine one browser extension enabled, so do so only if you are actually testing more
than one browser in an automated session.
User’s Guide 77
4 RECORDING A TEST FRAME
Preparing to record window declarations
78 User’s Guide
4 RECORDING A TEST FRAME
How to record window declarations
Procedure To record declarations for the main window and menu hierarchy
of your application:
1 Start up your application.
2 Select File/New.
The New dialog appears.
User’s Guide 79
4 RECORDING A TEST FRAME
How to record window declarations
Note If you are using QA Organizer, you can also use the
QuickStart Wizard to easily record a test frame with window
declarations (see “Getting started” on page 19). If you do not have
QA Organizer, the wizard icon (the hat) shown in the preceding
dialog does not appear.
The New Test Frame dialog is displayed, allowing you to create a test
frame file for an application displayed in the Application list box. The
Application list box displays all applications that are open and not
minimized; if your test application is not listed, click Cancel, open your
application, and select File/New again.
The default file name for the new test frame file is frame.inc. (The .inc
extension denotes an include file, a file that stores declarations.) By
default, subsequent frame files in the same directory are named
frame1.inc, frame2.inc, and so on. You can edit the path, file name, or
both. Click the Browse pushbutton if you need help editing the name or
path.
4 Select your application from the Application list box.
The Command Line, Working Directory, and 4Test Identifier fields are
updated for your application:
- The Command Line field displays the path of the application’s
executable.
- The Working Directory field names the directory in which the
application’s program files and documents are located.
- The 4Test Identifier field displays the default identifier of the
application’s main window, which is based on the actual caption of
the main window.
80 User’s Guide
4 RECORDING A TEST FRAME
How to record window declarations
Window declarations appear in the outline editor, which means that the
declarations for individual GUI objects can be expanded to show detail,
collapsed to hide detail, and edited if necessary.
File name is stored in When you record a test frame, the full path of the test frame file is added to
runtime options the Use Files field of the Runtime Options dialog. This means that SilkTest
can use of the information contained in the declarations and recognize the
GUI objects in your application when you record testcases.
Note If you don’t want to use the new test frame you just recorded
and instead want to use the test frame you had been using, be sure to
respecify the pathname of the old test frame in the Runtime Options
dialog (select Options/Runtime).
Recording After you record your test application’s main window and menus, you record
declarations for a all the dialogs you want to test.
dialog
Procedure To record your application’s dialogs, use this procedure once for
each dialog in your application:
1 Make sure that the test frame file that contains the declarations for the
application’s main window is open. The dialog declarations will be
appended to this file.
2 Select Record/Window Declarations.
The Record Window Declarations dialog is displayed.
User’s Guide 81
4 RECORDING A TEST FRAME
How to record window declarations
3 Make your application active and invoke one of its dialogs, referred to in
this procedure as the target dialog.
4 If necessary, arrange windows so that you can see the target dialog.
5 Position the cursor on the title bar of the target dialog.
Note that as you move the cursor toward the title bar, the contents of the
Window Declaration list box change dynamically to reflect the object at
which you’re pointing, as well as any contained objects. When the cursor
is positioned correctly, the Window Detail group box (upper left) shows
the caption of the dialog in the Identifier field.
6 Press Ctrl+Alt.
The declaration is frozen in the lower half of the dialog.
82 User’s Guide
4 RECORDING A TEST FRAME
Window declaration syntax and structure
The wStartup constant When you record the test frame, a constant called
wStartup is created. By assigning the identifier of the login window to
wStartup and by recording a new invoke method, your tests can start the
application, enter any required information into the login window, then
dismiss the login window. For more information, see “Handling login
windows” on page 297.
Saving the test frame After you finish recording declarations for each one of your application’s
dialogs, save the test frame file.
Procedure To save a test frame, select File/Save when the test frame is the
active window. If it is a new file, it is automatically named frame.inc (if you
already have a frame.inc file, a number is appended to the file name). You can
use File/Save As to select another name.
When saving a file, SilkTest does the following:
• Saves a source file, giving it the .inc extension; the source file is an
ASCII text file, which you can edit
• Saves an object file, giving it the .ino extension; the object file is a binary
file that is executable, but not readable by you
Example If you name a frame file myframe and save it, you will end up with
two files: the source file myframe.inc, in the location you specify, and the
object file myframe.ino.
About object files Object files are used for test frames and script files.
Object files are described in “Using object files” on page 134.
User’s Guide 83
4 RECORDING A TEST FRAME
Window declaration syntax and structure
Tag Identifier
Class
About classes
The class indicates the type, or kind, of GUI object being declared.
Classes are cross Note that this is the 4Test class, not the class that the GUI itself uses
platform internally. For example, although the class might be Label on one GUI and
Text on another, 4Test uses the class name StaticText to refer to text strings
which cannot be edited.
A class defines data The class also defines methods (actions) and properties (data) that are
and behavior inherited by the GUI object. For example, if you record a declaration for a
pushbutton named OK, a testcase can legally use a method like Click on the
pushbutton because the Click method is defined at the class level. In other
words, the definition of what it means to click on a pushbutton is included
within the definition of the 4Test class itself, and this definition is inherited
by each pushbutton in the GUI. If this were not true, you would have to
define within each GUI object’s window declaration all the methods you
wanted to use on that object.
The class cannot be The class as recorded cannot be changed. The one exception is that if the
changed recorded class is CustomWin—meaning that SilkTest does not recognize the
object—you can, when appropriate, map the class to one that is recognized,
as described in “Mapping custom classes to standard classes” on page 95.
Related topics For more information on using object-oriented classes, see Chapter 16,
“Extending the Class Hierarchy”.
84 User’s Guide
4 RECORDING A TEST FRAME
Window declaration syntax and structure
Refer to the online Help for descriptions of the variables, methods, and
properties defined by each class.
About identifiers
The identifier is the GUI object’s logical name.
Default identifier is By default, SilkTest derives the identifier from the object’s actual label or
based on the label/ caption, removing any embedded spaces or special characters (such as
caption accelerators). So, for example, the Save As label becomes the identifier
SaveAs.
Note Identifiers can contain single-byte international characters,
such as é and ñ.
Identifier can be based If the object does not have a label or caption, SilkTest constructs an identifier
on the index by combining the class of the object with the object’s index. The index is the
object’s order of appearance, from top left to bottom right, in relation to its
sibling objects of the same class. For example, if a text field does not have a
label or caption, and it is the first text field within its parent object, the default
identifier is TextField1.
Identifier can be Note that the identifier is arbitrary, and you can change the generated one to
changed the unique name of your choice. See “Modifying identifiers and tags” on
page 93.
About tags
The tag is the actual name of the object, as opposed to the identifier, which is
the logical name. SilkTest uses the tag to identify objects in the application
under test when recording and when executing testcases.
Testcases never use the tag to refer to an object; they always use the identifier.
Five kinds of tags Unlike identifiers, tags are not arbitrary. There are five types of tags:
User’s Guide 85
4 RECORDING A TEST FRAME
Window declaration syntax and structure
Note Not all types of objects have all five tags. Dialogs, for
example, do not have window IDs, so they cannot have a Window ID
tag.
Example Here is what you would see if you record declarations for the Case
Sensitive check box in the Text Editor’s Find dialog.
86 User’s Guide
4 RECORDING A TEST FRAME
Window declaration syntax and structure
These are the possible tags that can be used by SilkTest to identify the Case
Sensitive check box when recording or executing testcases.
What happens by You can record more than one tag for an object. Doing so makes scripts less
default sensitive to changes when the tests are run. For example, if you record a
caption and a window ID for a control, then even if the caption on the control
changes (such as the caption “Case Sensitive” changing to “Case is
significant”), SilkTest can still find the control based on its window ID.
This is particularly an issue in situations where captions change dynamically,
such as in MDI applications where the window title changes each time a
different child window is made active.
By default, when you record window declarations, each object is given two
tags: the caption (if there is one) and the Window ID (if there is one). Notice
in the Record Declarations dialog shown above that two tags are checked in
the Tag Information box: Caption and Window ID.
For example, here is the default recorded declaration for the Case Sensitive
check box:
CheckBox CaseSensitive
multitag "Case sensitive"
"$1041"
Procedure To change which tags are recorded by default for GUI objects:
1 Select Record/Window Declarations.
User’s Guide 87
4 RECORDING A TEST FRAME
Window declaration syntax and structure
3 Select and unselect the check boxes in the Default Multitags group box
as appropriate.
4 Click OK.
The next time you record window declarations, SilkTest will use the tag
types you selected by default. You can always override the defaults for a
particular object.
Using class-specific You can specify which multiple-tag types to use for an individual class. For
multiple tags example, maybe you don’t want window ID used with a particular class, even
though you want window ID used with all other classes. You can specify this
by including a setting statement in the declaration for the class.
For more information, see “winclass declaration” in online Help.
If an object doesn’t By default, SilkTest follows these steps to create a Caption tag for an object:
have a caption or if the
caption is not unique 1 SilkTest uses the literal label or caption of the object, if there is one.
2 If the object has a sibling object with the same label or caption, SilkTest
appends the object’s index number to the tag. The index number is the
object’s order of appearance in relation to other sibling objects of the
same class, from top left to bottom right within the parent object.
For example, if a dialog has two objects labeled Find, the tag of the one
nearest the top left of the dialog is Find[1] and the tag of the one nearest
the bottom right of the dialog is Find[2].
3 If the object does not have a label or caption, SilkTest uses the index
number.
For example, if a dialog contains two unnamed text fields, the text field
closest to the upper left corner of the dialog has the tag #1, and the other
has the tag #2.
88 User’s Guide
4 RECORDING A TEST FRAME
Window declaration syntax and structure
Changing tags Sometimes you need to change tags from what SilkTest
named them by default. See “Modifying identifiers and tags” on page 93.
How SilkTest uses When running your testcases, the Agent tries to resolve each part of a
multiple tags at multiple tag from top to bottom until it finds an object that matches.
runtime
Consider this declaration:
CheckBox CaseSensitive
multitag "Case sensitive"
"#1"
User’s Guide 89
4 RECORDING A TEST FRAME
Window declaration syntax and structure
Now when you record window declarations, SilkTest will default to the
tag type you selected and record the tag in a tag statement. You can
always override the default for a particular object.
For more information, see “tag statement” in online Help.
As is true for all window declarations, the declaration for the main window is
composed of a class, identifier, and tag.
The window statement The main window declaration begins with the 4Test reserved word window.
The term window is historical, borrowed from operating systems and window
manager software, where every GUI object (for example main windows,
dialogs, menu items, and controls) is implemented as a window.
90 User’s Guide
4 RECORDING A TEST FRAME
Window declaration syntax and structure
The sCmdLine and When you record the declaration for your application’s main window and
wMainWindow menus, the sCmdLine and wMainWindow constants are created. These
constants constants allow your application to be started automatically when you run
your testcases.
The sCmdLine constant specifies the path to your application’s executable.
The following example shows sCmdLine constants for different operating
environments:
msw const sCmdLine = "d:\mytests\texted2.exe"
motif const sCmdLine = "textedit"
mac const sCmdLine = "Centris610:Sample:Text Editor"
The wMainWindow constant specifies the 4Test identifier for the main
window of your application. For example, here is the definition for the
wMainWindow constant of the Text Editor application on all platforms:
const wMainWindow = TextEditor
The menu The following example from the Text Editor application shows the default
declarations main window declaration and a portion of the declarations for the File menu:
window MainWin TextEditor
multitag "Text Editor"
"$D:\MYTESTS\TEXTED2.EXE"
.
.
.
Menu File
tag "File"
MenuItem New
multitag "New"
"$100"
Note Menus do not have window IDs, but menu items do, so by
default menus are declared with the tag statement while menu items
are declared with the multitag statement. For more information, see
“About tags” on page 85.
Declarations are The declarations for the menus are nested (indented) within the declaration
nested for the main window, and the declarations for the menu items are nested
within their respective menus. This nesting denotes the hierarchical structure
of the GUI, that is, the parent-child relationships between GUI objects.
In the sample Text Editor application, MainWin is the parent of the File
menu; the File menu is considered a child of the MainWin. Similarly, all the
menu items are child objects of their parent, the File menu. A child object
belongs to its parent object, which means that it is either logically associated
with the parent or physically contained by the parent.
User’s Guide 91
4 RECORDING A TEST FRAME
Window declaration syntax and structure
Because child objects are nested within the declaration of their parent object,
the declarations for the child objects do not need to begin with the reserved
word window.
Declarations are The declarations for the controls contained by a dialog are nested within the
nested dialog’s declaration to show the GUI hierarchy.
The parent statement Although a dialog is not physically contained by the main window, as is true
for menus, the dialog nevertheless logically belongs to the main window.
Therefore, a parent statement within each dialog declaration is used to
indicate that it belongs to the main window of the application.
92 User’s Guide
4 RECORDING A TEST FRAME
Modifying identifiers and tags
Why the declaration is The message box declaration is generic for three reasons:
generic
• The dialog’s tag specifies that its parent is the current active application.
• The most likely names for pushbuttons are accounted for: OK, Cancel,
Yes, and No.
• The tag of the message is an index number, not the text of the message.
You can enhance the generic declaration. See “Modifying the message box
declaration” on page 100.
User’s Guide 93
4 RECORDING A TEST FRAME
Modifying identifiers and tags
Why change the tags By default, the GUI object’s caption and index are used for the tag, because
they are the most portable. In most cases, these are what you want to use.
However, there are situations in which the default tag is not suitable. For
complete information about using alternative tags, see the section “How to
choose a tag-string form” in the “multitag statement” topic in online Help.
Modifying in the You can modify the identifier or tag for a dialog as you are recording it.
Record dialog
Procedure To modify a declaration in the Record Window Declarations
dialog:
1 In the Window Declaration list box at the bottom of the dialog, click the
line for the object containing the tag or identifier you want to change.
The Window Detail group box in the upper left of the dialog updates to
include the information from the line you clicked on.
2 To change the identifier, simply replace the existing identifier with one
of your choice.
To change the tag, select the tag types you want to include in the
generated multitag. You can edit the contents of each tag type in the text
fields in the Tag Information group box.
The Window Declaration list box updates dynamically as you enter the
new information.
3 When finished making modifications, press the Paste to Editor
pushbutton.
The declarations for the dialog are appended to the test frame file.
Example of changing tag You might want to provide more than one caption
for a control if the control’s caption can change dynamically. For example, if
a push button sometimes says Yes and sometimes says Continue, you could
change the tag as shown here:
Note that you separate different tag components using the pipe character ( | ).
94 User’s Guide
4 RECORDING A TEST FRAME
Mapping custom classes to standard classes
Modifying after If you need to modify the identifier or tag for the application’s main window
pasting or menus, you do so after you have pasted the declarations to the test frame
file, by editing the file directly.
User’s Guide 95
4 RECORDING A TEST FRAME
Mapping custom classes to standard classes
2 In the Window Declarations list box, click on the line containing the
declaration for the custom object.
The line is highlighted and the declaration for the CustomWin appears in
the Window Detail group box.
3 In the Window Detail group box, press the Class Map pushbutton.
The Class Map dialog appears. The name of the custom class is
displayed in the Custom Class text field.
4 In the Standard Class field, enter the name of the built-in 4Test class to
which you are mapping the custom class.
5 Click Add.
6 Click OK.
When you resume recording, the object has the standard 4Test class. Any
subsequent similar objects encountered will automatically be mapped to the
correct 4Test class. You must modify any prerecorded declarations containing
these objects to use the standard class.
96 User’s Guide
4 RECORDING A TEST FRAME
Defining a method for a GUI object
To enable the recording of these declarations, map the Custom Class of the
Ignore class widget onto itself. For example, to “un-ignore” the class
XmForm, enter XmForm in both the Custom Class and Standard Class
windows of the Class Map dialog.
You can also use Class Mapping to add custom windows to the Ignore class.
While both are equivalent, using the Invoke method makes your testcases
more maintainable. For example, if the menu pick changes, you only have to
change it in your window declarations, not in any of your testcases.
The Invoke method To use the Invoke method, the declaration for the dialog needs to specify the
dialog’s wInvoke variable, which contains the identifier of the menu item or
button that invokes the dialog. For example:
window DialogBox Open
tag "Open"
parent TextEditor
WINDOW wInvoke = TextEditor.File.Open
User’s Guide 97
4 RECORDING A TEST FRAME
Defining a method for a GUI object
3 Name the method by typing the name or selecting one of the predefined
methods: BaseState, Close, Invoke, or Dismiss.
4 Record the actions that make up the method.
5 Edit the 4Test statements that were recorded, if necessary.
6 Paste the code to your include file.
Example For example, suppose you want to create a method named SetLineNum for a
dialog named GotoLine, which does the following:
• Invokes the dialog
• Enters a line number
• Clicks the OK button
The following 4Test code shows how to add the definition for the
SetLineNum method to the GotoLine dialog’s declaration:
window DialogBox GotoLine
tag "Goto Line"
parent TextEditor
const wInvoke = TextEditor.Search.GotoLine
98 User’s Guide
4 RECORDING A TEST FRAME
Defining a data structure for an object
Where to define the When you include a method definition in a GUI object’s window declaration,
method you can only use the method on that one object. If you want to define a
method that you can use on all GUI objects of a given class, you should
define the method at the class level. For more information, see “Adding
methods to a class” on page 306.
User’s Guide 99
4 RECORDING A TEST FRAME
Modifying the message box declaration
Testcases
Introduction This chapter shows you how to use the powerful SilkTest recorder to
automatically capture the actions you perform as you interact with your
application, creating an automated testcase.
The cornerstone of an automated test is the verification stage, in which the
test verifies that the state of the application matches the expected (baseline)
state. Using the recorder, you can record object-appropriate verification of
your application’s state, data, or appearance.
What you will learn This chapter contains the following sections:
Topic Page
Preparing to design and record testcases 102
Test running the built-in recovery system 105
Testcase design principles 106
Defining test requirements 108
Overview of recording the stages of a testcase 109
Recording the setup stage 112
Recording the verification stage 112
Recording the cleanup stage and pasting the recording 132
Saving a script file 133
Using object files 134
Recording from within a QA Organizer testplan 136
How recorded commands uniquely identify GUI objects 138
Topic Page
Recording without window declarations 138
Recording 4Test components 139
2 Enable support for the browsers you want to use, if they aren’t already
enabled. In the Primary Extension field for a browser, choose Enabled.
3 Select any other extensions that the browser needs for testing such as
ActiveX or fault trapping. We recommend that you do not turn on fault
trapping until you really need it.
4 Click OK to close the Extension Enabler dialog.
Enabling browser You enable support for browsers on the host machine using the Extensions
support on a host dialog. Be advised that there is overhead associated with having more than
machine one browser extension enabled, so do so only if you are actually testing more
than one browser in an automated session.
If the built-in recovery system cannot close one of the three representative
dialogs, you need to modify the recovery system so that it understands how to
close the dialog. For more information, see “Specifying new window closing
procedures” on page 293.
2 It verifies that the actual state matches the expected (correct) state. (Your
QA department might use the term baseline to refer to this expected
state.) This stage is the heart of the testcase.
3 It cleans up the application, in preparation for the next testcase, by
undoing the steps performed in stage 1.
Each testcase is Each testcase you record should perform its own setup in stage 1, and should
independent undo this setup in stage 3, so that the testcase can be executed independently
of every other testcase. In other words, the testcase should not rely upon the
successful or unsuccessful completion of another testcase, and the order in
which it is executed should have no bearing on its outcome.
If a testcase relies on a prior testcase to perform some setup actions, and an
error causes the setup to fail or, worse yet, the application to crash, all
subsequent testcases will fail because they cannot achieve the state where the
test is designed to begin.
A testcase has a Each testcase you record should verify a single aspect of the application in
single purpose stage 2. When a testcase designed in this manner passes or fails, it’s easy to
determine specifically what aspect of the target application is either working
or not working.
If a testcase contains more than one objective, many outcomes are possible.
Therefore, an exception may not point specifically to a single failure in the
software under test but rather to several related function points. This makes
debugging more difficult and time-consuming and leads to confusion in
interpreting and quantifying results. The net result is an overall lack of
confidence in any statistics that might be generated.
Note But there are techniques you can use to do more than one
verification in a testcase. See “Performing more than one verification
in a testcase” on page 211.
A testcase starts from In order for a testcase to be able to function properly, the application must be
a base state in a stable state when the testcase begins to execute. This stable state is called
the base state. The recovery system is responsible for maintaining the base
state in the event the application fails or crashes, either during a testcase’s
execution or between testcases.
If these conditions are not sufficient for your application, you can customize
the recovery system. For more information on how the recovery system
works and how it can be customized, see Chapter 15, “Understanding the
Recovery System”.
When a user enters the criteria for the search and clicks the Find Next
pushbutton, the searching feature attempts to locate the string. If the string is
found, it is selected (highlighted). Otherwise, an informational message is
displayed.
What data does the A user can enter three pieces of information in the Find dialog:
feature expect
• The search can be case sensitive or insensitive, depending on whether the
Case Sensitive check box is set or unset.
• The search can be forward or backward, depending on whether the Down
or Up radio button is selected.
• The search can be for any combination of characters, depending on the
value entered in the Find What text field.
Create meaningful To organize this information, it is helpful to construct a table that lists the
data combinations possible combinations of inputs. From this list, you can then decide which
combinations are meaningful and should be tested. A partial table for the
Find dialog is shown below:
Note for QA Organizer users After learning the basics of recording in this
section, read about “Recording from within a QA Organizer testplan” on
page 136, which makes recording easier by automatically generating for you
the links that connect the testplan to the testcase.
Preparing to record
Procedure To invoke the Record Testcase dialog and set up the test:
1 Select Record/Testcase.
a If a script file is not the active window, you are prompted for a file
name.
2 Type the name of your testcase in the Testcase Name field of the Record
Testcase dialog. By default the name is Test1 (unless that name is already
used in the current script file, in which case the default name is Test2,
and so on). Testcase names are not case sensitive; they can be any length
and consist of any combination of alphabetic characters, numerals, and
underscore characters.
3 Select DefaultBaseState in the Application State field to have the built-in
recovery system restore the default base state before the testcase begins
executing. Select another application state if you want the recovery
system to perform an additional sequence of steps after the default base
state is restored but before the testcase begins executing. For more
information on application states, see “Using application states” on
page 185.
The application state you chose is made the default application state and
is suggested in the Record Testcase dialog the next time you record a
testcase.
Note If you choose “(None)” as the application state, the
recovery system will not be used at all. This means that you must
manually put your application in the starting state before running
the testcase.
4 If you do not want SilkTest to display the status window it normally
shows during playback when driving the application to the specified base
state—perhaps because the status bar obscures a critical control in the
application you are testing—uncheck the Show AppState status window
check box.
5 Click the Start Recording pushbutton.
The following sequence of events occurs:
- The Record Testcase dialog closes.
- Your application is started, if it was not already running.
- The editor window disappears from the display.
- The Record Status window appears.
Using the Record As you move around the application while recording, the Record Status
Status window window dynamically updates to show the object that the recorder thinks
the mouse is currently over or the control that has focus.
Use this information to make sure that the recorder has kept up with your
actions (sometimes the recorder can fall behind if your system’s
resources are low).
In particular, check the Record Status window before pressing Ctrl+Alt
to do your verification in order to make sure you are verifying the correct
object.
window you want to verify active and press Ctrl+Alt. How you make a
window active on Motif depends on how you set the
keyboardFocusPolicy resource.)
The Verify Window dialog appears over your application window.
The Window field, in the top-left corner of the dialog, displays the name
of the object you were pointing at when you pressed Ctrl+Alt.
2 If the name in the Window field is incorrect, press Cancel to close the
dialog and return to the application. Point to the object you want to verify
and press Ctrl+Alt again.
Motif note: if Ctrl+Alt If your application under test is running under OSF/Motif using an X
doesn’t work Window emulator on a PC and Ctrl+Alt does not work, do the following:
1 Quit SilkTest.
2 Create an environment variable named QAP_VERIFY_KEY and assign
it the value 1.
In the C shell, do this:
setenv QAP_VERIFY_KEY 1
In the Bourne shell, do this:
QAP_VERIFY_KEY=1; export QAP_VERIFY_KEY
In the Korn shell, do this:
export QAP_VERIFY_KEY=1
With this environment variable set, when you restart SilkTest, you can press
F5 to display the Verify Window dialog.
About the Verify The Verify Window dialog has three tabs.
Window dialog
The dialog has these tabs by default.
With Verify Using Properties deselected, the next time you go to verify an
object, the Verify Window dialog will have an Attribute tab, instead of a
Property tab. For information on verification using attributes, see “Verifying
object attributes” on page 130.
Caption is the text that appears in the dialog box’s title bar; Children is a list
of all the objects contained in the dialog box, DefaultButton is the button that
is invoked when you press Enter, and so on. In your testcases, you can verify
the state of any of these properties.
You can also, in the same testcase, verify properties of children of the
selected object. In the preceding illustration, notice that the child objects in
the Find dialog box (such as the text field FindWhat and the check box
CaseSensitive) are also selected for verification.
By recording verification statements for the values of one or more of an
object’s properties, you can determine whether the state of the application is
correct or in error when you run your testcases.
Properties can be To make your testing easier, properties are organized into sets.
verified as a set
Predefined sets Here are the predefined property sets:
The property sets listed above are the general built-in property sets.
If you have enabled an extension to provide enhanced support for testing an
application built with a particular development environment, there might be
additional property sets. For more information, see the online Help for the
extension.
If you are testing a Web application, there are additional property sets. For
more information, see Testing Web Applications with SilkTest.
Configuring property sets You can configure property sets to suit your
needs, even combining frequently used property sets into a new larger
property set. For more information, see “Configuring your own property sets”
on page 121.
• The Property Value field displays the value of the selected property.
You can edit the value in this field if it is not what you want to verify
against. The value specified in this field is the value you expect at
runtime, that is, the baseline value.
3 Choose the object(s) to verify. To verify all or most objects, click the
Check All pushbutton and then uncheck individual check boxes.
4 Choose the properties to verify in one of the following ways:
• Select an object and check one or more check boxes in the Properties
to Verify list box. Repeat this for all the properties of all objects that
you want to verify.
• Select a set of properties from the Property Set drop-down list box.
The list box displays the predefined property sets (Values is the
default), and any property sets you’ve defined yourself.
When you select a property set, all properties in that property set are
selected and all properties not in that property set are deselected. You
can select only one property set at a time.
5 Click OK to close the Verify Window dialog.
6 If you are writing a complete testcase, record the cleanup stage and paste
the testcase into the script. If you have added a verification statement to
an existing testcase, paste it into your script and close the Record Actions
dialog.
VerifyProperties A VerifyProperties method statement is added to your script. The
method statement VerifyProperties method verifies the selected properties of an object and its
children.
For more information about VerifyProperties, see the online Help.
Fuzzy verification There are situations when SilkTest cannot see the full contents of a control
(such as a text field) because of the way that the application paints the control
on the screen. One known case is text fields in cells in Dev2000Tables.
For example, consider a text field in a Dev2000Table whose contents are
wider than the display area. In some situations (but not in others), the
application clips the text to fit the display area before drawing it, meaning
that SilkTest only see the contents that are visible; not the entire contents.
Consequently, when you later do a VerifyProperties against this text field, it
may fail inappropriately. For example, say the true contents of the text field
are “29 Pagoda Street”, but only “29 Pagoda” displays. Depending on exactly
how the test is created and run, the expected value might be “29 Pagoda”
whereas the value seen at runtime might be “29 Pagoda Street” (or vice
versa). So the test would fail, even though it should pass.
To work around this problem, you can use fuzzy verification, where the rules
for when two strings match are loosened. Using fuzzy verification, the
expected and actual values do not have to exactly match. The two values are
considered to match when one of them is the same as the first or last part of
the other one. Specifically, VerifyProperties with fuzzy verification will pass
whenever any of the following functions would return TRUE (where actual is
the actual value and expected is the expected value):
• MatchStr (actual + "*", expected)
• MatchStr ("*" + actual, expected)
• MatchStr (actual, expected + "*")
• MatchStr (actual, "*" + expected)
(In string comparisons, * stands for any zero or more characters.)
For example, all the following would pass if fuzzy verification is enabled:
FuzzyVerifyWhich
value Result
FALSE (default) Fuzzy verification is disabled.
FuzzyVerifyWhich
value Result
One class Fuzzy verification is enabled for all objects of that
class.
Example window.VerifyProperties ({…},
Dev2000Table) enables fuzzy verification for all
Dev2000Tables in window (but no other object).
List of classes Fuzzy verification is enabled for all objects of each
listed class.
Example window.VerifyProperties ({…},
{Dev2000Table, TextField}) enables fuzzy verification
for all Dev2000Tables and text fields in window (but no
other object).
TRUE Fuzzy verification is enabled only for those objects
whose FuzzyVerifyProperties member is TRUE.
To set the FuzzyVerifyProperties member for an object,
add the following line within the object's declaration:
FUZZYVERIFY FuzzyVerifyProperties = TRUE
Example If in the application's include file, the
DeptDetails Dev2000Table has its
FuzzyVerifyProperties member set to TRUE:
window ChildWin EmpData
. . .
Dev2000Table DeptDetails
FUZZYVERIFY FuzzyVerifyProperties = TRUE
And the test has this line:
EmpData.VerifyProperties ({...}, TRUE)
Then fuzzy verification is enabled for the DeptDetails
table (and other objects in EmpData that have
FuzzyVerifyProperties set to TRUE), but no other
object.
Fuzzy verification takes more time than standard verification, so only use it
when necessary.
For more information, see “VerifyProperties method” in online Help.
Defining your own You can also define your own verification properties. For more information,
verification properties see “Defining new verification properties” on page 308.
The Property Sets dialog opens. The Property Sets list box displays all
existing property sets.
3 Specify a name for the new property set in the Name field. Property set
names are not case sensitive; they can be any length and consist of any
combination of alphabetic characters, numerals, and underscore
characters.
4 Specify a class in the Class field and then a property of that class in the
Property field. For more information, see “Specifying a class-property
pair” on page 123.
5 Click Add.
The class-property pair is added to the list box.
Note The class or property name is not validated here, so type
carefully. Invalid names are ignored at runtime.
If you make a mistake, select the class-property pair and click Edit.
6 Repeat steps 4 and 5 for as many class-property pairs as you want to add.
Delete any class-property pairs you don’t want to include by selecting
them and pressing Remove.
7 Once the list of classes and properties is correct, click OK.
The New Property Set dialog closes and the new property set is displayed
in the Property Sets list box.
8 Click Close to dismiss the Property Sets dialog.
3 Specify a name for the new property set in the Name field.
4 Select at least two property sets from the Property Sets to Combine list
box and click OK.
5 The Combine Property Sets dialog closes and the new property is
displayed in the Property Sets list box, as are the constituent sets.
Note If you modify any of the constituent sets, the combined set will
be modified as well.
How to edit a property You can modify an existing property set at any time to rename it or to edit the
set class-property pairs that compose it.
• A bitmap comparison will fail if the image being verified does not have
the same screen resolution, color, window frame width, and window
position when the testcase is run on a different machine than the one on
which the baseline image was captured.
• Make sure that your testcase sets the size of the application window to
the same size it was when the baseline bitmap was captured.
• Capture the smallest possible region of the image so that your test is
comparing only what is relevant.
• If practical, do not include the window’s frame (border), since this may
have different colors and/or fonts in different environments.
Option Description
OPT_BITMAP_MATCH_COUNT The number of successive snapshots that must
be the same for the bitmap to be considered
stable.
OPT_BITMAP_MATCH_INTERVAL The time interval between snapshots to use
for ensuring the stability of the image.
OPT_BITMAP_MATCH_TIMEOUT The total time allowed for a bitmap to
become stable
OPT_BITMAP_PIXEL_TOLERANCE The number of pixels of difference below
which two bitmaps are judged equivalent.
This option is not supported on OSF/Motif.
To globally set these options for a group of scripts, use the Agent Options
dialog. To locally set these options within a script or testcase, use the
SetOption method, for example:
Agent.SetOption (OPT_BITMAP_MATCH_COUNT, 3)
Using the Verify You can use the built-in Verify function to verify that two values are equal
function and generate an exception if they are not. Typically, you use Verify to test
something that doesn’t map directly to a built-in property or method. Verify
has the following syntax:
Example Say you want to verify the number of radio buttons in the
Direction RadioList in the Replace dialog in the Text Editor. There is no
property or method you can directly use to verify this. But there is a method
for RadioList, GetItemCount, which returns the number of radio buttons. You
can use the method to provide the actual value, then specify the expected
value in the script, as follows:
When doing the verification, position the mouse pointer over the RadioList
and press Ctrl+Alt. Select the Method tab in the Verify Window dialog, and
select the GetItemCount method. Click OK to close the Verify Window
dialog, and complete your testcase. Paste it into a script. You now have the
following script:
testcase VerifyFuncTest ()
TextEditor.Search.Replace.Pick ()
Replace.Direction.GetItemCount ()
Replace.Cancel.Click ()
Now use the Verify function to complete the verification statement. Change
the line
Replace.Direction.GetItemCount ()
to
Verify (Replace.Direction.GetItemCount (), 2)
That is, the call to GetItemCount (which returns the number of radio buttons)
becomes the first argument to Verify. The expected value, in this case, 2,
becomes the second argument.
Your completed script is:
testcase VerifyFuncTest ()
TextEditor.Search.Replace.Pick ()
Verify (Replace.Direction.GetItemCount (), 2)
Replace.Cancel.Click ()
• Selected range, which is the beginning and ending position of the text
string currently selected in the field, in (line, column) format. For
example a value of {1,12,1,16} means that the selected text begins on
line 1, column 12 and ends on line 1, column 16.
• Selected Text, which is the string that is currently selected, if any, in the
text field.
• Text, which is the entire contents of the text field.
By recording verification statements for the values of one or more of an
object’s attributes, you can determine whether the state of the application is
correct or in error when you run your testcases. That is: did the feature you
are testing have the expected result?
Note Attributes have been essentially rendered obsolete and have
been replaced by properties. See “Verifying using properties” on
page 115.
The list box on the left shows the attributes for the current object.
3 Select an attribute from the list box.
The current value of the attribute (that is, the value that exists when you
are recording) is shown in the Attribute Value field.
4 If the current value of the attribute is not the value you want to test for at
runtime, edit the attribute value field. The value specified in this field is
the value you expect at runtime, that is, the baseline value.
5 Click OK to accept the attribute and its value.
The Verify Window dialog is closed, and the Record Status window is
opened.
The testcase will verify that the object has the attribute value selected. If not,
an error is written to the results file.
Verifying all attributes By selecting the Verify All Attributes check box,
you can record a test that verifies the state, contents, and value of a GUI
object and any objects it contains. This is commonly called a smoke test or a
Level 1 test. A smoke test uses the VerifyEverything method to verify every
aspect of a particular GUI object.
Defining additional attributes If you need to, you can define and add your
own attributes to the built-in hierarchy of GUI classes. For more information,
see “Defining new attributes” on page 311.
If Then
All the information in the Click OK. SilkTest closes the Record
window is complete and what Testcase dialog and places the new testcase
you want. in your script file.
The 4Test code is not what Edit the code in the Testcase Code field.
you expected.
The testcase name is not what Edit the name in the Testcase Name field.
you want.
The application state is not Delete the code in the Testcase Code field,
the one you want. select a new application state from the
drop-down list and click Resume
Recording to rerecord the testcase.
The testcase is not finished. Click Resume Recording. The Record
Testcase window is reopened. You can
continue to record your interactions.
If you chose “(none)” as your application state, the testcase is recorded as:
testcase testcase_name () appstate none
The recording When you paste a recorded testcase (or other recorded actions, such as when
statement you use Record Actions) into a script, SilkTest indents the code under a
recording statement to facilitate playback. For more information about the
recording statement, see online Help.
• Saves a source file, giving it the .t extension; the source file is an ASCII
text file, which you can edit
• Saves an object file, giving it the .to extension; the object file is a binary
file that is executable, but not readable by you
Example If you name a script file mytests and save it, you will end up with
two files: the source file mytests.t, in the location you specify, and the object
file mytests.to.
Procedure To save a new version of a script’s object file when the script file
is in view-only mode, select File/Save Object File.
About object files Object files are described in the next section.
Example Here’s a simple example of how you might distribute object files
so that others cannot see the code.
In file test.inc, place the definition of a function called TestFunction. (When
you save the file, the entire include file is compiled into test.ino.)
TestFunction ()
ListPrint (Desktop.GetActive ())
In the file test.t use the test.inc include file. SilkTest and QA Partner will load
the .ino equivalent. Call TestFunction, which was defined in the include file
use "test.inc"
main ()
TestFunction () // call the function
Distribute test.t and test.ino. Users can open test.t and run it but do not have
access to the actual routine, which resides only in compiled form in test.ino.
How object files are Object files are always used if they are available. When you open a script file
used or an include file, the corresponding object file is loaded as well, if there is
one. If the object file is not older than the source file, the source file will not
be recompiled; the script is ready to run. If the source file is more recent, the
source file is recompiled before the script is run. If you then later save the
source, a new object file will be saved automatically.
If a file is loaded during compilation (that is, if a file is included in another
file that is being compiled), only the object file will be loaded, if it exists and
is newer than the corresponding source file.
Where object files are By default, an object file is read from and written to the same directory as its
located corresponding source file. But you can specify different directories for object
files.
Procedure To specify where object files should be written to and read from:
1 Select Options/Runtime.
The Runtime Options dialog is displayed.
2 Specify a directory in the Objfile Path field, as follows:
- Leave the field empty if you want to store object files in the same
directory as their corresponding source files
- Specify an absolute path if you want to store all object files in the
same directory
- Specify a relative path if you want object files to be stored in a
directory relative to the directory containing the source files
3 Click OK.
Object files will be saved in the location you specify here. In addition,
SilkTest will try to find object files in these locations. If it fails to find an
object file, it will look in the directory containing the source file.
Examples Specifying d:\obj in the Objfile Path field tells SilkTest to read
and write all object files in the d:\obj directory, regardless of where the source
files are located.
Specifying obj in the Objfile Path field tells SilkTest to read and write an
object file in the directory obj that is a subdirectory of the directory
containing the source file. In this scenario, each directory of source files will
have a different directory of object files. For example, if a source file is in
d:\src, its corresponding object file would be read from and written to
d:\src\obj.
2 Select Record/Testcase.
The Record Testcase Script dialog prompts you to name a script file to
contain the testcase.
Note You won’t be prompted for a script file if there is a script
defined at a higher level and inherited by the testcase you are now
recording. If there is, the testcase will be put in that script.
3 If prompted, select an existing script from the list or enter the name of a
new script in the File Name field, then click OK.
SilkTest displays the Record Testcase dialog.
4 Enter the name for the testcase and optionally select an application state
to be run before the recording starts.
5 Click the Start Recording pushbutton.
The Recording Status dialog displays. The dialog flashes the word
Recording for the duration of the session.
6 When finished recording the actions that comprise the testcase, click the
Done pushbutton in the Recording Status dialog.
The status dialog closes, and the Record Testcase dialog reappears.
7 Click the Paste to Editor pushbutton.
The Record Testcase dialog closes, the testcase is inserted into the script
file, and the script and testcase statements are added to the testplan on a
new line and are indented appropriately.
Note If the script file is inherited by the testcase you are
recording, only the testcase statement is pasted.
The fully qualified identifier for main windows and dialogs does not need to
include ancestors because the declarations begin with the keyword window.
Record/Class If you are using ActiveX, Visual Basic, or Java classes (controls) that are not
predefined, you can record the classes for use in your tests.
For more information about recording classes, see online Help.
Interpreting Results
Introduction You can run testcases individually, as a script, as a group of scripts (called a
suite), or from a testplan if you are running QA Organizer. Whenever you run
tests, SilkTest generates a results file, which indicates how many tests passed
and how many failed, describes why tests failed, and provides summary
information. You can invoke comparison tools from within the results file that
pinpoint exactly how the runtime results differ from your known baselines.
Testplan results files offer additional features, such as the ability to generate a
Pass/Fail report or compare different runs of the testplan.
What you will learn This chapter contains the following sections:
Topic Page
Preparing to run tests 141
Running tests 144
Results files 148
Using the results file to find errors 152
Fixing errors in a script 155
Managing results file information 156
If you do not plan to test Web applications, you must disable all browser
extensions on the host machine.
2 Enable support for the browsers you want to use, if they aren’t already
enabled. In the Primary Extension field for a browser, choose Enabled.
3 Select any other extensions that the browser needs for testing such as
ActiveX or fault trapping. We recommend that you do not turn on fault
trapping until you really need it.
4 Click OK to close the Extension Enabler dialog.
Enabling browser You enable support for browsers on the host machine using the Extensions
support on a host dialog. Be advised that there is overhead associated with having more than
machine one browser extension enabled, so do so only if you are actually testing more
than one browser in an automated session.
Running tests
You can run testcases individually, as a script, as a group of scripts (called a
suite), or from a testplan if you are running QA Organizer.
Preparing to run tests If you plan to test Web applications, make sure you enable the correct
browser extension(s), either during installation or using the Agent Options
dialog. If you do not plan to test Web applications, you must disable all
browser extensions using the Agent Options dialog. This is because the
recovery system works differently when testing Web applications than when
testing non-Web applications. For more information, see “The recovery
system for Web applications” in Chapter 3 of Testing Web Applications with
SilkTest.
After you have turned off multiple-tag support, 4Test discards all segments of
a multiple tag except the first one.
Stopping a running Procedure To stop a running testcase before it completes:
testcase
• If your test application is on a target machine other than the host
machine, select Run/Abort.
• If your test application is running on your host machine, press
Shift+Shift.
Grouping scripts in a After you have created a number of script files, you might want to collect
suite them into a test suite. A suite is a file that names any number of scripts.
Instead of running each script individually, you run the suite, which executes
in turn each of your scripts and all the testcases they contain. Suite files have
a .s extension.
If an argument is more than one word, enclose it with quotation marks. For
example, the following passes in three arguments:
"Bob H" "Emily M" "Craig J"
Examples The following testcase simply prints a list of all the passed
arguments:
testcase ProcessArgs ( )
LIST OF STRING lsArgs
lsArgs = GetArgs ( )
ListPrint (lsArgs)
You can also process the arguments individually. The following testcase
prints the second argument passed:
testcase ProcessSecondArg ( )
LIST OF STRING lsArgs
lsArgs = GetArgs ( )
Print (lsArgs[2])
The following testcase adds the first two arguments:
testcase AddArgs ()
LIST OF STRING lsArgs
lsArgs = GetArgs ( )
NUMBER nArgSum
Results files
When you run a testcase, script, suite, or testplan, SilkTest creates a results
file. The following figure shows a results file for a script (find.t). The menu
bar includes the Results menu, which allows you to manipulate the results file
and locate errors. The Results menu appears only when the active window
displays a results file
Definition of results A results file provides information about the execution of the testcase, script,
file suite, or testplan. By default, the results file has the same name as the
executed script, suite, or testplan, but with a .res extension (for example,
find.res).
To change the default name and directory of the results file, edit the Runtime
Options dialog. For more information, see “Options/Runtime...” on page 484.
Overall summary By default, the results file displays an overall summary at the top of the file,
including the name of the script, suite, or testplan; the machine the tests were
run on; the number of tests run; the number of errors and warnings; actual
errors; and timing information. The overall summary for find.t is shown in
the preceding figure.
Procedure To hide the overall summary, click on the summary and select
Results/Hide Summary. (You can change the default behavior by deselecting
the Show Overall Summary check box in the Runtime Options dialog. See
page 484.)
Individual test You can see a summary for an individual test in a results file.
summary
For a script or suite results file, the individual test summaries contain timing
information and errors or warnings. For a testplan results file, the individual
test summaries contain the same information as in the overall summary plus
the name of the testcase and script file.
Hierarchical format of The format for the rest of a testplan results file follows the hierarchy of test
testplan results file descriptions that were present in the testplan. Test statements in the testplan
that are preceded by a pound sign (# ) as well as comments (using the
comment statement) are also printed in the results file, in context with the
test descriptions.
Customizing colors You can modify the colors for these elements in the results file in the Editor
Colors dialog. For more information, see “Using color to indicate structure”
on page 56.
Using Find Error Procedure To find and expand the next error or warning message in the
results file, select Edit/Find Error.
Tip To skip warning messages and find error messages only,
deselect Find Error Stops at Warnings check box in the Runtime
Options dialog.
You can also use the Find, Find Next, and Go to Line commands on the Edit
menu to navigate through a results file.
Navigating to errors SilkTest provides several ways to move from the results file to the actual error
in the script in the script:
• Double-click in the margin next to an error line to go to the script file that
contains the 4Test statement that failed.
• Click on an error message and select Results/Goto Source.
What happens when you click on the box icon depends on the error message.
• If the error message relates to an application’s appearance, as in Bitmaps
have different sizes, SilkTest opens the bitmap tool for your
platform.
• If the error message relates to an application’s behavior, as in Verify
selected text failed, SilkTest opens the Difference Viewer.
The bitmap tool compares baseline and results bitmaps, whereas the
Difference Viewer compares actual and expected values for a given testcase.
Bitmap tool The following figure shows the bitmap tool. The baseline bitmap for the Font
dialog in the Text Editor is compared to the result bitmap, which shows the
Font dialog when the testcase was run. The Differences window shows the
area of difference between the two bitmaps, that is, the different selection for
Font Style.
The Differences
window shows the
differences between
the baseline and
result bitmap
The bitmap tool supports several comparison commands, which let you
closely inspect the differences between the baseline and results bitmaps.
These commands are described in “Comparing bitmaps” on page 414. In
particular, see “Zooming in on the differences” on page 416.
figure below; however, in reviewing the text, you can see, for example, that
SilkTest expects the Case Sensitive check box to be unchecked (FALSE),
while the actual value found during testcase execution was checked (TRUE).
When you have more than one screen of values or are using a black-and-
white monitor, use Results/Next Result Difference to find the next difference.
Use Update Expected Values, described next, to resolve the differences.
Debugging tools You might need to use the debugger to explore and fix errors in your script. In
the debugger, you can use the special commands available on the Breakpoint,
Debug, and View menus.
For more information, see Chapter 7, “Using the Debugger”.
Marking failed When a testplan results file shows testcase failures, you might choose to fix
testcases and then rerun them one at a time. You might also choose to rerun the failed
testcases at a slower pace, without debugging them, simply to watch their
execution more carefully.
4 Select the results set you want to merge into the currently opened results
set and click OK.
What happens QA Organizer combines the two results sets by merging the results set you
selected in the Merge Results dialog into the currently open results set. The
open results set is altered; no additional results set is created. The date and
time of the altered results set reflect the more recent test run.
For example, say you ran a section of the testplan consisting of 20 tests
yesterday and ran a different section of the testplan consisting of 10 tests
today. The merged results set would have today's date and would consist of
the results of 30 tests.
Note QA Organizer uses the test descriptions as well as the script,
testcase, and testdata statements to identify and locate the various
cases in the testplan and in the results set. When test results overlap
in the two results set that were merged, the more recent run is used.
If you change a test description between runs or modify the
statements, QA Organizer might be unable to find the test when you
try to merge results. QA Organizer will place these so-called
orphaned tests at the top of the results set.
3 Select a results set from the list box and click OK.
When the results set reappears, a colored arrow is positioned in the left
margin for every test that is different. A red arrow indicates that the
difference is due to the pass/fail state of the test changing. A magenta
arrow indicates that the difference is due to the addition or removal of the
test in the compared test run.
4 Select Results/Next Result Difference to search for the next difference or
select Results/Next Error Difference to search for the next difference that
is due to the change in a pass/fail state of a test.
Including results of You can mark manual tests as having passed or failed in the Update Manual
manual tests Tests dialog (see “Documenting manual tests in the testplan” on page 67).
The Pass/Fail report includes in its statistics the manual tests that you have
documented as having passed or failed.
By default, whenever you generate a report, it will include information on the
tests run for that results file, plus the current results of any manual tests
specified in the testplan. If the manual test results are subsequently updated,
the next time you generate the report, it will incorporate the latest manual
results, which might not be what you want. Instead, you might want the
report to use a snapshot of manual results, not the most recent manual results.
You do this by merging the results of manual tests into the results file.
You might need to resize the window so there is enough room to display
the chart well.
• If you have not generated a report, as described in “Generating a
testplan Pass/Fail report” on page 162, a default chart is displayed,
which allows you to modify chart parameters before actually
generating the chart.
2 Take one of the following actions:
Changing advanced Usually you can get the chart you want using the default and basic charting
properties properties. But if you want more customization, you can modify just about
any property in the chart. For example, you can:
• Change the text that appears for the title and footnote
• Change the font used for any text in the chart
• Specify the location for the title, legend, and footnote
• Change the colors used for the data
• Change the size and spacing of the bars in bar charts
• Add borders and shading to the background (backdrop) of any area
4 Make your changes. For information about the properties, click Help.
5 Click Apply to apply the changes without closing the dialog. Click OK to
close the dialog.
You will find out about many of the errors or inconsistencies in your scripts
when SilkTest automatically raises an exception in response to them. Some
problems, however, cause a script to work in unexpected ways, but do not
generate exceptions. You can use SilkTest debugger to solve these kinds of
problems.
Using the debugger, you can step through a script a line at a time and stop at
specified breakpoints, as well as examine local and global variables and enter
expressions to evaluate.
But the SilkTest debugger is more than just a tool for fixing scripts. You can
also use the debugger to help find problems in your application, using the
debugging facilities to step through the application slowly so you can
determine just where a problem occurs.
What you will learn This chapter contains the following sections:
Topic Page
Starting the debugger 172
Setting breakpoints 173
Executing a script in the debugger 175
Viewing the debugging transcript 177
Viewing the call stack 178
Viewing other script elements 178
Evaluating expressions 179
Running, resetting, and terminating execution 180
Tips on debugging 180
When you enter the debugger, you can execute the script under your control
(but note that you cannot edit a script when you are in the debugger).
The following figure shows the debugging environment.
Debugger menus In debugging mode the menu bar includes three additional menus—Debug,
Breakpoint, and View.
• Debug menu commands allow you to control the script’s flow.
• Breakpoint menu commands add or remove a breakpoint.
• View menu commands display different elements of the running script
(for example, local and global variables, the call stack, and breakpoints)
and evaluate expressions.
Exiting the debugger You can leave the debugger whenever execution is stopped.
Setting breakpoints
One useful way to debug a script is to pause it, observe its behavior and check
its state, then restart it. This is useful when you are not sure what lines of
code are causing a problem.
The debugger lets you stop execution on any line by setting breakpoints. A
breakpoint is a line in the script where execution stops, so that you can check
the script’s status. During debugging, you can set breakpoints on any
executable line where you want to check the call stack (described in
“Viewing the call stack” on page 178), examine the values in one or more
variables, or just see what a script has done so far. You cannot set breakpoints
on blank lines or comment lines.
Breakpoints
Seeing a list of Procedure To view a list of all the breakpoints in a script, select View/
breakpoints Breakpoints.
The list shows the name of the file each breakpoint is in, and the number
of the line it is set on or the name of the function if the breakpoint was set
on the first line of the function.
Deleting breakpoints You can delete breakpoints in any of three ways.
To delete Then
All breakpoints 1 Select Breakpoint/Delete All.
2 Click Yes to confirm the deletions.
An individual breakpoint • Place the cursor on the line where the breakpoint
is set and select Breakpoint/Toggle.
or
• Double-click in the left margin of the line
One or more breakpoints 1 Select Breakpoint/Delete to open the Delete
Breakpoint dialog.
2 Select one or more breakpoints from the list box
and click OK.
Line
stopped at
Finish Function
Step Into
Step Over
Step Into command You use the Step Into command to execute the next line in a script. If the line
contains a function call, control passes into the function. You can use Step
Into to step through the function one line at a time, executing each line in turn
as you go.
Step Over command Step Over also executes the next line in a script. However, if the line is a
function call, SilkTest executes the function without stopping. You use Step
Over to speed up debugging if you know a particular function is bug-free.
Finish Function You use Finish Function to execute the script until the current function
command returns. The focus will be at the line where the function returns. Try using
Finish Function in combination with Step Into to step into a function and then
run it.
Procedure To enable viewing the trace listing, check the Print Agent Calls
option (and if you want, the Print Tags with Agent Calls option) in the
Runtime Options dialog before running a script.
Procedure To view a list of all the local variables that are in scope
(accessible) from the current line, including their values, select View Local
Variables.
Procedure To change the value of an active variable, select the variable and
type its new value in the Set Value field.
When you resume execution, the new values will be used.
Evaluating expressions
When a script reaches a breakpoint, you can evaluate expressions.
Properties and methods for a class are valid in expressions, as long as the
declaration for the class they belong to is included in one of the modules used
by the script being debugged.
If an expression evaluates to a complex value, like an array, SilkTest may
display its result in collapsed form. Use View/Expand Data or View/Collapse
Data (or double-click on the plus icon) to manipulate the display.
Tips on debugging
This section describes some common bugs and how to find them. It also
includes some ideas for designing and testing scripts with error detection in
mind.
Chapter Page
Chapter 8, “Generalizing a Testcase” 185
Chapter 9, “Handling Exceptions” 209
Chapter 10, “Adding Data to a Testplan” 221
Chapter 11, “Categorizing and Marking Testplans” 235
Chapter 12, “Working With Large Testplans” 253
Topic Page
Using application states 185
Using data-driven testcases 189
Testing applications that have been made Year 2000 compliant 195
For more information on any of the 4Test statements and reserved words
discussed in this chapter, see the online Help.
4 Type the name of your new application state in the Application State
Name field.
5 Select an application state from the Based On drop-down list.
6 Press the Start Recording pushbutton.
SilkTest closes the Record Application State dialog and displays the
Record Status window. The Status field flashes Recording.
7 Drive your application to the state you want to record. At any point, you
can record a verification by pressing Ctrl+Alt.
8 When you have finished recording an application state, press Done on
the Record Status window. SilkTest redisplays the Record Application
State dialog. The Application State Code field contains all the 4Test code
you’ve just recorded. You can take the following actions:
If Then
All the information in the Click Paste to Editor. SilkTest closes the
window is complete and Record Application State dialog and places
what you expect. the new application state in your file.
You want to alter the code. Edit the Application State Code field.
The application state name Edit the name in the Application State Name
is not what you want. field.
The application state on Delete the code in the Application State Code
which this application state field, select a new application state from the
is based is not the one you drop-down list, and click Resume Recording
want. to rerecord the application state.
The application state Click Resume Recording. SilkTest opens the
routine is not finished. Record Status window.
Example Here is a sample application state that performs the setup for all forward
case-sensitive searches in the Find dialog:
appstate Setup () basedon DefaultBaseState
TextEditor.File.New.Pick ()
DocumentWindow.Document.TypeKeys ("Test Case<Home>")
TextEditor.Search.Find.Pick ()
Find.CaseSensitive.Check ()
Find.Direction.Select ("Down")
2 Select the application state you want to run and click Run.
3 If there are compilation errors, SilkTest displays an error window. Fix the
errors and rerun the application state.
Note that the record variable name (Data) is preceded by the record type
name (SEARCHINFO).
Finally, we revised the testcase to use the defined data:
testcase FindTest (SEARCHINFO Data)
TextEditor.File.New.Pick ()
DocumentWindow.Document.TypeKeys (Data.sText+Data.sPos)
TextEditor.Search.Find.Pick ()
Find.FindWhat.SetText (Data.sPattern)
Find.CaseSensitive.SetState (Data.bCase)
Find.Direction.Select (Data.sDirection)
Find.FindNext.Click ()
Find.Cancel.Click ()
DocumentWindow.Document.VerifySelText ({Data.sExpected})
TextEditor.File.Close.Pick ()
MessageBox.No.Click ()
Note that the command that checks the Case Sensitive check box has been
changed to use the SetState method instead of the recorded Check method, so
that it can receive the BOOLEAN parameter bCase.
Note also that the following statement in the rewritten testcase uses the {}
operator (list constructor operator).
DocumentWindow.Document.VerifySelText({Data.sExpected})
• If you are using QA Organizer, you do not pass data to a testcase from
the script’s main function. Instead, you embed the data in the testplan and
QA Organizer passes the data when you run the testplan. Therefore,
QA Organizer users should skip this section and instead follow the
instructions in “Adding Data to a Testplan” on page 221.
Using a main function Although most of the script files you create contain only testcases, in some
in the script instances, you need to add a function named main to your script. You can use
the main function to pass data to testcases as well as control the order in
which the testcases in the script are executed.
When you run a script file using Run/Run:
• If the script file contains a main function, the main function is executed,
then execution stops. Only testcases and functions called by main will be
executed, in the order in which they are specified in main.
• If the script does not contain a main function, the testcases are executed
from top to bottom.
The following template shows the structure of a script that contains a main
function that passes data to a data-driven testcase:
main ()
Example of using main Using this structure, the following example shows how to create a script that
defines data records and then calls the sample testcase once for each record in
the list:
type SEARCHINFO is record
STRING sText // Text to type in document window
STRING sPos // Starting position of search
STRING sPattern // String to look for
BOOLEAN bCase // Case-sensitive or not
STRING sDirection // Direction of search
STRING sExpected // The expected match
main ()
SEARCHINFO Data
list of SEARCHINFO lsData = {...}
{"Test Case", "<END>", "C", TRUE, "Up", "C"}
{"Test Case", "<END>", "Ca", TRUE, "Up", "Ca"}
// additional data records can be added here
for each Data in lsData
FindTest (Data)
When the script is run When you select Run/Run, the main function is called and the FindTest
testcase will be executed once for every instance of Data in lsData (the list of
SEARCHINFO records). In the script shown above, the testcase will be run
twice. Here is the results file that is produced:
Script findtest.t - Passed
Passed: 2 tests (100%)
Failed: 0 tests (0%)
Totals: 2 tests, 0 errors, 0 warnings
Testcase FindTest ({"Test Case", "<END>", "C", TRUE, "Up", "C"}) - Passed
Testcase FindTest ({"Test Case", "<END>", "Ca", TRUE, "Up", "Ca"}) - Passed
Notice that with data-driven testcases, SilkTest records in the results file the
parameters that are passed in.
Using FileReadInfo to In this sample data-driven testcase, the testcase data is stored in a list within
read external data the script itself. It is also possible to store the data externally and read records
into a list using the FileReadValue function. For more information on this
function, see online Help.
MessageBox.Message.VerifyValue (Data.sMessage)
MessageBox.OK.Click ()
Find.Cancel.Click ()
TextEditor.File.Close.Pick ()
MessageBox.No.Click ()
The VerifyValue method call in this testcase verifies that the message box
contains the correct string. For example, the message should be “Cannot find
Ca” if the user enters Ca into the Find dialog and the document editing area
does not contain this string.
Using do...except to The VerifyValue method, like all 4Test verification methods, raises an
handle an exception exception if the actual value does not match the expected (baseline) value.
When this happens, SilkTest halts the execution of the testcase and transfers
control to the recovery system. The recovery system then returns the
application to the base state. (For more on the recovery system, see Chapter
15, “Understanding the Recovery System”.)
However, suppose you don’t want SilkTest to transfer control to the recovery
system, but instead want to “trap” the exception and handle it yourself. For
example, you might want to log the error and continue executing the testcase.
To do this, you can use the 4Test do...except statement and related
statements, which allow you to handle the exception yourself.
For more information, see Chapter 9, “Handling Exceptions”.
Field Description
Input Date Format Specifies how the dates are specified in your scripts.
You can select one of the predefined formats in the list
or define your own. See “Specifying date formats”
below.
Input Year Threshold Specifies whether a two-digit date is transformed into
a 19xx date or a 20xx date.
Dates whose years are less than or equal to the
threshold are transformed into 20xx dates; dates whose
years are greater than the threshold are transformed
into 19xx dates.
Specify 0 to ignore this field.
Minimum Year Don’t transform the date if the date after
transformation would be less than the year specified
here.
Specify a four-digit date; specify 0 to ignore this field.
Field Description
Maximum Year Don’t transform the date if the date after
transformation would be greater than the year
specified here.
Specify a four-digit date; specify 0 to ignore this field.
Output Date Format Specifies how the Year 2000–complaint application
expects the dates. You can select one of the predefined
formats in the list or define your own. See “Specifying
date formats” below.
Output Year Aging Number of years to add to the transformed date. You
can use aging so that your tests can test different dates
without your having to modify the scripts.
Specify 0 to ignore this field.
Specifying date Date formats consist of date masks, which specify how the date is
formats represented, and delimiters.
The date masks Date import and output formats in transformation rules
support the following masks (which are a subset of the masks supported by
the FormatDateTime function):
Mask Description
yy Two-digit year
yyyy Four-digit year
m Month with one or two digits (1–12)
mm Month with exactly two digits (01–12)
mmm Abbreviated month name (Jan, Feb, Mar, Apr, May,
Jun, Jul, Aug, Sep, Oct, Nov, or Dec); case insensitive
mmmm Full month name (January, February, and so on); case
insensitive
d Day of month with one or two digits (1–31)
dd Day of month with exactly two digits (01–31)
ddd Abbreviated day of the week (Sun, Mon, Tue, Wed,
Thu, Fri, or Sat); case insensitive
dddd Full day of the week (Monday, Tuesday, and so on);
case insensitive
Note Though it is unlikely that you will need any masks other than
those listed above, date output formats can use any of the masks
supported by FormatDateTime.
Valid delimiters In addition to the date masks, date formats can include the
following delimiters:
• Space (matches one or more spaces)
• / - : , . (forward slash, hyphen, colon, comma, or period)
Any other character in the format generates an error.
Examples of date Here are some examples of date formats you can use in the transformation
formats rules:
Getting dates from Similarly, SilkTest will transform dates that come back from the application
the application through the following method:
• GetText for a ComboBox, DynamicText, StaticText, or TextField
Verifying properties Each time VerifyProperties is called, SilkTest traverses the entire property
tree (the WinPropTree argument to the function) and converts all the expected
values using the defined rules. The transformed dates are used as the true
expected values in the verifications.
How the dates are When the methods listed above are called, SilkTest does the following:
transformed
1 It compares the string argument to all the input formats that have been
defined in any of the rules.
2 If the argument matches an input format, the string is transformed to the
output format, subject to the values specified in the rule for Input Year
Threshold, Minimum Year, Maximum Year, and Output Year Aging.
If a string doesn’t match an input format, it is not transformed.
SilkTest uses the same criterion when transforming dates coming back
from the application as when transforming dates sent to the application:
If the date coming back through GetText matches an input format, it is
transformed.
3 It sets or gets the text in the application using the transformed date and
notifies you in the results file that it transformed a date.
Note SilkTest does all this transformation in memory at runtime.
Your scripts are not changed.
Finding all the dates in If you want SilkTest to find all dates in your tests but not transform them,
your tests create a rule where the output format is the same as the input format. SilkTest
will log a message whenever it encounters such a date, but won’t transform it.
That rule says to transform dates in the form m/d/yy (such as 9/18/97) to the
form mm/dd/yyyy (such as 09/18/1997). Dates whose years are less than or
equal to 30 are transformed to 20xx.
Here are some tests that illustrate the date transformation (it uses the Text
Field window in the test application shipped with SilkTest):
Example 1: Date testcase GetandSetText1 ()
conversion
STRING sReturnedText
TextFieldWindow.Invoke ()
Print ("Entering the value '9/18/97'")
TextFieldWindow.TheTextField.SetText ("9/18/97")
Print ("Getting ...")
sReturnedText = TextFieldWindow.TheTextField.GetText ()
Print (sReturnedText)
TextFieldWindow.Close ()
// Result:
// Entering the value '9/18/97'
// *** Y2K Notification: date value '9/18/97' has been
changed to '09/18/1997' in
TextFieldWindow.TheTextField.SetText
// Getting ...
// 09/18/1997
// Result:
// Entering the value '9/18/20'
// *** Y2K Notification: date value '9/18/20' has been
changed to '09/18/2020' in
TextFieldWindow.TheTextField.SetText
// Getting ...
// 09/18/2020
Note the following about this test:
• The argument to SetText is “9/18/20”, also of the input form m/d/yy, so it
matches the rule.
• In this case, the year (20) is below the threshold in the rule (30), so the
date is converted to 2020, instead of 1920.
Example 3: No date testcase GetandSetText3 ()
conversion
STRING sReturnedText
TextFieldWindow.Invoke ()
Print ("Entering the value '9-18-97'")
TextFieldWindow.TheTextField.SetText ("9-18-97")
Print ("Getting ...")
sReturnedText = TextFieldWindow.TheTextField.GetText ()
Print (sReturnedText)
TextFieldWindow.Close ()
// Result:
// Entering the value '9-18-97'
// Getting ...
// 9-18-97
You can also use the Y2KRules declaration to apply rules to individual child
controls. The Y2KRules construct applies the rule only to the control in
which it is declared. If you want to apply a rule to a control and all of its
children, use the Y2KRulesInherited construct.
Y2KRulesInherited Y2KRulesInherited allows you to specify transformation rules selectively for
particular objects or classes. You must declare Y2KRulesInherited rules
within the declarations of specific objects or classes. The rule you specify
will apply to the object and all of its children. Rules set for specific child
controls override any rules they might inherit.
For example, if you want to specify the same Y2K transformation rule for all
text fields on a dialog, add the Y2KRulesInherited rule in the declaration for
the dialog. If you later want one of the text fields to apply a different
transformation rule, add a second Y2KRulesInherited rule in the declaration
for that text field. This rule will override the inherited one.
The syntax for setting Y2KRulesInherited is:
setting Y2KRulesInherited = "rule" | NULL
Default behavior When you first install SilkTest, your winclass.inc file contains the following
declaration within the AnyWin class:
setting Y2KRules = "UseContainer"
Later, you realize that the dates in one of the text fields, called
TheSingleLine, must be transformed to the format mm-dd-yyy for a
particular report. To meet this requirement, add the following statement
inside the declaration for TheSingleLine:
setting Y2KRules="mm/dd/yy|0|0|0|mm-dd-yyyy|0"
This new rule will override the inherited rule for TheSingleLineTextField text
field. Note that we use Y2KRules because we are applying this rule to a
single control. If TheSingleLineTextField had child controls that needed to
inherit this rule, we would have used the Y2KRulesInherited construct
instead.
Here’s how these declarations appear in the test frame file:
If you don’t want the warnings logged in the results file, change the statement
to:
setting Y2KWarnings = FALSE
either for the AnyWin class in winclass.inc or for any other specific
declaration.
This chapter describes techniques you can use to handle exceptions (errors)
that are generated when you run your scripts.
What you will learn This chapter contains the following sections:
Topic Page
Default error handling 209
Using do...except 210
Performing more than one verification in a testcase 211
Adding to the default error handling 212
Trapping the exception number 214
Writing an error-handling function 214
Programmatically logging an error 216
Defining your own exceptions 216
Enabling fault trapping 217
These runtime errors are called exceptions. They indicate that something did
not go as expected in a script. They can be generated automatically by
SilkTest, such as when a verification fails, when there is a division by zero in
a script, or when an invalid function is called.
You can also generate exceptions explicitly in a script.
However, suppose you don’t want SilkTest to transfer control to the recovery
system when an exception is generated, but instead want to trap the exception
and handle it yourself. To do this, you use the 4Test do...except statement.
Using do...except
Using do...except you can handle exceptions locally, instead of passing
control to SilkTest’s built-in error handler (which is part of the recovery
system). The statement has the following syntax:
do
statements
except
statements
If an exception is raised in the do clause of the statement, control is
immediately passed to the except clause, instead of to the recovery system. If
no exception is raised in the do clause of the statement, control is passed to
the line after the except clause—the statements in the except clause are not
executed.
...
do
control passes exception raised here
into except clause except
statements
...
...
do
no exception raised here
control passes except
to first statement statements
after except clause
...
That testcase contains three verification statements. However, if, for example,
the first verification (VerifyCaption) fails, an exception is raised and the
testcase terminates. The second and third verifications never happen.
To verify more than one thing in a testcase, you can trap all but the last one in
a do...except statement, such as:
testcase MultiVerify2 ()
TextEditor.Search.Find.Pick ()
do
Find.VerifyCaption ("Find")
except
ExceptLog ()
do
Find.VerifyFocus (Find.FindWhat)
except
ExceptLog ()
Find.VerifyEnabled (TRUE)
Find.Cancel.Click ()
Here, all the verifications will happen each time the testcase is run. If either
of the first two fails, the 4Test function ExceptLog is called. That function
logs the error information in the results file, then continues execution of the
script.
Find.CaseSensitive.SetState (Data.bCase)
Find.Direction.Select (Data.sDirection)
Find.FindNext.Click ()
do
MessageBox.Message.VerifyValue (Data.sMessage)
except
sMatch = DocumentWindow.Document.GetSelText ()
if (sMatch != "")
Print ("Found " + sMatch + " not " + Data.sPattern)
reraise
MessageBox.OK.Click ()
Find.Cancel.Click ()
TextEditor.File.Close.Pick ()
MessageBox.No.Click ()
As the example shows, following the do keyword is the verification
statement, and following the except keyword are the 4Test statements that
handle the exception. The exception-handling statements in this example do
the following:
1 Call the GetSelText method to determine what text, if any, is currently
selected in the document.
2 If the return value from the GetSelText method is not an empty string, it
means that the application found a false match. If this is the case, then
print the false match and the search string to the results file.
3 Reraise the exception to transfer control to the recovery system and
terminate the testcase.
reraise The reraise statement raises the most recent exception again and passes
control to the next exception handler (in the preceding testcase, it passes
control to the built-in recovery system). We used reraise in the preceding
testcase because if the exception-handling code does not explicitly reraise the
exception, flow of control passes to the next statement in the testcase.
So this example performs a test after an exception is raised, prints a statement
to the results file if text was selected, then calls the recovery system, which
terminates the testcase, logs the error, and restores the test application to its
default base state.
ErrorHandler ()
CALL Call
LIST OF CALL lCall
lCall = ExceptCalls ()
The second line is the result of printing the information from ExceptData.
The rest of the lines show the processing of the information from
ExceptCalls.
And note that this testcase passes, because the error was handled locally and
not reraised.
Function Action
LogError (string) Writes string to the results file as an error (displays
in red or italics, depending on platform) and
increments the error counter.
This function is called automatically if you don’t
handle the error yourself.
LogWarning (string) Same as LogError, except it logs a warning, not an
error.
ExceptLog ( ) Calls LogError with the data from the most recent
exception.
Windows 95 and Fault trapping is available on Windows 95 and Windows NT. This is
Windows NT especially important for Windows NT users, because unexpected program
terminations are likely to cause the entire system to hang.
Tip We recommend that you keep fault trapping turned off until you
need to turn it on.
Setting on the host and target machines Fault trapping for the specific
application under test is enabled through the Extensions dialog and the
Extension Enabler dialog. Once you enable fault trapping, you can also
selectively turn on/off a number of fault trapping options.
Topic Page
Linking a testplan to a data-driven testcase 222
Specifying data that is unique to a single test description 223
Specifying data that is shared by multiple tests 226
Converting the data in a results file into a plan 233
The following testplan is associated with the FindTest testcase (the testcase
statement is highlighted for emphasis). The statement occurs at the Find
dialog group description level, so that each of the eight test descriptions in the
group can call the testcase, passing it a unique set of data:
This figure shows the testdata statement, which is used to specify the data
that a test description passes to a data-driven testcase. The testdata statement
is described in detail in the next section.
To provide context, the multi-line list box at the top of the dialog displays
the line in the testplan that the cursor was on when the dialog was
invoked, indicated by the black arrow icon.
If the testcase and script associated with the current test description are
inherited from a higher level in the testplan, they are shown in blue;
otherwise, they are shown in black.
3 Enter the data in the Test Data field, separating each data element with a
comma. For example:
b If the testcase expects a record, use the same syntax as above, but
open and close the list of record fields with curly braces:
testdata: {data [,data]}
where data is any valid 4Test expression.
Be sure to follow the testdata keyword with a colon. If you enter the
keyword correctly, the statement appears in dark red, the default color.
Otherwise, the statement appears in either blue or black, indicating the
compiler is interpreting the line as a description.
For example:
• The $Case symbol is assigned the value TRUE at the Case sensitive group
description level and the value FALSE at the Case insensitive group
description level.
• The $Direction symbol is assigned the value Down at the Forward group
description level, and the value Up at the Backward group description
level.
• The $Position symbol is assigned the value <HOME> at the Forward
group description level, and the value <END> at the Backward group
description level.
Because the data that is common is factored out and defined at a higher level,
it is easy to see exactly what is unique to each test.
The same symbol can If a symbol is defined at a level in the plan where it can be shared by a group
have several local of tests, each test can assign its own local value to the symbol, overriding
values whatever value it had at the higher level. You can tell whether a symbol is
locally assigned by using the Testplan Detail dialog:
• Locally assigned symbols appear in black. Symbols that inherit their
values appear in blue.
Specifying symbols as Procedure To specify symbols as arguments when entering a testcase
arguments in a statement in the Testplan Detail dialog:
testcase statement
1 Place the insertion cursor in the testplan at the location where the testcase
statement is to be inserted. For example:
2 Select Testplan/Detail.
4 Click OK.
2 Select Testplan/Detail.
4 Enter the symbol definition in the text field to the left of the Add
pushbutton. For example:
Note that you do not need to enter the $ character; QA Organizer will
take care of this for you when it inserts the definitions into the testplan.
5 Click Add.
The symbol is added to the list box above the Add text field, for example:
6 Define additional symbols in the same manner, and then click OK when
finished.
The Testplan Detail dialog closes, and the symbol definition(s), including
the $ character, are entered in the plan. For example:
How inheritance is The Testplan Detail dialog uses color and font to indicate whether a symbol’s
indicated value is inherited or assigned locally. Locally assigned symbols appear in
black, and symbols that inherit their value appear in blue.
Assigning values You can also define symbols and assign them values by typing them into the
manually testplan, using this syntax:
$symbolname = symbolvalue
where symbolname is any valid 4Test identifier name, prefixed with the
$ character and symbolvalue is any string, list, array, or the ? character (which
indicates an undefined value).
For example, the following statement defines a symbol named Color and
assigns it the STRING value “Red”:
$Color = "Red"
Because the results file was for a script, not a testplan, there are no group
or testcase descriptions in the testplan, only test data. Therefore,
QA Organizer uses the # symbol so that when this testplan is run, the
testdata statement will double as description. The # symbol can be used
with any QA Organizer statement so that the statement will double as
description. For more information on the syntax of QA Organizer
statements, see Appendix A, “QA Organizer Keywords”.
Testplans
What you will learn This chapter contains the following sections:
Topic Page
Overview 235
Defining and modifying attributes and values 237
Assigning attributes and values to a testplan 240
Marking a testplan 243
Creating and modifying testplan queries 246
Overview
You might want to work with only selected testcases at any one time, for
example, to run only those tests that exercise a particular area of the
application or to report on only the tests that were assigned to a particular QA
engineer.
Definition of mark To work with selected tests rather than the entire testplan, you denote or mark
those tests in the testplan. Marks are temporary; you can remove them at any
time, and they last only as long as the current work session.
Ways of marking a By choice You can mark an individual test description, a group description,
testplan or the entire plan simply by selecting it and then choosing the appropriate
marking command on the Testplan menu.
By test failure After running a testplan, the generated results file might
indicate test failures. You can mark these failures in the plan by selecting
Results/Mark Failures in Plan. You then might fix the errors and rerun the
failed tests. (For information on results files, see “Results files” on page 148.)
Definition of attributes Attributes are site-specific characteristics that you can define for your
testplan and assign to test descriptions and group descriptions. You can use
test attributes to build queries.
How attributes are By assigning attributes to parts of the testplan, you can:
used
• Group tests in the plan to distinguish them from the whole testplan
• Report on the testplan based on a given attribute value
• Run parts of the testplan that have a given attribute value
For example, you might define an attribute called Engineer that represents the
set of QA engineers that are testing an application through a given testplan.
You might then define values for Engineer like Bob, Emily, Craig, and Zoe,
the individual engineers who are testing this plan. You can then assign the
values of Engineer to the tests in the testplan. Certain tests are assigned the
value of Emily, others the value of Craig, and so on. You can then run a query
to mark the tests that have a given value for the Engineer attribute. Finally,
you can run just these marked tests.
Attributes are also used to generate reports. For example, to generate a report
on the number of passed and failed tests for Engineer Craig, simply select
this value from the Pass/Fail Report dialog. You do not need to mark the tests
or build a query in this case.
Note This chapter does not describe how to run parts of a testplan,
nor does it describe how to generate a report using attributes. These
are described in Chapter 6, “Running Tests and Interpreting
Results”.
Attribute Description
Developer Specifies the group of QA engineers who developed
the testcases called by the testplan.
Component Specifies the application modules to be tested in this
testplan.
Category Specifies the kinds of tests used in your QA
Department, for example, Smoke Test.
What you can do You can define up to 254 attributes. You can also rename the predefined
attributes.
Naming rules Here are the rules for naming attributes:
• Attribute names can be up to 11 characters long
• Attribute and value names are not case sensitive
Attributes reside in the Attributes and values as well as queries are stored by default in testplan.ini
initialization file (the initialization file is specified in the Data File for Attributes and Queries
field in the General Options dialog). The testplan.ini file is located in the
SilkTest installation directory
Make sure that all the QA engineers in your group use the same initialization
file.
How to define You can define up to 254 attributes.
attributes and their
values Procedure To define an attribute and its values:
1 Select Testplan/Define Attributes.
2 Click New.
The New Attribute dialog appears.
Type Description
Normal You specify values when you define the attribute. Users
of the attribute in a testplan pick one value from the list.
Edit You don’t specify values when you define the attribute.
Users type their own values when they use the attribute
in a testplan.
Set Like Normal, except that users can pick more than one
value.
a Type a value in the text field and click Add. Repeat this step for each
value.
The values display in the Values list in alphabetical order.
How to assign an Procedure To assign an attribute from the Testplan Detail dialog:
attribute via the
Testplan Detail dialog 1 Place the cursor in the testplan where you would like the assignment
statement to appear, either after the test description or the group
description.
2 Select Testplan/Detail to invoke the Testplan Detail dialog.
3 Click on the Test Attributes tab.
The Test Attributes tab is displayed.
The arrow in the list box at the top of the dialog identifies the test
description at the cursor position in the testplan, for example, “Case
sensitive” in the preceding figure. The attribute will be added to this test
description. The Test Attributes tab lists all your current attributes at this
level of the testplan.
4 Do one of the following:
• If the attribute is of type Normal, select a value from the list.
• If the attribute is of type Set, select one or more values from the list.
• If the attribute is of type Edit, type a value.
5 Click OK.
The dialog closes and the assignment statement(s) are placed in the
testplan.
Adding or removing Tests can be assigned more than one value at a time for attributes whose type
members of a Set is Set.
attribute
For example, you might have a Set variable called RunWhen with three
values: UI, regression, and smoke. You can assign any combination of these
three values to a test or group of tests. Separate each value with a semicolon.
You can use the + or – operator to add or subtract elements to what were
previously assigned.
adds the value smoke to the previously assigned values, so Test 2 has the
values UI, regression, and smoke.
RunWhen: - regression
Test 2
testcase: t2
In this example, Test 1 has the values UI and regression. The statement
RunWhen: - regression
removes the value regression from the previously assigned values, so Test2
has the value UI.
Note the following about using + and –:
• You must follow the + or – with a space.
• You can add or remove any number of elements with one statement.
Separate each element with a semicolon.
• You can specify + element(s) even if no assignments had previously been
made; the result is that the element(s) are now assigned.
• You can specify – element(s) even if no assignments had previously been
made; the result is that the set’s complement is assigned. Using the
previous example, specifying
RunWhen: - regression
when no RunWhen assignment had previously been made results in the
values UI and smoke being assigned.
Marking a testplan
Marks are temporary denotations that allow you to work with selected tests in
a testplan. Marks can be removed at any time, and they last only as long as
the current work session.
Black stripe denotes a You can recognize a marked testcase by the black stripe in the margin. The
mark following figure shows two marked tests in the outline4 testplan.
Marking commands The Testplan menu contains seven marking commands — Mark, Mark All,
are on the Testplan Unmark, Unmark All, Mark by Query, Mark by Named Query, and Find Next
and Results menus Mark. In addition, the Results menu contains the Mark Failures in Plan
command. (The Results menu appears only when a results file is in the active
window. For more information on results files, see “Results files” on
page 148.)
The following table describes each of the commands.
Marking commands Four of the commands are also available on the tool bar:
on the tool bar
How to mark one or Procedure To mark a single test, place the cursor on the test description and
more tests select Testplan/Mark.
The test description and all the statements are marked.
Procedure To mark a group of related tests, place the cursor on the group
description and select Testplan/Mark.
The group description, its associated statements, and all test descriptions
and statements subordinate to the group description are marked.
Procedure To mark two or more adjacent tests and their subordinate tests,
select the test description of the adjacent tests and select Testplan/Mark.
QA Organizer marks the test descriptions and statements of each selected
test and any subordinate tests.
How the marking When you apply a mark using the Mark command, the new mark is added to
commands interact existing marks.
When you mark tests via the query marking commands, QA Organizer by
default clears all existing marks before running the query. Mark by Named
Query supports sophisticated query combinations, and it would not make
sense to retain previous marks. However, Mark by Query, which allows one-
time-only queries, lets you override the default behavior and retain existing
marks. To retain existing marks, deselect the Unmark All Before Query check
box in the Mark by Query dialog.
Printing marked tests Procedure To print the testplan showing only marked tests, select File/Print
and make sure the Print Marked Only check box is selected in the Print
dialog, as well as any other options you want. Select OK to print the listing.
Example: You want to mark all tests in the testplan that meet all three of
these criteria: They are defined in the script file find.t, the value of the
Developer attribute is Dave, and the value of the symbol $TestLevel is 2.
Named versus Queries can be named or unnamed. However, unnamed queries can be run
unnamed queries only once. If you name the query, you can have QA Organizer run it in the
same or subsequent work sessions without having to rebuild the query or
manually remark the tests that you’re interested in rerunning or reporting on.
Two query marking Testplan/Mark by Query or Testplan/Mark by Named Query both create
commands queries; however, Mark by Named Query provides extra features, like the
ability to combine queries or to create a query without running it
immediately. If the query-creation function and the query-running function
are distinct in your company, then use Mark by Named Query. If you intend
to run a query only once, or run a query while keeping existing marks, then
use Mark by Query.
The following table highlights the differences between the two commands.
Queries reside in the Named queries are stored by default in testplan.ini (the initialization file is
initialization file specified in the Data File for Attributes and Queries field in the General
Options dialog). The testplan.ini file is in the SilkTest installation directory.
Make sure that all the QA engineers in your group use the same initialization
file.
How to create a new You can create a new query through either Testplan/Mark by Query or
query Testplan/Mark by Named Query.
• Specify a name in the Query Name (Optional) • Click the New pushbutton. The New Testplan
field if you want to save the query. Query dialog appears. It has the same tabs as the
• Deselect the Unmark All Before Query check Mark by Query dialog (shown at left).
box if you want to retain existing marks. By • Specify a name for the query in the Query Name
default, the marks are cleared. field.
Note: To build a query that marks only manual tests, enter the
keyword manual in the Testcase field.
Note: The wildcard characters * (asterisk) and ? (question mark) are
supported for partial matches: * is a placeholder for 0 or more
characters, and ? is a placeholder for 1 character.
Example 1: If you type find_5 (* in the Testcase field, the query
would search all the testcase statements in the plan and mark those
test descriptions that matched as well as all subordinate descriptions
to which the matching testcase statement applied; that is, those
where the find_5 testcase passed in data.
Example 2: If you type find.t in the Script field, the query would
search all script statements in the plan and mark those test
descriptions that matched exactly as well as all subordinate
descriptions to which the matching script statement applied; that is,
those in which you had specified find.t exactly. It would not match
any script statements in which you had specified a full path.
- To include existing attributes and values in the query, use the Test
Attributes tab (shown in the following figure).
If you need to modify the symbol in the query, select it from the list
box and click Edit. QA Organizer places it in the text field and
changes the Add pushbutton to Replace. Edit the symbol or value
and click Replace.
If you need to exclude the symbol from the query, select it from the
list box and click Remove. QA Organizer deletes it from the list box.
4 Once you are satisfied with the query, do one of the following, depending
on which command you chose to create the query.
Creating a new query You can combine two or more existing queries into a new query through the
by combining existing Mark by Named Query dialog. The new query can represent the union of the
queries constituent queries (logical OR) or the intersection of the constituent queries
(logical AND).
Query1 Query2
Developer: Emily Developer: Craig
Component: Searching TestLevel: 2
The new query created from the union of Query1 and Query2 will first mark
those tests that match all the criteria in Query1 (Developer is Emily and
Component is Searching) and then mark those tests that match all the criteria
in Query2 (Developer is Craig and TestLevel is 2).
3 Specify a name for the new query in the Query Name field.
4 Select two or more queries to combine from the Queries to Combine list
box.
5 Select a radio button representing the combination method to use: either
Union of Queries or Intersection of Queries.
6 Click OK to save the new query.
The Combine Testplan Queries dialog closes and the Mark by Named
Query dialog is once again visible.
How to edit queries You can modify an existing query through the Mark by Named Query dialog.
Testplans
For large or complicated applications, the testplan can itself become quite
large. This raises three issues:
• How to keep track of where you are in the plan and what is in scope at
that level
• How to determine which portions of the plan have been implemented
• How to allow several staff members to work on the plan at the same time
To determine context and scope, you can use the Testplan Detail dialog. To
determine which portions of the plan have been implemented, you can
produce a Completion report. To allow multiple users to work on the same
plan, you can structure your testplan as a master plan with one or more
subplans. These topics are all discussed in this chapter.
What you will learn This chapter contains the following sections:
Topic Page
Navigating through a large testplan 254
Measuring testplan completion 256
Dividing a testplan into a master plan and subplans 257
Editing a master plan in a multi-user environment 260
If this were all you could see on your display, you would have difficulty
understanding the current context of symbols and statements.
3 Once you find the level at which a symbol, attribute, or statement was
defined, you can change the value at that level, causing the inherited
value at the lower levels to change also.
3 In the Report Scope group box, indicate whether the report is for the
entire plan or only for those tests that are marked
4 To subtotal the report by a given attribute, select an attribute from the
Subtotal by Attribute field.
For example, suppose you have an attribute named Developer and have
specified the values that Developer can take. If each test has been
assigned a Developer value, you can generate a Completion report that
shows how many of the tests for each developer are complete.
contain the remaining levels of group descriptions and test descriptions. This
approach allows multi-user access, while at the same time maintaining a
single point of control for the entire project.
For example, consider the following testplan:
If you wanted to break this testplan up into a master plan and subplans, one
approach would be to place the case-sensitive tests in one subplan and the
case-insensitive tests in another, as shown in the following figure:
Note that subplans are specified with an include statement. To expand the
subplan files so that they are visible within the master plan, double-click in
the left margin next to the include statement.
For example, here is how the preceding master testplan looks when the
casesens.pln and the caseinsv.pln testplan files are expanded inline:
Once a subplan is expanded inline, the subplan statement changes from red
(the default color for statements) to magenta, indicating that the line is now
read-only and that the subplan is expanded inline. At the end of the expanded
subplan is the <eof> marker, which indicates the end of the subplan file.
Subplans inherit Statements, attributes, symbols, and test data defined in the master plan are
Information accessible within each of the subplans.
For example, consider the following master plan:
The subplans contained in the casesens.pln file and the caseinsv.pln file each
inherit, and can therefore use, the following from the master plan:
• The script statement
• The testcase statement
• The assignment statement for the $Text symbol
Creating the subplan You create a subplan in the same way you create any testplan: by opening a
new testplan file and entering the group descriptions, test descriptions, and
QA Organizer statements that comprise the subplan, either manually or using
the Testplan Detail dialog.
Adding the include To connect the master plan to a subplan file, you enter an include statement
statement to the in the master plan at the point where the subplan logically fits. The include
master plan statement cannot be entered through the Testplan Detail dialog, so it must be
entered manually. The include statement uses this syntax:
include: myinclude.pln
where myinclude is the name of the testplan file that contains the subplan.
If you enter the include statement correctly, it appears in red, the default
color used for QA Organizer statements. Otherwise, the statement appears in
blue or black, indicating you’ve made a syntax error (the compiler is
interpreting the line as a description, not a statement).
Sharing a testplan All QA engineers working on a large testplan that is broken up into a master
initialization file plan and subplans must use the same testplan initialization file. To do this, all
engineers should specify the same file name in the Data File for Attributes
and Queries field in the General Options dialog (invoked by selecting
Options/General).
Copying a subplan When you copy and paste the include statement and the contents of an open
include file, note that only the include statement will be pasted. To view the
contents of the subplan, open the pasted include file by selecting Include/
Open or by double-clicking in the margin to the left of the include statement.
Saving changes When finished editing, select Include/Save to save the changes to the subplan
currently being edited. This is distinct from File/Save, which saves all open
master plans and subplans.
Refreshing a local When another user modifies a subplan, those changes are not automatically
copy reflected in your read-only copy of the subplan. Once the other user has
released the lock on the subplan, there are two ways to refresh your copy:
• Close and then reopen the subplan
• Acquire a lock for the subplan
Chapter Page
Chapter 13, “Porting Tests to Other GUIs” 265
Chapter 14, “Supporting Internationalized Applications” 281
Introduction Using SilkTest, you can create portable testcases which will test your
application on any of the supported GUIs. The reason for this is that your
testcases use logical names, called identifiers, to refer to the GUI objects, and
not actual names, called tags. Therefore, if there are differences in the ported
application’s appearance, you need only change the window declarations, not
the testcases themselves.
What you will learn This chapter contains the following sections:
Topic Page
Marking 4Test code as GUI-specific 265
Conditional compilation 269
Supporting GUI-specific executables 271
Supporting GUI-specific captions 272
Supporting GUI-specific menu hierarchies 273
Supporting extra controls 273
Supporting different implementations of the same control 274
Supporting differences in application behavior 277
List of available GUI For a complete list of GUI specifiers, see “GUITYPE data type” in online
specifiers Help.
Syntax of a GUI A GUI specifier has this syntax:
specifier
[[gui-type [, gui-type]] | [! gui-type]]
gui-type is the GUI. You can express this in one of two mutually exclusive
ways. For example, you can specify one or more GUIs separated by commas,
as in:
motif, msw
Or you can specify all but one GUI, as in the following, which indicates that
what follows applies to all environments except Macintosh:
! mac
// NOT legal:
switch (i)
msw case 1
Print ("hello")
motif case 1
Print ("goodbye")
if statements You can use GUI specifiers in if statements, as long as GUI specifiers used
within the statement are subsets of any GUI specifiers that enclose the entire
if statement:
type statements You can use a GUI specifier before a type ... is enum or type ... is set
statement, but not before an individual value within the declaration.
do...except statements You can use GUI specifiers to enclose an entire do...except statement and
before individual statements, but you cannot use GUI specifiers before the
except clause.
// legal:
do
msw Verify (expr1,expr2)
motif Verify (expr3,expr4)
except
motif reraise
msw if (ExceptNum () == 1)
Print ("err, etc.")
// NOT legal:
msw do
Verify (expr,expr)
msw except
reraise
Class declarations Be careful using GUI specifiers before class declarations; they can be
ambiguous. Any ambiguities must be resolvable at compile-time.
// bad style:
msw winclass myclass
mac winclass myclass
window myclass inst // Ambiguous. Is it an instance of
With inheritance When using GUI specifiers for parent classes, you must explicitly use the
GUI specifiers with the descendants:
msw95 winclass newclass
msw95 winclass subclass : newclass
msw95 window subclass inst
With global variables Be careful when using GUI specifiers with global variables: SilkTest
initializes global variables before connecting to an Agent. This might not
give you the results you want if you are doing distributed testing.
Let’s say that you are running tests on a remote machine that is listed in
Runtime Options. Because SilkTest initializes all global variables before
connecting to an Agent, any GUI specifier at the global level will initialize to
the host machine, not the target machine you want to test against.
For example, say the host machine is running Window 95 and the target
machine is running Windows NT 4.0. Consider the following script:
mswnt40 STRING sVar1 = SYS_GetEnv("UserName")
msw95 STRING sVar1 = SYS_GetRegistryValue
(HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Control",
"Current User")
main()
print(sVar1)
This script fails, with the error message
*** Error: Registry entry 'Current User' not found
because sVar1 is initialized to the value for the host system (the GUI specifier
msw95), not the target system (mswnt40).
So, if you want GUI specifiers on variables at the global level, put the
initialization of these variables in the main function or another function that
is called after the Agent is contacted.
Conditional compilation
If you have GUI-specific code in your scripts and declarations, you can have
SilkTest conditionally compile your code based on the values of the GUI
specifiers—only code specific to a particular GUI is compiled (as well, of
course, as all code that is not GUI-specific). This has two advantages:
• The compilation is faster
• The resulting code is smaller and require less memory to run
You can also cause conditional compilation by using constants, which are
evaluated at compile time.
Note Constants are not restricted to conditional compilation. You
can use constants for any value that you want resolved at compile
time.
Note Setting a GUI target affects which classes are listed in the
Library Browser.
3 To conditionalize code based on the value of constants you define, do the
following:
a Click the Compiler Constants pushbutton in the Runtime Options
dialog.
The Compiler Constants dialog is displayed.
Procedure To load different include files for different versions of the test
application:
1 Define a compiler constant, as described above. For example, you could
define a constant named MyIncludeFile.
2 Insert the following statement into your 4Test file:
use constant
For example, if you defined a constant MyIncludeFile, insert the
following statement:
use MyIncludeFile
You should declare as many sCmdLine variables as there are GUIs on which
your application runs, beginning each declaration with the appropriate GUI
specifier.
For example, the following constants specify how SilkTest should start the
Text Editor application on Windows and Macintosh:
msw const sCmdLine = "c:\sample\texted2.exe"
mac const sCmdLine = "{SYS_GetDrive ()}:Text Editor:TextEd2"
Deciding which form When an object’s caption or label changes on a different GUI, it is usually
of tag to use preferable to use multiple tags, each based on the GUI-specific label or
caption, instead of using the index. Not only does it make your declarations
easier to understand, but it shields your testcases from changes to the
sequence of child objects. For example, if the Exit item changes so that it is
the fourth item and not the fifth, your testcases will still run.
For menus that SilkTest does not recognize as platform-specific, you should
preface the window declaration with the appropriate GUI specifier.
Example: different To illustrate the second case, suppose that the Edit menu for the Text Editor
items on a menu application has a menu item named Clear which appears on the Windows
version only. The declaration for the Edit menu should look like this:
Menu Edit
tag "Edit"
msw MenuItem Clear
tag "Clear"
MenuItem Undo
tag "Undo"
Dialog on Windows The Windows GUI allows an end user to enter the file name to be opened as a
path name in a text field; the Macintosh GUI does not. Therefore, the
following two declarations are recorded under Windows, but not under
Macintosh:
StaticText FileNameText
tag "File Name:"
TextField FileName1
tag "File Name:"
When merging the window declarations for the two GUIs, you need to
preface these declarations with the msw GUI specifier:
msw StaticText FileNameText
tag "File Name:"
msw TextField FileName1
tag "File Name:"
Dialog on Macintosh Similarly, the Macintosh version of the dialog has two pushbuttons which do
not appear on the Windows version. The first pushbutton allows the user to
eject a floppy disk from the disk drive. The second pushbutton allows the user
to navigate to the desktop directory. Therefore, the following two
declarations are recorded under Macintosh, but not under Windows:
PushButton Eject
tag "Eject"
PushButton Desktop
tag "Desktop"
So, when merging the window declarations for the two GUIs, you need to
preface these declarations with the mac GUI specifier:
mac PushButton Eject
tag "Eject"
mac PushButton Desktop
tag "Desktop"
For example, the methods for the RadioList and PopupList classes have
identical names, because the actions being performed by the methods are
similar. Therefore, if a control in your application is a popup list on one
GUI and a radio list on another, your scripts are already portable.
• If the two object classes do not have similar methods, or if the methods
have different names, then you need to follow the steps outlined in this
section to port your scripts.
Creating a class that Consider the Direction control in the Find dialog of the Text Editor
maps to several application, which allows a user to specify the direction (up or down) of
SilkTest classes searches. Suppose that this control is implemented as a check box on the
Macintosh, but as a radio list on all other GUIs. As a radio list, the user
selects either the Up or the Down radio button. On the Macintosh, the user
checks the check box to select Up, and leaves the check box unchecked to
select Down.
The first step in solving this portability scenario is to create a new window
class that you will use for the object on all platforms. The class you need to
define, in effect, generalizes several distinct 4Test classes into one logical
class.
To achieve this generalization, you:
• Derive the new class from the 4Test Control class, since both radio lists
and check boxes derive from this class.
• Define the class with a GUI-specific tag statement for each platform.
Each tag statement states the actual class of the control on the particular
GUI. This allows SilkTest to know what the actual class on the control
will be at runtime on each of the GUIs.
• Define generalized methods that use a switch statement to branch to the
4Test code that implements the method on the particular GUI.
Here is the class declaration, which is arbitrarily named DirButton:
// The class is derived from Control
default
RadioList (WndTag).Select (Item)
DirButton Direction
tag "Direction"
Using cross-platform Finally, in your scripts, you can use your cross-platform method names. The
methods in your window declarations map the cross-platform method names you use in your
scripts scripts to the actual methods required to carry out the actions you want on
each of the GUIs.
Continuing the example from the last section, you use the Select method in
your code to select the control named Direction.
testcase SearchBackward ()
LISTITEM Item
Item = "Up"
Find.Invoke ()
Find.Direction.Select (Item)
.
.
.
Find.Dismiss ()
Note that the script does not indicate that anything unusual is happening; all
of the steps necessary to make the Select method work properly, regardless of
the class of the object, are encapsulated in the class and window declarations.
case mswdos:
// code to test feature on windows
case motif:
// code to test feature on motif
case pm:
// code to test feature on Presentation Manager
Text field requires On some GUIs, the Enter/Return key must be pressed after data is entered
Return keystroke into a text field. Suppose you want to create a testcase which enters invalid
data into the field, and then checks if the application detects the error. After
the testcase enters the invalid data, it needs to use the GetGUIType function
to determine the GUI, and then press the Return key if the GUI requires it.
For example:
// code to enter an invalid string into field
if (GetGUIType () == motif)
MyTextField.TypeKeys ("<Return>")
Error messages are The VerifyErrorBox function, shown below, illustrates how to solve the
different problem of different error messages on each GUI platforms. In this example,
Motif always adds the prefix “Error:” to its message, while the other
platforms do not.
VerifyErrorBox (STRING sMsg)
// verifies that the error box has the correct error
// message, then dismisses the error box
Mechanism for In the Windows Open dialog, to navigate to a directory you can enter the path
navigating directories name of the directory into a text field. The Macintosh GUI, however, does not
is different allow you to enter paths; you must instead navigate to a working directory by
selecting successive directories from a list.
The SetWorkingDirectory function, shown below, illustrates how to solve this
problem. The function receives the sPath parameter, which is the path of the
directory to change to. The function contains a case statement which
• For the Macintosh, parses sPath into its constituent directories and
double-selects each directory name in succession until the working
directory is reached.
• For all other GUIs, enters the value of the sPath variable directly into the
appropriate text field.
SetWorkingDirectory (STRING sPath)
// this function sets the working directory
UserSetup.Invoke ()
SelectFolder.CurrentFolder.Click ()
UserSetup.Dismiss ()
default
UserSetup.Invoke ()
UserSetup.Directory.SetText (sPath)
UserSetup.Accept ()
Applications
Topic Page
Built-in support for international keyboards 281
Internationalizing tags 282
Internationalizing tags
If your application is localized into multiple languages, then the captions and
labels upon which SilkTest bases the tags it records are valid only for the
localized version you ran when recording your window declarations.
One alternative is to use index numbers for the tags instead of the caption/
label. However, this approach has the drawback that you are not able to verify
that the labels are correct when you are testing your application’s GUI
objects.
To make your declarations valid for all localized versions, one alternative is:
1 Replace every tag in your declarations with a STRING variable, if the tag
would otherwise be constructed from a caption or label.
2 Create an enumerated type of the possible languages. You use a variable
of this new enumerated type as an index into the STRING variable.
3 Assign to each STRING variable the correct localized string, by indexing
into a table of strings.
At runtime, SilkTest substitutes the appropriate localized string for each
variable. The rest of this section explains each of these steps.
Replace tags with For example, the following portion of the window declarations for the Text
string variables Editor application shows the declarations for the File menu and for the Find
dialog. Note that each tag has been replaced with a STRING variable.
window MainWin TextEditor
msw tag sMainWin
Menu File
tag sFile
MenuItem New
tag sNew
MenuItem Open
tag sOpen
MenuItem Save
tag sSave
MenuItem SaveAs
tag sSaveAs
MenuItem Exit
tag sExit
// omitted menu declarations
StaticText WhatText
tag sFindWhatText
TextField What
tag sFindWhat
CheckBox CaseSensitive
tag sCaseSensitive
RadioList Direction
tag sDirection
PushButton FindNext
tag sFindNext
PushButton Cancel
tag sCancel
SilkTest retrieves the value for each STRING variable using the language
index. You define the index and initialize the STRING variables as described
in the next two sections.
Create an enumerated You should create an enumerated type like the one shown below which
type for the languages defines the set of possible languages into which your application can be
localized. Although the values of the enumerated type can be any meaningful
set of strings, you might want to use the two-letter International Standard IS
639 language codes, as shown in this example:
type LANGUAGETYPE is enum
// uses the 2-letter IS 639 language code.
LT_DE // German
LT_EN // English
LT_ES // Spanish
Use current language Then, create and initialize a constant or a variable that specifies the language
as index that you want to test when you run your scripts. For example, to test the
English version of an application, you can create and initialize this constant:
Load string variables The final step is to create and initialize a STRING variable for each object in
with the localized the application, as shown in this example:
strings
STRING sFile = {"Datei", "File", "Archivo"}[LANGUAGE]
STRING sNew = {"Neu", "New", "Nuevo"}[LANGUAGE]
STRING sOpen = {"?ffnen", "Open", "Abrir"}[LANGUAGE]
STRING sSave = {"Speichern", "Save", "Guardar"}[LANGUAGE]
STRING sSaveAs = {"Speichern unter","Save As", "Guardar como"}[LANGUAGE]
STRING sExit = {"Beenden", "Exit", "Salir"}[LANGUAGE]
STRING sFind = {"Suchen", "Find", "Buscar"}[LANGUAGE]
STRING sWhatText = {"Suchen nach:", "Find What:", "Buscar"}[LANGUAGE]
STRING sWhat = {"Suchen nach:", "Find What:", "Buscar"}[LANGUAGE]
STRING sDirection = {"Suchrichtung", "Direction", "Direcci?n"}[LANGUAGE]
STRING sUp = {"Aufw?rts", "Up", "Arriba"}[LANGUAGE]
STRING sDown = {"Abw?rts", "Down", "Abajo"}[LANGUAGE]
STRING sFindNxt={"Weitersuchen","Find Next",Buscar siguiente"}[LANGUAGE]
STRING sCancel = {"Abbrechen", "Cancel", "Cancelar"}[LANGUAGE]
Each of those statements picks a value from a list based on the language,
which serves as the index into the list. For example, if LANGUAGE is LT_EN,
which is the second member of the enumerated data type LANGUAGETYPE,
then the first statement would evaluate to:
STRING sFile = {"Datei", "File", "Archivo"}[2]
So sFile would be assigned the value “File”, which is the second element in
the list. For more information about using lists, see “LIST data type” in online
Help.
Note If your application uses distinct captions/labels for each GUI,
then you should create multiple versions of each STRING variable
using GUI specifiers. See Chapter 13, “Porting Tests to Other
GUIs”, for more information. An alternative approach is to create a
two-dimensional list of strings, and use the GUI type and the
language as the indices.
Chapter Page
Chapter 15, “Understanding the Recovery System” 287
Chapter 16, “Extending the Class Hierarchy” 303
Chapter 17, “Supporting Custom Objects” 317
Chapter 18, “Modifying the Library Browser” 327
Recovery System
What you will learn This chapter contains the following sections:
Topic Page
Introduction to the recovery system 287
Setting up the recovery system 288
The recovery system’s flow of control 291
How the recovery system starts the application 292
How the recovery system closes windows 292
Specifying new window closing procedures 293
Specifying windows to be left open 296
Handling login windows 297
Adding to the default base state 298
Overriding the default recovery system 299
Modifying the default recovery system 301
For more information about application states, see “Using application states”
on page 185.
The base state The base state is the state you expect the application to be in at the start of
each testcase. Although this can differ from application to application, the
default recovery system considers an application to be at the base state when
• The application is running
• The application is not minimized
• The application is active
• No other window besides the main window is open
If this is not sufficient or if the recovery system is not able to achieve one or
more of these conditions, you need to modify the recovery system.
If you do not plan to test Web applications, you must disable all browser
extensions on the host machine.
2 Enable support for the browsers you want to use, if they aren’t already
enabled. In the Primary Extension field for a browser, choose Enabled.
3 Select any other extensions that the browser needs for testing such as
ActiveX or fault trapping. We recommend that you do not turn on fault
trapping until you really need it.
4 Click OK to close the Extension Enabler dialog.
Enabling browser You enable support for browsers on the host machine using the Extensions
support on a host dialog. Be advised that there is overhead associated with having more than
machine one browser extension enabled, so do so only if you are actually testing more
than one browser in an automated session.
Testcase
error?
BaseState method
To ensure that only the main window is open, the recovery system attempts to
close all other open windows, using an internal procedure that you can
customize when necessary.
To make sure that there are no application windows open except the main
window, the recovery system calls the built-in CloseWindows method. This
method starts with the currently active window and attempts to close it using
the sequence of steps below, stopping when the window closes.
If any of the steps fails, none of the following steps is executed and the
recovery system raises an exception.
1 If a Close method is defined for the window, call it.
2 Click on the Close menu item on the system menu (on platforms and
windows that have system menus).
3 Click on the window’s close box (if one exists).
4 If the window is a dialog, type each of the keys specified by the OPT_
CLOSE_DIALOG_KEYS option and wait one second for the dialog box to
close. By default, this option specifies the Esc key.
5 If there is a single button in the window, click that button.
6 Click each of the buttons specified by the OPT_CLOSE_WINDOW_
BUTTONS option. By default, this option specifies the Cancel, Close,
Exit, and Done keys.
7 Select each of the menu items specified by the OPT_CLOSE_WINDOW_
MENUS option. By default, this option specifies the File/Exit and the
File/Quit menu items.
8 If the closing of a window causes a confirmation dialog to appear,
CloseWindows attempts to close the dialog by clicking each of the
buttons specified with the OPT_CLOSE_CONFIRM_BUTTONS option. By
default, this option specifies the No button.
When the window (and any resulting confirmation dialog) closes,
CloseWindows repeats the preceding sequence of steps with the next window,
until all windows are closed.
The Close panel contains the following options, each of which takes a
comma-delimited list of character string values:
List of Buttons Used To Close a Window If a button press will close the
dialog, add the button name to the default list.
List of Menus Used To Close a Window If selecting a menu item will close
the dialog, add the menu item to the default list.
Name of Close Item On System Menu If selecting a menu item from the
system menu will close the dialog, add the menu item to the default list.
For a complete listing of the defaults for these options on all supported
platforms, see “Close options” on page 469.
Specify dynamically As you set Close options in the Agent Options dialog, the informational text
at the bottom of the dialog shows the 4Test command you can use to specify
the same option from within a script; add this 4Test command to a script if
you need to change the option dynamically as a script is running.
Specify for individual If you want to specify for an individual dialog the keys, menu items, and
objects buttons that the recovery system should use to close that dialog, define the
appropriate variable in the window declaration for the dialog:
• lsCloseWindowButtons
• lsCloseConfirmButtons
• lsCloseDialogKeys
• lsCloseWindowMenus
2 Remove the comment characters (the two forward slash characters) at the
beginning of the declaration.
For example, the following 4Test code shows how to set the lwLeaveOpen
constant so that the recovery system leaves open the window with the 4Test
identifier DocumentWindow when it restores the base state.
const lwLeaveOpen = {DocumentWindow}
Example After following this procedure, your test frame might look like this:
window MainWin MyApp
tag "My App"
const wStartup = Login
// the declarations for the MainWin should go here
Invoke ()
derived::Invoke ()
Login.Name.SetText ("Your name")
Login.Password.SetText ("password")
Login.OK.Click ()
About the derived keyword and scope resolution operator Notice the
statement derived::Invoke ( ). That statement uses the derived keyword
followed by the scope resolution operator ( :: ) to call the built-in Invoke
method, before performing the operations needed to fill in and dismiss the
login window.
For more information about the scope resolution operator, see online Help.
3 Place the insertion point on the declaration for the application’s main
window.
4 Select Record/Method.
SilkTest displays the Record Method dialog, which allows you to record
a method for a class or window declaration.
5 Press the arrow button (located to the right of the Method Name field)
and select BaseState from the drop-down list.
How the default The default recovery system is implemented through several functions, as
processing is described in the following table.
implemented
Function Purpose
DefaultBaseState Restore the default base state, then call the application’s
BaseState function, if defined.
DefaultScriptEnter Executed when a script file is first accessed.
Default action: none.
DefaultScriptExit Executed when a script file is exited.
Default action: Call the ExceptLog function if the script
had errors.
DefaultTestcaseEnter Executed when a testcase is about to start.
Default action: Set the application state.
DefaultTestcaseExit Executed when a testcase has ended.
Default action: Call the ExceptLog function if the script
had errors, then set the base state.
Overriding the default You can write functions that override some of the default behavior of the
processing recovery system.
Example You might want to specify that certain test files get copied from a
server in preparation for running a script. You would specify such processing
in a function called ScriptEnter in your test frame.
Hierarchy
The 4Test class hierarchy defines the methods and properties that enable you
to query, manipulate, and verify the data or state of any GUI object in your
application.
You can define your own methods and properties, as well as define your own
classes. You can also define your own attributes, which are used in the
verification stage in testcases.
What you will learn This chapter contains the following sections:
Topic Page
The class hierarchy 304
Adding methods to a class 306
Defining new class properties 308
Defining new verification properties 308
Defining new attributes 311
Defining new classes 313
The inheritance The inheritance depicted in the class hierarchy diagram can be summarized
embodied in the class as follows:
hierarchy
This class Defines the methods common to
AnyWin Any type of GUI object.
BrowserChild The window that contains the contents of a Web page.
Control Check boxes, combo boxes, list boxes, popup lists,
pushbuttons, radio lists, scales, scroll bars, static text
(labels), text fields, and so on.
CustomWin Does not apply.
Assigned to objects that do not correspond to any class
in the built-in class hierarchy. Because the CustomWin
class derives from the AnyWin class, you can use the
primitive methods of the AnyWin class, like PressKeys,
to define higher level actions to perform on your
custom GUI objects.
DesktopWin The desktop.
GuptaTable Gupta tables.
HtmlColumn A column within an HtmlTable in a Web application.
HtmlTable A series of two or more columns in a Web application.
Menu Apple menus, help menus, menu items, popup menus,
and system menus.
MoveableWin Child windows (free-standing windows, including MDI
sheets) dialog boxes, message boxes, and the
application’s main window.
TaskbarWin The taskbar in Windows 95 or Windows NT 4.0.
Logical classes The AnyWin, Control, and MoveableWin classes are logical (virtual) classes
that do not correspond to any actual GUI objects, but instead define methods
common to the classes that derive from them. This means that SilkTest never
records a declaration that has one of these classes.
CursorClass, There are three classes that are not part of the AnyWin class hierarchy,
ClipboardClass, and because they define methods for objects that are not windows:
AgentClass
• CursorClass, which defines the three methods you can use on the cursor:
GetPosition, GetType, and Wait.
• ClipboardClass, which defines the two methods you can use on the
system clipboard: GetText and SetText.
• AgentClass, which defines the methods you can use to set options in the
4Test Agent. (The 4Test Agent is the component of SilkTest that
translates the method calls in your testcases into the appropriate GUI-
specific event streams.)
The predefined identifiers for the Cursor, Clipboard, and Agent You do
not record declarations for the cursor, the clipboard, or the Agent. Instead,
you use predefined identifiers for each of these objects when you want to use
a method to act against the object:
• The predefined identifier for the 4Test Agent is Agent
• The predefined identifier for the clipboard is Clipboard
• The predefined identifier for the mouse pointer is Cursor
For example, to set a 4Test Agent option, you use a call such as the following:
Agent.SetOption (OPT_VERIFY_COORD, TRUE)
DesktopWin Because the desktop is a GUI object, it derives from the AnyWin class.
However, unlike other GUI objects, you do not have to record a declaration
for the desktop. Instead, you use the predefined identifier Desktop when you
want to use a method on the desktop.
For example, to call the GetActive method on the desktop, you use a call like
the following:
wActive = Desktop.GetActive ()
Redefining a method There may be some instances in which you want to redefine an existing
method. For example, to redefine the GetCaption method of the Anywin
class, you use this 4Test code:
winclass AnyWin : AnyWin
GetCaption ()
// insert method definition here
Deriving a new To derive a new method from an existing method, you can use the derived
method from an keyword followed by the scope resolution operator (::).
existing method
The following example defines a GetCaption method for DialogBox that
prints the string “Dialog” before calling the built-in GetCaption method
(defined in the AnyWin class) and printing its return value:
winclass DialogBox : DialogBox
GetCaption ()
Print ("Dialog")
Print (derived::GetCaption ())
Polymorphism If a class defines its own version of a method or property, that method or
property overrides the one inherited from an ancestor. This is referred to as
polymorphism. For example, the ListBox class has its own GetContents
method, which overrides the GetContents method inherited from the AnyWin
class.
Defining a new To define a new method to use on a single GUI object, not for an entire class
method for a single of objects, you add the method definition to the window declaration for the
GUI object individual object, not to the class. The syntax is exactly the same as when you
define a method for a class. For examples, see “Defining a method for a GUI
object” on page 97.
And let’s say that you have defined a class property, NumChildren, that you
want to make available to the verification system.
// user-defined property
property NumChildren
INTEGER Get ()
return ListCount (GetChildren ())
INTEGER NumChild2
Here is the 4Test code in the winclass.inc file that implements the Default
Button attribute:
attribute "Default button", VerifyDefaultButton, QueryDefaultButton
As this 4Test code examples shows, each attribute definition begins with the
attribute statement, followed by three comma-delimited values:
1 The text that you want to appear in the Attribute panel of the Verify
Window dialog. This text must be a string.
2 The method SilkTest should use to verify the value of the attribute at
runtime.
3 The method SilkTest should use to get the actual value of the attribute at
runtime.
Syntax To add one or more attributes to an existing class, use the following syntax:
winclass ExistingClass : ExistingClass...
attribute_definitions
Each attribute definition must begin and end on its own line.
When you define a new attribute, you usually need to define two new
methods (steps 2 and 3 above) if none of the built-in methods suffice.
For example, to add a new attribute to the DialogBox class that verifies the
number of children in the dialog, you add code like this to your test frame (or
other include file):
winclass DialogBox:DialogBox
attribute "Number of children", VerifyNumChild, GetNumChild
integer GetNumChild()
return ListCount (GetChildren ()) // return count of children of dialog
As this example shows, you use the hidecalls keyword when defining the
verification method for the new attribute.
About the hidecalls keyword The keyword hidecalls hides the method
from the call stack listed in the results. Using hidecalls allows you to update
the expected value of the verification method from the results. If you don’t
use hidecalls in a verification method, the results file will point to the frame
file, where the method is defined, instead of to the script. We recommend that
you use hidecalls in all verification methods so you can update the expected
values.
The Number of Children attribute is listed in the Verify Window dialog when
you do an attribute verification:
StaticText DirectoriesText
tag "Directories:"
StaticText PathText
tag "#3"
ListBox Path
tag "#2"
StaticText ListFilesOfTypeText
tag "List Files of Type:"
PopupList ListFilesOfType
tag "List Files of Type:"
StaticText DrivesText
tag "Drives:"
PopupList Drives
tag "Drives:"
PushButton OK
tag "OK"
PushButton Cancel
tag "Cancel"
PushButton Network
tag "Network"
To make use of this new class, you
1 Rewrite the declarations for the Open and Save As dialogs, changing the
class to FileDialog.
2 Remove the declarations for the child objects inherited from the new
class.
Here are the rewritten declarations for the Open and Save As dialogs:
window FileDialog SaveAs
tag "Save As"
window FileDialog Open
tag "Open"
For more information on the syntax used in declaring new classes, see
“winclass declaration” in online Help.
Introduction While you record window declarations, SilkTest attempts to identify the class
of each object in your GUI and assign the appropriate class from the built-in
class hierarchy. If an object does not correspond to one of the built-in classes,
SilkTest designates the object as a custom window and assigns it to the class
CustomWin.
This chapter shows you how to support certain kinds of custom objects by
modifying your window declarations and by creating new classes and
methods.
What you will learn This chapter contains the following sections:
Topic Page
Why SilkTest sees objects as custom windows 317
Mapping custom classes to standard classes 318
Supporting graphical controls 319
Supporting nongraphical controls 322
4 In the Standard Class field, enter the name of the built-in 4Test class to
which you are mapping the custom class.
5 Click Add.
6 Click OK.
When you resume recording, the object has the standard 4Test class. Any
subsequent similar objects encountered will automatically be mapped to the
correct 4Test class. You must modify any prerecorded declarations containing
these objects to use the standard class.
Add a location suffix You can, however, create declarations for each discrete object. To do this,
to the declaration’s make as many copies of the original recorded declaration as there are discrete
tag objects. Then add a location suffix to the tag in each declaration, which is the
location of the object within the graphical area.
SilkTest provides two ways to specify the location suffix of “contained”
graphical objects, depending on the size and spacing of the control:
• Controls that are sized and spaced evenly in a grid
• Controls that are irregularly sized and spaced
Evenly sized and If a group of graphical controls are equal in size and evenly spaced in a grid,
spaced controls you can specify the location of each control as column y of the total number
of columns and row x of the total number of rows. This syntax is both
cross-platform and resolution independent.
Example Consider the following declaration for a tool bar that contains a
single row of 26 icons:
CustomWin PaintWindow1
mswdos tag "[PaintWindow]#1"
Note the following:
• Because there is no label or caption for the tool bar, SilkTest uses an
index for the tag.
• SilkTest qualifies the tag with the name of the class, PaintWindow,
enclosed within square brackets ([]).
• SilkTest bases the identifier PaintWindow1 on the internal class, since
the tool bar has no label or caption.
(3:26, 1:1)
You append this location to the tag with the forward slash (/) character.
Example To alter the previous declaration for the tool bar so that it becomes
the declaration for the Printer icon, which is the third icon, you write this
declaration:
CustomWin Printer
mswdos tag "[PaintWindow]#1/(3:26, 1:1)"
The declarations for the first seven icons in the tool bar are shown here:
CustomWin FileOpen
mswdos tag "[PaintWindow]#1/(1:26, 1:1)"
CustomWin Save
mswdos tag "[PaintWindow]#1/(2:26, 1:1)"
CustomWin Printer
mswdos tag "[PaintWindow]#1/(3:26, 1:1)"
CustomWin Preview
mswdos tag "[PaintWindow]#1/(4:26, 1:1)"
CustomWin Cut
mswdos tag "[PaintWindow]#1/(5:26, 1:1)"
CustomWin Copy
mswdos tag "[PaintWindow]#1/(6:26, 1:1)"
CustomWin Paste
mswdos tag "[PaintWindow]#1/(7:26, 1:1)"
Using the modified Once you create window declarations like these for the graphical objects in
declaration your application, you can manipulate them as you would any other object.
For example, if the tool bar was contained in an application named MyApp,
to click on the FileOpen icon in the tool bar, you use this command:
MyApp.FileOpen.Click ()
Note You need to write this statement—and others that access the
objects declared above, such as Save and Printer—by hand.
Record/Testcase and Record/Actions will not use these identifiers.
Irregularly sized or If the graphical controls in a group are not the same size or are not evenly
spaced controls spaced in a grid, you need to specify in the declaration the location suffix of
each control as an exact x,y point. This x,y point typically corresponds to the
center of the object. This syntax is not necessarily cross-platform or
resolution independent.
You specify a location in its declaration as an x,y coordinate using the
following syntax:
(x, y)
You append this location to the tag with the forward slash (/) character.
Example The following declaration specifies that the FileOpen icon on the
tool bar is located at position 74, 22.
CustomWin FileOpen
mswdos tag "[PaintWindow]#1/(74,22)"
• Otherwise, you can add partial support for the custom object by creating
a new class derived from AnyWin and writing methods that use the
primitive methods of the AnyWin class, like TypeKeys and MoveMouse,
to implement, as much as is possible, the methods appropriate for the
non-graphical control.
Supporting custom Suppose your application has an object that acts like a text field, but which is
text fields not implemented using your GUI’s standard text field object. The following
example illustrates how you can derive a new class from AnyWin and define
methods for the custom object. The example defines the ClearText,
GetMultiText, SetMultiText, and GetMultiSelText methods.
// This new class defines methods that re-implement
// the methods of the TextField class so that they will
// work on custom text fields. To be able to use these
// methods, you must change the class of object in the
// declarations from CustomWin to CustomTextField.
Using Clipboard If you are having trouble getting or setting information with a custom object
methods that contains text, you might want to try the 4Test Clipboard methods. For
example, assume you have a class, CustomTextBuffer, which is similar to a
TextField, but using the TextField’s GetText and SetText methods do not
work with the CustomTextBuffer. You can use the ClipboardClass GetText
and SetText methods as follows.
Getting text The following sample code retrieves the contents of the
CustomTextBuffer by placing it on the Clipboard, then printing the Clipboard
contents:
// Go to beginning of text field
CustomTextBuffer.TypeKeys ("<Ctrl-Home>")
// Highlight it
CustomTextBuffer.TypeKeys ("<Ctrl-Shift-End>")
// Copy it to the Clipboard
CustomTextBuffer.TypeKeys ("<Ctrl-Insert>")
// Print the contents of the Clipboard
Print (Clipboard.GetText())
Setting text Similarly, the following sample code inserts text into the custom
object by pasting it from the Clipboard:
// Go to beginning of text field
CustomTextBuffer.TypeKeys ("<Ctrl-Home>")
// Highlight it
CustomTextBuffer.TypeKeys ("<Ctrl-Shift-End>")
// Paste the Clipboard contents into the text field
CustomTextBuffer.TypeKeys ("<Shift-Insert>")
Tip You can wrap this functionality in GetText and SetText methods
you define for your custom class, similar to what was shown in
“Supporting custom text fields” on page 323.
Supporting custom To support custom list boxes, you can write “low-level” methods as
list boxes demonstrated in the previous section.
A second approach is to implement the necessary Microsoft Windows
messages and set the necessary Windows style bits so that you can use the
standard list box methods on your custom list box. The Microsoft Windows
messages which you must implement are:
• LB_GETCOUNT
• LB_GETTEXT
• LB_GETITEMRECT
• LB_GETTOPINDEX
• LB_GETSEL
• LB_GETTEXTLEN
And the Windows style bits that you must set are:
• WS_VSCROLL
• LBS_EXTENDEDSEL
• LBS_MULTIPLESEL
Browser
Topic Page
The standard Library Browser contents 327
Modifying the Library Browser contents 329
User-defined methods The Library Browser displays the same information for user-defined methods
that it does for built-in methods, except that it does not include descriptive
text for the method or its parameters. It displays User defined as the method
description and displays the data type for each parameter.
Note If the Library Browser isn’t displaying your user-defined
objects, close the Library Browser, recompile the include files that
contain your user-defined objects, then reopen the Library Browser.
Here is the default display for the user-defined method VerifyNumChild:
Properties The Library Browser also shows all built-in and user-defined properties. As
with user-defined methods, the description for user-defined properties by
default is User defined.
Functions By default, the Library Browser shows all built-in functions but does not
display your user-defined functions.
Edit your copy of 4test.txt to add the information you want, as shown in the
following examples.
User-defined methods To fully document your methods, add the method descriptions to the
appropriate class section in 4test.txt, such as:
#******************************************************
class: DialogBox
...
#** custom method
Compare this with the figure under “User-defined methods” on page 328,
which shows the default display for VerifyNumChild.
User-defined To include full documentation for your custom properties in the Library
properties Browser, add the property descriptions to the appropriate class section in
4test.txt, such as:
#***********************************************
class: DialogBox
...
#** custom property
property: iNumChild
notes: The number of child objects in the dialog box.
User-defined If you want your user-defined functions to display in the Library Browser,
functions create a group called User-defined functions and document your functions,
such as:
group: User-defined functions
Applications
Chapter Page
Chapter 19, “Introduction to Client/Server Testing” 337
Chapter 20, “Configuring SilkTest for Client/Server Testing” 351
Chapter 21, “Implementing Client/Server Testing” 363
Chapter 22, “Multi-Application Testing” 381
Testing
Topic Page
Client/server testing challenges 337
Client/server testing with SilkTest 338
The kinds of network testing you can perform 339
Client/server testing configurations 343
QA DBTester features 346
Concurrent programming issues 347
also communicate across a network with remote devices and with a variety of
other applications. The step from testing a stand-alone application running on
a single PC or workstation to testing an application that runs on multiple
different platforms and communicates across a network with other
applications is a big one. Here are just a few of the testing methodology
problems raised by such a step:
• Managing simultaneous automatic regression tests on different
configurations and/or platforms.
• Ensuring the reproducibility of client/server tests that modify a server
database.
• Verifying a client application’s server operations independently without
relying on the application under test.
• Testing the concurrency features of a client/server application.
• Testing the intercommunication capabilities of networked applications.
• Closing down multiple failed applications and bringing them back to a
particular base state (recovery control).
• Testing the functioning of the server application when driven at peak
request rates and at maximum data rates (peak load and volume testing).
• Automated regression testing of multi-tier client/server architectures.
To test for problems with concurrent access to the same database record, you
need to write specific scripts that synchronize two clients to make requests of
the same records in your server’s database(s) at the same time. Your goal is to
encounter faulty read/write locks, software deadlocks, or other concurrency
problems. See “Code for concurrency test example” on page 389 for an
example of a test that demonstrates the necessary 4Test coding techniques.
Once the application passes the functional tests, you can test the boundary
conditions that might be reached by large numbers of transactions. The next
sections describe this kind of testing.
Stress testing Stress testing verifies that running a particular command or set of commands
a large number of times does not break the software. For example, stress
testing discovers a failure to release resources. This is no different than stress
testing for a non-distributed application, but as with functional testing, you
need to discover whether the application functions properly with a remote
server under stress conditions. You can do this type of stress testing with a
single client.
You can create stress tests by repeating the same commands a large number
of times. Usually, the 4Test code looks like this:
for i = 1 to 50
DoSomeCommands ()
For example, you can stress-test remote logins or you can stress-test all
application operations that allocate significant amounts of memory (to see if
they release that memory). One simple form of stress testing is to set up your
functional tests to repeat indefinitely and then run them night and day over a
two- to four-day period.
Load testing Load testing verifies that running a large number of concurrent clients does
not break the server or client software. For example, load testing discovers
deadlocks and problems with queues. SilkTest’s unique architecture provides
built-in load testing. You can implement your functional tests using multi-
application techniques and then, after completing functional testing on a
single machine, you can run them on multiple targets. Unlike simulated load
testing tools, SilkTest allows you to create a real-life scenario that can
determine the actual impact of multi-machine operation on your application,
the server, the network, and all related elements.
There are two variations of load testing: peak load (which is sometimes
called stress testing in the client/server environment) and volume.
Peak load testing is placing a load on the server for a short time to emulate
the heaviest demand that would be generated at peak user times—for
example, credit card verification between noon and 1 PM on Christmas Eve.
chart the response times for different numbers of representative users. You
can then repeat the test with a more (or less) powerful server. See the
extension kit guide for more information.
While SilkTest does not provide support for sophisticated performance
analysis, this is an easy method for characterizing user-perceived
performance. It provides a method for gathering data for configuration
guidelines that requires little more effort than the repackaging of tests that
were already written for functional testing.
Machine 1 Machine 2
SilkTest SilkTest
Agent Agent
Client
Application
Server
Testing a stand-alone Server
application with
SilkTest Testing a client/server
application with software all
in one machine
The next figure shows a testing configuration in which the client application
runs on the same machine as SilkTest and the server application runs on a
separate machine. Note that in this configuration and with Machine 2 in the
previous figure, there is no communication between SilkTest and the server.
This means that you must manage the work of starting and initializing the
server database manually. For some kinds of testing this is appropriate.
This configuration lets you test the remote client-to-server connection and is
appropriate for many stress tests. It allows you to do volume load testing
from the point of view of the client application, but not the server.
Machine 1 Machine 2
SilkTest
Server
Agent
Client
The next figure shows multiple copies of the client application running on
separate machines, with SilkTest driving the client application by means of
the Agent process on each client machine, and the client application driving
the server application. This is just the multi-client version of the previous
configuration. You could run a fourth instance of the client application on the
SilkTest machine. The actual number of client machines used is your choice.
This configuration is appropriate for load testing and configuration testing if
you have no need to automatically manipulate the server. You must have at
least two clients to test concurrency and mutual-exclusion functionality.
Target Machines
Agent
Client
Agent
SilkTest Server
Client
Agent
Client
Once you are running SilkTest, it makes sense to have your script initialize
your server automatically. The last figure uses the same hardware
configuration as the previous figure, but SilkTest is also driving the server
directly.
This figure shows SilkTest using an Agent on the server machine to drive the
server’s GUI (the lower connecting arrow); this approach can be used to start
the server’s database and sometimes can be used to initialize it to a base state.
The upper arrow shows SilkTest using SQL commands to directly manipulate
the server database; use this approach when using the Agent is not sufficient.
After starting the database with the Agent, use SQL commands to initialize it
to a base state. The SQL commands are submitted by means of SilkTest’s
database functions, which do not require the services of the Agent.
This is the most complete testing configuration. It requires QA DBTester.
You can use it for all types of SilkTest testing, including volume load testing
of the server, peak load testing, and performance testing.
QA DBTester
Agent
Server
SilkTest
Client Agent
The special features that allow SilkTest to provide rigorous testing for client/
server applications are the following:
• Automatic control of multiple applications
• Multithreading for automatic control of concurrent applications
• Reporting results by thread ID
• Testing across networks using a variety of protocols
The added value that QA DBTester provides for the client/server environment
is: Direct database access from the test script.
These different features are discussed in the sections that follow.
QA DBTester features
You may be testing a distributed application that accesses a database or you
may be directly testing database software. In either of these cases, you might
want to manipulate the database directly from SilkTest for several purposes:
• To exercise certain database functions that are present in a GUI that runs
directly on the server machine and is not a client application. For
example, administrative functions used for setting up the database.
• To set the server database to a known state.
• To verify an application’s database results without using the application.
• To read information from the database to use as input to a testcase.
SilkTest can drive a server application’s GUI by means of the SilkTest Agent
exactly as it drives a client application. In addition, QA DBTester, a
companion product to SilkTest, provides direct access, using SQL, from a test
script to any database supported by ODBC drivers. These database functions
enable you to read and write database records without using the client
application. Thus you can verify client test results without assuming the
ability of the client to do that verification.
In addition to using the SQL functions in your tests, you can also use these
functions to help manage your testing process as follows:
• Maintain a bug database, updating it with the results of your testing.
• Manage your test data in a database instead of in a text file.
The database functions, among other things, allow you to connect to a
database, submit an SQL statement, read data from the selected record(s) if
the SQL statement was SELECT, and subsequently disconnect from the
database. About a dozen of these functions allow you to access your
database’s catalog tables.
The QA DBTester functions that support these operations are documented in
online Help. They begin with the letters “DB_”.
Please refer to the machine handle entry in the 4Test Language Reference for
a detailed description of the machine handle operator.
Since 4Test allows you to pass variables to these functions, you can write a
block of code that sends the same operations to a particular set of target
machines and you can pass the SetMachine function in that block of code a
variable initialized from a list that specifies the machines in that set. Thus
specifying which machines receive which operations is very simple. Chapter
22, “Multi-Application Testing” gives examples and detailed explanations of
these techniques.
Recovering multiple There are three major categories of operations that an Agent executes on a
tests target machine:
• Setup operations that bring the application to the state from which the
next test will start.
• Testing operations that exercise a portion of the application and verify
that it executed correctly.
• Cleanup operations that handle the normal completion of a test plus the
case where the test failed and the application is left in an indeterminate
state. In either case, the cleanup operations return the application to a
known base state.
When there are multiple machines being tested and more than one
application, the Agent on each machine must execute the correct operations
to establish the appropriate state, regardless of the current state of the
application. Chapter 22, “Multi-Application Testing” explains how to use the
4Test support functions with which a testcase accomplishes this and gives
code examples.
Concurrency For SilkTest, concurrent processing means that Agents on a specified set of
machines drive the associated applications simultaneously. To accomplish
this, the host machine interleaves execution of the sets of code assigned to
each machine. This means that when you are executing identical tests on
several machines, each machine can be in the process of executing the same
operation (for example, select the Edit.FindChange menu item).
At the end of a set of concurrent operations, you will frequently want to
synchronize the machines so that you know that all are ready and waiting
before you submit the next operation. You can do this easily with 4Test.
There are several reasons for executing testcases concurrently:
• You want to save testing time by running your functional tests for all the
different platforms at the same time and by logging the results centrally
(on the host machine).
• You are testing cross-network operations.
• You need to place a multi-user load on the server.
• You are testing the application’s handling of concurrent access to the
same database record on the server.
To accomplish the last example, testing concurrent database accesses, you
simply set all the machines to be ready to make the access and then you
synchronize. When all the machines are ready, you execute the operation that
commits the access operation—for example, clicking the OK button.
Consider the following example:
// [A] Execute 6 operations on all machines concurrently
for each sMachine in lsMachine
spawn
SixOpsFunction (sMachine)
rendezvous // Synchronize
Client/Server Testing
Topic Page
Understanding host and target machines 351
Networking protocols used by SilkTest 352
The configuration process 353
Selecting the SilkTest protocol 354
Configuration tasks 355
The following table lists the protocols that SilkTest can use for each of the
platform types.
NetBIOS/
Platform TCP/IP NetBEUI IPX/SPX AppleTalk
Windows 3.1 • •
Windows 3.11 • • •
Windows 95 • •
Windows NT • •
OS/2 • •
Solaris •
SunOS •
HP-UX •
AIX •
IRIX •
Macintosh •
There is no limit on the protocol or API an application under test may use.
Just make sure that the protocol required by SilkTest and the protocol
required by your application are running at the same time.
• Gather the information that your test scripts will need when making
explicit connections. For example, you can edit the Agent names into a
list definition and have your testplan pass the list variable name as an
argument for testcases controlled by that plan. The testcases then pass
each Agent name to a Connect or SetUpMachine function and that
function makes the explicit host-to-Agent connection.
The rest of this chapter provides configuration details specific to the different
protocols and operating systems you might be using. This detailed
information makes it possible for you to implement the configuration process
outlined above. Look for headings that mention topics applicable to your
configuration.
Use TCP/IP...
If the host or any Agent is Motif-based, you must use TCP/IP.
Or you can use TCP/IP just because you prefer to.
Use IPX/SPX...
If SilkTest and all its Agents are running MS Windows 3.11 on 16-bit PCs in
a Novell network, you can save the overhead of NetBIOS by using Novell’s
IPX/SPX protocol. The PCs must all be running some kind of Novell network
software, such as Netware or Personal Netware, that can use IPX/SPX as the
communications protocol. If you are running under 32-bit Windows or
running Agents under OS/2, NetBIOS does not present an overhead problem
because it is included in these systems.
Note Since all Agents must use the same protocol, if you want to
drive Agents on both 16-bit and 32-bit PCs you must use NetBIOS
or TCP/IP.
Use NetBIOS/NetBEUI...
If this is the protocol you normally use and you will not be including any
Motif-based workstations in your testing configuration, you can use
NetBIOS.
Configuration tasks
This section presents some configuration requirements that must be met to
use SilkTest to test applications over a network. In general you should set up
your Agents and make all adjustments to the partner.ini file or environment
variables before enabling networking on the host machine.
As you look through the rest of this chapter, please be sure to read all sections
that apply to your configuration. You can ignore topics that do not apply to
your configuration.
that your scripts will use to direct test operations to the associated
application. Use the following procedure to enable networking and assign the
Agent its network name:
1 Start the Agent software.
2 Left-click the Agent icon to display the icon menu.
3 Select the Network item from that menu.
4 In the dialog that appears, select the network type from the Network
pulldown list.
5 For NetBIOS, enter the Agent name into the Agent name field. For TCP/
IP, the default port number appears in the Port number field. You
normally want to accept the default, but if you have port number
conflicts, read the section “And you have port number conflicts...” on
page 356.
6 Click OK.
You only need to execute this procedure the first time you run that Agent.
Multiple remote When you enable networking by selecting the networking type in the
applications Runtime Options dialog on the host, do not set the Agent Name field to an
Agent name if you have multiple remote Agents. This field only accepts a
single Agent name and using it prevents you from handling all your client
machines the same way. If you specify one Agent name from your set of
Agents, then you cannot issue a Connect call to that Agent and thus would
not receive the machine handle that the Connect function returns. Since you
have to issue some Connect calls, be consistent and avoid writing exception
code to handle a machine that is automatically connected.
number, you have a port number conflict. If you start an Agent on a Motif
workstation and there is a port conflict, UNIX assigns a new port and prints
its number to standard output. If you start an Agent on a PC and the default
port is already in use, you get an error message. In either case, you can use a
different port number just for this machine (using the default number for the
rest) or you can have all your machines use the same available port number.
When you have an Agent that uses other than the default port number, you
have to specify the port number in every reference to that Agent. The syntax
is:
"AgentName:nnnn"
where AgentName is the target machine name and nnnn is the port number.
Since you will normally use a file or a list variable to hold your Agent names,
you can just add the ":nnnn" where needed.
In most cases, the default TCP/IP port specified by UNIX will be available
for SilkTest. If there are no port conflicts, there is nothing you have to do to
specify ports. If you have a conflict, the port number used for that machine
must change and you can choose to change the port numbers used by all your
PCs and workstations so that all use the same number. See the instructions
below for PCs and for workstations.
Enable networking on Once the protocol has been picked for any PC Agents and the port settings
TCP/IP host are consistent, you can enable networking on the host. Do this by selecting
Options/Runtime and setting the port number and/or Agent name. You can
skip this step if you do not have to change the default port number and you
are not specifying an Agent name for a single-remote-application
configuration.
For Windows 95/NT, move the mouse pointer over the Agent icon and wait
two seconds; the icon name will appear.
Enable networking on You must enable networking for each Agent PC by selecting the protocol
Agent type to be used in the Agent window (this operation is described in “One or
more remote applications” on page 355). When you select TCP/IP as the
protocol, you will see the port number field displayed with the default TCP/
IP port number. When you click OK, the selection is accepted if the default
port is available.
Handle port number If when you click OK in the Agent window on a PC, the system responds
conflict with an error message stating that the number is in use, then you have a port
number conflict. You can find an available port number for that machine and
leave the rest as the default. Or you can find a port number that all your
machines can use. Invoke the Agent window again and specify an alternative
port number. The value of the port number must be in the range 1000 to 9999.
If this number is accepted, you can try setting all of your PCs to this port—
again using the Agent window. When you find an available port for all your
PCs, check its availability with any workstations in your testing network (as
described below). Port number conflicts are rare.
Note You only have to set the port number once.
The value of nnnn must be in the range 1000 to 9999. If any workstation
cannot use the proposed value (an unlikely possibility), try another. Then set
this number on any PCs you have in the testing network (as described above).
Note When you start an Agent that does not use the default port
number, you must always specify the number you want used;
otherwise UNIX selects a number for you.
Handle limited If your testbed seems to be short one SilkTest Agent license, this may be
licenses because SilkTest has started up an unplanned Agent on the local workstation.
If you do not want to use the local workstation as a test machine, set the
Runtime Options dialog’s Agent Name field to “(none)” instead of “(local).”
This will free up one license for a remote Agent.
The first four bytes are the network address, which is usually all zeros in a
local area network. The last six bytes are the host address. You can find the
host address by means of the Netware Diagnostics program, or by asking
your system manager.
Enable networking on You assign the selected name to the Agent when you enable networking for
Agent each Agent PC (this operation was described in “One or more remote
applications” on page 355).
Enable networking on Once the protocol has been picked for all the Agents and they are named, you
NetBIOS host can enable networking on the host. Do this by selecting Options/Runtime and
selecting the NetBIOS network type. Then fill in the Agent name if you have
a single-remote-application configuration.
Testing
What you will learn This chapter contains the following sections:
Topic Page
Evolving a testing strategy 364
Connecting to a target machine 364
Using 4Test’s parallel processing features 365
Reporting distributed results 372
Incremental functional test design 373
Running tests on one remote target 374
Running tests serially on multiple targets 374
Testing clients plus server serially 375
Testing clients concurrently 376
Testing in parallel, but not synchronously 378
Application states and the recovery system 380
As a runtime option: You can specify the target’s name in the host’s
Runtime Options dialog before you run a script or suite. This is sufficient
when you are connecting to only one Agent. If you have been testing a script
by running SilkTest and Agent on the same system, you can then test the
script on a remote system without editing your script by using this method.
Threads and In the 4Test environment, a thread is a mechanism for interleaving the
concurrent execution of blocks of client code assigned to different Agents so that one
programming script can drive multiple client applications simultaneously. A thread is part
of the script that starts it; a thread is not a separate script. Each thread has its
own call stack and data stack. However, all the threads that a script spawns
share access to the same global variables, function arguments, and data types.
A file that one thread opens is accessible to any thread in that script.
While the creation of a thread carries no requirement that you use it to submit
operations to a client application, the normal reason for creating a
multithread script is so that each thread can drive test functions for one client,
which allows multiple client application operations to execute in parallel.
When a script connects to a machine, any thread in that script is also
connected to the machine. Therefore you must direct the testing operations in
a thread to a particular Agent machine. The following sections, in particular
“Specifying the target” provide more details.
Note Threads interleave at the machine instruction level, therefore
no single 4Test statement is atomic with respect to a statement in
another thread.
Parallel processing You create and manage multiple threads using combinations of the 4Test
statements statements parallel, spawn, rendezvous, and critical:
• A spawn statement begins execution of the specified statement or block
of statements in a new thread. Since the purpose of spawn is to initiate
concurrent test operations on multiple machines, the structure of a block
of spawned code is typically:
a A SetMachine command, which directs subsequent machine
operations to the specified Agent.
b A set of machine operations to drive the application.
c A verification of the results of the machine operations.
• A rendezvous statement blocks execution of the calling thread until all
threads that were spawned by the calling thread have completed. When
the last child thread exits, it unblocks the parent thread. The rendezvous
statement is unnecessary if it is the last statement at the end of the
testcase. If the calling thread has no child threads, rendezvous does
nothing.
• A parallel statement spawns a statement for each machine specified and
blocks the calling thread until the threads it spawns have all completed. It
condenses the actions of spawn and rendezvous and can make code
more readable.
• A critical statement guarantees that no other thread will execute until the
specified statement or indented block of code has executed and passed
control to the next statement at the level of the critical statement. Only
one thread can have “critical” status at any time.
In 4Test, all running threads (those not blocked) have the same priority with
respect to one another. 4Test executes one instruction for a thread, then passes
control to the next thread. The first thread called is the first run, and so on.
All threads run to completion unless they are deadlocked. 4Test will detect a
script deadlock and raise an exception (see “How 4Test handles script
deadlock” on page 372). Note that the 4Test exit statement terminates all
threads immediately when it is executed by one thread.
The parallel statement The parallel statement executes a single statement for each thread. Thus if
you wish to run complete tests in parallel threads, use the invocation of a test
function (which may execute many statements) with the parallel statement,
or use a block of statements with spawn and rendezvous.
To use the parallel statement, you have to specify the machines for which
threads are to be started. You can follow the parallel keyword with a list of
statements, each of which specifies a different Agent name. For example:
parallel
DoSomething ("Client1")
DoSomething ("Client2")
The above example could achieve the same result when written as follows:
for each sMachine in lsMachine
spawn
[sMachine]DoSomething ()
rendezvous
Specifying the target While the typical purpose for a thread is to direct test operations to a
particular test machine, you have total flexibility as to which machine is
being driven by a particular thread at any point in time. For example, in the
code below, the spawn statement starts a thread for each machine in a
predefined list of test machines. The SetMachine command directs the code
in that thread to the Agent on the specified machine. But the [“server”]
machine handle operator directs the code in the doThis function to the
machine named “server.” The code following the doThis invocation
continues to be sent to the sMachine specified in the SetMachine command.
for each smachine in lsMachine
spawn // start thread for each sMachine
SetMachine (sMachine)
// ... code executed on sMachine
Note that while the machine handle operator takes only a machine handle,
4Test implicitly casts the string form of the Agent machine’s name as a
machine handle and so the machine name is effectively the same as a
machine handle.
More about machine You can use the SetMachine function to change target machines for an entire
handle operator block of code. To specify the target machine for a single command, use the
machine handle operator on the command. There are two forms of this
operator: the arrow form (sMachine->) and the bracket form ([sMachine]).
For example, to execute the SYS_SetDir function on the target machine
specified by the sMachine1 variable, you do this:
sMachine1->SYS_SetDir (sDir)
You can use this syntax with a method call, for example:
sMachine-> TextEditor.Search.Find.Pick
But when invoking a method, this form of the machine handle must be the
first token in the statement. The following examples show valid and invalid
syntax:
You can also use this form of the machine handle operator with a function
that is not being used to return a value or with a method, as in the following
examples.
for each sMachine in lsMachine
[sMachine] FormatTest7 ()
Protecting access to When a new thread is spawned, 4Test creates a new copy of all local variables
global variables and function arguments for it to use. However, all threads have equal access
to global variables. To avoid a situation in which multiple threads modify a
variable “simultaneously,” you need to declare the variable as “shareable.” A
shareable variable is available to only one thread at a time.
Note Instances where threads modify variables simultaneously will
generate unpredictable results. Errors of this kind are very hard to
detect. Make variables shareable wherever the potential for conflict
exists.
void IncrementTestNum ()
access iTestNum
iTestNum = iTestNum + 1
The declaration of the semaphore is global; each thread contains the code to
acquire and release the semaphore. The initial value of three ensures that no
more than three threads will ever be executing the database access code
simultaneously.
How 4Test handles It is possible for a multi-threaded 4Test script to reach a state in which
script deadlock competing threads block one another, so that the script cannot continue. This
is called script deadlock.
When the 4Test runtime environment detects deadlock, it raises an exception
and halts the deadlocked script.
For example, the following script will never exit successfully:
share INTEGER iIndex1 = 0
share INTEGER iIndex2 = 0
main ()
parallel
access iIndex1
Sleep (1)
access iIndex2
Print ("Accessed iIndex1 and iIndex2")
access iIndex2
Sleep (1)
access iIndex1
Print ("Accessed iIndex2 and iIndex1")
• Elapsed time — sorts results for all threads and all machines in event
order. This enables you to see the complete set of results for a time
period and may give you a sense of the load on the server during that
time period or may indicate a performance problem.
• Machine —sorts results for all threads running on one machine and
presents the results in time-sorted order for that machine before reporting
on the next machine.
• Thread — sorts results for all tests run under one thread and presents the
results in time-sorted order for that thread before reporting on the next
thread.
The rest of this chapter describes and gives examples of the techniques you
use to direct test operations to one or more remote target applications and to
control whether those tests run serially or concurrently.
[Montana] script2.t
[Montana] script3.t
Note that any spaces between the target Agent’s name and the script name are
not significant.
Alternatively, to run testcases serially on multiple target machines, switch
among the target machines from within the script using 4Test’s built-in
Connect and Disconnect functions. For example, the following script
contains a function named DoSomeTesting that is called once for each
machine in a list of target machines, with the target Agent’s name as an
argument:
testcase TestSerially ()
STRING sMachine
// Define list of agent names
LIST OF STRING lsMachines = {...}
"Ohio"
"Montana"
3 Call the UpdateDatabase function once for each client machine. The
function sets the target machine to the specified client, then does a
database update. It creates a timer to time the operation on this client.
4 Disconnect from all target machines.
testcase TestClient_Server ()
Connect ("server")
Connect ("client1")
Connect ("client2")
DoSomeSetup ("server")
UpdateDatabase ("client1")
UpdateDatabase ("client2")
DisconnectAll ()
rendezvous // synchronize
Disconnect ("client1")
Disconnect ("client2")
DoSomeSetup ("server")
Disconnect ("server")
parallel
UpdateDatabase ("client1") // thread for client1
UpdateDatabase ("client2") // thread for client2
// automatic synchronization
Disconnect ("client1")
Disconnect ("client2")
RandSeed (3)
STRING sClientName
SelectTest ()
INTEGER i = RandInt (1, 12)
Test2 ()
// Do the test . . .
. . .
Test12 ()
// Do the test . . .
Topic Page
Multi-application startup and recovery 382
Testcase structure in a single-application environment 382
Testcase structure in a multi-application environment 383
Recording window declarations for remote machines 386
Code for template.t 386
template.t explained 387
Code for concurrency test example 389
Concurrency test explained 391
Code for notification example 1 395
Notification example 1 explained 396
Code for notification example 2 398
Notification example 2 explained 399
spawn
SetMachine (sMach1)
spawn
SetMachine (sMach2)
rendezvous
// . . .
template.t explained
The following line of code is the first required line in a multi-application
testcase file. It is the testcase declaration. Note that it does not pass an
application state as in the stand-alone environment.
multitestcase MyTest (STRING sMach1, STRING sMach2)
The next two code lines are the first required lines in the testcase:
SetUpMachine (sMach1, MyFirstApp, "MyFirstAppState")
SetUpMachine (sMach2, My2ndApp, "My2ndAppState")
You must execute the SetUpMachine function for every client machine that
will be tested. For each SetUpMachine call, you specify the application to be
tested (by passing the name of the main window) and the state to which you
want the application to be set (by passing the name of the application state if
you defined one).
The SetUpMachine function issues a Connect call for a machine you want to
test and then configures either the base state or a specified application state.
It does this as follows:
• It associates the client application’s main window name with the
specified machine so that the DefaultBaseState function can
subsequently retrieve it to set the base state.
• It associates the name of the application’s base state (if one is specified)
with the specified machine so that the SetMultiAppStates function can
subsequently retrieve it and set the application to that state at the start of
the testcase.
The first argument for SetUpMachine is the machine name of one of your
client machines. The second argument is the name you supply in your main
window declaration in your test frame (frame.inc) file. For this example, the
frame.inc file specifies the following:
window MainWin MyFirstApp
rendezvous
// . . .
The second spawn statement starts the thread for the second machine in this
template. The rendezvous statement blocks the execution of the calling
thread until all threads spawned have completed. You can use the rendezvous
statement to synchronize machines as necessary before continuing with the
testcase.
SetMultiAppStates ()
SetMachine (sMachine)
/*** ATTEMPT TO HAVE EACH MACHINE SAVE THE EMPLOYEE RECORD ***/
SetMachine (sMachine)
Employee.OK.Click ()
if (MessageBox.Exists (ACCEPT_TIMEOUT))
SetMachineData (NULL, "sMessage",
MessageBox.Message.GetText ())
MessageBox.OK.Click ()
Employee.Cancel.Click ()
else if (Employee.Exists ())
AppError ("Employee dialog not
dismissed after {ACCEPT_TIMEOUT} seconds")
rendezvous
iSucceed = 0
The first line of the testcase file defines the timeout value (in seconds) to be
used while waiting for a window to appear.
multitestcase MyTest (LIST OF STRING lsMachine)
with all currently running threads. In this example, a thread is started for each
machine in the list of machines being tested. 4Test sends the statements in the
indented code block to the Agents on each machine and then waits at the
rendezvous statement until all Agents report that all the code statements have
been executed.
The following is the code defined for the spawn statement:
// The code to be executed in parallel by
// all machines:
SetMachine (sMachine)
Employee.Salary.SetText
[STRING] RandInt (50000, 70000))
Each thread executes operations that prepare for an attempt to perform
concurrent writes to the same database record. The SetMachine function
establishes the Agent that is to execute the code in this thread. The next two
statements drive the application’s user interface to select John Doe’s record
from the employee list (a PopupList) and then to pick the Edit option from
the Employee menu. This opens the Employee dialog box and displays John
Doe’s employee record. The last thread operation sets the salary field in this
dialog box to a random number. At this point the client is prepared to attempt
a write to John Doe’s employee record. When this point has been reached by
all clients, the rendezvous statement is satisfied, as described above, and
4Test can continue with the next script statement.
for each sMachine in lsMachine
spawn
SetMachine (sMachine)
Employee.OK.Click ()
if (MessageBox.Exists (ACCEPT_TIMEOUT))
SetMachineData (NULL, "sMessage",
MessageBox.Message.GetText ())
MessageBox.OK.Click ()
Employee.Cancel.Click ()
Now that all the clients are ready to write to the database, the script creates a
thread for each client, in which each attempts to save the same employee
record at the same time. There is only one operation for each Agent to
execute: Employee.OK.Click, which clicks the OK button to commit the edit
performed in the previous thread.
The test expects the application to report the concurrency conflict with
message boxes for all but one client and for that client to close its dialog box
within 15 seconds. The if-else construct saves the text of the message in the
error message box by means of the SetMachineData function. It then closes
the message box and the Employee window so that the recovery system will
not report that it had to close windows. This is good practice because it
means fewer messages to interpret in the results file.
The “else if” section of the if-else checks to see whether the Employee
window remains open (presumably because it is held by a deadlock
condition); this is a testcase failure. In this case, the AppError function places
the string “***ERROR:” in front of the descriptive error message and raises
an exception; all Agents terminate their threads and the testcase exits.
iSucceed = 0
use "ccmail.inc"
LogMeIn()
LogInUser(GetMachineData( NULL, "Username" ),
GetMachineData( NULL, "Password" ) )
//------------------------------------------------------------
SetMultiAppStates()
CcMail.xWindow.GoToInbox.Pick ()
Verify( CcMail.Message.DeleteMessage.IsEnabled(), TRUE,
"MESSAGE WAITING" )
Note This script continues in “Code for notification example 2” on
page 398.
The next line is an include statement. The explanations for each fragment of
code follow that code.
use "ccmail.inc"
The ccmail.inc file is defined for this testcase. It contains the window
declarations for the application plus application state definitions and
definitions for general purpose utility functions also needed by other
testcases designed for this application. You can find the ccmail.inc file in the
SilkTest Examples directory. Code fragments from that file are included as
needed in this discussion.
LogMeIn()
LogInUser(GetMachineData(NULL, "Username"),
GetMachineData(NULL, "Password") )
The utility function LogMeIn is called by the Invoke method for the CC Mail
main window, called CcMail. The LogInUser function is defined in
ccmail.inc. The machine data that LogInUser retrieves for its arguments gets
established by each test before the application state function for each
machine is invoked.
multitestcase SingleUserNotification (STRING sMachine1)
The function declaration for the testcase passes in the name of the SilkTest
Agent for the machine on which the application is running.
if(sMachine1 == NULL)
sMachine1 = "(local)"
This if statement allows you to invoke the testcase without specifying a
machine name when you want to run on the local machine.
SetUpMachine(sMachine1, CcMail, "EnsureInBoxIsEmpty")
This is a single-user testcase and therefore does not actually need the setup
methodology required by a multi-application test. However, this chapter
describes client/server testing, which is frequently multi-application testing,
and therefore all the example testcases use the multi-application coding
methods. We recommend that you also follow this practice, since consistency
of testing styles reduces coding errors in your testcases.
One difference for this testcase is that this is an application that requires the
user to log in. Therefore the following code fragment provides the user name
and password for the application under test:
SetMachineData (sMachine1, "Username", "QAtest1")
SetMachineData (sMachine1, "Password", "QAtest1")
These statements associate two pieces of information, named “Username”
and “Password,” with the specified machine. In both cases the value of the
associated information is the same, “QAtest1.” Now that this information is
available to the application state function, that function can log the user in.
This will happen as a result of the next statement.
SetMultiAppStates()
In this test, SetMultiAppStates will actually only set the application state for
the one machine.
SimpleMessage ("QAtest1", "Message to myself",
"A message to myself")
The above line invokes the following utility function from ccmail.inc, which
sends the short message to the local machine:
Utility function void SimpleMessage (STRING sRecipient, STRING sSubject,
STRING sBody)
CcMail.Message.NewMessage.Pick()
NewMessage.MailingLabel.Recipient.SetText (sRecipient)
NewMessage.MailingLabel.Recipient.TypeKeys ("<Enter>")
NewMessage.MailingLabel.Recipient.TypeKeys ("<Enter>")
NewMessage.MailingLabel.SubjectField.SetText (sSubject)
NewMessage.MailingLabel.SubjectField.TypeKeys ("<Enter>")
NewMessage.EditBody.Body.TypeKeys (sBody)
NewMessage.EditBody.Body.TypeKeys ("<Ctrl-s>")
This function uses standard methods on Ccmail window components (defined
in ccmail.inc) to do the following:
1 Pick the NewMessage item from the Message menu.
2 Enter the string in argument one into the Recipient field and press the
Enter key twice to move to the Subject field.
3 Enter the string in argument two into the Subject field and press Enter to
move to the message body portion of the window (EditBody.Body).
4 Type the string in argument three into the Body field and type <Ctrl-s> to
send the message.
The next block of code verifies the results of the test.
Verify(CcMailNewMailAlert.Exists(NOTIFICATION_TIMEOUT),
TRUE )
Verify(CcMailNewMailAlert.IsActive(), TRUE, "ALERT")
CcMailNewMailAlert.OK.Click()
CcMail.xWindow.GoToInbox.Pick ()
Verify(CcMail.Message.DeleteMessage.IsEnabled(), TRUE,
"MESSAGE WAITING")
The above code does the following:
1 Verifies that the sent message was received (as indicated by the
NewMailAlert message box). The NOTIFICATION_TIMEOUT value
causes the Verify function to wait for that period of time for the window
to exist. If the timeout value is reached, the Verify raises an exception.
2 Verify that the dialog box CcMailNewMailAlert is active.
3 If the Verify executes without an exception, click on the OK button in the
CcMailNewMailAlert dialog box.
4 Pick the GoToInbox menu item from the Window menu.
5 Verify that a message exists in the Inbox by checking to see that the
Message menu has its DeleteMessage menu item enabled. If the menu
item is not enabled, there is no message in the Inbox and the Verify
function raises an exception.
SetMultiAppStates()
//---------------------------------------------------------
// Switch to the second machine:
SetMachine( sMachine2 )
Chapter Page
Chapter 23, “Using the Windows Bitmap Tool” 403
Chapter 24, “Calling Windows DLLs From 4Test Scripts” 425
Chapter 25, “Using PVCS with SilkTest” 431
Tool
What you will learn This chapter contains the following sections:
Topic Page
Introduction to the Windows Bitmap Tool 403
Windows Bitmap Tool window 405
Starting the Windows Bitmap Tool 408
Capturing bitmaps 410
Comparing bitmaps 414
Creating and applying masks 419
the Windows Bitmap Tool, where you can compare them using the tool’s
special comparison features. Conversely, bitmaps captured by the bitmap tool
can be compared by SilkTest’s bitmap functions.
When to use the You might want to use the Windows Bitmap Tool in these particular
Windows Bitmap Tool situations:
• To compare a baseline bitmap against a bitmap generated during testing
• To compare two bitmaps from a failed test
For example, suppose during your first round of testing you create a bitmap
using one of SilkTest’s built-in bitmap functions, CaptureBitmap. Assume
that a second round of testing generates another bitmap, which your test
script compares to the first. If the testcase fails, SilkTest raises an exception
but cannot specifically identify for you the ways in which the two images
differ. At this point you can open the Windows Bitmap Tool from the results
file to inspect both bitmaps.
Features for The Windows Bitmap Tool can create and graphically locate the differences
comparing bitmaps between two bitmaps. You can use all Windows functionality to resize, save,
and otherwise manipulate bitmaps, in addition to the special comparison
features included in the tool. Using Windows Bitmap Tool, you can
• Show the areas of difference
• Zoom in on the differences
• Jump from one zoomed difference to the next
• View on-line statistics about the bitmaps
• Edit (copy and paste), print, and save bitmaps
• Create masks
Masks are screen images that are used to exclude bitmap differences or any
part of a bitmap from comparison by the bitmap tool. You might consider
masking any differences that you decide are insignificant or that you know
will vary in an effort to avoid testcase failure. For example, suppose a testcase
fails because one bitmap includes a flashing area of a dialog. In the bitmap
tool you can block the flashing area from the two bitmaps by creating and
applying a mask to them. Once a mask is applied and the masked bitmaps are
saved, the mask becomes a permanent part of the baseline bitmaps you are
comparing. Masks can also be saved in separate files and used in SilkTest
testcases.
Bitmap Tool menus This section documents the menus that are unique to the Windows Bitmap
Tool — Edit, Bitmap, Differences, and Capture.
Edit menu
Menu item Description
Copy Copies the selected region to the clipboard.
Paste Inserts the contents of the clipboard into the bitmap tool.
New Mask Creates a child window into which you can place a
monochrome (black-and-white) mask bitmap.
Apply Mask Applies the current mask bitmap to the baseline and result
bitmaps in memory (not on disk).
Bitmap menu
Menu item Description
Set Baseline Toggles the bitmap in the active window as the baseline
bitmap.
Set Result Toggles the bitmap in the active window as the result
bitmap.
Differences menu
Menu item Description
Show Creates a Differences window that contains a monochrome
bitmap of the differences between the current baseline
bitmap and the current result bitmap. Black represents areas
with no differences; white represents areas with differences.
Zoom Available only when the Differences window is open. Fits
and stacks the Baseline, Differences, and Result windows,
and creates a special (not sizable) Zoom window with three
panes. When you place the mouse cursor at a given location
in the Baseline Bitmap, Differences, or Result Bitmap
window
• The top pane of the Zoom window shows the same
portion of the Baseline Bitmap window
• The middle pane shows the same zoomed portion of the
Differences window
• The bottom pane shows the same zoomed portion of the
Result Bitmap window.
Moving the mouse within one of the three windows
generates a simultaneous and synchronized real-time
display in all three panes of the Zoom window. The zoom
ratio is 4 to 1.
Scan Available only when the Zoom window is open. Starts
automated zoom mode, in which the bitmap tool scans for
differences from left to right and top to bottom. When the
tool finds the first difference, it displays a square in the
same relative location of the Baseline Bitmap, Differences,
and Result Bitmap windows. In addition, that location is
shown in all three panes of the Zoom window.
Selecting Scan while the tool is in scan mode exits Scan and
returns to zoom mode.
Capture menu
Menu item Description
Window Captures window that you select.
Client Area Captures client area of window you select.
Rectangle Captures rectangular area of choice. Move the mouse cursor
to desired location to begin capture. As you press and hold
the left button, drag the mouse to the screen location to end
capture and then release the button.
Desktop Captures entire desktop.
Hide Window on Hides bitmap tool window temporarily when you select a
Capture window or area to be captured. By default, menu item is
checked (selected). Select the item again to deselect it.
Use before beginning capture procedure.
To Action
Open an existing bitmap Select File/Open and specify a file in the Open
file dialog. See “Comparing bitmaps” on page 414.
Capture a new bitmap See “Capturing bitmaps” on page 410.
From the Run dialog To start the Windows Bitmap Tool from the Run dialog, specify the full path
of the tool’s executable file, bitview.exe, which is in the SilkTest installation
directory.
Optionally, you can also specify parameters for opening bitmaps files. For
example, if you want to open two bitmap files to be compared as well as an
associated mask bitmap file, you might type:
install-dir\bitview.exe bitmap1.bmp bitmap2.bmp mask.msk
Procedure To start the Windows Bitmap Tool from the Run dialog:
1 Select Run from the Start menu.
The Run dialog appears.
2 Type the pathname of the tool’s executable file and any parameters in the
Command Line text field and click OK.
The Windows Bitmap Tool starts. Any bitmaps you specified on the
command line are opened. Go to “Comparing bitmaps” on page 414.
If you didn’t specify any files in the command line, go to step 3.
3 You can now open existing bitmaps created in SilkTest or in the tool, or
you can capture new bitmaps:
To Action
Open an existing Select File/Open and specify a file in the Open dialog.
bitmap file Go to “Comparing bitmaps” on page 414.
Capture new Go to “Capturing bitmaps” on page 410.
bitmaps
From SilkTest results When the verification of a bitmap fails in a testcase, SilkTest saves the actual
file result in a bitmap file with the same name as the baseline bitmap but with the
extension .rmp. So, if the bitmap file testbase.bmp fails the comparison,
SilkTest names the result bitmap file testbase.rmp. It also logs an error
message in the results file, as shown in the following figure.
Note In some cases this error message does not reflect an actual
error. In particular, when SilkTest compares a bitmap it captured
with one captured in the bitmap tool, the comparison fails because
SilkTest stores footer information in its bitmap. The bitmaps might
in fact be identical in all ways except for this information.
As shown in the preceding figure, SilkTest opens the bitmap tool, opens both
baseline (expected, .bmp file) and result (actual, .rmp file) bitmaps, creates a
Differences window and places it in between the baseline and result bitmaps.
To the right, the tool displays a three-paned Zoom window. (The Differences
and the Zoom windows are described in “Zooming in on the differences” on
page 416.)
Capturing bitmaps
Two ways to capture You can capture bitmaps in two ways:
bitmaps
• Windows Bitmap Tool
• SilkTest’s bitmap functions and methods embedded in testcases
This section explains how to capture bitmaps in the Windows Bitmap Tool
and how to save them.
Capturing bitmaps in You use the Capture menu in the Windows Bitmap Tool to capture a bitmap
the Windows Bitmap for any of the following in your application:
Tool
• A window
To capture Action
Window Select Capture/Window. Click on the window you
want to capture.
Client area Select Capture/Client Area. Click on the client
area you want to capture.
Selected rectangular Select Capture/Rectangle. Move the mouse cursor
area to desired location to begin capture. While
pressing and holding the left mouse button, drag
the mouse to outline a rectangle, and then release
the mouse button to capture it. During outlining,
the size of the rectangle is shown in pixels.
Desktop Select Capture/Desktop.
The bitmap tool creates a new MDI child window containing the newly
captured bitmap. The following figure shows a bitmap of the Font dialog.
Note that the title bar reads Bitmap - (Untitled) and the status line at
the bottom right of the window gives the dimensions of the bitmap
(height by width), and the number of colors.
Example You save baseline and result bitmaps of the Open dialog as
open.bmp and open.rmp. Alternatively, you might name them openbase.bmp
and openres.bmp, respectively.
The following table lists the file extensions supported by the Windows
Bitmap Tool. We recommend that you use .bmp for baseline bitmaps and
.rmp for result bitmaps.
If you are And you want the file name Then use this file
saving to be extension
Baseline bitmap Identical to the result bitmap’s .bmp
Result bitmap Identical to the baseline bitmap’s .rmp
If you are And you want the file name Then use this file
saving to be extension
Either baseline Unique .bmp or .dib (Device
or result bitmap Independent Bitmap)
Note SilkTest uses .rmp for bitmaps that are captured within a
testcase and fail verification.
Capturing bitmaps in You can compare a baseline bitmap captured in the bitmap tool with one
SilkTest captured in a SilkTest testcase of your application.
• If you write testcases by hand, you can use SilkTest’s built-in bitmap
functions.
• If you prefer to record testcases via Record/Testcase, the Verify Window
dialog allows you to record a bitmap-related verification statement.
Using bitmap CaptureBitmap, SYS_CompareBitmap, WaitBitmap, and VerifyBitmap are
functions built-in bitmap-related 4Test functions. In particular, VerifyBitmap is useful
for comparing a screen image during the execution of a testcase to a baseline
bitmap created in the bitmap tool. If the comparison fails, SilkTest saves the
actual bitmap in a file. In the following example, the code compares the
testcase bitmap (the baseline) against the bitmap of TestApp captured by
VerifyBitmap:
TestApp.VerifyBitmap ("c:\sample\testbase.bmp")
3 Enter a file name in the Bitmap File Name field. Use the Browse
pushbutton to help select a directory name.
4 Select a radio button: Entire Window, Client Area of Window, or Portion
of Window, and click OK.
Note The Portion of Window radio button is identical to Capture/
Rectangle in the bitmap tool. To capture a portion of the window,
move the mouse cursor to the desired location to begin capture.
While pressing and holding the left mouse button, drag the mouse to
outline a rectangle, and then release the mouse button to capture the
bitmap.
Note Keep in mind that during the capture of a bitmap, SilkTest
always adds a bitmap footer to the bitmap file on the disk. This
means that the physical size of the bitmap will be slightly bigger
than if you capture the bitmap in the bitmap tool. The bitmap footer
always contains the window tag for a given bitmap.
Comparing bitmaps
The Windows Bitmap Tool has four major comparison commands — Show,
Zoom, Scan, and Comparison Statistics. You can also use masks to alter the
tool’s comparison capabilities. Masks are described in “Creating and
applying masks” on page 419.
Baseline and Result To compare two bitmaps, you must designate one bitmap in the comparison
bitmaps as the baseline and the second bitmap as the result. While you may have
many bitmap files open in the tool, at any one time only one bitmap can be set
as the baseline and one as the result. If you want to set new baseline and
result bitmaps, you must first unset the current assignments.
These designations are temporary; you can at any point set and reset a bitmap
as a baseline, result, or neither.
Showing areas of The Show command creates a Differences window, which is a child window
difference containing a black-and-white bitmap. Black represents areas with no
differences and white represents areas with differences.
Zooming in on the The Zoom command creates a special (not sizable) Zoom window with three
differences panes and resizes and stacks the Baseline, Differences, and Result window.
The top pane of the Zoom window contains a zoomed portion of the Baseline
Bitmap window.The middle pane shows a zoomed portion of the Differences
window. The bottom pane shows a zoomed portion of the Result Bitmap
window. All three zoomed portions show the same part of the bitmap. When
you move the mouse within any of the three windows, the bitmap tool
generates a simultaneous and synchronized real-time display in all three
panes of the Zoom window.
.
Jumping from one The Scan command on the Differences menu automates zoom mode and
difference to the next causes the bitmap tool to scan for differences from left to right and top to
bottom. When the first difference is found, a small square (32 x 32 pixels) is
shown in the Baseline Bitmap, Result Bitmap, and Differences bitmap
windows in the same relative location. In addition, that location is shown in
all three panes in the Zoom window. Use Differences/Next to jump forward
and Differences/Previous to jump back.
Note You must first create a Differences window and a Zoom
window using Differences/Show and Differences/Zoom.
Capturing the Zoom window in scan mode While in scan mode, you can
capture the Zoom window to examine a specific bitmap difference.
To capture all or part of the Zoom window while in scan mode:
1 Make sure the Capture/Hide Window is unchecked. If necessary, select
the item to remove the check mark.
2 Select Next or Previous until the Zoom window contains the difference
you want to capture.
3 Press one of the following:
To capture Action
Entire Zoom window Press Ctrl+W and select the Zoom window.
Client area of Zoom Press Ctrl+A and select the Zoom window.
window
Selected area of Zoom Press Ctrl+R. Move the mouse cursor to desired
window location to begin capture. While pressing and
holding the left mouse button, drag the mouse to
the screen location to end capture, and release
the mouse button.
4 Optionally, you can fit the bitmap in its window, resize it, and save it.
Looking at statistics The Comparison Statistics command displays information about the baseline
and result bitmaps, with respect to width, height, colors, bits per pixel,
number of pixels, and the number and percentage of differences (in pixels).
How to create a mask To create a mask that excludes some differences or just selected areas, and
that includes selected apply it:
areas
1 Select Edit/New Mask.
The bitmap tool creates an empty Mask Bitmap child window that is the
same size as the baseline bitmap.
2 Using the Differences window to help you locate differences, place the
mouse cursor in the baseline bitmap window at the position where you
want to begin creating the mask. As you press and hold the left mouse
button, drag the mouse cursor to outline a rectangle. Then release the left
mouse button.
The following figure provides an example of a user-created mask for the
baseline bitmap of the Font dialog. Note that the rectangular outline in
the baseline map changes to a filled-in rectangle. The mask bitmap
window also contains a like-sized rectangle in the same relative location.
4Test Scripts
What you will learn This chapter explains how to call a DLL from within a 4Test script. The
chapter covers these topics:
Topic Page
Declaring a DLL function 425
Aliasing a DLL name 426
Using DLL support files installed with SilkTest 427
Passing arguments to DLL functions 428
dllname The name of the dll file that contains the functions you want to
call from your 4Test scripts.
prototype A function prototype of a DLL function you want to call.
Prototype syntax A function prototype has this form:
return-type func-name ( [arg-list] )
return-type The data type of the return value, if there is one.
func-name An identifier that specifies the name of the function.
arg-list A list of the arguments passed to the function, specified as
follows:
[pass-mode] data-type identifier
pass-mode Specifies whether the argument is passed into the function (in),
passed out of the function (out), or both (inout). If omitted, in
is the default.
To pass by value, make a function parameter an in parameter.
To pass by reference, use an out parameter if you only want to
set the parameter’s value; use an inout parameter if you want to
get the parameter’s value and have the function change the
value and pass the new value out.
data-type The data type of the argument.
identifier The name of the argument.
Limitation You can call DLL functions from 4Test scripts, but you cannot call member
functions in a DLL.
// 4Test declaration
type REC is record
...
STRING sName, size=32
...
Using SilkTest with PVCS allows you to manage your SilkTest test files with
a powerful software configuration management tool. You can retrieve and run
the latest version or any earlier version of a script, suite, include file, or
testplan at any time in a consistent, repeatable manner. You can keep track of
who makes a change, when, and why, and also ensure that files aren’t
inadvertently overwritten or lost.
Note This chapter assumes that you are familiar with PVCS. If not,
you should consult your PVCS documentation.
What you will learn This chapter contains the following sections:
Topic Page
Overview of using PVCS 431
Setting up PVCS with SilkTest 434
Using PVCS with SilkTest 434
Possible problems 440
ArchDir 1 Tests
vcs.cfg
vcs.cfg
ArchDir 2
(shared)
App2 tests
vcs.cfg
ArchDir 3
The SilkTest/PVCS Supplied with SilkTest is a default PVCS configuration file that has been
default configuration customized for archiving SilkTest test files. It is named pvcsqap.cfg and is
file installed in the Windows directory. This file is the source from which
SilkTest creates all your the local vcs.cfg files that manage your SilkTest
work files.
Before you can perform any PVCS operations under SilkTest on the files in a
working directory, a local vcs.cfg file must exist. The easiest way to generate
one and assign some corresponding archive directories is with the Archive
Setup feature in SilkTest (see “Setting up your archives” on page 435). All
the PVCS directives in pvcsqap.cfg will be copied to your new local
configuration file (vcs.cfg) in the working directory along with the archive
and reference directories you assign. These local vcs.cfg files are used in
conjunction with the PVCS master configuration file (master.cfg) and
override specific settings for handling SilkTest test files.
Modifying pvcsqap.cfg If you want to customize pvcsqap.cfg—for example, add or change some
directives—you should modify the file before you have SilkTest generate
your local vcs.cfg files from it. Changes you make to the pvcsqap.cfg file are
used only when SilkTest creates a new local configuration file.
2 Select the files you want to check out. You can check out multiple files at
once by highlighting them in the multiple-select list of files.
Checking in files
You check in a file to store a new version of a SilkTest file in a PVCS archive.
The archive keeps track of who made the changes, when they were done, and
any comments you want to enter about the changes that were made. To be
able to check a modified file back into an archive, it must have first been
checked out with a lock.
2 Select one or more files in the directory to be checked in. (If one of the
files you have selected doesn’t yet have an archive, see “Creating a new
file archive” below before proceeding.)
3 Click OK.
The following dialog is displayed so you can make comments and/or
assign a version label to the new revision of the file(s) being checked in.
Creating a new file If the archive for a work file does not exist, it is automatically created when
archive you check the work file in. The new archive is created in the first archive
directory in the list of archive directories for the current working directory if
it is allowed by the local and master PVCS configuration files. You can also
create the archive with PVCS Version Manager.
Tip If you want to create a new archive in an archive directory other
than the one listed first for the working directory, do the following
before checking the file in: Open the Archive Setup dialog and move
the directory where you want to store the new archive to the top of
the list. See “Setting up your archives” on page 435.
2 Select the work files and enter the label in the Version Label field. If the
label has already been given to an earlier revision of a file, the label will
be reassigned to the most recent version.
Viewing differences
Procedure To see the differences between the currently active file in
SilkTest and the last revision of its archive, select Options/PVCS Options/
View Differences.
The differences are put into a new file named arcdifxx.tmp in the current
working directory and opened in the editor.
Possible problems
This section lists some of the problems you might encounter using the PVCS
extension to SilkTest.
• If you get “PVCS ERROR: PVCS_E_CANT_OPEN_CONFIG_FILE,”
you do not have a pvcsqap.cfg file in your Windows directory.
• If you get “PVCS ERROR: PVCS_E_LAF...,” the PVCS file islv.ini in
the Windows directory has problems.
• Only one program can use the PVCS DLLs at a time. So if you are
running PVCS Version Manager, SilkTest might not be able to use the
PVCS extension.
Chapter Page
Chapter 26, “Command Line Syntax” 445
Chapter 27, “Menu Commands” 449
What you will learn This chapter explains the syntax of the partner command, which you can use
to start the SilkTest executable from the command line.
There are two ways to start the SilkTest executable program from the
command line:
• In the Run dialog by selecting Run from the Start menu.
• From the command-line prompt in a DOS window.
The options The following table lists all the options to the partner command.
Option Description
-m Specifies the target machine. The default is the current machine.
(On UNIX, get the machine name using the “hostname”
command.) Call the 4Test built-in function Connect to connect
to a different machine at runtime.
-r Must be the last option specified, followed only by the name of
a script (and, optionally, arguments that the script takes), a suite,
testplan, or link file. If you specify a link file, tells SilkTest to
resolve the link and attempt to open the link target. Otherwise,
tells SilkTest to run the specified script, suite, or testplan,
optionally passing args as arguments to a script file
-q Quits SilkTest after the script, suite, or testplan completes.
-query Specifies a query. Must be followed by the name of a saved
query. Tells SilkTest to perform an Include/Open All, then
Testplan/Mark By Named Query, then Run/Marked Tests.
-p Provided for use with a Windows shell program that is running
SilkTest as a batch task. The option enables another Windows
program to receive a message containing the number of errors
that resulted from the script(s) run. SilkTest broadcasts this
message using the Windows PostMessage function, with the
following arguments:
• hWnd = HWND_BROADCAST
• uiMsg = RegisterWindowMessage (mess)
• wParam = 0
• lParam = number of errors
To take advantage of the -p option, the shell program that runs
SilkTest should first register mess, and should look for mess
while SilkTest is running.
-resexport Tells SilkTest to export the most recent results sets to .rex files
automatically. Specifying -resexport has the same effect as if
each script run invokes the ResExportOnClose function during
its execution.
script.t/ The name of the SilkTest script, suite, testplan, or link file to
suite.s/ load, run, or open.
plan.pln/
link.lnk
Option Description
args Optional arguments to a script file. You can access the
arguments using the GetArgs function and use them with your
scripts.
For more information, see “Passing arguments to a script” on
page 147.
Examples
To load SilkTest, type:
partner
To load SilkTest on system “sys1” and run the test.t script, type:
partner -m sys1 -r test.t
To load SilkTest and run the test.t script with arguments, type:
partner -r test.t arg1 arg2
To load SilkTest, run tests.pln, and export the most recent results set from
tests.res to tests.rex, type
partner -q -resexport -r tests.pln
To load SilkTest and run the tests marked by the query named query3 in
tests.pln, type
partner -query query3 -r tests.pln
What you will learn This chapter provides reference information about the menus in SilkTest,
including the Include and Testplan menus of QA Organizer. Menus and menu
items are listed in alphabetical order, not in the order they appear on the
menu bar.
SilkTest and The following table describes each of the menus.
QA Organizer menus
Menu Use the commands on this menu to
Breakpoint Add or remove a breakpoint, which is a line of 4Test code
where execution stops so that you can check the status of
the application. A breakpoint appears as a large bullet in
the left margin. On color monitors, it appears in red. The
Breakpoint menu appears only in debugging mode.
Debug Find and fix runtime errors. The Debug menu appears only
in debugging mode.
Edit Cut, copy, delete, and paste 4Test statements; undo
changes; go to a specified line in a window; locate and
replace text strings; and locate errors in 4Test code
associated with a line in a results window or errors window.
You can also toggle Visual 4Test on and off.
File Open, save, close, and print files. You can also run suites
and run and debug 4Test scripts.
Help Learn about functions, methods, and properties; access
online help.
Include Open, lock, unlock, save, and close include files containing
subplans and master testplans.
Breakpoint menu
The Breakpoint menu contains the following commands:
• Add...
• Delete...
• Delete All
• Toggle
Note For complete information about debugging testcases, see
Chapter 7, “Using the Debugger”.
Breakpoint/Add...
Available only in debugging mode. Opens the Add Breakpoint dialog, which
allows you to add a breakpoint at any executable line of a function.
To set a breakpoint, double-click on a module from the Modules list box,
which displays all currently loaded scripts and include files. When you select
a module, the Function field displays all the functions and testcases in the
selected module. Select the function that you want to enter a breakpoint into.
SilkTest enters the function name in the Breakpoint field. Select OK to have
SilkTest set a breakpoint on the first line of that function. A breakpoint is
denoted as a large bullet.
Breakpoint/Delete...
Available only in debugging mode. Opens the Delete Breakpoint dialog,
which lets you delete one ore more breakpoints. The dialog lists all
breakpoints by file name and line number.
Breakpoint/Delete All
Available only in debugging mode. Deletes all breakpoints, first prompting
you for confirmation.
Breakpoint/Toggle
Available only in debugging mode. Adds a breakpoint on the current line, if
none exists, or deletes a breakpoint, if one exists, on the current line. A
breakpoint is denoted as a large bullet.
Debug menu
The Debug menu contains the following commands:
• Abort
• Continue
• Exit
• Finish Function
• Reset
• Run
• Step Into
• Step Over
Note For complete information about debugging testcases, see
Chapter 7, “Using the Debugger”.
Debug/Abort
Available only in debugging mode when you have executed scripts on a target
machine. Terminates execution of the script you are debugging.
Debug/Exit
Available only in debugging mode. Exits debugging mode.
Debug/Finish Function
Available only in debugging mode. Executes the script until the current
function returns.
Debug/Reset
Available only in debugging mode. Frees memory and all variables, and
clears the call stack of the script you are debugging.
Debug/Step Into
Available only in debugging mode and after using Debug/Run and execution
has stopped at a breakpoint. Executes the current line of 4Test code in the
active script or in a file called by the active script. If the current line has a
breakpoint, SilkTest executes the line. If the current line contains a function
call, control passes into the function; SilkTest stops at the first statement.
Debug/Step Over
Available only in debugging mode, and after using Debug/Run and execution
has stopped at a breakpoint. Executes the current line of 4Test code in the
active script or in a file called by the active script, without stepping into any
functions called by the current line. Control stops at the next statement.
Edit menu
The Edit menu contains the following commands:
• Copy
• Cut
• Delete
• Find...
• Find Error
• Find Next
• Find Next Difference
• Goto Line...
• Paste
• Redo
• Replace...
• Undo
• Visual 4Test
Edit/Copy
Copies the selected text to the clipboard. Use with Edit/Paste.
Edit/Cut
Deletes the selected text from the active SilkTest window and copies it to the
clipboard. Use with Edit/Paste.
Edit/Delete
Deletes the selected text. Unlike the Cut command, Delete does not copy text
to the clipboard.
Edit/Find...
Opens the Find dialog. SilkTest searches in the active window for the text
string you specify in the Find What text field. The default string is the last
string you searched for. You can also specify the direction of the search (by
default, down) and whether or not the search should be case sensitive (by
default, case insensitive).
If the search is successful, SilkTest highlights the first occurrence of the text
string, expanding outlined code, if necessary, to do so. Select Find Next to
have SilkTest locate the next occurrence of that text string. If the search is not
successful, SilkTest displays an informational message.
Edit/Find Error
Available in the active error window or results window. How this command
works depends on whether an error window or results window is active:
If this window is
active Then SilkTest...
An errors window or In errors window, scans downward from the cursor
script position until it finds a line referring to a script file. It
looks for that file among open editor windows (or
opens it if it is not found) and places the cursor on the
appropriate line of the script.
In script, places the cursor on the appropriate line.
A results window Positions the cursor to the next error message or
warning in the results window.
By default, Find Error finds both errors and warnings. If you want to skip
over warnings, deselect the Find Error Stops at Warning check box in the
Runtime Options dialog. See “Options/Runtime...” on page 484.
You can select Find Error with the accelerator key F4.
By pressing F4
when You can
An errors window or Step through each source line that contains an error. As
script is active you view the code, SilkTest displays the relevant error
message on the status line.
A results window is Cycle through each error message or warning in the
active results file
Edit/Find Next
Repeats the last find or replace operation (see Edit/Find... and Edit/
Replace...). This menu item changes to Replace Next after a replace
operation, or to Replace All again after a global replace operation.
Edit/Go to Line...
Opens the Go to Line dialog, which locates a specified line. In the outline
editor, SilkTest expands the outline before it locates the specified line
number.
Once you click OK, SilkTest positions the cursor at the beginning of the
specified line in the active window.
Edit/Paste
Inserts text that you have cut or copied to the clipboard into the active
SilkTest window at the current insertion point.
Note You cannot paste graphics into a SilkTest window.
Edit/Redo action
Reverses the undo you just accomplished through Edit/Undo. The Redo
menu item describes the undo. For example, after undoing a cutting action,
the Redo menu item becomes Redo Cut. When you select Edit/Redo action,
SilkTest performs the action again; in this example, it cuts some text again.
Edit/Replace...
Opens the Replace dialog. When you click the Replace pushbutton, SilkTest
replaces the string you specify in the Find What field with the string in the
Replace With field. The default string is the one you specified in the last
replacement operation. If there are no more occurrences of the text string to
replace, SilkTest displays an information message.
Edit/Undo action
Backs out of the most recent change you have made in the editor. When you
make a change in the editor, the Undo menu item describes the action. For
example, if you had just cut some text, the menu item would read Undo Cut.
When you select Edit/Undo action, SilkTest undoes the editing action, in this
example, the cutting action, and also changes the Redo menu item to describe
the undone action, for example, Redo Cut.
To reverse the undo you just did, use Edit/Redo.
Edit/Visual 4Test
If checked, starts Visual 4Test, an outline editor mode. All the commands on
the Outline menu become available. Default is checked.
To start the plain editor, select the menu item; the check mark is removed. To
have the plain editor start by default, change the Outline Editor by Default
field in the General Options dialog. See “Options/General...” on page 480.
Note Changing from the plain editor to Visual 4Test might not work
due to syntax errors.
File menu
The File menu contains the following commands:
• Check In
• Check Out
• Close
• Debug...
• Exit
• New...
• Open...
• Open Selection
• Page Setup
• Print...
• Printer Setup...
• Quit
• Run...
• Save
• Save As...
• Save All
• Save Object File
In addition, the File menu contains a dynamic menu item of the form n
operation file-name that allows you to repeat the most recent open, run, or
debugging operation on a file name.
File/Check In
Available if PVCS is installed only. Checks a file into PVCS. For more
information, see Chapter 25, “Using PVCS with SilkTest”.
File/Check Out
Available if PVCS is installed only. Checks a file out of PVCS. For more
information, see Chapter 25, “Using PVCS with SilkTest”.
File/Close
Closes the active window. If you made changes to the window since the last
time you saved the file, SilkTest prompts you to save the file.
File/Debug...
Opens the Debug dialog, from which you can choose a script to debug. Once
you click OK, SilkTest opens the selected file in debugging mode. Special
debugging commands are available. See the menu items on the Debug,
Breakpoint, and View menus.
For complete information about debugging testcases, see Chapter 7, “Using
the Debugger”.
File/Exit
Closes SilkTest. SilkTest prompts you to save any changes made to files since
the last time you saved them.
File/New...
Opens the New dialog, which you use to create a new file—a test frame,
testplan, 4Test script, 4Test include file, suite, or text file. If you are using
QA Organizer, you can click the wizard icon to invoke the QuickStart
Wizard.
When you select any of these file types except test frame, SilkTest opens a
new editing window. The title bar of the new editing window displays the file
type you’ve chosen and the string “untitled.” When you select test frame,
SilkTest displays the New Test Frame dialog, which allows you to create a
test frame file for an open application.
File/Open...
Opens the Open dialog, from which you select an existing file. If the
specified file is already open, SilkTest makes its window the active window,
rather than opening another copy of the file.
File/Print...
Opens the Print dialog, which you use to print all or part of the contents of the
active editor or results window.
To Action
Print the entire file Select All Lines in the Print Range group box.
or or
Print a range of lines Highlight the lines you want to print and select
Selected Lines Only in the Print Range group box.
Print a header and/or footer Type literal text, variables, or a combination in the
on each page Header and/or Footer fields:
• &f prints the file name
• &p prints the page number
Example: If you type the following in the header
field:
Script: &f
and the file name is testapp.t, the printed header
will be: Script: testapp.t
Print entire contents of file Check the Print Fully Expanded check box.
or or
Print the file as shown on Uncheck the Print Fully Expanded check box.
the screen
Adjust the margins Enter new values for the left, right, top, and
bottom margins in the Margins group box. Default
value is 0.50 inch.
Change the output Select the output resolution you want from the
resolution Print Quality list box.
Change the font family, Click the Font pushbutton. The Editor Font dialog
size, and style of the text appears. See page 479.
To Action
Reconfigure the printer Click the Setup pushbutton. The standard Printer
options Setup dialog appears.
File/Printer Setup...
Opens the Printer Setup dialog, which you use to select printer settings, such
as the printer, paper size and source, and page orientation of the output.
File/Run...
Opens the Run dialog, from which you select a script, suite, or testplan (if
available) to run.
Note SilkTest will run a script, suite, and testplan that has an
extension of .t, .s, or .pln, respectively.
File/Save
Writes the contents of the active window to disk.
If you open a file that is owned by someone else or that someone else
currently has open, SilkTest opens a view-only copy of the file and changes
the menu item File/Save to File/Save Object File. You cannot save the source
file (since it is locked), but you can select File/Save Object File to write a a
new object file. For more information, see “Using object files” on page 134.
File/Save As...
Opens the Save As dialog, which you use to save the contents of the active
window with the name you specify.
File/Save All
Writes to disk all the open files that have changed since the last time you
saved them.
Help menu
The Help menu contains the following commands:
• Help Topics
• Library Browser...
• Online User’s Guide...
• About...
Help/Help Topics...
Starts SilkTest online Help. For information on its contents, look at the topic
“About this Help”.
Help/Library Browser...
Starts the Library Browser, an online facility for quickly looking up
documentation for built-in and user-defined 4Test methods, properties, and
functions.
Note For information about using the Library Browser, see “The
Library Browser” on page 38. For information about modifying the
contents of the Library Browser, see Chapter 18, “Modifying the
Library Browser”.
Help/About ...
Opens a dialog that lists the releases of software that are currently running.
Include menu
The Include menu contains the following commands:
• Acquire Lock
• Close
• Close All
• Open
• Open All
• Release Lock
• Save
Include/Acquire Lock
When the cursor is positioned on an open, read-only testplan or subplan,
locks the subplan. The gray bar changes from gray to yellow, indicating that
you have read-write access to the file.
Include/Close
When the cursor is positioned on an open subplan in a testplan, closes the
subplan.
Include/Close All
Closes all open subplans in the testplan.
Include/Open
When the cursor is positioned on an include statement in a testplan, opens
the subplan in the testplan.
Include/Open All
Opens all subplans (indicated by include statements) within the active
testplan.
Include/Release Lock
When the cursor is positioned on a open, locked subplan or testplan, unlocks
the subplan or testplan. The margin bar for the testplan or subplan turns from
yellow to gray. QA Organizer prompts you to save any unsaved changes.
Include/Save
When the cursor is positioned on a subplan, writes the subplan to disk. When
the cursor is positioned on a master testplan, writes the testplan, but not the
subplans, to disk.
Note To save the master testplan and all subplans, use File/Save.
Options menu
The Options menu contains the following commands:
• Agent...
• Class Map...
• Close Options Set
• Editor Colors...
• Editor Font...
• Enable/Disable Wizard
• General...
Options/Agent...
Opens the Agent Options dialog, which allows you to set global options
relating to these categories: timing, verification, closing windows, bitmaps,
faults, compatibility, extensions, and other. Select the tab for a category to
display a panel of options for that category.
The following figure shows the Timing category.
Whenever you select an option from a panel, the dialog displays the 4Test
statement that represents the selected option. For example, if you select the
Window Timeout (Seconds) option in the Timing category,
the following 4Test code is displayed in a field at the bottom of the dialog.
The statement uses the SetOption method to operate on the Agent object.
OPT_WINDOW_TIMEOUT is the internal 4Test name of the Window Timeout
option, and 5 is the current, default value of the option, five seconds.
This code is useful if you want to set a local value for a given Agent. You
simply place the statement in the main routine of your script, making sure to
set the value you want.
• The 4Test Statement Which Sets the Option field displays the code. You
can either type it yourself or select the Copy to Clipboard pushbutton to
have SilkTest copy the statement to the clipboard.
Creating multiple sets of Agent options In some situations you might want
to have more than one set of Agent options; for example, if you’re working
on multiple projects, you might want to have a custom set of options for each
project. In this case, you need to save your options in a file other than the
default file, which is partner.ini. SilkTest saves your options automatically in
partner.ini whenever you exit the dialog. To save your custom options in
another file, see “Options/Save New Options Set...” on page 489.
Timing options Window Timeout (Seconds) Specifies the number of seconds SilkTest
waits for a window to appear and be in the correct state. If a window does not
appear in the correct state within the specified timeout, SilkTest raises an
exception. The correct state of the window depends on how you set the
options on the Verification tab, which determine whether SilkTest checks
whether a window is enabled, active, exposed, or unique.
The default is 5 seconds (unless you have enabled enhanced support for
Visual Basic, in which case the default is 20 seconds for 32-bit applications
and 10 seconds for 16-bit applications).
Keyboard Event Delay (Seconds) Specifies the delay used before each
keystroke in a script.
Default is 0 seconds.
Mouse Event Delay (Seconds) Specifies the delay used before each mouse
event in a script. The delay affects moving the mouse, pressing buttons, and
releasing buttons.
Default is 0 seconds.
App Ready Timeout (Seconds) Specifies the number of seconds that the
Agent waits for an application to become ready. If the application is not ready
within the specified timeout, SilkTest raises an exception. To require the
Agent to confirm the ready state of an application, check the Verify That an
Application is Ready (Requires an Extension) option on the Verification tab.
This option applies only if the application or extension knows how to
communicate to the Agent that it is ready. To find out whether the extension
has this capability, see the documentation that comes with the extension.
Default is 10 seconds.
Verify That Windows Are Active If checked, verifies that windows are
active before interacting with them. See the topic on active and enabled status
in the 4Test Language Reference for information about how this option
affects various SilkTest methods.
Default is checked.
Verify That Windows Are Enabled If checked, verifies that windows are
enabled before interacting with them. See the topic on active and enabled
status in the 4Test Language Reference for information about how this option
affects various SilkTest methods.
Default is checked.
Verify That Windows Are Exposed If checked, verifies that windows are
exposed (that is, not covered, obscured, or logically hidden by another
window) before interacting with them.
Default is checked.
Verify the Class Tag for Methods of Class Control If checked, verifies
that objects are of the specified type before interacting with them. This option
is unchecked and disabled for SilkTest.
Don't Group Radio Buttons into a List If unchecked, treats radio buttons
as a group of class RadioList. Otherwise, uses the Release 1.x method of
treating radio buttons as individual objects.
Default is unchecked.
Use Release 1.x Version of the Class Library If checked, uses QA Partner
Release 1 versions of GetChildren, GetClass, and GetParent. Otherwise uses
Release 2 and later versions.
Default is unchecked.
Use Release 1.x Window Tags If checked, generates and operates on tags
compatible with SilkTest releases earlier than QA Partner Release 2.
Otherwise, uses Release 2 or later tags.
Default is unchecked.
The Release 2 algorithm affects tags that use index numbers and some tags
that use captions. In general, Release 2 tags are more portable, while the
earlier algorithm generates more platform-dependent tags. Use this option
only if you must run old scripts without any changes.
Add ‘Window Tags’ to the Record Menu If checked, includes the Record
Window Tags menu item on the Record menu. Selecting the Record Window
Tags menu item opens the Record Window Tags dialog. This dialog is
equivalent to the QA Partner Release 1.x Paste Window dialog, allowing you
to paste Release 1.x style window tags into your script.
Default is unchecked.
Other options Tolerance to Use When Sizing Windows (Pixels) Specifies an integer
representing the number of pixels allowed for a tolerance when a resized
window does not end at the specified size.
For some windows and GUIs, you can’t always resize the window to the
particular size specified. If the ending size is not exactly what was specified
and the difference between the expected and actual sizes is greater than the
tolerance, SilkTest raises an exception.
Windows cannot be sized smaller than will fit comfortably with the title bar.
Default is 0 pixels.
Show Windows Which Are out of View If checked, allows controls which
are not currently scrolled into view to be seen by the Agent. This option is
useful for testing Web applications. If unchecked, controls not currently in
view are invisible.
Default is checked.
Options/Extensions...
Opens the Extensions dialog, which you use to enable extensions and fault
trapping for applications under test on your host machine. When you first
open the Extensions dialog, it displays the runtime environments for all
extensions that you have installed or that come with SilkTest. The
environments listed in the Extensions dialog are used for running the
applications you wish to test.
For example, if you install the Forté extension with SilkTest, the Extensions
dialog looks like this:
The environments that you will use for testing applications appear in the
Application column. The Forté runtime environment appears because you
installed it; the browser and Java runtime environments appear because
browser support and Java support come with SilkTest. By default, extensions
for testing applications in each runtime environment are disabled.
Application Displays the browser and runtime environments that the installation program
installed and recognizes automatically. If you installed all development
environments, the installation program automatically adds the following
runtime environments to the Extensions dialog.
For runtime environments that do not use standard names or for stand-alone
applications, you must add the name of the executable or DLL file to the
dialog yourself. For more information, see Chapter 3, “Enabling Extensions
for Applications Under Test”.
Primary Extension The Primary Extension field displays Enabled, Disabled, or None (for not
applicable), or names the primary extension to use for applications launched
in the runtime environment listed in the Application column. Click on an
entry in the field to activate a drop-down list box. Then click on the down
arrow to see the choices.
Other Extensions Check the boxes that represent the other extensions that are required for
testing the application that is launched in the runtime environment listed in
the Application column.
Example: For a Forté application that uses embedded ActiveX controls, you
would select the ActiveX check box in the row containing Forté application.
Java To enable Java support for an application running on the host machine,
check the Java check box. When Java is enabled, you may also click the Java
pushbutton to select Java options. See Options later in this section for a
description of Java options.
Fault Trap To enable fault trapping for an application on the host machine,
check the Fault Trap check box. You may also click the Fault Trap pushbutton
to turn on/off specific fault trapping options. For more information, see
“Enabling fault trapping” on page 217.
Options The pushbuttons in this group display options for different extensions.
Java Active only when Java support is enabled, the Java pushbutton opens
the Extension Options dialog. Select Redirect Java Console Output and enter
a file name to redirect output from the Java console to a local file where you
can more easily scroll and copy the text. For more information, see the online
Help on Testing Java Applications and Applets.
Fault trap Fault options determine the actions taken by SilkTest when a
general protection fault is encountered. Active only when fault trapping is
enabled (the Fault Trap check box is selected), the Fault Trap pushbutton
opens the Fault Trap Options dialog, as shown below for a Forté application:
The first four options, Fault Actions, set SilkTest’s behavior when a fault
occurs, as described in the following table:
The next five options, Fault Log Information, relate to logging fault
information, as described in the following table:
Fault Log
Information Description Default
Call Stack If checked, logs the current calling stack. checked
Not applicable to 32-bit environments.
Fault Log
Information Description Default
Registers If checked, logs registers. checked
Memory If checked, logs memory management checked
Information information.
System If checked, logs information about the checked
Information system.
Task List If checked, logs a list of all programs checked
currently running.
New Click the New pushbutton to add one or more runtime environments to the
dialog for testing applications. For more information, see Chapter 3,
“Enabling Extensions for Applications Under Test”.
Duplicate Active only when you select a runtime environment in the Application
column that you entered manually. (You cannot duplicate runtime
environments that are installed by default.) Click the Duplicate pushbutton to
add a new application that has all the same settings (primary and other
extensions, fault trapping, and other options) as the selected application, but
with a different name. For more information, see Chapter 3, “Enabling
Extensions for Applications Under Test”.
Remove Active only when you select a runtime environment in the Application
column that you entered manually. (You cannot remove runtime
environments that are installed by default.) Click the Remove pushbutton to
delete an runtime environment from the dialog. For more information, see
Chapter 3, “Enabling Extensions for Applications Under Test”.
Details Active only when you select a runtime environment with an enabled Primary
Extension. Click the Details pushbutton to see detailed information about the
runtime environment, including version, help text, and executable module(s).
For example, here is the Extension details dialog for Forté extension.
Options/Class Map...
Opens the Class Map dialog, which you use to map custom classes to
standard classes supported by SilkTest. By establishing a class mapping, you
can use the methods and properties of existing standard classes on the
appropriate custom classes. You can also display the Class Map dialog while
you are recording declarations by clicking the Class Map pushbutton in the
Record Window Declarations dialog.
Note For more information about mapping custom classes to
standard classes, see “Mapping custom classes to standard classes”
on page 95.
Options/Editor Colors...
Opens the Editor Colors dialog, shown in the following illustration, which
you use to set the screen colors for various elements of 4Test code, results
information, and the testplan (if available). You can select any of the screen
colors provided or create your own by modifying the RGB values of these
colors. Modify the values by using the slider bar to change the relative value
of red, green, or blue, or enter specific values in the field next to each color.
When you are satisfied, click OK. If you want to revert to the default colors,
click Reset.
Editor Item Select an item from the list box. You can set colors for the
following categories:
• 4Test code
• Results information
• Testplans
• Difference Viewer
Editor Item Color Select one of the 16 colors from the palette for the
selected editor item. If you are satisfied with this color, click OK. Otherwise,
you can modify the RGB values of the selected colors by sliding the bar to
the left or right until you get the color you want or by entering specific RGB
values.
Options/Editor Font...
Opens the Editor Font dialog, shown below, which allows you to select a
screen font in the family, size, and style of your choice. As you make your
selections, the Sample field is updated.
When you are satisfied, click OK. SilkTest changes the font family, size, and
style for all open windows.
Font Select a font family from the available fonts displayed in the list box.
The default is Courier.
Font Style Select a style from those that are available for the selected font,
for example, regular, italic, bold, and bold italic.
Default is regular.
Size Select a point size from those that are available for the selected font.
Default is 9.
Sample This read-only area displays a line of text in the font family, style,
and point size currently selected and changes dynamically as you make
selections from the list boxes in the dialog.
Options/Enable/Disable Wizard
Available with QA Organizer only. Enables or disables the QuickStart
Wizard, which guides you through the creation of test frames, testplans, and
testcases.
For more information, see “Getting started” on page 19.
Options/General...
Opens the General Options dialog, which you use to set such aspects of
general system behavior as the editor and your workspace.
.
Editor Save Files Before Running If checked, writes all open modified files to disk
before executing a script, suite, or testplan. If unchecked, only writes the
individual script, suite, or testplan to disk before running it if it has been
modified.
Default is checked.
Create Backups If checked, SilkTest creates a backup file each time you
save a file. The backup file has an underscore ( _ ) appended to the extension.
Default is checked.
Visual 4Test by Default If checked, SilkTest starts the outline editor when
you open a script or include file. Otherwise, SilkTest opens the file in the
plain editor.
Default is checked.
File History Size Specifies the number of recent file actions, from 0 to 9, to
display on the File menu. See “File/n operation file-name” on page 459.
Default is 4.
4Test Include File Extensions Specifies the default extensions that SilkTest
supports for include files. Files with other extensions will be treated like text
files. They are displayed in the plain editor only.
Defaults are inc and lib.
Testplan Data File for Attributes and Queries Specifies the default path for the
testplan initialization file. The file extension is arbitrary. Available only if you
have installed QA Organizer.
Default is testplan.ini.
Help Help Files for Library Browser Specifies the files that SilkTest uses to
compile its help file (4test.hlp) for the Library Browser. Separate all entries
with commas. Default is 4test.txt, which includes information for all the
built-in classes and functions.
For more information about the Library Browser, and how to update its
contents with information about your own classes and functions, see Chapter
18, “Modifying the Library Browser”.
Options/n option-file-name
Displays a list of from 1 to n file names. Each is a custom options file that
you have loaded during the current work session. File names are prefaced by
an integer, n, where the last-loaded options file is listed first. You can click on
a file name to have a new set of options in effect.
Alternatively, you can select a file name by its number; for example, to load
the options in the second file in the list, press Alt+O+2.
Options/Property Sets
Opens the Property Sets dialog, which allows you to create, modify, combine,
and delete property sets, which are used to verify properties in testcases.
For more information, see “Configuring your own property sets” on
page 121.
Options/PVCS Options
Available if PVCS is installed only. Displays cascading menu that allows you
to manage your SilkTest test files with PVCS. For more information, see
Chapter 25, “Using PVCS with SilkTest”.
Options/Recorder
Opens the Recorder Options dialog, shown below, which allows you to
specify settings that SilkTest uses when recording.
General Options Change Hotkey To Ctrl+Shift If checked, SilkTest waits for you to press
Ctrl+Shift for verifications, instead of the default Ctrl+Alt.
Default is unchecked.
Data File for Property Sets Specifies the file containing the definitions for
the property sets being used.
For more information about verification using properties and property sets,
see “Verifying using properties” on page 115.
Recorded Events Record Only Low Level Events If checked, specifies that you want to
record events at a lower level for selected controls.
Ignore Mouse Move Events If checked, SilkTest does not record mouse
movements when you select the Record Testcase and Record Actions
commands. If unchecked, SilkTest records mouse movements that cannot be
built into higher-level actions and that occur while a mouse button is pressed.
Typically, you leave this checked unless you are testing an application, such
as a drawing application, where mouse movements themselves are
significant.
Default is checked.
Options/Runtime...
Opens the Runtime Options dialog, shown below, which allows you to
specify settings that SilkTest uses when it runs a script.
4Test Agent Name Specifies the name of the Agent on the target machine, that is,
the machine on which you want to run a script or suite. Specify a value only if
you want to run your script on a machine other than the host machine.
Note If you don’t know the Agent name, click on the Agent icon on
the target machine.
Arguments Specifies the arguments, if any, that you want to pass to the
script at runtime. Separate multiple arguments with spaces.
For example, suppose your script takes two arguments: the number of
iterations to perform and the name of a test data file. In this case, you would
enter 5 test1.dat
For more information, see “Passing arguments to a script” on page 147.
Use Path Specifies one or more paths along which SilkTest searches for
include files. Include files can be named in the Use Files field or in a script’s
use statement. If you specify a path, SilkTest searches the current directory
and then each of the directories in the path named here. If you do not specify
a search path, SilkTest searches the current directory only.
Type one or more paths, separated by commas. The syntax for a path is the
same as that used by the native operating system.
Use Files Specifies the names of one or more include files for SilkTest to
automatically load at startup. Do not specify files in this field if you intend to
include a use statement for the files in a script.
Type the file name(s) as you would any other operating system path. Separate
file names with a comma.
Tip You can also click the Add button and select an include file to
add from a standard file selection dialog.
Objfile Path Specifies the location where SilkTest reads and writes object
files from.
• Leave the field empty if you want to store object files in the same
directory as their corresponding source files
• Specify an absolute path if you want to store all object files in the same
directory
• Specify a relative path if you want object files to be stored in a directory
relative to the directory containing the source files
For more information, see “Using object files” on page 134.
Gui Targets Specifies the platforms that you want to compile your scripts
and include files for (using conditional compilation). You can specify as
many GUI targets as you want; separate each GUI specifier by a comma.
For more information, see “Conditional compilation” on page 269.
Default Browser Specifies the Web browser to use when testing Web
applications.
Compiler constants... Opens the Compiler Constants dialog, where you can
define constants and assign them values. You can use the defined constants in
scripts and include files anywhere you can specify an expression. Constants
are evaluated and replaced with their values at compile time.
To define a constant, specify its name in the Constant Name field and its
value in the Value field, then click Add. You can edit or delete an existing
constant by selecting it and clicking Edit or Remove.
Y2K Rules... Opens the Y2K Date Transformation Rules dialog where you
can define rules that SilkTest should use to transform dates.
For more information, see “Testing applications that have been made Year
2000 compliant” on page 195.
Results Directory/File Specifies the name of the directory, or the name of the
directory and file that stores the results of script runs.
Note SilkTest always assigns the extension .res to all results files. If
you supply a different extension, SilkTest will override it.
If you leave the field empty (the default), SilkTest gives the results file the
same name as the script and stores it in the same directory as the script.
If you supply only a directory name, SilkTest gives the results file the same
name as the script and stores it in the directory you specify.
Write to Disk After Each Line If checked, writes the results file to disk
whenever the script generates output, as in the case of a print statement.
Selecting this option ensures that in the event of system failure, SilkTest will
produce a results file containing output up to the time of system failure. The
disadvantage of selecting this option is that file I/O slows down script
execution.
Default is checked.
Show Overall Summary If checked, displays the summary of results for the
entire script, suite, or testplan, including the start and elapsed time of
execution, and the total number of errors and warnings. For more information
on how this option is used, see “Results/Show Summary” on page 509 and
“Results/Hide Summary” on page 507.
Default is checked.
Log Elapsed Time, Thread, and Machine For Each Output Line If
checked, records this information in the results file for each line that is
written.
Default is unchecked.
Note To view this information, make the results file active, then
select Results/View Options and check Elapsed time, Thread
number, and Current machine.
Execution Minimize While Running If checked, SilkTest runs minimized while you
run a script, suite, or testplan.
Default is unchecked.
Print Tags With Agent Calls If checked and Print Agent Calls is also
checked, includes tags with the method calls in your results files.
Default is unchecked.
Outline menu
The Outline menu contains the following commands:
• Collapse
• Collapse All
• Comment Block
• Expand
• Expand All
• Move Left
• Move Right
• Transpose Down
• Transpose Up
• Uncomment Block
Outline/Collapse
When the current line or selected line(s) is preceded by a minus sign (-),
hides subitems.
Outline/Collapse All
Hides all subitems in the window.
Outline/Comment Block
Available when the active window is a 4Test script, include file, or plan.
Places a single-line comment (indicated by //) at the beginning of the current
line or selected lines.
Outline/Expand
When a plus sign (+) precedes the current line or selected lines, displays the
hidden subitems (children) of the current line or selected lines.
Outline/Expand All
Displays all subitems in the window.
Outline/Move Left
Moves the current line, selected lines, or the text cursor (if line is empty) to
the left. Repeat to achieve the desired outdent. If you shift the code to an
invalid position in a script or include file, SilkTest marks the invalid line(s)
with an X.
Outline/Move Right
Moves the current line, selected lines, or the text cursor (if line is empty) to
the right. Repeat to achieve the desired indent. If you shift the code to an
invalid position in a script or include file, SilkTest marks the invalid lines
with an X.
Outline/Transpose Down
Exchanges the current line or selected lines with the next line. Adjust the
indent or outdent using Edit/Move Right and Edit/Move Left.
Outline/Transpose Up
Exchanges the current line or selected lines with the previous line. Adjust the
indent or outdent using Edit/Move Right and Edit/Move Left.
Outline/Uncomment Block
Available when the active window is a 4Test script, include file, or plan.
Removes a single-line comment (indicated by //) from the current line or
selected lines.
Record menu
The Record menu contains the following commands:
• Actions...
• Application State...
• Class...
• Method...
• Testcase...
• Window Declarations...
• Window Identifiers...
• Window Locations...
Record/Actions...
Opens the Record Actions dialog, shown below, which records the actions
you perform to test an application, translates them into 4Test statements, and
displays them in the Recorded Actions window. In addition, you can add a
4Test verification statement or method for a GUI object.
Typically, you then insert (paste) the statements into a script file at the cursor
position, where you can edit them and write additional statements. If no file is
open, SilkTest inserts the actions into an untitled script window.
Recorded Actions The Recorded Actions list box displays your interactions with your
application as 4Test statements. Furthermore, the list box provides all the
editing capabilities of the SilkTest editor, so you can manipulate any actions
you record.
4Test statements have this format: fully-qualified-object-name.method.
After recording one or more actions, you might want to
• Add a verification statement or a method for a GUI object by positioning
the mouse cursor over the GUI object and pressing Ctrl+Alt.
• Halt recording to bring your application to another state before
continuing to record actions. (This avoids recording of extraneous
statements.) Press the Pause Recording pushbutton.
• Insert the statements in the Recorded Actions list box into a script file.
Press Paste to Editor, or Copy to Clipboard and related commands.
• Delete the statements from the list box, for example, because you
recorded the wrong actions. Press Clear All.
Keep on Top If checked, keeps the Record Actions dialog in the foreground even when you
are tracking the cursor over your application.
Default is checked.
Pause Recording/ Pause Recording/Resume Recording is a toggle pushbutton that suspends or
Resume Recording continues the recording of your interactions with your application. Typically,
you use Pause Recording to halt recording while you bring your application
to the correct state before beginning recording again. Typically, you use
Resume Recording after pasting recorded actions into the editor.
Ctrl+Alt (Ctrl+Shift) To add a verification statement or method for an object, position the mouse
cursor over the object and press Ctrl+Alt. (Press Ctrl+Shift if you changed
the hotkey in Recorder Options.)
For more information, see “Recording the verification stage” on page 112.
Paste to Editor The Paste to Editor pushbutton inserts the 4Test code in the Recorded Actions
list box into the active SilkTest editing window. The Record Actions dialog is
cleared. Each time you select Paste to Editor, SilkTest inserts the recorded
actions at the end of the same file.
Note If you use an editor other than that provided with SilkTest to
write your test scripts, use Copy to Clipboard to insert actions rather
than Paste to Editor.
Copy to Clipboard The Copy to Clipboard pushbutton copies the recorded actions in the list box
to the clipboard. The Record Actions dialog is cleared. Use Edit/Paste to
insert the actions into an editing window of your choice.
Clear All Empties the Recorded Actions window.
Close Closes the Record Actions dialog and redisplays the SilkTest editing window.
If you have not pasted your recorded actions to the editor, SilkTest prompts
you to do so.
Record/Application State...
Opens the Record Application State dialog, which allows you to define an
application state routine that SilkTest runs before it executes your testcase. If
the current window is not a script or include file, SilkTest prompts you to
create a new include file.
Application State Name Specifies the name of the application state you are
going to record. The application state name must be less than 64 characters.
Paste to Editor Closes the Record Application State dialog and places the
4Test code, if any, in the associated file.
Cancel Closes the Record Application State dialog without placing any code
in the associated file.
The presence of the Record Status window indicates that you can begin
recording an application state. The Status field flashes the word Recording.
In addition, the Record Status window lets you
• Temporarily pause recording, for example, to leave your desk or begin
another task, and then resume recording. Press Pause Recording and then
Resume Recording in the Record Status window.
• Add verification statements to your application state routine. Position the
mouse pointer over the GUI object you want to verify and press Ctrl+Alt.
The Record Status window shows the object that the mouse pointer is
over.
• See what you’ve recorded and then resume recording. Press Pause
Recording to review the Application State Code field in the dialog and
then press Resume Recording in the dialog.
• End recording. Press Done.
Pause Recording/ Pause Recording/Resume Recording is a toggle pushbutton that suspends or
Resume Recording continues the recording of your interactions with your application. Typically,
you use Pause Recording to halt recording while you bring your application
to the correct state before beginning recording again. Typically, you use
Resume Recording after pasting your recording into the editor.
Ctrl+Alt To add a verification statement or method, position the mouse cursor over the
object and press Ctrl+Alt. (Press Ctrl+Shift if you changed the hotkey in
Recorder Options.)
For more information, see “Recording the verification stage” on page 112.
Done When you have finished recording an application state or just want to see
what you have recorded, press the Done pushbutton on the Record Status
window. SilkTest redisplays the Record Application State dialog. The
Application State Code field contains all the 4Test code you’ve just recorded.
You can take the following actions:
If Then
All the information in the Click OK. SilkTest closes the Record
window is complete and what Application State dialog and places the new
you want. application state in your file.
The code is not what you want. Edit the code in the Application State Code
field.
The application state name is not Edit the name in the Application State
the one you want. Name field.
The application state on which Delete the code in the Application State
this application state is based is field, select a new application state from the
not the one you want. drop-down list, and click Resume
Recording to rerecord the application state.
The application state routine is Click Resume Recording. SilkTest opens
not yet finished. the Record Status window. You can continue
to record your application state.
Record/Class...
Records a new Visual Basic, ActiveX, or Java class declaration.
For more information, see the online Help.
Record/Method...
Available only when an include file or script is in the active window and the
insertion point is within a class or object declaration (methods are necessarily
part of a class or object definition). Opens the Record Method dialog, which
allows you to record a method for a class or window declaration.
Method Name Specifies the name of the method. You can add a new
method name or select a name from the drop-down list: BaseState, Close,
Invoke, or Dismiss.
Paste to Editor Closes the Record Method dialog and places the method in
the associated include file at the cursor position.
Cancel Closes the Record Method dialog without placing any code in the
associated include file.
Start Recording Closes the Record Method dialog and displays the Record
Status window. For more information about this dialog, see “Record/
Application State...” on page 493.
Pause Recording/ Pause Recording/Resume Recording is a toggle pushbutton that suspends or
Resume Recording continues the recording of your interactions with your application. Typically,
you use Pause Recording to halt recording while you bring your application
to the correct state before beginning recording again. Typically, you use
Resume Recording after pasting your recording into the editor.
Ctrl+Alt To add a verification statement or method, position the mouse cursor over the
object and press Ctrl+Alt.
For more information, see “Recording the verification stage” on page 112.
Done When you have finished recording a method or just want to see what you
have recorded, click Done in the Record Status window. SilkTest redisplays
the Record Method dialog. The Method Code field contains all the 4Test code
you’ve just recorded. You can take the following actions:
If Then
All the information in the Click OK. SilkTest closes the Record
window is correct and complete. Method dialog and places the new method
in your include file.
If Then
The code is not what you want. Edit the code in the Method Code field.
The method name is not what you Edit the name in the Method Name field.
want.
The method is not yet finished. Click Resume Recording. SilkTest opens
the Record Status window. You can continue
to record your method.
Record/Testcase...
Compiles the currently active script file or testplan, if not already compiled,
and opens the Record Testcase dialog. If the current window is not a script,
SilkTest prompts you to open a script file or create a new script file before
opening the dialog.
Record/Window Declarations...
Opens the Record Window Declarations dialog, which allows you to record
descriptions, called window declarations, of the GUI objects in your
application and insert them into a declarations file, called an include file.
SilkTest uses the declarations in the include file to identify the objects named
in your test scripts. The declaration is a combination of class, identifier, and
tag.
The Record Window Declarations dialog, shown below, is composed of a
Window Declaration list box, the Window Detail group box, the Tag
Information group box, a check box and several pushbuttons. As you move
the mouse pointer over your application, the contents of the dialog change to
reflect the object under the pointer. When you position the mouse pointer
over the object you want to declare and press Ctrl+Alt, the Record Window
Declarations dialog freezes the current contents, in effect, taking a snapshot
of the declaration. The following figure shows a snapshot of the declaration
for the Text Editor.
Window Detail The Window Detail group box (upper left) displays the class, identifier, and
tag of the object highlighted in the list box (bottom of the dialog). You can
edit the class and identifier of an object displayed in the list box by selecting
it. The Window Detail group box contains the following fields:
Class The class is the GUI object type and determines which methods can
operate on the object.
SilkTest assigns the class CustomWin to all custom objects. If the custom
object can be mapped to a standard SilkTest class, you should create the
mapping before you paste (insert) the declarations into an include file. Click
the Class Map... pushbutton to display the Class Map dialog, shown below, to
change the class of a custom object.
Tag A tag is the actual internal name SilkTest uses to refer to an object. By
default SilkTest constructs the tag from the object’s caption and Window ID,
but it can also be based on the object’s prior text, index, and location, as
shown in the Tag Information group box. To include other components in the
object’s tag, check the appropriate check boxes in the Tag Information group
box. The tag is updated in the Window Detail group box.
To change how SilkTest constructs the default tag, click the Options
pushbutton.
Tag Information The Tag Information group box (upper right) displays the components that
SilkTest can use to construct the tag for the object.
For more information about tags, see “About tags” on page 85.
Window Declaration The Window Declaration list box (across bottom of dialog) displays the
declarations you’ve recorded. It shows the class, identifier, and tag of the
recorded object and its child (subordinate) objects in the GUI hierarchy. The
declarations for child objects are embedded within the declaration of the
parent (superior) object and are shown indented in the list box. To modify the
declaration of an object displayed in the list box, select it. Its class, identifier,
and tag are displayed in the Window Detail group box, where you can edit
this information.
By default, custom objects of class CustomWin are shown in red on color
monitors (bold on black-and-white monitors). This highlighting serves as a
visual reminder for you to map the class of custom objects to a standard class
supported by SilkTest. Also, by default, custom objects of class Ignore are
not included in the declarations. To change these default behaviors, click the
Options pushbutton.
Resume Tracking The Resume Tracking pushbutton continues the recording of object
declarations. Typically, you use Resume Tracking after pasting declarations
into the editor. When you are ready to record another declaration, press
Resume Tracking.
Paste to Editor The Paste to Editor pushbutton inserts the “frozen” declarations in the
Window Declaration list box into an untitled file if the current window is not
a script or include file. The Record Window Declarations dialog is cleared.
Each time you select Paste to Editor, SilkTest inserts the declarations at the
end of the same file.
Note Use Copy to Clipboard instead of Paste to Editor if you want
to paste declarations elsewhere in the file, or if you use an editor
other than that provided with SilkTest to write your test scripts.
Copy to Clipboard The Copy to Clipboard pushbutton copies the declarations in the list box to
the clipboard. The Record Window Declarations dialog is cleared. Use Edit/
Paste to insert the declarations into a different editing window or to insert
them into the current window at the location of your choice.
Keep on Top If checked, the Keep on Top check box keeps the Record Window
Declarations dialog in the foreground even when you are tracking the cursor
over your application. Default is checked.
Close The Close pushbutton closes the Record Window Declarations dialog. If you
have pasted declarations into a file, SilkTest displays the editing window that
contains your declarations.
Options The Options pushbutton opens the Record Window Declarations Options
dialog, which lets you customize settings for the default tag, identifier, and
the recording window.
The Default MultiTags group box controls the default manner in which
SilkTest constructs the tag. The default is to construct the tag from the
object’s Caption and Window ID.
Note If you deselect Record Multiple Tags, the Default Multitags
group box becomes the Default Tag group box, and the check boxes
for the tag types become radio buttons, allowing you to choose only
one tag component.
Select one or more of the following choices (select only one if you deselected
Record Multiple Tags):
The Window Declaration Identifiers group box controls the default manner in
which SilkTest constructs the identifiers in the declaration.
The Window Declaration Viewer group box lets you determine how SilkTest
displays and declares objects of custom classes.
Record/Window Identifiers...
Opens the Record Window Identifiers dialog, shown below, which records a
4Test window identifier as a fully qualified GUI object name.
Window Identifier When you place the mouse cursor over a GUI object, the Window Identifier
text field displays the 4Test identifier for the object. The actual name
recorded by SilkTest depends on whether or not you have previously recorded
declarations for the GUI objects you want to test.
• If you have compiled the include file for the GUI objects, then a fully
qualified object name is composed of the identifier of the object and as
many of its ancestors as necessary to uniquely identify the object.
Example: The fully qualified name of the Open menu item is:
TextEditor.File.Open
The ancestor of the Open menu item is the File menu. The ancestor of the
File menu is the top-level GUI object, TextEditor, the application itself.
• If you have not previously compiled the include file for the GUI objects,
then SilkTest creates a fully qualified object name on the fly (called
dynamic instantiation) based on the class and tag of the object. For more
information, see “Recording without window declarations” on page 138.
When the text field displays the identifier you want to record, you can halt
tracking and insert (paste) it into a script file. Press Ctrl+Alt (Ctrl+Shift if
you changed the hotkey in Recorder Options) to freeze the contents of the
text field, and then press Paste to Editor, or Copy to Clipboard and related
commands, to insert the identifiers into a script.
Keep on Top If checked, always keeps the Record Window Identifiers dialog in the
foreground.
Default is checked.
Resume Tracking The Resume Tracking pushbutton continues the recording of identifiers.
Typically, you use Resume Tracking after pasting an recorded identifier into
the editor. When you are ready to record another identifier, select Resume
Tracking.
Paste to Editor The Paste to Editor pushbutton inserts the recorded identifier into the active
SilkTest editing window. The identifier is inserted at the cursor position. The
Record Window Identifiers dialog is cleared.
Note If you use an editor other than that provided with SilkTest to
write your test scripts, use Copy to Clipboard to insert identifiers
rather than Paste to Editor.
Copy to Clipboard The Copy to Clipboard pushbutton copies the identifier in the Window
Identifier list box to the clipboard. The window is cleared. Use Edit/Paste to
insert the identifiers into an editing window of your choice.
Close The Close pushbutton closes the Record Window Identifiers dialog.
Record/Window Locations...
Opens the Record Window Locations dialog, shown below, which you use to
record the x, y location of a graphical control, such as a tool bar. A location is
recorded relative to the screen, frame, and client window.
Location When you move the mouse cursor over the active application, the text field
displays the 4Test identifier of the GUI object tracked by the mouse. The
Screen, Frame, and Client radio buttons display the x, y location of the GUI
object relative to the screen, frame, and client window, respectively. Select
the location of your choice. Then press Ctrl+Alt (Ctrl+Shift if you changed
the hotkey in Recorder Options) to halt recording.
Paste to Editor The Paste to Editor pushbutton inserts the relative location of your choice
into the active SilkTest editing window. The location is inserted at the cursor
position. The Record Window Locations dialog is cleared.
Note If you use an editor other than that provided with SilkTest to
write your test scripts, use Copy to Clipboard to insert locations
rather than Paste to Editor.
Copy to Clipboard The Copy to Clipboard pushbutton copies the location of the object named in
the text field to the clipboard. The window is cleared. Use Edit/Paste to insert
the location into an editing window of your choice.
Keep on Top If checked, the Keep on Top check box keeps the Record Window Locations
dialog in the foreground even when you are tracking the cursor over your
application.
Default is checked.
Resume Tracking The Resume Tracking pushbutton continues the recording of locations.
Typically, you use Resume Tracking after pasting a location into the editor.
When you are ready to record another location, select Resume Tracking.
When you are ready to record another location, select Resume Tracking. The
label changes to Pause Tracking,
Close The Close pushbutton closes the Record Window Locations dialog.
Results menu
The Results menu contains the following commands:
• Close
• Compact
• Compare Two Results...
• Convert to Plan...
• Delete...
• Export...
• Extract...
• Find Next Difference
• Goto Source
• Hide Summary
• Mark Failures in Plan
• Merge...
• Pass/Fail Report...
• Select...
• Send to QA Radar...
• Show Summary
• Update Expected Value
• View Differences...
• View Options...
Results/Compact
Available only when the active window is a results window. Removes unused
space in a results file, thereby reducing the file size.
Results/Convert to Plan...
Available only when the active window is a results window and you have run
a script, a suite, or a single testcase from a script. Opens the Convert Results
to Plan dialog, which allows you to transform a results file into a testplan.
For more information, see “Converting the data in a results file into a plan”
on page 233.
Results/Delete...
Available only when the active window is a results window. Opens the Delete
Results dialog. When you select a set of results and click OK, SilkTest
deletes it.
Results/Export...
Available only when the active window is a results window. Opens the Export
Results dialog, which you can use to export your results to a structured file
that is suitable for further processing by an application such as a spreadsheet.
For more information, see “Exporting structured information” on page 158.
Results/Extract...
Available only when the active window is a results window. Opens the
Extract Results dialog, which allows you to place selected information from a
results file in ASCII format into a new editor window or a file or send the
information to a printer.
For more information, see “Storing results in an unstructured ASCII format”
on page 156.
Results/Goto Source
Available only when the active window is a results window. Opens a script
file associated with the current results file, if it is closed, and makes it the
active window. If the cursor was positioned at an error message in the results
file, SilkTest positions the cursor at the error line in the script. If the cursor
was positioned at the results for a particular testcase, SilkTest positions the
cursor at the beginning of the testcase in the script file.
Results/Hide Summary
Available only when the active window is a results window. Hides the display
of results summary of the current suite, script, testplan, or testcase. To have
the summary hidden by default, see “Show Overall Summary” on page 488.
Results/Merge...
Available only when the active window is a QA Organizer results window.
Opens the Merge Results dialog, which you use to combine the active results
file with the results file of your choice.
For more information, see “Merging testplan results sets” on page 160.
Results/Pass/Fail Report...
Available only when a results file produced by running a testplan is the active
window. Opens the Results Pass/Fail Report dialog, which generates an on-
screen report on the number and percentage of tests that have passed.
For more information, see “Generating a testplan Pass/Fail report” on
page 162.
Results/Select...
Available only when the active window is a results window. Opens the Select
Results dialog, which allows you to chose which set of results to display. You
can also use this dialog to add a comment to individual results sets.
By default, SilkTest saves the results of five executions per script, suite, or
testplan. To change the default number of results saved, edit the History Size
option in the Runtime Options dialog, available by selecting Runtime on the
Options menu.
Results/Send to QA Radar...
Available only when the active window is a results window. Opens the Send
Results to QA Radar dialog, which you can use to send your results directly
to QA Radar, the Segue product that you can use to manage your
application’s bug reports, enhancement requests, and documentation issues.
For more information, see “Sending results to QA Radar” on page 158.
Results/Show Summary
Available only when the active window is a results window. Displays the
results summary for the current suite, testplan, script, or testcase, including
the start and elapsed time, and error totals. By default, a summary is shown
for the script. To change this default behavior, see “Show Overall Summary”
on page 488.
Results/View Differences...
Available only when the active window is a results window, and the current
line of the results file displays a box icon preceding an error message. Opens
the Difference Viewer when you click on the icon.
For more information, see “Finding application logic errors” on page 154.
Results/View Options...
Displays a dialog that lets you specify which information you want displayed
in the results window and how you want the information sorted.
Run Menu
The Run menu contains the following commands:
• Abort
• Application State...
• Compile
• Compile All
• Debug
• Run
• Run All Tests
• Run Marked Tests
• Show Status
• Testcase...
Run/Abort
Available only when a program is being run or debugged on a target machine
other than the host machine. Terminates the script, suite, testcase, or testplan
that is currently executing.
Run/Application State...
Opens the Run Application State dialog, which allows you to run or debug an
application state defined in your test frame file or the active script file.
If you click Debug, SilkTest displays the file in which the application state
resides and enters debugging mode. Special debugging commands are
available on the Breakpoint, Debug, and View menus. In addition, SilkTest
places a breakpoint at the first line of the application state.
Run/Compile
Compiles the testplan, suite, or script and all dependent files (such as include
files), if they have changed since they were last compiled. It also compiles
the following if needed: the files listed in the Use Files field in the Runtime
Options dialog, the compiler constants declared in the Runtime Options
dialog, and the include files loaded at startup.
Compile-time errors are automatically written to an error window. Compiling
a suite is equivalent to compiling each script in the suite individually.
Compare to Run/Compile All.
Run/Compile All
Compiles the script or suite and all dependent include files, even if they have
not changed since they were last compiled. It also compiles files listed in the
Use Files field in the Runtime Options dialog and the compiler constants
declared in the Runtime Options dialog. Finally, it compiles the include files
loaded at startup, if needed.
Compile-time errors are automatically written to an error window. Compiling
a suite is equivalent to compiling each script in the suite individually.
Compare to Run/Compile.
Run/Debug
Available only when the file in the active window is a script. Reads the script
in the active window into a debugging window and enters debugging mode.
Special debugging commands are available.
For more information, see Chapter 7, “Using the Debugger”.
Run/Run
Compiles and runs the 4Test script, suite, or testplan in the active window.
Compilations errors are written to an errors window. Once the file is
compiled, all runtime errors (exceptions), reports, and other results
information are automatically written to the corresponding results file (file-
name.res), which SilkTest displays in a results window after the script, suite,
or testplan terminates.
SilkTest will not run a script, suite, or testplan whose file name has an
extension other than .t, .s, or .pln.
About saving files SilkTest always saves the suite, script, or testplan before running it if you
made any changes to it since the last time you saved it.
By default, SilkTest also saves all other open modified files whenever you run
a script, suite, or testplan. To prevent this automatic saving of other open
modified files, deselect the Save Files Before Running check box in the
General Options dialog. See “Options/General...” on page 480.
Run/Show Status
Hides or shows the Runtime Status window when running a script, suite,
testcase, or testplan on a target machine other than the host machine.
Run/Testcase...
Available only when the active window contains either a testcase or a results
file in which the current line is a testcase.
Opens the Run Testcase dialog, which displays all the testcases contained in
the current script. You can select a testcase to run or debug. If you select
Debug, SilkTest displays the file in which the testcase resides and enters
debugging mode and places a breakpoint at the first line of the testcase.
Note Debugging mode makes available all the debugging
commands on the Breakpoint, Debug, and View menus.
Testplan menu
The Testplan menu contains the following commands:
• Completion Report...
• Define Attributes...
• Detail...
• Find Next Mark
• Go to Script
• Insert Template...
• Manual Tests...
• Mark
• Mark All
• Mark by Named Query
• Mark by Query
• Unmark
• Unmark All
Testplan/Completion Report...
Opens the Testplan Completion Report dialog, which you use to generate a
report on the number of “complete” tests.
For more information, see “Measuring testplan completion” on page 256.
Testplan/Define Attributes...
Opens the Define Attributes dialog, which allows you to define or rename
QA Organizer attributes as well as add, rename, or remove attribute values.
For more information, see Chapter 11, “Categorizing and Marking
Testplans”.
Testplan/Detail...
When the cursor is positioned at a test or group description, opens the
Testplan Detail dialog. Use this command to add important information to the
current test description or group description.
The dialog has a list box and a panel for the following categories:
• Test Execution
• Test Attributes
• Symbols
To display a particular tab, select the appropriate tab.
List box The list box at the top of the dialog provides context. The black arrow icon in
the left margin identifies the test at the cursor position in the testplan. This is
the test description you are adding details to, for example, character search in
the preceding figure. Any details will apply to this test and any tests that
inherit details from this test. Group descriptions that apply to this test are
shown (in blue) at the top of the list box.
Test Execution tab The Test Execution tab, shown in the preceding figure, helps you specify
valid script, testcase, and testdata statements.
Script Specifies the full path of the script that you want to appear on the
current line of the testplan. To browse for the correct script, click the Scripts
pushbutton, which displays the Testplan Detail -- Scripts dialog, a standard
file-selection dialog. QA Organizer writes the full path, relative path, or just
the file name of the selected script into the testplan, beginning the line with
the reserved word script, followed by a colon.
Testcase Specifies the name of the testcase that you want to appear on the
current line of the testplan. To browse for the correct testcase, press the
Testcases pushbutton, which displays the Testplan Detail -- Testcase dialog
that displays the testcases contained in the script specified in the Script field.
QA Organizer writes the selected testcase into the testplan, beginning the line
with the reserved word testcase, followed by a colon.
Test Data Specifies the data values that you want to pass to the testcase.
Separate each value with a comma. QA Organizer writes the data to the
current line of the testplan, beginning the line with the reserved word
testdata, followed by a colon. For more information on specifying data
values, see Chapter 10, “Adding Data to a Testplan”.
Test Attributes tab The Test Attributes tab allows you to assign attribute values to your testplan.
For more information, see Chapter 11, “Categorizing and Marking
Testplans”.
Symbols tab The Symbols panel allows you to add, edit, or delete a symbol associated
with the test.
For more information, see “Specifying data that is shared by multiple tests”
on page 226.
Testplan/Goto Script
Opens the script file associated with the current line of the testplan and makes
it the active window.
Testplan/Insert Template...
Opens the Insert Testplan Template dialog, which you use to create a
template, or outline, of your testplan.
For more information, see “Using a template of the GUI hierarchy as an
outline” on page 60.
Testplan/Manual Tests...
Opens the Update Manual Tests dialog, which you can use to document your
manual tests and have them included in Completion and Pass/Fail reports.
For more information, see “Documenting manual tests in the testplan” on
page 67.
Testplan/Mark
Marks one or more tests in the testplan, depending upon the position of the
insertion cursor. The following table shows how to use Testplan/Mark.
Testplan/Mark All
Marks all the tests in the testplan.
Testplan/Mark by Query
Opens the Mark by Query dialog, which allows you to construct a query from
attributes, symbols, the script, the testcase, and test data.
For more information, see “Creating and modifying testplan queries” on
page 246.
Testplan/Unmark
Unmarks one or more tests and any subordinate tests, depending on the
position of the insertion cursor. The following table shows how to use
Testplan/Unmark.
Testplan/Unmark All
Unmarks all the tests in the testplan.
View menu
The View menu contains the following commands:
• Breakpoints
• Call Stack
• Collapse Data
• Expand Data
• Expression
• Global Variables
• Local Variables
• Module...
• Transcript
Note For complete information about debugging testcases, see
Chapter 7, “Using the Debugger”.
View/Breakpoints
Available only in debugging mode. Displays in the Breakpoints window,
which lists the module names and line numbers where breakpoints are set.
Double-click on an entry to move the focus to that line in the script.
View/Call Stack
Available only in debugging mode. Displays the current call stack in the Call
Stack window. A call stack is a listing of all the function calls that have been
called to reach the current function in the script you are debugging.
View/Collapse Data
Available only in debugging mode. Collapses the data about a variable. A
plus (+) sign in front of a variable name indicates that the data about the
variable is already collapsed. A minus sign (-) in front of a variable name
indicates that the data can be collapsed. A period (.) indicates that all the data
about a variable is already displayed.
Note You can also collapse the data about a variable by double-
clicking on the minus (-) symbol.
View/Expand Data
Available only in debugging mode. Expands the data about a variable. A
minus (-) in front of a variable name indicates that more data is available. A
plus (+) in front of a variable name indicates that the data about it is already
expanded. A period (.) indicates that all the data about a variable is already
displayed.
Note You can also expand the data about a variable by double-
clicking on the (+) symbol.
View/Expression
Available only in debugging mode. Opens an Expression window in which
you can evaluate an expression and check the result. To evaluate an
expression, type it in the input area and press Enter. The result appears
directly beneath the expression. You use the Expression window to query the
application you are running, thus performing actions different from those
specified in your script.
If you type an identifier name, the result is the value that variable currently
has in the running script. If you type an expression, the result is the value of
that expression. Any function you specify must return a value, and must be in
scope (accessible) at the current source line.
You can reference class properties and methods that return values in
expressions, as long as the declaration for the class they belong to is included
in one of the modules used by the script being debugged.
If an expression evaluates to a complex value, like an array, SilkTest may
display its result in collapsed form. Use View/Expand Data or View/Collapse
Data to manipulate the display, or double-click on the plus (+) and minus (-)
symbols.
View/Global Variables
Available only in debugging mode. Displays the Globals window, which lists
all global variables that are in scope (accessible) from the current source line
and their current values. You can set a new value for the variable in the Set
Variable text field.
If a variable is uninitialized, SilkTest labels it “<unset>.” If a variable has a
complex value, like an array, SilkTest may display its result in collapsed
form. Use View/Expand Data or View/Collapse Data to manipulate the
display, or double-click on the plus (+) and minus (-) symbols.
For more information on 4Test scope rules, see online Help.
View/Local Variables
Available only in debugging mode. Displays the Locals window, which lists
all local variables that are in scope (accessible) in the current function
declaration and their current values. You can set a new value for the variable
in the Set Variable text field.
If a variable is uninitialized, SilkTest labels it “<unset>.” If a variable has a
complex value, like an array, SilkTest may display its result in collapsed
form. Use View/Expand Data or View/Collapse Data to manipulate the
display, or double-click on the plus (+) and minus (-) symbols.
For more information on 4Test scope rules, see online Help.
View/Module...
Available only in debugging mode. Opens the View Module dialog, which
lists the modules used by the script being debugged.
Double-click on a module name to view it in a debugging window. The list
includes all the modules that loaded by startup.inc, so you can set breakpoints
in GUI functions, classes, and so forth.
View/Transcript
Available only in debugging mode. Makes the transcript window for the
current debugging session the active window. SilkTest sends error
information and output from print commands to the transcript window. The
contents of the transcript window are not written to disk.
Window menu
The Window menu contains the following commands:
• Arrange Icons
• Cascade
• Close All
• Next
• Previous
• Tile Horizontally
• Tile Vertically
Window/Arrange Icons
Arranges the icons in a row in the SilkTest main window.
Window/Cascade
Layers one SilkTest child window on top of the other. Except for the top
window, which is completely visible, only the title bar of each window is
showing.
Window/Close All
Closes all open windows. If any file has changed since you last saved it,
SilkTest prompts you to save the file before closing it.
Window/n
Lists all open windows. Each window in the history is numbered from 1 to n
and is identified by its file type and the window title bar. If the file has never
been saved, then the title bar is “(untitled).” Otherwise the file name is
shown.
The windows are numbered in the order you opened them, with first window
numbered 1. If more than nine windows are open, the More Windows... item
appears; click More Windows to open the Select Window dialog, which lists
all open windows.
The currently active window is preceded by a check mark. To make another
window active, do one of the following:
• Select a window from the menu.
• Select a window by its number; for example, to select the second
window, press Alt+W+2.
• Cycle through the history by using Window/Next or Window/Previous.
Window/Next
Makes the next window in the numbered window history the active window.
Repeat this command to cycle through the window history.
Window/Previous
Makes the previous window in the window history the active window. Repeat
this command to cycle through the window history.
Window/Tile Horizontally
Makes each child window of the SilkTest main window the same size and
positions them in a north to south orientation along the y axis. Each window
is as wide as the SilkTest window frame and is as tall as the height of the
display divided by the total number of child windows.
Window/Tile Vertically
Makes each child window of the SilkTest main window the same size and
positions them in an east to west orientation along the x axis. Each window is
as tall as the SilkTest window frame and is as wide as the width of the frame
divided by the total number of child windows.
Chapter Page
Appendix A, “QA Organizer Keywords” 527
Appendix B, “Using Drag-and-Drop” 531
Appendix C, “Supporting Owner-Draw List Boxes” 533
Appendix D, “Useful Multitestcase Code Templates” 535
Appendix E, “Glossary” 539
Keywords Just as the 4Test language has a set of keywords that you use to construct
testcases, QA Organizer also has its own distinct set of keywords that you use
when creating testplans. The keywords are described in the following table.
Statements You use QA Organizer keywords to construct statements, using this syntax:
keyword : value
keyword One of the QA Organizer keywords.
value A comment, script, testcase, include file, attribute name, or data
value.
For example, this statement associates the script myscript.t with the plan:
script : myscript.t
The # operator When a # character precedes a QA Organizer statement, the statement will
double as a test description in the testplan. This helps eliminate possible
redundancies in the testplan. For example, the following test description and
script statement:
Script is test.t
script:test.t
can be reduced to one line in the testplan:
#script: test.t
DragMouse combines the functionality of the begin and end drag methods.
However, DragMouse operates only within a single window.
For more information, see online Help.
Boxes
An owner-draw list box is a list box that has the owner-draw style bit set. This
is distinct from a custom object that looks like a standard list box but is not.
The following procedure describes how developers can modify an application
so that SilkTest can access the text of a standard list box that is owner-draw
and that does not have the HasStrings style bit turned on. (If the HasStrings
style bit of the owner-draw list box is turned on, then you do not need to
make the modifications described here.) The procedure entails modifying
each owner-draw list box’s parent window procedure so that SilkTest can
query the parent about what is in the list box.
Procedure To turn on the HasStrings style bit of the owner-draw list box:
1 Include owndraw.h, supplied with SilkTest, in your source files.
2 For each owner-draw list box, add code, such as that shown below, to its
parent’s WndProc.
LONG FAR PASCAL ListParentWndProc (HWND hWnd, UINT uiMsg,WPARAM wParam,
LPARAM lParam)
{
//Use a static for the registered message number
static UINT uiMsgGetItem Text = 0;
LPGETITEMTEXTSTRUCT LpGetItemText;
USHORT usItem;
PSZ pszItemText;
switch (uiMsg)
{
...
default;
//Process the QAP_GETITEMTEXT message
if (uiMsg == uiMsgGetItemText)
{
//lParam points to a LPGETITEMTEXTSTRUCT structure
lpGetItemText = (LPGETITEMTEXTSTRUCT) lParam;
case ODA_GETITEMTEXT:
//Return the text for the requested list item
//(lpGetItemText->itemID is the index of the item in the
//list box -- the same number passed to LB_GETITEMDATA)
usItem = UINT (lpGetItemText->itemID);
pszItemText = <pointer to text of item[usItem]>;
strncpy (lpGetItemText->lpstrItemText, pszItemText,
lpGetItemText->nMaxItemText);
lpGetItemText->lpstrItemText[lpGetItemText->nMaxItemText-1]
= ‘\0’;
lpGetItemText->bSuccess = TRUE;
break;
case ODA_GETITEMTEXTSIZE:
//Return the length of the requested list item
usItem = UINT (lpGetItemText->itemID);
lpGetItemText->nMaxItemText = <length of item[usItem]> + 1;
lpGetItemText->bSuccess = TRUE;
break;
...
}
}
}
}
Templates
This appendix contains two templates that you might find useful as a basis for
your multitestcases.
What you will learn This appendix contains the following section:
Topic Page
Parallel template 535
Client/Server template 536
Parallel template
This template is stored as parallel.t in the Examples subdirectory of the
SilkTest installation directory. The code tests a single application that runs on
an externally defined set of machines.
This multitestcase template accepts a list of machine names. The application
whose main window is MyMainWin is invoked on each machine. The same
operations are then performed on each machine in parallel. If any testcase
fails, the multitestcase will be marked as having failed; however, a failed
testcase within a thread does not abort the thread.
You can use this template by doing three edits:
• Include the file that contains your window declarations.
• Substitute your application’s MainWin name defined in your MainWin
window declaration for the template’s “MyMainWin” MainWin name.
• Insert the calls to one or more tests, or to the main function, where
indicated.
use "myframe.inc"
rendezvous
Client/Server template
This template is stored as multi_cs.t in the Examples subdirectory of the
SilkTest installation directory. This testcase invokes the server application
and any number of client applications (based on the list of machines passed to
it) and runs the same function on all clients concurrently, after which the
server will perform end-of-session processing.
You can use this template by doing the following edits:
• Include the files that contains your window declarations for both the
client application and the server application.
• Substitute your server application’s MainWin name defined in your
MainWin window declaration for the template’s “MyServerApp”
MainWin name.
• Substitute your client application’s MainWin name defined in your
MainWin window declaration for the template’s “MyClientApp”
MainWin name.
Agent The 4Test Agent is the software process that translates the commands in your
4Test scripts into GUI-specific commands. In other words, it is the Agent that
actually drives and monitors the application you are testing. One Agent can
run locally on the host machine. In a networked environment, any number of
Agents can run on remote machines.
application state The state you expect your application to be in at the beginning of a testcase.
This is in addition to the conditions required for the base state. See base state.
attributes In QA Organizer, site-specific characteristics that you can define for your
testplan and assign to test descriptions and group descriptions. Each attribute
has a set of values. For example, you define the Developer attribute and
assign it the values of Bob, Carol, Ted, and Alice, the names of the QA
engineers in your department. Attributes are useful for grouping tests, in that
you can run or report on parts of the testplan that have a given attribute value;
for example, all tests that were developed by Bob can be executed as a group.
In SilkTest, an attribute is a characteristic of an application that you verify in
a testcase. Used in the Verify Window dialog.
base state The known, stable state you expect the application to be in at the start of each
testcase. See DefaultBaseState.
call stack In debugging mode, a list of functions and testcases executing at the time an
error occurred in a script. Listed in reverse order, from the last one executed
back to the first.
child object Subordinate object in the GUI hierarchy. A child object is either logically
associated with or physically contained by its superior object, the parent. For
example, the File menu, as well as all other menus, are physically contained
by the main window. See parent object.
class GUI object type. The class determines which methods can operate on an
object. Each object in the application is an instance of a GUI class.
class mapping Association of nonstandard custom objects with standard objects understood
by SilkTest.
client area A window’s internal area less the scroll bars, title bar, and borders.
custom object Nonstandard object that SilkTest does not know, by default, how to interact
with.
data-driven testcase Generalized testcase that receives many combinations of data from 4Test
functions and/or a testplan.
data member Variable defined within a class or window declaration.
declarations See window declarations.
DefaultBaseState Built-in application state function that returns your application to its base
state. By default, the built-in DefaultBaseState ensures that the application is
running and is not minimized, the main window of the application is open,
and all other windows (for example, dialogs and messages boxes) are closed.
See also base state.
Difference Viewer Dual-paned display-only window that lists every expected value in a testcase
and its corresponding actual value. Highlights all occurrences where
expected and actual values differ. You display the Difference Viewer by
selecting the box icon in the results file.
dynamic instantiation Creation of a fully qualified object name for a GUI object based on the
object’s class and tag. Done automatically whenever the recorded object has
no associated declaration defined in the test frame file. See also fully
qualified object name.
dynamic link library A library of reusable functions that allow code, data, and resources to be
(DLL) shared among programs using the module. Programs are linked to the module
dynamically at runtime.
exception Signal that something did not work as expected in a script. Logs the error in
the results file.
frame file See test frame file.
fully qualified object Name that uniquely identifies a GUI object. The actual format depends on
name whether or not a window declaration has been previously recorded for the
object and its ancestors. See also dynamic instantiation.
group description In QA Organizer, one or more lines in an outline that describe a group of
tests, not a single test. Group descriptions by default are displayed in black.
See also test description.
Hungarian notation Naming convention in which a variable’s name begins with one or more
lowercase letters indicating its data type. For example, the variable name
sCommandLine indicates that the data type of the variable is STRING.
identifier Name used in test scripts to refer to an object in the application. Logical,
GUI-independent name. Identifier is mapped to the tag in a window
declaration.
include file File that contains window declarations, and can contain constant, variable,
and other declarations.
load testing Testing that determines the actual (not simulated) impact of multi-machine
operations on an application, the server, the network, and all related elements.
localize an application To make an application suitable for a specific locale: for example, to include
foreign language strings for an international site.
manual test In QA Organizer, a test that is documented but cannot be automated and,
therefore, cannot be run within the testplan. You might chose to include
manual tests in your testplan in order to centralize the testing process. To
indicate that a test description is implemented manually, you use the keyword
value manual in the testcase statement.
mark In QA Organizer, a technique used to work with one or more tests as a group.
A mark is denoted by a black stripe in the margin bar of the testplan. Marks
are temporary and last only as long as the current work session. Tests that are
marked can be run or reported on independently as a subset of the total plan.
master plan In QA Organizer, that portion of a testplan that contains only the top few
levels of group descriptions. Within the master plan you can expand (display)
its subplans, which contain the remaining levels of group description and test
description. The master plan/subplan approach allows multi-user access to a
testplan, while at the same time maintaining a single point of control for the
entire project. A master plan file has a .pln extension. See also subplan.
message box Dialog box that has only static text and pushbuttons. Typically, message
boxes are used to prompt a user to verify an action (such as “Save changes
before closing?”) or to alert a user to an error.
method Operation, or action, to perform on a GUI object. Each class defines its own
set of methods. Methods are also inherited from the class’s ancestors.
minus (-) sign In a file, an icon that indicates that all information is displayed. Click on the
minus sign to hide the information. The minus sign becomes a plus sign.
modal A dialog box that presents a task that must be completed before continuing
with the application. No other part of the application can be accessed until the
dialog box is closed. Often used for error messages.
modeless A dialog box that presents a simple or ongoing task. May be left open while
accessing other features of the application, for example, a Search dialog box.
negative testing Tests that deliberately introduce an error to check an application’s behavior
and robustness. For example, erroneous data may be entered, or attempts
made to force the application to perform an operation that it should not be
able to complete. Generally a message box is generated to inform the user of
the problem.
nested declarations Indented declarations that denote the hierarchical relationships of GUI
objects in an application.
object file A script file or include file that is executable but not editable or viewable—as
opposed to a source file, which is editable but must be compiled before it is
executed.
outline In QA Organizer, a structured, usually hierarchical model that describes the
requirements of a testplan and contains the statements that implement the
requirements. The outline supports automatic, context-sensitive coloring of
testplan elements.
In SilkTest, the outline is a 4Test editor mode that supports automatic,
context-sensitive coloring and indenting of 4Test elements.
parent object Superior object in the GUI hierarchy. A parent object is either logically
associated with or physically contains its subordinate object, the child. For
example, the main window physically contains the File menu as well as all
other menus.
performance testing Testing to verify that an operation in an application performs within a
specified, acceptable period of time. Alternately, testing to verify that space
consumption of an application stays within specified limits.
plus (+) sign In a file, an icon that indicates that there is hidden information. You can show
the information by clicking on the plus sign. The plus sign becomes a minus
sign.
polymorphism Different classes or objects performing the same named task, but with
different execution logic.
properties Characteristics, values, or information associated with an object, such as its
state or current value.
subplan Testplan that is referenced by another testplan, normally the master testplan,
by using an include statement. Portion of a testplan that lives in a separate file
but can be expanded inline within its master plan. A subplan may contain the
levels of group description and test description not covered in the master
plan. A subplan can inherit information from its master plan. You add a
subplan by inserting an include statement in the master plan. A subplan file
has a .pln extension, as in subplan-name.pln. See also master plan.
suite A file that names any number of 4Test test script files. Instead of running
each script individually, you run the suite, which executes in turn each of
your scripts and all the testcases it contains.
symbols In QA Organizer, used in a testplan to pass data to 4Test testcases. A symbol
can be defined at a level in the testplan where it can be shared by a group of
tests. Its values are actually assigned at either the group or test description
level, depending on whether the values are shared by many tests or are unique
to a single test. Similar to a 4Test identifier, except that its name begins with a
$ character.
tag Object’s actual name or index as it appears in the GUI. The name by which
SilkTest locates and identifies objects in the application.
template A hierarchical outline in QA Organizer that you can use as a guide when
creating a new testplan. Based on the window declarations in the frame file.
test description In QA Organizer, a terminal point in an outline that specifies a testcase to be
executed. Test descriptions by default are displayed in blue. See also group
description.
test frame file Contains all the data structures that support your scripts: window
declarations, user-defined classes, utility functions, constants, and variables,
as well as other include files.
testcase In a script file, an automated test that ideally addresses one test requirement.
Specifically, a 4Test function that begins with the testcase keyword and
contains a sequence of 4Test statements. It drives an application to the state to
be tested, verifies that the application works as expected, and returns the
application to its base state.
In a testplan, a testcase is a keyword whose value is the name of a testcase
defined in a script file. Used in an assignment statement to link a test
description in a testplan with a 4Test testcase defined in a script file.
testplan In general, a document that describes test requirements. In QA Organizer, a
testplan is displayed in an easy-to-read outline format, which lists the test
requirements in high-level prose descriptions. The structure can be flat or
many levels deep. Indentation indicates the level of detail. A testplan also
contains statements, which are keywords and values that implement the test
descriptions by linking them to 4Test testcases. Large testplans can be
divided into a master plan and one or more subplans. A testplan file has a .pln
extension, such as find.pln. See also master plan and subplan.
variable A named location in which you can store a piece of information. Analogous
to a labeled drawer in a file cabinet.
verification statement 4Test code that checks that an application is working by comparing an actual
result against an expected (baseline) result.
window declarations Descriptions of all the objects in the application’s graphical user interface,
such as menus and dialogs. Declarations are stored in an include file which has
a .inc extension, typically the frame.inc file. See also test frame file.
window part Predefined identifiers for referring to parts of the window. Associated with
common parts of MoveableWin and Control classes, such as LeftEdge,
MenuBar, ScrollBar.
X
X
meaning of 490
X Window System
changing verification
keystroke 113
Y
Year 2000 compliance
date transformations 195
setting Year 2000 transformations at
runtime 487
testing converted applications 195
Z
Zoom menu item (Differences)
bitmap tool 406
Zoom window 416
capturing 418
zooming
areas of difference in bitmaps 416