Kapow Mashup Server 6.

Users Guide

ModelMaker

ModelMaker
Users Guide

Copyright 1999-2007 Kapow Technologies http://www.kapowtech.com All rights reserved.

CONTENTS

iii

Contents
INTRODUCTION ........................................................................................ 1 What is ModelMaker? ..........................................................................1 Before You Read On ............................................................................1 Other Resources .................................................................................1 MODELMAKER BASICS ............................................................................... 2 Objects .............................................................................................2 Attributes ..........................................................................................2 Domain Models...................................................................................2 MODELMAKER USER INTERFACE ................................................................... 4 TUTORIAL: CREATING A DOMAIN MODEL ........................................................ 5 Creating the Object.............................................................................5 Adding Attributes................................................................................6 Validating the Domain Model ................................................................7 Previewing the Object .........................................................................8 Saving the Domain Model ....................................................................8 Creating the Database Table .............................................................. 10 EXAMPLE DOMAIN MODELS ....................................................................... 13 FEATURES IN DEPTH ................................................................................ 14 Database Output Objects Explained ..................................................... 14 Validating Domain Models .................................................................. 15 Configuring Objects .......................................................................... 15 Configuring Attributes ....................................................................... 16 Modifying an Existing Object .............................................................. 17 Creating and Deleting Database Tables ................................................ 18 INDEX.................................................................................................. 19

INTRODUCTION

Introduction
What is ModelMaker?
ModelMaker is an application for modeling the objects used by a robot, such as the objects that the robot extracts and the objects that it accepts as input objects.

Before You Read On

Before you proceed to the next chapter, make sure that you have installed ModelMaker correctly as described in the Installation Guide.

Other Resources
Additional, mostly referential documentation on ModelMaker is available in the ModelMaker entry in RoboHelp, which is accessible from the Help menu in ModelMaker. You should also check out the support site at this URL: http://support.kapowtech.com/

MODELMAKER USER'S GUIDE

ModelMaker Basics
ModelMaker is an application for creating the objects that are used by robots. This chapter introduces you to the key concepts in ModelMaker. With ModelMaker, you are able to design objects that are modeled after realworld data. In the most common case, an object is designed to hold the data that a robot collects or extracts from a data source. In RoboMaker, you are able to build your robots using the objects you create with ModelMaker. See the RoboMaker Users Guide for more information about how RoboMaker uses the objects created by ModelMaker.

Objects
Objects can be used by robots for different purposes. For example, a robot can extract objects, return error or status objects, or it can receive objects as input, known as input objects. An object has an object type that determines how the object can be used. The following object types are available: Input Object this is an object that can be given as input to a robot, to provide information that the robot needs to perform its task. Output Object this is an object that can be extracted by a robot. This type can also be used for other objects to be returned by a robot, such as error or status objects. Database Output Object this is an object that can be extracted by a robot and stored in a database. In essence, objects are used as data carriers, either as input to a robot in the form of input objects, or as output from a robot in the form of output objects or database output objects.

Attributes
An object has a number of attributes. The attributes are where the data is held inside an object. For example, a news article object could have attributes such as title, author, date, and content. An attribute has an attribute type, which determines what kind of data the attribute can contain. For example, the title attribute of a news article object could have the type Short Text. There are many different attribute types available, such as Short Text, Long Text, Character, Integer, Number, Date, Boolean, and Binary. The data in an attribute is called an attribute value.

Domain Models
Objects are organized into domain models. A domain model defines one or more objects. A domain model typically models a specific domain, such as

MODELMAKER BASICS

news, stocks, or airline information, by defining the objects that are relevant for that domain. ModelMaker is an application for creating domain models, just as RoboMaker is an application for creating robots. In ModelMaker, you are always working on a current domain model. A domain model must satisfy certain requirements to be used in the other applications of Kapow Mashup Server. The domain model is said to be valid if it satisfies these requirements. Checking whether a domain model is valid is called validating the domain model.

MODELMAKER USER'S GUIDE

ModelMaker User Interface

This chapter gets you started with using ModelMaker, by introducing you to the user interface of the application. The main window of ModelMaker is shown below:

Attribute Table Object List

Add, Remove, and Configure Attributes

Add, Remove, and Preview Objects

Figure 1: ModelMaker Main Window To the left in the window is the Object List. This is the list of objects in the domain model that you are working on. You can add new objects, remove objects, change the order of the objects, and preview objects (to see how they will appear in RoboMaker), using the buttons below the Object List. The currently selected object is configured in the right part of the main window. Among other things, you can configure the attributes of the object. This is done in the Attribute Table. You can add new attributes, remove attributes, change their order, and configure attributes using the buttons below the Attribute Table.

TUTORIAL: CREATING A DOMAIN MODEL

Tutorial: Creating a Domain Model

Let us create a domain model containing a news article object that we can use for extracting news articles from web sites.

Creating the Object

Start ModelMaker and choose Create a new domain model.... ModelMaker will start with an empty domain model. Add a new object to the domain model by clicking the button below the Object List. Choose Database Output Object and click OK. This creates an object that can be extracted by a robot and stored in a database. The ModelMaker Main Window should now look like this:

Note that a number of attributes have already been added to the object, such as robotId, robotRunId, etc. These attributes are for administration purposes and are required for a database output object. Therefore, you should not remove or change these attributes. They are shown in gray as a reminder that they are required and should not be modified.

MODELMAKER USER'S GUIDE

Adding Attributes
Now, let us add a title attribute to the article object. Click the button below the Attribute Table. The Attribute Configuration window will pop up. Configure the attribute as shown below:

Click the OK button. As you return to the main window, notice that the title attribute has been added to the Attribute Table.

TUTORIAL: CREATING A DOMAIN MODEL

Add another attribute for the body text of the article, and configure it as shown below. Note that you should set the attribute type to Long Text:

Click OK to return to the main window. Of course, a real-world article object would have more attributes than these, but let us skip those in this example.

Validating the Domain Model

Try validating your domain model by clicking the message: icon. You should see this

As you can see, your domain model is not yet valid, since you havent given the object a name. Therefore, set the name of the object to MyArticle.

MODELMAKER USER'S GUIDE

Then try to validate the domain model again by clicking the should see this message:

icon. Now you

The domain model is now valid.

Previewing the Object

In order to see what the object would look like when inspected in RoboMaker, you can preview it by pressing the button below the Object List. This will show the window below:

This can be especially useful if you want to verify that attributes that are not marked as Visible are indeed hidden inside the object. Note that in our case, only our two new attributes are shown. The six attributes that were grayed when we created our database output object all have the Visible property unchecked in their configurations.

Saving the Domain Model

To make your domain model accessible to your robots, you must save the domain model in the robot library that the robots belong to. Normally, this is the robot library of the current robot project, which is the robot project that

TUTORIAL: CREATING A DOMAIN MODEL

you are currently working on. For more information on robot libraries and robot projects, see the RoboMaker Users Guide. To save your domain model, click the icon, and choose the folder to save the domain model in. ModelMaker will default to the robot library folder of the current robot project, which is where we want to save the domain model in this tutorial. Enter MyArticle as the file name and click Save. Your domain model will now be saved as the file MyArticle.model. Now, let us check that the domain model is accessible in RoboMaker. To use a newly created domain model in another Kapow Mashup Server application that is already running, you must reload the domain models in that application by clicking the icon in the application. If you already have RoboMaker running, reload its domain models now to make it use the new domain model, otherwise start RoboMaker now. In RoboMaker, go to the Output Objects tab of the Objects View, located in the lower right corner of the RoboMaker Main Window, and shown below.

10

MODELMAKER USER'S GUIDE

Click the Add/Remove... button to open the Add/Remove Output Objects icon to add a new output object to the robot. You should window. Click the now be able to select your MyArticle object in the drop-down box. Do that, and then click OK to close the window. You can now view your object in the Output Objects tab, which should look like this:

As you can see, the MyArticle object has the title and body attributes that you defined in ModelMaker, and looks as when we previewed it in ModelMaker. Note also that the administration attributes (robotId, robotRunId, etc.) are not visible or accessible. Important Note: You cannot have two objects with the same name in the same robot library, even if the objects are in two different domain models in the robot library.

Creating the Database Table

If you want to be able to store your database output objects in a database, you should create matching tables in an appropriate database. ModelMaker can assist you in creating these tables. If your system administrator has not yet set up a database connection for you, you should skip this portion of the tutorial.

TUTORIAL: CREATING A DOMAIN MODEL

11

To see how to create the database table for the MyArticle object, select Create Database Table ( ) in the Tools menu of ModelMaker. This will open the Create Database Table Window. Select the MyArticle object in the Object property, as shown below:

Next, select the database name along with the relevant database vendor type for that database. Click Generate SQL. This will generate a suggestion for an SQL statement for creating the table, similar to the one shown below:

You can edit the SQL statement to fit your needs, and then execute it or save it. However, in this tutorial, just click Close in the two windows to close them without doing anything with the statement.

12

MODELMAKER USER'S GUIDE

This completes the tutorial. Youve now learned how to create a new domain model with an object from scratch, and make it ready for use in robots you create with RoboMaker.

EXAMPLE DOMAIN MODELS

13

Example Domain Models

The installation includes a number of example domain models representing some common domains. These example domain models are located in the Project/Library/Examples subfolder in the installation folder. If you want to create an object with the same name as an object in an example domain model, you need to remove the example domain model, since a robot library cannot contain multiple objects with the same name. Alternatively, you can use another project. Please see the Installation Guide for information on how to change the current project.

14

MODELMAKER USER'S GUIDE

Features in Depth
This chapter discusses the features of ModelMaker in more detail. For more specific information about configuring domain models, objects, and attributes, please consult the online RoboHelp entries for ModelMaker.

Database Output Objects Explained

If your robot uses a database output object, it will need to handle the returned objects in some way. Typically, the objects will be saved in a storage location, such as in a file or in a database. This way, other applications will then be able to access the data that has been collected by the robot. In order to be able to successfully store and retrieve objects, the following attributes are included in all database output objects: Attribute Name robotId robotRunId refindKey firstExtractionDate latestExtractionDate extractedInLatestRun Attribute Type Integer Integer Refind Key Date Date Boolean

When storing an object in a storage location, an important issue is what to do if the storage already contains the particular object. A simple mechanism is used for refinding existing objects in storage and updating these instead of inserting multiple copies of the object. When a database output object is ready to be added to storage, a search is made for an existing object with the same refindKey in the storage. If such an object exists, it will be updated, and if not, the extracted object will be inserted. You must select the attributes to include in the comparison when refinding objects, by setting the Part of Refind Key option of those attributes. As for the remaining attributes required, the firstExtractedDate is the date and time when the object was extracted for the first time, and the latestExtractedDate is the date and time when the extracted object was most recently inserted or updated. The extractedInLatestRun attribute will be true if the object was extracted in the latest robot run. The way in which your extracted objects are physically stored depends on the Storage Environment that is used when the robot is run. See the RoboServer Users Guide or the RoboRunner Users Guide for more on running

FEATURES IN DEPTH

15

robots with environments. Note that not all storage environments support refinding of objects. In the case of saving a database output object to database storage using the Database Storage Environment, it is of course necessary that the appropriate database table exists in a database that is available. This means that it is a requirement that the table contains columns that match the attributes listed above, as well as columns for all of the attributes specific to the object. See the Creating and Deleting Database Tables section for more information on how ModelMaker can assist you in setting up the appropriate database tables. Consult the Installation Guide for more information on setting up database connections. For more information about environments, see the RoboRunner Users Guide and the Environments section of the RoboRunner section in the online RoboHelp.

Validating Domain Models

A domain model must satisfy certain requirements to be used in the other applications of Kapow Mashup Server. For example, each object in a domain model must have a unique name. A domain model that satisfies all the requirements is said to be valid. Checking that a domain model is valid is called validating the domain model. Every time a domain model is changed in ModelMaker it is automatically validated. This includes any modifications to the objects in the domain model, as well as any of the attributes on those objects. In the title of the main window of ModelMaker, you can see if the domain model is valid or not. If the domain model is invalid, select Validate Domain Model in the Edit menu or press and you will be presented with a message box explaining the reason.

Configuring Objects
When working with an object within a domain model, it is important that you configure all of the relevant properties of the object. Otherwise, the domain model will be invalid, or the object will not behave as expected when used in RoboMaker. To configure an object, you must first select it in the Object List. The selected object is then displayed in the right side of the main view of ModelMaker, where you can make changes. All objects within a domain model must have a valid name. If a domain model contains only one object, it is common practice to give the same name to both the domain model and the object. This will not cause any problems, and can in fact help simplify things in many cases. You can set the name of an object above the Attribute Table in the object configuration view. Also, as mentioned previously, all objects must have an object type, which indicates how the object will be used. You can choose the object type for an object using the Type drop-down box below the Attribute Table.

16

MODELMAKER USER'S GUIDE

Configuring Attributes
The attributes within an object must also be correctly added and configured in order for the object to be valid. You must specify both a name and a type for each attribute. The available attribute types are shown in Figure 2. Attribute Type Integer Number Boolean Character Short Text Long Text Password Description An integer, e.g. 12. A number, e.g. 12.345. A boolean value, i.e. either "true" or "false". A single character, e.g. "A". A short text. Will be displayed in a one-line text field. A long text. Will be displayed in a multi-line text box. A password. Will be diplayed in a password field that shows asterisks instead of the characters in the password. An HTML clip. This is the same as a Long Text, except that you can preview the clip in a browser window. An XML document. This is the same as a Long Text, except that only well-formed XML documents are allowed. A date. The date must be of the form yyyy-mm-dd hh:mm:ss.n, e.g. "1992-04-25 10:33:06.0". Binary data, i.e. any sequence of bytes. An image. This is the same as Binary Data, except that you can preview the image. A PDF document. This is the same as Binary Data, except that you can preview the PDF document. A session (containing cookies, authentications, etc.). A currency code, as defined by the ISO-4217 standard, e.g. "EUR" for Euro. A country code, as defined by the ISO-3166 standard, e.g. "DE" for Germany.

HTML

XML

Date Binary Image PDF Session Currency Country

FEATURES IN DEPTH

17

Attribute Type Language Refind Key

Description A language code, as defined by the ISO-639 standard, e.g. "de" for German. A special refind key used for refinding objects. Figure 2: Attribute Types

The Refind Key attribute type is used when storing database output objects see the section Database Output Objects Explained for more information. An attribute has a number of other configurable properties, such as whether or not the attribute is visible from within RoboMaker, and whether or not the attribute is required to contain data before the object can be stored. Refer to the online RoboHelp entries in ModelMaker on attribute configuration for more information.

Modifying an Existing Object

If you need to change an object after you have written robots that use it, you need to be careful. Your robots might stop working if you do something wrong. You should never perform any of the following changes to an object that is already used by existing robots if you do, you will not be able to use the object in those robots (however, the robots may still be loaded without the objects that were modified): Change the name of an object. Change the type of an object. Delete an object. Remove or rename an attribute that is assigned a value different from its default value in the Robot Configuration. Change the type of an attribute that is assigned a value different from its default value in the Robot Configuration to a type than is not compatible with the assigned value.

You may perform the following changes to an object, but some of the existing robots may generate errors when they are executed (you can subsequently fix these errors): Change the name of an attribute. Change the Required property of an attribute from false to true. Add a new attribute that has the Required property set to true. Delete or rename an attribute that has its default value in the Robot Configuration.

18

MODELMAKER USER'S GUIDE

Change the type of an attribute that has its default value in the Robot Configuration.

You can always make the following changes, without affecting the existing robots: Change the Required property of an attribute from true to false. Change the name and revision of a domain model. Change a comment (no matter where). Add a new object with a new name. Add a new attribute that has the Required property set to false.

Creating and Deleting Database Tables

If you want to store your extracted objects in a database, you should create matching tables in the database. ModelMaker can assist you in creating these tables. It does so by examining the object you have created, and generating SQL that is appropriate for creating a new database table. In the Tools menu, select Create Database Table ( ) to open a window where you can choose the name of your database (as defined in Settings see the Installation Guide), the type of database, and the object that you want to create the table for. By clicking Generate SQL, you will be shown a suggestion for an SQL statement for creating the table, which you then have the option to modify as well as execute or save. The SQL shown is a recommended suggestion by ModelMaker you can change the statement to fit your needs, if required. For example, you might change the column type for a Short Text attribute from "VARCHAR(255)" to "VARCHAR(50)" in order to conserve database space, or you could add an auto-incrementing primary key. However, under normal circumstances, you should not modify the table name or any of the column names, nor remove any of the columns. You can also delete an existing table in the database using the Drop Database Table ( ) command under the Tools menu.

INDEX

19

Index
A
attributes, 2 adding to object, 6 configuring, 17 types, 2, 17

O
objects, 2 adding attributes, 6 configuring, 15 creating, 5 database output, 2, 14 input, 2 modifying existing, 18 naming, 10 output, 2 previewing, 8 refinding, 14 storing extracted, 15 types, 2 output object, 2

C
current robot project, 9

D
database output object, 2 database tables creating, 10, 19 deleting, 19 domain models, 3 saving, 9 validating, 7, 15

R
refind key, 14 robot library, 9

E
example domain models, 13

S
Storage Environment, 15

I
input objects, 2

V
validation, 7, 15

M
ModelMaker, 1, 2