WebSphere Transformation Extender (Mercator)

Rajesh Mamidi Venkata Brundavanam Date: 11/27/2006

Agenda
Agenda EAI What is WTX (Mercator) WTX Design Studio
Type Designer Database Interface Designer Map Designer Integration Flow Designer Command Server Event Server

Enterprise Integration

CRM ODBC Database Flat Files

ERP

FTP Legacy SMTP

EDIFACT

VANs

X12

Enterprise Integration

Enterprise Integration
Enterprise Application Integration

Enterprise Application Integration is the process of coordinating the operations of various applications across an enterprise.

EAI solutions offer the following features: Reduced development and maintenance cost. Enhanced performance and reliability Extension of legacy system lifecycle

What is WebSphere Transformation Extender (Mercator)?

WebSphere TX is a unique data integration technology for use in Application to Application and Business to Business enablement.
A powerful data transformation engine that natively across the enterprise operating environments that has the ability to speak directly to native interfaces and APIs without the need for coding or re-tooling.

The Benefits of Websphere Transformation Extender

Legacy-to-Anything Integration
Enhanced visibility enables single view of key data within front- and back-office applications/systems

Automated processes for integration of data with transactional and operational systems Automated decisions on routing based on content of messages, eliminating costly and time- consuming hand coding High interoperability within a ServiceOriented Architecture to leverage existing and future IT enhancements Compliance with industry and government standards and regulations governing data transfer and integration (EDI, ACORD, AL3, HIPAA, SWIFT)

EDI-to-Anything Integration
Enhanced data connectivity simplifies the complex and speeds B2B implementations

Transformation & Routing

Enhanced synchronization provides real-time validation and transformation

Web Services Enablement

IBM WebSphere DataStage TX Map

Application

MAP

Application

Database

Type Tree

SOA Edition

Database

File

File

Web Web Service Service Enterprise Data Integration Requirements IBM Websphere DataStage TX Value

Expose legacy applications containing key data elements Use a common methodology for interaction across apps, data, processes Leverage mainframe power with Web-based technologies

Automated, complex data transformation and routing across many points of integration Integrated tools and environment for complete enterprise data integration Enterprise-wide interoperability supported by an SOA

What is Mercator?

WebSphere TX (Mercator)

Components

TYPE DESIGNER MAP DESIGNER DATABASE INTERFACE DESIGNER INTEGRATION FLOW DESIGNER EVENT SERVER COMMAND SERVER

10

WebSphere TX (Mercator)
Design Studio

Components
Server Production System

Client Development System

Integration Flow Designer to model and manage transformation workflows Type Designer to define and import data objects Database Interface Designer to generate data definitions from metadata imported from database Map Designer to define data transformation and routing rules Transformation engines to develop, test and run transformations in development environment

Platform-specific transformation engines for production environment Adapters to integrate with specific types of data sources and targets Databases Messaging systems Packaged applications Business integration utilities

11

Modeling the Transformation Process

Analyze and Define the Data Properties Structure Validation rules Apply Integration Rules to Models Interfaces Structure Formats Semantics Resources Communication Storage Services Type Designer Database Interface Designer

Map Designer

Deploy and Manage Processes Flow Events Timing

12

Integration Flow Designer

Type Designer

TYPE DESIGNER

13

Type Designer
What is Type Tree? A type tree is a data dictionary that defines how types are classified. A type tree describes: Syntax Structure Semantics

14

Type Designer

Specify, define and manage Type Trees. Maintain Type Trees Properties of data Structure Data Validation Rules Different Character Sets

15

Type Designer
(Input and Output)

Step 3

File Record Record Record Data Items Simple Complex Complex Complex Complex

Step 2

Step 1

Simple Simple Simple

16

Type Designer
For modeling data objects Structure/containment of data Syntax properties Semantic validation rules Graphically design in a Type Tree Import or use industry data definitions

17

WebSphere DataStage TX Importers and Templates

Automatic data definitions: Importers Automatic data definitions: Importers

XML DTDs and Schemas XML DTDs and Schemas WSDL WSDL COBOL Copybooks COBOL Copybooks PeopleSoft PeopleSoft Java classes Java classes

All definitions are modifiable though GUI

Java Messaging Service Java Messaging Service

Database Structures Database Structures Application Definitions Application Definitions SAP: BAPI, IDoc, DXOB, BDC SAP: BAPI, IDoc, DXOB, BDC PeopleSoft 7: Business Document, Message Agent PeopleSoft 7: Business Document, Message Agent PeopleSoft 8: Component Interface PeopleSoft 8: Component Interface Siebel: COM Business Object, EAI, EIM Siebel: COM Business Object, EAI, EIM COM Components & Methods COM Components & Methods CORBA IDL CORBA IDL EJB API EJB API Text Files Text Files Tuxedo FML and View Tuxedo FML and View

Built-in Data Definitions Built-in Data Definitions

EDIFACT EDIFACT X12 X12 TRADACOMS TRADACOMS EANCOM EANCOM ODETTE ODETTE HIPAA X12 HIPAA X12 HL7 HL7 NCPDP NCPDP SWIFT SWIFT

18

Building In Additional Validation

Additional validation can be added to check if data, once isolated, matches specific data value or set of values
Use component rules to Use component rules to specify condition(s) that must specify condition(s) that must be met for a particular be met for a particular component to be valid component to be valid

Use restriction lists to limit Use restriction lists to limit item to particular value item to particular value or set of values or set of values

19

Database Designer

DATABASE INRERFACE DESIGNER

20

Database Interface Designer

Specify databases to use for data source or target Define query statements Generate type trees for tables/views, queries, and stored procedures by importing metadata from RDBMS Identify additional tree definition, mapping and execution requirements

21

Database Designer

Imports metadata about Queries Tables Stored procedures Database connectivity is supported under the control of Relational Database Management Systems (RDBMS).

22

2-23

Generating Type Trees from Databases

Table/View Type Tree

Database

Query

Type Tree

Database

Stored Procedure

Type Tree

Database

23

Database Designer

24

Map Designer

MAP DESIGNER

25

Map Designer
Create maps to: Validate data from data sources Identify data targets Specify data transformation logic (rules)

26

Map Designer
The Map Designer is used to:

Create maps to specify the logic necessary to transform the input data to the desired output data. Identify the source and data objects of the input data. Identify the target and data objects of the output data. Specify and build the output data according to the map rules. Provide information about data validation by generating trace files. View the run results of the map execution.

27

Map Designer
Map source file name Map source file name

Map name Map name Rule bar Rule bar Rule column Rule column

Navigator Navigator

From From Window Window Cards Cards

To To Window Window

28

How DataStage TX Engine Works

1. Validates input data against type tree starting with Input card #1 2. Applies map rules starting with Output card #1

Input card #1

Output card #1

Input card #2 Output card #2

29

Map Designer

30

Functions - Rules

Around 100 Functions Most can be used both in Component Rules as well as Mapping Rules

Component Rules evaluate to True or False Mapping Rules evaluate to some value

31

Integration Flow Designer

INTEGRATION FLOW DESIGNER

32

Integration Flow Designer

The integration flow designer (IFD) is a component of the Mercator design studio that provides a graphical interface to combine collection of maps and run them as a single unit There are three stages of Using the IFD : Defining Systems Verifying Component Relationships Preparing Systems to Run

33

Integration Flow Designer

Define systems that graphically and logically Define systems that graphically and logically organize transformation workflows into logical organize transformation workflows into logical collections based on requirements collections based on requirements Generate system control information from Generate system control information from system definitions for real-time processing system definitions for real-time processing
34

What is a System?

A system is: A set of logically related maps Assigned to a server on which it will run Run by WTX transformation servers A system can include: Components that reference maps Components that reference other systems

35

Event & Command Server

TRANSFORMATION ENGINES

36

WebSphere DataStage TX Transformation Engines

Command Server Can be run on variety of platforms and operating systems Platform-specific Runs one map at a time

Event Server/Launcher Can be run on variety of platforms and operating systems Platform-specific Manages execution of systems of maps based on triggers Message events Time events File state changes Database triggers

37

Command Server
Command Server is used to execute maps

38

Command Server
Command Server Output

39

Event Server

The Event Server automates the execution of maps. The Event Server runs systems of maps that are created and generated using the Integration Flow Designer (IFD).

These systems of maps that are generated specifically to run in the Event Server are called Event Server system files sometimes referred to as Event Server control files.

40

Event Server/Launcher Architecture

Event Server Event Server Administration Administration Configuration information

Manageme Manageme nt Console nt Console

Listening port

Event Server Event Server service service (Windows) or (Windows) or daemon (Unix) daemon (Unix)

Listening port

Event Event Server Server Monitor Monitor

Control information

Status information
41

Event Event Server Server

Monitoring information

Event Server

42

WebSphere DataStage TX Administrative Tools

Event Server Administration allows configuration of secure access rights to monitor and control Event Servers and the DataStage TX systems they are running Management Console allows user to view and control the status of transformation workflows running with the Event Server Event Server Monitor gives detailed information about maps running with Event Server Snapshot Viewer enables analysis of snapshots of Event Server activity Resource Registry defines resource name aliases for data sources and targets

43

Event Server Monitor

Server name Server name Name of watch Name of watch

Vertical grid Vertical grid time interval time interval Green line indicates Green line indicates map success map success Yellow line indicates Yellow line indicates map warning map warning

Red line indicates Red line indicates map failure map failure 44

Thank You
Italian Traditional Chinese

Grazie

Thai

Russian

Obrigado
Brazilian Portuguese

Merci
French

Gracias
Spanish Arabic

Danke
German Simplified Chinese

Japanese

45