Ab Initio EME

Ab Initio EME
Overview on Ab Initio EME

BT Project Team
TCS, Kolkata
1
Confidential
Asias Largest Global Software & Services Company
Ab Initio EME
CONTENTS
INTRODUCTION
HISTORY
ENVIRONMENT STRUCTURE
EME DATA STORE/REPOSITORY CONNECTION SETTINGS
PROJECT
SANDBOX
PARAMETERS
VERSION CONTROL -- CHECK IN , CHECK OUT
WORKING WITH PREVIOUS VERSIONS OF GRAPHS/OBJECTS
KNOWN PROBLEMS
Confidential
Ab Initio EME
INTRODUCTION
What is EME?
EME, Enterprise Meta Environment, is an object oriented data storage
system that version controls and manages various kinds of information
associated with Ab Initio applications, which may range from design
information to operational data. In simple terms, it is a repository, which
contains data about data metadata.
Why EME?
Avoid setup scripts, use EME parameters -- more standardisation
Source Control
dependency and impact analysis for the graphs in the repository
Confidential
Ab Initio EME
A Bit of History
Initially, two metadata products were available from Ab Initio :

Re>Posit -- Base technical metadata product
BM>R -- Business metadata repository on top of Re>Posit
These two products are now bundled together with a new name EME
Confidential
Ab Initio EME
Environment Structure
stdenv (Standard
Environment)
Public Sandboxes
(Optional)
Data Area
EME
Repository
User
Sandboxes
Check in
Check out
Inclusion of public projects/
Reference to public sandboxes
(Optional)
_REPO Variables
Inclusion of stdenv
5
Confidential
EME Data store/Repository

connection settings
Ab Initio EME
EME datastore is a specific instance of EME in the environment. This is a repository

where different versions of code and its related data like the record formats,
transformations etc are maintained. At any point of time a user can connect to only
one such EME repository instance. To access an EME Datastore, go to
Project>EME Datastore Settings in the GDE Menu and details are to be filled up in the
following boxlike:
6
Confidential
EME Data store/Repository

connection settings
Ab Initio EME
Following details are to be filled up in the EME Datastore Settings

Method
: Remote Execution (Rexec)/Telnet
Host
: The host where the EME Datastore resides
Login and Password
: Unix Login credentials for the host
Co>Operating system Location : Path to where the Ab Initio Co>Operating
system is installed
EME Datastore Location
: Path to where the EME Datastore is located
Mode
: Source Code Control
After filling in the detail press on the Connect button to test the connection. If the
details are filled in correctly you will get a message box confirming the connection.
7
Confidential
Project in EME
Ab Initio EME
Project
A Project is a collection of related graphs and its associated elements like
dml, xfr etc in the EME Datastore.
Project structure
Typically a project should contains maximum of 5 to 10 graphs. This helps in
organising the code efficiently within EME. With increase in the number of graphs in
a Project, the time taken to perform dependency analysis on the graphs and related
data increases. Before adding a Project to an existing application, which already has
a number of Projects in place, the impact it might have on other Projects and on the
Application as a whole must be considered.
8
Confidential
Ab Initio EME
Structure of a Project in EME
/Projects
Project1
mp
Sub directory for AI Graphs
run
Sub directory for deployed shell scripts
xfr
Sub directory for transform files
dml
Sub directory for record format files
db
Sub directory for database interface files
bin
Sub directory for tools and utilities
sql
Sub directory for sql queries
Project2
9
Confidential
Different Types of Projects
Ab Initio EME
Private vs. Public Project

There is often information common to multiple Projects. For instance several
Projects may share some record format files or transform files. Such elements
which are used across Projects can be made widely reusable by making them part
of a Project and including this Project in other projects to access the common
elements. A Project that is included by other Projects is termed as a Public Project
and the Projects including public Projects are known as Private Projects.
A public Project is public in the sense that their data and metadata are expected
to be shared with other Projects and a private Project is private in the sense that
their data and metadata are not expected to be shared with other Projects.
10
Confidential
Different Types of Projects
Ab Initio EME
The Environment Project (Stdenv)

There is a special Project associated with every instance of Ab Initio environment
known as the Environment Project or stdenv (Standard Environment). This is no
different from a regular Project in the structure. It contains machine and Application
specific settings like the data directory mount points, max-core settings and application
wide parameters like current date, which are used across all Projects. During creation
of any Project, stdenv is included in it by default. A single stdenv is required for an entire
set of applications on a single machine and sharing a single EME Datastore.
11
Confidential
Concept of Sandbox
Ab Initio EME
What is a Sandbox?
Projects held in the EME Datastore cant be manipulated directly. To work on
Projects, they must be checked out to a working area on the file system where
we can develop and modify code. This working area on the file system is known
as a Sandbox. It has exactly the similar directory structure as that of a Project
in the Datastore.
Each object that needs to be worked on is checked out to a sandbox where
modifications or enhancements are carried out. After the changes are complete
the code is checked in from the sandbox area to the EME Datastore.
This action creates a new version of the code in the EME Datastore.
12
Confidential
Sandbox Projects vs. EME

Datastore Project
Ab Initio EME
Sandboxes are work areas used to develop, test or run code associated with a given
project. Only one version of the code can be held within the sandbox at any time.
The EME Datastore contains all versions of the code that have been checked into it.
A particular sandbox is associated with only one Project where as a Project can be
checked out to a number of sandboxes.
Project in EME Datastore
User Sandbox
Check-in
Project1
Project1
v5
File1
Check-out
v1
File1
File1
v2
v3
File1
File1
v4
v5
File1
13
Confidential
Parameters
Ab Initio EME
A parameter is a name-value pair with some additional attributes that determine

when and how to interpret or resolve its value. Parameters are used to provide
logical names to physical location and should always be used instead of hard
coded paths in graphs. We can have two types of parameters, graph and Project
parameters.
Graph parameters
Graph parameters, as the name suggests are specific to the individual graphs and
are private to them. They affect execution of the graph for which they have been
defined. Graph parameters can be defined by navigating to Edit>Parameters in the
GDE which opens the graph parameters editor.
14
Confidential
Parameters
Ab Initio EME
Project parameters
Project parameters are inherited by all the graphs in the Project and are accessed
from the GDE by the sandbox parameter editor in Project>Edit
Sandbox>Parameters. This shows a dialog box prompting to enter the sandbox
path. Choose the correct host and the sandbox path and press OK to open the
sandbox parameter editor, which exactly like the graph parameter editor shown as
above.
15
Confidential
Parameters
Ab Initio EME
Editing parameters
To add a new Project parameter or to modify the value of an existing one, we
should first lock the parameters in the sandbox parameter editor by clicking the
lock button on the menu. If nobody has locked it in their sandboxes, then the lock
symbol turns green indicating a successful lock. This implies we can add or modify
the parameters now. If a lock is already there before, then while opening the
parameter editor it shows a warning saying the parameters are already locked and
the lock symbol is red in such a case. After getting a lock, others are disabled from
editing the parameters.
16
Confidential
Major Parameter Attributes
Ab Initio EME
Scope: Scope of a parameter can be formal or local. A local parameter is internal to the
sandbox and most of the parameters have their scope as local. Its value is taken from
the value column in the parameter editor. A formal parameter is one whose value can
be set from outside, i.e. from the environment where the graph is run. Its value is
supplied from the command line. A green diamond can identify the formal parameters
with an arrow mark.
Kind: If scope is local, kind is left unspecified, but if it is formal, the kind is automatically
set to keyword.
Type: This determines the nature of the parameter. Project parameters have four types
as string, common Project, switch and dependent. Graph parameters have different set
of types.
Export: When this check box is checked, the corresponding parameter value is
exported as an environment variable, otherwise it is generated as a local shell variable.
Private Value: If a parameter is specified as a private value, any subsequent
changes to it remain private to the local sandbox and are not checked in into the EME.
This is useful when different users want different values for the same parameter.
17
Confidential
Ab Initio EME
Major Parameter Attributes
Value: This column specifies the value of the parameter.

Interpretation: This determines how the parameter is going to be evaluated.
Constant: Value is taken literally.
$ Substitution: Variables with $ prefixes are replaced with their values
${} Substitution: Variables within {} and with $ prefixes are replaced by their
values but other occurrences of $ are ignored.
Shell: Korn shell syntax is used to evaluate the value of the parameter.
Required: This attribute can take two values, required (the default) or optional. If it
is required, the value column cant be left blank but if it is optional, it can
be left blank.
18
Confidential
Version Control and Tagging
Ab Initio EME
Each object under EME source control, which may be a file, a directory or a Project,
exist as a series of versions, each of which is a representation of what was checked
in by some user. It can optionally have a textual description attached to it called a
tag and a description as a comment. Each version is separately numbered and can
be accessed by either the version number or the tag attached to it. Version
numbers, which are integers and tags, are global to the whole EME datastore. Tags
are the basic units during migration of code across EME datastore instances.
19
Confidential
How to access EME
Ab Initio EME
Ways to access EME in BT environment :

GDE
TBD display screen shot ???
EME web Interface using valid user and password

TBD ??? Display web interface .. URL ??
20
Confidential
Check out of files using GDE
Ab Initio EME
Check out
The Ab Initio GDE provides wizards to check out code from the EME to
sandbox.Check out updates the sandbox with the particular version of code that is
being checked out from the EME. By default the latest version of any object is
checked out, but we can check out any version of code we want. Any object that is
version controlled in the EME datastore can be checked out to a sandbox, which
may be pre-existing or may be created during check out process itself. While
checking out a Project or any objects belonging to the Project to a sandbox, stdenv
and any common Projects associated with it also need to be referenced in the
sandbox. If the sandbox to which you are checking out is an existing one, it would
have the information as to where to reference for the common projects (The stdenv
sandbox and the public sandboxes). In case it is a new sandbox, during check out
we have to point to the stdenv and public sandbox (if any) paths.
21
Confidential
Ab Initio EME
Check out wizard is invoked by navigating to Project>Check Out, which looks as

follows:
Select the Project /directory or file you want to check out by browsing to the
particular Project /directory or file.
In sandbox host dropdown list select the host on which the sandbox resides.
Enter the path to an existing sandbox (the sandbox must be associated with the
concerned Project, which is being checked out) or mention a new one in the
directory field, which would be created during check out.
22
Confidential
Ab Initio EME
The advanced options dialog can be seen by clicking on advanced button.
The first two options specify whether to check out the required files from the
parent project and whether to check out required files from the common Projects.
The default is check out the required files from the parent project. A file is required
if it is directly referenced in a graph or if it is referenced in an include in a dml or xfr.
While checking out a whole project these two options are disabled as shown above.
Run host setup script makes sure to run the host profiles set up script before
check out and mark files read only on check out does exactly what it says. The
default is on for both of these options.
23
Confidential
Ab Initio EME
We can select a particular tagged version of the object we want to check out from
the tag drop down list. By default the latest version is checked out.
On clicking next, if the sandbox doesnt exist then a confirmation is asked whether
to create the new sandbox or not. Clicking yes creates the sandbox and checks out
the object mentioned to this sandbox.
24
Confidential
Ab Initio EME
You will be prompted to enter the sandbox locations of stdenv and any common
projects associated with the project, unless the sandbox has already these values
specified or the sandbox is a pre-existing one.
25
Confidential
Ab Initio EME
Clicking on Do Checkout performs the checkout operation and on its completion

a window shows the operations performed.
26
Confidential
Locking
Ab Initio EME
A lock must be acquired on the object to be modified in the sandbox after

successful completion of check out. To modify a graph that has been checked
out, first open the graph in the GDE and then click on the lock symbol
on
the menu. This checks whether the version in the sandbox is the latest version of
the object in the datastore and if it is, the lock symbol turns green
showing
that the graph is now locked and is editable.
If the graph has already been locked in some other sandbox, after opening the
graph in the GDE the lock is red in colour
denoting that there is already a lock
on it. A lock can be acquired on an object only if the sandbox version and the
current version of the object in the EME are the same.
Once a lock is acquired and the changes are complete the object must be
checked in to the datastore to create a new version in the Datastore.
For non-Ab Initio objects which cant be locked from the GDE, a lock can be
obtained from the Unix command line using the air commands available to obtain
a lock on the particular object.
27
Confidential
Check in of files using GDE
Ab Initio EME
Once the project files have been edited and updated they need to be checked in to
create a new version in the EME datastore, which will be available for other users.
Check in wizard is invoked by navigating to Project>Check in. Before starting the
check in wizard, it checks for any unsaved file in the sandbox and prompts whether to
save them or not. The check in wizard looks as follows:
28
Confidential
Ab Initio EME
Choose the Sandbox host from the drop down list.

In the Directory or file field, browse to the particular file in the sandbox that you want to check in. You may select a file under
the sandbox or you may also select the whole sandbox in which case the whole project would be checked in to the EME
datastore.
Browse to the parent Project in Project Directory field, which points to the Project directory in the EME datastore where the
object would be checked in.
To go to the advanced options in check in click on the advanced button.
The check in tab indicates how you want the check in to be performed. By default Force overwrite is unchecked. Once it is
checked the object is checked in even if there are conflicts and becomes the latest version in the datastore. Run Host Setup
script causes to run the host profiles set up script before each check in. It is advised not to change any settings here.
29
Confidential
Ab Initio EME
The analysis tab specifies how much dependency analysis is done and on which
objects during check in.
30
Confidential
Ab Initio EME
A tag, which is a descriptive piece of text and a comment, can be attached to

the version that will be checked in. This can be mentioned in the tag tab of
advanced options dialog box. The tagging standards are described in another
document.
31
Confidential
Ab Initio EME
After filling in the tag information, on clicking next in the check in wizard a check
in ready dialog is displayed.
Clicking on Do Checkin performs the actual check in and displays a window similar to the
check out finished window with the results of check in and dependency analysis (if specified
in the advanced option).
32
Confidential
Working with previous versions

of graphs/objects
Ab Initio EME
Many a time a previous version of a graph may be required to check out and update
rather than working with the latest or current version of the graph as available in the
EME data store. Using check out wizard in GDE, you may check out a tagged
version of a graph, which is not the latest version available. But GDE doesnt allow
locking such versions. In such a case, the procedure to be followed is:
33
Confidential
Working with previous versions

of graphs/objects
Ab Initio EME
Check out the required previous tagged version of the graph to your sandbox.
(V1 in figure below).
Check it back in with Force Overwrite in advanced option in check in wizard.
This will make it the current version in the data store. (V4 in figure below).
Lock the graph now to make the changes.
Check in the graph back to the EME data store. This updated version will
become the latest version in the EME data store. (V5 in figure below)
User Sandbox
EME data store

V1
V1
V2
V3
V4
Updated
V5
Check in +force overwrite
Check in
Check out
34
Confidential
How EME will be used in Volume/AMP project
Ab Initio EME
E.g . To be Done ????

Define Project
Define Parameters
Check-in
Check-out
--- Module Leads

--- Module Leads
--- Team Member
--- Team Member
35
Confidential
Known Problems Accessing EME from offshore
Ab Initio EME
E.g . To be Done ???? With resolution...

Few experienced resources in EME -->>> Training
Considerable Time taken to check out , check in from offshore -->>>
check out at start , check-in at end , no intermediate movement
Help on EME
EME documentation kit . TBD ??
36
Confidential
Ab Initio EME
THANK YOU
37
Confidential

Ab Initio EME

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ab Initio EME

Uploaded by

Copyright:

Available Formats

Ab Initio EME

Overview on Ab Initio EME

Asias Largest Global Software & Services Company

Asias Largest Global Software & Services Company

Asias Largest Global Software & Services Company

Initially, two metadata products were available from Ab Initio :

Asias Largest Global Software & Services Company

Asias Largest Global Software & Services Company

EME Data store/Repository

EME datastore is a specific instance of EME in the environment. This is a repository

Asias Largest Global Software & Services Company

EME Data store/Repository

Following details are to be filled up in the EME Datastore Settings

Asias Largest Global Software & Services Company

Asias Largest Global Software & Services Company

Structure of a Project in EME

Sub directory for AI Graphs

Sub directory for deployed shell scripts

Sub directory for transform files

Sub directory for record format files

Sub directory for database interface files

Sub directory for tools and utilities

Sub directory for sql queries

Asias Largest Global Software & Services Company

Different Types of Projects

Private vs. Public Project

Asias Largest Global Software & Services Company

Different Types of Projects

The Environment Project (Stdenv)

Asias Largest Global Software & Services Company

Asias Largest Global Software & Services Company

Sandbox Projects vs. EME

Asias Largest Global Software & Services Company

A parameter is a name-value pair with some additional attributes that determine

Asias Largest Global Software & Services Company

Asias Largest Global Software & Services Company

Asias Largest Global Software & Services Company

Major Parameter Attributes

Asias Largest Global Software & Services Company

Major Parameter Attributes

Value: This column specifies the value of the parameter.

Asias Largest Global Software & Services Company

Version Control and Tagging

Asias Largest Global Software & Services Company

How to access EME

Ways to access EME in BT environment :

EME web Interface using valid user and password

Asias Largest Global Software & Services Company

Check out of files using GDE

Asias Largest Global Software & Services Company

Check out of files using GDE

Check out wizard is invoked by navigating to Project>Check Out, which looks as

Asias Largest Global Software & Services Company

Check out of files using GDE

The advanced options dialog can be seen by clicking on advanced button.

Asias Largest Global Software & Services Company

Check out of files using GDE

Asias Largest Global Software & Services Company

Check out of files using GDE

Asias Largest Global Software & Services Company

Check out of files using GDE

Clicking on Do Checkout performs the checkout operation and on its completion

Asias Largest Global Software & Services Company

A lock must be acquired on the object to be modified in the sandbox after