EME

<Enterprise Meta Environment>

Basics
INTRODUCTION

What is EME?

EME is an object oriented data storage

system that version controls and manages
various kinds of information associated with
Ab Initio applications, which may range
from design information to operational data.
In simple terms, it is a repository, which
contains data about data – metadata.
Environment Structure

Public Sandboxes stdenv (Standard

(Optional) Environment)

Data Area
EME
Repository

User
Sandboxes

Check in _REPO Variables

Check out Inclusion of stdenv
Inclusion of public projects/
Reference to public sandboxes
(Optional)
Revisiting Sandbox Concept

What is a Sandbox?

Projects held in the EME Datastore can’t be manipulated directly. To work on

Projects, they must be checked out to a working area on the file system where
we can develop and modify code. This working area on the file system is known
as a Sandbox. It has exactly the similar directory structure as that of a Project
in the Datastore.

Each object that needs to be worked on is checked out to a sandbox where

modifications or enhancements are carried out. After the changes are complete
the code is checked in from the sandbox area to the EME Datastore.
This action creates a new version of the code in the EME Datastore.
 Sandbox Projects
vs.
EME Datastore Project
Sandboxes are work areas used to develop, test or run code
associated with a given project. Only one version of the code can
be held within the sandbox at any time. The EME Datastore
contains all versions of the code that have been checked into it. A
particular sandbox is associated with only one Project where as a
Project can be checked out to a number of sandboxes.

User Sandbox Project in EME Datastore

Check-in

Project1 Project1

v5 File1 Check-out v1 File1

v2 File1
v3 File1
v4 File1
v5 File1
 EME Data store/Repository connection
settings

EME datastore is a specific instance of EME in the environment. This is a repository

where different versions of code and its related data like the record formats,
transformations etc are maintained. At any point of time a user can connect to only
one such EME repository instance. To access an EME Datastore, go to
Project>EME Datastore Settings in the GDE Menu and details are to be filled up in the
following boxlike:
EME Data store/Repository
connection settings

Following details are to be filled up in the EME Datastore Settings

Method : Remote Execution (Rexec)/Telnet

Host : The host where the EME Datastore resides
Login and Password : Unix Login credentials for the host
Co>Operating system Location : Path to where the Ab Initio Co>Operating
system is installed
EME Datastore Location : Path to where the EME Datastore is located
Mode : Source Code Control

After filling in the detail press on the Connect button to test the connection. If the
details are filled in correctly you will get a message box confirming the connection.
 Project

Project
A Project is a collection of related graphs and its associated
elements like
dml, xfr etc in the EME Datastore.

Project structure
Typically a project should contains maximum of 5 to 10 graphs.
This helps in organising the code efficiently within EME. With
increase in the number of graphs in a Project, the time taken to
perform dependency analysis on the graphs and related data
increases. Before adding a Project to an existing application,
which already has a number of Projects in place, the impact it
might have on other Projects and on the Application as a whole
must be considered.
Structure of a Project in EME

/Projects mp Sub directory for AI Graphs

run Sub directory for deployed shell scripts

Project1

xfr Sub directory for transform files

Project2
dml Sub directory for record format files

db Sub directory for database interface files

bin Sub directory for tools and utilities

sql Sub directory for sql queries

 Different Types of Projects
Private vs. Public Project

zThere is often information common to multiple Projects. For

instance several Projects may share some record format files or
transform files. Such elements which are used across Projects can
be made widely reusable by making them part of a Project and
including this Project in other projects to access the common
elements. A Project that is included by other Projects is termed as
a Public Project and the Projects including public Projects are
known as Private Projects.

zA public Project is public in the sense that their data and

metadata are expected to be shared with other Projects and a
private Project is private in the sense that their data and metadata
are not expected to be shared with other Projects.
The Environment Project (Stdenv)

There is a special Project associated with every instance of Ab Initio

environment known as the Environment Project or stdenv (Standard
Environment). This is no different from a regular Project in the structure.
It contains machine and Application specific settings like the data
directory mount points, max-core settings and application wide
parameters like current date, which are used across all Projects. During
creation of any Project, stdenv is included in it by default. A single stdenv
is required for an entire set of applications on a single machine and
sharing a single EME Datastore.
 Version Control and Tagging

Each object under EME source control, which may be a file, a

directory or a Project, exist as a series of versions, each of which
is a representation of what was checked in by some user. It can
optionally have a textual description attached to it called a tag
and a description as a comment. Each version is separately
numbered and can be accessed by either the version number or
the tag attached to it. Version numbers, which are integers and
tags, are global to the whole EME datastore. Tags are the basic
units during migration of code across EME datastore instances.
Check out of files using GDE

zCheck out wizard is invoked by navigating to Project>Check Out, which

looks as follows:

zSelect the Project /directory or file you want to check out by browsing to
the particular Project /directory or file.
zIn sandbox host dropdown list select the host on which the sandbox
resides.
zEnter the path to an existing sandbox (the sandbox must be associated
with the concerned Project, which is being checked out) or mention a new
one in the directory field, which would be created during check out.
z The advanced options dialog can be seen by clicking on advanced
button.

zThe first two options specify whether to check out the required files
from the parent project and whether to check out required files from the
common Projects. The default is check out the required files from the
parent project. A file is required if it is directly referenced in a graph or
if it is referenced in an include in a dml or xfr. While checking out a
whole project these two options are disabled as shown above.
zRun host setup script makes sure to run the host profile’s set up script
before check out and mark files read only on check out does exactly
what it says. The default is on for both of these options.
zWe can select a particular tagged version of the object we want to
check out from the tag drop down list. By default the latest version is
checked out.

zOn clicking next, if the sandbox doesn’t exist then a confirmation is

asked whether to create the new sandbox or not. Clicking yes creates
the sandbox and checks out the object mentioned to this sandbox.
zYou will be prompted to enter the sandbox locations of stdenv and any
common projects associated with the project, unless the sandbox has
already these values specified or the sandbox is a pre-existing one.
•Clickingon Do Checkout performs the checkout operation and
on its completion a window shows the operations performed.
Locking

z A lock must be acquired on the object to be modified in the sandbox

after successful completion of check out. To modify a graph that has
been checked out, first open the graph in the GDE and then click on
the lock symbol on the menu. This checks whether the version
in the sandbox is the latest version of the object in the datastore and
if it is, the lock symbol turns green showing that the graph is
now locked and is editable.
z If the graph has already been locked in some other sandbox, after
opening the graph in the GDE the lock is red in colour denoting
that there is already a lock on it. A lock can be acquired on an object
only if the sandbox version and the current version of the object in
the EME are the same.
z Once a lock is acquired and the changes are complete the object must
be checked in to the datastore to create a new version in the
Datastore.
z For non-Ab Initio objects which can’t be locked from the GDE, a lock
can be obtained from the Unix command line using the air commands
available to obtain a lock on the particular object.
Check in of files using GDE

Once the project files have been edited and updated they need to be
checked in to create a new version in the EME datastore, which will be
available for other users. Check in wizard is invoked by navigating to
Project>Check in. Before starting the check in wizard, it checks for any
unsaved file in the sandbox and prompts whether to save them or not.
The check in wizard looks as follows:
z Choose the Sandbox host from the drop down list.
z In the Directory or file field, browse to the particular file in the
sandbox that you want to check in. You may select a file under
the sandbox or you may also select the whole sandbox in
which case the whole project would be checked in to the EME
datastore.
z Browse to the parent Project in Project Directory field, which
points to the Project directory in the EME datastore where the
object would be checked in.
z To go to the advanced options in check in click on the
advanced button.
z The check in tab indicates how you want the check in to be
performed. By default “Force overwrite” is unchecked. Once it
is checked the object is checked in even if there are conflicts
and becomes the latest version in the datastore. “Run Host
Setup script” causes to run the host profiles set up script
before each check in. It is advised not to change any settings
here.
z The analysis tab specifies how much dependency analysis is done and
on which objects during check in.
A tag, which is a descriptive piece of text and a comment, can be
attached to the version that will be checked in. This can be
mentioned in the tag tab of advanced options dialog box. The
tagging standards are described in another document.
z After filling in the tag information, on clicking next in the check in
wizard a check in ready dialog is displayed.

z Clicking on “Do Checkin” performs the actual check in and displays a

window similar to the “check out finished” window with the results of
check in and dependency analysis (if specified in the advanced
option).
Working with previous versions of graphs/objects

• Check out the required previous tagged version of the graph to your sandbox.
(V1 in figure below).
• Check it back in with “Force Overwrite” in advanced option in check in wizard.
• This will make it the current version in the data store. (V4 in figure below).
• Lock the graph now to make the changes.
• Check in the graph back to the EME data store. This updated version will become
the latest version in the EME data store. (V5 in figure below)
User Sandbox EME data store
V1 V1

V2

V3

V4
Updated V5
Check in +force overwrite

Check in

Check out
 Parameters
A parameter is a name-value pair with some additional attributes that
determine when and how to interpret or resolve its value. Parameters
are used to provide logical names to physical location and should always
be used instead of hard coded paths in graphs. We can have two types
of parameters, graph and Project parameters.

Graph parameters
Graph parameters, as the name
graphs and are private to them.
which they have been defined.
navigating to Edit>Parameters
parameters editor.
suggests are specific to the individual
They affect execution of the graph for
Graph parameters can be defined by
in the GDE which opens the graph
 Project parameters
Project parameters are inherited by all the graphs in the Project and are
accessed from the GDE by the sandbox parameter editor in Project>Edit
Sandbox>Parameters. This shows a dialog box prompting to enter the
sandbox path. Choose the correct host and the sandbox path and press
OK to open the sandbox parameter editor, which exactly like the graph
parameter editor shown as above.
Major Parameter Attributes

Scope: Scope of a parameter can be formal or local. A local parameter is internal to the
sandbox and most of the parameters have their scope as local. Its value is taken from
the value column in the parameter editor. A formal parameter is one whose value can
be set from outside, i.e. from the environment where the graph is run. Its value is
supplied from the command line. A green diamond can identify the formal parameters
with an arrow mark.

Kind: If scope is local, kind is left unspecified, but if it is formal, the kind is automatically
set to keyword.
Type: This determines the nature of the parameter. Project parameters have four types
as string, common Project, switch and dependent. Graph parameters have different set
of types.
Export: When this check box is checked, the corresponding parameter value is
exported as an environment variable, otherwise it is generated as a local shell variable.
Private Value: If a parameter is specified as a private value, any subsequent
changes to it remain private to the local sandbox and are not checked in into the EME.
This is useful when different users want different values for the same parameter.
Value: This column specifies the value of the parameter.

Interpretation: This determines how the parameter is going to be evaluated.

Constant: Value is taken literally.

$ Substitution: Variables with $ prefixes are replaced with their values

${} Substitution: Variables within {} and with $ prefixes are replaced by

their values but other occurrences of $ are ignored.

Shell: Korn shell syntax is used to evaluate the value of the parameter.

Required: This attribute can take two values, required (the default) or
optional. If it is required, the value column can’t be left blank but if
it is optional, it can be left blank.
Thank You