Professional Documents
Culture Documents
1. What is the function you would use to transfer a string into a decimal?
In this case no specific function is required if the size of the string and decimal is same. Just
use decimal cast with the size in the transform function and will suffice. For example, if the source
field is defined as string(8) and the destination as decimal(8) then (say the field name is field1).
out.field :: (decimal(8)) in.field
If the destination field size is lesser than the input then use of string_substring function can be used
likie the following.
say destination field is decimal(5).
out.field :: (decimal(5))string_lrtrim(string_substring(in.field,1,5)) /* string_lrtrim used to trim
leading and trailing spaces */
7. What is the difference between partitioning with key and round robin?
Partition by Key - Distribute according to the key value
Partition by Round Robin - Distribute a predefined number of records to one flow and then the same
numbers of records to the next flow and so on. After the last flow resumes the pattern and almost evenly
distributes the records... This patter is called round robin fashion.
10.
The ramp parameter contains a real number that represents a rate of reject events in the
number of records processed.
11.
If the user wants to group the records on particular field values then rollup is best way to
do that. Rollup is a multi-stage transform function and it contains the following mandatory
functions.
1. initialise
2. rollup
3. finalise
Also need to declare one temporary variable if you want to get counts of a particular group.
For each of the group, first it does call the initialise function once, followed by rollup function
calls for each of the records in the group and finally calls the finalise function once at the end of
last rollup call.
12.
This is a tricky question, number of component in a graph has nothing to do with the level of knowledge
a person has. On the contrary, a proper standardized and modular parametric approach will reduce the
number of components to a very few. In a well thought modular and parametric design, mostly the graphs
will have 3/4 components, which will be doing a particular task and will then call another sets of graphs to
do the next and so on. This way total numbers of distinct graphs will drastically come down, support and
maintenance will be much more simplified. The bottomline is, there are lot more other things to plan rather
than to add components.
13.
Lookup File consists of data records which can be held in main memory. This makes the
transform function to retrieve the records much faster than retirving from disk. It allows the
transform component to process the data records of multiple files fastly.
14.
In RDBMS the relationship between the two tables is represented as Primary key and
foreign key relationship.Wheras the primary key table is the parent table and foreignkey
table is the child table.The criteria for both the tables is there should be a matching column
15.
An outer join is used when one wants to select all the records from a port - whether it has
satisfied the join criteria or not
16.
A Cartesian join will get you a Cartesian product. A Cartesian join is when you join every row of one table
to every row of another table. You can also get one by joining every row of a table to every row of itself.
17.
Main Purpose of Stored Procedure for reduse the network trafic and all sql statement executing in cursor
so speed too high.
18. Why might you create a stored procedure with the 'with recompile' option?
Recompile is useful when the tables referenced by the stored proc undergoes a lot of
modification/deletion/addition of data. Due to the heavy modification activity the execute plan
becomes outdated and hence the stored proc performance goes down. If we create the stored
proc with recompile option, the sql server wont cache a plan for this stored proc and it will be
recompiled every time it is run.
18.
What is a cursor? Within a cursor, how would you update fields on the row
just fetched?
The oracle engine uses work areas for internal processing in order to the execute sql
statement is called cursor.There are two types of cursors like Implecit cursor and Explicit
cursor.Implicit cursor is using for internal processing and Explicit cursor is using for user open
for data required.
19.
How would you find out whether a SQL query is using the indices you
expect?
explain plan can be reviewed to check the execution plan of the query. This would guide if
the expected indexes are used or not.
20.
21.
Because implicit is using for internal processing and explicit is using for user open data
requied
22.
23.
Describe the process steps you would perform when defragmenting a data
1) We can move the table in the same or other tablespace and rebuild all the indexes on the table.
alter table <table_name> move <tablespace_name> this activity reclaims the defragmented space in the
table
analyze table table_name compute statistics to capture the updated statistics.
2)Reorg could be done by taking a dump of the table, truncate the table and import the dump back into
the table.
24.
When running a stored procedure definition script how would you guarantee
the definition could be "rolled back" in the event of problems
There are quite a few factors that determines the approach such as what type of version control are
used, what is the size of the change, what is the impact of the change, is it a new procedure or replacing
an existing and so on.
If it is a new, then just drop the wrong one
if it is a replacement then how big is the change and what will be the possible impact, depending upon
you can have the entire database backed up or just create a script for your original procedure before
messing it up or you just do an ed and change the file back to original and reapply. you may rename the
old procedure as old and then work on new and so on.
few issues to keep in mind are synonyms, dependancies, grants, any job calling the procedure at the time
of change and so on. In nutshell, scenario can be varied and solution also can be varied.
25.
Basically,This is a part of D.B.A responsibilities GRANT means permissions for example GRANT
CREATE TABLE ,CREATE VIEW AND MANY MORE .
REVOKE means cancel the grant (permissions).So,Grant or Revoke both commands depend
upon D.B.A.
26.
Describe how you would ensure that database object definitions (Tables,
Indices, Constraints, Triggers, Users, Logins, Connection Options, and Server
Options etc) are consistent and repeatable between multiple database instances
(i.e.: a test and production copy of a database).
27.
Lookup is basically a specific dataset which is keyed. This can be used to mapping values as per
the data present in a particular file (serial/multi file). The dataset can be static as well dynamic ( in case
the lookup file is being generated in previous phase and used as lookup file in current phase).
Sometimes, hash-joins can be replaced by using reformat and lookup if one of the input to the join
contains less number of records with slim record length.
AbInitio has built-in functions to retrieve values using the key for the looku
28.
What is the difference between look-up file and look-up, with a relevant
example?
A lookup is a component of abinitio graph where we can store data and retrieve it by using a key
parameter.
A lookup file is the physical file where the data for the lookup is stored.
29.
If the DML changes dynamically then both dml and xfr has to be passed as graph level
parameter during the runtime.
30 How do you add default rules in transformer?
Double click on the transform parameter of parameter tab page of component properties, it
will open transform editor. In the transform editor click on the Edit menu and then select Add
Default Rules from the dropdown. It will show two options - 1) Match Names 2) Wildcard.
30.
To run a graph infinitely, the end script in the graph should call the .ksh file of the graph.
Thus if the name of the graph is abc.mp then in the end script of the graph there should be a
call to abc.ksh.
Like this the graph will run infinitely.
31.
Basically there are serial and parallel layouts supported by AbInitio. A graph can have
both at the same time. The parallel one depends on the degree of data parallelism. If the multifile system is 4-way parallel then a component in a graph can run 4 way parallel if the layout is
defined such as it's same as the degree of parallelism.
32.
Aggregation and Rollup both can summerise the data but rollup is much more
convenient to use. In order to understand how a particular summerisation being rollup is much
more explanatory compared to aggregate. Rollup can do some other functionalities like input
and output filtering of records.
33.
EME is said as enterprise metdata env, GDE as graphical devlopment env and Co-operating sytem
can be said as asbinitio server
34.
how to retrive data from database to source in that case whice componenet
is used for this?
To unload (retrive) Data from the database DB2, Informix, or Oracle we have
components like Input Table and Unload DB Table by using these two components we can
unload data from the database
35.
36.
Contineous components used to create graphs,that produce useful output file while running
continuously
Ex:- Contineous rollup,Contineous update,batch subscribe
37.
which one is faster for processing fixed length dmls or delimited dmls and
why ?
Fixed length DML's are faster because it will directly read the data of that length without
any comparisons but in delimited one,s every character is to be compared and hence delays
38.
39.
You can test a dbc file from unix command prompt using m_db test <name-of-the dbc file>
which tests the data base connection, data base version, user name, password, database objects and
other required mandatory values checking.
Here, the syntax is on below
m_db test <name-of-the dbc file>
Example:
40.
make it csv formatted , deliminated file and read it thru input table comp.
Co-Operating systems, that itself means a lot, it's not merely an engine or
interpretor. As it says, it's an operating system which co-exists with another operating
system. What does that mean.... in layman's term abinitio, unlike other applications, does
not sit as a layer on top of any OS. It itself has quite a lot of operating system level
capabilities such as multi files, memory management and so on and this way it completely
integrate with any other OS and work jointly on the available hardware resources. This
sort of Synergy with OS optimize the utilization of available hardware resources. Unlike
other applications (including most other ETL tools) it does not work like a layer and
interprete the commands. That is the major difference with other ETL tools , this is the
reason why abinitio is much much faster than any other ETL tool and obviously much
much costlier as well.
The appropriate parameter need to be selected in the component layout for the type of partioning..
45.
46.
Every component in Ab initio has a memory of 7MB (anybody correct me if I'm wrong) except a
few components like In-Memory components such as join, Rollup, Scan, join with DB etc., which uses an
additional memory called Max-Core. This parameter is only available if your data is not explicitly sorted
and your component also needs to sort the data besides the processing.This additional memory is also
used to store the temporary results.There is no specific fixed value for the max core and each in-memory
component has a default max core value like 10MB for sort and 64 MB for join component etc.,
47.
dml expression means abinitio dml are stored or saved in a file and dml describs the data
interms of expressions that performs simple computations such as files, dml also contains transform
functions that control data transforms,and also describs data interms of keys that specify grouping or non
grouping ,that means dml expression are non embedded record format files
.xfr means simply say it is non embedded transform files ,Transform function is express business rules
,local variables, statements and as well as conn between this elements and the input and the ouput fields.
44. What are the different versions and releases of ABinitio (GDE and Co-op version)
for GDE 1.10, 1.11, 1.12, 1.13, and 1.15 is latest one. For Co-Op latest one is 2.14.
45. How to run the graph without GDE?
In RUN ==> Deploy >> As script , it create a .bat file at ur host directory ,and then run .bat file
from Command prompt
46. can anyone give me an exaple of realtime start script in the graph?
Here is a simple example to use a start script in a graph:
In start script lets give as:
export $DT=`date '+%m%d%y'`
Now this variable DT will have today's date before the graph is run.
Now somewhere in the graph transform we can use this variable as;
out.process_dt::$DT;
48.
49.
what is m_dump
m_dump command prints the data in a formatted way.
50.
51.
Eg - You have 2 incoming flows (This can be data parallelism or component parallelism) on Broadcast
component, one with 10 records & other with 20 records. Then on all the outgoing flows (it can be any
number of flows) will have 10 + 20 = 30 records
Replicate - It replicates the data for a particular partition and send it out to multiple out ports of the
component, but maintains the partition integrity.
Eg - Your incoming flow to replicate has a data parallelism level of 2. with one partition having 10 recs &
other one having 20 recs. Now suppose you have 3 output flos from replicate. Then each flow will have 2
data partitions with 10 & 20 records respectively.
52.
Two are graph level parameters but in local you need to initialize the value at the time of
declaration where as globle no need to initialize the data it will promt at the time of running the graph for
that parameter.
53.
roll up generates data records that summarizes group of data records the rollup can be
used to perform the data aggregatuin like sum,avg,max,min etcScan component generates a
series of cumulative summary records such as successive yera to date totals for group of data
records
54.
55.
When you set the sorted-input parameter of "JOIN" component to "In memory:
Input need not be sorted", you can find the driving port.
Generally driving port use to improve performance in a graph.
The driving input is the largest input. All other inputs are read into memory.
For example, suppose the largest input to be joined is on the in1 port. Specify a port number of 1 as the
value of the driving parameter. The component reads all other inputs to the join for example, in0, and
in2 into memory.
Default is 0, which specifies that the driving input is on port in0.
JOIN also improves performance by loading all records from all inputs except the driving input into main
memory.
56.
57.
what is semi-join
in abinitio,there are 3 types of join...
1.inner join.
2.outer join
58.
How would you do performance tuning for already built graph ? Can you let
me know some examples?
example :- suppose sort is used in fornt of merge component its no use of using sort ! bcz we
hv sort component built in merge.
2) we use lookup instead of JOIN,Merge Componenet.
3) suppose we wnt to join the data comming from 2 files and we dnt wnt dupliates we will use union
funtion instead of adding addtional component for duplicate remover.
59.
60.
where as
.dbc is the database configuration file used for the other database components
61.
Sift links to MFS files on Unix for Ab Initio ? what is this ....
2) $pound what is this
3) $? what for it is used
4) types of loading
5 overwrite when it used ?
Link is a command where in unix we use for when the original file is deleted when we create a
link the other replaces file exists.
Example: ln file 1 file2
$# Total number of positional parameters.
$? exit status of the last executed command
62.
63.
64.
Hi can anyone tell me what happens when the graph run? i.e The Cooperating System will be at the host, We are running the graph at some other
place. How the Co-operating System interprets with Native OS?
when ever you press Run button on your GDE,the GDE genarates a script
and the genarated script will be transfered to your host which is specified in to your GDE run settings.
then the Co>operating system interprets this script and executes the script on different mechins(if
required) as a sub process(threads),after compleation of each sub process,these sub_processes will
return status code to main process this main process in tern returns error or sucess code of the job to
GDE
65.
One of the main purpose of the parameterized graphs is that if we need to run the same
graph for n number of times for different files, we set up the graph parameters like $INPUT_FILE,
$OUTPUT_FILE etc and we supply the values for these in the Edit>parameters.These parameters are
substituted during the run time. we can set different types of parameters like positional, keyword, local
etc.
The idea here is, instead of maintaining different versions of the same graph, we can maintain one
version for different files.
66. What are the most commonly used components in a Abinition graph?
can anybody give me a practical example of a trasformation of data, say customer
data in a credit card company into meaningful output based on business rules?
The most commonly used components in to any Ab Initio project are
input file/output file
input table/output table
lookup file
reformat,gather,join,runsql,join with db,compress components,sort,trash,partition by expression,partition
by key ,concatinate
66.
There are somany ways to improve the performance of the graphs in Abinitio.
I have few points from my side.
1.Use MFS system using Partion by Round by robin.
2.If needed use lookup local than lookup when there is a large data.
3.Takeout unnecessary components like filter by exp instead provide them in reformat/Join/Rollup.
4.Use gather instead of concatenate.
5.Tune Max_core for Optional performance.
6.Try to avoid more phases
67.
68.
The graph paramaters are one which are added to the respective graph. You can added the graph
parameters by selecting the edit>parameters from the menu tab. Here's the example for the graph
parameters.
If you want to run a same graph for n number of files in a directory, You can assign a graph parameter to
the input file name and you can supply the paramter value from the script before invoking the graph.
There are 2 types of graph parameters in AbInitio
1. local parameter
2. Formal parameters.(those parameters working at runtime)
69.
Please let me know whether we have abinitio GDE version 1.14 and what is
the latest GDE version and Co-op version?
Ab Initio GDE 1.14.7 is the latest version with some of the additional features which replaces the DTM
plans in the older versions
70.
API and UTILITY are the two possible interfaces to connect to the databases to perform certain
user specific tasks. These interfaces allows the user to access or use certain functions (provided by the
database vendor) to perform operation on the databases.The functionality of each of these interfaces
depends on the databases.
API has more flexibility but often considered as a slower process as compared to UTILITY mode. Well the
trade off is their performance and usage
71.
There are lot many ways to handle the DMLs which changes dynamically with in a single file. Some of
the suitable methods are to use a conditional DML or to call the vector functionality while calling the DMLs
72.
Environemental variables server as global variables in unix envrionment. They are used for passing on
values from a shell/ process to another.
They are inherited by Abinitio as sandbox variables/ graph parameters like
AI_SORT_MAX_CORE
AI_HOME
AI_SERIAL
AI_MFS etc.
To know what all variables exist, in your unix shell, find out the naming convention and type a command
like "env | grep AI". This will provide you a list of all the variables set in the shell. You can refer to the
graph parameters/ components to see how these variables are used inside Abinitio.
73.
74.
What is the difference between sandbox and EME, can we perform checkin
and checkout through sandbox/ Can anybody explain checkin and checkout?
The project is under source control under EME. You can't do any manipulations on the
project which is in EME. So if you want to do any manipulations you have to check out the project into
your sandbox and do whatever manipulations you want and for updating the project you have to checkin
the project.
You have to set up one working area in your system to work on the project, that working area is your
sandbox.
75.
i hope abinitio testing is to be carried manually only no automation is available as of now.we need to
integrate the graphs n also run them manually n carry on the process.thats it.
76.
Load multiple files from my perspective means writing into more than one file at a time. If this is the
same case with you, Ab initio provides a component called Write Multiplefiles (in dataset Component group)
which can write multiple files at a time. But the files which are to be written must be local files i.e., they should
reside in your local PC. For more information on this component read in help file.
77.
How to execute the graph from start to end stages? Tell me and how to run
graph in non-Abinitio system?
there are so many ways to do this, i am giving one example due to time constraint
you can run components according to phasea how you defined.
by creating ksh, sh scripts also you can run,
78.
data mapping deals with the transformation of the extracted data at FIELD level i.e.
the transformation of the source field to target field is specified by the mapping defined on the target field.
The data mapping is specified during the cleansing of the data to be loaded.
For Example:
source;
string(35) name = "Siva Krishna
";
target;
string("01") nm=NULL("");/*(maximum length is string(35))*/
Then we can have a mapping like:
Straight move.Trim the leading or trailing spaces.
The above mapping specifies the transformation of the field nm.
79.
you can change your .profile file to include any commands that you want to execute whenever u logging
in.you can even put commands in your .profile file that overrides settings made in /etc/profile(this file is
set up by the system adminiastrator).
you can set the following in your .profile......
- Environment settings
- aliases
- path variables
- name and size of your history file
- primary and secondary command prompts..
................and many more.
if u got any more information related to this topic, pls let me know..