Professional Documents
Culture Documents
Pushdown Optimization which is a new concept in Informatica PowerCentre, allows developers to balance data transformation
load among servers. This article describes pushdown techniques.
Pushdown Optimization
Pushdown optimization is a way of load-balancing among servers in order to achieve optimal
performance. Veteran ETL developers often come across issues when they need to determine the
appropriate place to perform ETL logic. Suppose an ETL logic needs to filter out data based on some
condition. One can either do it in database by using WHERE condition in the SQL query or inside
Informatica by using Informatica Filter transformation. Sometimes, we can even "push" some
transformation logic to the target database instead of doing it in the source side (Especially in the case of
EL-T rather than ETL). Such optimization is crucial for overall ETL performance.
Suppose a mapping contains a Filter transformation that filters out all employees except those with a DEPTNO greater than 40.
The Integration Service can push the transformation logic to the database. It generates the following SQL statement to process
the transformation logic:
The Integration Service generates an INSERT SELECT statement and it filters the data using a WHERE clause. The Integration
Service does not extract data from the database at this time.
The Integration Service pushes as much transformation logic as possible to the source database. The Integration Service
analyzes the mapping from the source to the target or until it reaches a downstream transformation it cannot push to the source
database and executes the corresponding SELECT statement.
The Integration Service pushes as much transformation logic as possible to the target database. The Integration Service
analyzes the mapping from the target to the source or until it reaches an upstream transformation it cannot push to the target
database. It generates an INSERT, DELETE, or UPDATE statement based on the transformation logic for each transformation it
can push to the database and executes the DML.
The Integration Service pushes as much transformation logic as possible to both source and target databases. If you configure
a session for full pushdown optimization, and the Integration Service cannot push all the transformation logic to the database, it
performs source-side or target-side pushdown optimization instead. Also the source and target must be on the same
database. The Integration Service analyzes the mapping starting with the source and analyzes each transformation in the
pipeline until it analyzes the target. When it can push all transformation logic to the database, it generates an INSERT SELECT
statement to run on the database. The statement incorporates transformation logic from all the transformations in the mapping.
If the Integration Service can push only part of the transformation logic to the database, it does not fail the session, it pushes as
much transformation logic to the source and target database as possible and then processes the remaining transformation
logic.
The Rank transformation cannot be pushed to the database. If the session is configured for full pushdown optimization,
the Integration Service pushes the Source Qualifier transformation and the Aggregator transformation to the source, processes
the Rank transformation, and pushes the Expression transformation and target to the target database.
When we use pushdown optimization, the Integration Service converts the expression in the transformation or in the workflow
link by determining equivalent operators, variables, and functions in the database. If there is no equivalent operator, variable, or
function, the Integration Service itself processes the transformation logic. The Integration Service logs a message in the
workflow log and the Pushdown Optimization Viewer when it cannot push an expression to the database. Use the message to
determine the reason why it could not push the expression to the database.
1. To push Sequence Generator transformation logic to a database, we must configure the session for pushdown optimization
with Sequence.
2. To enable the Integration Service to create the view objects in the database we must configure the session for pushdown
optimization with View.
2. After the database transaction completes, the Integration Service drops sequence and view objects created for pushdown
optimization.
When we use native drivers, the Integration Service generates SQL statements using native database SQL. When we use
ODBC drivers, the Integration Service generates SQL statements using ANSI SQL. The Integration Service can generate more
functions when it generates SQL statements using native language instead of ANSI SQL.
When the Integration Service runs a session configured for full pushdown optimization and an error occurs, the database
handles the errors. When the database handles errors, the Integration Service does not write reject rows to the reject file.
If we configure a session for full pushdown optimization and the session fails, the Integration Service cannot perform
incremental recovery because the database processes the transformations. Instead, the database rolls back the transactions.
If the database server fails, it rolls back transactions when it restarts. If the Integration Service fails, the database server rolls
back the transaction.
Links
dwbiconcepts.com/…/21-pushdown-op… 3/4
2/9/2011 Pushdown Optimization In Informatica
Comparing Performance of SORT operation (Order By) in Informatica and Oracle
Informatica Reject File - How to Identify rejection reason
Implementing Informatica Incremental Aggregation
Using Informatica Normalizer Transformation
Informatica Dynamic Lookup Cache
Informatica Tuning - Step by Step Approach
Implementing Informatica Partitions
Implementing Informatica Persistent Cache
Aggregation with out Informatica Aggregator
dwbiconcepts.com/…/21-pushdown-op… 4/4