You are on page 1of 14

Cisco Systems, Inc.

<Measure/Subject Area/Project Name/Application Name>


ETL Specification Document
Author: Creation Date: Last Updated: Version: <Project Team Name> <date> <date> <version #>

1. Document Control
1.1. Revision History Date Version Number Author Details Of Change

1.2. Document Reviewers Date Reviewers Name Reviewers Job Title

1.3. Document Approvers Date Approvers Name Approvers Job Title

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 2 of 14

Table of Contents
<Any change to this ETL design document should be followed up by an update to the TOC field below. To update: right-click anywhere in the TOC field and select Update Field, then select Entire Table and click ok.>

1. Document Control..................................................................................................................................2 1.1. Revision History..........................................................................................................................2 1.2. Document Reviewers..................................................................................................................2 1.3. Document Approvers..................................................................................................................2 2. High level overview................................................................................................................................4 2.1. Purpose / Overview.....................................................................................................................4 2.2. ETL / Technical Architecture.......................................................................................................4 3. Specification Details..............................................................................................................................5 3.1. Table / View Structures..............................................................................................................5 Table Name: Table 1...........................................................................................................5 Table Name: Table 2...........................................................................................................5 Source Table Name: Table 1...............................................................................................6 Source Table Name: Table 2...............................................................................................6 Target Table Name: Table 3................................................................................................6 Target Table Name: Table 4................................................................................................7 3.2. Program List................................................................................................................................7 <Provide the name of the package/program/mapping>........................................................8
Subprogram 1 <Provide the name of the package/program >......................................................8 Subprogram 2 <Provide the name of the package/program>......................................................9 Source Qualifier Transformation 1 <Provide the transformation name>.......................................9 <Provide Transformation Type> Transformation 1 <Provide the transformation name>............10 Workflow 1 <Provide the name of the workflow >.......................................................................10 Session 1 <Provide the name of the session >...........................................................................10 Session 2 <Provide the name of the session >...........................................................................10

3.3. Detailed Column Mapping Specification....................................................................................11 3.4. Job Configuration and Scheduling Details................................................................................13 Job Name 1........................................................................................................................13 Job Name 2........................................................................................................................13 4. Appendix..............................................................................................................................................14 4.1. Livelink URL..............................................................................................................................14

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 3 of 14

2. High level overview


<The goal of this template is to capture all data movement and transformation occuring within a given measure, subject area, project, or application name. It is recommended to create and maintain a single document to capture all data movement and transformation occuring within a category mentioned above.>

2.1. Purpose / Overview


The purpose of this document is to record and communicate the ETL design for the <Measure/Subject Area/Project/Application Name> to be consumed by the IT and Business users. It explains the various components of the ETL design. The document includes the data flows as well as the outlines of the standards and guidelines that are followed during the development process.

2.2. ETL / Technical Architecture


<Include architecture information such as High Level System Flow or provide a link to where the diagram resides. Also included any detailed system flows or diagrams that would help provide a clearer picture of the data movement process.>

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 4 of 14

3. Specification Details
3.1. Table / View Structures
< Provide the list all the tables/views that are used in this measure/subject area/project.>
Database:

Schema: Last Revise Date:


Sizing rows Mb Key Type (PK/FK/U K) Current 3 mo 6 mo

Layer: By: 1 yr

Table Name: Table 1


Descript ion

IN DW

Column / Field

Format

Null

Description

Database:

Schema: Last Revise Date:


Sizing rows Mb Key Type (PK/FK/U K) Current 3 mo 6 mo

Layer:

Table Name: Table 2


Descript ion

By:
1 yr

IN DW

Column / Field

Format

Null

Description

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 5 of 14

< If identifying the table usage as Source, Target, Staging adds clarity to the ETL design, feel free to indicate that in the Table Name field as Source, Target, Staging, etc. However, if the table is used as a combination of table usage types, then using the more generic table specification template above would be preferred, and then clarify the table usage in the Program sections below..>

Database:

Schema: Last Revise Date:


Sizing rows Mb Key Type (PK/FK/U K) Current 3 mo 6 mo

Layer: By: 1 yr

Source Table Name: Table 1


Descript ion

IN DW

Column / Field

Format

Null

Description

Database:

Schema: Last Revise Date:


Sizing rows Mb Key Type (PK/FK/U K) Current 3 mo 6 mo

Layer:

Source Table Name: Table 2


Descript ion

By:
1 yr

IN DW

Column / Field

Format

Null

Description

Database:

Schema: Last Revise Date:


Sizing Current 3 mo 6 mo

Layer: By: 1 yr
3/2/2011 Page 6 of 14

Target Table Name: Table 3


Descript
Cisco Systems Inc. Confidential ETL Specification Document

ion

rows Mb Key Type (PK/FK/U K)

IN DW

Column / Field

Format

Null

Description

Database:

Schema: Last Revise Date:


Sizing rows Mb Key Type (PK/FK/U K) Current 3 mo 6 mo

Layer:

Target Table Name: Table 4


Descript ion

By:
1 yr

IN DW

Column / Field

Format

Null

Description

3.2. Program List


This section details the list of ETL programs for this measure/subject area/project. <Copy and past the following section along with at least 1 sub program section to document all programs as needed. Each program must have at least one sub program.>

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 7 of 14

Technology Used Program <Provide the name of the package/program/mapping>

< Indicate the ETL technology used to develop the program. Examples of ETL technologies may include Informatica, OWB, PL/SQL, etc. >

Description Program Level Filters Extraction Strategy Load Strategy

<Provide a textual description about what this package/program does. Also include any other diagrams or flow charts to help describe the ETL process.> <Describe any table/dataset level filters> <Provide the types of extraction being used by this program/package, i.e. Incremental or Complete. Describe in details if the extraction is incremental.> <Describe the types of load being used by this program/package, i.e. Insert, Update, or Delete. It may be a combination of these strategies. Is this an Incremental load versus a complete table rebuild? > <Describe the error strategy to be followed within the Extraction Strategy and Load Strategy.> Error handling within Extraction Strategy Error handling within Load Strategy

Error Strategy

<Copy and paste the following sections to describe the ETL process as needed. Sub programs sections are applicable to documenting procedural program design. Transformation sections have been provided for documenting ETL designs for implementation using Informatica (current ETL standard within the EDW environment. NOTE: Depending on your ETL design you may have either 1- All procedural sub programs 2- All Informatica transformatoins or 3- Mixture of procedural and Informatica ETL. The sub program and transformation templates below should be used according to your needs to document the ETL design. For example, if additional fields are required, then add them. Or, if your ETL design will be fully deployed with Informatica, then delete the sub program sections. Or, if you need to design a different type of transformation, such as an Aggregator transformation, copy the Transformation template below and tailor the fields according to the transformation being designed. > Sub Program Description Sources Subprogram 1 <Provide the name of the package/program > <Describe what this package/program does> <Source Table 1 Name> <Source Table 2 Name> <Source Table 3 Name>
3/2/2011 Page 8 of 14

Cisco Systems Inc. Confidential ETL Specification Document

Targets Sub Program Level Filters Table Join Condition DFD Reference BRD Reference Sub Program Description Sources Targets Sub Program Level Filters Table Join Condition DFD Reference BRD Reference

<Target Table 1 Name> <Target Table 2 Name> <Target Table 3 Name> <Describe any table/dataset level filters> <Describe the basic join information>

Subprogram 2 <Provide the name of the package/program> <Describe what this package/program does> <Source Table 1 Name> <Source Table 2 Name> <Source Table 3 Name> <Target Table 1 Name> <Target Table 2 Name> <Target Table 3 Name> <Describe any table/dataset level filters> <Describe the basic join information>

< Templates for documenting ETL designs to be implemented within Informatica. Only Source Qualifier and generic templates are provided for examples. Feel free to create new templates for different types of Transformations.> Transformation Table / Synonym / View Source Qualifier Transformation 1 <Provide the transformation name> <Provide the table, synonym, or view which the Source Qualifier references>

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 9 of 14

Filter(s)/Condition(s) SQL Override Transformation Field 1 Field 2

<Provide filters or join conditions which should be applied to the SQ query> <Provide psuedocode or the exact SQL which should drive this transformation>

<Provide Transformation Type> Transformation 1 <Provide the transformation name> <Field name required to clearly and accurately articulate the transformation design.> <Field name required to clearly and accurately articulate the transformation design.>

< This section is specific to Informatica and can be used to document ETL design implemented within Workflows and Sessions which uses the Mapping (Program) of this chapter. These sections will typically be used when a generic mapping has been defined which can be used by multiple Sessions along with special SQL filters or conditions defined in each session. > WorkFlow Description Session Targets Session Level Filters Session Targets Session Level Filters Workflow 1 <Provide the name of the workflow > < Provide a textual description about what this workflow does. Include the workflow diagram which consist of various tasks that are run concurrently or sequentially > Session 1 <Provide the name of the session >
Table Name Insert Update as Update Update as Insert Update else insert Delete Truncate Table

<Target table 1> <Target table 2> <Describe any table/dataset level filters> Session 2 <Provide the name of the session >
Table Name Insert Update as Update Update as Insert Update else insert Delete Truncate Table

<Target table 1> <Target table 2> <Describe any table/dataset level filters>

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 10 of 14

3.3. Detailed Column Mapping Specification


<Column Mapping with respect to process (Double click on the following spreadsheet to get to the details)>

Microsoft Excel Worksheet

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 11 of 14

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 12 of 14

3.4. Job Configuration and Scheduling Details


<This section provides details regarding how the backend jobs will be configured, which backend job will execute each ETL program within this design, and scheduling details. This section is assuming that the EDW standard job configuration (DW_JOBS tables) and job scheduling(Dollar Universe) environments are being used. If your environment uses a different job configuration or scheduling setup you may want to modify this section accordingly. Existing Job? Indicates if the following specifications are changes within an existing production job. Job Change Type further qualifies the job change type as a New, Modified, or Dropped change within the existing job. Job Change Type is not required for designing new backend jobs. >

Job Name

Job Name 1 DW Jobs Specifications


Prerequisites CONC Stream ETL program

Existing Job?

Dollar Universe Job Specifications


Job Change Type Session Name UPROC Name Dependencies

Scheduling Specs
Frequency Day/Time

Job Name

Job Name 2 DW Jobs Specifications


Prerequisites CONC Stream ETL program

Existing Job?

Dollar Universe Job Specifications


Job Change Type Session Name UPROC Name Dependencies

Frequency

Day/Time

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 13 of 14

4. Appendix

4.1. Livelink URL This latest version of this document can be found on LiveLink at:
http://ework.cisco.com/Livelink/livelink.exe?func=ll&objId=2248909&objAction=Open

Cisco Systems Inc. Confidential ETL Specification Document

3/2/2011 Page 14 of 14

You might also like