INTEGRATION SERVICES (IS)

In this Presentation

What is Integration Services? Data integration without SSIS Data integration with SSIS Architecture of SSIS Packages and SSIS Components Deployment of SSIS packages SQL Management Studio and SSIS Log Data for Troubleshooting Overview of SSIS security Sample SSIS Layout Summary

What is Integration Services?

SQL Server Integration Services (SSIS) is one of the most powerful features in SQL Server 2005. It is technically classified as a business intelligence feature and is a robust way to load data and perform tasks in a workflow. Even though it's mainly used for data loads, you can use it to do other tasks in a workflow like executing a program or a script, or it can be extended. We make use of Business Intelligence Development Studio to work with SSIS and integrate data. Integration Services Service is required for working with SSIS components.

Data integration without SSIS

Alerts & escalation

Call centre data: semi structured

Text Mining

ETL
Staging

Data mining

Staging Legacy data: binary files Hand coding

ETL

Warehouse

Cleansing & ETL

Application database

Reports Mobile data

ETL

Staging

Preparation of data requires different, often incompatible, tools. Reporting and escalation isa slow process, delaying smart responses.

Data Integration with SSIS

Alerts & escalation

Call centre: semi-structured data

Text mining components

Data mining components

Mobile data

Custom source
Legacy data: binary files

Merges
Warehouse

Standard sources
Application database

Data cleansing components

Reports

SQL Server Integration Services

Integration and warehousing are a seamless, manageable, operation. Source, prepare and load data in a single, auditable process.

SSIS Architecture

Package is an executable unit of work. Tasks are specific operations that are to be performed by SSIS. Tasks can be grouped into Containers. Data Flow is from one task to another task / container. Data Flow is managed by Data Flow Engine.

Package

Package is the basic unit of work, deployment, and execution. It is an organized collection of:

Package can be designed graphically or built programmatically Can be saved in XML format to the file system or SQL Server

Connection managers Control flow components Data flow components Variables Event handlers Configurations

Connection Manager

Logical representation of a connection Stored in the package and cannot be shared between packages used by package elements. Does not need to connect to SQL Server.

Control Flow

Control flow is a process-oriented workflow engine A package contains a single control flow Control flow elements

Containers Tasks Precedence constraints Variables

Containers

Provide structure and services for

Grouping tasks Implementing repeating flows

Execute in sequence defined by precedence constraints in control flow Manage variable and transactional boundaries

Tasks

Perform discrete operations at runtime Execute in sequence defined by precedence constraints in control flow Use properties configured once at design time or assigned dynamically at runtime by using expressions

Data Flow Task

SSIS Expressions

Deploying SSIS packages

Designer can build a deployment file set which includes a projects package(s), configuration files and installer files You move the installer file set to another server\environment and execute, to install packages to SQL or the File system. Installer can copy miscellaneous files but will not install them, such as custom component.

Design Package Bi Studio Add Configurations Add Miscellaneous files Set Project Deployment properties Build Project

Deployment Flow

Copy/Move Deployment folder\files You

Execute manifest file Installation Wizard Choose Destination (SQL File System) Modify protection level Choose location of supporting files Change configurations

Create desired agent jobs SQL Agent

SSMS and SSIS

Requires the SSIS service Allows Monitoring and Stopping of currently Executing packages Maintain stored package structure and set roles for SQL stored packages You can connect\view multiple SSIS servers at one time Adhoc Package execution from folder tree

Sample SSIS Layout

Packages on file system SSIS packages stored in SQL SSIS Package Execution Source data

SSIS package Logging

SSIS package error rows

SSIS support Servers

Destination data

S o u r c e F la t F ile s

Summary

SQL Server Integration Services is an exceptionally high performance integration and transformation tool Some processes benefit more from parallelism, some from memory 32-bit performance and scale can be increased by parallelism 64-bit enables highly scalable memory operations