Data Migration: Data migration is the process of making a copy of data and moving it from one device or system

to another, preferably without disrupting or disabling active business processing. After data is transferred, processing uses the new device or system. Data migration is the process of shifting the whole production data from one database to other new database. This process might be done by using data migration tool which is developed as per the data migration requirements. Reasons why enterprise goes for Data migration process includes: Deployment of a new operating or application system. Server or storage technology replacement or upgrade. Server or storage consolidation. Changes to database schema and structure. Relocation of the data center. Server or storage equipment maintenance. Workload balancing or other performance-related tuning. In most organizations, data migrations are routine operations, so they have to be done effectively and in a controlled environment to assure data integrity, compatibility, low downtime, and security as well as a seamless migration process. Effective data migration strategies: Many organizations migrate data as part of the process of upgrading controls, systems, or storage. Enterprises need to minimize the business impacts of data migration - downtime, data integrity issues, costs, control problems, and so on. Specific operational goals of effective data migration: There are specific goals associated with implementing an effective data migration strategy. Primarily,data must be migrated from the source platform to the target platform completely and accurately, and according to company and regulatory policies on information controls and security. This means no dropped or incomplete records, and no data fields that fail validation or other quality controls in the target environment.

Another goal of data migration is that the process be done quickly, with as short a downtime window as possible (given the enterprise's Maintenance Time Objective targets.) Finally, the cost of data migration must be manageable, in terms of technology and staff requirements.

There are many metrics that can measure the effectiveness and efficiency of data migrations: Number of on line customizations required. Percentage of migrated records. Percentage of migrated tables. Percentage of migrated records by technology. Percentage of migrated records by application. Percentage of data with quality problems. Number of migration errors. Migration impact on database size. Downtime due to migration. Required staging storage / hardware. Percentage of reconciliation errors. Percentage of cleansed data. Testing Phase in Data Migration: Each type of system may replace and/or enhance the functionality currently delivered by one or more legacy systems. Regardless of the type of project, some data conversion must take place. Complications arise when we attempt to take the information currently maintained by the legacy system and transform it to fit into the new system. We refer to the building of this bridge as data migration. Data migration is a common element among most system implementations. It can be performed once, as with a legacy system redesign, or may be an ongoing process as in a data warehouse. Even some legacy system migrations require ongoing data conversion if the incoming data requires continual cleansing. During this data migration process there may be possibility of data missing or data corruption. So after the data migration process one has to test whether the whole data is migrated successfully or not, whether something is missing or gets corrupted during migration.

Data Migration Testing Life Cycle The following are the steps to be followed in the process of Data Migration Testing: Data Migration Requirement Analysis. Test Strategy. Creation of POC. Test Planning. Design of Automated Test Scripts. Test Environment Setup. Test Execution. Result and Defect Analysis.

Checklist for Data Migration Testing: We can check for the table row count. We have to check database integrity. We have to check the referential integrity. Whether the data are loaded in the target database correctly. Check the constraints. All the entities are transformed or not like tables, Stored Procedure's, Views, functions etc.., Check the functionality of your application with target database, it should work as it is supposed to work. When we are speaking database migration testing. We can say ETL (Extract, transform,Loading) testing. Database migration is extracting the database from source and loads the data to target database. Two types of Loading: Direct Loading. Incremental Loading.

Direct Loading: The direct loading is nothing copy the data from source to target. The number of rows should be equal source and target.

Incremental Loading: The Incremental loading, when loading the data from source to target, applying some transformations. Cleaning the data or load the data with specific conditions. Challenge is two different databases have their own structures so just simple table/record import export function will not work. We need a tool (say ETL) in between source database and destination database where we will write the transformation logic, done by the Dev team. Testing team will validate that all those business rules in ETL are actually working in migration testing. Testing options for Data Migration: Pre-migration testing: These tests occur early in the migration process, before any migration and even after migration for testing is been completed. Verify scope of source systems, it should include data as well as excluded data and corresponding specific queries for migrations. Define the source to target high-level mappings for each category of data or content and verify that the desired type has been defined in the destination system. Verify destination system data requirements for ex, field names, type, mandatory fields and valid values lists and other field-level validation checks. Testing the source data against the requirements of the destination system, for suppose if the destination system has a mandatory field, ensure that the corresponding source is not null. If the destination system field contains list of valid values and make sure that the corresponding source field contains these valid values. Test the fields that uniquely link source and target records and ensure that there is a definitive mapping between the record sets. For ex: Joins concept implementation in both source and destination data bases. Test tool configuration against the migration specification which can be done via black box testing on a field-by-field basis.

Post-migration testing: Once a migration has been executed, additional end to end testing can be executed. Post-migration is typically performed in a test environment.

Test the throughput of the migration process (number of records per unit time). This testing will be used to verify that the planned downtime is sufficient. For planning purposes, consider the time to verify that the migration process was completed successfully.

Compare Migrated Records to Records Generated by the Destination System Ensure that migrated records are complete and of the appropriate context. Summary Verification There are several techniques that provide summary information including record counts and checksums. Here, the number of records migrated is compiled from the destination system and then compared to the number of records migrated. This approach provides only summary information and if any issue exists, it does not often provide insight to an issues root cause.

Compare Migrated Records to Sources Tests should verify that fields values are migrated as per the migration specification. In short, source values and the field level mappings are used to calculate the expected results at the destination. This testing can be completed using sampling if appropriate or if the migration includes data that poses significant business or compliance risk, 100% of the migrated data can be verified using an automated testing tool.

User Acceptance Testing: Functional subtleties related to the co-mingling of migrated data and data created in the destination system may be difficult to identify early in the migration process. User acceptance testing provides an opportunity for the user community to interact with legacy data in the destination system prior to production release, and most often, this is the first such opportunity for the users. Attention should be given to reporting, downstream feeds, and other system processes that rely on migrated data.

Types of Migration Migration of data from one database to another can be further classified into various subsets. a. Direct Migration from the source database to the target: In this type of migration the data elements that are present in the source database is directly transferred into the target database b. Migration after performing certain transformations on the source data: This migration requires some transformation of the source data before it is migrated to the new database. For ex: a field Priority in the source database may allow only one out of the three values High, Medium and Low. But in the target database the same field allows one out of the three values 1, 2 and 3. Thus the a High value in the source database will be transformed to a value 1 and will then be migrated to the target database. These are the migrations which were encountered during testing of data migration project(s): Change in data type. For ex: Data in the source table may be present as a character, but in the target table the mapped column is an integer. Concatenation of two or more columns. Mathematical transformations (ex: summing up of fields etc..,) Migration of values depending on other fields/tables present in the source database. c. Target Generated Data: This is the data that is generated at the target database. These values will not be present the source database but will be generated each time an insertion or an update takes place on a table. These columns generally are the primary key of a table. d. Reference at target system generated. This type of migration deals with data that has to comply with referential integrity constraints. This data is generated at the new database but the data comes from the parent attribute.

A reference key in the target database is usually encountered when a table in the source database is split up into two or more tables in the target. e. Manual setup data in the target database. Certain tables have to be populated before the migration takes place. These tables can be termed as Master Tables. These tables are the parent tables to the various referential constraints that are to be implemented in the child tables. The data for these tables may not come from the source database but will be setup manually in the target database.