(c) Infinity

Preparation Phase

Access Grants Assessment of Current Infrastructure Inventory of Current Setup Meta-Data Backup

New Cluster Setup with Hadoop 2.x . OR Inline Migration to Hadoop 2.x from Hadoop 1.x Methodology

Staging Environment

Replica of Current Setup in Staging Environment Upgrade in Staging Environment Validation of Migration in Staging Environment

Production Rollout

Upgrade of Production Environment Production Environment validation.

(c) Infinity

Upgrade a cluster by configuring a new cluster with same capacity and newer Hadoop version and then migrate the files/data from old cluster to new one.

(c) Infinity

CopyToLocal /CopyFromLocal

The process flows, the files are copied to local drive using Hadoop command CopyToLocal and the files are then pushed to the new cluster using CopyFromLocal, and the older cluster can be decommissioned.

Using Hadoop CP command

This is a kind of cluster to cluster copy, using Hadoop cp command the files are transferred from one HDFS to other HDFS. As the version are different we need a mechanism called copy from HFTP where the command is executed from the target cluster by defining source as old cluster with HFTP protocol and target as HDFS protocol.

Using Hadoop DISTCP command

This is a kind of cluster to cluster copy, using Hadoop distcp command the files are transferred from one HDFS to other HDFS. As the version are different we need a mechanism called copy from HFTP where the command is executed from the target cluster by defining source as old cluster with HFTP protocol and target as HDFS protocol.

(c) Infinity

 Upgrading

existing cluster from V1 to V2 inline by Installing/Configuring new cluster and updating metadata.
Hadoop V2

Hadoop V1

HDFS Upgraded Metadata HDFS

(c) Infinity

Shutdown of Cluster/Node s

Validation of Upgrade Activity

Production Rollout

Software Stack Stage

Upgrade Phase

(c) Infinity

 http://wiki.apache.org/hadoop/Hadoop_Upgrade

(c) Infinity

me mail @ dwivedishashwat@gmail.com my blog http://helpmetocode.blogspot.com

Visit

Drop

(c) Infinity