Reference Architecture Library

Reference Architecture Overview for Microsoft® Exchange Server 2007 with

Hitachi Adaptable Modular Storage Model AMS1000
White Paper

September 2007
Executive Summary
The release of Microsoft® Exchange Server 2007 incorporated significant improvements,
including changes to the Exchange workload profile and new options for data protection that
influence the underlying storage infrastructure. This technical brief provides an overview of the
ways some new features of Exchange Server 2007 impact the design and implementation of a
robust storage infrastructure.

The technical brief is intended to serve as a companion to the initial Hitachi Data Systems
submission to the Exchange Solution Reviewed Program (ESRP) for Exchange Server 2007,
and it describes solutions based on the Hitachi Adaptable Modular Storage model AMS1000
configuration tested for a range of 3,125 to 25,000 Exchange mailboxes. The paper discusses
how data protection alternatives can affect performance and introduces a recommended
building-block approach for designing and architecting Exchange Server 2007 storage
infrastructures on the AMS1000.

Specifically, the paper describes the reference test environment that was used during the
development of the ESRP submission. It also provides an overview of some of the
architectures available to provide varying degrees of Exchange 2007 data protection. In
addition, the paper provides insight into the various hardware, software, and service
components from Hitachi Data Systems required to implement a protected Exchange Server
2007 solution on the AMS1000 and address economy, performance, and growth.

Although the recommendations documented here may generally represent good practices,
configurations may vary. Please contact your Hitachi Data Systems representative, or visit
Hitachi Data Systems online at http://www.hds.com for further information on solutions by
Hitachi Data Systems.
Contents
Perform an ce a nd Dat a Prot ec tion for Mi crosoft E xch a nge Serv er 2007 ................................................... 1

Hitachi Adaptable Modular Storage Model AMS1000 for Exchange............................................................................... 1

AMS1000 and Exchange Server 2007 ESRP Performance Solution Profile.................................................................... 2

Reference ESRP Test Topology ....................................................................................................................................... 3

Reference ESRP Test Environment Specifications........................................................................................................... 3

Reference ESRP Test Storage Configurations ................................................................................................................. 4

Reference ESRP Test Summary Results.......................................................................................................................... 8

Exch ang e Data Prote ction .................................................................................................................... 9

Performance Planning for Backup and Replication.......................................................................................................... 9

Designing for Performance and Protection .................................................................................................................... 11

Key Takeaways ............................................................................................................................................................... 12

Storag e- bas ed Prote ction for Exc hang e .............................................................................................. 13

Adaptable Modular Storage model AMS1000: Hitachi Data Systems Protected Architectures for Exchange Server
2007................................................................................................................................................................................ 13

Examples of Building-block-driven Hitachi Data Systems Protected Solution Architectures ........................................ 16

Sum mar y ........................................................................................................................................... 22

Ap p endi x: Sup pl eme nt al E xch ang e S erv er 2007 Infor mation ................................................................. 23
Reference Architecture Library
Reference Architecture Overview for Microsoft® Exchange Server 2007
Hitachi Adaptable Modular Storage Model AMS1000
White Paper

Performance and Data Protection for Microsoft Exchange Server 2007

Microsoft Exchange Server 2007 changed the way that Exchange servers and their underlying storage interact.
Since the release of Exchange 2003, servers have increased in both power and the ability to affordably support
large amounts of memory. Microsoft took this into consideration when designing Exchange Server 2007, using 64-
bit processors with large internal cache memory as the new required platform. These changes led to a reduction in
the I/O workload on the disk storage system since more operations are now cached in server memory, resulting in
fewer I/Os to physical storage. These enhancements, along with other fundamental changes to the data
architectures of Exchange, improve performance and availability while minimizing unnecessary expense.

Hitachi Adaptable Modular Storage Model AMS1000 for Exchange

With those objectives in mind, the initial Hitachi Data Systems submission to the Microsoft Exchange Solution
Reviewed Program (ESRP) used the AMS1000 as the foundation for the solution. The ESRP submission can be
found at http://technet.microsoft.com/en-us/exchange/bb412164.aspx. The AMS1000’s high-availability
characteristics and its ability to scale easily to support a large number of active Exchange users make it an ideal
platform for a wide range of Exchange solutions. Microsoft offers the Exchange Solution Reviewed Program to
help customers and partners design the right environment for their requirements. Visit the abovementioned Web
page for additional information about this valuable program provided by the Exchange product group.

Fundamental Storage Considerations for Exchange Server 2007

Capacity, performance, and available budget are often at odds with each other when it comes to selecting a
storage solution for an Exchange environment. Even with performance improvements, Exchange Server 2007 still
places a significant demand on the underlying storage infrastructure, and this frequently translates into the need
for more physical disk resources to provide acceptable response times and the highest availability. Generally, the
design decisions for Exchange Server 2007 involve the following factors:

• Making sure there is enough space to store all of the data

• Making sure the solution provides an acceptable user experience as measured by response times

• Making certain that all critical Exchange components are properly protected to ensure operations resume
following an outage, disaster, or other event that leads to service disruption

The architect’s goal is to balance the above factors when designing an Exchange Server 2007 infrastructure that
addresses cost while improving availability.

1
The Exchange Server 2007 reference architecture overview that follows is designed to accommodate a scalable
Exchange Server 2007 deployment with a range of Hitachi Data Systems data protection options. It is strongly
recommended that the Microsoft documents listed in the Appendix be referenced when implementing or migrating
to a Microsoft Exchange Server 2007 environment.

AMS1000 and Exchange Server 2007 ESRP Performance Solution Profile

The initial task was to develop and test a scalable, high-performance platform for Microsoft Exchange Server
2007. Exchange Server 2007 presents two distinct workload patterns to storage: a random workload consisting of
8KB read/write operations to the databases, and a sequential write workload of varying record size (between 512
bytes and the log buffer size) to the transaction logs. A key for performance design is an expandable storage
architecture that supports varying I/O workloads while allowing for the preferred method of data protection. The
AMS1000-based solution targets organizations that are seeking to consolidate their Exchange Server 2007
storage on a high-performance, high-reliability storage system that offers a range of data protection options.

The tested ESRP configuration was designed to support up to 25,000 Exchange users in a modular, building-
block fashion, using the following profile specifications:

• Up to 8 Exchange Servers

• 3,125 users per Exchange Server

• 0.48 IOPS per user

• 200MB mailbox size

• 5 storage groups per server

• 1 mailbox store per storage group

• 625 mailboxes per mailbox store

2
Reference ESRP Test Topology
The following diagram shows the hardware components used for the AMS1000 ESRP submission.

Figure 1. Hardware Components for AMS1000 ESRP Submission

IBM® BladeCenter Hitachi Adaptable Modular

Two (2) HS20 Blades Storage AMS1000

IBM BladeCenter
Two (2) HS20 Blades
Fibre Channel
SAN

IBM BladeCenter
Two (2) HS20 Blades

IBM BladeCenter
Two (2) HS20 Blades

Reference ESRP Test Environment Specifications

The tested configuration is capable of supporting 25,000 users with a 0.48 IOPS per user profile and user mailbox
size of 200MB. The guiding design philosophy of this solution is modularity, so that this configuration can be
scaled in both directions, down to a minimum of 3,125 users, or up to 25,000 users. This also provides
organizations with the ability to expand an Exchange Server 2007 storage configuration incrementally by deploying
disk trays and disks as needed. See AMS1000 ESRP Submissions for additional information and details about this
and other Hitachi Data Systems submissions to the ESRP program.

The following tables summarize the testing environment:

Table 1. ESRP Test Environment Specificatons—Simulated Exchange Configuration

Simulated Exchange Configuration

Number of Exchange mailboxes simulated 25,000

Number of hosts 8

Number of mailboxes/host 3,125

Number of storage groups/host 5

Number of mailbox stores/storage group 1

Number of mailboxes/mailbox store 625

Number of mailbox store LUNs/storage group 1

Simulated profile: I/Os per second per mailbox 0.48

(IOPS, include 20 percent headroom)

Database LUN size 267GB

Log LUN size 30GB

Total database size for performance testing 5,468GB

Percentage of storage capacity used by Exchange database** 51 percent

3
Table 2. ESRP Test Environment Specifications—Primary Storage Hardware

Primary Storage Hardware

Storage Connectivity (Fibre Channel, server attached storage (SAS), Fibre Channel
SATA, iSCSI)

Storage model and OS/firmware revision Hitachi AMS1000—0760/B-H

Storage cache 16GB

Number of storage controllers 2

Number of storage ports 8

Maximum bandwidth of storage connectivity to host 32Gbit [8x4Gbit host bus adapter (HBA)]

Switch type/model/firmware revision Brocade 4 Port Enterprise Fibre Channel Switch for IBM®
BladeCenter®—v4.4.1a

HBA model and firmware QLogic QMC 2462S BIOS 1.47, FW 4.00.23

Number of HBAs per host 2 HBAs per host

Host server type IBM HS20 Blade 2X3.2GHz Intel Xeon Processor—8GB RAM

Total number of disks tested in solution 200

Maximum number of spindles can be hosted in the storage 480

Reference ESRP Test Storage Configurations

Table 3. ESRP Test Storage Configurations—Mailbox Store Disks

Primary Storage Disk Configuration (Mailbox Store Disks)

Disk type, speed, and firmware revision Fibre Channel disk 146GB 15K 4C56

Raw capacity per disk (GB) 146GB

Number of physical disks in test 160

Total raw storage capacity (GB) 23,360GB

Disk slice size (GB) N/A

Number of slices per LUN or number of disks per LUN 4 disks per LUN

RAID level RAID-1+0 at storage level

Total formatted capacity 10,640GB

Storage capacity utilization 46 percent

Database capacity utilization 52 percent

4
Table 4. ESRP Test Storage Configurations—Transaction Log Disks

Primary Storage Disk Configuration (Transaction Log Disks)

Disk type, speed, and firmware revision Fibre Channel disk 146GB 15K 4C56

Raw capacity per disk (GB) 146GB

Number of spindles in test 40

Total raw storage capacity (GB) 5,840GB

Disk slice size (GB) N/A

Number of slices per LUN or number of disks per LUN 2 disks per LUN

RAID level RAID-1 at storage level

Total formatted capacity 1200GB

The scalable design of the storage architecture for the AMS1000-based solution was shaped by the concept of
building blocks, which produced the storage module configuration. A storage module, which is described in Table 5,
is used as a fundamental element on which the protected solution architectures (described later) are based.

Table 5. ESRP Test Storage Configurations—Single Storage Module Configuration

Single Storage Module Configuration

Number of supported users 3,125

Number of hosts 1

IOPS per user 0.48

Mailbox size 200MB

Number of storage groups 5

Number of disk trays 2

Total number of disks 27

Number of database disks 20

Number of log disks 6

Number of spare disks 1

Disk type 146GB 15K RPM Fibre Channel

Database RAID level 1+0

Log RAID level 1

Figure 2 shows a graphical illustration of the physical disk allocation for a single storage module configuration.

5
Figure 2. Single Storage Module

Hitachi Adaptable Modular Storage AMS1000

Single Module (1 Host)

RKA 2

RKA 1

DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5
Log SG2 Log SG4

For the targeted 25,000-user Exchange environment, eight storage modules are stacked together with a
dedicated storage module allocated to each of the eight host servers. Figure 3 depicts the AMS1000
configuration, scaled to support 25,000 mailboxes by using single storage modules in a building-block fashion.

6
Figure 3. Eight Storage Modules

Hitachi Adaptable Modular Storage AMS1000

Eight Modules (8 Hosts)

RKA 16
Host 8

RKA 15

RKA 14
Host 7

RKA 13

RKA 12
Host 6

RKA 11

RKA 10
Host 5

RKA 9

RKA 8
Host 4

RKA 7

RKA 6
Host 3

RKA 5

RKA 4
Host 2

RKA 3

RKA 2

Host 1

RKA 1

DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5
Log SG2 Log SG4

7
Reference ESRP Test Summary Results
The ESRP storage performance tests exercise storage with a maximum, sustainable Exchange I/O workload for
two hours using the Microsoft JetStress tool. The test is designed to show I/O response times for a storage
infrastructure under a constant, intensive Exchange Server 2007 emulated workload. The test results below are
the sum of all logical disk I/Os and average of all the logical disks’ I/O latency. The complete set of test results,
logs, and reports can be found in the AMS1000 ESRP Submission for Exchange Server 2007.

Individual Server ESRP Performance Metrics

The sum of I/Os and the average latency across all servers are provided in Table 6.

Table 6. Individual Server ESRP Performance Metrics

Blade 1 Blade 2 Blade 3 Blade 4 Blade 5 Blade 6 Blade 7 Blade 8

Database I/O

Average Database Disk Transfers/sec

1,814 1,874 1,829 1,851 1,851 1,836 1,845 1,805
(IOPS)

Average Database Disk Reads/sec

963 996 971 984 982 976 978 957
(IOPS)

Average Database Disk Writes/sec

851 878 858 867 869 860 867 848
(IOPS)

Average Database Disk Read Latency (ms) 17 16 16 16 16 16 16 16

Average Database Disk Write Latency (ms) 4 4 4 4 4 4 4 4

Transaction Log I/O

Average Log Disk Writes/sec (IOPS) 557 573 560 568 564 564 563 553

Average Log Disk Write Latency (ms) 0 0 1 0 1 1 1 0

Aggregate ESRP Performance Metrics Across all Servers

The sum of I/Os and the average latency across all servers in solution are included in Table 7.

Table 7. Aggregate ESRP Performance Metrics Across All Servers

Aggregated Performance for Exchange Server 2007

Database I/O

Average Database Disk Transfers/sec (IOPS) 14,707

Average Database Disk Reads/sec (IOPS) 7,808

Average Database Disk Writes/sec (IOPS) 6,899

Average Database Disk Read Latency (ms) 16

Average Database Disk Write Latency (ms) 4

Transaction Log I/O

Average Log Disk Writes/sec (IOPS) 4,504

Average Log Disk Write Latency (ms) 1

8
ESRP Performance Test Summary
Using the AMS1000 storage module building blocks, basic Exchange Server 2007 performance can easily scale
between 3,125 and 25,000 mailboxes. The total expected IOPS based on the ESRP profile is 1,500 for one
storage module (3,125 * 0.48) and 12,000 for eight modules (25,000 * 0.48), with an average response time of
<= 20ms for database read and writes. The average storage module in the ESRP submission supported 1,838
database IOPS (+338) with 16ms and 4ms average read and write latencies, well below the critical 20ms
benchmark established by Microsoft. As these latencies breach the 20ms+ levels, end-user experience begins
to degrade significantly. The average ESRP storage module maintains roughly 20 percent of IOPS and latency
headroom above the tested profile expected results. The scalability of the AMS1000 is validated as the same
headroom factor is maintained when eight storage modules are scaled out to support 25,000 mailboxes.
However, this built-in headroom may be needed to provide the resources necessary to support the workloads that
correspond to certain protection requirements. The potential impact on performance of data protection options is
covered in the following section.

Exchange Data Protection

Performance Planning for Backup and Replication
The performance summary in the preceding section showed results for a standalone environment with minimal
data protection. Adding backup and replication functions to any application solution, including Exchange Server
2007, requires effective planning for these processes. Backup and replication place demands on one or more
components of an Exchange Server 2007 infrastructure and require additional storage resources, regardless of the
deployed technology. Some of the associated costs, benefits, and requirements that accompany host and storage
based backup and replication solutions are summarized in Table 8.

Table 8. Table Host and Storage-based Backup/Replication Considerations

Added Workload
Host RAM to Production
and CPU Exchange Required RPO (Recovery RTO (Recovery
Method Requirements Storage Backup Window Point Objective) Time Objective)
Host-based Backup Medium to High Medium to High Hours Hours to Days Hours to Days

Host-based Replication Medium to High Low to High N/A1 Minutes to Days Minutes to Days2

Storage-based Backup/ Low Low to Medium Seconds to Minutes Hours to Days Minutes to Hours
Replication
1 Backup not performed, but can offload backup process on the passive replica
2 If restore is required

E-mail has become a critical resource for conducting business and is thus a high priority for any business
continuity solution. Therefore, it is important to understand the areas where tradeoffs may be sought as the overall
costs of the solution are evaluated against the business value provided.

9
Host-based Backup Software
Traditional host-based backup applications for Exchange Server 2007 are a perfectly acceptable protection
solution for some organizations, especially for small environments with less stringent data protection requirements.
By nature, the streaming backup process for Exchange Server 2007 demands increased host resources during
normal backup operations. Specifically, additional CPU cycles are required to perform the page checksum
process, and RAM and disk resources must be sufficient to support the intensive sequential read and write
workloads associated with the streaming backup.

The amount of time required to perform this type of backup (commonly known as the backup window) is much
greater than the time required for storage-based backup methods, which prohibits this type of solution from
delivering attractive recovery point objective (RPO) and recovery time objective (RTO) levels. However, tape
backups are a critical business requirement for many organizations and provide enhanced data protection when
combined with storage-based replication as we will see later in this section. The AMS1000 ESRP Submission
contains the complete details and test results for the AMS1000, which include metrics for the simulated streaming
backup tests performed by JetStress during the various tests. Hitachi Data Protection Suite, powered by
CommVault®, is described later in this document. It is a comprehensive backup solution that provides Microsoft’s
Volume Shadow Copy service–based (VSS) and streaming API backup functionality that can be applied in a variety
of designs to provide enhanced RPO and RTO objectives.

Host-based Replication Software—Including Microsoft LCR and CCR

Exchange Server 2007 introduces new replication and availability features that may affect overall storage design.
Local Continuous Replication (LCR) and Cluster Continuous Replication (CCR) enhance the recoverability of
Exchange 2007 by complementing, not replacing, backup solutions. Each technology provides varying levels of
protection by providing a passive copy for rapid recovery. However, a VSS-based or steaming API backup
solution is still required for LCR or CCR implementations where true data protection is required. Please refer to
Microsoft’s specific guidelines and limitations for LCR and CCR.

Local Continuous Replication

LCR provides functionality that maintains a second copy of a storage group locally on a standalone Exchange
2007 server, which can facilitate recovery when the second copy is manually activated. In essence, when
deployed using modern, high-availability storage systems from Hitachi Data Systems, the value of LCR is limited to
offloading the backup process to the second copy and potential recovery capabilities from some logical
corruptions. LCR uses the Exchange Server 2007 Replication Service, which requires additional server hardware
resources, specifically processor cycles and memory address space. Standalone Exchange 2007 servers
commonly have the Hub Transport role configured, in addition to the Mailbox role, which also increases the server
CPU and RAM requirements to accommodate the additional workload. The recommendations provided by
Microsoft for additional resources for CPU and RAM, and other server and storage hardware requirements, should
be followed.

Adding LCR on the AMS1000 entails allocating additional storage group storage modules (described previously)
for the passive copy. This satisfies two key storage recommendations for LCR-enabled storage groups:
• Isolate the physical disks for the active and passive copies

• Ensure that the disks allocated to the passive copies have the same performance capabilities as the disks used
to support the active copy

Cluster Continuous Replication

CCR leverages Exchange Server 2007 asynchronous log-shipping technology to replicate storage groups
between the active and passive nodes of a cluster. In addition, CCR is integrated into Microsoft Cluster Services to
facilitate Exchange service management and failover. While still resource intensive, many of the key processes and
their associated performance impacts occur on the passive Exchange node. The primary Exchange server does

10
incur a resource penalty due to the overhead of managing the replication service, a network share, and other
components. Because log shipping technologies inherently operate at a point in time that is behind the production
system, the RPO and RTO levels delivered through host-based replication, in most cases, cannot match those of
a zero-data-loss storage-based replication solution.

The storage architecture for CCR on the AMS1000 using the storage module design from the ESRP submission
entails allocating storage modules for the passive copy on an additional, often remotely located, AMS1000. This
architecture, combined with the storage group and mailbox store design of the AMS1000-based storage module,
ensures that the disks allocated to maintain the passive copies have the same performance and capacity
capabilities as the disks on which the active copy resides. Native host-based replication for Exchange Server 2007
is new Microsoft technology; testing of those new features was not conducted during the initial ESRP submission
for the AMS1000. However, one can use the ESRP results for the AMS1000 along with Microsoft’s replication
guidelines to estimate the performance impact in terms of IOPS, which may be significant for larger Exchange
environments.

Storage-based Backup/Replication
The performance impact associated with using storage-based replication technologies of the AMS1000 will vary
based on a number of factors outlined in detail in AMS1000 documentation. However, as a rule of thumb, expect
to incorporate a distance-related performance allowance into the design to account for the increased data
protection workload.

The allowance may need to be increased for some of the larger, higher IOPS Exchange environments targeted at
the upper range of the 25,000-user scale when using one of the advanced replication architectures. Similarly,
performance impacts may be negligible for smaller environments using only one to four storage modules and
which would likely use more basic replication architectures. The protected Hitachi replication architectures will be
discussed in a later section.

Detailed planning and design are critical for Exchange Server 2007 environments that can be characterized as
having high IOPS, aggressive RPO/RTO, demanding business continuity requirements or otherwise possessing
advanced levels of technical or design complexity. These solutions may benefit significantly from the assistance of
performance and business continuity consultants from Hitachi Data Systems Global Solution Services (GSS).

Designing for Performance and Protection

Specific production requirements will rarely be identical to a test environment due to differing module and option
configurations, additional applications, and the specifics of the physical environment, including the distance
between sites and the type of connectivity. In this case, the strength of the scalable AMS1000 Exchange design is
apparent because it provides IOPS and response-time benchmarks that one can apply as building blocks in
conjunction with other technologies to meet specific demands.

Using the Exchange Server 2007 test profile described in the ESRP submission, each reference storage module
processed between 2,350 and 2,450 IOPS (including database and log I/Os). When eight storage modules were
stacked together, the AMS1000 supported a simulated Exchange Server 2007 environment consisting of eight
servers that maintain 25,000 mailboxes. The average database read and write latencies measured 16ms and 4ms
respectively, which equates to 25 percent and 75 percent below the 20ms response time target. Sustained IOPS
that occur within an acceptable response time (which is the primary validation goal of JetStress) are the key
metrics one needs to consider when building Hitachi Data Systems protected architectures for Exchange Server
2007.

Designing a high-performing and resilient solution from the storage module building blocks requires allowing for
the performance impact associated with the backup and/or replication processes in terms of IOPS and response
time. Regardless of the backup and/or replication technology, additional workload requirements are placed on the

11
server and/or the storage resources composing the Exchange Server 2007 infrastructure. It is necessary to
incorporate the increased resource demands of backup and replication functions into the storage design to ensure
application performance does not degrade during times of high activity.

Increasing the amount of memory (RAM) on the Exchange Server 2007 hosts can provide significant performance
benefits. Specifically, an increased number of IOPS, reduced response times, and/or reduction in host-based
replication impact can be realized with increased levels of RAM. Microsoft indicates that 32GB of RAM is the
practical, not technical, limit for Exchange Server 2007 hosts, as the benefit of increasing RAM beyond this level
does not provide enough return to justify the significant costs.

From a storage perspective, two possible methods of adapting a reference storage module from the ESRP
submission for use in building protected solution architectures involve:
• Decreasing the number of mailboxes allocated to the mailbox store in each storage group for the Exchange
instance. This decreases the total number of mailboxes housed by the Exchange instance, thereby reducing the
I/O workload to the physical storage resources associated with those storage groups.
• Increasing the number of RAID-1+0 disk spindles allocated to the Adaptable Modular Storage system RAID
groups, which comprise the production LUNs for the database file for a given storage group. This allocates the
mailbox store file over an increased number of physical disks from the larger RAID-1+0 RAID group, which
provides the ability to support an increased number of IOPS within an acceptable average response time.

By decreasing the number of mailboxes we are simply accommodating the IOPS and response time requirements
for the backup and/or replication process with the IOPS and response time requirements of the reduced number
of mailbox stores for the Exchange instance. The benchmark IOPS (2400, including logs) and response time
metrics (16ms) from the average reference storage module should be the targeted maximum levels when the
impact of backup and/or replication processes is included. However, ensure that a 10 percent to 15 percent factor
is built into the calculations in order to account for growth and spikes in latency.

Alternatively, by increasing the number of physical disk spindles that compose the RAID groups for the database
LUNs for a storage group we can account for backup/replication overhead. This method stripes the mailbox store
file (database) over an increased number of physical disks, providing the ability to support an increased number of
IOPS within an acceptable average response time. The benchmark IOPS (2400, including logs) and response time
metrics (16ms) from the average storage module now become the expected minimum levels supported when
additional physical spindles are added to the Adaptable Modular Storage system RAID groups for the mailbox
store LUNs. Again, ensure that a 10 percent to 15 percent factor is built into the calculations in order to account
for growth, spikes in latency-producing, nontransactional I/O, and maintenance activities. This calculation can
easily get complex and requires restructuring the reference storage module to account for the additional spindles.
Therefore, be sure to involve Hitachi Data Systems Global Solution Services when using this alternate technique.

Key Takeaways
The following list summarizes the key Exchange Data Protection takeaways and provides additional guidance for
designing and validating Exchange Server 2007 solution architectures:
• The AMS1000-based storage module architecture provides a benchmark building-block performance metric for
Exchange-solution construction. However, each situation is unique and the test results should be used only as a
guideline rather than as an absolute predictor of performance.
• When material changes to the basic storage module architecture are required, it is strongly recommended that
additional Microsoft JetStress testing be conducted against all anticipated I/O and non-transactional workloads,
and that the appropriate Hitachi Data Systems technical resources be engaged.

12
• Be conservative when developing workload and resource estimates and when mitigating peak periods of heavy
I/O and nontransactional workloads.
• JetStress testing is just one of the Microsoft recommended steps to establish readiness and supportability for
an Exchange Server 2007 environment. However, it is crucial for validating a storage configuration, especially
when designing for data protection.
• LoadGen validation testing that includes the architected backup and/or replication processes and overhead is
highly recommended.

Storage-based Protection for Exchange

The following components are used in the building blocks shown for Hitachi-protected Exchange Server 2007
solutions.

Hardware
• Hitachi Adaptable Modular Storage model AMS1000

Software
• Hitachi HiCommand® Device Manager 5.5—provides centralized management of distributed Hitachi
environments
• Hitachi HiCommand Dynamic Link Manager 5.8—provides robust path failover and load balancing for Hitachi
storage in open systems operating environments
• Hitachi HiCommand Protection Manager 5.7—protects mission-critical applications by automating split-mirror
backups and enabling rapid recovery using proven Hitachi replication technologies
• Hitachi Command Control Interface—allows scripted control of replication procedures

• Hitachi ShadowImage® Replication—provides nondisruptive, host-independent data replication for creating

copies of any customer accessible data within a single Hitachi storage system
• Hitachi TrueCopy® Synchronous—provides continuous, nondisruptive, host-independent remote data
replication for disaster recovery
• Hitachi Data Protection Suite (or other third-party backup software)—provides scheduled disk-to-tape or disk-
to-disk-to-tape backup of data volumes or files for offsite storage or local recovery

Adaptable Modular Storage model AMS1000: Hitachi Data Systems Protected

Architectures for Exchange Server 2007
In order to more clearly present the protection alternatives that follow, it may be useful to first map some of the
common elements between Exchange Server 2007 and the AMS1000. This paper first recaps the elements
associated with a single Exchange server and the disks that make up the volumes for a single Exchange Server
2007 storage group. Figure 4 highlights the physical layout of the first tray of a single storage module. For a single
storage group, note that two volumes are represented consisting of six physical disks—four disks for the mailbox
store and two for the logs and checkpoint file.

13
Figure 4. Single Storage Group Disks Extracted from a Single Storage Module

Hitachi Adaptable Modular Storage AMS1000

Single Module (1 Storage Group)

RKA 1

SG1 Mailbox Allocated SG1 Log & Spare

Store (.edb) Checkpoint
(.log & .chk)

Up to this point, discussions of the AMS1000-based storage modules have focused on the production Exchange
Server 2007 LUNs, RAID groups and physical disks. In order to build protected architectures that use Hitachi
storage–based replication solutions, storage modules must be allocated to hold one or more replication volumes,
commonly known as secondary volumes (S-VOLs), and then the S-VOLs must be mapped to the primary physical
disk configuration on the AMS1000. Additional disk trays (RKAs) and racks will be required to expand the
AMS1000 to house the S-VOL physical disks. Also, remember that the logical drawings that accompany the
Hitachi Data Systems protected reference architectures introduced in this section represent only one Exchange
storage group, which is composed of two dedicated LUNs—one for the Exchange database and one for the logs.

The straightforward method for developing the S-VOL storage modules is to allocate one additional storage
module for each S-VOL you want to maintain, providing S-VOL LUN construction and allocation that identically
matches the characteristics of the P-VOLs. This design provides optimal replication performance by isolating
workloads and storage resources and avoids disk contention areas also referred to as hot spots.

Figure 5 illustrates a single storage module for the P-VOLs, and the additional storage module architecture
required for two copies of S-VOLs. Additional S-VOLs are required to provide multiple recovery points in time,
which are used to rapidly restore and recover from multiple scenarios. The number of S-VOLs and the frequency
of refresh via the backup process are key factors in determining the recovery service level objectives that a
storage-based backup and replication solution can support. Each additional copy provides an enhanced level of
data protection, but an additional set of storage resources are required to maintain each backup copy.

14
Figure 5. Primary and Secondary Volume Storage Modules for Replication
Hitachi Adaptable Modular Storage AMS1000
Single Module (S-VOL #2)

RKA 20

RKA 19

Hitachi Adaptable Modular Storage AMS1000

Single Module (P-VOLs)
DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5 Spare
Log SG2 Log SG4
RKA 2

RKA 1

Hitachi Adaptable Modular Storage AMS1000

Single Module (S-VOL #1)
DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5 Spare
Log SG2 Log SG4
RKA 20

RKA 19

DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5 Spare
Log SG2 Log SG4

Notice that three AMS1000–based storage modules are required. These reside, in this example, on separate trays
depicted by the RKA number but connected to the same AMS1000. One storage module would be allocated to
the production Exchange server volumes and the other two storage modules would be used for unique S-VOL
copies. A fourth storage module would be required to maintain a third S-VOL if needed.

An alternative method that can reduce cost entails constructing the S-VOL LUNs using one or more of the
following techniques:

• Using larger capacity physical disks

• Increasing the number of physical spindles in the RAID group

However, realize that storage-based replication carries with it associated resource costs as well, so ensure that
during recovery, when using larger or slower drives in a different RAID configuration, your design can maintain the
RPO, RTO, and SLO solution goals without introducing response time or IOPS degradation to the production
Exchange Server 2007 workloads. Hitachi Data Systems Global Solution Services offers a variety of professional
services to assist with meeting these demands.

Figure 6 illustrates an AMS1000–based storage module configured to use 300GB, 10,000 RPM Fibre Channel
drives, in place of 146GB 15,000 RPM drives. This allows two S-VOLs to reside on a single storage module.

15
Figure 6. Single Storage Module for Replication Supporting Multiple S-VOLs

Hitachi Adaptable Modular Storage AMS1000 Hitachi Adaptable Modular Storage AMS1000
Single Module (P-VOLs) Single Module (Multiple S-VOL)

RKA 2 RKA 2

RKA 1 RKA 1

Secondary
Copy #1
DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5 Spare DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5
Log SG2 Log SG4 Log SG2 Log SG4

Spare
Secondary
Copy #2
DB SG1 DB SG2 DB SG3 DB SG4 DB SG5 Log SG1 Log SG3 Log SG5
Log SG2 Log SG4

For the diagrams that follow, the primary volumes (P-VOLs) represent the primary database and log volumes for an
entire storage group, and the secondary volumes (S-VOLs) represent image replicas that are maintained for rapid
recovery on local storage, remote storage, or both. Figure 7 illustrates how the Exchange Server 2007, Windows
Server 2003, ShadowImage and Protection Manager software, and AMS1000 map to one another.

Figure 7. Exchange Server Components and Resources Mapped to Storage Volumes

1:00 AM
.edb file

Mount Point 2:00 AM

PVOL
PVOL
.log file
1:00 AM

Mount Point 2:00 AM

.chk file PVOL

PVOL
Microsoft Storage Mailbox
Exchange Group Store
Server Hitachi Adaptable Modular Storage AMS1000

Examples of Building-block-driven Hitachi Data Systems Protected Solution

Architectures
This section discusses the Hitachi Data Systems protected architectures available for Exchange Server 2007 on
the AMS1000. As above, the protected architectures are applied in a building-block manner to produce a solution
architecture that meets specific Exchange Server 2007 protection requirements. In addition, this section provides
examples that illustrate how individual components can be combined to develop complete, protected solution
architectures.

16
Basic Replication Architectures for Hitachi HiCommand Protection Manager software
Figure 8 highlights the basic local replication architectures for Hitachi HiCommand Protection Manager when used
with ShadowImage Replication software. Each P-VOL and S-VOL container in the following drawings represents a
single Exchange Server 2007 storage group—not an entire Exchange instance or server. The P-VOL container
corresponds to the two volumes—one for the mailbox store and the other for the logs and checkpoint file—and
the S-VOL container represents the two volumes that contain the replicated recovery points of these storage
group files. The S-VOL containers that are dashed in the drawing represent multiple copies, which are recommended
to improve recoverability in the case of corruption or other data loss that is not discovered immediately. Each
additional S-VOL provides an additional recovery point, which enhances the protection level of the overall
Exchange Server 2007 solution.

Figure 8. Basic Replication Capabilities of HiCommand Protection Manager Software Used with ShadowImage
Software

Single Generation Multiple Generation

Hitachi ShadowImage® Hitachi ShadowImage®
Replication software Replication software

ShadowImage ShadowImage
PSUS PSUS

Figure 9 illustrates the basic local protection components for Exchange Server 2007 based on Protection Manager
and ShadowImage software. The backup (import) server is used to confirm the integrity of the single secondary
copy, via Eseutil (Extensible Storage Engine Utility). In most cases, the backup (import) server can be shared in
multi-Exchange instance environments. This is outlined in the HiCommand Protection Manager documentation.

17
Figure 9. Hitachi Protection Manager Base Components
LAN

Hitachi HiCommand® HiCommand Protection

Protection Manager software Manager software

CCI CCI
Hitachi Resource Manager™ Resource Manager Shadow
Shadow Copy Provider Copy Provider

Hitachi Data
Protection Suite
Production Server Media Server
Microsoft Exchange Microsoft VSS
or SQL Import Server

Backup Software’s Backup Software’s

Agent Master Agent
Switch

PVOLS SVOLS

Hitachi Adaptable Modular Storage AMS500 Backup on Tape Library

The primary Exchange volumes are replicated locally using Protection Manager along with Volume ShadowCopy
Service (VSS) and Virtual Disk Service (VDS) integration, to make image copies that can be used for rapid local
recovery or that can be accessed for other protection alternatives. This elementary level of protection can allow
recovery if corruption of the active Exchange instance is discovered before the copy is over-written by a
subsequent backup. However, with only a single copy and no offsite backup, there is little true data protection.
This point highlights the importance of maintaining multiple S-VOLs, providing multiple recovery points from which
service can be rapidly restored.

HiCommand Protection Manager software provides capabilities that can be used in conjunction with replication,
tape backup and/or Hitachi Storage Cluster architectures. These capabilities are applicable to Exchange instances
using the Single Copy Cluster and Majority Node Set cluster models available with Microsoft Cluster Services, in
addition to the standalone model as illustrated in the previous drawing.

18
Multi-generation Protection Manager with Hitachi Data Protection Suite
Figure 10 illustrates the components of a more robust solution architecture for Exchange Server 2007 that combines
HiCommand Protection Manager, ShadowImage Replication and Hitachi Data Protection Suite software.

Figure 10. Hitachi Protection Manager with Hitachi Data Protection Suite Protected Architecture Components
LAN

Hitachi HiCommand® HiCommand Protection

Protection Manager software Manager software

CCI CCI
Hitachi Resource Manager™ Resource Manager Shadow
Shadow Copy Provider Copy Provider

Hitachi Data
Protection Suite
Production Server Media Server
Microsoft Exchange Microsoft VSS
or SQL Import Server

Backup Software’s Backup Software’s

Agent Master Agent
Switch

PVOLS SVOLS

Hitachi Adaptable Modular Storage AMS500 Backup on Tape Library

In this example, the multiple generation replication architecture for ShadowImage software is applied to provide
the replication component of HiCommand Protection Manager software, which was discussed earlier. This
solution architecture combines the split-mirror backup, rapid recovery protected architecture of Protection
Manager software with the preservation and retention protected architecture of Hitachi Data Protection Suite to
create a resilient disk-to-disk-to tape backup solution. This is the most common deployment model for Protection
Manager in medium-sized organizations.

Advanced Replication Architectures with HiCommand Protection Manager Software

In addition to providing local copies for rapid recovery purposes, HiCommand Protection Manager software also
has the ability to maintain a remote copy using Hitachi TrueCopy Synchronous software, which can be used for
disaster recovery or for facilitating the replication leg of a solution designed to offload the tape backup process to
the remote site.

19
Figure 11. Advanced Replication Model Combining HiCommand Protection Manager, ShadowImage Replication
and TrueCopy Synchronous Software

Single Generation Hitachi TrueCopy® Synchronous and

Hitachi ShadowImage® Replication Software Generation(s)

TrueCopy PSUS

ShadowImage PSUS

Combining HiCommand Protection Manager Software and Hitachi Storage Cluster

Figure 11 highlights a more advanced replication architecture for Exchange Server 2007 using HiCommand
Protection Manager software and Hitachi Storage Cluster combined with ShadowImage Replication and TrueCopy
Synchronous software. Multiple S-VOLs provide additional recovery points, which enhance the protection level of
the overall Exchange Server 2007 solution.

Figure 12. Advanced Replication Model Combining Hitachi Storage Cluster and HiCommand Protection Manager,
ShadowImage Replication and TrueCopy Synchronous Software
Hitachi Storage Cluster Integration

Hitachi TrueCopy®
Synchronous Software Pair

Hitachi ShadowImage®
Replication Software PSUS ShadowImage PSUS

Hitachi Storage Cluster Integration Hitachi Storage Cluster Integration

(Normal Operation) (After Cluster Failover)

TrueCopy Pair TrueCopy Pair

ShadowImage PSUS ShadowImage PSUS ShadowImage PSUS SI PSUS

The grayed-out volumes in Figure 12 indicate when ShadowImage Replication software is inactive because a site
does not house the active Exchange cluster node.

20
For the following advanced protection Reference Architecture, multiple-generation ShadowImage copies are
combined with HiCommand Protection Manager and Hitachi TrueCopy Synchronous software and Hitachi
Storage Cluster.

Figure 13. Multiple-generation Protected Architecture Components for HiCommand Protection Manager Software
and Hitachi Storage Cluster
LAN/WAN

Hitachi HiCommand® Microsoft Windows Windows Windows HiCommand Protection Manager

Protection Manager Server 2003 x64
Software CC (x32/x64) Enterprise Edition
Hitachi Resource Manager™ Exchange 2007 (x64)
Shadow Copy Provider
Microsoft Cluster Server (MSCS)
Microsoft Exchange 2007 Majority Node Set
MC (x32/x64)
Hitachi Storage Cluster (HSC)
Generic Script Resource
HiCommand Protection Manager
for Exchange (x64)
HiCommand Protection Manager
CC (x64)
CCI/Resource Manager Shadow
Copy Provider
Primary Site
Import Server
Server 2003 (x84/x64) Server 2003 x64 CC (x32/x64)
Enterprise Edition Enterprise Edition
CCI/Resource Manager Shadow
MSCS Majority Exchange 2007 (x64) Copy Provider
Node Set
MSCS Majority Node Set Exchange 2007 MC (x32/x64)
HSC Generic Script Resource
HiCommand Protection Manager
for Exchange (x64)
HiCommand Protection
Manager CC (x64)
CCI/Resource Manager
Tie-breaker Site Shadow Copy Provider
Majority Node Set Secondary Site
File Share Witness Import Server

Primary Site Secondary Site

Majority Node Set Majority Node Set
Exchange Mailbox Exchange Mailbox
Server Active Server Passive
Cluster Node Cluster Node

Hitachi TrueCopy®
Synchronous Software

Hitachi Adaptable Modular Hitachi Adaptable Modular

Storage AMS1000 Storage AMS1000

This solution architecture integrates the local, rapid backup and recovery functionality of Protection Manager
software with the disaster recovery capabilities provided by Hitachi Storage Cluster to provide a superior level of
data protection for Exchange Server 2007. Multiple local recovery points complement the synchronously
connected remote Exchange cluster node to facilitate protection against a variety of outage scenarios, ranging
from an isolated virus, to a server crash, to a complete site failure. The architecture of this solution provides the
ability for organizations to realize extremely low RPO and RTO, and aggressive SLO.

21
Services
Planning, design and implementation services play a key role in the successful deployments of protected
environments. Hitachi Data Systems offers a range of services, many of which are crucial in constructing the
solutions introduced. The following services leverage the hardware and software components of the various
reference architectures to design, implement and validate highly protected, resilient Exchange Server 2007
infrastructures:

• Implementation Services for Hitachi Data Protection Suite, powered by CommVault®

• Hitachi SplitSecond™ Solutions

• Remote Copy Planning and Design Service

• Hitachi Storage Cluster for Microsoft® Environments Services

• Risk Assessment Service

Summary
As enhanced features and functionality drive broader adoption and migration to Exchange Server 2007, Hitachi
Data Systems expects increased demand for mission-critical infrastructures, like the AMS1000, to support that
deployment. As highlighted, a high-performing, resilient storage infrastructure is a critical success factor for
supporting a diverse set of Exchange Server 2007 deployments.

The ESRP submission for Exchange Server 2007 on the AMS1000 provides the foundation upon which solution
architectures supporting varying levels of scalability, protection and performance can be built. The storage
modules and reference architectures introduced in this paper provide the building-block components necessary to
architect a high-performing and highly available Exchange Server 2007 data protection solution leveraging the
AMS1000. These solutions are critical components in our customer’s overall efforts to meet their business needs
while leveraging and maximizing investments in Microsoft and Hitachi Data Systems technology.

The building-block framework introduced in this document can be applied to other Hitachi Data Systems ESRP
submissions for Exchange Server 2007 to arrive at the optimal storage configuration for a given set of budget,
performance, data protection and business requirements. However, be sure to follow the recommended best
practices outlined in the ESRP for that particular family of Hitachi storage. Workloads and data protection
requirements are unique for each organization. Accordingly, additional design and testing is always recommended,
especially when data protection requirements are taken into account.

Hitachi Data Systems will continue to develop Reference Architecture materials for Exchange Server 2007. These
documents will provide technical insights, recommendations and configuration details used in this document to
produce the data protection reference architectures. For additional information and contacts, please review
www.hds.com and the Hitachi Data Systems Microsoft partner page at www.hds.com/microsoft.

22
Appendix: Supplemental Exchange Server 2007 Information
The content in this paper applies the principles, guidance and best practices found in the publicly available
information contained in the following Microsoft publications.

Document Name Hyperlink

Deploying a Simple Exchange Organization http://technet.microsoft.com/en-us/library/bb201659.aspx

Deploying a Standard Exchange Organization http://technet.microsoft.com/en-us/library/bb201727.aspx

Deploying a Large Exchange Organization http://technet.microsoft.com/en-us/library/bb201708.aspx

Deploying a Complex Exchange Organization http://technet.microsoft.com/en-us/library/bb201752.aspx

Planning for Edge Transport Servers http://technet.microsoft.com/en-us/library/aa996562.aspx

Planning for Hub Transport Servers http://technet.microsoft.com/en-us/library/aa998616.aspx

Planning for Client Access Servers http://technet.microsoft.com/en-us/library/bb232184.aspx

Planning for Mailbox Servers http://technet.microsoft.com/en-us/library/bb201699.aspx

Planning for Unified Messaging Servers http://technet.microsoft.com/en-us/library/bb125141.aspx

Exchange Server 2007 Planning and Architecture http://technet.microsoft.com/en-us/library/aa998636.aspx

Best Practices for Deploying a New Exchange http://technet.microsoft.com/en-us/library/bb123517.aspx

Organization

Best Practices for Transitioning an Exchange http://technet.microsoft.com/en-us/library/bb124920.aspx

Organization

Best Practices for Migrating to an Exchange http://technet.microsoft.com/en-us/library/b841fe7e-8961-4962-8f4b-

Organization 55d89dc9ef51.aspx

Features of Exchange Server 2007 http://www.microsoft.com/exchange/evaluation/features/default.mspx

Exchange Server 2007 Product Documentation http://technet.microsoft.com/en-us/library/bb124558.aspx

Planning Processor and Memory Configurations http://technet.microsoft.com/en-us/library/aa998874.aspx

Planning Disk Storage for Exchange Server 2007 http://technet.microsoft.com/en-us/library/bb124518.aspx

Exchange Server 2007 Security and Protection http://technet.microsoft.com/en-us/library/aa996775.aspx

High Availability Strategies for Exchange Server 2007 http://technet.microsoft.com/en-us/library/bb123523.aspx

Local Continuous Replication http://technet.microsoft.com/en-us/library/bb125195.aspx

Cluster Continuous Replication http://technet.microsoft.com/en-us/library/bb124521.aspx

Single Copy Cluster http://technet.microsoft.com/en-us/library/bb125217.aspx

Exchange Server 2007 High Availability http://technet.microsoft.com/en-us/library/bb124721.aspx

Clustering Architecture Essentials http://technet2.microsoft.com/windowsserver/en/library/42ba8090-cbb8-4e90-

ad7f-bc09d956bdac1033.mspx?mfr=true

Scheduled and Unscheduled Outages http://go.microsoft.com/fwlink/?LinkId=65320

Optimizing Storage for Exchange Server http://www.microsoft.com/technet/prodtechnol/exchange/2003/library/optimizestor

age.mspx

Planning for Local Continuous Replication http://technet.microsoft.com/en-us/library/bb124704.aspx

Planning for Cluster Continuous Replication http://technet.microsoft.com/en-us/library/bb123996.aspx

23
Document Name Hyperlink

Disaster Recovery Strategies http://go.microsoft.com/fwlink/?LinkId=65320

Managing Storage Groups and Databases http://go.microsoft.com/fwlink/?LinkId=65320

24
 Hitachi Data Systems Corporation

Corporate Headquarters 750 Central Expressway, Santa Clara, California 95050-2627 USA

Contact Information: + 1 408 970 1000 www.hds.com / info@hds.com

Asia Pacific and Americas 750 Central Expressway, Santa Clara, California 95050-2627 USA

Contact Information: + 1 408 970 1000 www.hds.com / info@hds.com

Europe Headquarters Sefton Park, Stoke Poges, Buckinghamshire SL2 4HD United Kingdom

Contact Information: + 44 (0) 1753 618000 www.hds.com / info.uk@hds.com


Hitachi is a registered trademark of Hitachi, Ltd. and/or its affiliates in the United States and other countries. Hitachi Data Systems is a registered trademark and service
mark of Hitachi, Ltd., in the United States and other countries. HiCommand is a registered trademark of Hitachi, Ltd.
TrueCopy and ShadowImage are registered trademarks and SplitSecond is a trademark of Hitachi Data Systems Corporation.
IBM and BladeCenter are registered trademarks of International Business Machines Corporation.
Microsoft is a registered trademark of Microsoft Corporation.
CommVault is a registered trademark of CommVault Systems, Inc.
All other product and company names are, or may be, trademarks or service marks of their respective owners.
Notice: This document is for informational purposes only, and does not set forth any warranty, express or implied, concerning any equipment or service offered or to be
offered by Hitachi Data Systems. This document describes some capabilities that are conditioned on a maintenance contract with Hitachi Data Systems being in effect,
and that may be configuration-dependent, and features that may not be currently available. Contact your local Hitachi Data Systems sales office for information on
feature and product availability.
Hitachi Data Systems sells and licenses its products to certain terms and conditions, including limited warranties. To see a copy of these terms and conditions prior to
purchase or license, please go to http://www.hds.com/products_services/support/warranty.html or call your local sales representative to obtain a printed copy. If you
purchase or license the product, you are deemed to have accepted these terms and conditions.
© Hitachi Data Systems Corporation 2007. All Rights Reserved
WHP-263-00 September 2007