You are on page 1of 362

RecoverPoint

Version 4.4

Administrator's Guide
P/N 302-002-495
REV 01

Copyright 2015 EMC Corporation. All rights reserved. Published in USA.


Published November, 2015
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change
without notice.
The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a
particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.
EMC, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other
countries. All other trademarks used herein are the property of their respective owners.
For the most up-to-date regulatory document for your product line, go to EMC Online Support (https://support.emc.com).
EMC Corporation
Hopkinton, Massachusetts 01748-9103
1-508-435-1000 In North America 1-866-464-7381
www.EMC.com

RecoverPoint 4.4 Administrator's Guide

CONTENTS

Figures

Tables

11

Chapter 1

Getting Started

13

Logging into RecoverPoint............................................................................. 14


Logging into Unisphere for RecoverPoint.......................................... 14
Notification of new RecoverPoint software releases.......................... 15
Licensing RecoverPoint................................................................................. 17
Before you begin.............................................................................. 17
How RecoverPoint licensing works for new purchases...................... 17
Upgrade from RecoverPoint 4.0 or later......................................................... 25
Upgrade from RecoverPoint versions prior to 4.0........................................... 25
Get licensing support.................................................................................... 27
The Getting Started wizard............................................................................ 28
Welcome screen...............................................................................28
Add Licenses screen........................................................................ 28
Enable Support screen..................................................................... 29
Register RecoverPoint screen........................................................... 30
The Unisphere for RecoverPoint interface...................................................... 31
Set the time display of Unisphere for RecoverPoint...........................32
Set the language of Unisphere for RecoverPoint............................... 33
The RecoverPoint workflow............................................................................33
First steps in RecoverPoint............................................................... 34
Configuring replication.....................................................................34
Performing recovery activities...........................................................34
Monitoring and managing RecoverPoint........................................... 34
System analysis............................................................................... 34
RecoverPoint support....................................................................... 34

Chapter 2

Dashboard

37

The RecoverPoint Dashboard.........................................................................38


Overall Health............................................................................................... 39
RPA Clusters Map............................................................................. 40
Multipath monitoring....................................................................... 41
Alerts............................................................................................... 42
System Traffic.................................................................................. 43
Consistency Group Transfer Status................................................... 43
System Limits................................................................................................45
System.............................................................................................45
Groups.............................................................................................46
Storage............................................................................................ 46
Splitters........................................................................................... 46
System Events...............................................................................................47
Display events................................................................................. 47
Display additional event information................................................48
Sort events.......................................................................................48
RecoverPoint 4.4 Administrator's Guide

CONTENTS

Filter events..................................................................................... 48
Copy an event to the clipboard.........................................................49
Collect system information around a specific event.......................... 50

Chapter 3

Protection

51

RecoverPoint Protection................................................................................ 52
Protect Volumes............................................................................................ 53
Protecting volumes.......................................................................... 53
Policy Templates........................................................................................... 68
Display all policy templates..............................................................69
Create new policy templates............................................................ 69
Modify existing policy templates...................................................... 70
Remove a policy template................................................................ 71
Apply a policy template to a new consistency group......................... 71
Apply a policy template to an existing consistency group................. 72
Group Sets.................................................................................................... 73
How group sets work........................................................................73
Display all group sets.......................................................................74
Create a group set............................................................................ 74
Edit a group set................................................................................ 75
Remove a group set..........................................................................76
Enable or disable all groups in a group set....................................... 77
Pause or start transfer for all groups in a group set........................... 77
Apply a bookmark to all of the groups in a group set........................ 77
Initiate recovery activities for a group set......................................... 77
Manage Protection........................................................................................ 78
Before you begin.............................................................................. 80
The RecoverPoint protection policies................................................80
Monitoring and managing protection................................................90
Monitoring and managing groups.....................................................91
Monitoring and managing copies................................................... 105
Modifying consistency groups........................................................ 114
Modifying copies............................................................................129
Modifying links.............................................................................. 132
Modifying replication sets.............................................................. 132
Modifying journals......................................................................... 136
Modifying snapshots......................................................................138

Chapter 4

Recovery

145

RecoverPoint Recovery................................................................................ 146


Test a Copy................................................................................................. 147
How to test a copy..........................................................................148
Test a Copy and Recover Production............................................................ 160
How to recover production............................................................. 160
Test a Copy and Fail Over............................................................................ 165
How to fail over.............................................................................. 166
Manage Recovery........................................................................................ 173
Testing a copy................................................................................ 174
Failing over.................................................................................... 175
Recovering production................................................................... 178

Chapter 5

RPA Clusters

181

RecoverPoint clusters.................................................................................. 182


4

RecoverPoint 4.4 Administrator's Guide

CONTENTS

General Cluster Info.....................................................................................183


Storage....................................................................................................... 185
Display registered storage..............................................................186
Register storage............................................................................. 187
Define storage login credentials..................................................... 189
Unregister storage..........................................................................192
Edit storage registration details......................................................193
Display registered resource pools.................................................. 194
Register available resource pools...................................................195
Unregister resource pools.............................................................. 196
Splitters...................................................................................................... 197
Display splitter information............................................................ 198
Display volume information........................................................... 199
Add splitters.................................................................................. 200
Remove splitters............................................................................ 202
Attach volumes to the splitter........................................................ 202
Detach volumes from the splitter....................................................203
RPAs........................................................................................................... 204
Properties...................................................................................... 205
Status............................................................................................ 206
Interface........................................................................................ 207
Traffic Statistics............................................................................. 207
Usage Statistics............................................................................. 209
vCenter Servers........................................................................................... 211
Before you begin............................................................................ 211
Display vCenter Server information................................................ 212
Register vCenter Server.................................................................. 212
Edit vCenter Server Registration..................................................... 213
Remove vCenter Server.................................................................. 213
Rescan vCenter Server....................................................................214
Phoenix Systems.........................................................................................214

Chapter 6

Admin

215

RecoverPoint Administration....................................................................... 216


Manage Licenses........................................................................................ 216
Install a license..............................................................................217
Remove licenses............................................................................ 218
Monitor your license limits............................................................. 218
Export a detailed license report......................................................218
Display your RecoverPoint license information............................... 219
Register RecoverPoint..................................................................................220
Register your RecoverPoint system................................................. 221
Export the post-deployment form to a CSV file................................224
Register RecoverPoint by email or phone........................................225
Users and Roles.......................................................................................... 225
Manage Users................................................................................ 226
Manage Roles................................................................................ 230
Manage User Authentication.......................................................... 233
Change Password.......................................................................... 236
Switch User....................................................................................237
Change Security Level.................................................................... 238
Change Idle Logout Time................................................................ 239
System Analysis Tools................................................................................. 239
Detect Bottlenecks......................................................................... 239
Balance Load................................................................................. 242
RecoverPoint 4.4 Administrator's Guide

CONTENTS

Collect System Information............................................................ 247


System Notifications................................................................................... 253
Manage Event Filters...................................................................... 254
Configure System Reports and Alerts..............................................264
Manage Call Home Events.............................................................. 269

Chapter 7

Support

273

RecoverPoint Support..................................................................................274
RecoverPoint Documentation......................................................... 274
EMC Online Support....................................................................... 275
The RecoverPoint Community......................................................... 276

Chapter 8

Troubleshooting

279

First-time initialization is taking too long..................................................... 280


How does it work............................................................................280
Initialize from backup.................................................................... 280
Host applications are hanging.....................................................................281
When does application regulation happen?....................................282
How does application regulation work?.......................................... 282
How do I know application regulation is happening?...................... 282
What can I do to stop my group from being regulated?................... 282
One-phase distribution workarounds.......................................................... 282
Copy is being regulated...............................................................................282
When does control action regulation happen?................................283
How do I know control action regulation is happening?.................. 283
How does control action regulation work?...................................... 283
How do I release a copy from control action regulation?................. 283
How do I verify that regulation is over?........................................... 283
Copy is experiencing a high load................................................................. 283
How do I know a copy is experiencing a high load?.........................284
What is a permanent high load?..................................................... 284
When do permanent high loads occur?...........................................284
How do permanent high loads work?..............................................284
How can I tell a copy is under permanent high load?...................... 284
What can I do to come out of permanent high load?....................... 284
How do I verify that a permanent high load is over?........................ 285
What is a temporary high load?...................................................... 285
When do temporary high loads occur?............................................285
How do temporary high loads work?...............................................285
How can I tell a copy is under temporary high load?....................... 285
What should I know about temporary high loads?.......................... 286
How do I verify that a temporary high load is over?......................... 286
RPA keeps rebooting................................................................................... 286
When does reboot regulation happen?........................................... 286
How does reboot regulation work?................................................. 286
How do I know reboot regulation is happening?............................. 287
What should I do to stop reboot regulation?................................... 287
Host applications crash during initialization................................................287
What is my storage read-rate during initialization?......................... 287
How does I/O throttling work?........................................................ 287
When should I enable I/O throttling?..............................................288
How do I enable I/O throttling?.......................................................288
How do I check how throttling affects my system performance?......289
How can I tell whether or not I/O throttling is enabled?.................. 289
6

RecoverPoint 4.4 Administrator's Guide

CONTENTS

Add more LUNs than allowed per VNX storage group................................... 290
How do I know I need more VNX LUNs?...........................................290
What can I do if I need more VNX LUNs?......................................... 290
How can I verify that the storage groups were added successfully?.291
Swap the LUN numbers of journal volumes..................................................291
Find your array serial numbers.....................................................................291
Find the serial numbers of VNX arrays............................................ 291
Find the serial numbers of Symmetrix arrays.................................. 292
Find the serial numbers of VPLEX arrays......................................... 292
Find your RPA cluster IDs............................................................................. 293
Find your XtremIO cluster IDs.......................................................................293
Find your XtremIO cluster IDs (PSNTs) in XtremIO............................293
Find your XtremIO cluster IDs (PSNTs) in RecoverPoint....................294
Understanding the RecoverPoint license..................................................... 294
Calculate your required journal size............................................................ 297
Minimum journal size with snapshot consolidation........................298
Validate replication integrity....................................................................... 304
How can replication integrity be compromised?..............................304
When should I validate replication integrity?.................................. 304
How the integrity check tool works................................................. 305
How the periodic integrity validation tool works............................. 305
How do I configure integrity validation?..........................................306
How do I know whether or not integrity validation was successful?.307
Can I disable or enable integrity validation globally?...................... 307
What should I do if an issue is detected?........................................308
MetroPoint group conversion...................................................................... 308

Chapter 9

Events

309

Normal Events.............................................................................................310
Detailed events........................................................................................... 320
Advanced events.........................................................................................360

RecoverPoint 4.4 Administrator's Guide

CONTENTS

RecoverPoint 4.4 Administrator's Guide

FIGURES

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

LAC email...................................................................................................................... 21
Getting Started Wizard Welcome screen.........................................................................28
Getting Started Wizard Add Licenses screen.................................................................. 29
Getting Started Wizard Enable Support screen............................................................... 29
Getting Started Wizard Register RecoverPoint screen..................................................... 30
Unisphere for RecoverPoint............................................................................................32
DashboardOverall Health tab.........................................................................................39
RPA clusters map widget................................................................................................40
Alerts widget..................................................................................................................42
Alerts table.................................................................................................................... 42
System traffic widget..................................................................................................... 43
Consistency group transfer status widget.......................................................................43
Dashboard System Limits tab........................................................................................ 45
DashboardSystem Events.............................................................................................. 47
Identifying a distributed group.....................................................................................100
Identifying a distributed group.....................................................................................100
Recovery bar for consistency groups............................................................................ 173
Recovery bar for group sets..........................................................................................174
RPA Clusters: General Cluster Info................................................................................183
RPA Clusters: Storage.................................................................................................. 185
RPA Clusters: Splitters................................................................................................. 198
RPA ClustersRPA-Cluster-Name tabRPAs...................................................................... 204
RPA ClustersRPA-Cluster-Name tabRPAsProperties.......................................................205
RPA ClustersRPA-Cluster-Name tabRPAsStatus............................................................ 206
RPA ClustersRPA-Cluster-Name tabRPAsInterface ........................................................ 207
RPA ClustersRPA-Cluster-Name tabRPAsTraffic Statistics..............................................207
RPA ClustersRPA-Cluster-Name tabRPAsUsage Statistics..............................................209
RPA Clusters: vCenter Servers...................................................................................... 211
RecoverPoint Documentation page on EMC Online Support..........................................275
EMC Online Support page............................................................................................ 276
The RecoverPoint Community page.............................................................................. 277
License file format....................................................................................................... 294
Example licenses.........................................................................................................295

RecoverPoint 4.4 Administrator's Guide

FIGURES

10

RecoverPoint 4.4 Administrator's Guide

TABLES

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

Licensing support.......................................................................................................... 27
Dashboard options........................................................................................................ 38
RPA clusters map icons..................................................................................................40
RPA clusters map options.............................................................................................. 41
Transfer state.................................................................................................................43
System LimitsSystem tab...............................................................................................45
System LimitsGroup tab.................................................................................................46
System LimitsStorage tab ............................................................................................. 46
System LimitsSplitters tab............................................................................................. 46
Event Filter settings....................................................................................................... 48
Protection options......................................................................................................... 52
Toolbar options............................................................................................................. 56
Group summary screen.................................................................................................. 66
Default policy templates................................................................................................ 68
Multiple consistency group monitoring options............................................................. 79
Multiple consistency group actions................................................................................79
Group policies............................................................................................................... 80
Link policies.................................................................................................................. 83
Copy policies................................................................................................................. 86
Journal policies..............................................................................................................87
Status tab......................................................................................................................91
Status tab options when a group is selected..................................................................92
Priority setting default values........................................................................................ 97
Consolidated statistics output..................................................................................... 101
Status tab....................................................................................................................105
Status tab options when a group is selected................................................................106
Copy journal tab.......................................................................................................... 110
Journal Tab: Snapshot Consolidation Progress information.......................................... 112
Journal tab image list...................................................................................................112
Snapshot policies........................................................................................................139
Recovery options......................................................................................................... 146
Select an Image screen................................................................................................150
Image access mode..................................................................................................... 152
XtremIO snap-based replication mitigation after failover..............................................167
RPA cluster options......................................................................................................182
RPA cluster information............................................................................................... 184
Storage information.....................................................................................................186
Resource pool storage information.............................................................................. 194
Splitter information..................................................................................................... 199
Volume information..................................................................................................... 200
RPA Clusters: RPA table............................................................................................... 204
RPA Clusters: RPA properties tab................................................................................. 205
RPA Clusters: RPA status tab........................................................................................206
Virtual machine info.................................................................................................... 212
RPA Clusters: Add vCenter Server.................................................................................213
Edit vCenter Server Settings.........................................................................................213
Admin options............................................................................................................. 216
Displayed license information..................................................................................... 219
Automatically registering your RecoverPoint system.....................................................222
Pre-defined users........................................................................................................ 226
Pre-defined roles......................................................................................................... 230
RecoverPoint 4.4 Administrator's Guide

11

TABLES

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

12

Permissions.................................................................................................................231
Role settings................................................................................................................232
LDAP Configuration settings.........................................................................................235
Bottleneck detection settings...................................................................................... 240
Bottleneck detection commands................................................................................. 241
Load balancing commands.......................................................................................... 244
System information collection settings........................................................................ 249
Collect system information options..............................................................................252
Event Filter settings..................................................................................................... 255
Email settings..............................................................................................................257
Syslog settings............................................................................................................ 259
SNMP settings............................................................................................................. 259
RecoverPoint SNMP trap variables............................................................................... 261
System parameters sent to SyR....................................................................................265
Support options...........................................................................................................274
License file variables................................................................................................... 295
Journal size with snapshot consolidation equation legend...........................................299
Normal scope events................................................................................................... 310
Detailed scope events................................................................................................. 320
Advanced scope events............................................................................................... 360

RecoverPoint 4.4 Administrator's Guide

CHAPTER 1
Getting Started

This section describes the first steps to setting up your RecoverPoint system.
l
l
l
l
l
l
l
l

Logging into RecoverPoint..................................................................................... 14


Licensing RecoverPoint......................................................................................... 17
Upgrade from RecoverPoint 4.0 or later................................................................. 25
Upgrade from RecoverPoint versions prior to 4.0................................................... 25
Get licensing support............................................................................................ 27
The Getting Started wizard.................................................................................... 28
The Unisphere for RecoverPoint interface.............................................................. 31
The RecoverPoint workflow....................................................................................33

Getting Started

13

Getting Started

Logging into RecoverPoint


Unisphere for RecoverPoint is the RecoverPoint stand-alone GUI that includes all
RecoverPoint features and updates as soon as they are available. All of the routine
monitoring and managing of the RecoverPoint system is performed through Unisphere for
RecoverPoint, and this Administrators Guide is primarily aimed at the Unisphere for
RecoverPoint user. You can also manage and monitor RecoverPoint through the following
interfaces:
The RecoverPoint Command Line Interface (CLI)
For the IT professional who prefers a command line interface and wants to run
scripts to perform multiple functions at once. Some of the advanced protection
settings referenced in this guide can only be accessed through the CLI. For detailed
instructions on how to launch and use the CLI, see the EMC RecoverPoint Command
Line Interface Reference Guide.
The RecoverPoint plug-in for Unisphere
For IT generalists who prefer to manage all system-related tasks through one
interface. The RecoverPoint plugin for Unisphere contains most of the functionality of
Unisphere for RecoverPoint. For detailed instructions on how to download, install,
launch and use the RecoverPoint plug-in for Unisphere, see the EMC RecoverPoint
Deploying with VNX/VNXe/CLARiiON Arrays and Splitter Technical Notes.
The RecoverPoint REST (REpresentative State Transfer) API
For developers who wish to integrate RecoverPoint with their own applications and
for those who need to write scripts to automate RecoverPoint operations. For
detailed instructions on how to install, run and use the RecoverPoint API, see the
EMC RecoverPoint REST API Programming Guide and Reference Technical Notes.

Logging into Unisphere for RecoverPoint


Procedure
1. Open a web browser and enter the cluster management IP into your browser address
bar. The RecoverPoint login screen is displayed.

2. In the RecoverPoint login screen, log in as admin or monitor according to the predefined user on page 226 rules.
3. If:
l

14

this is your first time logging into Unisphere for RecoverPoint, The Getting Started
wizard on page 28 is displayed. Use the Getting Started wizard to add your
RecoverPoint licenses, enable support, and register your RecoverPoint system.

RecoverPoint 4.4 Administrator's Guide

Getting Started

this is not your first time logging into Unisphere for RecoverPoint and a new
version of RecoverPoint is available The Upgrade Recommendation dialog box on
page 15 is displayed.

Results
When you are finished entering the required information into The Getting Started
wizard on page 28, Unisphere for RecoverPoint is displayed. Select RPA Clusters > RPACluster-Name > RPAs . Click the tab of each RPA cluster and ensure that a green checkbox
appears in both the Status and Connectivity columns of all RPAs at each RPA cluster.

Notification of new RecoverPoint software releases


RecoverPoint software upgrades are described in relation to major RecoverPoint releases.
In RecoverPoint, major releases are labeled with an X.x (for example, RecoverPoint 4.1).
RecoverPoint upgrades are considered:
l

MajorIf you must move to another major release to upgrade them. For example,
RecoverPoint 3.5, 4.0 and 4.1 are all different major releases of RecoverPoint, and
upgrading from one major release to another requires a major upgrade.

MinorIf they are in the same major release. For example, upgrading to any of the
service packs or patches of RecoverPoint 4.1 are all minor upgrades.

The Upgrade Recommendation dialog box


Each time you log into Unisphere for RecoverPoint, if RecoverPoint detects that there is a
new recommended minor version or the target version is newer than the version of
RecoverPoint currently installed at the RPA cluster to which you have connected, the
Upgrade Recommendation dialog box is automatically displayed, containing the:
l

RecoverPoint version currently installed at the RPA cluster you are connected to.

RecoverPoint target versionThe target version is the most stable and current
RecoverPoint software code version across all major RecoverPoint releases. Best
practice is to ensure all RPA clusters in your RecoverPoint system are always running
the RecoverPoint target version.

RecoverPoint recommended minor versionThe recommended minor version is the


most current minor software version available for the major release of RecoverPoint
that you are running. If your environment is not qualified, or you do not wish to
upgrade to the target version, consider upgrading your RPA clusters to the
RecoverPoint recommended minor version. There is one recommended minor version
per major release of RecoverPoint.

You can enable or disable the display of the Upgrade Recommendation dialog box using
the Admin > System Notifications > System Reports and Alerts screen, by clearing or
marking the Enable New Release Notification checkbox. It is enabled by default.
If you connected to RecoverPoint using the management (floating) IP address, note the
relevant cluster name in the Upgrade Recommendation dialog box, as the dialog box is
displayed per RPA cluster.

What should I do when I see the Upgrade Recommendation dialog box?


When the Upgrade Recommendation dialog box is displayed, you should start thinking
about upgrading your RecoverPoint system to the target version or recommended minor
version indicated in the pop-up.

Deciding whether or not you want to upgrade


To help you decide whether or not you are qualified for, or want to upgrade, use the:
Notification of new RecoverPoint software releases

15

Getting Started

EMC RecoverPoint Release NotesThe RecoverPoint release notes contain descriptions


of all of the new features and know bugs of each release. Use them to decide whether
or not the main features of the target or recommended release warrant an upgrade for
your company. For:
n

RecoverPoint/SEdownload the release notes from https://support.emc.com/


products/1249_RecoverPoint-SE/Documentation/

RecoverPoint/CLdownload the release notes from https://support.emc.com/


products/1226_RecoverPoint-CL/Documentation/

RecoverPoint/EXdownload the release notes from https://support.emc.com/


products/27704_RecoverPoint-EX/Documentation/

EMC Service CenterThe EMC Service Center at https://support.emc.com/


servicecenter/ is the one stop shop where you manage all of your support-related
activities and information. If you are using RecoverPoint/CL or RecoverPoint/EX, use
the Service Center to open a Service Request and start planning your RecoverPoint
upgrade. All RecoverPoint users can use the Service Center to:
n

Track, view, manage and create service requests.

Download the latest EMC products and patches.

Engage with EMC Technical Support for answers to your questions.

Contact EMC Technical support.

Access information for sites, products, contacts, contracts, and environments.

Download the EMC Support mobile application.

Learn about EMC's Maintenance, Warranty Information and Parts Return Site.

Connect with EMC customers and technical experts in a worldwide community.

Register your product to access software downloads, manage or obtain licenses.

EMC MatrixThe RecoverPoint EMC Simple Support Matrices (ESSMs) can be found
on EMCs E-lab Web site. E-Lab has tested and qualified millions of different data
center configurations and EMC solutions. To find out if your environment is qualified
for upgrade to the suggested target and recommended versions, download the
relevant RecoverPoint ESSM:
n

For upgrade to RecoverPoint 4.1.xDownload the ESSM from https://


elabnavigator.emc.com/vault/pdf/EMC_RecoverPoint_4.1.x_ESSM.pdf

For upgrade to RecoverPoint 4.0.xDownload the ESSM from https://


elabnavigator.emc.com/vault/pdf/EMC_RecoverPoint_4.0.x_ESSM.pdf

Upgrading your RecoverPoint environment


If you are upgrading your RecoverPoint system and are running:
l

RecoverPoint with VNX or VNXe storage and RecoverPoint/SE licensesYour


RecoverPoint system is customer upgradeable.
1. Download the RecoverPoint Customer Upgrade Kit from the Recommended
Resources area in the upper right corner of: https://support.emc.com/products/
1249_RecoverPoint-SE
2. Unzip the kit.
3. From the kit, open the RecoverPointCustomer Upgrade Guide, and follow the
instructions to upgrade your system.

16

Any other RecoverPoint systemYour RecoverPoint system is not customer


upgradeable. EMC Customer Support will plan and execute the upgrade for you.

RecoverPoint 4.4 Administrator's Guide

Getting Started

Contact your EMC representative or open a service request at https://


support.emc.com/servicecenter/

Licensing RecoverPoint
This section describes how to license new RecoverPoint purchases, and upgrade your
RecoverPoint license.

Before you begin


l

All RecoverPoint licenses are per array, an XtremIO cluster or an RPA cluster.
Licensing is enforced using the Locking ID.

You cannot add a volume that resides on an unlicensed array to a replication set.
Only volumes that reside on licensed arrays are displayed in the volume list. This
limitation does not exist for journal volumes or repository volumes.

If a license Locking ID contains an array serial number that is not recognized by


RecoverPoint, you will be able to install the license in RecoverPoint but it will not be
tied to an RPA cluster and the license wont be activated until the array is mapped and
zoned to an RPA cluster.

For consistency groups containing both a local copy and a remote copy: If you select
to fail over to the remote copy and the remote RPA cluster does not have enough
licensed replication capacity for two times the capacity of the remote copy, the
failover will fail and you will be prompted to remove the local copy for the failover to
succeed.

When trying to add or modify a group, copy or replication set, the licenses of all
relevant arrays and RPA clusters are checked and the action will fail if it causes any of
the arrays or RPA clusters in the system to exceed their maximum licensed replication
capacity.

In RecoverPoint/CL:
n

If a license Locking ID contains an RPA Cluster ID that is not recognized by


RecoverPoint, you will not be able to install the license.

RecoverPoint/CL licenses cannot be added if a RecoverPoint/SE license already


exists.

RecoverPoint/CL licenses cannot coexist on the same RPA cluster with


RecoverPoint/EX licenses. If both are added to the system, the RecoverPoint/EX
license is disabled until the RecoverPoint/CL license is removed.

When RecoverPoint/SE is installed, only RecoverPoint/SE licenses can be added.

RecoverPoint/EX licenses cannot:


n

Be added if a RecoverPoint/SE license already exists.

Coexist on the same RPA cluster with RecoverPoint/CL licenses. If both are added
to the system, the RecoverPoint/EX license is disabled until the RecoverPoint/CL
license is removed.

How RecoverPoint licensing works for new purchases


RecoverPoint licensing is performed through EMC Online Support at http://
support.emc.com.
1. Immediately upon submission of your sales order, a LAC email is sent to the address
provided during order entry, containing a new License Authorization Code (LAC) for all
Licensing RecoverPoint

17

Getting Started

of your purchased entitlements. A sales order can contain multiple LACs. In this case,
a separate LAC email is sent for each LAC contained in the sales order.
2. Click the Click here link in the LAC email to automatically access the RecoverPoint
licensing area of the EMC Online Support site and display all of the entitlements
associated with your LAC.
3. In the RecoverPoint licensing area of the EMC Online Support site, activate and
generate a License File for each entitlement. You will generate a separate license for
each RecoverPoint license model associated with each unique array or RPA cluster.
There is no special licensing model for virtual RPAs.
4. Install your license files on the RecoverPoint system through Unisphere for
RecoverPoint by selecting Admin > Manage Licenses, or by running the
add_license command in the RecoverPoint CLI.

What you will need


What you will need before you start licensing RecoverPoint.
l

Your LAC email(s) (or sales order numbers) to perform Step 1: Quickly access your
entitlements on page 20.

Access to EMC Online Support to perform Step 2 - Activate your entitlements on page
21.

Unisphere for RecoverPoint to perform Step 3 - Install your license file(s) in


RecoverPoint and enable support on page 24.
Note

If you do not have Unisphere for RecoverPoint installed, see the EMC RecoverPoint
Installation and Deployment Guide for installation instructions, and install
RecoverPoint before starting this procedure.

Search for entitlements associated with a sales order


Procedure
1. Access and log into EMC Online support at http://support.emc.com
l

Existing EMC Online Support users; enter your EMC Support username and
password.

New EMC Online Support users; click Register Now and complete the registration
process.

2. Select Service Center > Get and Manage Licenses > R > RecoverPoint Family.
3. Click the Activate link in the RecoverPoint version 4.0 and later section to access the
EMC licensing portal.

18

RecoverPoint 4.4 Administrator's Guide

Getting Started

4. In the LAC Request screen, click Manage Entitlements to display the Search
Entitlements screen:

5. In the Search Entitlements screen:

How RecoverPoint licensing works for new purchases

19

Getting Started

a. Enter a sales order number in the Sales Order # field.


b. Select Available in the Status field.
c. Click Search Entitlements to display all unactivated entitlements associated with
the entered sales order.
6. Go on to Step 2 - Activate your entitlements on page 21.

Step 1: Quickly access your entitlements


You will need to create a new RecoverPoint purchase order if you are a new RecoverPoint
customer, or if you want to:
l

(In XtremIO) Upgrade a basic license to a full-featured license.

Add replication capacity in RecoverPoint/CL.

Add registered capacity in RecoverPoint/EX.

Upgrade from RecoverPoint/SE to RecoverPoint/EX.

Upgrade from RecoverPoint/EX to RecoverPoint/CL.

Add storage arrays in RecoverPoint/EX.

Add RPAs to your RecoverPoint system.

Add an RPA cluster to your RecoverPoint system.

Add raw capacity (for RecoverPoint/EX customers with VMAX 10K arrays with
registered capacity).

Add back-end array replication (for RecoverPoint/EX customers with VPLEX arrays).

Dynamically change capacity allocations on RPA clusters (for ELA customers).

In all of these cases, you will need to make a purchase with EMC sales, and obtain a
sales order. As soon as your sales order is approved, a LAC email is automatically sent to
the email addresses provided during order entry containing the LAC (License Authorization
Code) issued to you, validating your purchase and your entitlement(s).
20

RecoverPoint 4.4 Administrator's Guide

Getting Started

Figure 1 LAC email

A unique LAC is usually generated and emailed to you for each purchase, but this is not
always the case for evaluation licenses or purchases of certain products. Also, in some
cases (such as the addition of multiple arrays in RecoverPoint/EX), multiple LACs will be
generated and multiple LAC emails will be emailed to you.
To start the licensing process:
l

If you have your LAC email(s), for each LAC, click the Click here link in the LAC email
to automatically access the Powerlink licensing area of EMC Online Support, and
display all entitlements associated with the LAC. Then, perform Step 2 - Activate your
entitlements on page 21.
Note

If you do not have your LAC email on page 21, search your email inbox for email(s)
containing EMC License Authorization in the subject line. If you still cant find your LAC
email(s), contact EMC Worldwide Licensing Support at: https://
powerlinklicensing.emc.com/poeticWeb/session/license_request_emc.jsp
l

If you have your sales order number(s), Search for entitlements associated with a
sales order on page 18.

Step 2 - Activate your entitlements


Before you begin
In the licensing section of the EMC Online Support Web site:
l
Existing EMC Online Support users; enter your EMC Support username and password.
l
New EMC Online Support users; click Register Now and complete the registration
process.
After you have logged-into EMC Online Support, the Search Entitlements to Activate
screen is displayed. The Search Entitlements to Activate screen is the first screen of a
How RecoverPoint licensing works for new purchases

21

Getting Started

five screen wizard, that will guide you, step-by-step, through the process of turning your
LACs into license files (redeeming your entitlements).
Procedure
1. In the The Search Entitlements to Activate screen:
The link in the LAC email pre-populates the LAC field and searches for all entitlements
associated with the specified LAC. All relevant entitlements are displayed.

a. Select ONE entitlement to activate. Each entitlement must be selected and


activated separately. There can be multiple RecoverPoint models listed under each
entitlement.
b. Click Start Activation Process. The Search Machines dialog box is displayed.

c. In the Search Machines dialog box, click Add a Machine. The Add Machine dialog
box is displayed.
d. In the Add Machine dialog box, enter a new machine name, and click Save.
A machine name is like a folder, it is used to group items together logically. A
unique machine name must be specified for each entitlement. Machine names can
22

RecoverPoint 4.4 Administrator's Guide

Getting Started

be whatever unique name you choose but it is best practice to relate the machine
name to the Locking ID. For example, in RecoverPoint/SE and RecoverPoint/EX, the
Locking ID is an array serial number. Therefore, the machine name should be RPAPM00120701240 if APM00120701240 is the array serial number used for the
Locking ID. In RecoverPoint/CL, the Locking ID is the RPA Cluster ID, so the
machine name should be the RPA Cluster ID.

2. In the Register screen, verify the machine name, and click Next.
3. In the Activate screen, enter the Locking ID, and click Next.

The Locking ID is the field that is displayed under the Machine Information column;
its value is whichever entity the license is enforced to.
Depending on the entitlement, the Locking ID can be either an RPA Cluster ID (in
RecoverPoint/CL), or an array serial number (in RecoverPoint/SE and RecoverPoint/
EX). To display instructions for finding your RPA Cluster IDs and array serial numbers,
either click the Locking ID help link or see Find your RPA cluster IDs on page 293, Find
your XtremIO cluster IDs on page 293 and Find your array serial numbers on page
291.
4. In the Confirm screen, enter the email addresses of the recipients of the license file in
the Email to field of the Additional Email Options section, and click Finish.
By default, the customer contact provided during order entry and the user performing
the entitlement activation are automatically emailed an activation certificate
containing the license file as an attachment. You can enter multiple email addresses
in the Email to field. Separate multiple email addresses with commas. You can also
include optional comments which will be displayed in the certificate e-mail in the
Email comments field, or add optional notes which can only be viewed online in
Powerlink Licensing in the Notes for this transaction section.

How RecoverPoint licensing works for new purchases

23

Getting Started

5. In the Complete screen, click Save to File to download the license file and save the file
locally. The resulting license file has a *.lic extension and is in plain text format
(can be opened in any text editor).

6. Redo this procedure from Step 1: Quickly access your entitlements on page 20 for
each entitlement in each LAC email or sales order.
7. After you have turned all of your entitlements into license files, physically transfer the
license file(s) to the computer from which you will be running Unisphere for
RecoverPoint, so you can install them in the RecoverPoint system in the next step.

Step 3 - Install your license file(s) in RecoverPoint and enable support


Before you begin

24

Multiple warnings will be issued before an expiring license is about to expire. After
license expiration, replication continues but no further operations will be allowed on
the RecoverPoint system.

The license limitations and their current values (for example, capacity, expiration) for
each licensed RPA cluster and storage array is displayed in the Dashboard > System
Limits pane.

Adding a new permanent or expiring license with the same Locking ID as an existing
expiring license replaces the existing expiring license.

RecoverPoint 4.4 Administrator's Guide

Getting Started

After you have your license files, launch Unisphere for RecoverPoint as described in the
EMC RecoverPoint Installation and Deployment Guide.
When Unisphere for RecoverPoint loads for the first time The Getting Started wizard on
page 28 is displayed. The Getting Started Wizard will guide you, step-by-step, through
the process of installing your license file(s) in RecoverPoint and enabling RecoverPoint
support. If The Getting Started wizard on page 28 is not displayed, install your license
files using Manage Licenses on page 216 and Configure System Reports and Alerts on
page 264.

Upgrade from RecoverPoint 4.0 or later


When upgrading from RecoverPoint 4.0 or later, all existing permanent licenses are
automatically carried over and no further licensing is required.

Upgrade from RecoverPoint versions prior to 4.0


Before you begin
When upgrading to RecoverPoint 4.1 from RecoverPoint versions prior to 4.0, previous
licenses are automatically converted to the new license format. The new licenses are
temporary and valid for 90 days from the date of upgrade. After that, you must have
installed one or more permanent license files to replicate with RecoverPoint appliances.
Use the following procedure to upgrade your RecoverPoint software license from
RecoverPoint 3.5.x to RecoverPoint 4.0 and later versions.
Procedure
1. Open a Service Request with EMC Customer Support:
You can open the Service Request yourself or have an EMC representative open the
Service Request for you.
To have an EMC representative open the Service Request for you:
l

Call 24/7 License Key Live Support for North America, EMEA and APJK by dialing
1-800-782-4362, selecting option 4, and selecting option 4 again.

Go to http://support.emc.com and click the Live Chat button.


To open the Service Request yourself:

a. Go to http://support.emc.com
l

When upgrading to RecoverPoint 4.0, previous licenses will automatically be


converted to the new license format. The new licenses are temporary and valid
for 90 days from the date of upgrade. After that, you must have installed one or
more permanent license files to replicate with RecoverPoint appliances.

Existing EMC Online Support users; when prompted, enter your EMC Support
username and password.

New EMC Online Support users; click Register Now and complete the
registration process.

b. In EMC Online Support, click Service Center in the main toolbar, and then click
Create a Service Request.
c. Fill out the fields in the service request according to the instructions in the
following table:
Upgrade from RecoverPoint 4.0 or later

25

Getting Started

Field

Value

Service
Select Technical Problem/Question from the drop-down.
Request Type
Company and Product Information
Search by
Site ID

This is the geographical location of the data center for which you are
requesting the license. If you know the site ID, enter it in this field.
If you don't know the site ID:
Expand the Advanced Site Search.
Enter your company name and details.
Click Search.
Select the relevant site ID in the Search Results table.
Note

If you don't see your companys site ID in the list, contact EMC Customer
Support.
Product
Details

Select the Choose from all EMC Products radio button.


Start typing the product name into the field to bring up all of the relevant
EMC products.
Select the EMC product for which you are requesting an upgrade from the
search results.

Version
Number

Select the version number of the EMC product that you want to upgrade
from the drop-down.

Contact Information
Initial
Contact is
Best Made
By

Select Email.

Primary
Contact
Name

Your license will be emailed to the email address specified here.


Select the name of the primary contact person at your company for
RecoverPoint management.

Note

If you select Online, all communication regarding your RecoverPoint


license key will be through the SR tracking system on EMC Online Support.

Note

If the email address/es listed under the primary contact name are incorrect
or missing, edit them to update the contact information.
Phone

Enter the phone number as an alternative contact method.


Note

If the phone numbers listed under the primary contact name are incorrect
or missing, edit them to update the contact information.
Problem Details
Problem
Severity

26

RecoverPoint 4.4 Administrator's Guide

Enter a problem severity of 4

Getting Started

Field

Value

Problem
Summary

Type the following text into the problem summary field:


License Upgrade Request for SO#: <The Sales Order Number of Your Last
RecoverPoint Purchase>

Problem
Description

Enter the following:


l

Your current RecoverPoint System ID

All of the array serial numbers of the EMC arrays (VNX, VMAX, VPLEX,
etc.) that require RecoverPoint protection.

Your required replication types (local replication, remote replication)

Your required replication capacity

Your required configuration details (splitter type, splitter sharing,


Stretch CDP, etc.)

d. When youre done filling out the SR details, click the Submit button. An SR
Tracking number is displayed. Write this number down and save it for future
reference.
Note

In most cases, 48 hours after submitting your service request, your RecoverPoint
licenses will be emailed to the specified email address. If you have not received
your RecoverPoint licenses within a month from the time you submitted your
service request, contact EMC Customer Support.
2. Install your RecoverPoint license(s).
When Unisphere for RecoverPoint loads for the first time The Getting Started wizard on
page 28 is displayed. The Getting Started Wizard will guide you, step-by-step,
through the process of installing your license file(s) in RecoverPoint and enabling
RecoverPoint support. If The Getting Started wizard on page 28 is not displayed,
install your license files using Manage Licenses on page 216 and Configure System
Reports and Alerts on page 264.

Get licensing support


Licensing support is available for product ordering, upgrades, license activation and
license installation, according to the following table.
Table 1 Licensing support

Process

Americas

EMEA

APJ

Ordering

1-866-EMC-7777> Option 2

+1-508-435-1000, ext. 54777 > Option 2

Presalessupportcenter@emc.com
Upgrades

1-508-249-5401

US number: +1-508-249-5401

License
Activation or
Installation

http://support.emc.com
Service Center > Create Service Request

Get licensing support

27

Getting Started

Table 1 Licensing support (continued)

Process

Americas

EMEA

APJ

353 (0) 21 487-9862 >


Option 1

353 (0) 21 487-9883

licensing@emc.com
1-800-782-4362 > Option 4 >
Option 4

The Getting Started wizard


The first time Unisphere for RecoverPoint is run after a RecoverPoint installation, the
Getting Started Wizard is displayed to guide users through the steps necessary to
configure the basic RecoverPoint settings needed to ensure their RecoverPoint systems
are up and running as soon as possible.
The Getting Started Wizard contains the following screens:
1. The Welcome screen on page 28 provides a brief description of the wizard and its
functionality.
2. The Add Licenses screen on page 28 allows you to install your RecoverPoint license
and display the license information.
3. The Enable Support screen on page 29 allows you to configure and test the
RecoverPoint system notifications mechanism.
4. The Register RecoverPoint screen on page 30 allows you to fill out the RecoverPoint
post-deployment form and automatically send it to the EMC Install Base (IB) group.

Welcome screen
The first time you access the Unisphere for RecoverPoint Management Application, the
Getting Started Wizard is displayed. To start protecting your data, in the Welcome screen,
click Next Add Licenses to go on to the Add Licenses screen on page 28.

Figure 2 Getting Started Wizard Welcome screen

Add Licenses screen


The Add Licenses screen allows you to install your RecoverPoint license(s) and display
your license information.
28

RecoverPoint 4.4 Administrator's Guide

Getting Started

Figure 3 Getting Started Wizard Add Licenses screen


Note

The settings in the Add Licenses screen are displayed both through the Getting Started
Wizard the first time you launch Unisphere for RecoverPoint, and by selecting Admin >
Manage Licenses.
Procedure
1. Install a license on page 217.
2. Click Next Enable Support to go to the Enable Support screen on page 29.

Enable Support screen


The Enable Support screen allows you to configure the RecoverPoint system reports
mechanism (SyR) and provide one-way communication between a RecoverPoint
installation and the EMC System Reports database (ESRS).
Figure 4 Getting Started Wizard Enable Support screen

Enable Support screen

29

Getting Started

Note

The settings in the Enable Support screen are displayed both through the Getting Started
Wizard the first time you launch Unisphere for RecoverPoint, and by selecting Admin >
System Notifications > Configure System Reports and Alerts.
Procedure
1. Configure system reports and alerts on page 266.
2. Ensure that system reports are configured correctly on page 268 by clicking the Test
Connectivity button.
Note

The connectivity test, if successful, sends a call home event (number 30999) to the
EMC Install Base and opens a Service Request with EMC Customer Support. If a
verification email is provided, EMC Customer Support sends an email to the entered
email address to verify that the system reports mechanism (SyR) has been
successfully configured.
3. Click Next Register RecoverPoint

Register RecoverPoint screen


The Register RecoverPoint screen allows you to fill out and automatically send the
RecoverPoint post-deployment form to the RecoverPoint IB group.
Figure 5 Getting Started Wizard Register RecoverPoint screen

Note

The settings in the Register RecoverPoint screen are displayed through both the Getting
Started Wizard the first time you launch Unisphere for RecoverPoint, and by selecting
Admin > Register RecoverPoint each subsequent time Unisphere for RecoverPoint is
launched.
Procedure
1. Register your RecoverPoint system on page 221.
2. Click the Finish button to close the Getting Started Wizard and apply your changes.
30

RecoverPoint 4.4 Administrator's Guide

Getting Started

A dialog box is displayed allowing you to decide whether or not you would like to
display the Getting Started Wizard in subsequent RecoverPoint launches.

NOTICE

It is not recommended to select the checkbox in the pop-up until you have completed
the tasks of adding your licenses, enabling RecoverPoint support, and registering your
RecoverPoint system.

The Unisphere for RecoverPoint interface


The RecoverPoint interface has been developed using Adobe Flex technology and
simplified to promote ease of use and a common user experience between all EMC
products. The RecoverPoint wizards are streamlined and optimized for a simpler and
more intuitive user experience, targeted at the IT generalist. The Dashboard provides an
overview of system health at a glance, and all of the information displayed through
Unisphere for RecoverPoint is consistently auto-refreshed.
Note

Up to 20 instances of Unisphere for RecoverPoint can be run concurrently on each RPA


cluster.

The Unisphere for RecoverPoint interface

31

Getting Started

Figure 6 Unisphere for RecoverPoint

The Status Bar that consistently appears at the bottom of the interface displays the
numbers of alerts that are currently in the system, and their severity (for example, Alerts:
1 Critical, 3 Errors, 3 Warnings. When you click the status bar from any screen in the
application, you are redirected back to The RecoverPoint Dashboard on page 38.

Set the time display of Unisphere for RecoverPoint


Use the following procedure from any screen in Unisphere for RecoverPoint to set the time
display.
Procedure
1. Click Time displayed on the Unisphere for RecoverPoint status bar.

2. In the Set Time Display dialog box, select the radio button of your preferred time
display option.

NOTICE

Local time is the local time at the RPA that Unisphere for RecoverPoint is connected to.
Results
The time display option you choose will be used by RecoverPoint when performing
operations which require a time stamp such as collecting system information and logging
events.
32

RecoverPoint 4.4 Administrator's Guide

Getting Started

Set the language of Unisphere for RecoverPoint


By default, the English version of th Unisphere for RecoverPoint interface is installed.
However, you can change the language of Unisphere for RecoverPoint and update
Unisphere for RecoverPoint with the most currently released language packs as soon as
they are available, and without waiting for a new RecoverPoint release.
Procedure
1. Download the most current language files from support.emc.com, and save them to
your local drive.
2. In Unisphere for RecoverPoint, select the Settings icon in the upper-right corner to
display the Preference Settings dialog box.

3. In the Preference Settings dialog box, either:

select one of the display languages that are built-into the current RecoverPoint
release from the Display Language combo-box.

click the Click here to update the display language files or help files of Unisphere
for RecoverPoint link to display the Change Unisphere for RecoverPoint Language
Web page.

4. In the Change Unisphere for RecoverPoint Language Web page:


a. Click Choose File, and select the file that you downloaded in step 1 on page 33.
b. Click Apply to upload the file to the RPA and apply the changes to Unisphere for
RecoverPoint.

The RecoverPoint workflow


The RecoverPoint workflow consists of:
Set the language of Unisphere for RecoverPoint

33

Getting Started

First steps in RecoverPoint on page 34

Configuring replication on page 34

Performing recovery activities on page 34

Monitoring and managing RecoverPoint on page 34

System analysis on page 34

RecoverPoint support on page 34

First steps in RecoverPoint


Perform the following steps after completing the installation process, and before starting
to configure your replication environment.
Procedure
1. Access your licenses on page 20 and activate them on page 21 on support.emc.com.
2. Install your RecoverPoint licenses, enable support, and register your RecoverPoint
system using the Getting Started Wizard on page 28.
3. (Optionally) Set the language on page 33 and the time display on page 32.
4. Manage users on page 226, manage roles on page 230, and manage user
authentication on page 233.
5. Register storage on page 187 and Register resource pools on page 195.
6. (Optionally) Change the user security level on page 238.

Configuring replication
After you have completed the First steps in RecoverPoint on page 34, configure the
RecoverPoint system to start replication. The entire process is described step-by-step in
Protecting volumes on page 53.

Performing recovery activities


In case of a disaster, or simply to perform routine maintenance on your production site,
you may wish to test a copy, recover production or fail over operations to another site.
Use cases and procedures for doing so are outlined here on page 145.

Monitoring and managing RecoverPoint


Once replication has been configured and started, you normally wont need more than
minimal involvement with the system.
To troubleshoot your system, if required, refer to The RecoverPoint Dashboard on page
38, Managing Protection on page 78 and Managing Recovery on page 173.
The types of event notification in RecoverPoint, and how to configure events is detailed in
System Notifications on page 253.

System analysis
RecoverPoint contains intelligent System Analysis Tools on page 239 to help you make
your replication environment as efficient as possible.

RecoverPoint support
If you need support, use the Support on page 274 tab to easily access all of the
RecoverPoint support options on support.emc.com.
34

RecoverPoint 4.4 Administrator's Guide

Getting Started

For troubleshooting issues, be sure to collet system information on page 247 before
contacting EMC Customer Support.

RecoverPoint support

35

Getting Started

36

RecoverPoint 4.4 Administrator's Guide

CHAPTER 2
Dashboard

This section describes the RecoverPoint dashboard and how to use it to analyze and
monitor replication in RecoverPoint.
l
l
l
l

The RecoverPoint Dashboard.................................................................................38


Overall Health....................................................................................................... 39
System Limits........................................................................................................45
System Events.......................................................................................................47

Dashboard

37

Dashboard

The RecoverPoint Dashboard


The RecoverPoint Dashboard provides a high-level overview of the RecoverPoint system.
It presents important system information to help you analyze and monitor your
RecoverPoint environment.

Table 2 Dashboard options

Option

When to use

Overall Health To graphically display an overview of the current status of the RecoverPoint
system.
System Limits

To monitor how close you are to breaching your system limitations.

System Events To display the RecoverPoint event log.

38

RecoverPoint 4.4 Administrator's Guide

Dashboard

Overall Health
The Overall Health tab graphically displays an overview of the current status of the
RecoverPoint system. The overall health tab contains widgets that allow you to easily
monitor and analyze the current state of the system.

Figure 7 Dashboard > Overall Health tab

The Overall Health tab of the Dashboard contains the following widgets:
l

RPA Clusters Map on page 40

Alerts on page 42

System Traffic on page 43

Consistency Group Transfer Status on page 43

Overall Health

39

Dashboard

RPA Clusters Map


The RPA Clusters Map widget provides a schematic representation of all RPA clusters in
the RecoverPoint system, the connectivity between them, and any system alerts received
by the clusters.

Figure 8 RPA clusters map widget

The RPA clusters map displays the type of connection that exists between all RPA clusters
in the system. A dashed line represents an IP (WAN) connection and a solid line
represents a Fibre Channel (LAN) connection.
The RPA clusters map also displays the status of all major components in the system that
are seen by the RPA cluster, including; hosts, switches, storage devices, individual RPAs,
the LAN, and the WAN.
When the system detects a problem with one of these major components, the RPA
clusters map displays:
Table 3 RPA clusters map icons

Icon

Description
An error icon on each component that is not functioning properly.
A warning icon on each component that has warnings associated with it.
The status of a component, particularly until you have completed logging in to
the system, may also be Unknown. In that case, the icon is displayed on the
component.

Note

The errors and warnings in the RPA clusters map widget are also displayed in the
Alerts on page 42 widget.
Hover your cursor over an RPA cluster in the map to display a tooltip with the following
links:

40

RecoverPoint 4.4 Administrator's Guide

Dashboard

Table 4 RPA clusters map options

Option

Description

Display Alerts

To display the Alerts dialog box containing the number of alerts registered by
the RPA cluster.
Click Details in the Alerts dialog box to display the alert texts.

Display RPA
To display the General Cluster Info on page 183.
Cluster Details

Multipath monitoring
When multipathing monitoring is active, the system analyzes network errors at the level
of individual paths, and generates a warning in the RPA Clusters Map whenever there is
not full redundancy between the RPA and splitters or volumes. Full redundancy is defined
as follows:
l

For RPA-to-volume links, there must be at least two distinct paths between each RPA
and volume; that is, each RPA has access to at least two storage WWNs (and
controllers, where relevant) via non-overlapping paths.

For RPA-to-splitter links, there must be at least two distinct paths between each RPA
and each splitter using different RPA ports and host (or switch) WWNs.

When RPA Multipath Monitoring is enabled, the system issues a warning upon logging in
regarding any existing links without full redundancy. In addition, warning events are
written to the log.
By default, multipath monitoring is active for all copies, for links both to storage and to
splitters.
To enable or disable RPA multipath monitoring, use the
config_multipath_monitoring CLI command.
Note

Path information is displayed in the Storage on page 185 screen.

Multipath monitoring

41

Dashboard

Alerts
The Alerts widget provides a graphical overview of system health categorized by alert
level.
Figure 9 Alerts widget

Click the More details link at the bottom-right corner of the graph to display the alerts
table.
The alerts table contains system errors and warnings. The errors and warnings are
composed of messages from the System Limits screen, and other general system
warnings.
Figure 10 Alerts table

Hover over an error or warning in the table to display a tooltip containing the error or
warning properties.
When relevant, the dialog will show a severity, category, relevant RPA cluster, and
description of the error or warning.
Note

A system limit with a status of Critical equates to an Error in the alerts table. A system limit
with a status of Major or Minor equates to a Warning in the alerts table.

42

RecoverPoint 4.4 Administrator's Guide

Dashboard

Click the Less details link at the bottom of the alerts table to close the alerts table and
display the Alerts widget.

System Traffic
The System Traffic widget provides a graphical overview of the amount of SAN and WAN
traffic passing through each RPA cluster.

Figure 11 System traffic widget

Consistency Group Transfer Status


The Consistency Group Transfer Status widget displays the data Transfer state on page
43 of all consistency groups in the RecoverPoint system.

Figure 12 Consistency group transfer status widget

Each consistency group and consistency group copy can be in any one of the following
transfer states:
Table 5 Transfer state

Transfer
State

Description

Active

Data is being transferred asynchronously to a copy.


System Traffic

43

Dashboard

Table 5 Transfer state (continued)

Transfer
State

Description

Active
Data is being transferred synchronously to a copy.
(Synchronized
)
Paused

Data is not being transferred to a copy, because transfer has been paused by the
user.

Paused by
system

Data is not being transferred to a copy, because transfer has been paused by the
system.

Init (n%)

A copy is being initialized, or undergoing a full sweep or volume sweep.

High-load

The system enters a permanent high-load state while data is being transferred to
a copy.

High-load (n
%)

The system enters a temporary high-load state while data is being transferred to
a copy.

Replicating
snap

The link is replicating in snap-based replication mode and is currently performing


any of the steps in the snap-based replication process or is performing XtremIO
snap-based replication.

Snap idle

The link is replicating in snap-based replication mode and is about to begin or


has completed a snap-based replication process or has completed an XtremIO
snapshot.

N/A

Data is not being transferred to a copy, because the copy has been disabled by
the user.

The pie chart in the consistency group transfer status widget presents a breakdown of the
consistency groups in the system based on their current data transfer states, as follows:
l

Active (green) - includes active and active (synchronized)

Error (red)

Init (orange) - includes initializing, high load, and high load (n%)

Paused (yellow) - includes paused and paused by system

Disabled (grey) - includes N/A

Other (brown) - includes flushing, flushing buffer, flushing remote buffer to journal, and
unknown

Note

A consistency groups Transfer state on page 43 is determined by its most severe link
state. For example, in a RecoverPoint system that is replicating to two copies, if one link
is paused and one link is active, the transfer state will be set as paused.
If there are no consistency groups in the system, the chart will be replaced by the text No
consistency groups.
Hover your mouse over a transfer state in the chart to see the names of the consistency
groups in that state.

44

RecoverPoint 4.4 Administrator's Guide

Dashboard

System Limits
The System Limits tab monitors important system values to let the user know how close
they are to their limits. The limits may be determined by the system, policies, licensing, or
limitations of external technologies.
Figure 13 Dashboard > System Limits tab

You can sort the system limits by clicking in a column header.


You can filter monitored limits by clicking the Filter icon in the upper-right corner and
entering text to filter by in the text box.
All status severities of minor, major, and critical are also displayed in the Alerts on page
42 widget.

System
The following information is displayed in the System tab:
Table 6 System Limits > System tab

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its
limit.

Description

The description of the monitored system limit.

RPA Cluster

The relevant RPA cluster.

RPA

The relevant RecoverPoint appliance.

System Limits

45

Dashboard

Table 6 System Limits > System tab (continued)

Column

Description

Current Value of
Limit

The current value of the setting and its limit.

Groups
The following information is displayed in the Groups tab:
Table 7 System Limits > Group tab

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its limit.

Description

The description of the monitored system limit.

Group

The relevant consistency group.

Copy

The relevant consistency group copy.

Current Value
of Limit

The current value of the setting and its limit.

Storage
The following information is displayed in the Storage tab:
Table 8 System Limits > Storage tab

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its limit.

Description

The description of the monitored system limit.

RPA Cluster

The relevant RPA cluster.

Array

The name of the relevant storage array.

Current Value
of Limit

The current value of the setting and its limit.

Splitters
The following information is displayed in the Splitters tab (not relevant to XtremIO, which
does not use a splitter):
Table 9 System Limits > Splitters tab

46

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its limit.

RecoverPoint 4.4 Administrator's Guide

Dashboard

Table 9 System Limits > Splitters tab (continued)

Column

Description

Description

The description of the monitored system limit, including:


l

The number of RPA clusters attached to the splitter.

The total number of volumes attached to the splitter.

RPA Cluster

The relevant RPA cluster.

Splitter

The name of the relevant splitter.

Current Value
of Limit

The current value of the setting and its limit.

System Events
The System Events tab displays the RecoverPoint event log which contains all events
logged by the RecoverPoint system, in the order that they are logged. A total of 9000
events can be displayed in the event log.
Figure 14 Dashboard > System Events

Display events
To display system events; select Dashboard > System Events tab.
The RecoverPoint event log is displayed, containing the last 9000 events logged in the
RecoverPoint system. Each event in the event log is displayed along with its relevant
event ID, summary, level, scope, point in time, RPA cluster, and topic.

System Events

47

Dashboard

Note

By default, only Normal scope events are displayed, but you can also Filter events on
page 48 to display Detailed events.

Display additional event information


When an event is selected in the event log, the Details tab under the event log displays
additional event information. To display more information, such as a recommended user
action in cases that warrant user intervention, click the More Information tab.

Sort events
To sort the events in the event log by ID, summary, level, scope, point in time, RPA
cluster, or topic, click on the relevant column header but you can also Filter events on
page 48 based on advanced criteria.

Filter events
Use the following procedure to filter the events in the System Events on page 47 log, and
save the settings for the current RecoverPoint user.
Procedure
1. Select Dashboardand click the System Events tab.
2. Click Filter Events. The Event Filter dialog box is displayed.

3. Events can be filtered using the settings in the table below.


Table 10 Event Filter settings

48

Setting

Values

From

The date and time (in GMT) from which events are to be displayed.
Select Unbounded to display the 9000 events preceding the date and time
specified in the To field.

To

The date and time (in GMT) until which events are to be displayed. Select
Unbounded to display all events up until the current date and time.

RecoverPoint 4.4 Administrator's Guide

Dashboard

Table 10 Event Filter settings (continued)

Setting

Values

Events to
Exclude

The IDs of the events that should not be displayed. Multiple event IDs can be
entered. Separate multiple event IDs with a comma.

Topics

The topic of the events to display in the event log. Valid values are:

Scope

Level

RPA cluster

Splitter

Consistency group

Management

RPA

The scope of the events to display in the event log:


l

Normal: To display only the root cause for an entire set of Detailed and
Advanced events. In most cases, these events are sufficient for effective
monitoring of system behavior.

Detailed: To display all events.

Advanced: In specific support cases, EMC Customer Support may ask you to
display events in an Advanced scope. These events contain information that
is intended primarily for the technical support engineers.

The level of the events to display in the event log:


l

Info: Messages are informative in nature, usually referring to changes in the


configuration or normal system state.

Warning: Message indicates a warning, usually referring to a transient state


or an abnormal condition that does not degrade system performance.

Error: Message indicates an important event that is likely to disrupt normal


system behavior and/or performance.

Containing the Enter the terms that must exist in the event summary in order for the event to be
text
displayed in the event log. Use commas to separate multiple terms. Select:
l

Match all to display all events containing all of the specified terms in the
event summary.

Match any to display all events containing any of the specified terms in the
event summary.

Reset to
Default

To reset all of the settings in the Event Filter dialog box to their default settings.

OK

To close the Event Filter dialog box, and apply the specified settings.

Cancel

To close the Event Filter dialog box, without applying the specified settings.

Copy an event to the clipboard


Procedure
1. Select Dashboardand click the System Events tab.
2. Select the event in the event log and click Copy to Clipboard.
Copy an event to the clipboard

49

Dashboard

Results
The event info that is copied to the clipboard is in the following format:
Summary: RPA is successfully communicating with its cluster.
Level: Clear
ID: 3000
Topic: RPA
Scope: Normal
Point in Time (GMT): 4/17/2013 8:46:35 AM
RPA Cluster: London
Description : RPA is successfully communicating with its
cluster(RPA2)

Collect system information around a specific event


Procedure
1. Attach volumes to the splitter on page 202 for all splitters in the system.
2. Select Dashboardand click the System Events tab.
3. Select an event in the event log.
4. Click Collect System Information. The collect system information wizard is displayed
and the collection start and end fields are automatically set to 10 minutes before and
after the selected event.
5. Continue from step 3 on page 248 of Collect system information on page 248.

50

RecoverPoint 4.4 Administrator's Guide

CHAPTER 3
Protection

This section describes how to protect volumes, and how to monitor and manage
protection and consistency.
l
l
l
l
l

RecoverPoint Protection........................................................................................ 52
Protect Volumes.................................................................................................... 53
Policy Templates................................................................................................... 68
Group Sets............................................................................................................ 73
Manage Protection................................................................................................ 78

Protection

51

Protection

RecoverPoint Protection
In Unisphere for RecoverPoint, data protection is guided by wizards, and performed
through the Protection menu.

Table 11 Protection options

52

Option

When to Use

Protect
Volumes

To add a new consistency group and define its protection plan.


Displays the Protect Volumes wizard to guide you through the process of
defining the production and its protection plan, the copies and their protection
plans, and the production and copy journals and their protection plans.

Policy
Templates

To view or modify existing protection policy templates, or create a new protection


policy template for copies or links.

Group Sets

To create or modify sets of consistency groups, perform recovery activities or


periodically bookmark the same consistent point in time, across volumes in
multiple groups, simultaneously.

Manage
Protection

To manage and monitor existing consistency groups and their protection plans,
control transfer, or bookmark consistent points in time across groups and sets of
groups.

RecoverPoint 4.4 Administrator's Guide

Protection

Protect Volumes
Use Protection > Protect Volumes to create consistency groups (CGs) to protect your
volumes. In RecoverPoint, CGs are used to configure protection policies, and set RPO and
RTO policies according to specific resource allocation and prioritization.

Protecting volumes
Creates a new enabled consistency group.
Before you begin
l

Journal volumes can be automatically provisioned. To automatically provision journal


volumes, RecoverPoint requires dedicated resource pools. If you are planning to use
the automatic journal provisioning feature, make sure you have dedicated resource
pools on storage. You must register storage and register resource pools to enable
automatic volume provisioning.

For RecoverPoint with VNX or VNXe with RecoverPoint/SE licenses, there is a


limitation of one array per side. If you have a local and a remote copy, make sure the
local copy is stored on the same VNX/VNXe/CLARiiON array as the production copy.

Ensure you have your RecoverPoint environment planning document describing the
groups and journals you will need to create, the volumes that you will need to define
in each, and the RPAs that will handle the data transfer. If you do not have this
document, contact your RecoverPoint account executive.
Your environment planning document should detail:

The number of consistency groups that you will need, and their names.

The LUNs to include in each group, and at each copy.

The journal LUNs at each copy.

Which consistency groups to set as distributed.

The required RPO and RTO settings per consistency group.

The RPA that will handle the replication of each group.

The priority of each consistency group with relation to the other groups in the
system.

Decide whether you will want to initialize the groups from a backup on page 280,
and prepare accordingly.
When a consistency group is first created, or a replication set is first added to the
system, the system automatically starts a first-time initialization process at the
Protect Volumes

53

Protection

copies (that is, synchronizing the copy volumes with the production volumes by
sending the complete image of each production volume to each copys storage).
Depending on the amount of data being transferred, this process can be both timeconsuming and bandwidth-consuming. To save time and bandwidth, you can
initialize the volumes from a backup as presented in First-time initialization is taking
too long on page 280 for this procedure).
l

If your hosts are part of a Windows clustered environment or running on an AIX


operating system, you will need to perform a first-time failover as soon as possible
after creating your consistency groups.
n

If your hosts are part of a Windows clustered environment, see the First-time
failover section of the EMC RecoverPoint Replicating the Windows File System
Technical Notes for the specific procedure.

If your hosts are running on an AIX operating system, see the First-time failover
section of the EMC RecoverPoint Deploying with AIX Hosts Technical Notes for the
specific procedure.

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that copy
must be on the same array.

Ensure you have mapped all replication LUNs to RecoverPoint WWNs. Only the
volumes that are mapped and zoned to the RPA cluster are displayed in the available
volumes list in RecoverPoint.
Note

In RecoverPoint/SE only, all of the volumes of the VNX Series array are displayed in
the available volumes list.
l

For applications running on a host cluster:


The RecoverPoint system supports both local and remote high-availability host
clusters.
n

Place all volumes that are resources of the host cluster in a single consistency
group. Simplify management by assigning the name of the host cluster as the
name of the consistency group.
Note

The Journal volumes must not be resources of the host cluster.

54

Assign each replication set the same name as its disk resource in the cluster. If all
cluster nodes (hosts) at the side are down, you may not be able to create
replication volumes. To correct this problem, either bring up the nodes, or run a
rescan_san command (from the CLI), with volumes=FULL.

Best practice is to perform first-time failover as soon as configuration is


completed. First-time failover instructions are different for different host clusters.
Refer to the EMC Technical Notes for your host cluster to perform first-time
failover.

Prior to enabling a consistency group, the system Dashboard may show an Error
on the source-side storage and display the message Volume cannot be accessed
by any RPA. The errors are removed when the consistency group is enabled.

In VNX storage systems:


n

There is a limit on the number of LUNs that are supported per storage group (see
the EMC RecoverPoint Release Notes for this limit).

If you need more LUNs than are currently available to support your replication sets
and journals, either because you have calculated your system needs or because

RecoverPoint 4.4 Administrator's Guide

Protection

you have received a message that you have exceeded the maximum number of
supported LUNs, you may be able to create additional storage groups, according
to the procedure presented in Add more LUNs than allowed per VNX storage
group on page 290.
l

The VNXe storage system does not use storage groups.

In Symmetrix splitter environments:

Volumes attached to VNX splitters and volumes attached to Symmetrix splitters


may be mixed in one consistency group. See the EMC RecoverPoint Deploying with
Symmetrix Arrays and Splitter Technical Notes for other splitter-specific limitations.

If any volume added is a device on a Symmetrix VMAX 10K, 20K or 40K array, refer
to Device Tagging in the EMC RecoverPoint Deploying with Symmetrix Arrays and
Splitter Technical Notes.

In VNX splitter environments:


n

The same consistency group can contain volumes attached to multiple splitters.

The same consistency group can contain volumes using a VNX splitter to split
writes to one RPA cluster and volumes using a different splitter type to split writes
to a different RPA cluster.

In VNXe splitter environments, A consistency group can contain only volumes


attached to VNXe splitters.

In VPLEX splitter environments:

The VPLEX consistency group name will only be displayed in the VPLEX Group
column of the volume list if you Attach volumes to the splitter on page 202 for the
VPLEX splitter (including certificate).

After you register your VPLEX storage, you will not be able to select VPLEX volumes
and non-VPLEX volumes in the same consistency group copy. It is recommended
that all volumes in a VPLEX consistency group be configured in a single
consistency group copy, and all volumes in a non-VPLEX consistency group be
configured in another consistency group copy.

A VPLEX MetroPoint consistency group can contain a maximum of one remote


copy and from zero to two local copies. If there is no remote copy, there must be
two local copies, one at each side of the VPLEX Metro.

To turn a non-MetroPoint group into a MetroPoint group, when the production


volume is a VPLEX distributed volume and there exists no more than one remote
copy, you can Add a copy on page 116.

To turn a MetroPoint group into a non-MetroPoint group, you can select the
groups standby production copy in the Manage Protection on page 78 screen,
and Remove a copy on page 123.

For replicating Oracle databases, see the instructions on how to replicate an Oracle
database, including using Oracle hot backup procedures with RecoverPoint
bookmarks for point-in-time snapshots and quick testing and disaster recovery in the
EMC RecoverPoint Replicating Oracle with EMC RecoverPoint Technical Notes.

Procedure
1. Select Protection > Protect Volumes.
2. In the Select Production Volumes screen:
a. Define the consistency group name, the production name, and the RPA cluster that
will manage the production.

Protecting volumes

55

Protection

Note

A MetroPoint Group checkbox is displayed if one or more distributed volumes


exists at the production copy, and is mapped to the RPA cluster of the production
copy. To create a MetroPoint group, select the MetroPoint group check box. When
selected, only VPLEX distributed volumes are displayed in the volume list.
b. Select your production volumes in the volume list.
Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that
copy must be on the same array. Use the toolbar at the top-right corner of the
screen to help you identify the volumes.
Table 12 Toolbar options

Icon

Description
To filter the volumes in the list by Size, Product, Vendor, Name, UID, LUN,
or VPLEX Group.
Note

The VPLEX Group column is only displayed after you Attach volumes to the
splitter on page 202 for your VPLEX splitter.
To select the table columns that you want to display.
Note

You can also click and drag the column dividers to make the columns
wider or thinner. In most cases, RecoverPoint will remember the column
width that you define, and display the columns accordingly in each
subsequent launch of this wizard.
To export the data in the volume list to a *.CSV file.

56

RecoverPoint 4.4 Administrator's Guide

Protection

Table 12 Toolbar options (continued)

Icon

Description
To display context-sensitive help for this wizard screen.

c. Optionally, click Modify Policies to define the group policies and copy policies
now, or skip this step and define the policies later.
The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is a
specific business need to set other policies.
d. Click Next Define the Production Journal.
3. In the Define Production Journal screen:

If the volume that you specified in the last step is not on a VNX/VNXe/CLARiiON array,
skip to step a on page 59
If the volume that you specified in the last step resides on a VNX/VNXe/CLARiiON
array that has already been registered in RecoverPoint, and a resource pool was
already registered for the same VNX/VNXe/CLARiiON array, the Journal Provisioning
Method screen is displayed.

Protecting volumes

57

Protection

If you want to allow RecoverPoint to automatically provision your journal volumes,


ensure Automatically Provision Journal Volumes is selected, select the relevant
resource pool, and select one of the following options:
Option

Description

Journal size

Enter a value for the journal size to define an absolute journal


size at the production copy.

Predicted
incoming
write-rate

Select this option and enter a value for predicted incoming writerate and required protection window to have RecoverPoint/SE
calculate the required journal size based on your consistency
groups predicted write-rate and a required protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.

Required
protection
window

Note

Only relevant if Predicted incoming write-rate is selected.


Enter a value for the required protection window to have
RecoverPoint calculate the required journal size based on your
consistency groups predicted write-rate and a required
protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.

Tiering Policy

(Optionally) Select one of the tiering policies available on storage


to automatically apply it to the selected volumes.

The following information is displayed for each resource pool in the table:

58

RecoverPoint 4.4 Administrator's Guide

Protection

Information

Description

Name

The name of the registered resource pool in the


RecoverPoint system.

Type

The resource pool type. Valid resource pool types are:


n

RAID groups

Storage groups

Datastores

Total Size

The total capacity of the resource pool.

Available Size

The total capacity of the resource pool minus the capacity


of the volumes that were already allocated in the resource
pool.

Tiers

The physical type of the storage. For example, Fibre


Channel, SATA, etc.

Available Tiering
Policies

The tiering policies that are available on storage and can


be automatically applied to volumes in RecoverPoint when
users automatically provision volumes.

Storage Name

The name of the storage array in RecoverPoint.

Serial Number

Serial number of the storage array.

Array Type

The type of the array.

IP

The arrays IP address.

If there are Available Tiering Policies on storage, you can select a Tiering policy to
apply to the journal volume/s from the combo-box at the bottom of the resource
pool table now.
l

If you do not want RecoverPoint to automatically provision journal volumes on


storage for you, ensure the Select Provisioned Journal Volumes radio button is
selected.
In the volume list:
a. Select the volumes to add to the journal.
For XtremIO arrays, the journal need not be more than 10 GB for nondistributed consistency groups and 40 GB for distributed consistency groups,
since the XtremIO journal contains only pointers to snapshots and metadata.
For best performance, select volumes that are identical in size. If identically
sized volumes are not available, select volumes that are similar in size.
Use the toolbar icons at the top-right corner of the screen to help you identify
the required volumes in the volume list.
In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped
and zoned to the RPA cluster are displayed in the available volumes list.
In RecoverPoint/SE, all of the volumes of the supported VNX Series arrays are
displayed in the available volumes list.
b. Optionally, click Modify Policy to define the journal policies now, or skip this
step and define the policies later from the Manage Protection screen. The
policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is
a specific business need to set other policies.
Protecting volumes

59

Protection

If you want to change the protection settings using policy templates, use the
Select Template drop-down to select one of your templates. Click Load to load
the settings in the template and click OK to apply the settings.
c. Click Next Add a Copy.
4. If you did not select the MetroPoint group checkbox in step 2, continue to step 5 on
page 60. If you did select the MetroPoint group checkbox in step 2, the Standby
Production screen is displayed. In the Standby Production screen:

a. Enter a name for the copy in the Standby production copy name field.
b. In the volume list, mark the checkboxes of each journal volume that you want to
use in your standby production journal.
Use the toolbar icons at the top-right corner of the screen to help you identify the
required volumes in the volume list. For best performance, select volumes that are
identical in size. If identically sized volumes are not available, select volumes that
are similar in size.
These are the copy journal volumes for the standby copy, not the copy volumes.
c. Optionally, click Modify Policy to define the journal policy now, or skip this step
and define the policies later from the Manage Protection screen.
The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is a
specific business need to set other policies.
If you want to change the protection settings using policy templates, use the
Select Template drop-down to select one of your templates. Click Load to load the
settings in the template and click OK to apply the settings.
d. Click Next Add a Copy.
5. In the Add a Copy screen:

60

RecoverPoint 4.4 Administrator's Guide

Protection

a. Define the copy name, the RPA cluster that will manage the copy volumes, and the
replication mode on page 82.
For each production volume, click the Select volume link to add a volume to the
copy. The selected volume is the volume that the production volume is replicated
to.
The volume list is displayed. The volume list only contains volumes of sizes that
are equal to, or greater than, the specified production volume.
Note

If one volume in a copy is on an XtremIO array, all the volumes in the consistency
group at that copy must be on an XtremIO array. If the copy is on an XtremIO array,
it must be exactly the same size as all other copies in the consistency group. It
cannot be either larger or smaller.

Note

In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped and
zoned to the RPA cluster are displayed in the available volumes list. In
RecoverPoint/SE, all of the volumes of the supported VNX Series array are
displayed in the available volumes list.
Use the toolbar icons at the top-right corner of the screen to help you identify the
required volumes in the volume list. For best performance during failover, select
Protecting volumes

61

Protection

volumes that are the same size as the production volume. If volumes of the same
size are not available, select volumes that are as similar in size as possible.
NOTICE

You cannot proceed to the next step until the replication sets icon under the
volume list shows a green check mark to indicate that the number of replication
sets is equal to the number of production volumes.

b. Optionally, click Modify Policy to define the copy policies and link policies now, or
skip this step and define the policies later from the Manage Protection screen.
If you want to change the protection settings using policy templates, in the Link
tab, use the Select Template drop-down to select one of your templates. Click Load
to load the settings in the template and click OK to apply the settings.
Note

The Modify Policy link is only enabled after volumes are selected in the volume
list. You can set a copy policy for the production, however, the copy policy of the
production is only relevant after failover, when the production becomes a copy.
c. To continue configuring the group, click Next Define Copy Journal.
6. In the Define Copy Journal step:
If the copy volume that you specified in the last step is not on a VNX/VNXe/CLARiiON
array, skip to step a on page 65
If the volume that you specified in the last step resides on a VNX/VNXe/CLARiiON
array that has already been registered in RecoverPoint, and a resource pool was
already registered for the same VNX/VNXe/CLARiiON array, the Journal Provisioning
Method screen is displayed.

62

RecoverPoint 4.4 Administrator's Guide

Protection

If you want to allow RecoverPoint to automatically provision your journal volumes,


keep the Select Provisioned Journal Volumes radio button selected, select the
relevant resource pool from the table, and select one of the following options:
Option

Description

Journal size

Enter a value for the journal size to define an absolute journal


size at the copy.

Predicted
Select this option and enter a value for predicted incoming
incoming write- write-rate and required protection window to have RecoverPoint
rate
calculate the required journal size based on your consistency
groups predicted write-rate and a required protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate
and the required protection window is dynamically updated in
the Calculated journal size field.
Required
protection
window

Note

Only relevant if Predicted incoming write-rate is selected.


Enter a value for the required protection window to have
RecoverPoint calculate the required journal size based on your
consistency groups predicted write-rate and a required
protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate
and the required protection window is dynamically updated in
the Calculated journal size field.

Tiering Policy

(Optionally) Select one of the available tiering policies


displayed in the Resource pool provisioning information table
below to automatically apply the policy to the selected
volumes.

Protecting volumes

63

Protection

Option

Description

Back to Select
Production
Volumes

Go back to the previous screen and re-select the production


volumes, or (optionally) redefine the production protection
policy.

The following information is displayed for each resource pool in the table:
Information

Description

Name

The name of the registered resource pool in the


RecoverPoint system.

Type

The resource pool type. Valid resource pool types are:


n

RAID groups

Storage groups

Datastores

Total Size

The total capacity of the resource pool.

Available Size

The total capacity of the resource pool minus the capacity


of the volumes that were already allocated in the resource
pool.

Tiers

The physical type of the storage. For example, Fibre


Channel, SATA, etc.

Available Tiering
Policies

The tiering policies that are available on storage and can


be automatically applied to volumes in RecoverPoint when
users automatically provision volumes.

Storage Name

The name of the storage array in RecoverPoint.

Serial Number

Serial number of the storage array.

Array Type

The type of the array.

IP

The arrays IP address.

If there are Available Tiering Policies on storage, you can select a Tiering policy to
apply to the journal volume/s from the combo-box at the bottom of the resource
pool table now.
When you are done configuring your journal settings, click Next Display Group
Summary.
l

If you do not want RecoverPoint to automatically provision your journal volumes on


storage, ensure Select Provisioned Journal Volumes is selected and:
For applications running on a host cluster, the journal volumes must not be
resources of the host cluster.

64

RecoverPoint 4.4 Administrator's Guide

Protection

a. Select the volumes to add to the journal.


The journal for an XtremIO array need not be larger than 10 GB for a copy with
non-distributed consistency groups, 40 GB for distributed consistency groups.
Use the toolbar icons at the top-right corner of the screen to help you identify
the required volumes in the volume list. For best performance, select volumes
that are identical in size. If identically sized volumes are not available, select
volumes that are similar in size.
In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped
and zoned to the RPA cluster are displayed in the available volumes list. In
RecoverPoint/SE, all of the volumes of the supported VNX Series array are
displayed in the available volumes list.
b. Optionally, click Modify Policy to define the journal policies now, or skip this
step and define the policies later using the Manage Protection screen.
If you want to change the protection settings using policy templates, use the
Select Template drop-down to select one of your templates. Click Load to load
the settings in the template and click OK to apply the settings.
Note

The Modify Policy link is only enabled after volumes are selected in the volume
list.
c. Click Next Display Group Summary
7. In the Group Summary screen:

Protecting volumes

65

Protection

a. Use the following options until the consistency group diagram reflects your
requirements:
Table 13 Group summary screen

Action

Description

Edit

Opens the Add a Copy screen of the copy whose Edit link you clicked,
allowing you to edit the copy settings, and modify the copy volume
configuration, replication mode, or protection plan as described in
step 5 on page 60.

Delete

Removes the copy whose Delete link you clicked, from the consistency
group.
Note

You cannot delete the production copy.

Replication
Sets

Displays detailed volume information on all replication set volumes that


have been defined for the consistency group. Select a row in the
replication sets table in the top section to display the associated volume
information in the bottom section.
For each volume the following information is displayed; vendor, array,
serial number, volume, UID, size and VPLEX group.
Click on the replication set name to open the Edit Replication Set dialog
box, and rename the replication set.

66

RecoverPoint 4.4 Administrator's Guide

Protection

Table 13 Group summary screen (continued)

Action

Description
Note

When you add a replication set to a consistency group, RecoverPoint


ensures consistency between the copies and the production source by
performing a volume sweep synchronization on all of the volumes in the
new replication set. Selecting the Attach as Clean checkbox overrides the
default synchronization process by informing the system that the new copy
volumes are known to be an exact image of their corresponding production
volumes. If Attach as Clean is checked, and the copy volumes are
inconsistent with their corresponding production volumes, they will remain
inconsistent. To ensure consistency, best practice is not to use Attach as
Clean and allow RecoverPoint to perform the volume sweep.

Start
replication
to all
copies in
group when
I click
'Finish'

When selected, starts transfer for the group as soon as the group is
created.
NOTICE

If a consistency group contains one journal created using automatic volume


provisioning and one journal whose volumes were selected manually this
setting will not work and you will have to start transfer for the group
manually.
A first-time initialization (or fast first-time initialization) occurs when a
consistency group is enabled for the first time.

Add a Copy

Finish

Opens the Add a Copy screen, allowing you to create another copy, define
the new copys settings, and define the copys volume configuration,
replication mode, and protection plan.
Note

Before you start transfer to any copy, ensure that the copy volumes are
unmounted from any hosts and any volume groups are deported from the
logical volume manager (AIX, HP-UX, Windows, and Solaris have volume
managers built into the operating system; Veritas Volume Manager can be
used with any of these operating systems).
Creates the group, applies all of the specified settings, and starts a firsttime initialization process to synchronize the production volumes to the
copy volumes.
Before you click Finish, note the following:
l

If First-time initialization is taking too long on page 280, instead of


allowing RecoverPoint to send the initialization data to the copy
storage, you can initialize the copy volumes from a backup disk, saving
the time it would otherwise take to synchronize the data over a WAN or
FC connection.

By default, RecoverPoint writes the first snapshot directly to the copy


storage, bypassing the copy journal. Using the fast_first_init
parameter of the config_link_policy CLI command, you can
override the default setting and write the initialization snapshot first to

Protecting volumes

67

Protection

Table 13 Group summary screen (continued)

Action

Description
the journal (which is more time-consuming but provides greater data
protection).
l

To enable failover during initialization, set the value of the


allow_long_resync parameter of the config_copy_policy
CLI command to no (default is yes).
If your hosts are part of a Windows clustered environment or running
on an AIX operating system, you will need to perform a first-time
failover as soon as possible after first-time initialization is complete.

If your hosts are part of a Windows clustered environment, see the


First-time failover section of the EMC RecoverPoint Replicating the
Windows File System Technical Notes for the specific procedure.

If your hosts are running on an AIX operating system, see the Firsttime failover section of the EMC RecoverPoint Deploying with AIX Hosts
Technical Notes for the specific procedure.

b. Click Finish to create the group, and all of it's copies.


The group is created and the Manage Protection screen is displayed.
After you finish
You can now manage protection on page 78 or start recovery on page 146.

Policy Templates
Use Protection > Policy Templates to view or modify existing protection policy templates
or create new protection policy templates, for copies or links.

Three default policy templates are included in RecoverPoint:


Table 14 Default policy templates

68

Name

Description

default copy

Provides a practical configuration for copy policies on page 86 and journal


policies on page 87.

RecoverPoint 4.4 Administrator's Guide

Protection

Table 14 Default policy templates (continued)

Name

Description

default remote Provides a practical configuration for remote link policies on page 83.
link
default local
link

Provides a practical configuration for local link policies on page 83.

Display all policy templates


Use the following procedure to display the list of policy templates currently configured in
the RecoverPoint system.
Procedure
1. Select Protection > Policy Templates.

2. You cannot remove the default policy templates, but you can create new ones on page
69 based on the default template settings and modify on page 70 any of the userdefined policy templates.

Create new policy templates


Before you begin
l

Only users with Group permissions can create new policy templates.

Use the Copy policy template to control your journal policies on page 87.

If a policy template contains an invalid value for an array, you will not be able to load
it to the array. For example, In XtremIO, if you select Enable Snapshot Consolidation
and then try to apply the template to copy on an XtremIO array, loading the template
will fail because snapshot consolidation is not supported on XtremIO arrays.

Procedure
1. In the list of policy templates, click Add.

Display all policy templates

69

Protection

The Add Policy Template dialog box is displayed.


2. In the Type field, select the type of policy template that you want to create.
Valid types of policy templates are Copy and Link.
The settings change according to the type of template that you select.
3. In the Name field, enter a descriptive name for your new policy template.
4. Depending on the type of template that you selected, configure the settings in the
template according to the instructions in copy policies on page 86, journal
policies on page 87 or link policies on page 83.
5. Click OK.

Modify existing policy templates


Before you begin
l

Only users with Group permissions can modify policy templates.

When you modify a policy template, all settings based on the template do not
automatically change. You will need to re-load and apply the modified policy
template to use the new settings.

Use the Copy policy template to control your journal policies on page 87.

If a policy template contains an invalid value for an array, you will not be able to load
it to the array. For example, In XtremIO, if you select Enable Snapshot Consolidation
and then try to apply the template to copy on an XtremIO array, loading the template
will fail because snapshot consolidation is not supported on XtremIO arrays.

Note

You cannot modify any of the default policy templates on page 68.
Procedure
1. In the list of policy templates, select a template and click Edit. The Edit Policy
Template dialog box is displayed.

70

RecoverPoint 4.4 Administrator's Guide

Protection

NOTICE

You cannot change the Type of an existing template.


2. Depending on the type of template that you selected, modify the settings in the
template according to the instructions in copy policies on page 86, journal
policies on page 87 or link policies on page 83.
3. Click OK.

Remove a policy template


Before you begin
l

Only users with Group permissions remove policy templates.

You cannot remove the default Copy and Link policy templates, and you cannot
change their settings. You can, however, create new policy templates and base them
on the settings of the default policy templates.

Procedure
1. In the list of policy templates, select a template.
2. Click Remove, and click Yes in the confirmation dialog box.
NOTICE

You cannot delete any of the default policy templates on page 68.

Apply a policy template to a new consistency group


Use the Protect Volumes on page 53 wizard to apply policy templates to copies, journals
and links during group creation according to the following procedure.
Procedure
1. Define the Journal policies on page 87 in the production journal and copy journal
creation screens:
a. Select the Modify Policy link.
b. Use the Select Template drop-down to select one of your templates.
c. Click Load to load the settings in the template.
Remove a policy template

71

Protection

d. Click OK to apply the settings.


2. Define the Link policies on page 83 in the copy creation screen:
a. Select the Modify Policies link.
b. Select the Link tab.
c. Use the Select Template drop-down to select one of your templates.
d. Click Load to load the settings in the template.
e. Click OK to apply the settings.

Apply a policy template to an existing consistency group


Use the Protection > Manage Protection to apply policy templates to copies, journals and
links after group creation according to the following procedure.

Define a protection policy for existing group links


Procedure
1. Select Protection > Manage Protection
2. In the left pane, expand the consistency group and select one of its copies.
3. In the right pane, click the Link Policy tab.
4. In the Link Policy tab, for each link:
a. Select the link to which you want to apply the template settings.
b. Use the Select Template drop-down to select the template.
c. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
d. Click Apply to apply the settings to the selected link.

Define a protection policy for existing group journals


Procedure
1. Select Protection > Manage Protection
2. In the left pane, expand the consistency group and select one of its copies.
3. In the right pane, click the Copy Policy tab.
4. In the Copy Policy tab:
a. Use the Select Template drop-down to select the template.
b. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
c. Click Apply to apply the settings to the copy journal.

Define link and journal policies while adding a copy to an existing group
Procedure
1. Select Protection > Manage Protection

72

RecoverPoint 4.4 Administrator's Guide

Protection

2. In the left pane, select a consistency group.


3. In the right pane, select Status > Add Copy.
4. In the production journal and copy journal creation screens, define the journal
policies on page 87:
a. Select the Modify Policy link.
b. Use the Select Template drop-down to select one of your templates.
c. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
d. Click OK to apply the settings.
5. In the copy creation screen, define the link policies on page 83:
a. Select the Modify Policies link.
b. Select the Link tab.
c. Use the Select Template drop-down to select one of your templates.
d. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
e. Click OK to apply the settings.

Group Sets
Use Protection > Group Sets to create or modify sets of consistency groups, perform
operational and recovery activities, and periodically bookmark the same consistent point
in time across volumes in multiple groups, simultaneously.

Note

Parallel bookmarks cannot be enabled for group sets containing consistency groups with
volumes that reside on an XtremIO array or a Data Domain.

How group sets work


The user selects a set of groups, enters a name for the group set, and (optionally) sets the
frequency at which parallel bookmarks will be periodically applied to the group set.
Group Sets

73

Protection

If parallel bookmarking is enabled, a bookmark is generated at the specified interval for


all groups in the group set. The bookmark will consist of the group set name that the user
specifies and an automatically incremented number starting at zero, and incremented up
to 65535 (and then, beginning at zero again).
Group sets can be used to perform actions on all of the groups defined in it
simultaneously, such as; enabling and disabling image access, enabling direct access,
undoing writes, failing over, recovering production, starting and stopping transfer, and
enabling or disabling groups.

Display all group sets


Use the following procedure to display all group sets currently configured in the
RecoverPoint system.
Before you begin
l

Only users with Group permissions can add, edit or remove group sets.

The Enable, Disable, Start Transfer, and Pause Transfer buttons are only enabled if
they are relevant for at least one of the groups in the group set.

Procedure
1. Select Protection > Group Sets.

2. All of the group sets in the system are displayed, along with their system information.

Create a group set


Before you begin

74

Parallel bookmarks cannot be enabled for group sets containing consistency groups
with volumes that reside on an XtremIO array or a Data Domain.

Only users with Group permissions can add, edit or remove group sets.

The maximum number of group sets in a RecoverPoint system is limited to 128.


However, each group set can contain an unlimited number of consistency groups,
unless parallel bookmarking is enabled.

While parallel bookmarking is enabled for a group set:


n

The maximum number of groups in the group set is limited to 32.

You cannot add or remove groups from the group set.

The parallel bookmarking interval should be greater than 20 seconds.

You cannot create the group set if any of the groups that you want to include are
already part of another group set that has parallel bookmarking enabled.

RecoverPoint 4.4 Administrator's Guide

Protection

You cannot create the group set if any of the groups that you want to include
contain links for which snap-based replication mode has been enabled.

You cannot create a group set if one of its groups is not replicating in the same
direction (that is, from the same source) as other groups in the group set.

Although groups with mixed topologies (that is, with a different number of copies)
can be included in the same group set, for recovery purposes, this is not
recommended.

Procedure
1. Select Protection > Group Sets .
2. Click Add. The Add Group Set dialog box is displayed.

3. In the Add Group Set dialog box:


a. In the Name field, enter a descriptive name for the group set.
b. In the list of consistency groups, select one or more consistency groups to add to
the group set.
All consistency groups in a group set must be replicating in the same direction.
You cannot enable parallel bookmarking for a group set if any of the groups in the
group set are part of another group set that has parallel bookmarking enabled.
c. Ensure that the Enable Parallel Bookmarking checkbox reflects your requirements.
If it is selected, ensure that the Frequency list box contains the required value.
Enabling parallel bookmarking limits the maximum number of groups in the group
set to 32. You cannot add or remove groups from a group set with parallel
bookmarking enabled. The parallel bookmarking frequency should be no less than
20 seconds. Parallel bookmarks cannot be enabled for group sets containing
consistency groups with volumes that reside on an XtremIO array.
d. Click OK.

Edit a group set


Before you begin
l

Parallel bookmarks cannot be enabled for group sets containing consistency groups
with volumes that reside on an XtremIO array.
Edit a group set

75

Protection

Only users with Group permissions can edit group sets.

Procedure
1. Select Protection > Group Sets .
2. Select a group set in the list of group sets.
3. Click Edit. The Edit Group Set dialog box is displayed.

4. In the Edit Group Set dialog box:


a. In the Name field, modify the group set name as required.
Although you can modify the group set name, it is not recommended.
b. Select or clear the checkboxes of groups in the list of groups to include or exclude
them from the groups set.
You will not be able to enable parallel bookmarking for the group set if any of the
groups in the group set are part of another group set that has parallel bookmarking
enabled.
c. Ensure that the Enable Parallel Bookmarking checkbox reflects your requirements.
If it is selected, ensure that the Frequency list box contains the required value.
Parallel bookmarks cannot be enabled for group sets containing consistency
groups with volumes that reside on an XtremIO array. You cannot disable parallel
bookmarking if it is enabled, but you can change the Frequency. Enabling parallel
bookmarking limits the maximum number of groups in the group set to 32. You
cannot add or remove groups from a group set with parallel bookmarking enabled.
The parallel bookmarking frequency should be no less than 20 seconds.
d. Click OK.

Remove a group set


Before you begin
l

Only users with Group permissions can remove group sets.

This action cannot be undone.

Procedure
1. Select Protection > Group Sets .
76

RecoverPoint 4.4 Administrator's Guide

Protection

2. Select one or more group sets in the list of group sets.


3. Click Remove.

Enable or disable all groups in a group set


Before you begin
l

When transfer starts for a disabled group, a full sweep occurs.

The Enable and Disable buttons are only enabled if they are relevant for at least one
of the groups in the group set.

Procedure
1. Select Protection > Group Sets .
2. Select the group set from the list of group sets.
3. Click Enable or Disable.

Pause or start transfer for all groups in a group set


Use the following procedure to start or pause the transfer of writes between the
production and all copies in all groups of a group set.
Before you begin
l

After transfer starts, a short initialization occurs.

The Start Transfer and Pause Transfer buttons are only enabled if they are relevant
for at least one of the groups in the group set.

Procedure
1. Select Protection > Group Sets .
2. Select the group set from the list of group sets.
3. Click Start Transfer or Pause Transfer.

Apply a bookmark to all of the groups in a group set


Procedure
1. Select Protection > Group Sets .
2. Select the group set from the list of group sets.
3. Click Apply Bookmark.

Initiate recovery activities for a group set


You can trigger recovery activities for a group set from both the Protection and the
Recovery menus.
l

To test a copy, undo writes, enable direct access or disable image access for all
consistency groups in a group set:
n

Select Protection > Group Sets. Select a group set from the list of group sets, click
Test Copies, and follow the instructions in Test a Copy on page 147.

Select Recovery > Test a Copy and follow the instructions in Test a Copy on page
147.

To test a copy and recover production of all consistency groups in a group set:

Enable or disable all groups in a group set

77

Protection

Select Protection > Group Sets. Select a group set from the list of group sets, click
Recover Production, and follow the instructions in Test a Copy and Recover
Production on page 160.

Select Recovery > Recover Production and follow the instructions in Test a Copy
and Recover Production on page 160.

To test a copy and fail over all consistency groups in a group set:
n

Select Protection > Group Sets. Select a group set from the list of group sets, click
Fail Over, and follow the instructions in Test a Copy and Fail Over on page 165.

Select Recovery > Fail Over and follow the instructions in Test a Copy and Fail
Over on page 165.

Manage Protection
Use the Manage Protection screen to manage and monitor your existing consistency
groups and their protection plans, control transfer, and mark consistent points in time
across groups and sets of groups.

When the Manage Protection screen is first displayed, the Consistency Groups node is
selected in the left pane, and the list of all consistency groups in the system is displayed
in the right pane along with the status of every consistency group in the RecoverPoint
system.

Use this screen for monitoring all consistency groups, as follows:

78

RecoverPoint 4.4 Administrator's Guide

Protection

Table 15 Multiple consistency group monitoring options

Column

Description

Name

The name of the consistency group.

Status

Whether a group is Enabled or Disabled.

Transfer

The group transfer status (Paused, Active, Initializing, etc.). When a group is
configured to transfer data synchronously, the word Synchronized is displayed
next to the state of transfer.
Note

You can also see the data transfer state on page 43 of consistency groups
through the Consistency Group Transfer Status on page 43 widget in the
Dashboard.
Alerts

The number of alerts that have been logged for a group. If alerts have been
logged, click the link to display the alert texts.

Protected Size The size of the data that is being protected by the group.
Active RPA

The RPA that is the preferred RPA.

Group Set

The group set name, if the group has been added to a group set.

You can also perform the following actions on multiple groups in the system
simultaneously, by clicking the relevant button at the bottom of the screen:
Table 16 Multiple consistency group actions

Button

Description

Remove Group Remove the selected consistency groups.


Note

Cannot be undone.
Enable Group
Note

Only enabled if the selected consistency groups are disabled. Enables the
selected consistency groups.
Enabling a disabled consistency group starts replication and causes a full
sweep.
Disable Group

Only enabled if the selected consistency groups are enabled. Disables the
selected consistency groups.
Note

Disabling a consistency group stops all replication, deletes journals, and causes
a full sweep when the group is enabled.
Create
Bookmark

Create a bookmark and define snapshot policies across multiple groups on page
141, ensuring the same point in time can be identified across multiple
consistency groups.

Manage Protection

79

Protection

Before you begin


Before you begin, note that if your host is part of a Windows clustered environment or
running on an AIX operating system, during initialization, the production volume is
replicated in its entirety, including disk information (disk signature, partition table, and
other disk information). This means that the disk information on the copy disk will have
been created by the production host and not by the copy host. Therefore, it is necessary
to invalidate the copy hosts disk information.
To rectify this, best practice is to run a planned failover as soon as possible after
initialization of a consistency group; that is, as soon as possible after the consistency
group transfer status changes from Initializing to Active.
l

To perform this procedure on an AIX host, see the EMC RecoverPoint Deploying with AIX
Hosts Technical Notes for the correct procedure.

To perform this procedure on a Windows host, see EMC RecoverPoint Deploying with
Windows File System Technical Notes.

The RecoverPoint protection policies


RecoverPoint protection is policy-driven. A protection policy, based on the particular
business needs of your company, is uniquely specified for each consistency group, each
copy (and copy journal) and each link. Each policy comprises settings that collectively
govern the way in which replication is carried out. Replication behavior changes
dynamically during system operation in light of the policy, the level of system activity,
and the availability of network resources.
All of the protection policy settings that are available through Unisphere for RecoverPoint
are presented in Group policies on page 80, Link policies on page 82, Copy
policies on page 86, and Journal policies on page 86.
You can save group Link policies on page 82 and Journal policies on page 86 and reuse them using Policy Templates on page 68.
Note

There are various advanced protection policies that cannot be accessed through
Unisphere for RecoverPoint, and can only be configured using the RecoverPoint CLI. See
the EMC RecoverPoint CLI Reference Guide for your RecoverPoint product version for more
information on these policies.

Group policies
The tables in this section describe the policy settings for consistency groups. To display
these settings, select Protection > Manage Protection, select a consistency group in the
left pane, and click the Group Policy tab in the right pane.
Table 17 Group policies

80

Setting

Description

Group Name

The name of the consistency group.

Primary RPA

The RPA that you prefer to replicate the consistency group. When the primary RPA
is not available, the consistency group will switch to another RPA in the RPA
cluster. Whether data will transfer when replication is switched to another RPA

RecoverPoint 4.4 Administrator's Guide

Protection

Table 17 Group policies (continued)

Setting

Description
depends on the value of the transfer_by_non_preffered parameter of the
config_group_policy CLI command.
Note

Best practice is to ensure that groups that replicate in synchronous replication


mode are set to use different RPAs than groups that replicate in asynchronous
replication mode . Mixing between the two may result in low I/O rates for the
synchronous groups. It is also recommended that dynamic sync and purely
synchronous consistency groups reside on different RPAs, whenever possible.

Priority

Default = Normal
Only relevant for remote replication over the WAN or Fibre Channel, when two or
more consistency groups are using the same Primary RPA.
Select the priority assigned to this consistency group. The priority determines the
amount of bandwidth allocated to this consistency group in relation to all other
consistency groups using the same Primary RPA.
Possible values are: Idle, Low, Normal, High, and Critical
In asynchronous replication, groups with a priority of Critical are provided ten
times the priority of normal groups. Groups with a priority of High are provided
three times the priority of normal groups. Groups with a priority of Low are
provided 50% of the priority of normal groups. Groups with a priority of Idle are
provided 1% of the priority of normal groups.

Preferred
Cluster

Note

Only relevant when MetroPoint group is selected.


Default = Follow VPLEX bias rules
Sets the preferred RPA cluster of the MetroPoint group to the active or standby
production, or to follow the VPLEX bias rules.

Distribute
group writes
across
multiple
RPAs

Default=disabled
Only supported if the journal is at least 40 GB.
Note

Both enabling and disabling this setting causes the journal of all copies in the
consistency group to be lost.
Distributed consistency groups across multiple RPAs to significantly increase the
maximum available RPA throughput, allowing for a significantly larger group. For
throughput performance statistics (during synchronous and asynchronous
replication) and feature limitations, see the EMC RecoverPoint Performance Guide.
When enabled, a minimum of one and a maximum of three secondary RPAs can
be selected.
Note

Before changing this setting, ensure all preferred RPAs (both primary and
secondary) are connected by Fibre Channel and can see each other in the SAN .

The RecoverPoint protection policies

81

Protection

Table 17 Group policies (continued)

Setting

Description

External
Application

Default=none
Possible values are None and SRM.

Recovery
Copy

None: Using an external management application (SRM) is disabled.

SRM: To enable support for VMware Site Recovery Manager. This option is
only valid if a RecoverPoint Storage Replication Adapter for VMware Site
Recovery Manager is installed on the vCenter Servers. For more information,
refer to the EMC RecoverPoint Adapter for VMware Site Recovery Manager
Release Notes.

Specifies which copy the external application should fail over to.

External
Only relevant for MetroPoint configuration and if External Application = SRM.
Management SRM supports only one production copy. When RecoverPoint is managed by
external management, the external management application (SRM) must be
Side

connected via a fixed management IP address. If the other cluster becomes


active, SRM will fail. External Management Side determines the RecoverPoint IP
address of which RecoverPoint production copy is used for management by SRM.
This IP address cannot be changed, even if the active production copy changes
over to the other side, and SRM therefore fails.
The External Management Side parameter cannot be changed unless:

Managed by

RecoverPoint is replicating correctly.

In vCenter, neither SRM Test nor a Recovery Plan is running.


After changing the External Management Side parameter, Rescan Scan must
be run in vCenter, otherwise SRM will fail.

When any value other than None is selected, Managed By is enabled and you
have the following options:
l

RecoverPoint: Check this option for planned or unplanned maintenance of


the RecoverPoint system. When activated, external application support is
disabled and user-initiated RecoverPoint capabilities are enabled. When
activated, RecoverPoint user-initiated capabilities, such as image access,
image testing, changing policies and creating bookmarks are available.

External Application: When activated, the specified external application


manages RecoverPoint. All RecoverPoint user-initiated capabilities are
disabled. The user cannot access images, change policies, or change
volumes. Bookmarks cannot be created in the RecoverPoint GUI, but they
can be created using the RecoverPoint CLI bookmark commands.
When Managed by = External Application, Site Recovery Manager manages
the group, and can perform automatic failover and test failover.

Link policies
The tables in this section describe the policy settings for the links between two copies.
To display these settings, select Protection > Manage Protection, select a consistency
group in the left pane, and click the Link Policy tab in the right pane.

82

RecoverPoint 4.4 Administrator's Guide

Protection

Table 18 Link policies

Setting

Description

Copy 1

The 1st copy that serves as an endpoint of the specified link.

Copy 2

The 2nd copy that serves as an endpoint of the specified link.

Asynchronous

Default=enabled
When enabled, RecoverPoint replicates the consistency group data in
asynchronous replication mode.

Snap
Replication

Default=disabled
When enabled, RecoverPoint replicates the consistency group data in
asynchronous, snap-based replication mode.
Possible values:
l

None: Disables snap-based replication mode.

Periodic: A new snap starts after the specified interval has passed since
the previous snap was started. If the time interval has passed, but the
previous snap has not been finished, the next snap will start as soon as
the previous one is finished. Sets the interval between snaps to any
value between 30 and 1440 minutes. When set to Periodic, an interval
should be defined. The default interval is 30 minutes. The RPO will be
twice the specified interval, and cannot be changed.
Note

The recommended snap replication mode for XtremIO replication is


Periodic. For sizing guidelines when using Continuous snap-based
replication mode consult EMC Customer Support.

Synchronous

On Highload: Uses asynchronous replication mode as the default


replication mode and dynamically switches to snap-based replication
mode upon high load. When highload is over, dynamically switches back
to asynchronous replication mode .

Continuous: A new snap starts as soon as the previous snap is


completed. Specify the RPO. RPO is determined by the XtremIO snapshot
pruning policy, which cannot be changed and by the Maximim Number of
Snaps specified. The RPO specified here can shorten the RPO determined
by the other parameters, but cannot lengthen it. Default RPO = 1 minute.

Note

Not relevant in ProtectPoint and XtremIO.


Default=disabled
When enabled, RecoverPoint replicates the consistency group data in
synchronous replication mode.

Dynamic by
latency

Note

Not relevant in ProtectPoint.


Default=disabled

The RecoverPoint protection policies

83

Protection

Table 18 Link policies (continued)

Setting

Description
Only relevant for synchronous replication mode.
When enabled, RecoverPoint uses dynamic sync mode and alternates
between synchronous and asynchronous replication modes, as necessary,
according to latency conditions (the number of milliseconds or microseconds
between the time the data is written to the local RPA and the time that it is
written to the RPA or journal at the remote site).

Dynamic by
throughput

Start async replication above: When the specified limit is reached,


RecoverPoint automatically starts replicating in asynchronous replication
mode.

Resume sync replication below: When the specified limit is reached,


RecoverPoint goes back to replicating in synchronous replication mode.

Note

Not relevant in ProtectPoint.


Default=disabled
Only relevant for synchronous replication mode.
When enabled, RecoverPoint uses dynamic sync mode and alternates
between synchronous and asynchronous replication modes, as necessary,
according to throughput conditions (the total writes that reach the local RPA,
per copy, in kb/s).

RPO

Start async replication above: When the specified limit is reached,


RecoverPoint automatically starts replicating in asynchronous replication
mode.

Resume sync replication below: When the specified limit is reached,


RecoverPoint goes back to replicating in synchronous replication mode.

Default = 25 seconds
This setting defines the required lag of the each link in a consistency group,
and is set manually in MB, GB, writes, seconds, minutes, or hours.
In RecoverPoint, RPO starts being measured when a write made by the
production host reaches the local RPA, and stops being measured when the
write reaches either the target RPA or the target journal (depending on the
value of the transfer_by_non_preferred parameter of the
config_group_policy CLI command).
When the value of the regulate_application parameter of the
config_link_policy CLI command is set to no (default is no), the
specified RPO is not guaranteed. RecoverPoint will try its best to replicate
within the specified RPO without affecting host performance.

Compression

Note

Not relevant in ProtectPoint.


Default = Low

84

RecoverPoint 4.4 Administrator's Guide

Protection

Table 18 Link policies (continued)

Setting

Description
To compress data before transferring it to a remote RPA cluster. Can reduce
transfer time significantly.
Note

Only relevant for asynchronous remote replication. Both the enabling and
disabling of compression causes a short pause in transfer and a short
initialization. Compression decreases transfer time, but increases the source
RPAs CPU utilization.

Enable
Deduplication

High: Yields the highest bandwidth reduction ratio and requires the most
RPA resources.

Medium: Yields an average bandwidth reduction ratio and requires an


average amount of RPA resources.

Low: Yields the lowest bandwidth reduction ratio and requires the least
RPA resources.

None: Compression is disabled and requires no RPA resources.


See the EMC RecoverPoint Release Notes for the supported throughput for
each compression level.

Note

Not relevant in ProtectPoint.


Default = disabled
To eliminate repetitive data before transferring the data to a remote RPA
cluster. Can reduce transfer time significantly.
Note

Only relevant for asynchronous remote replication. Compression must be


enabled before deduplication can be enabled. Both the enabling and
disabling of deduplication causes a short pause in transfer and a short
initialization. Deduplication decreases transfer time, but increases the source
RPAs CPU utilization.

Snapshot
Granularity

Note

Not relevant in ProtectPoint and XtremIO.


Default = fixed (per second)
l

Fixed (per write): To create a snapshot for every write operation, over a
specific (local or remote) link.

Fixed (per second): To create one snapshot per second, over a specific
(local or remote) link.

Dynamic: To have the system determine the snapshot granularity of a


specific (local or remote) link, according to available resources.

The RecoverPoint protection policies

85

Protection

Table 18 Link policies (continued)

Setting

Description
Note

When you use distributed consistency groups, the snapshot granularity


of all links in the consistency group can be no finer than one second.

Copy policies
Describes the policy settings for copies.
To display these settings, select Protection > Manage Protection, expand a consistency
group in the left pane and select one of its copies, and click the Copy Policy tab in the
right pane.
Table 19 Copy policies

Setting

Description

Copy Name

The name of the copy.

Host OS

Default = Other/Mixed
Select the operating system of the host writing to the volumes in the consistency
group. If the hosts are not all running the same operating system, select Others/
Mixed. If one or more of the volumes in the consistency group resides on an ESX
server, set as follows:

Maximum
Number of
Snaps

VMware ESX Windows -when running a guest Windows OS with a VNX splitter

VMware ESX -when running a guest OS other than Windows, with a VNX
splitter

Note

Only relevant in XtremIO


Specify the maximum number of snaps to be saved at the copy (2500).

Clear journal
Note
upon
Only relevant in ProtectPoint
removal

Journal policies
Describes the policy settings for copy journals.
To display these settings, select Protection > Manage Protection, expand a consistency
group in the left pane, and select one of its copies. Then click the Copy Policy tab in the
right pane.

86

RecoverPoint 4.4 Administrator's Guide

Protection

Table 20 Journal policies

Setting

Description

Journal
Compression

Note

Not relevant in ProtectPoint or XtremIO.


Default = none
When enabled, instructs the target RPA of the consistency group to compress
the snapshots in the journal so that more images (that is, a longer protection
window) can be saved in the same journal capacity, saving storage cost.
Enabling this option is encouraged if you have cost considerations, a low
incoming write-rate, or limited bandwidth. It is also recommended to
compress the journal when forcing synchronous replication.
Compression impacts the CPU resources of the target RPA of the consistency
group, and can impact that RPAs ability to sustain its write-load. If the target
RPA of the consistency group for which you wish to enable this option is also
transferring the data of other consistency groups across the WAN, note that
enabling this setting will affect the RPAs transfer rate. See the EMC
RecoverPoint Release Notes for throughput limitations.
Note

Enabling journal compression while a consistency group is enabled will result


in the loss of all snapshots in the journal. To change the value of journal
compression, the consistency group must be in one of the following states:
disabled, direct image access, or distributing.

Maximum
Journal Lag

Note

Not relevant in ProtectPoint or XtremIO.


Default = unlimited
Defines the data access aspects of RTO for each copy.
Note

RTO includes the time for trying to fix the problem without a recovery, the
recovery itself (RecoverPoint's role in an organization's RTO, which is
controlled by this setting), tests, and the communication to the users.
Decision time for users is not included.
When data is received by the RPA faster than it can be distributed to storage
volumes, it accumulates in the journal. The Maximum Journal Lag is the
maximum amount of snapshot data (in bytes, KB, MB, or GB) that is
permissible to hold in the copy journal before distribution to the copy
storage. In other words, the amount of data that would have to be distributed
to the copy storage before failover to the latest image could take place. In
terms of RTO, this is the maximum time that would be required in order to
bring the copy up-to-date with production.
When the Maximum Journal Lag value is reached, the system switches to
three-phase distribution mode, and no longer retains rollback information. As
soon as the lag is within the allowed limits, rollback data is retained again.

The RecoverPoint protection policies

87

Protection

Table 20 Journal policies (continued)

Setting

Description

Required
Protection
Window

Default = disabled
Note

For XtremIO arrays, the Required Protection Window is determined by the


XtremIO snapshot pruning policy, which cannot be changed, and the

Maximum Number of Snapshots


The protection window indicates how far in time the copy image can be rolled
back. Select the checkbox to define a required protection window. Specify the
length of the required protection window.
When the required protection window is defined, the status of the current
protection window and the predicted protection window are displayed in the
Journal tab of the copy and can be displayed when Monitoring copy
journals on page 109. In addition, the system will raise an event in any of the
following cases:

Enable
RecoverPoint
snapshot
consolidation

Current Protection Window becomes insufficient.

Current Protection Window becomes sufficient.

Predicted Protection Window becomes insufficient.

Predicted Protection Window was insufficient and becomes sufficient.

Note

Not relevant in ProtectPoint or XtremIO.


Default = disabled
When selected, enables automatic snapshot consolidation to allow for the
storage of a longer history in the copy journal.
The system will issue an event in any of the following cases:
l

The specified group is not distributing.

The specified snapshots have not reached the copy storage.

The consolidation times are not far apart enough for significant changes
to have occurred. There must be a least 1 GB of space between the
snapshots being consolidated.

Another consolidation is in progress on the same RPA.


Note

Automatic snapshot consolidation cannot be enabled for a group that is


part of a group set. When enabled, the Predicted Protection Window is
not calculated. The Minimum journal size with snapshot consolidation on
page 298 is 30 GB.

Do not
Note
consolidate any
Not relevant in ProtectPoint or XtremIO.
snapshots for
at least
Default = 2 days

88

RecoverPoint 4.4 Administrator's Guide

Protection

Table 20 Journal policies (continued)

Setting

Description
Minimum = 12 hours.The period during which snapshot data is not to be
consolidated. The periods start time is always today, and the periods end
time is n hours / days / weeks / monthsago.
The following conditions apply:

Consolidate
snapshots that
are older than x
to one snapshot
per day for y
days

Must be a minimum of12 hrs.

If no daily or weekly consolidations are specified, the remaining


snapshots are consolidated monthly.

Note

Not relevant in ProtectPoint or XtremIO.


Default = 5 days
Snapshots are consolidated every ~24 hours.
Select the Indefinitely checkbox to consolidate all subsequent snapshots in
~24 hour intervals.
The following conditions apply:
l

If the Indefinitely checkbox is not selected, and no weekly consolidations


are specified, the remaining snapshots are consolidated monthly.

If the Indefinitely checkbox is selected, weekly and monthly


consolidations are disabled, and the remaining snapshots are
consolidated daily.
Note

The snapshot consolidation process will fail if at least 24 hours worth of


data do not exist in the journal in addition to the unconsolidated
duration, or if there is not a minimum of 1 GB of data between the
starting and closing points of the consolidation.

Consolidate
snapshots that
are older than x
to one snapshot
per week for y
weeks

Note

Not relevant in ProtectPoint or XtremIO.


Default = 4 weeks
Snapshots are consolidated every ~168 hours.
Select the Indefinitely checkbox to consolidate all subsequent snapshots in
~168 hour intervals.
The following conditions apply:
l

If the Indefinitely checkbox is not selected, the remaining snapshots are


consolidated monthly.

If the Indefinitely checkbox is selected, monthly consolidations are


disabled, and the remaining snapshots are consolidated weekly.

The RecoverPoint protection policies

89

Protection

Table 20 Journal policies (continued)

Setting

Description
Note

The first weekly consolidation will fail unless there are at least seven
daily snapshots in the journal, in addition to the required number of
weekly snapshots and the specified unconsolidated duration. The first
monthly consolidation will fail unless there are at least four weekly
snapshots, or 28 daily snapshots in the journal in addition to the
required number of monthly snapshots.

Monitoring and managing protection


After a consistency group has been created, use the Manage Protection screen to monitor
and modify the goup's settings and policies.

Use Manage Protection for:

90

Monitoring and managing groups on page 91

Monitoring and managing copies on page 105

Monitoring group performance on page 93

Monitoring copy journals on page 109

Modifying consistency groups on page 114

Modifying copies on page 129

Modifying links on page 132

Modifying replication sets on page 132

Modifying journals on page 136

Modifying snapshots on page 138

RecoverPoint 4.4 Administrator's Guide

Protection

Monitoring and managing groups


Use the following procedure to monitor the status of replication for a specific group.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a group.
3. In the right pane, select the Status tab.
Results

The Status tab displays a graphical representation of the consistency group and copy
statuses. The following information is displayed:
Table 21 Status tab

Label

Displays

<Group
Name>
Running on
<RPA>

The group name and the RPA which is handling the group data, both of which can
be modified through the group policies on page 80.

<Copy Name>
at <Cluster
Name>

The relevant copy and RPA cluster names. The copy name can be modified
through the copy policies on page 86.

Replication
modes and
states

A visual representation of the current replication modes and states.


l

A dashed green line means that the system is replicating asynchronously.

A dashed green line on top of a solid line means that the system is
replicating synchronously.

A greyed out line means that replication has stopped.

Monitoring and managing groups

91

Protection

Table 21 Status tab (continued)

Label

Displays
Note

The lines move in the direction of replication.


The transfer state on page 43 is displayed in the center of each link between
the production and the copy. When replicating synchronously, the word
synchronized is displayed. The Transfer state displayed here correlates to
what is displayed in the Consistency Group Transfer Status on page 43
widget of the system Dashboard.

Role

The current role of the copy with regards to failover and regulation.
l

Before failover: Production, Local Copy and Remote Copy

After failover: Target at Production, Local Source and Remote Source

During regulation: Regulated

Storage

The image access modes.

Image

The image currently being distributed to storage in the distribution phase.

When a group is selected in the left pane, you can click the buttons at the bottom of the
Status tab to perform the following actions:
Table 22 Status tab options when a group is selected

Button

Description

Apply
Bookmark

Create a bookmark and define snapshot policies for one group on page 140 by
displaying the Apply Bookmark dialog box and providing a bookmark name for
the current snapshot. When the snapshot is closed, it will be listed with the
bookmark name that you give it.

Add Copy

Displays the Add a copy on page 116 wizard, enabling you to add a copy to the
selected consistency group.

Test Copy

Displays the Test a Copy on page 147 wizard, enabling you to select and test a
copy of the selected consistency group.

Fail Over

Note

Not relevant in ProtectPoint


Displays the Test a Copy and Fail Over on page 165 wizard, enabling you to
select and test a copy of the selected consistency group, as well as fail over to it
permanently or temporarily.

92

Recover
Production

Displays the Test a Copy and Recover Production on page 160 wizard, enabling
you to select and test a copy of the selected consistency group, and recover
production from it.

Start
Transfer

Only displayed if transfer is paused for the selected consistency group. Causes
the transfer of writes from the production host to all copies of the selected
consistency group to start.

RecoverPoint 4.4 Administrator's Guide

Protection

Table 22 Status tab options when a group is selected (continued)

Button

Description

Pause
Transfer

Only displayed if transfer is active for the selected consistency group. Causes the
transfer of writes from the production host to all copies of the selected
consistency group to pause.
Note

Best practice is not to use this command. Use Group policies on page 80 to set
policies for use of bandwidth. Pause transfer may be used when WAN bandwidth
is very limited and you wish to give the largest bandwidth possible to another
consistency group. In that case, you may temporarily pause transfer for lowerpriority consistency groups.

Disable
Group

Only displayed if the group is enabled for the selected consistency group.
Disables the selected consistency group.
Note

Disabling a consistency group stops all replication, deletes journals, and causes
a full sweep when the group is enabled.

Enable
Group

Only displayed if the group is disabled for the selected consistency group.
Enables the selected consistency group.
Note

Enabling a disabled consistency group starts replication and a full sweep


commences.

Remove
Group

Removes the selected consistency group from the system.


Note

Cannot be undone.

Monitoring group performance


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a group.
3. In the right pane, select the Statistics tab.

Monitoring and managing groups

93

Protection

The following traffic statistics are displayed in the Traffic sub-tab:


l

The Total Traffic widget displays the total traffic that the consistency group data
generates on the RPA transferring the group data at the production and copy sites,
as well as between RPA clusters in your RecoverPoint system (in Mb/sec). The total
traffic includes the Initialization Traffic and the Application Traffic.

The Application Traffic widget displays the traffic that the consistency group
generates on the RPA transferring the group data at the production and copy sites,
as well as between RPA clusters in your RecoverPoint system (in Mb/sec). The
application traffic includes Incoming Writes as well as RecoverPoint replication
process traffic.

The Initialization Traffic widget displays the traffic that the consistency group
generates between copies and RPA clusters in your RecoverPoint system, during
consistency group initialization (in Mb/sec). Initialization traffic can be high, as
old writes are transferred between source and target copies to synchronize
between the production volumes and the copy volumes.

The Incoming Writes widget displays the writes to the production storage that the
consistency group generates at copies in your RecoverPoint system (in writes per
second), where each write is one I/O to the production storage.

The following statistics are displayed in the Replication Performance sub-tab:

94

RecoverPoint 4.4 Administrator's Guide

Protection

The Bandwidth Reduction widget displays the ratio by which RecoverPoint was
able to compress the consistency group data that is transferred between RPA
clusters in your RecoverPoint system. The bandwidth reduction ratio includes both
data compression and deduplication.
Note

Compression and deduplication are only relevant for asynchronous replication to a


remote RPA cluster.
The gains realized will vary depending on the compressibility of the replicated
data. Typically, enterprises can save 50% 90% of their bandwidth by enabling
the compression setting.
NOTICE

While enabling compression saves bandwidth, it also increases RPA CPU


utilization, which lowers RPA throughput. See the Performance section of the
EMC RecoverPoint Release Notes for the impact of each compression setting on RPA
throughput. Also, both the enabling and disabling of compression causes a short
pause in transfer and a short initialization.
l

The Time Lag widget displays the lag of the selected consistency group (in
seconds), when transferring data between all local and remote RPA clusters in your
RecoverPoint system.

The Write Lag widget displays the lag of the selected consistency group (in writes),
when transferring data between all local and remote RPA clusters in your
RecoverPoint system. The write lag includes any writes generated by RecoverPoint
replication processes as well as internal RecoverPoint overhead.

The Data Lag widget displays the lag of the selected consistency group (in MB),
when transferring data between all local and remote RPA clusters in your
RecoverPoint system.

Monitoring and managing groups

95

Protection

Enabling deduplication
Before you begin
l

Deduplication is only relevant for asynchronous remote replication.

Both the enabling and disabling of deduplication causes a short pause in transfer
and a short initialization.

Data compression must be enabled before deduplication can be enabled.

Deduplication is only supported in RecoverPoint environments in which all (local and


remote) RPAs are:
n

Physical RPAs, running on a GEN4 or GEN5 hardware platform with at least 8GB of
RAM.

Virtual RPAs, with at least 4GB RAM.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Link Policy tab.
a. In the Select link to set policy diagram, select the relevant link.
b. In the Bandwidth Reduction section, if Compression is set to None, enable
compression by changing this value to Low, Medium, or High.
c. Check the Enable Deduplication checkbox.
4. Click the Apply button.
NOTICE

There is a short pause in transfer, followed by a short initialization.


After you finish
To display the effects of deduplication on your bandwidth:
l

Display short-term deduplication statistics on page 96the system displays the


current (sampled over the past seconds) and average (sampled over the past
minutes) performance statistics.

Display long-term deduplication statistics on page 97the user defines the period
in time for which to generate performance statistics.

Note

A bandwidth reduction or deduplication ratio value that is not significantly higher than 1
would indicate that the deduplication (and/or compression) process is not very effective.
If so, consider disabling these policies (as they do consume RPA CPU resources).

Display short-term deduplication statistics


To display the short-term effects of deduplication on your bandwidth:
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
96

RecoverPoint 4.4 Administrator's Guide

Protection

3. In the right pane; click the Statistics tab and click on the Replication Performance tab.
Note the performance statistics in the Bandwidth Reduction pane. The Bandwidth >
Reduction pane displays the current bandwidth reduction ratio, including
compression and deduplication.
1. To display the short-term effects of deduplication on your bandwidth through the
RecoverPoint CLI:
4. Run the get_group_statistics command.
5. Note the Current bandwidth reduction ratio and Average bandwidth reduction ratio
values. These values are the current (seconds) and average (minutes) bandwidth
reduction ratios, including compression and deduplication.
6. Note the Current deduplication ratio, and Average deduplication ratio values. These
values are the current (seconds) and average (minutes) bandwidth reduction ratios, of
only the deduplication process.

Display long-term deduplication statistics


The long-term effects of deduplication on your bandwidth can only be monitored through
the RecoverPoint CLI.
To display the long-term effects of deduplication on your bandwidth:
l

Detect bottlenecks on page 239

Export consolidated statistics on page 100

Both bottleneck detection and exporting consolidated statistics allows you to define the
period in time for which to generate long-term bandwidth reduction and deduplication
ratios.

Configuring lag optimization


Describes how to set the lag optimization of a specific consistency group.
Before you begin
l

You should:
n

Document the RPO of your consistency groups (for example, according to


regulation rules).

Evaluate the importance and value of your consistency groups data in relation to
each other, and verify that the default values of the priority on page 81 setting (as
described in the following table) reflect your business objectives. If they do not,
contact EMC Customer Support and ask them to change the values.
Table 23 Priority setting default values

Setting Value
Idle

0.01 x Normal

Low

0.5 x Normal

Normal

Normal

High

3 x Normal

Critical

10 x Normal

Monitoring and managing groups

97

Protection

Note

Changing the default values is optional, as they provide a practical configuration.


It is recommended to accept the default settings unless there is a specific
business need to set other policies.
l

Lag optimization is performed per RPA.

The lag is optimized only if the bandwidth is the cause of the lag. It is not relevant, for
example, if the lag is caused by the distribution process at the remote site (which can
happen when replicating over Fibre Channel).

The lag optimizer is only relevant for groups replicating asynchronously, that are not
experiencing a high load or an initialization. Groups replicating synchronously are
always prioritized over groups replicating asynchronously.

Procedure
1. Select Protection > Manage Protection
2. In the left pane, select a consistency group.
3. In the right pane, select the Group Policy tab
4. Specify the group Policy setting.
5. In the left pane, expand the group and select a copy.
6. Specify the copy RPO policy, according to your company's business objectives.

Distributing group writes across RPAs


Before you begin
NOTICE

When modifying a groups topology, journal loss will occur.


l

The snapshot granularity of all links in the consistency group can be set in the link
policies to no smaller than one second.

Journal loss will occur when modifying a groups topology (setting a non-distributed
group as distributed, or setting a distributed group as non-distributed).

When configuring journals for a distributed consistency group, keep the following in
mind:

All copies of distributed consistency groups must have journals that are at least
20 GB in size.

The recommended journal size for distributed groups with snapshot consolidation
enabled is at least 40GB.

If the capacity of an existing copy journal is less than the minimum journal size
required for distributed consistency groups (see the EMC RecoverPoint Release
Notes for this limit) the consistency group will need to be disabled and then
enabled again after you add journal volumes or change the size of a journal LUN
on storage
Enabling a disabled group causes a full sweep.

Distributed consistency groups are only supported if there is a Fibre Channel


connection between all RPAs in each RPA cluster in the RecoverPoint system,
therefore:
n

98

In Fibre Channel environments, make sure all of the RPAs at each RPA cluster are
connected to the SAN through a Fibre Channel switch, and zoned together so that
they all see each other in the SAN.

RecoverPoint 4.4 Administrator's Guide

Protection

In iSCSI environments, make sure all RPAs are physically connected to each other
through their HBA Fibre Channel ports.
Note

In iSCSI configurations, there can be a maximum of two RPAs per RPA cluster
because two of the four existing Fibre Channel ports in the RPAs HBA are already
connected directly to the storage. If more than 2 RPAs per RPA cluster are
required, connect all of the RPAs in the RPA cluster through Fibre Channel
switches (two should be used for high availability) and zone them together.
If any of the primary or secondary RPAs associated with a consistency group becomes
unavailable, there will be a brief pause in transfer on all of the groups primary and
secondary RPAs, and all of the group segments will undergo a short initialization.
l
Under certain circumstances (for example, if one of the primary or secondary RPAs
becomes unavailable) two consistency group segments could be handled by the
same RPA.
l
In general, distributed consistency groups offer better performance than nondistributed (regular) consistency groups, as distributed groups run on a minimum of
two RPAs (one primary RPA and one secondary RPA). There is only a small
improvement in performance when a group is run on three RPAs. However, there is a
steep improvement in performance when a group is run on four RPAs.
l
The following are example journal configurations for distributed consistency groups:
n
One 40 GB volume + optional smaller volumes
n
Two 20 GB volumes + optional smaller volumes
n
Three 16 GB volumes + optional smaller volumes
n
Four 10 GB volumes + optional smaller volumes
Procedure
1. Review the relevant limitations and performance statistics of distributed groups in the
EMC RecoverPoint Release Notes
2. Select Protection > Manage Protection.
3. In the left pane, select a consistency group.
4. In the right pane, click the Group Policy tab
5. Check or clear the Distribute group checkbox. The Secondary RPA checkboxes are
enabled or disabled accordingly.
6. If you are enabling this feature, select one to three secondary RPAs, by checking the
relevant checkboxes in the Secondary RPA section.
Results
The group writes are distributed across the selected RPAs.
l

After you finish


To monitor each group segments performance separately, and in relation to the group as
a whole, detect botttlenecks on page 239.
You can tell whether or not a group is distributed by navigating to Protection > Manage
Protection, clicking on a consistency group name in the left pane and looking at the top
of the consistency groups Status tab. If the group is distributed, the text (primary)
followed by a comma-separated list of numbers indicating the designated secondary
RPAs is displayed.

Monitoring and managing groups

99

Protection

Figure 15 Identifying a distributed group

If the group is not distributed, only one RPA is specified in this area. You can also tell
which of all of your groups are distributed by navigating to Protection > Manage
Protection, clicking on Consistency Groups in the left pane and looking in the Active RPA
column of the consistency group table. Distributed consistency groups are displayed with
the text (primary) followed by a comma-separated list of numbers indicating the
designated secondary RPAs.

Figure 16 Identifying a distributed group

For groups that are not distributed, only one RPA is specified in this column.

Exporting consolidated statistics


The export_consolidated_statistics CLI command provides data series for a selection of
important RecoverPoint operational statistics. It enables advanced users, customer
support representatives, and implementation specialists to analyze system traffic and
workload trends, to identify correlation between spikes in two or more settings, and to
discover the root causes of high loads and other significant system behaviors.
With the export_consolidated_statistics command, you specify the granularity at which to
collect statistics (minute, hour, and/or day) and, for each granularity, the time frame over
which to collect the statistics. The resulting CSV file organizes the output for each entry
according to the standard bottleneck detection settings.
Procedure
1. Create a secure SSH connection to the RecoverPoint CLI and log in one of the Predefined users on page 226 with admin privileges.
2. Run the export_consolidated_statistics command.
3. When prompted, specify the required time frame and report granularity.
The consolidation statistics are collected, placed in multiple comma-separated-value
(*.csv) files, and archived into one compressed file with the extension tar.gz.
4. Open an Internet browser window and connect to an RPA with the address:
http://<RPA IP address>/info
Note

You can connect to the RPA through either an HTTP or an HTTPS connection.
5. When prompted, log in to the RPA as one of the Pre-defined users on page 226 with
webdownload privileges.
6. In the browser, navigate to the following directory:
../info/long_term_stats
100

RecoverPoint 4.4 Administrator's Guide

Protection

7. Download and unzip the long_term_stats.tar.gz file into a folder on your local
machine.
Results
The following table lists the statistics that are output by the
export_consolidated_statistics command. All statistics are valid for the sampled time
interval, that is, the difference between the time from and time to values on that row.
Table 24 Consolidated statistics output

Statistic

Unit of
measure

Description

Incoming writes rate for


link(1)

MB/sec

Throughput of writes that arrived at a link on the


production side.

Incoming IOs rate for


link

IOs/sec

Number of writes that arrived at a link on the production


side. Average IO size can be computed as (Incoming
writes rate for link) / (Incoming IOs rate for link).

Non - initialization
output rate for link

MB/sec

Rate at which normal write traffic was transferred for the


link.

Initialization output rate MB/sec


for link

Rate at which data was actually transferred for the link


for purpose of synchronization.

Data synchronization
rate for link

MB/sec

Rate at which data was checked for possible transfer for


the link for purpose of synchronization. In first-time
initialization, this is the rate at which the data from the
replication set volumes is transferred. In other cases,
comparison of signatures increases the rate.

Compression CPU
utilization

Depending on the relevance (link, site or RPA), either:


l

Link: Portion of processor time used to compress the


incoming data over a link. This statistic is relevant
only for remote links.

Site: Portion of processor time used to compress the


incoming data over all links for all RPAs at a site.

RPA: Portion of processor time used to compress the


incoming data over all links on the RPA.

Percentage time in
transfer

% of time

Portion of time link was in active transfer state.

Percentage time of
initialization

% of time

Portion of time link was in initializing transfer state.

RPO - lag in time


sec
between replicas during
transfer after init

Actual recovery point objective(2) for link, as measured


in seconds. It is measured only when the transfer state
for the link was active.

RPO - lag in data


MB
between replicas during
transfer after init

Actual recovery point objective for link, as measured in


megabytes of data. It is measured only when the transfer
state for the link was active.

RPO - lag in IOs


IOs
between replicas during
transfer after init

Actual recovery point objective for link, as measured in


number of IOs. It is measured only when the transfer
state for the link was active.

Monitoring and managing groups

101

Protection

Table 24 Consolidated statistics output (continued)

102

Statistic

Unit of
measure

Description

Group-Link utilization

Aggregate measure of portion of RPA capacity used by


the link, based on number of IOs, writes throughput, and
CPU utilization.

Distributor receiver
regulation duration

% of time

Depending on the relevance (link, site or RPA), either:


l

Link: Portion of time that the distribution process for


a copy was forced to regulate the incoming data rate.
A high value indicates a slow storage journal
volumes relative to the rate of the incoming data.

RPA: Portion of time that the distribution process


across all copies served by the RPA was forced to
regulate the incoming data rate. A high value
indicates slow storage journal volumes relative to
the rate of the incoming data.

Site: Portion of time that the distribution process


across all copies at a site was forced to regulate the
incoming data rate. A high value indicates slow
storage journal volumes relative to the rate of the
incoming data.

Distributor phase 1
thread load

% of time

Portion of time used to receive data and write it to the


journal volumes for the link. The value is dependent on
the performance of the journal volumes.

Distributor phase 1
effective speed

MB/sec

Depending on the relevance (link, site or RPA), either:


l

Link: Rate at which data was received and written to


the journal volumes for the link. The value is
dependent on the performance of the journal
volumes.

RPA: Rate at which data was received and written to


the journal volumes for all copies served by the RPA.
The value is dependent on the performance of the
journal volumes.

Site: Rate at which data was received and written to


the journal volumes for all copies at the site. The
value is dependent on the performance of the
journal volumes.

Distributor phase 2
thread load

% of time

Portion of time used to read data from the journal


volumes and write to the copy volumes, using either
normal or fast-forward distribution mode. The value is
dependent on the performance of the journal and copy
volumes.

Distributor phase 2
effective speed

MB/sec

Depending on the relevance (link, site or RPA), either:

RecoverPoint 4.4 Administrator's Guide

Link: Rate at which data was read from the journal


volumes and written to the copy volumes, using
either normal or fast-forward distribution mode. The

Protection

Table 24 Consolidated statistics output (continued)

Statistic

Unit of
measure

Description
value is dependent on the performance of the
journal and copy volumes.
l

RPA: Rate at which data was read from the journal


volumes and written to the copy volumes for all
copies served by the RPA, using either normal or
fast-forward distribution mode. The value is
dependent on the performance of the journal and
copy volumes.

Site: Rate at which data was read from the journal


volumes and written to the copy volumes for all
copies at a site, using either normal or fast-forward
distribution mode. The value is dependent on the
performance of the journal and copy volumes.

Fast forward
distribution duration

% of time

Portion of time at copy when distribution was in fastforward mode.

WAN throughput from


box (over IP)
or
Cross-site throughput
from box (over FC)

Mb/sec

Outgoing throughput from RPA to remote site.

Total incoming writes


rate for box

MB/sec

Throughput of writes that arrived at an RPA on the


production side. For concurrent local and remote
replication, this is the sum of throughputs for all local
and remote links on this RPA (which is double the actual
incoming writes rate).

Incoming IOs rate for


box

IOs/sec

Number of writes that arrived at an RPA on the


production side. For concurrent local and remote
replication, this is the sum of incoming IOs for all local
and remote links (which is double the actual number of
incoming writes). Average IO size can be computed as
(Total incoming writes rate for box) / (Incoming IOs rate
for box).

Non - initialization
output rate for box
(average over all
period)

MB/sec

Rate at which normal write traffic was transferred by the


RPA.

Initialization output rate MB/sec


for box (average over all
period)

Rate at which data was actually transferred by the RPA


for purpose of synchronization.

Data synchronization
rate for box (average
over all period)

Rate at which data was checked for possible transfer by


the RPA for purpose of synchronization. In first-time
initialization, this is the rate at which the data from the
replication set volumes is transferred. In other cases,
comparison of signatures increases the rate.

MB/sec

Monitoring and managing groups

103

Protection

Table 24 Consolidated statistics output (continued)

104

Statistic

Unit of
measure

Description

Replication process
CPU utilization per
box

Portion of processor time used for the replication


process on the RPA, including compression and other
replication activities.

SAN target thread load

% of time

Portion of time used by RPA for initial processing of


writes, before assigning them to the relevant links.

Box utilization

Aggregate measure of portion of RPA capacity used,


based on both IO load and processor utilization.

WAN throughput from


site (over IP)
or
Cross-site throughput
from site (over FC)

Mb/sec

Combined outgoing throughput from a site (to remote


site).

Total incoming writes


rate for site

MB/sec

Throughput of writes that arrived on the production side.


For concurrent local and remote replication, this is the
sum of throughputs for all local and remote links at the
site (which is double the actual incoming writes rate).

Incoming IOs rate for


site

IOs/sec

Number of writes that arrived on the production side. For


concurrent local and remote replication, this is the sum
of incoming IOs for all local and remote links at the site
(which is double the actual number of incoming writes).
Average IO size can be computed as (Total incoming
writes rate for box) / (Incoming IOs rate for box).

Non - initialization
output rate for site
(average over all
period)

MB/sec

Rate at which normal write traffic was transferred by all


RPAs at the site.

Initialization output rate MB/sec


for site (average over all
period)

Rate at which data was actually transferred by all RPAs at


the site for purpose of synchronization.

Data synchronization
rate for site (average
over all period)

MB/sec

Rate at which data was checked for possible transfer by


all RPAs at the site for purpose of synchronization. In
first-time initialization, this is the rate at which the data
from the replication set volumes is transferred. In other
cases, comparison of signatures increases the rate.

Line latency between


sites

msec

Time it takes for data to be transferred from a site to the


other site.

Packet loss

% of
packets

Measure of the reliability of the line between a site and


the other site, as measured in portion of packets that
must be resent. Latency and packet loss impact the
effective throughput on a line between two sites.

RecoverPoint 4.4 Administrator's Guide

Protection

Table 24 Consolidated statistics output


(1) In an RPA, a link is a logical channel from production to copy, either local or remote, for a
given consistency group.
(2) An actual recovery point objective is the time/data/writes that were waiting to be
transferred over the link to the copy, averaged over the sampled time interval.

Monitoring and managing copies


Use the following procedure to monitor the status of replication for a specific copy.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a group, and select a copy.
3. In the right pane, select the Status tab.
Results

The Status tab displays a graphical representation of the copy statuses. The following
information is displayed:
Table 25 Status tab

Label

Displays

<Group
Name>
Running on
<RPA>

The group name and the RPA which is handling the group data, both of which can
be modified through the group policies on page 80.

<Copy Name>
at <Cluster
Name>

The relevant copy and RPA cluster names. The copy name can be modified
through the copy policies on page 86.

Monitoring and managing copies

105

Protection

Table 25 Status tab (continued)

Label

Displays

Replication
modes and
states

A visual representation of the current replication modes and states.


l

A dashed green line means that the system is replicating asynchronously.

A dashed green line on top of a solid line means that the system is
replicating synchronously.

A greyed out line means that replication has stopped.


Note

The lines move in the direction of replication.


The transfer state on page 43 is displayed in the center of each link between
the production and the copy. When replicating synchronously, the word
synchronized is displayed. The Transfer state displayed here correlates to
what is displayed in the Consistency Group Transfer Status on page 43
widget of the system Dashboard.

Role

The current role of the copy with regards to failover and regulation.
l

Before failover: Production, Local Copy and Remote Copy

After failover: Target at Production, Local Source and Remote Source

During regulation: Regulated

Storage

The image access mode.

Image

The image currently being distributed to storage in the distribution phase.

When a copy is selected in the left pane, you can click the buttons at the bottom of the
Status tab to perform the following actions:
Table 26 Status tab options when a group is selected

Button

Description

Test Copy

Displays the Test a Copy on page 147 wizard, enabling you to test the selected
copy of the selected consistency group.

Fail Over

Note

Not relevant in ProtectPoint


Displays the Test a Copy and Fail Over on page 165 wizard, enabling you to test
and fail over to the selected copy permanently or temporarily.

106

Recover
Production

Displays the Test a Copy and Recover Production on page 160 wizard, enabling
you to test and recover production from the selected copy.

Start
Transfer

Only displayed if transfer is paused for the selected copy.


Causes the transfer of writes from the production host to the selected copy to
start.

Pause
Transfer

Only displayed if transfer is active for the selected copy.


Causes the transfer of writes from the production host to the selected copy to
pause.

RecoverPoint 4.4 Administrator's Guide

Protection

Table 26 Status tab options when a group is selected (continued)

Button

Description
Note

Best practice is not to use this command. Use Group policies on page 80 to set
policies for use of bandwidth. Pause transfer may be used when WAN bandwidth
is very limited and you wish to give the largest bandwidth possible to another
consistency group. In that case, you may temporarily pause transfer for lowerpriority consistency groups.

Disable Copy Only displayed if the selected copy is enabled. Disables the selected copy.
Note

Disabling a copy stops all replication, deletes journals, and causes a volume
sweep when the copy is re-enabled.

Enable Copy

Only displayed if the selected copy is disabled. Enables the selected copy.
Note

Enabling a disabled copy starts replication and a volume sweep commences.

Remove
Copy

Removes the copy from the consistency group.


Note

You cannot remove the production copy.

Controlling RTO
Describes how to set and monitor the RPO at a specific copy.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select a copy.
3. In the right pane, click the Copy Policy tab.
4. Control the part of RTO dedicated to data access using the Maximum journal lag
setting in the Journal policies on page 86.
Results
When the system approaches the value of Maximum journal lag, it moves to three-phase
distribution mode.
After you finish
To monitor the RTO:
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Journal tab.
The value displayed in the Journal Lag field indicates the RTO in terms of data size.
Alternatively, run the get_group_statistics CLI command to display the journal
lag.
Monitoring and managing copies

107

Protection

Controlling RPO
Describes how to set and monitor the RTO at a specific copy.
Before you begin
Only relevant in asynchronous replication mode
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Link Policy tab.
a. In the Select link to set policy diagram, select the relevant link.
b. In the Protection Settings, set the RPO policy and click the Apply button.
4. Control the RPO for each consistency group through the RPO setting in the group Link
policies on page 82.

In RecoverPoint, RPO can be expressed in terms of time, quantity of data, ornumber of


writes. To guarantee the RPO, host applications can be throttled upon approaching the
defined RPO setting, using application regulation. To control application regulation,
run the regulate_application parameter of the config_link_policy CLI
command. In synchronous replication mode, the regulate_application
108

RecoverPoint 4.4 Administrator's Guide

Protection

parameter of the config_link_policy CLI command is automatically enabled,


and cannot be modified.
After you finish
To monitor the RPO:
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Statistics tab and then, click the Replication Performance
sub-tab to monitor the consistency group's RPO in terms of time, writes, or quantity of
data.
Note

You can also select Dashboard > System Limits, and then click on the Groups tab to
monitor the RPO of all consistency groups in RecoverPoint simultaneously.
In the RecoverPoint CLI, run the get_group_statistics command, and identify the
Lag output in the Link stats area.

Monitoring copy journals


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a group and select a copy.
3. In the right pane, select the Journal tab.

Monitoring and managing copies

109

Protection

Results

The following table describes the information displayed in the Journal tab of each copy.
Table 27 Copy journal tab

Info

Description

Current

Snapshot currently being distributed to the journal.

Storage

Storage access status. The current condition of a volume at the copy storage can
be:

Journal Lag

Compression
Ratio

No access, during normal replication.

Direct Access, during direct image access.

Logged Access, during logged access.

Amount of data (represented by snapshots) in the copy journal that has not yet
been distributed to the copy storage. The maximum journal lag setting defines
the current recovery point objective. In the event of a disaster, this is the
maximum amount of data loss that may be incurred.
Note

This feature is relevant for both synchronous and asynchronous replication.


Default = none
When enabled, instructs the target RPA of the consistency group to compress the
snapshots in the journal so that more images (that is, a longer protection
window) can be saved with the same journal capacity (saving storage cost).
Enabling this option is encouraged if you have cost considerations, a low
incoming write-rate, and/or limited bandwidth.
Take note of the following:

110

RecoverPoint 4.4 Administrator's Guide

Compression is not available in RecoverPoint/SE.

Compression is not relevant for the production journal (since the production
journal does not contain snapshots).

Compression impacts the CPU resources of the target RPA of the consistency
group, and can impact that RPAs ability to sustain its write-load. If the target

Protection

Table 27 Copy journal tab (continued)

Info

Description
RPA of the consistency group for which you want to enable this option is also
transferring the data of other consistency groups across the WAN, note that
enabling this setting will affect the RPAs transfer rate. See the EMC
RecoverPoint Release Notes for throughput limitations.

Required
Protection
Window

Default = disabled
Indicates how far in time the copy image can be rolled back.

Current
Protection
Window

Indicates how far the copy journal can be rolled back. If the Required Protection
Window is defined, the Current Protection Window will be in one of the following
statuses (indicated in parentheses after the Current Protection Window):

Predicted
Protection
Window

Sufficient: Image can be rolled back far enough to meet the Required
Protection Window

Insufficient: Image cannot be rolled back far enough to meet the Required
Protection Window

Extending: Replication has not been running long enough to be roll backed
as far as the Required Protection Window.
If the Required Protection Window is not defined, the status will be N/A.

Systems prediction of the eventual size of the protection window. Note that
there is no guarantee on how long it will take to reach the predicted protection
window, and no guarantee that it will ever be reached (conditions may change
before it is reached).
If the Required Protection Window is defined, the Predicted Protection Window is
in one of the following statuses:
l

Sufficient: Predicted Protection Window is large enough to meet the


Required Protection Window

Insufficient: Predicted Protection Window is not large enough to meet the


Required Protection Window
If the Required Protection Window is not defined or replication has not been
running long enough to make predictions, no status will be indicated. It can
take 24 hours or longer of journal entries before the system finishes
calculating the predicted protection window.
Note

When snapshot consolidation is enabled, the Predicted Protection Window


value is displayed as N/A.
Space Saved
by
Consolidation

Amount of space saved by snapshot consolidation. This value is updated only


after a consolidation process completes.

If a snapshot consolidation job is in process, the following information is also displayed:

Monitoring and managing copies

111

Protection

Table 28 Journal Tab: Snapshot Consolidation Progress information

Info

Description

Consolidation
Type

Type of consolidation: Manual, Daily, Weekly, Monthly.

Consolidation
Range

Start and end times of snapshots being consolidated.

Progress

Completion percentage. A pending status indicates that the consolidation is


waiting for additional snapshots in the journal to be distributed to the user
volume. The consolidation will begin automatically once the snapshots have
been distributed.

Stop

Cancels the consolidation process. The consolidation process stops after it


completes processing the PIT it is currently working on. Stopping a consolidation
process returns the journal to the same state it was in before the consolidation
started.

The image list in the copy Journal tab displays the following information:

Table 29 Journal tab image list

112

Info

Description

Point in Time

Closing time of the snapshot.

Size

Size of snapshot.

Bookmark

Displays the bookmark name.


For consolidated snapshots, displays the consolidation type (manual, daily,
weekly, monthly). A tool tip indicates how much space was saved by the
consolidation.

Consistency

Whether the snapshot is known by the user to be application-consistent or not.


The two possible values for this field are Application-Consistentand CrashConsistent. All snapshots are crash-consistent by default. Snapshots are
automatically marked application-consistent by KVSS during snapshot creation,
and can be manually marked by the user using the Snapshot Policy button in
the copy Journal tab.

RecoverPoint 4.4 Administrator's Guide

Protection

Table 29 Journal tab image list (continued)

Info

Description
Note

Marking a snapshot as application-consistent does not create an applicationconsistent snapshot, it only tags the snapshot as application-consistent.
Snapshots marked as application-consistent have a higher probability of being
included in the snapshots displayed in the image list. If snapshot consolidation
is enabled, consider setting a Consolidation Policy of Never Consolidate for
snapshots that are marked as application-consistent.
Consolidation
Policy

Consolidation policy applied to this snapshot (Always Consolidate, Never


Consolidate, Survive Daily, Survive Weekly, Survive Monthly).
XtremIO snapshot pruning policy cannot be changed. It is displayed as Always
Consolidate.

Monitoring distribution performance


Describes how to monitor distribution performance at a specific copy.
Procedure
1. Detect bottlenecks on page 239.
2. Select the General detection including initialization and high load periods with peak
writing analysis option.
3. Select both Advanced overview and Detailed overview.
4. Specify the other required information.
5. In the output, scroll-down until you reach a section that starts with the text: System
overview of the copy:<copyname

6. In five-phase distribution mode, note the process steps labeled as Distributor phase
1 and Distributor phase 2, and note their performance statistics.

7. In three-phase distribution mode, note the process labeled Fast forward distribution
duration, and note the statistics.
Monitoring and managing copies

113

Protection

Enabling snapshot consolidation


Before you begin
l

The total size of all snapshots between the specified start and end times must be at
least 1 GB.

Snapshots that account for 90% of the consolidation period must be available in
journal.
For example, for daily consolidation to take place, the starting and ending snapshots
must be at least 22 hours apart. Likewise, automatic consolidation will not take place
if the snapshots in the journal exceed 110% of the consolidation period. For
example, daily consolidation will not take place if the starting and ending snapshots
are more than 26 hours apart.

You cannot enable automatic consolidation on a consistency group that belongs to a


group set.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select a copy.
3. In the right pane, click the Copy Policy tab.
4. Select the Enable RecoverPoint Snapshot Consolidation checkbox to begin
consolidating the snapshots in your copy journal according to the default settings, or
adjust the settings to your specific requirements.
Results
Automatic snapshot consolidation begins according to the values set in the Copy Policy
tab, provided the system is in the distribution phase and initialization is over.
After you finish
To display the consolidation progress and space saved by the process:
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select a copy.
3. In the right pane, click the Journal tab.

Modifying consistency groups


Use the Manage Protection screen to:

114

Modify a groups protection policy on page 115

Modify a links protection policy on page 115

Remove a group on page 116

Add a copy on page 116

RecoverPoint 4.4 Administrator's Guide

Protection

Remove a copy on page 123

Add replication sets on page 123

Remove replication sets on page 128

Pause transfer on page 128

Start transfer on page 128

Disable groups on page 129

Enable groups on page 129

Rename a replication set on page 133

Modify a groups protection policy


Use this procedure to modify the protection policy of an existing consistency group.
Procedure
1. Select Protection > Manage Protection.
2. Select the consistency group in the left pane.
3. Click the Group Policy tab in the right pane.

Results
All of the Group policies on page 80 can be modified through the Group Policy tab.

Modify a links protection policy


Use this procedure to modify the protection policies of the links between existing
consistency group copies.
Procedure
1. Select Protection > Manage Protection.
2. select the consistency group in the left pane and click the Link Policy tab in the right
pane.

Modifying consistency groups

115

Protection

Results
All of the Link policies on page 82 can be modified through the Link Policy tab.
If you have created Policy Templates on page 68, use the Select Template drop-down to
select a policy template. Click Load to load the template settings into the dialog box.
NOTICE

After loading a policy template, modify the Link policies on page 82 as required, and click
Apply to use the settings. The settings are not applied until the Apply button is clicked.

Remove a group
NOTICE

This action cannot be undone.


To remove one or more groups from the system, select Protection > Manage Protection,
and then, either:
l

Remove one group by selecting the consistency group in the left pane, clicking the
Status tab, and clicking Remove Group.

Remove multiple groups by clicking Consistency Groups in the left pane, selecting
one or more groups in the right pane, and clicking Remove Group.

Add a copy
Use the following procedure to add a copy to an existing consistency group.
Before you begin
In the Add a Copy wizard:

116

The Cancel button closes the wizard and disables image access without creating the
copy.

The Finish button closes the wizard, creates the copy, and optionally, starts
replication to the new copy.

RecoverPoint 4.4 Administrator's Guide

Protection

NOTICE

VPLEX Metro users can also use this procedure to turn a non-MetroPoint group into a
MetroPoint group.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, select the Status tab, and click Add Copy. The Add a Copy wizard is
displayed.
4. In the Add a Copy screen:
Note

This screen changes depending on whether or not VPLEX distributed volumes have
been mapped to RPA cluster at the production, and the group contains no remote
copies, or one remote copy.
l

If the there are VPLEX distributed volumes mapped to the RPA cluster at the
production, and the group contains no remote copies, or one remote copy, the Add
a Copy screen will contain two options for adding a copy.

If you are not a VPLEX Metro user or you do not want to turn a non-MetroPoint
group into a MetroPoint group, skip to the next bullet. If you are a VPLEX Metro
user, and want to turn a non-MetroPoint group into a MetroPoint group:
a. Select the Add Standby > Production Copy radio button.
b. Enter a name for the copy in the Standby production copy name field.
c. In the volume list, mark the checkboxes of each journal volume that you want
to use in your standby production journal.
Use the Toolbar options on page 56 at the top-right corner of the screen to help
you identify the required volumes in the volume list. For best performance,
select volumes that are identical in size. If identically sized volumes are not
available, select volumes that are similar in size.
These are the copy journal volumes for the standby copy, not the copy volumes.

Modifying consistency groups

117

Protection

d. Optionally, click Modify Policy to define the Journal policies on page 86 now, or
skip this step and define the policies later when you Modify a copys protection
policies on page 130.
The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is
a specific business need to set other policies.
If you want to change the protection settings using Policy Templates on page
68, use the Select Template drop-down to select one of your templates. Click
Load to load the settings in the template and click OK to apply the settings.
Note

This link is only enabled after volumes are selected in the volume list.
e. Click Next Display Copy Summary, and continue to Step 5 on page 213 on page
122.
l

If the groups production volumes are not VPLEX distributed volumes or the group
contains more than one remote copy, in the Add a Copy screen:

a. Define the copy name, the RPA cluster that will manage the copy volumes, and
the replication mode on page 82.
b. For each production volume, click the Select volume link to add a volume to
the copy. The selected volume is the volume that the production volume is
replicated to.
Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at


that copy must be on the same array.
The volume list is displayed. The volume list only contains volumes of sizes
that are equal to, or greater than, the specified production volume.

118

RecoverPoint 4.4 Administrator's Guide

Protection

Note

In VPLEX splitter environments, the VPLEX consistency group name will only be
displayed in the VPLEX Group column of the volume list if you Attach volumes
to the splitter on page 202 for the VPLEX splitter (including certificate). After
you enter VPLEX credentials, you will not be able to select VPLEX volumes and
non-VPLEX volumes in the same consistency group copy. It is recommended
that all volumes in a VPLEX consistency group be configured in a single
consistency group copy, and all volumes in a non-VPLEX consistency group be
configured in another consistency group copy.
In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped
and zoned to the RPA cluster are displayed in the available volumes list. In
RecoverPoint/SE, all of the volumes of the supported VNX Series array are
displayed in the available volumes list. See the appropriate technical notes for
your splitter for more information.
Use the toolbar options on page 56 at the top-right corner of the screen to help
you identify the required volumes in the volume list. For best performance
during failover, select volumes that are the same size as the production
volume. If volumes of the same size are not available, select volumes that are
as similar in size as possible.
NOTICE

You cannot proceed to the next step until the replication sets icon under the
volume list shows a green check mark to indicate that the number of
replication sets is equal to the number of production volumes.

a. Optionally, click Modify Policy to define the Copy policies on page 86 and Link
policies on page 82 now.
Modifying consistency groups

119

Protection

The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is a
specific business need to set other policies.
If you want to change the protection settings using Policy Templates on page 68, in
the Link tab, use the Select Template drop-down to select one of your templates.
Click Load to load the settings in the template and click OK to apply the settings.
b. To continue configuring the group, click Next Define Copy Journal.
5. In the Define Copy Journal step:
If the volume that you specified in the last step is not on a VNX/VNXe/CLARiiON array,
skip to step a on page 121
l

If the volume that you specified in the last step resides on a VNX/VNXe/CLARiiON
array that has already been registered in RecoverPoint, and a resource pool was
already registered for the same VNX/VNXe/CLARiiON array, the Journal
Provisioning Method screen is displayed.

If you want to allow RecoverPoint to automatically provision your journal volumes


for you, keep the Automatically Provision Journal Volumes radio button selected,
and select one of the following options:

120

Option

Description

Journal size

Enter a value for the journal size to define an absolute journal


size at the copy.

Predicted
incoming
write-rate

Select this option and enter a value for predicted incoming writerate and required protection window to have RecoverPoint/SE
calculate the required journal size based on your consistency
groups predicted write-rate and a required protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.

RecoverPoint 4.4 Administrator's Guide

Protection

Option

Description

Required
protection
window

Note

Only relevant if Predicted incoming write-rate is selected.


Enter a value for the required protection window to have
RecoverPoint/SE calculate the required journal size based on
your consistency groups predicted write-rate and a required
protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.

When you are done configuring your journal settings, click Next Display Copy
Summary.
l

If you do not want RecoverPoint to automatically provision your journal volumes on


storage, ensure the Select Provisioned Journal Volumes radio button is selected.

For applications running on a host cluster, the journal volumes must not be
resources of the host cluster.
In the volume list:
a. Select the volumes to add to the journal.
On XtremIO arrays, the journal does not need to be larger than 10 GB for nondistributed consistency groups, 40 GB for distributed consistency groups.
Use the toolbar options on page 56 at the top-right corner of the screen to help
you identify the required volumes in the volume list. For best performance,
select volumes that are identical in size. If identically sized volumes are not
available, select volumes that are similar in size.
b. Optionally, click Modify Policy to define the Journal policies on page 86 now.
The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is
a specific business need to set other policies.
Modifying consistency groups

121

Protection

If you want to change the protection settings using Policy Templates on page
68, use the Select Template drop-down to select one of your templates. Click
Load to load the settings in the template and click OK to apply the settings.
c. Click Next Display Copy Summary.
6. In the Copy Summary screen:
a. Select a copy to display its volume details and ensure the copy summary reflects
your anticipated copy configuration:

In RecoverPoint/SE:
l

If you selected to Automatically provision journal volumes based on a predefined Journal Size, the Journal Size is displayed at the bottom of the Copy
Summary screen.

If you selected to Automatically provision journal volumes based on a Predicted


incoming write-rate, and a Required protection window, the Calculated Journal
Size is displayed at the bottom of the Copy Summary screen.

b. Use the following options to create the copy, and optionally, start replication.
Action

Description

Start
replication
to copy
when I click
'Finish'

When selected, starts transfer to the copy as soon as the copy is created.

Finish

Creates the copy, applies all of the specified settings, and starts a firsttime initialization process to synchronize the production volumes to the
copy volumes.

Note

Before you start transfer to any copy, ensure that the copy volumes are
unmounted from any hosts and any volume groups are deported from the
logical volume manager (AIX, HP-UX, Windows, and Solaris have volume
managers built into the operating system; Veritas Volume Manager can be
used with any of these operating systems).

Before you click Finish, note the following:

122

RecoverPoint 4.4 Administrator's Guide

Protection

Action

Description
l

If First-time initialization is taking too long on page 280, instead of


allowing RecoverPoint to send the initialization data to the copy
storage, you can initialize the copy volumes from a backup disk, saving
the time it would otherwise take to synchronize the data over a WAN or
FC connection.

By default, RecoverPoint writes the first snapshot directly to the copy


storage, bypassing the copy journal. Using the fast_first_init
parameter of the config_link_policy CLI command, you can
override the default setting and write the initialization snapshot first to
the journal (which is more time-consuming but provides greater data
protection).

To enable failover during initialization, set the value of the


allow_long_resync parameter of the config_copy_policy CLI
command to no (default is yes).
If your hosts are part of a Windows clustered environment or running
on an AIX operating system, you will need to perform a first-time
failover as soon as possible after first-time initialization is complete.

If your hosts are part of a Windows clustered environment, see the


First-time failover section of the EMC RecoverPoint Replicating the
Windows File System Technical Notes for the specific procedure.

If your hosts are running on an AIX operating system, see the Firsttime failover section of the EMC RecoverPoint Deploying with AIX Hosts
Technical Notes for the specific procedure.

Remove a copy
Use the following procedure to remove a copy of an existing consistency group.
NOTICE

This action cannot be undone.


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select the copy.
Note

If you want to turn a MetroPoint group into a non-MetroPoint group, select the standby
production copy.
3. In the right pane; select the Status tab, click Remove Copy.

Add replication sets


Before you begin
l

When a replication set is added to a consistency group all of the consistency group
volumes undergo a short pause in transfer and a short initialization, and a volume
sweep occurs on the newly added volumes.

Modifying consistency groups

123

Protection

The journal of all copies of the consistency group will be lost if:
n

All RPA clusters in your system are not running RecoverPoint 4.4 or later.

The initialization snapshot is too large for the capacity of the journal, triggering a
long initialization.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Replication Sets tab, and click Add Replication Sets.

The Add Replication Sets wizard opens and the Select Production Volumes screen is
displayed.
4. In the Select Production Volumes screen:
a. Select your production volumes in the volume list.
Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that
copy must be on the same array.

124

RecoverPoint 4.4 Administrator's Guide

Protection

Use the toolbar options on page 56 at the top-right corner of the screen to help you
identify the volumes.
The VPLEX consistency group name will only be displayed in the VPLEX Group
column if you have entered the VPLEX storage credentials (including certificate).
When you enter VPLEX credentials, you will not be able to select VPLEX volumes
and non-VPLEX volumes in the same RecoverPoint consistency group copy. It is
recommended that all volumes in a VPLEX consistency group be configured in a
single RecoverPoint consistency group copy, and vice versa.
b. To continue configuring the replication set, click Next Select Copy Volumes.
5. In the Select Copy Volumes screen:

a. Click the Select volume link to add a volume to the copy.


The selected volume is the volume that the production volume is replicated to. The
volume list is displayed.
The volume list only contains volumes of sizes that are equal to, or greater than,
the specified production volume.
Modifying consistency groups

125

Protection

Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that
copy must be on the same array.

Use the toolbar options on page 56 at the top-right corner of the screen to help you
identify the required volumes in the volume list.
For best performance during failover, select volumes that are the same size as the
production volume. If volumes of the same size are not available, select volumes
that are as similar in size as possible.
You cannot go on to the next step until the Replication sets icon under the volume
list shows a green check mark to indicate that the number of Replication sets is
equal to the number of production volumes.

b. When you are done selecting the volumes at each copy, click Next Review
Replication Sets.
6. In the Review Replication Sets screen:

126

RecoverPoint 4.4 Administrator's Guide

Protection

a. Select a row in the replication sets table in the top section to display the
associated volume information in the bottom section.
b. Click on the replication set name to open the Edit Replication Set dialog box.

c. If required, rename the replication set.


d. Optionally, select the Attach as Clean checkbox (advanced users ONLY).
NOTICE

When you add a new replication set to an existing consistency group, RecoverPoint
ensures consistency between the new copy volume(s) and their production
source(s) by performing a volume sweep on all of the volumes in the new
replication set. Selecting the Attach as Clean checkbox overrides the default
synchronization process by informing the system that the new copy volumes are
known to be an exact image of their corresponding production volumes. If Attach
as Clean is checked, and the new copy volumes are inconsistent with their
corresponding production volumes, they will remain inconsistent. To ensure
consistency, best practice is not to use this option and allow the default
RecoverPoint synchronization process to ensure consistency between the new
copy volumes and the production.
7. Click Finish.

Modifying consistency groups

127

Protection

Remove replication sets


Before you begin
l

When a replication set is removed from a consistency group, all copy journals are
lost, and all of the consistency group volumes undergo a short pause in transfer and
a short initialization.

If the replication set is removed during a long initialization, the consistency group will
undergo a full sweep.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
Enabled consistency groups must contain at least one replication set. If you want to
remove all of the replication sets in the group, click the Status tab and then click
Disable Group now.
3. In the right pane:
a. Select the Replication Sets tab.
b. Select one or more replication sets to remove.
Note

Clicking on a cell in any of the copy columns displays the relevant volume
information in the bottom pane.
4. Click Remove Replication > Set and click yes in the confirmation dialog boxes.
If you disabled the group after step 1 on page 128, select the Status tab and click
Enable Group now.
NOTICE

All copy journals are lost, and all of the consistency group volumes undergo a short
pause in transfer and a short initialization.

Pause transfer
Use the following procedure to pause transfer between production and all group copies.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Status tab, click Pause Transfer, and click yes in the
confirmation dialog box.

Start transfer
Before you start transfer to any copy, ensure that the copy volumes are unmounted from
any hosts and any volume groups are deported from the logical volume manager.
Use the following procedure to start transfer between production and all group copies.

128

RecoverPoint 4.4 Administrator's Guide

Protection

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Status tab, click Start Transfer, and click yes in the
confirmation dialog box.

Disable groups
Use the following procedure to disable one or more enabled consistency groups.
Note

Disabling a group stops replication, deletes all copy journals, and causes a full sweep on
all copies in the group when the group is re-enabled.
Procedure
1. Select Protection > Manage Protection.
2. Disable the group(s):
l

To disable one group, select the group in the left pane, click the Status tab, and
click Disable Group.

To disable multiple groups, click Consistency Groups in the left pane, select one or
more groups in the right pane, and click Disable Group.

Enable groups
Use the following procedure to enable one or more disabled consistency groups.
Note

Enabling a consistency group automatically starts transfer. Before you start transfer to
any copy, ensure that the copy volumes are unmounted from any hosts and any volume
groups are deported from the logical volume manager.
Procedure
1. Select Protection > Manage Protection.
2. Enable the group(s):
l

To enable one group, select the group in the left pane, click the Status tab, and
click Enable Group.

To enable multiple groups, click Consistency Groups in the left pane, select one or
more groups in the right pane, and click Enable Group.
NOTICE

When transfer starts, a full sweep occurs.

Modifying copies
Use the Manage Protection screen to:
l

Modify a copys protection policies on page 130

Remove a copy on page 130


Modifying copies

129

Protection

Pause transfer on page 131

Start transfer on page 131

Disable a copy on page 131

Enable a copy on page 131

Modify a copys protection policies


Use this procedure to modify the protection policies of an existing copy and its journal.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select a copy.
3. In the right pane, click the Copy Policy tab.

Results
All of the Copy policies on page 86 and Journal policies on page 86 can be modified
through the Copy Policy tab.
If you have created Policy Templates on page 68, use the Select Template drop-down to
select one of them, and click Load to load their settings into the dialog box.
NOTICE

After loading a policy template, modify the Copy policies on page 86 and Journal
policies on page 86 as required, and click Apply to use the settings. The settings are not
applied until the Apply button is clicked.

Remove a copy
Use the following procedure to remove a copy of an existing consistency group.
NOTICE

This action cannot be undone.


130

RecoverPoint 4.4 Administrator's Guide

Protection

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group, and select the copy.
3. In the right pane; select the Status tab, click Remove Copy, and click yes in the
confirmation dialog box.

Pause transfer
Use the following procedure to pause transfer between production and a copy.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Pause Transfer, and click yes in the
confirmation dialog box.

Start transfer
Before you start transfer to any copy, ensure that the copy volumes are unmounted from
any hosts and any volume groups are deported from the logical volume manager.
Use the following procedure to start transfer between production and a copy.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Start Transfer, and click yes in the
confirmation dialog box.

Disable a copy
Disabling a copy stops replication, deletes the copy journal, and causes a volume sweep
when the copy is re-enabled.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Disable Copy, and click yes in the
confirmation dialog box.

Enable a copy
Enabling a copy automatically starts transfer.
Before you begin
Before you start transfer to any copy, ensure that the copy volumes are unmounted from
any hosts and any volume groups are deported from the logical volume manager.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Enable Copy, and click yes in the
confirmation dialog box.
Modifying copies

131

Protection

NOTICE

A volume sweep commences when transfer starts.

Modifying links
To modify the protection policy of the links between existing consistency group copies:
Procedure
1. Select Protection > Manage Protection.
2. Select the consistency group in the left pane
3. In the right pane, click the Link Policy tab.

Results
All of the Link policies on page 82 can be modified through the Link Policy tab.
If you have created Policy Templates on page 68, select a link in the diagram and use the
Select Template drop-down to select a policy template. Click Load to load the template
settings into the Link Policy dialog box.
NOTICE

After loading a policy template, modify the Link policies on page 82 as required, and click
Apply to use the settings. The settings are not applied until the Apply button is clicked.

Modifying replication sets


Use the Replication Sets dialog box to:

132

Rename a replication set on page 133

Edit replication sets on page 133

Resize volumes in a replication set on page 134

RecoverPoint 4.4 Administrator's Guide

Protection

Rename a replication set


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Replication Sets tab, select a replication set, and click
Edit Replication Set.

4. Change the value in the Name field, and click OK.

Edit replication sets


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Replication Sets tab, select a replication set, and click
Edit Replication Set to display the Edit Replication Set dialog box:

a. Click the blue link next to the name of the RPA cluster containing the volume you
want to replace. The Select Volumescreen is displayed.

Modifying replication sets

133

Protection

Note the production volume specified in the Production volume area at the top of
the screen.
b. From the Volumes list, select the volume to which you want to replicate the
specified production volume.
NOTICE

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that
copy must be on the same array. The volume list only displays volumes at the RPA
cluster that are equal to, or greater than, the specified production volume. For best
performance during failover, select a volume that is the same size as the
Production volume. If a volume of the same size is not available, select a volume
that is as similar in size as possible.
c. Click OK to return to the Edit Replication Set dialog box.
d. Click OK to apply the new replication set volume configuration, and click yes in the
confirmation dialog.
NOTICE

Data transfer is briefly paused for the group, and a volume sweep occurs on the
new volume(s) when data transfer is enabled again.

Resize volumes in a replication set


Note

Not relevant in XtremIO.


Volumes in a replication set can be resized in two ways:
l

Dynamic LUN resizing on page 135, which does not result in a full sweep.

Non-dynamic LUN resizing on page 136, which does result in a full sweep.
NOTICE

You cannot Resize volumes in a replication set on page 134 that are based on thick
LUNs while snap-based replication is enabled. Disable snap-based replication before
you Resize volumes in a replication set on page 134 that are based on thick LUNs.

134

RecoverPoint 4.4 Administrator's Guide

Protection

Dynamic LUN resizing


Before you begin
l

All volumes in the replication set are attached to a VNX/VNXe splitter.

VNX OE for Block 05.32 or later is available at all copies.

All volumes should be resized at the same time and to the same size. If they are not
the same size, the allocated LUNs will not be fully utilized.

Only dynamic expanding of LUNs is supported, not shrinking of LUNs.

After resizing the volumes in a replication set, virtual image access with roll will not
be allowed to a point-in-time before the volume was resized. Virtual image access
(without roll) and logged (physical) image access to a point-in-time before the
resizing are supported.

Procedure
1. On the VNX/VNXe array (in Unisphere), resize the LUNs:
a. Click the LUN tab.
b. Select the copy LUN or LUNs, right-click on the LUN name, select Expand, and enter
the new size.
The final size of all LUNs in the replication set should be exactly the same.
c. Select the production LUN, select Expand, and enter the new size.
Once the LUNs have been resized, RecoverPoint will fake the sizes (as if they were
still the old sizes). Replication continues using the faked sizes.
2. In Unisphere for RecoverPoint, select Protection > Manage Protection.
3. In the left pane, select the consistency group.
4. In the right pane; select the Replication Sets tab, select the replication set, and click
Edit Replication Set to display the Edit Replication Set dialog box.

5. Click the blue link of the copy whose volume you want to resize. When prompted,
confirm that you wish to resize.
RecoverPoint runs SAN discovery on the new volumes and starts a short initialization
of the expanded areas. It may take a few minutes for RecoverPoint to correctly display
the new sizes of volumes.
After resizing the volumes in a replication set, virtual image access with roll will not be
allowed to a point-in-time before the volume was resized. Virtual image access
(without roll) and physical image access to a point-in-time before the resizing are
supported.
Modifying replication sets

135

Protection

6. Make any necessary updates in hosts and applications (such as re-scanning the SAN).

Non-dynamic LUN resizing


NOTICE

This procedure will cause a full sweep.


Procedure
1. In RecoverPoint:
a. For all attached volumes, Detach volumes from the splitter on page 203 (not
relevant for XtremIO).
b. Remove the replication set according to Remove replication sets on page 128.
2. On the storage, resize all LUNs in the replication set: Either resize the LUN or delete
the LUN and define a larger one.
3. In RecoverPoint:
a. Create a new replication set containing the resized or new LUNs according to Add
replication sets on page 123.
b. Attach the resized or new LUNs to the splitter according to Attach volumes to the
splitter on page 202 (not relevant for XtremIO).
4. Make any necessary updates in hosts and applications (such as re-scanning the SAN).
Some hosts and applications are sensitive to changes in the volume size.

Modifying journals
There are two ways to change the storage capacity of an existing journal:
l

Add or edit journal volumes on page 136; this process does not trigger a full sweep
synchronization, nor does it erase all history in the existing journal. After this
procedure, the copy can still be rolled back to a previous point-in-time.

Change the size of a journal LUN on storage on page 137; this process does trigger a
full sweep synchronization, and it does erase all history in the existing journal. After
this procedure, the copy cannot be rolled back to a previous point-in-time.

Add or edit journal volumes


Use the following procedure to change the storage capacity of a journal at a copy by
adding (or removing) journal volumes, for each copy in the group.
Before you begin
l

In RecoverPoint/CL and RecoverPoint/EX only: Only the volumes that are exposed to
the RPA cluster are displayed in the available volumes list. If the required volumes
are not in the available volumes list, mask the LUNs to RecoverPoint WWNs, and click
the Rescan button to update the list of available volumes.

In RecoverPoint/SE, all of the volumes of the supported VNX Series array are
displayed in the available volumes list.

For best performance, select volumes that are identical in size. If identically sized
volumes are not available, select volumes that are similar in size.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group, and select a copy.
136

RecoverPoint 4.4 Administrator's Guide

Protection

3. In the right pane:


a. Select the Journal tab.
b. Click Edit Journal Volumes. The Edit Journal Volumes dialog box is displayed.

c. Mark the checkboxes of the volumes that you want to add to the journal.
Clear the checkboxes of the volumes that you want to remove from the journal.
Multiple volumes can be selected.
d. Click OK.
After you finish
If you performed this procedure to enlarge the journal capacity after distributing the
writes of this consistency group across multiple RPAs, disable and then re-enable this
group to trigger a full sweep.

Change the size of a journal LUN on storage


Note

Not relevant in XtremIO.


Use the following procedure to resize journals by resizing an existing journal volume.
Procedure
1. In RecoverPoint, select Recovery > Test a Copy (see Test a Copy on page 147):
a. Enable access to the Latest Image at the copy containing the journal volume(s), in
Logged Access mode.
b. Exit the wizard without disabling image access.
Note

This ensures that all writes up until the latest image are transferred from the copy
journal to the copy storage, and only the host writes from that point forward will
need to be synchronized at the end of this procedure.
2. In RecoverPoint, disable the copy and remove the journal volume(s). Select Protection
> Policy Templates (see Policy Templates on page 68):
Modifying journals

137

Protection

a. In the left pane, expand the group that the journal volume belongs to and select
the copy.
b. In the right pane; select the Journal tab, and click Edit Journal > Volumes. The Edit
Journal Volumes dialog box is displayed.

c. Clear the checkbox of the journal volume whose size you want to change and click
OK. The journal volume is removed from the RecoverPoint system.
3. At the copys SAN, dedicate more storage resources to the required LUN(s).
4. In RecoverPoint, click Edit Journal Volumes .
The RecoverPoint SAN discovery utility automatically detects the change in the
physical size of the volume.
5. Select the resized LUN(s).
6. Click the OK button to apply the changes and exit the dialog box.
7. In the right pane, select the Status tab.
8. Select Recovery > Manage Recovery (see Manage Recovery on page 173), and select
Finish Testing in the Select Next Action combo box of the copy containing the
expanded journal volume(s).
NOTICE

Data transfer is briefly paused for the group, and a short initialization may occur, but
only the writes that occurred after image access was enabled to the copy will be
synchronized.

Modifying snapshots
All of the points in time (snapshots) created by RecoverPoint are crash-consistent by
default. You can set or modify one or more snapshots' default policies by using the
following procedures:

138

Modify the policies of one snapshot on page 140

Create a bookmark and define snapshot policies for one group on page 140

RecoverPoint 4.4 Administrator's Guide

Protection

Note

To automatically create parallel bookmarks at pre-defined intervals, Create a group


set on page 74.
l

Create a bookmark and define snapshot policies across multiple groups on page 141

Use KVSS to Create and mark an application-consistent point in time on page 142.
The snapshot settings and policies are:
Table 30 Snapshot policies

Setting

Description

Bookmark

The name applied to a point in time snapshot. Bookmarks mark a consistent


point in time across all volumes in the group.

Mark as

The two possible values for this field are Application-Consistentand CrashConsistent. All snapshots are Crash-Consistent by default. Select ApplicationConsistentfrom the drop down if the selected snapshot is known to be
application-consistent.
Selecting Application-Consistent does not create an application-consistent
snapshot, it only tags the snapshot as application-consistent.
Note

Snapshots are automatically marked as application-consistent by KVSS


during snapshot creation, and can be manually marked by the user using the
Snapshot Policy button. Snapshots marked as application-consistent have
a higher probability of being displayed in the image list. If snapshot
consolidation is enabled, consider selecting Define a Consolidation Policy for
this Snapshot and Never Consolidate this Snapshot for snapshots that are
marked as application-consistent.

Define a
Consolidation
Policy for this
Snapshot

Note

Not relevant in ProtectPoint


Sets the consolidation policy for the snapshot.
The default consolidation policy for a snapshot is Always Consolidate, which
means that the snapshot is consolidated the next time that the consolidation
process runs.
Select Never Consolidate this Snapshot if you never want this snapshot
consolidated.
Select Snapshot must survive Daily/Weekly/Monthly consolidations
for:
l

Daily: Snapshot remains after daily consolidations, but is consolidated in


weekly, monthly, and manual consolidations.

Weekly: Snapshot remains after daily and weekly consolidations, but is


consolidated in monthly and manual consolidations.

Monthly: Snapshot remains after daily, weekly, and monthly


consolidations, but is consolidated in manual consolidations.

Modifying snapshots

139

Protection

Table 30 Snapshot policies (continued)

Setting

Description
Note

The XtremIO snapshot pruning policy cannot be changed.

Define a
Retention
Policy for this
Snapshot

Note

Only relevant in ProtectPoint

Modify the policies of one snapshot


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select a copy.
3. In the right pane; select the Journal tab, select a snapshot in the list, and click Edit.
The Snapshot Policy dialog box is displayed.

4. In the Snapshot Policy dialog box, enter or modify any of the snapshot policies on
page 139 and click OK.

Create a bookmark and define snapshot policies for one group


Before you begin
l

You can only bookmark a snapshot in a consistency group that is enabled and
actively transferring.

Latest is a reserved term and therefore cannot be used when marking snapshots.

Use the following procedure to change a snapshots settings and policies across all
volumes in a group.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane; select the Status tab, and click Apply Bookmark. The Apply
Bookmark dialog box is displayed.

140

RecoverPoint 4.4 Administrator's Guide

Protection

4. Enter a descriptive name in the Bookmark field.


5. Enter or modify any of the snapshot policies on page 139.
6. Click OK.
Note

All RecoverPoint snapshots are crash-consistent. Use KVSS to create applicationconsistent on page 142 snapshots.

Create a bookmark and define snapshot policies across multiple groups


Use this procedure to identify a similar recovery point in multiple consistency groups
running on different RPAs.
Before you begin
l

You can only bookmark a snapshot in a consistency group that is enabled and
actively transferring.

Latest is a reserved term and therefore cannot be used when marking snapshots.

When applying a bookmark to multiple groups in parallel, the system creates a bookmark
with the same name, snapshot policy, and consistency tag at a similar point in time on
each of the relevant consistency group volumes.
Note

This procedure does not create parallel bookmarks. To create parallel (consistent)
bookmarks at pre-defined intervals, create a group set on page 74.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select Consistency Groups.
3. In the right pane; select two or more consistency groups, click Create Bookmark.

The Create Bookmark dialog box is displayed.


Modifying snapshots

141

Protection

4. Enter a descriptive name in the Bookmark field.


5. Enter or modify any of the snapshot policies on page 139.
6. Click OK.

Create and mark an application-consistent point in time


The RecoverPoint KVSS utility is a command-line utility that enables applying bookmarks
to Windows 2012 R2 and 2008-based applications that support Microsoft Volume
Shadow Copy Service (VSS). Microsoft Exchange and SQL are examples of Windows
applications that support Volume Shadow Copy Service.
A single bookmark can be used to bookmark VSS-aware applications in many
consistency groups. VSS guarantees that the applications are in a consistent state at the
point-in-time when each bookmark is applied to an image. As a result, recovery using an
image with a KVSS bookmark is faster than recovering from normal RecoverPoint images.
The EMC RecoverPoint Replicating Microsoft SQL Server Technical Notes and the EMC
RecoverPoint Replicating Microsoft Windows File Systems Technical Notes contain additional
detailed information on how to download, install and use RecoverPoint KVSS to create
application-consistent bookmarks.
KVSS bookmarks are created using the kvss.exe bookmark command in Windows 32-bit
environments, and kvssx64.exe bookmark command in Windows 64-bit environments.
The working folder for running KVSS commands is %SystemDriver%/
EMCRecoverPointVSSProvider/.
Note

Any bookmarks created using KVSS are automatically marked as Application-Consistent in


the RecoverPoint snapshot policies on page 139.
When using KVSS to apply bookmarks:
l

Parameter values should be surrounded by quotation marks.

You can use the vssadmin list writers command to obtain a list of registered
writers on the host machine.

You can use the kvss.exe list command to display the components of each of
the writers found using the vssadmin list writers command.

You can run the kvss.exe set_credentials command once per Windows user
to define the ip, user and password. After doing so, you will not need to enter these
values again.

Multiple writers and groups can be entered simultaneously if they are separated by a
space.

To use KVSS in a Microsoft Cluster Server environment with Symmetrix DMX storage, the
SPC-2 flag must be enabled on the Symmetrix ports.The syntax is as follows:
kvss.exe bookmark
bookmark=<bookmark_name>
writers=<writer_name> <writer_name>
[groups=<group_name> <group_name>]
[consolidation_policy=never|survive_daily|survive_weekly|
survive_monthy|always]
[type=[FULL|COPY]]
[ip=<RecoverPoint_cluster_management_ip_address>]
[user=<RecoverPoint_username>]
[password=<RecoverPoint password>]

142

RecoverPoint 4.4 Administrator's Guide

Protection

Note

Parameters that are surrounded by square brackets [ ] are optional.


where:
l

writer = a VSS-aware host application

group = RecoverPoint consistency group

bookmark = name by which you can identify the applied bookmark

consolidation_policy = consolidation policy to set for this snapshot. Valid values are:
n

never; Snapshot is never consolidated.

survive_daily; Snapshot remains after daily consolidations, but is consolidated in


weekly, monthly, and manual consolidations.

survive_weekly; Snapshot remains after daily and weekly consolidations, but is


consolidated in monthly and manual consolidations.

survive_monthly; Snapshot remains after daily, weekly, and monthly


consolidations, but is consolidated in manual consolidations.

always; Snapshot is consolidated in every consolidation process, whether manual


or automatic.
Note

The default policy is always. If the consolidation_policy parameter is not


specified, the snapshot is consolidated in both automatic and manual
consolidation processes.
l

type =The shadow copy type: either FULL or COPY. This setting is optional. The default
is COPY. The settings full and copy are implemented by the writer application.
Generally, when type = full,backup logs are truncated; when type = copy, backup logs
are not truncated.

ip = RPA cluster management IP

user = RecoverPoint username

password = RecoverPoint password


NOTICE

You can use the kvss.exe set_credentials command to define the IP


address, user, and password one time per Windows user, instead of entering these
values each time you create a bookmark.
For example, to create a bookmark for a Microsoft Exchange application for the first time:
kvss.exe set_credentials
ip=10.10.0.145
user=admin
password=admin
kvss.exe bookmark
writers="Microsoft Exchange Writer"
groups="exchange\comp1" exchange\comp2
bookmark="exchange hourly snapshot"
consolidation_policy=survive_daily

Modifying snapshots

143

Protection

To create a bookmark every subsequent time for a Microsoft Exchange application after
defining the ip, user, and password through the kvss.exe set_credentials
command:
kvss.exe bookmark
writers="Microsoft Exchange Writer"
groups="exchange\comp1" exchange\comp2
bookmark="exchange hourly snapshot"
consolidation_policy=survive_daily

144

RecoverPoint 4.4 Administrator's Guide

CHAPTER 4
Recovery

This section describes how to test copies, recover from disasters, and migrate to a
different storage system.
l
l
l
l
l

RecoverPoint Recovery........................................................................................ 146


Test a Copy......................................................................................................... 147
Test a Copy and Recover Production.................................................................... 160
Test a Copy and Fail Over.....................................................................................165
Manage Recovery................................................................................................ 173

Recovery

145

Recovery

RecoverPoint Recovery
In Unisphere for RecoverPoint, data testing and recovery is guided by wizards, and
performed through the Recovery menu. Before you begin the following procedures, you
should be well acquainted with the concept of RecoverPoint recovery and the role that
copies, journals, and snapshots play in it. Recovery activities can be performed on both
consistency groups and group sets.
NOTICE

Test a Copy and Fail Over is not relevant in ProtectPoint.

Table 31 Recovery options

Option

When to Use

Test a Copy

Displays the Test a Copy wizard to guide you through the process of selecting
and accessing a copy image, and testing the selected image.

Test a Copy
and Recover
Production
Test a Copy
and Fail Over

To ensure that your copies can be used to restore data, recover from
disaster, or seamlessly take over production.

To offload a task. For instance, if you need to run a large query on a


database, and you do not want to tie up your production, you can run the
query on a copy. Of course, this assumes that you do not need the very
latest information (your data will be out of date by your lag time, possibly a
few seconds, plus the length of time it takes to run the query).

Displays the Recover Production wizard to guide you through the process of
selecting and accessing a copy image, verifying the image, and using a copy
journal to roll the production storage back to a previous point in time to correct
file or logical corruption.
Note

Note relevant in ProtectPoint.


Displays the Failover wizard to guide you through the process of selecting and
accessing a copy image, verifying the image, and failing over or failing back to
the production:

146

RecoverPoint 4.4 Administrator's Guide

Recovery

Table 31 Recovery options (continued)

Option

When to Use
l

To maintain the production system, and resume production activities at


the copy until the production system is repaired. This option is only
available for groups with more than one non-production copy.

To permanently migrate all production activities to a copy.

To perform a first-time failover (if your production or copy hosts are


running on AIX operating systems or are part of a Windows cluster server).
See the EMC RecoverPoint Deploying Windows File Server Technical Notes or
the EMC RecoverPoint Deploying with AIX Hosts Technical Notes for the
specific procedure.

Note

Migration requires advanced planning to ensure risk-free migration without


data loss. Contact EMC Customer Support for assistance before migrating. You
can also access the Failover wizard when Monitoring and managing groups on
page 91 and Monitoring and managing copies on page 105. You can also
Initiate recovery activities for a group set on page 77.

Manage
Recovery

To display all of the images that are currently being accessed in the
RecoverPoint system, continue testing, monitor, or continue recovery activities
for consistency groups or a group set.

Test a Copy
Describes the RecoverPoint procedure for accessing and testing copy images.
In RecoverPoint, image access is designed to allow for the brief verification, backup,
cloning, or data analysis of a copy, before failover or production recovery.

Note

You can also access the Test a Copy wizard when Monitoring and managing groups on
page 91, Monitoring and managing copies on page 105, or you can Initiate recovery
activities for a group set on page 77.

Test a Copy

147

Recovery

How to test a copy


Before you begin
l

For added protection, back up the copy image. In XtremIO this is not required as
image access works on a copy of the volume snapshot.

Decide which image access mode best suit your purpose.

If you need more time to test than your image access log or journal will allow, you
can:

Add journal volumes; any additional journal volumes should be permanent, as


removing a journal volume causes all of the snapshots in the journal to be deleted
and requires a full sweep.

Change the image_access_log_size parameter of the


config_copy_policy CLI command to allow more writes to be made while
image access is enabled.
Changing this policy is not recommended since it causes all of the snapshots in
the journal to be deleted and affects the minimum journal size limitation. See the
EMC RecoverPoint Release Notes for the minimum journal size limitation. On
XtremIO arrays, there are no time or space limits for image testing.

In any screen of the Test a Copy wizard:


n

Click Cancel to disable image access and exit the wizard. Click Cancel only when
you are done testing and don't plan to run any further tests on the copy in the near
future. After you click Cancel, RecoverPoint will disable image access and use the
information in the image access log to automatically undo any writes made to the
copy while image access was enabled.
NOTICE

Click Save & Close to exit the wizard without disabling image access and display
the Manage Recovery screen without starting the recovery activity. Use Save &
Close if you have a test plan running and you want to exit the wizard now but keep
image access enabled and return to testing later.

Before testing the copies of a group set, note that:


n

The warning The requested operation will be performed on all of the groups in the
group set excluding the groups listed above.is displayed to the user for a group in
the group set when the requested activity is illegal for one or more groups in the
group set. A recovery activity is legal for a group at a target cluster only if the
group is enabled, and the action is legal for one or more copies of the group.
When this warning is displayed, note that the requested action will not be
performed for the listed groups.

If an action that is part of a recovery activity fails for one or more of the groups in a
group set, you may want to manually perform the action for each group on which
the action failed in order to bring all of the group set's groups to the same
recovery state.

During image access on a group set:


When selecting the latest image, the latest available image of each relevant
group copy will be accessed.
When selecting an image from the image list, the images of the first group in
the group set are displayed to the user. When the user selects an image from
the list, RecoverPoint constructs a search query containing the parameters of

148

RecoverPoint 4.4 Administrator's Guide

Recovery

the first groups image and queries the rest of the groups in the group set
according to the first groups parameters.
When selecting a specific point in time by entering advanced search options,
the same image search query is sent for all groups in the group set (not
supported in XtremIO).
Procedure
1. Select Recovery > Test a Copy.
2. In the Select a Copy screen:
a. Select either a consistency group or a group set by marking the appropriate radio
button and selecting an item from the list.
l

If you selected a consistency group, select a non-production copy from the list
of copies or the copy diagram.

While selecting a copy, you can use the diagram to identify the state of data
transfer between the production and all copies in the group. The direction of
replication and type of replication are both illustrated.
Asynchronous replication is represented by a dashed line, while synchronous
replication is represented by a dashed line on top of a solid line.
Place your icon over any copy icon to display a tooltip with the copy Role, the
Storage state, and the state of Image access.
l

If you selected a group set, select a target RPA cluster from the list. When a
group set is selected, the table under the list of target RPA clusters can be used
to identify the direction of replication and number of copies in each group.

How to test a copy

149

Recovery

b. Click Next Select an Image.


3. In the Select an Image screen:

a. Select the copy image that you want to access. To test the image, you may want to
start with the last image known to be valid. To analyze data, you generally want the
latest image.
Table 32 Select an Image screen

150

Setting

Description

The latest
image

The last snapshot that was created at the production, and transferred to
the copy journal. This is the image at the top of the image list and the
most current snapshot sent from production.

An image
from the
image list

Opens a list of images in the copy journal, and enables you to easily
identify and select an image from the list by point in time, snapshot size
(or the delta between the selected snapshot and the snapshot before it),
bookmark, consistency, and consolidation policy.

RecoverPoint 4.4 Administrator's Guide

Recovery

Table 32 Select an Image screen (continued)

Setting

Description
Note

Use the toolbar icons to manage and filter the image list.

A specific
point in time
or bookmark

On arrays with splitters, there is a limited number of snapshots


(1014) available for selection in the image list. Snapshots that are
not available for selection in the image list may still be selected by
selecting a specific point in time from the Image to access combo
box.

On XtremIO arrays, the maximum number of snapshots available is


specified by the user, but cannot exceed 500. All available snapshots
are listed in the image list. Only the latest snapshot or a snapshot
from the image list can be selected.

The system constantly adds new snapshots to the image list and
discards others in order to maintain the display limit. This is done by
snapshot dilution, a priority selection algorithm in which
bookmarked images receive precedence over other images.

Note

Not relevant in ProtectPoint


Displays advanced search criteria, allowing you to perform a customized
search:
l

Point in Time: Searches for an image that was created at the


specified date and time. Not supported on XtremIO arrays.

Bookmark: Searches for an image with the specified bookmark


name.

Exact: Searches for a bookmark that contains the exact text entered
in the Bookmark field.

Image Type: Searches for the specified image type with the
specified bookmark name.

Max Range: Searches for an image that was created the specified
number of minutes/hours before and after the specified Point in
Time. Not supported on XtremIO arrays.

Select the required Image access mode. Virtual Access without roll image in
background is preferred. However, if you need to test images for a longer period of
time or need maximum performance while testing, select Virtual Access and check
the Roll to image in background checkbox, or select Logged Access.
Note

XtremIO supports only logged image access mode, accessing the selected pointin-time is instantaneous, there are no time or space limits on image access,
replication and distribution continue during image access, and the journal is not
affected.

How to test a copy

151

Recovery

Table 33 Image access mode

Setting

Description

Logged
Access

An image access mode that physically rolls the copy storage to the
selected image (bookmark, or point in time).
Use Logged Access if you need to:
l

run many commands.

test data from large areas of the copy.

test for a longer period of time (depending on your journal size,


portion of journal size reserved for image access, and rate of
incoming writes from production).

better performance while testing (than virtual access).

perform a recovery activity using the selected image.

You should not test for longer than a few hours, as the time that you can
spend accessing the image is limited by the size of the image access log
and the size of the copy journal.
You can roughly estimate the time you can spend accessing an image in
logged access mode; if the image access log is 20% of the journal and
the journal provides a Protection Window of 48 hours, the image access
log should allow for 20% of 48 hours, or 4.8 hours of testing. However,
since there are many factors that can increase the writes to the image
access log (such as write-load, how far a snapshot is from the current
point-in-time, and so on), as a rule of thumb, you can use 10% of the
Protection Window as a very rough estimate.

Virtual
Access

When image access is enabled, there is a pause in distribution while


the system rolls to the specified image. The length of the delay
depends on how far the selected image is from the current image (the
image currently being distributed to the production storage) and your
storage capabilities.

When you disable image access any writes made to the copy volumes
while physical image access was enabled will be rolled back and
distribution will continue from the current image forward.

An image access mode that creates only the required parts of the image
you selected in a separate virtual volume, enabling virtual access to the
image almost instantly, as the system does not physically roll the copy
storage to the selected image.
Use this mode if you need quick access to a copy image.
Your testing time should be counted in minutes, and should not exceed
an hour, as the size of the image access log in virtual access mode is set
to 40 GB.

152

RecoverPoint 4.4 Administrator's Guide

Available only for VNX/VNXe splitter.

When image access is enabled, there is a pause in distribution while


the system has access to the image.

When you disable image access the virtual volume and all writes
made to it while image access was enabled are discarded.

Recovery

Table 33 Image access mode (continued)

Setting

Description
l

In the Recover Production and Failover wizards, while virtual


access is enabled, roll image in background is also enabled, and
cannot be disabled.

Roll image in An image access setting that physically rolls the copy storage to the
background selected image in the background when virtual access is selected as the
image access mode. Once the roll is complete, the system starts
accessing the volume physically (in Logged Access mode).
Select this option if you selected virtual access as your image access
mode, but also want to (when the virtual roll is complete) run many
commands, test data from large areas of the copy, fail over or recover
production from the selected image, or have better performance while
testing (than virtual access).
l

Only available when virtual access is selected.

Only available for VNX/VNXe splitter.

As logged (physical) image access is enabled, there is a pause in


distribution while the system rolls to the specified image. The length
of the delay depends on how far the selected image is from the
current image (the image currently being distributed to the
production storage) and your storage capabilities.

When you disable image access, all writes performed while image
access was enabled are discarded.

More writes are made to the journal during virtual access than during
logged access. Therefore, in virtual access with roll, the Image access
log capacity progress bar will show a higher capacity in virtual access
mode, and then a lower capacity once the system switches to logged
access mode.

The virtual volume and the physical volume have the same SCSI ID,
therefore, the switch from one to the other will be transparent to
servers and applications.

b. Click Next Test the Image.


RecoverPoint starts enabling image access to the copy volumes.
4. The Test the Image screen differs based on whether you selected a consistency group
or a group set.
a. Use the screen to guide you through the process of testing your copy images:
l

If you selected a consistency group in step 2 on page 149:

How to test a copy

153

Recovery

Dialog

Description

Activity
Diagram

Illustrates the effects of the selected recovery activity on data transfer


between the copy and the production.
Used to identify:

Activity
Pane

The state of transfer between the selected copy and the production.

The direction of replication between the selected copy and the


production.

The write activity between the storage, host, journal, and RPAs at the
copy.

The replication mode (a dashed line represents asynchronous


replication, and a dashed line with a solid line through it represents
synchronous replication).

Guides users through the process of testing the copy image.


Note

Use the Next steps (at the copy host) to guide you through the process of
testing your copy.
n

Role: The copys current role in replication (source or target).

Storage: Current state of storage access at the copy.

Image: Current image being distributed at the copy.

Transfer: The state of transfer between the production and the copy is
displayed in the middle of the link between them.

Image access status icon: Indicates when image access is enabled,


and you can start testing.

154

RecoverPoint 4.4 Administrator's Guide

The Image access log capacity progress bar indicates how long you
can access the copy image before the image access log is full and all
writes to the copy fail. On XtremIO arrays, there is no time and space
limits on image access.

Recovery

Dialog

Description
Note

More writes are made to the journal during virtual access than during
logged access. Therefore, in virtual access with roll, the Image access log
capacity progress bar will show a higher capacity in virtual access mode,
and then a lower capacity once the system switches to logged access
mode.
After image access is enabled:
n

In the Test a Copy wizard, you can Undo Writes, Enable Direct
Access and Roll to Image.

In the Recover Production and Failover wizards, you can Undo


Writes.

Note

In XtremIO, you can only Undo Writes.

If you selected a group set in step 2 on page 149, use the following table to
navigate the screen.

Dialog

Description

Activity
Table

Depicts the recovery activity-related states of all relevant copies of all relevant
groups in the group set, at the specified RPA cluster.
n

The Group column contains the names of all of the relevant groups in the
group set. Each group can be listed multiple times, once for each copy at
the target cluster.

The Copy column contains the names of all relevant copies at the target
cluster.

The Copy Role column indicates each copys current role in replication
(local copy or remote copy).

The target copy of each group. If a group contains multiple copies at the
specified RPA cluster, the first copy in the group copy list is automatically
How to test a copy

155

Recovery

Dialog

Description
selected by the system. If the first copy isn't valid for the specified activity,
the second copy in the copy list is automatically selected.
Note

You can identify which copy was selected by the system in the Storage
column, as the copy will be in a state other than No Access or N/A.
n

The time stamp of the image being accessed at the target copy is
displayed in the Image column.

The Image Access Progress column indicates when image access is


enabled, and you can start testing.

The Image Access Log Capacity column contains a progress bar


indicates how long you can access the copy image before the image
access log is full and all writes to the copy fail. On XtremIO arrays, there is
no time and space limits on image access.
Note

More writes are made to the journal during virtual access than during
logged access. Therefore, in virtual access with roll, the Image access log
capacity progress bar will show a higher capacity in virtual access mode,
and then a lower capacity once the system switches to logged access
mode.

Activity
Pane

Guides users through the process of testing their copy images of all relevant
copies in a group set.
After image access is enabled:
n

In the Test a Copy wizard, you can Undo Writes, Enable Direct Access
and Roll to Image.

In the Recover Production and Failover wizards, you can Undo Writes.
Note

In XtremIO, you can only Undo Writes.

b. Wait for image access to be enabled. If the image is being accessed in logged
access mode, wait for the system to roll to the selected image before you begin
testing.
c. At the copy host, prepare the copy volumes for testing.
Note

If you selected to access the image in virtual access mode, mount the volume
anyway. The virtual image has the same SCSI ID as the physical image.
d. Access the copy volumes, and test them as required.
e. In RecoverPoint, while testing:

156

RecoverPoint 4.4 Administrator's Guide

Recovery

Note

This section does not apply to XtremIO. When accessing an image on XtremIO, log
capacity is irrelevant, and there is no time or space limit on image access.
Take note of the Image access log capacity progress. If the image access log gets
full, writes to the copy are blocked and you will have to select a corrective action:
Action

Description

Add journal Add journal volumes to the journal, enlarging the journals capacity. Note
that any additional journal volumes should be permanent, as removing a
volumes

journal volume causes all of the snapshots in the journal to be deleted and
requires a full sweep.

Note

Adding journal volumes is not needed on XtremIO arrays, since the


journals only contain metadata and pointers to snapshots, but no
replicated data.

Undo writes Undo Writes to use the information in the image access log to
automatically undo any writes made to the copy while image access was
enabled, without disabling image access.

Enable
direct
access

Note

Direct access is not supported on XtremIO arrays. In releases prior to


RecoverPoint 4.4 and XtremIO 4.0.2, XtremIO volumes are always
accessible; therefore, take great care not to allow any access to XtremIO
volumes except when orchestrated by RecoverPoint using logged image
access.
Enable direct access to bypass the image access log and write directly to
the copy storage. direct access allows you to access the image indefinitely,
however, enabling direct access disables RecoverPoint protection, and
erases the journal, so a snapshot history is no longer available.
Note

Both the production and the copy volumes are unprotected during direct
access.
When image access is disabled, a short initialization will commence to
synchronize the copy with the production, and from that point on, your
volumes will be protected by RecoverPoint again.

Disable
image
access

Before disabling image access, stop the application and dismount the
copy host.
Click Cancel to disable image access and exit the wizard.

The following table describes all of your options during testing.


Setting

Description

Undo
Writes

Clears the image access log, removing any writes made to the copy while
image access was enabled, and allowing you to keep testing, without
disabling image access.

How to test a copy

157

Recovery

Setting

Description
Use whenever one test case is complete, and before running a new test case
that is not dependent on the outcome of the previous test case, or when
resuming distribution.

Roll to
Image

Before using this option, at the copy host, shut down all applications
accessing the group volumes, unmount all group volumes, and deport
volume groups (if using logical disk management).

After using this option, at the copy host, mount the group volumes and
import volume groups to continue testing.

Note

Not relevant for XtremIO arrays. Only available through the Test a Copy
wizard.
Rolls the copy storage to the selected image in the background when Virtual
Access is selected as the image access mode. Once the roll is complete, the
virtual volume is discarded, and the physical volume takes its place.
Note

The virtual volume and the physical volume have the same SCSI ID, and
therefore, the switch from one to the other will be transparent to servers and
applications.
Use if you selected Virtual Access as your image access mode, but also need
to run many commands, test data from large areas of the copy, test for a
longer period of time, or maximum performance while testing.

Enable
Direct
Access

158

RecoverPoint 4.4 Administrator's Guide

Only available when Virtual Access is selected.

Not available for Symmetrix or VPLEX splitters.

As logged (physical) image access is enabled, there is a delay in transfer


until the system rolls to the specified image. The length of the delay
depends on how far the selected image is from the current image (the
image currently being distributed to the production storage).

Once logged (physical) image access is enabled, hosts in the SAN will
have access to the storage volumes, and the RPA will not have access.
Distribution of images from the copy journal to the copy storage is paused.

More writes are made to the journal during virtual access than during
logged access. Therefore, in virtual access with roll, the Image Access Log
Capacity progress indicator will show a higher capacity in virtual access
mode, and then a lower capacity once the system switches to logged
access mode.

When you disable image access, the virtual volume and all writes made to
it are discarded, and any writes made to the copy volumes while logged
(physical) image access was enabled are rolled back. Distribution from the
copy journal to the copy storage continues from the current image forward.

Note

Only available through the Test a Copy wizard. Not relevant on XtremIO
arrays.

Recovery

Setting

Description
Enabling direct access to the copy image is not recommended, since it
disables RecoverPoint protection. In this mode, RecoverPoint stops imposing
a limit to the amount of data that you can write to storage because it stops
saving these writes in the image access log. That being said, direct image
access does provide better system performance when accessing the copy,
because rollback information is not being written to the image access log in
parallel with the ongoing disk I/Os.
Use if you have already physically rolled to the required image (in Logged
access mode or Virtual access with roll) and want to carry out processing that
generates a high volume of write transactions at the copy, and you don't mind
that both the production copy and the local/remote copy image will be
unprotected for as long as you are accessing the image.

Finish

After enabling direct access, you cannot roll back to an earlier image of the
accessed copy if you discover corrupted data. Moreover, in the event of a
disaster at the production during testing, you will be unable to remove any
new data that you have written to the copy unless the group contains
another non-production copy.

While direct access is enabled, the copy host has access to the storage
volumes, and the RPA does not have access, so distribution of images
from the copy journal to the copy storage is paused.

While direct access is enabled, replication stops and the transfer state for
the consistency group is Paused. The system continues to write markers to
the production journal volume, and uses these markers to efficiently
synchronize between the copy and production volumes when image
access is disabled.

When image access is disabled, a short initialization is required to


synchronize the copy volumes with the production volumes. Only after
synchronization is RecoverPoint protection restored.

Note

Disabled in the Test a Copy wizard.


To disable image access, start the specified recovery activity, exit the wizard,
and display the Manage Recovery screen.
Use when you are done testing a copy, and are ready to start the recovery
activity.
Before you click Finish, shut down all applications at the copy host accessing
the volumes of the group, unmount the volume/s from the copy host, and
deport volume groups (if using logical disk management).
After you click Finish, the system will disable image access and use the
information in the image access log to automatically undo any writes made to
the copy while image access was enabled.

After you finish


See Manage Recovery on page 173.

How to test a copy

159

Recovery

Test a Copy and Recover Production


Use Test a Copy and Recover Production to display the Recover Production wizard. The
Recover Production wizard guides you through the process of correcting file or logical
corruption, by rolling the production back to a previous point-in-time.

Note

You can also access the Recover Production wizard when Monitoring and managing
groups on page 91 and Monitoring and managing copies on page 105, or you can Initiate
recovery activities for a group set on page 77.

How to recover production


Before you begin
l

By default, you cannot recover production from a copy image during initialization. For
production recovery to succeed, there must be a complete image in the copy journal
to recover production from; therefore, production recovery is not available during
initialization, while the snapshot being distributed is larger than the capacity of the
copy journal. To allow production recovery during initialization, set the value of the
allow_long_resync parameter of the config_copy_policy CLI command to
no (default is yes). When allow_long_resync is set to no, the system pauses
transfer whenever the snapshot being transferred is larger than the capacity of the
journal, at which point you can Add or edit journal volumes on page 136. Note that
any additional journal volumes should be permanent, as removing a journal volume
causes all of the snapshots in the journal to be deleted and requires a volume sweep.

In any screen of the Recover Production wizard:


n

Click Cancel to disable image access and exit the wizard. Click Cancel only when
you are done testing and don't plan to run any further tests on the copy in the near
future. After you click Cancel, RecoverPoint will disable image access and use the
information in the image access log to automatically undo any writes made to the
copy while image access was enabled.
NOTICE

160

Click Save & Close to exit the wizard without disabling image access and display
the Manage Recovery screen without starting the recovery activity. Use Save &

RecoverPoint 4.4 Administrator's Guide

Recovery

Close if you have a test plan running and you want to exit the wizard now but keep
image access enabled and return to testing later.
l

Before performing recovery activities on a group set, note that:


n

During image access on a group set:


When selecting the latest image, the latest available image of each relevant
group copy will be accessed.
When selecting an image from the image list, the images of the first group in
the group set are displayed to the user. When the user selects an image from
the list, RecoverPoint constructs a search query containing the parameters of
the first groups image and queries the rest of the groups in the group set
according to the first groups parameters.
When selecting a specific point in time by entering advanced search options,
the same image search query is sent for all groups in the group set (not
supported if there is an XtremIO volume anywhere in the group set).

The warning The requested operation will be performed on all of the groups in the
group set excluding the groups listed above.is displayed to the user for a group in
the group set when the requested activity is illegal for one or more groups in the
group set. A recovery activity is legal for a group at a target cluster only if the
group is enabled, and the action is legal for one or more copies of the group.
When this warning is displayed, note that the requested action will not be
performed for the listed groups.

If an action that is part of a recovery activity fails for one or more of the groups in a
group set, you may want to manually perform the action for each group on which
the action failed in order to bring all of the group set's groups to the same
recovery state.

Procedure
1. In RecoverPoint, select Recovery > Test a Copy and Recover Production.
2. In the Select a Copy screen:
a. Select either a consistency group or a group set by marking the appropriate radio
button and selecting an item from the list.
l

If you selected a consistency group, select a non-production copy from the list
of copies or the copy diagram.

How to recover production

161

Recovery

While selecting a copy, you can use the diagram to identify the state of data
transfer between the production and all copies in the group. The direction of
replication and type of replication are both illustrated.
Asynchronous replication is represented by a dashed line, while synchronous
replication is represented by a dashed line on top of a solid line.
Place your icon over any copy icon to display a tooltip with the copy Role, the
Storage state, and the state of Image access.
l

If you selected a group set, select a target RPA cluster from the list. When a
group set is selected, the table under the list of target RPA clusters can be used
to identify the direction of replication and number of copies in each group.

b. Click Next Select an Image.


3. In the Select an Image screen:

a. Select the image that you want to verify.


To restore your production from a copy, you generally want the most recent copy
image that you know to be valid. For example, if you are using Microsoft Volume
Shadow Copy Service, you probably want to select the most recent shadow copy.
162

RecoverPoint 4.4 Administrator's Guide

Recovery

The shadow copies will be bookmarked with the name that you assigned to
shadow copies in the Microsoft Volume Shadow Copy Service configuration.
b. Select your preferred Image access mode on page 152. In the Recover Production
wizard, while Virtual Access is enabled, Roll image in background is also enabled
and cannot be disabled.
c. Click Next > Start Recovery. RecoverPoint starts enabling image access to the copy
volumes.
NOTICE

Ensure image access is completely enabled, and then, mount the copy volumes at
the Data Domain copy host.
4. At the copy host, prepare the copy volumes for testing.
After image access is enabled, mount the copy volumes that you want to access. If the
volumes are in a volume group managed by a logical volume manager, import the
volume group.
5. Access the copy volumes, and test them as required.
Note

While testing the copy image, undo information for any changes made to the copy is
written to the image access log, and the changes are automatically undone when
image access is disabled.
Note

If the copy is on an XtremIO array, testing is performed upon a copy of the relevant
snapshot; the snapshot itself is unaffected. When testing is disabled, the copy of the
snapshot is deleted, without affecting the original snapshot.
6. In RecoverPoint, the Start Recovery screen differs based on whether you selected a
consistency group or a group set.
a. Use the screen to guide you through the process of testing your copy image/s:
l
If you selected a consistency group, use the activity diagram for groups on page
154 to navigate the screen.

How to recover production

163

Recovery

If you selected a group set, use the activity table for group sets on page 155 to
navigate the screen.

If any of the groups in the group set are not legal for the requested operation, a
warning will be displayed, and they will not be included in the Relevant copies
at target cluster table.
If any of the groups in the group set contain multiple relevant copies at the
target cluster, you can identify which copy was automatically selected by the
system in the Storage column. As illustrated in the screenshot above, group g4
contains two copies at the target cluster (r1, and r2) and the state of Storage at
copy r1 is Logged Access while the state of Storage at copy r2 is No Access,
meaning the system has automatically selected to recover the production of
group g4 using copy r1.
NOTICE

If you click Finish (start the recovery activity) before image access is enabled,
the action will only be performed on the groups for which image access has
already been enabled.
b. Wait for image access to be enabled and the system to roll to the selected image
before you begin testing the image. You know that image access is enabled by the
Image access progress indicator. On XtremIO arrays, access to the desired
snapshot is instantaneous.
c. Optionally, use the Test the Image options on page 157 to test the copy image.
d. While testing, take note of the Image access log capacity progress indicator. If the
image access log gets full, all writes to the copy are blocked and you will have to
select Corrective action on page 157. On XtremIO arrays, there are no time or
space limits for image access.
e. Ensure the Start replication to all copies in group(s) after recovery checkbox is:

164

selected, if you want RecoverPoint to automatically start replicating the group


(or group set) as soon as the production has been recovered.

cleared, if you do not want RecoverPoint to automatically start replicating the


group (or group set) as soon as the production has been recovered, and would
rather start replication at a later time using Start transfer on page 131.

RecoverPoint 4.4 Administrator's Guide

Recovery

f. When testing is complete, start recovering production.


Click Finish to Disable image access, start recovering production from the selected
image, and display the Manage Recovery on page 173 screen.
7. In the Manage Recovery on page 173 screen, click the Show Activity Diagram link and
note that, for each accessed copy in the group or group set:
a. The direction of replication changes, and replication proceeds from the copy image
to the production, as the copy image is being applied to the production. The Image
status of the production becomes Distributing Pre-replication image and the
production Role becomes Production (being restored).
b. After the copy image is completely applied to the production:
l
The marking information in the production journal is deleted.
Note

Not relevant to XtremIO arrays.


The direction of replication is restored from the production to the copy and
normal replication is resumed. The copy Transfer status reverts back to
Active(or Replicating Snapon XtremIO arrays). The production Transfer status
reverts back to Direct access, and the production Role reverts back to
Productionas image access at the production is enabled.
8. At the production host, test the production image.
9. Mount the production volumes and import the volume groups (if any) to verify the
data.
10.At the copy host, unmount the copy volumes and deport volume groups (if any).
After you finish
See Manage Recovery on page 173.
l

Test a Copy and Fail Over


Use Test a Copy and Fail Over to display the Failover wizard. The failover wizard guides
you through the process of migrating all of your production activities to a copy. For groups
with more than one non-production copy, you can also use the failover wizard to
temporarily fail over to a copy while performing routine maintenance on the production
system, and then fail back to the production.
Note

Migration should be planned well in advance, and the image to select for migration
should be part of an overall migration strategy.

Test a Copy and Fail Over

165

Recovery

Note

You can also access the Failover wizard when Monitoring and managing groups on page
91 and Monitoring and managing copies on page 105, or you can Initiate recovery
activities for a group set on page 77.

How to fail over


Use the following procedure to migrate production activities to another copy.
Before you begin
l

If you are failing over temporarily in order to maintain the production system while
resuming production activities at a copy, go straight to step 1 on page 168. You can
only fail over temporarily if you have more than one non-production copy in the
group.

If you are failing over permanently, before you begin, gracefully shut down source-site
host activities on Windows Server platforms at the production: To ensure an up-todate image of the file system, flush all file systems that reside on the production
volumes.
NOTICE

Some applications, such as Exchange, have their own cache, which should be
flushed as well. Flushing the file system does not flush application-level data.
Close all applications that are using the consistency groups volumes at the
production site (refer to the RecoverPoint Technical Notes for the relevant
application). Place each drive volume used by the consistency group into an offline
state:

166

Using Windows GUI: by opening the Windows Disk Management console, rightclicking the RecoverPoint protected volume, and selecting the OFFLINE option.

Using Windows CLI: by opening a Command Prompt window, running the


DISKPART command, selecting a disk and selecting the OFFLINE option.

If your production or copy hosts are running on AIX operating systems or are part of a
Windows cluster server, you should perform a first-time failover on each consistency
group as soon as it is configured. See the EMC RecoverPoint Deploying Windows File
Server Technical Notes or the EMC RecoverPoint Deploying with AIX Hosts Technical Notes
for the first-time failover procedure of your specific host type.

If a consistency group contains both a local copy and a remote copy, and you start to
fail over to the remote copy while the local site does not have enough licensed
remote replication capacity for two times the capacity of the consistency group, you
will receive the error message Remote replication limit exceeded. and will have to
mitigate the situation for the failover to succeed. If this happens, consider removing
the local copy, or Install a license on page 217 for more remote replication capacity.

By default, you cannot fail over to a copy image during initialization. For failover to
succeed, there must be a complete image in the copy journal to fail over to; therefore,
failover is not available during initialization, while the snapshot being distributed is
larger than the capacity of the copy journal. To allow failover during initialization, set
the value of the allow_long_resync parameter of the config_copy_policy
CLI command to no (default is yes). When allow_long_resync is set to no, the
system pauses transfer whenever the snapshot being transferred is larger than the
capacity of the journal, at which point you can Add or edit journal volumes on page
136 (not relevant in XtremIO). Note that any additional journal volumes should be

RecoverPoint 4.4 Administrator's Guide

Recovery

permanent, as removing a journal volume causes all of the snapshots in the journal
to be deleted and requires a volume sweep.
l

In XtremIO, after failover, if either the production or the copy is on an XtremIO array,
the snap-based replication mode after failover may require user intervention before
failover can proceed. Refer to the following table for guidance.
Table 34 XtremIO snap-based replication mitigation after failover

Production
array

Copy array

Replication
mode
before failover

Replication mode
after failover

XtremIO

XtremIO

Continuous or
Periodic

Same as before
failover.

XtremIO

VNX

Continuous

An error will occur.

Periodic

Remains Periodic

Mitigation plan

Manually set the


snap-based
replication mode to
Periodic.

XtremIO

VPLEX,
Symmetrix,
VNXe, VNX

No snap-based
replication
configured

Asynchronous is
Enabled, RTO is25
seconds.

VNX

XtremIO

On Highload

An error will occur.

VNX

XtremIO

Periodic

Remains Periodic.

VNX

XtremIO

No snap-based
replication
configured

An error will occur.

Manually set the


snap-based
replication mode to
Periodic.

VPLEX,
Symmetrix,
VNXe

XtremIO

An error will occur.

Manually set the


snap-based
replication mode to
Periodic.

Manually set the


snap-based
replication mode to
Periodic.

Before failing over a group set, note that:


n

During image access on a group set:


When selecting the latest image, the latest available image of each relevant
group copy will be accessed.
When selecting an image from the image list, the images of the first group in
the group set are displayed to the user. When the user selects an image from
the list, RecoverPoint constructs a search query containing the parameters of
the first groups image and queries the rest of the groups in the group set
according to the first groups parameters.
When selecting a specific point in time by entering advanced search options,
the same image search query is sent for all groups in the group set (search
options are not supported for selecting XtremIO snapshots).

How to fail over

167

Recovery

The warning The requested operation will be performed on all of the groups in the
group set excluding the groups listed above.is displayed to the user for a group in
the group set when:
The requested activity is illegal for one or more consistency groups in the
group set. A recovery activity is legal for a group at a target cluster only if the
group is enabled, and the action is legal for one or more copies of the group.
When this warning is displayed, note that the requested action will not be
performed for the listed groups.
All of the links between the new production and all target copies in the group
have not been manually defined before failover. After failover, the option to set
a copy on the target RPA cluster as the new production will only be available
for groups if all links from the new production to all target copies have already
been defined.

While parallel bookmarking is enabled for a group set, changing the direction of
replication will temporarily disable the creation of parallel bookmarks for all
groups that are part of the group set. Parallel bookmarks will not be created while
groups in the group set are replicating in different directions.

If an action that is part of a recovery activity fails for one or more of the groups in a
group set, you may want to manually perform the action for each group on which
the action failed in order to bring all of the group set's groups to the same
recovery state.

Procedure
1. Select Recovery > Test a Copy and Fail Over.
2. In the Select a Copy screen:
a. Select either a consistency group or a group set, marking the appropriate radio
button and selecting an item from the list.
l

If you selected a consistency group, select a non-production copy from the list
of copies or the copy diagram.

While selecting a copy, you can use the diagram to identify the state of data
transfer between the production and all copies in the group. The direction of
replication and type of replication are both illustrated. Asynchronous
replication is represented by a dashed line, while synchronous replication is
represented by a dashed line on top of a solid line. Place your icon over any
168

RecoverPoint 4.4 Administrator's Guide

Recovery

copy icon to display a tooltip with the copy Role, the Storage state, and the
state of Image access.
l

If you selected a group set, select a target RPA cluster from the list. When a
group set is selected, the table under the list of target RPA clusters can be used
to identify the direction of replication and number of copies in each group.

b. Click Next Select an Image.


3. In the Select an Image screen on page 150:

a. Select an image to fail over to.


To permanently fail over to a copy, you generally want the most recent copy image
that you know to be valid, that was created after placing all consistency group
drives into an offline state at the production.
If you are using Microsoft Volume Shadow Copy Service, you may want to select
the most recent shadow copy. The shadow copies will be bookmarked with the
name that you assigned to shadow copies in the Microsoft Volume Shadow Copy
Service configuration.
How to fail over

169

Recovery

b. Select your preferred image access mode on page 152. In the Failover wizard,
while Virtual Access is enabled, Roll image in background is also enabled and
cannot be disabled.
c. Click Next Start > Failover.
RecoverPoint starts enabling image access to the relevant copy volumes.
4. In RecoverPoint, the Start Failover screen differs based on whether you selected a
consistency group or a group set.
a. Use the screen to guide you through the process of testing your copy image(s):
l

If you selected a consistency group in step 2 on page 168, use the activity
diagram for groups on page 154 to navigate the screen.

If you selected a group set in step 2 on page 161, use the activity pane for
group sets on page 155 to navigate the screen.

If any of the groups or copies in the group set are not legal for the requested
operation, they will not be displayed in the Relevant copies at target cluster
table.
170

RecoverPoint 4.4 Administrator's Guide

Recovery

If any of the groups in the group set contain multiple relevant copies at the
target cluster, you can identify which copy was automatically selected by the
system in the Storage column. As illustrated in the screenshot above, group g4
contains two copies at the target cluster (r1, and r2) and the state of Storage at
copy r1 is Logged Access while the state of Storage at copy r2 is No Access,
meaning the system has automatically selected to fail over group g4 to copy r1.
b. Wait for image access to be enabled and the system to roll to the selected image
to begin testing. You know that image access is enabled by the Image access
progress indicator. On XtremIO arrays, image access is nearly instantaneous.
If you click Finish (to start the recovery activity) before image access is enabled,
the action will only be performed on the groups for which image access has
already been enabled.
c. If you are permanently failing over; resume production disk activities by placing
each production drive volume used by the consistency group into an online state:
Note

If you are only failing over temporarily, and planning to fail back as soon as the
production is restored, skip this step.
l

Using Windows GUI: by opening the Windows Disk Management console, rightclicking the RecoverPoint protected volume, and selecting the ONLINE option.

Using Windows CLI: by opening a Command Prompt window, running the


DISKPART command, selecting a disk and selecting the ONLINE option.

d. At the copy host, prepare the volumes for testing: After image access is enabled,
mount the copy volumes that you want to access and import the volume groups (if
any).
e. Access the copy volumes, and test them as required. While testing the copy image,
undo information for any changes made to the copy is written to the image access
log, and the changes are automatically undone when image access is disabled.
f. In RecoverPoint, use the Test the Image options on page 157 to test the copy
image.
g. Ensure the Start replication to all copies in group(s) after recovery checkbox is:
l

selected, if you want RecoverPoint to automatically start replicating the group


(or group set) as soon as the failover is complete.

cleared, if you do not want RecoverPoint to automatically start replicating the


group (or group set) as soon as the failover is complete, and would rather start
replication at a later time using Start transfer on page 131.

h. While testing; take note of the Image access log capacity progress indicator. If the
image access log gets full, all writes to the copy are blocked and you will have to
select a corrective action on page 157.
5. When testing is complete:
Note

If the group only contains one non-production copy, unmount the copy volumes and
deport any volume groups now.

How to fail over

171

Recovery

Click Finish to disable image access, start failing over to the selected image, and
display the Manage Recovery on page 173 screen.
Note

If your consistency group contains both a local copy and a remote copy, and you
selected to fail over to the remote copy while the local site does not have enough
licensed remote replication capacity for two times the capacity of the consistency
group, you will receive the message Remote replication limit exceeded. and will have
to mitigate the situation for the failover to succeed. Consider removing the local copy
or Install a license on page 217 for more remote replication capacity.
6. In the Manage Recovery on page 173, click the Show Activity Diagram link and note
that the direction of replication changes as the selected copy image is applied to the
production. The Image status of the new production becomes Distributing Prereplication image and the new copy Role becomes Failing Over. During image access,
the Back to Wizard on page 179 link can be used to return to the wizard until the
selected image is completely applied to the production.
After failover:
l

If there is only one non-production copy in the group:


a. The marking information in the production journal is deleted, and the copy
journal is deleted.
The group undergoes a full sweep.
b. Normal replication is resumed. The direction of replication is restored from the
production to the copy. The copy Transfer status reverts back to Active. The
production Transfer status reverts back to Direct access, and the production
Role reverts back to Production.

If there is more than one non-production copy in the group, you can repair the
production now. Your applications and business operations continue at the copy.
The production journal and the production storage (assuming they are online) are
kept up to date from the copy. The Role of the production becomes Target at
Productionand the Role of the selected copy becomes Local Source if a local copy
was selected, or Remote Source if a remote copy was selected.

After you finish


When you are done repairing the production, use the Manage Recovery on page 173
screen to:

172

fail back to production on page 178

set your copy (or copies) as the new production on page 176

RecoverPoint 4.4 Administrator's Guide

Recovery

Manage Recovery
Use Manage Recovery to monitor and manage all of the recovery activities currently
occurring in the system. All of the accessed copies in the system are displayed in the
Manage Recovery screen.

In the Manage Recovery screen, the options that you have and the information that is
displayed in the recovery bars depends on the specific recovery activity, and whether you
have selected to perform the activity on a consistency group or a group set.
Figure 17 Recovery bar for consistency groups

All of the recovery bars can be used to identify the:


l

The Image Access status

Recovery Activity

Relevant Copy

State of Transfer between the copy and production

In the recovery bar for consistency groups:


l

Use the Progress bar to identify:


n

How full the image access log is getting.


Manage Recovery

173

Recovery

n
l

During production recovery, the activity progress.

Click the Show Activity Diagram link to monitor the activity progress.

Figure 18 Recovery bar for group sets

In the recovery bar for group sets on page 173, click the Show Copy Activity link to
expand the group set and display a recovery bar for consistency groups on page 173 for
each relevant copy in the group set.
NOTICE

If an action that is part of a recovery activity fails for one or more of the groups in a group
set, you will have to manually perform the action for each group on which the action
failed in order to bring all of the group set's groups to the same recovery state.
l

Use the Progress bar to identify:


n

How full the image access log is getting.

During production recovery, the activity progress.

Click the Show Activity Diagram link of a group to monitor the activity progress.

Testing a copy
When the Activity type is Testing a copy, the following options are available in the Select
Next Action combo box:

174

Continue Testing

Finish Testing

RecoverPoint 4.4 Administrator's Guide

Recovery

Continue Testing
To open the Test a Copy wizard to the Test the Image screen and continue testing using
the Activity Diagram for Groups on page 154 or the Activity Table for Group Sets on page
155, and the Test the Image options on page 157.

Finish Testing
To stop testing and Disable image access to the relevant group or group set copies.

Failing over
When the Activity type is Failing Over, the following options are available in the Select
Next Action combo box:
NOTICE

If you are failing over a group set, you can select Back to Wizard on page 176, Set Copy/
Copies as Production on page 176, or Fail Back to Production on page 178 in the
Recovery bar for group sets to apply the action to all relevant copies at the target RPA
cluster.

Failing over

175

Recovery

Back to Wizard
Opens the Fail Over wizard to the Start Failover screen.
Note

This option is displayed until you manually click Back to Wizard and in the Failover
wizard, click Finish.

Set Copy/Copies as Production


Permanently fails the production over to the accessed copy (or copies, if you are failing
over a group set).
Note

This option is only relevant for groups (or group sets containing groups) with more than
one non-production copy.
Use the following procedure to permanently fail over to the accessed copy/copies.
Procedure
1. At the copy host, close any applications accessing the copy, unmount the copy
volumes and deport volume groups (if any) now.
2. In RecoverPoint, select Set Copy/Copies as Production:
l

If the accessed copy is the only non-production copy in the group, in the Activity >
Diagram, the Role of the Target at Productionbecomes Production. The Role of the
Local Sourcebecomes Local Copy.
There is a short initialization between the new production and the former
production. All local and remote copies that are not the new production or former
production undergo a full sweep.

If you are failing over a group and the accessed copy is not the only nonproduction copy in the group, the Set Production dialog box is displayed,
prompting you to decide what you want to do with any target copies of the new
production, as a consequence of failover.
If you are failing over a group set containing groups with more than one nonproduction copy, the Set Production dialog box is not displayed and before setting
a target copy as the new production, you will need to Modify a links protection
policy on page 115 for all of the links between the new production and all of the
new target copies.

176

RecoverPoint 4.4 Administrator's Guide

Recovery

Setting

Description

Configure a new link To define a new link between the two copies, and set the
between <CopyName> link policy. When selected, one of the replication modes
and <CopyName>
must be defined.
Note

Only one synchronous remote link and one synchronous


local link are supported per consistency group.
Usage
If Replication Mode is set to asynchronous,the RPO must
be defined.
Disable Copy
<CopyName>

To disable the copy.

Remove Copy
<CopyName>

To delete the copy data from the storage, and the


RecoverPoint settings.

Enter the required information and click OK.


The journal of the old remote copy (that is, the new production) is deleted.
Replication starts from the remote copy to the old production, updating the old
production data with the selected copy image.
In the Activity Diagram, the Role of the Target at Productionbecomes Production. The
Role of the Remote Sourcebecomes Remote Copy.
There is a short initialization between the new production and the former
production. All local and remote copies that are not the new production or former
production undergo a full sweep.
3. At the production host, test the production image: Mount the production volumes to
verify the data, and import the volume group (if any).
4. At the copy host, unmount the copy volumes and deport volume groups (if any).

Failing over

177

Recovery

Fail Back to Production


When repairs at the production site are complete, fails back to the production after failing
over to a copy (or copies, if you failed over a group set) to resume production activities at
the original production site.
Before you begin
Only relevant for groups (or group sets containing groups) with more than one nonproduction copy
Procedure
1. Select Fail Back to Production.
2. In the Select an Image screen, select an image that pre-dates any corruption.
3. Click Finish.
a. The production journal is erased. If you have more than one non-production copy,
transfer to the other copies is automatically resumed.
b. Normal replication is resumed. The direction of replication is restored from the
production to the copy.
c. The production Transfer status reverts back to Direct access, and the production
Role reverts back to Productionas image access at the production is enabled.
4. At the copy host, unmount the copy volumes and deport volume groups (if any).

Recovering production
When the Activity type is Recovering Production, only Back to Wizard is available.
NOTICE

If you are recovering production for a group set, you can select Back to Wizard in the
recovery bar for group sets to apply the action to all relevant copies at the target RPA
cluster.

178

RecoverPoint 4.4 Administrator's Guide

Recovery

Back to Wizard
Opens the Recover Production wizard to the Start Recovery screen so you can continue
testing using the Activity Diagram for Groups on page 154 or the Activity Pane for Group
Sets on page 155, and the Test the Image options on page 157.

Recovering production

179

Recovery

180

RecoverPoint 4.4 Administrator's Guide

CHAPTER 5
RPA Clusters

This section describes the RPA Clusters menu and how to use it to manage and monitor
replication in RecoverPoint.
l
l
l
l
l
l
l

RecoverPoint clusters.......................................................................................... 182


General Cluster Info.............................................................................................183
Storage............................................................................................................... 185
Splitters.............................................................................................................. 197
RPAs................................................................................................................... 204
vCenter Servers................................................................................................... 211
Phoenix Systems.................................................................................................214

RPA Clusters

181

RPA Clusters

RecoverPoint clusters
Monitoring or administration of components in a RecoverPoint system that are managed
by RPA clusters is performed through the RPA Clusters interface. The information for each
RPA cluster is displayed in a separate tab of the interface.

Table 35 RPA cluster options

Option

When to use

General
Cluster Info

To display the specified RPA cluster graphically as part of a site map and view
general cluster information, or to display any alerts that may have been logged
for any of the physical components of your RecoverPoint system, at the selected
RPA cluster.

Storage

To register arrays, vCenter Servers, and resource pools and manage their
registration in the RecoverPoint system.

Splitters

Note

Not relevant in ProtectPoint


To display the splitter table, add/remove splitters, or attach/detach volumes
from splitters at the selected RPA cluster.

RPAs
vCenter
Servers

To display the RPA table containing all RPAs in the selected RPA cluster, and the
status and properties of each.
Note

Not relevant in ProtectPoint


To monitor, register, or remove vCenter Servers.

Pheonix
Systems

Note

Not relevant in ProtectPoint


To use, monitor or disable Phoenix System RP Protection.

182

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

General Cluster Info


Use the RPA Clusters > RPA-Cluster-Name tab > General Cluster Info link to graphically
display an overview of the current state of individual components in your RecoverPoint
system.
Figure 19 RPA Clusters: General Cluster Info

The General Cluster Info section contains a Cluster Diagram and a Cluster Details section
that allow you to easily monitor and analyze the current state of the RPA cluster. The
cluster diagram contains all of the major components of the RecoverPoint system (host,
storage, and RPAs) and the connectivity between them and the RPA cluster. An icon is
displayed on any system component for which system Alerts on page 42 have been
registered.

The following information is displayed in the details section below the cluster diagram:

General Cluster Info

183

RPA Clusters

Table 36 RPA cluster information

Information

Description

RecoverPoint
Version

The version of the RecoverPoint software that is currently installed on the RPA
cluster.
If the system is in maintenance mode, the RecoverPoint version will display N/A
System is in maintenance mode.

Internal RPA
Cluster Name

The internal system name given to RPA clusters, and used in upgrade scenarios.

Management
IPv4

The IP address assigned to the LAN interface of the RPA cluster in order to define
the management interface network, in IPv4 format.
Note

Use this IP to connect to the RPA in the RPA cluster that is running the cluster
control, in IPv4 format.
Management
IPv6

The IP address assigned to the LAN interface of the RPA cluster in order to define
the management interface network, in IPv6 format.
Note

Use this IP to connect to the RPA in the RPA cluster that is running the cluster
control, in IPv6 format.
Software
Serial ID

One of the system parameters sent to SyR on page 265.

Number of
RPAs

The number of RPAs in the RPA cluster.

RPA Type

The type of the RPA cluster. Valid values are Physical or Virtual.

Maintenance
Mode

Unisphere for RecoverPoint will enter maintenance mode:


l

During minor version upgrade.

During major version upgrade.

When replacing an RPA in an existing cluster.

When adding new RPAs to existing clusters.

When converting an environment to a RecoverPoint/SE or RecoverPoint/CL


configuration.

When entering maintenance mode through CLI.

If an external system causes the system to enter maintenance mode.


When in this mode, Unisphere for RecoverPoint can only monitor the system;
user-initiated capabilities are disabled. Once the system exits this mode,
Unisphere for RecoverPoint will return to standard managing and monitoring
function.
Whether or not the system is in maintenance mode. If so, the specific
maintenance activity that is currently being performed is displayed.
RecoverPoint system maintenance is performed using the Deployment
Manager. See the EMC RecoverPoint Installation and Deployment Guide for
more information on maintenance mode.
Valid values are:

184

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

Table 36 RPA cluster information (continued)

Information

Time Zone

Description
l

None (when the system is not in maintenance mode)

Minor Upgrade

Major Upgrade

Hardware Replacement

RPA Cluster Upgrade

Repository Change

RP/SE Conversion

System Modification

User-Initiated Maintenance Mode

RPA Cluster Connection

Unknown

The time zone used by the RPA cluster.

Storage
Use the RPA Clusters > RPA-Cluster-Name tab > Storage link to enable RecoverPoint
storage awareness by registering new arrays, vCenter Servers and resource pools or
manage existing registered arrays, vCenter Servers, and resource pools.
Note

In RecoverPoint, Data Domain vDisk pools are referred to as resource pools

Figure 20 RPA Clusters: Storage

Use the Storage link to:


Storage

185

RPA Clusters

Figure 20 RPA Clusters: Storage (continued)


l

Display registered storage on page 186

Register storage on page 187

Unregister storage on page 192

Edit storage registration details on page 193

Display registered resource pools on page 194

Register available resource pools on page 195

Unregister resource pools on page 196

Display registered storage


Use the following procedure to display all of the arrays and vCenter servers that are
currently registered at an RPA cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Storage.

In the right pane, the Registered Storage table is displayed. The Registered Storage
table contains the list of all of the storage currently registered at the selected RPA
cluster and the following storage-specific information is displayed for each:
Table 37 Storage information

Information

Description

Name

The name of the registered storage in the RecoverPoint system.

Serial Number The name of the splitter in the RecoverPoint system.


Type

186

RecoverPoint 4.4 Administrator's Guide

The type of array or vCenter Server. Valid storage types are:


l

CLARiiON

ScaleIO

RPA Clusters

Table 37 Storage information (continued)

Information

Description
l

VNX

VPLEX

vCenter Server

XtremIO

Data Domain

IP

The IP address of the array or vCenter Server management interface. In the case
of ScaleIO storage, it may be a list of IP addresses.

Version

The version of the operating system running on the storage.

Splitter

The name of the splitter in the RecoverPoint system.


Note

Only available in VNX and CLARiiON.


Connectivity
Status

Whether or not there is connectivity between the array or vCenter Server and the
RPA cluster.

Register storage
Before you begin
Note

This procedure is not relevant to VNXe, which registers itself with RecoverPoint
automatically.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Storage.
Note

You can also register vCenter Servers by selecting RPA Clusters > <RPA-Cluster-Name>
tab > vCenter Servers .

Register storage

187

RPA Clusters

In the right pane, the Registered Storage table is displayed. The Registered Storage
table contains the list of all of the Data Domains, arrays and vCenter Servers that are
currently registered at the selected RPA cluster and their storage information on page
186.
3. In the right pane, in the Registered Storage tab, click Add. The Register Storage
dialog box is displayed.
4. In the Register Storage dialog box, you can:
l

Register an array that has already been masked to the RPA cluster

a. Select Register an array seen by this RPA cluster


b. Select an array from the list
c. Define storage login credentials on page 189
l

Register any storage


To register any Data Domain, XtremIO, VPLEX, ScaleIO, CLARiiON, VNX or vCenter
Server, regardless of whether or not it is masked to the RPA cluster:

188

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

a. Select Register any storage of type.


b. Select the storage type from the drop-down.
c. Define storage login credentials on page 189

Define storage login credentials


Use the following procedure to define the login credentials for storage in RecoverPoint.
l

For VNX/CLARiiON storage, all of the fields are mandatory. Enter the IP addresses of
SP-A and SP-B, the username and password of your storage and an authentication
scope.

For VPLEX storage, the storage IP address, username and password are mandatory.
Alternatively, retrieve a certificate from system or manually select it.

Define storage login credentials

189

RPA Clusters

For ScaleIO storage, the ScaleIO MDM credentials and IP1 address are mandatory.

For Data Domain, enter the Data Domain IP or hostname, the SU-vDisk user and
password, and enter the SU name.
Note

Requires additional configuration on the Data Domain (addition of the RPAs as hosts
and FC connectivity). By default, the Select FC for data transfer checkbox is enabled.
If your Data Domain doesn't have Fibre Channel connectivity and you want to connect
it using IP, uncheck this checkbox.

190

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

For XtremIO storage, the IP address is of the XtremIO Management Server (XMS).
Select the PSNT (Product Serial Number) of the desired XtremIO cluster (not required
if the array is already exposed to RecoverPoint and listed here).

If the XtremIO Management Server manages more than one cluster, register each
cluster separately, always using the IP address of the XtremIO Management Server.
Then enter the XtremIO rp_user as the User; and the rp_user password as the
Password. The Password can be changed in the XtremIO user interface. If the XtremIO
Management Server IP or rp_user changes, use this screen to change the registration
in RecoverPoint.
l

For vCenter Server storage, all of the fields are mandatory and the default TCP port is
443. Best practice is to configure the vCenter Server to require the use of a certificate
because once RecoverPoint has read the certificate, it does not need further access to
the location. The default certificate locations are:
Define storage login credentials

191

RPA Clusters

Windows 2003 Server:


C:\Documents and Settings\All Users\Application Data
\VMware\VMware VirtualCenter\SSL\rui.crt.

Windows 2008 Server:


C:\Users\All Users\Application Data\VMware\VMware
VirtualCenter\SSL\rui.crt.
For more information about the location of the security certificate, refer to
Replacing vCenter Server Certificates VMware vSphere 4.0, available at
www.vmware.com.

Unregister storage
Use the following procedure to unregister storage from an RPA cluster.
Note

You can also unregister vCenter Servers by selecting RPA Clusters > RPA-Cluster-Name >
vCenters Server.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Storage.

In the right pane, the Registered Storage tab is displayed. The Registered Storage tab
contains the list of all of the storage currently registered at the selected RPA cluster,
and its storage information on page 186.
3. In the right pane:

192

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

a. Select your storage in the Registered Storage tab.


b. Click Remove, and click Yes in the confirmation dialog box.

Edit storage registration details


Use the following procedure to edit the registration details of a Data Domain, an array, or
a vCenter Server at an RPA cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Storage.
Note

You can also edit the registration details of a vCenter Servers from the vCenter
Servers on page 211 link.
In the right pane, the Registered Storage table is displayed. The Registered Storage
table contains the list of all of the arrays and vCenter Servers that are currently
registered at the selected RPA cluster and their storage information on page 186.

3. In the right pane:


a. Select storage in the Registered Storage table.
b. Click Edit. The Edit Storage Registration dialog box is displayed.

Edit storage registration details

193

RPA Clusters

c. In the Edit Storage Registration dialog box:


If required, modify the storage name.
a. If required, Define storage login credentials on page 189
b. Click OK.

Display registered resource pools


Use the following procedure to display all of the resource pools that have been registered
at an RPA cluster.
Note

This procedure is not relevant for XtremIO arrays.


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Storage.

In the right pane, the Registered Storage table is displayed.


3. In the Registered Storage table, select an array or vCenter Server.
The Registered Resource Pools table is displayed under the Registered Storage tab.
The Registered Resource Pools table contains the list of all of the resource pools of
the selected storage, that are currently registered at the selected RPA cluster. The
following storage-specific information is displayed for each registered resource pool:
Table 38 Resource pool storage information

194

Information

Description

Name

The name of the registered resource pool in the RecoverPoint system.

Type

The resource pool type.


Valid resource pool types are:

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

Table 38 Resource pool storage information (continued)

Information

Description
l

RAID groups

Storage pools

Data stores

vDisk pools

Total Size

The total capacity of the resource pool.

Available
Size

The total capacity of the resource pool minus the capacity of the volumes that
were already allocated in the resource pool.

Tiers

The physical type of the storage. For example, Fibre Channel, SATA, etc.

Available
Tiering
Policies

The tiering policies that are available on storage and can be automatically
applied to volumes in RecoverPoint when users automatically provision volumes.

Register available resource pools


Use the following procedure to register resource pools of an array, Data Domain or
vCenter Server at an RPA cluster.
Before you begin
l

Pre-configure resource pools on the Data Domain in the same user storage unit and
vDisk pool.

After registering a vDisk pool, only a manual change on the Data Domain will be able
to unregister them and make them available for other clusters or uses.

Procedure
1. Select RPA Clusters.
2. Select the tab of the relevant RPA cluster.
3. In the left pane, select Storage.
In the right pane, the Registered Storage tab is displayed. The Registered Storage tab
contains the list of all of the Data Domains, arrays and vCenter Servers that are
currently registered at the selected RPA cluster and their storage information on page
186.

Register available resource pools

195

RPA Clusters

4. In the Registered Storage tab, select an array or vCenter Server.


The Registered Resource Pools table is displayed.
5. Under the Registered Resource Pools table, click Add.
The Register Available Resource Pools dialog box is displayed.

6. In the Register Available Resource Pools dialog box:


a. Select the resource pool that you want to register.
b. Click OK.

Unregister resource pools


Use the following procedure to unregister resource pools of an array or vCenter Server at
an RPA cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Storage.
196

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

In the right pane, the Registered Storage table is displayed. The Registered Storage
table contains the list of all of the arrays and vCenter Servers that are currently
registered at the selected RPA cluster and their storage information on page 186.

3. In the Registered Storage tab, select the array or vCenter Server whose resource pools
you want to unregister.
The Registered Resource Pools table is displayed in the Registered Storage tab. The
Registered Resource Pools table contains the list of all of the resource pools of the
selected array or vCenter Server, that are currently registered at the selected RPA
cluster and their storage information on page 194.
4. In the Registered Resource Pools table:
a. Select the resource pool you want to unregister.
b. Click Remove, and click Yes in the confirmation dialog box.

Splitters
Note

This section is not relevant in XtremIO, which is a splitter-less solution.


Select RPA Clusters > RPA-Cluster-Name > Splitters to display all of the splitters that have
been added to RecoverPoint, add and remove splitters, and attach or detach volumes
from splitters.

Splitters

197

RPA Clusters

Figure 21 RPA Clusters: Splitters

In RecoverPoint, all installed splitters are automatically added to the system, and all
volumes that are masked and zoned to the RPA are automatically attached to all existing
splitters at a site. In the rare case that a splitter is added to the system after the system is
already installed, you will need to add the splitter to the system, and attach the relevant
volumes to the splitter. Multiple splitters can be attached to one RPA cluster. See the EMC
RecoverPoint Release Notes for the limitation on the maximum number of splitters that can
be attached to a cluster.
If there are splitters that have been installed on the storage but have not been added to
the system or volumes that have not yet been attached to any splitters, an alert is
displayed in the:
l

RPA Clusters Map on page 40

Alerts on page 42

General Cluster Info on page 183

If an alert is displayed, select the Splitters link in the left pane to:
l

Display splitter information on page 198

Display volume information on page 199

Add splitters

Remove splitters on page 202

Attach volumes to the splitter on page 202

Detach volumes from the splitter on page 203

Display splitter information


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Splitters.

198

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

Results

The Splitters table is displayed in the right pane, with the following splitter-specific
information:
Table 39 Splitter information

Information

Description

Status

Whether the splitter is up or down.

Name

The name of the splitter in the RecoverPoint system.

Type

The type of storage that the splitter is installed on.

RPA Link

Whether or not there is connectivity between the RPA and the splitter.

Attached RPA
Clusters

The number of RPA clusters attached to the splitter.

When a VPLEX splitter is selected in the Splitters table, the Directors are displayed above
the Volumes Attached to Selected Splitter section.

Display volume information


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Splitters. The Splitter table is displayed in the right pane.
3. In the right pane; select a splitter in the Splitter table.
The Volumes Attached to Selected Splitter section displays the volumes attached to
the selected splitter, and the following volume-specific information:

Display volume information

199

RPA Clusters

Table 40 Volume information

Information

Description

Group

The name of the group that the volume belongs to.

Copy

The name of the copy that the volume belongs to.

Replication
Set

The name of the replication set that the volume belongs to.

Access

The data access state of the volume, with regards to the splitter. Once a volume
is configured in RecoverPoint, including attachment to a splitter, the RPA can
place the volume in any of the displayed states.

Also
Replicated by
SRDF

Whether or not the volumes is also replicated by the Symmetrix replication


technology, SRDF.

Add splitters
Use the following procedure to connect your RecoverPoint write-splitters to RecoverPoint.
Before you begin

200

Not relevant for XtremIO, which is a splitter-less solution.

If splitters have been installed on your arrays before installing RecoverPoint, they will
automatically be added to the RecoverPoint system, see splitter installation in the
EMC RecoverPoint Installation and Deployment Guide and the RecoverPoint technical
notes for your specific splitter type. However, if a storage array was added to the
system after deployment, the splitter will need to be added to the system.

All splitters that are masked and zoned to the RPAs are automatically added to the
system, unless they are already attached to another RPA cluster. VPLEX, VNX and
Symmetrix splitters are automatically added to the system regardless of masking and
zoning, unless they are already attached to the maximum allowed number of RPA
clusters. See the EMC RecoverPoint Release Notes for the maximum number of RPA
clusters that can be attached to a splitter.

Splitters residing on arrays that are not licensed are not automatically added to the
system.

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

Splitters removed by the user will not be re-added automatically by the system.

ScaleIO splitters are added to the system after the automatic SAN rescan cycle is
complete. If the ScaleIO splitter is not visible in the system, manually run the
rescan_san CLI command.

Volumes added to replication sets are automatically attached to all splitters.

For VNX splitters: Although the two storage processors are listed as separate entities
(Splitter 1-A and Splitter 1-B), they are managed as a single entity. If you add or
remove a splitter, the second storage processor instance is automatically added or
removed.

If a production storage volume is rolled back by a VNX SnapView session, the VNX
splitter will automatically initialize a full sweep of any of the volumes attached to
the splitter.

A single VNX splitter can be shared by up to four 4 RPA clusters. While attaching a
VNX splitter to a fifth RPA cluster appears to succeed, the splitter is in an error
state for the newly attached RPA cluster. All splitter operations for this RPA cluster
fail and return the error: Maximum RPA clusters per splitter exceeded. In this case,
remove the VNX splitter from the fifth RPA cluster.

For VNXe splitters:


n

The VNXe splitter is represented as a single entity.

The VNXe splitter has a designated Snapshot Mount Point for every LUN. No full
sweep occurs when adding or rolling back array snaps.

The VNXe splitter does not support multiple clusters.

Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane, click Add.

Note

The Add button is only enabled when there are splitters installed on the storage and
ready to add to the system.
4. In the Add Splitter dialog box; select the splitter that you want to add, and click OK.
If the selected splitter is a VNX/VNXe/CLARiiON splitter, both SP-A and SP-B are added
when one of them is selected.

Add splitters

201

RPA Clusters

After you finish


Attach volumes to the splitter on page 202

Remove splitters
Note

You must Detach volumes from the splitter on page 203 before you can remove it from
the system. Only splitters that are not attached to any volumes can be removed.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane; select one or more splitters in the Splitters table, click Remove, and
click Yes in the confirmation dialog box. If the selected splitter is a VNX/CLARiiON
splitter, both SP-A and SP-B are removed when one of them is selected.

Attach volumes to the splitter


Before attaching volumes to splitters, note the following:
l

RecoverPoint automatically detects all paths from the splitter to the volume. If no
path exists between the splitter and the volume, you cannot attach the volume to that
splitter.

You cannot replicate a volume until it is attached to a splitter. You cannot fail over to
a local or a remote copy of a volume until it is attached to a splitter.

For VNX splitters:


n

Although the two storage processors are listed as separate entities (Splitter 1-A
and Splitter 1-B), they are managed as a single entity. If you attach or detach a
volume at one instance, the same volume will automatically be attached or
detached at the other storage processor instance.

If you attach a volume to an RPA cluster, and that volume is already attached to a
different RPA cluster that shares the same VNX splitter, the volume appears to
attach successfully to the second RPA cluster, but then faults to the Attached to
other RPA cluster/s error state. The volume cannot be used by the RPA cluster to
which it was just attached. To correct this error, click Detach to detach the volume
from the RPA cluster.
Note

To avoid this problem, a volume should be masked to a single RPA cluster. A


volume that is masked for one RPA cluster should not be masked for another RPA
cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane; select a splitter in the Splitters table, and click Attach in the
Volumes Attached to Selected Splitter section. The volumes that can be attached to
the splitter are displayed. Select one or more volumes to attach to the splitter. Click
OK to attach to selected volume to the splitter.

202

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

NOTICE

When you attach a volume to a splitter, RecoverPoint ensures consistency between


the copy and the production source by performing a full sweep synchronization of the
volume. Checking the Attach as Clean checkbox overrides the default synchronization
process by informing the system that the copy volume is known to be an exact image
of its corresponding production volume. If Attach as Clean is checked, and the copy
volume being attached is inconsistent with its corresponding production volume, it
will remain inconsistent. To ensure consistency, best practice is not to select this
checkbox and use the default RecoverPoint synchronization process.

Detach volumes from the splitter


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane; select a splitter in the Splitters table, select a volume in the
Volumes Attached to Selected Splitter section, click Detach, and click Yes in the
confirmation dialog box.

Detach volumes from the splitter

203

RPA Clusters

RPAs
Select RPA Clusters > RPA-Cluster-Name > RPAs to display all of the RPAs in the
RecoverPoint system and the status and properties of each RPA.

Figure 22 RPA Clusters > RPA-Cluster-Name tab > RPAs

The RPA table displays the following information for each RPA:
Table 41 RPA Clusters: RPA table

Column

Description

Status

Indicates if the RecoverPoint appliance, LAN interface card, and WAN interface
card are alive.

ID

The ID of the RPA. Each RPA in an RPA cluster has an RPA ID that starts with the
string RPA and ends with a number from 1-8. Only RPA1 and RPA2 can run the
cluster control.

WAN IP

The WAN IP address of the RPA.

Management
IPv4

The LAN IP address of the RPA, in IPv4 format. Use this address to open a direct
session with an RPA.

Connectivity

Indicates if communication to all RPAs in the RPA cluster is alive and if the
storage link, repository volume, data link, and communication with the remote
RPA are all alive; if one RPA is down, connectivity of every RPA in the cluster will
report an error.

When an RPA in the RPA table is selected, the tabs at the bottom of the screen display the
RPA:

204

Properties on page 205

Status on page 206

Interface on page 207

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

Properties
The Properties tab displays the main properties of each RPA.

Figure 23 RPA Clusters > RPA-Cluster-Name tab > RPAs > Properties

Select an RPA in the RPA table and click the Properties tab to display:
Table 42 RPA Clusters: RPA properties tab

Field

Description

RPA Cluster

The RPA cluster that the RPA belongs to.

ID

The ID of the RPA. Each RPA in an RPA cluster has an RPA ID that starts with the
string RPA and ends with a number from 1-8. Only RPA1 and RPA2 can run the
cluster control.

WAN IP

The WAN IP address of the RPA.

Management
IPv4

The LAN IP address of the RPA, in IPv4 format. Use this address to open a direct
session with an RPA.

Management
IPv6

The LAN IP address of the RPA, in IPv6 format. Use this address to open a direct
session with an RPA.

RecoverPoint
Version

The RecoverPoint software release running on the RPA.

Hardware
Platform

The RPA hardware platform.

Hardware
Serial ID

The hardware serial ID of the RPA.

Memory

The amount Random Access Memory (RAM) in the RPA.

CPUs

The number of Central Processing Units (CPUs) in the RPA.

Properties

205

RPA Clusters

Status
The Status tab displays the connectivity statuses of each RPA.

Figure 24 RPA Clusters > RPA-Cluster-Name tab > RPAs > Status

Select an RPA in the RPA table and click the Status tab to display:
Table 43 RPA Clusters: RPA status tab

Field

Description

RPA Status

Whether or not the RPA is connected.

Repository
Volume

Whether or not the RPA can access the Repository Volume.

Storage Link

Whether or not the RPA can access storage.

WAN Interface

Whether or not the physical WAN port is alive, the interface card is functional,
and communication with other local RPAs exists.

FC Connectivity

Whether or not the RPA has access to the intranet.

Communication
with Remote RPA
Cluster

Whether or not the RPA has access to the remote RPA that it is paired with for
replication.

Data Link

Whether or not data can be replicated to the remote RPA that the selected
RPA is paired with for replication.

LAN IPv4 Interface Whether or not the physical LAN port is alive, the interface card is functional,
and communication with other local RPA (if any) exists.

206

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

Interface
The Interface tab displays the Fabric Interface settings of each RPA.

Figure 25 RPA Clusters > RPA-Cluster-Name tab > RPAs > Interface

Select an RPA in the RPA table and click the Interface tab to display the Fabric Interface
settings for each RPA HBA port. These settings are required when replacing a faulty RPA.

Traffic Statistics
Select an RPA in the RPA table and click the Traffic Statistics tab to display the RPAs
traffic statistics.

Figure 26 RPA Clusters > RPA-Cluster-Name tab > RPAs > Traffic Statistics

The Traffic Statistics tab displays the following statistics for the selected RPA:

Interface

207

RPA Clusters

Figure 26 RPA Clusters > RPA-Cluster-Name tab > RPAs > Traffic Statistics (continued)
l

Total traffic on page 208

Application traffic on page 208

Initialization traffic on page 209

Total traffic
The Total Traffic widget displays the total amount of data that is transferred by the
selected RPA in the SAN and between clusters, in Mb/s.

Application traffic
The Application Traffic widget displays the total amount of application data that is
transferred by the selected RPA in the SAN and between clusters, in Mb/s.

208

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

Initialization traffic
The Initialization Traffic widget displays the total amount of data that is transferred by
the selected RPA in the SAN and between clusters during RecoverPoint initialization, in
Mb/s.

Usage Statistics
Select an RPA in the RPA table and click the Usage Statistics tab to display the RPAs
resource usage statistics.

Figure 27 RPA Clusters > RPA-Cluster-Name tab > RPAs > Usage Statistics

The Usage Statistics tab displays the following statistics for the selected RPA:
l

Incoming writes on page 210

Bandwidth reduction on page 210

CPU usage on page 210

Usage Statistics

209

RPA Clusters

Incoming writes
The Incoming Writes widget displays the amount of incoming writes that are transferred
by the selected RPA, per second.

Bandwidth reduction
The Bandwidth Reduction widget displays the ratio by which the bandwidth usage of the
selected RPA is reduced per minute due to RecoverPoint.

CPU usage
The CPU Usage widget displays the percent of the selected RPAs CPU that is utilized per
minute.

210

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

vCenter Servers
Select RPA Clusters > RPA-Cluster-Name > vCenter Servers to display the vCenter Server
table, monitor, register, or remove vCenter Servers. The vCenter user should have the
necessary privileges to perform the desired tasks.
Figure 28 RPA Clusters: vCenter Servers

When the vCenter Server's IP address is selected, the bottom pane displays the vCenter
Servers object, and under it, all VMware vCenter Servers registered with RecoverPoint.
The view displays data extracted from the VMware vCenter Server together with
RecoverPoint replication data.
In addition to displaying ESX servers and all their virtual machines, datastores, and RDM
drives, the RecoverPoint vCenter Server view also displays the replication status of each
volume. The RecoverPoint vCenter Servers view is for monitoring only (read-only).
Select the vCenter Servers link to:
l

Display vCenter Server information on page 212

Register vCenter Server on page 212

Edit vCenter Server Registration on page 213

Remove vCenter Server on page 213

Rescan vCenter Server on page 214

Before you begin


Before you begin, note that the best practice is to configure the vCenter Server to require
the use of a certificate because once RecoverPoint has read the certificate, it does not
need further access to the location.
The default certificate locations are:
l

Windows 2003 Server:


C:\Documents and Settings\All Users\Application Data\VMware
\VMware VirtualCenter\SSL\rui.crt.

Windows 2008 Server:


C:\Users\All Users\Application Data\VMware\VMware
VirtualCenter\SSL\rui.crt.
vCenter Servers

211

RPA Clusters

For more information about the location of the security certificate, refer to Replacing
vCenter Server Certificates VMware vSphere 4.0, available at www.vmware.com.

Display vCenter Server information


To display data from the VMware vCenter Server in Unisphere for RecoverPoint, select
RPA Clusters > RPA-Cluster-Name > vCenter Servers. When selected, the top-right pane
displays the vCenter Server table containing all vCenter Servers registered with
RecoverPoint and their user names. For detailed information about a specific vCenter
Server, in the top pane, click the vCenter Server's IP address to display:
l
Each ESX server in the vCenter Server and its IP address
l
Each virtual machine running in the ESX server, with the following details:
Table 44 Virtual machine info

Information
Replication
status

Icon

Description
Fully configured for replication
Partially configured for replication
Not configured for replication

IP address
l

The IP address of the virtual machine

Every LUN and raw device mapping accessed by each virtual machine, with the
following details:
n
Replication status: fully configured for replication or not configured for replication.
n
For LUNs or devices configured for replication by RecoverPoint, the following are
displayed: consistency group, copy (Production, Local, Remote), replication set,
and which datastore for each LUN or raw device mapping is configured for
replication.

Register vCenter Server


Use the following procedure to create a connection between RecoverPoint and a VMware
vCenter Server, which allows RecoverPoint to display the VMware view of volumes
configured for replication.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
3. In the right pane, click Add. The Register vCenter Server dialog box is displayed.

212

RecoverPoint 4.4 Administrator's Guide

RPA Clusters

4. In the Register vCenter Serverdialog box, enter the following information:


Table 45 RPA Clusters: Add vCenter Server

Setting

Description

vCenter Server IP address of the vCenter Server. This is also the display name of the vCenter
IP
Server in the top pane.
Port

TCP port number of the vCenter Server. Default = 443.

Username

vCenter Server username.

Password

vCenter Server password.

Certificate

If you wish to specify a certificate, browse to and select the certificate file.

Edit vCenter Server Registration


Use the following procedure to edit the credentials of an existing connection between
RecoverPoint and a VMware vCenter Server.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
3. In the right pane; select a vCenter Server, and click Edit. The Edit vCenter Server
Registration dialog box is displayed.
4. In the Edit vCenter Server Registration dialog box, you can edit the following
information:
Table 46 Edit vCenter Server Settings

Setting

Description

vCenter Server IP address of the vCenter Server. This is also the display name of the vCenter
IP
Server in the top pane.
Port

TCP port number of the vCenter Server.

Username

vCenter Server username.

Password

vCenter Server password.

Certificate

If you wish to specify a certificate, browse to and select the certificate file.

Remove vCenter Server


Use the following procedure to remove a connection between RecoverPoint and a VMware
vCenter Server.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
Edit vCenter Server Registration

213

RPA Clusters

3. In the right pane; select a vCenter Server, click Remove, and click Yes in the
confirmation dialog box.

Rescan vCenter Server


Use the following procedure to obtain the latest information from the VMware vCenter
Servers registered with RecoverPoint, and update Unisphere for RecoverPoint.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
3. In the right pane, click the Rescan icon.

Phoenix Systems
Select RPA Clusters > RPA-Cluster-Name > Phoenix Systems to use, monitor or disable
Phoenix System RP Protection. The top-right pane allows you to choose which Phoenix
system to use. It also displays the status, disk capacity, and available disk capacity. For
more details, see EMC RecoverPoint Deploying with Phoenix System RP Technical Notes.

214

RecoverPoint 4.4 Administrator's Guide

CHAPTER 6
Admin

This section describes how to license and register RecoverPoint, how to manage users,
roles, and user authentication, how to configure system notifications, detect bottlenecks
in replication, analyze the load on each RPA, and collect system information for support
purposes. This section also describes how to export statistics and use the statistics tool
through the RecoverPoint CLI.
l
l
l
l
l
l

RecoverPoint Administration............................................................................... 216


Manage Licenses................................................................................................ 216
Register RecoverPoint..........................................................................................220
Users and Roles.................................................................................................. 225
System Analysis Tools......................................................................................... 239
System Notifications........................................................................................... 253

Admin

215

Admin

RecoverPoint Administration
Administration of RecoverPoint users and the RecoverPoint system is performed through
the Admin menu.

Table 47 Admin options

Option

When to Use

Manage
Licenses

To add or remove your RecoverPoint license files from the system or view your
RecoverPoint license details.

Fill out or update your RecoverPoint post-deployment form and automatically


Register
RecoverPoint send it to the EMC Install Base Group, registering your RecoverPoint product.

Users and
Roles

To manage users, roles, and user authentication. To log off and log back in as a
different user, to change the password of the currently logged in user , or to
change after how much idle time to log off the user.

System
Analysis
Tools

To detect bottlenecks in replication, analyze the load on each RPA, and collect
system information for support purposes.

To configure system reports and alerts, manage event filters and call home
System
Notifications events.

Manage Licenses
Select Manage Licenses to add or remove RecoverPoint licenses from the system, when
Licensing RecoverPoint on page 17.

216

RecoverPoint 4.4 Administrator's Guide

Admin

Install a license
Before you begin
When the first license file is added to the system, the Software Serial ID (SSID) in the new
license is automatically used as the identifier of your RecoverPoint system.
NOTICE

When subsequent licenses are added to the system, if they contain a different SSID than
the one currently in the system, you will need to register your RecoverPoint system again
with the new SSID.
Procedure
1. Select Admin > Manage Licenses to display the Manage Licenses screen.

Then for each license file:


2. Click Add to display the Add License dialog box.

3. Click Browse... and select a license file.


4. Click OK to add the license file to the RecoverPoint system.

Install a license

217

Admin

Note

To ensure that your license file contains the anticipated configuration, Display your
RecoverPoint license information on page 219 and check this information against the
license file variables on page 295.

Remove licenses
Procedure
1. Select Admin > Manage Licenses.
2. Select the license that you want to remove, and click Remove.
Note

A warning is issued if removing the license may cause the system to exceed the
allowed replication capacity (for example, when an array license is removed but an
existing consistency group contains volumes that reside on the removed array).

Monitor your license limits


Use the following procedure to ensure that you are replicating well within the limits of
your RecoverPoint license.
Procedure
1. Select Dashboard > System Limits > System.
2. Note the value of the following licensing-related system limits:
l

Number of bytes replicated locally per licensed RPA cluster

Number of bytes replicated remotely per licensed RPA cluster

Number of bytes replicated locally per licensed array

Number of bytes replicated remotely per licensed array

Number of days until RecoverPoint license expires

Export a detailed license report


You can export a detailed license report that includes all of the displayed license
information on page 219. The report is exported as a comma-delimited file (*.csv) that
can be opened in MS Excel or any other reader of CSV files.
Procedure
1. Select Admin > Manage Licenses.
2. Click the Export button at the top right corner of the screen.
3. In the Export dialog box:

218

RecoverPoint 4.4 Administrator's Guide

Admin

a. Select the desired export options and click Next.


b. Click Download to download the file to your computer, select a location on your
computer to store the file, and click Save.
4. Open the exported file:
a. Open MS Excel.
b. Open the *.csv file from the location you selected in the previous step. The Excel
Text Import Wizard is displayed to help you set the import options.
c. In the Excel dialog box, select Delimited, and click Next.
d. In the Delimiters field, select Comma, and click Next.
e. Click Finish.

Display your RecoverPoint license information


After you have added one or more licenses to RecoverPoint, you can display your
RecoverPoint license information through the Admin > Manage Licenses screen.

The following attributes are displayed, per license:


Table 48 Displayed license information

Parameter Values
Type

Description

RecoverPoint/CL Uses the RecoverPoint/CL licensing model.


RecoverPoint/EX Uses the RecoverPoint/EX licensing model.
RecoverPoint/SE Uses the RecoverPoint/SE licensing model.
RecoverPoint/BK Uses the RecoverPoint/BK licensing model.

RPA Cluster <RPA Cluster>

The name of the licensed RPA cluster.

Array Serial <Array Serial>

The serial number of the licensed storage array.


l

RecoverPoint/CL: None
Display your RecoverPoint license information

219

Admin

Table 48 Displayed license information (continued)

Parameter Values

Description
l

Replication Local

RecoverPoint/SE and RecoverPoint/EX:


An array serial number.

The license supports replication to a local copy.

Remote

The license supports replication to a remote copy.

Capacity

<Capacity>

The replication capacity supported by the license (in terabytes).

Expiration

<Date>

The date at which the license expires.

Permanent

The license does not have an expiration date.

Active

The license is active.

Expired

The expiration date of the license has passed.

Inactive

The license is not active. This can happen when:

Status

If a licensed array is not mapped and zoned to the RPA.

If a RecoverPoint/EX license and a RecoverPoint/CL license


are both added at the same RPA cluster, and the
RecoverPoint/EX license becomes inactive.

Note

Check the displayed information against the license file variables on page 295 for
correctness and Monitor your license limits on page 218 to ensure that you are
replicating well within the limits of your RecoverPoint license.

Register RecoverPoint
Select the Register RecoverPoint menu to fill out or update your RecoverPoint postdeployment form and automatically send it to the EMC Install Base Group, registering
your RecoverPoint product.

220

RecoverPoint 4.4 Administrator's Guide

Admin

Register your RecoverPoint system


Before you begin
l

Register your RecoverPoint system after:


n

installing a RecoverPoint system

connecting RPA clusters in a RecoverPoint system

upgrading a RecoverPoint system

The RecoverPoint registration process will fail if you do not first configure your SMTP
address on page 266.

The registration process will be incomplete if valid values are not provided for every
field in the post-deployment form.

When a new RPA cluster is added to the system, it is automatically added to the postdeployment form with empty parameter values. Ensure you have entered the values
for all new RPA clusters before sending the post-deployment form to the IB group.

Procedure
1. Select Admin > Register RecoverPoint to display the RecoverPoint Post-Deployment
Form.

Register your RecoverPoint system

221

Admin

2. Click Update Form to display the Update Post-Deployment Form Details dialog box.

3. In the Update Post-Deployment Form Detailsdialog box:


a. Select the relevant RPA cluster from the RPA Cluster drop-down.
b. Update the form information according to the instructions in the following table.
Table 49 Automatically registering your RecoverPoint system

222

Field

Description

For Example

Activity Type

Enter the type of activity you are performing


(upgrade, installation)

Upgrade

Location

The city, state and country where the customer is


located.

New York, New York,


USA

Upgrade/
Installation
Performing
Resource

Enter the role of the person performing this


upgrade or installation activity

Customer

Sales Order
Number

The customer or Customer Engineer should


provide this information

62276566

VCE

Indicate whether or not this RecoverPoint


Selected=Yes
implementation is operating within a VCE (Vblock) Unselected=No
environment.
or Unknown
VCE = VMware+Cisco+EMC

Connect
Home

The method used to send configuration reports


and alerts to EMC. Enabling this feature is
recommended as it allows EMC to pro-actively
address issues within your RecoverPoint
environment, should they arise.
Configure System Reports and Alerts on page
264 to establish connectivity for your
RecoverPoint environment.

Software: ESRS
(preferred), E-mail,
Not Allowed
Hardware: Not
Applicable

Connect In

The method used to allow remote connectivity to


your RecoverPoint environment. Enabling this
feature is recommended as it allows secure
access to your RecoverPoint environment to

Software: ESRS
(preferred), WebEx, or
Not Allowed

RecoverPoint 4.4 Administrator's Guide

Admin

Table 49 Automatically registering your RecoverPoint system (continued)

Field

Description

For Example

gather logs and resolve issues as expeditiously as Hardware: ESRS


possible.
(preferred, WebEx, or
If you already have an ESRS Gateway servicing
Not Allowed
other EMC products, simply use the ESRS Config
Tool (refer to the EMC Secure Remote Support
Gateway Operation Guide for further instructions on
Config Tool usage) to add your RecoverPoint
devices (or have your gateway administrator do it
for you) to the list of ESRS monitored
environments. Once the device is added, click the
request update button to send the new device
information to EMC and contact your local EMC
Customer Engineer to approve the update.
If you do not have an ESRS Gateway at your site,
contact your EMC Account Manager to find out
more about the benefits of ESRS.

c. Click Apply to save each change to the form, without closing the dialog box.
d. Click OK to save all changes and close the dialog box.
NOTICE

If your company does not have SyR connectivity, and therefore, you cannot
Configure system reports and alerts on page 266, you can skip the rest of the
steps in this procedure, Export the post-deployment form to a CSV file on page
224 and Register RecoverPoint by email or phone on page 225 now.
4. Click Send Form to automatically send the post-deployment form to the EMC Install
Base group. The Send Form dialog box is displayed.

5. In the Send Form dialog box, enter the email address of the person at your company
that is in charge of RecoverPoint maintenance and operation.
Note

Call home event 8000 is sent to EMC with your updated registration information and
an email is sent to the specified email address from EMC Customer Support to verify
that your registration details were updated successfully in the EMC Install Base.
6. Wait a couple of minutes.
7. Select Dashboard > System Events and look for events 8000 Calling homeCalling
home. If event 8000 is logged; the registration details were successfully sent to the
EMC Install Base.
Register your RecoverPoint system

223

Admin

Results
The following information is automatically sent along with the information that you
entered into the post-deployment form, per RPA cluster:
l

Software serial ID

Hardware serial IDs for RPAs 1-8

RecoverPoint software version

Cluster name

License type

Connect home method

The System ID (identical for all RPA clusters in a RecoverPoint system)

Export the post-deployment form to a CSV file


You can save your RecoverPoint registration information or Register RecoverPoint by
email or phone on page 225 by exporting the RecoverPoint post-deployment form and all
of its contents as a comma-delimited file (*.csv) that can be opened in MS Excel or any
other reader of CSV files. The Activity Date field in the exported CSV file contains the date
and time that the file was exported.
Procedure
1. Select Admin > Register RecoverPoint.
2. Click the Export button at the top right corner of the screen.

3. In the Export dialog box:

a. Select the desired export options and click Next.


b. Click Download to download the file to your computer, select a location on your
computer to store the file, and click Save.
4. Open the exported file:
a. Open MS Excel.
b. Open the *.csv file from the location you selected in the previous step. The Excel
Text Import Wizard is displayed to help you set the import options.
c. In the Excel dialog box, select Delimited, and click Next.
d. In the Delimiters field, select Comma, and click Next.

224

RecoverPoint 4.4 Administrator's Guide

Admin

e. Click Finish.

Register RecoverPoint by email or phone


Registers RecoverPoint if your company does not have external connectivity, and
therefore you cannot register your RecoverPoint system online.
Before you begin
l

Register your RecoverPoint system after:


n

installing a RecoverPoint system

connecting RPA clusters in a RecoverPoint system

upgrading a RecoverPoint system

The registration process will be incomplete if valid values are not provided for every
field in the post-deployment form.

Procedure
1. Access https://support.emc.com
2. Search for the term Post-Deployment Form
3. Download and fill out the RecoverPoint and RP for VMs Post-Deployment Form
4. Send the information to the EMC Install Base group:
l

EMC customers and partners: Email the post-deployment form to the Install Base
group at rp.registration@emc.com.

EMC employees:
n

(Preferred) Use the IB Portal at http://emc.force.com/BusinessServices.

Call in the information to the Install Base group at 1-866-436-2411 Monday


to Friday (regular Eastern Time Zone working hours).

Users and Roles


Select the Users and Roles menu to manage users, roles and user authentication, to log
off and log in as a different user, to change the password of the currently logged in user,
to change the security level, or to change after how much idle time to log off the user.

Register RecoverPoint by email or phone

225

Admin

Manage Users
Select Admin > Users and Roles > Manage Users to view, add, remove and modify
RecoverPoint users.

How users work


The RecoverPoint appliance is shipped with the following local users already defined:
Table 50 Pre-defined users

User

Role

Initial Password Permissions

security-admin security

security-admin

Security: changing users and roles, security levels,


LDAP configuration

admin

admin

admin

All, except security

boxmgmt

boxmgmt boxmgmt

Boxmgmt/Upgrade

You add users, edit users and remove users by selecting Admin > Users and Roles >
Manage Users.
Note

User authorization grants or denies users access to resources managed by RecoverPoint.


User authorization is identical, regardless of whether the user was authenticated by
RecoverPoint or by a Microsoft Windows Active Directory LDAP server. User authorization
can be limited to specific consistency groups.

Add new users


Before you begin

226

Only users with security permissions on page 231 can add users, and can remove
and edit permissions for users that have previously been added.

You cannot remove pre-defined users on page 226.

You cannot change the permissions of pre-defined users on page 226.

To change the passwords of pre-defined users on page 226, Switch User on page
237 and Change Password on page 236.

It is recommended that you change the passwords of predefined users.

If you wish to implement a purely LDAP-based authentication system, you need not
give out the passwords of any pre-defined users on page 226.

To be able to access RecoverPoint, every user in an LDAP Active Directory must be


added to RecoverPoint users list and assigned a role, or be a member of a group that
has access.

RecoverPoint 4.4 Administrator's Guide

Admin

Procedure
1. Select Admin > Users and Roles > Manage Users. The RecoverPoint user list is
displayed.

2. Click Add. The Add User dialog box is displayed.

3. Select Local User or LDAP User or Group.


l

To configure a local user:


a. Provide a username and a password, according to your security level.
b. Select the role in the user settings section.
c. To limit access to specific consistency groups, click Only for selected groups
and select the consistency groups to which you are granting this user access.

To configure an LDAP user or group:


a. Click Select... to select either a user name or a group from the lists of users and
groups.
b. Select the role from the drop-down in the user settings section.
c. To limit access to specific consistency groups, click Only for selected groups
and select the consistency groups to which you are granting this user access.
Manage Users

227

Admin

Binding to the LDAP server will fail unless at least one of the following is true:
a. The connection to the server uses LDAP over SSL (only if the Active Directory
supports SSL).
b. Anonymous binding is used.
c. Signing requirements are set to "none" in Active Directory.
Settings

Description

Local User
User Name

Name of new user to add for appliance-based


authentication. A user name must start with a lower-case
alphabetic character. All subsequent characters must be
lower-case alphabetic, numeric, or hyphen (-). No other
characters are legal in user names.

Password

Password for new user. All printing ASCII characters are


legal in passwords.

Confirm
Password

Password for new user. All printing ASCII characters are


legal in passwords.

LDAP User or
Group

To add users that already exist in the Active Directory on


the LDAP server, and grant them access to the
RecoverPoint appliance.

User Name

Name of a user to add to RecoverPoint user list.

Groups

Name of a group or groups (in Active Directory) to add to


the authorized users of RecoverPoint.

User Settings
Role

The role of the specified local user, LDAP user, or LDAP


group.

Only for selected When checked, limits the access of this local user, LDAP
groups
user, or LDAP group to the specified consistency groups.
Groups

Select the consistency groups that the specified user or


group may access.

Edit existing users


Note

Only users with securitypermissions on page 231 can edit users. You cannot edit predefined users on page 226.
Procedure
1. Select Admin > Users and > Roles > Manage Users.
2. To edit a local user, an LDAP user or LDAP group:
l

To modify the users password:


a. Read how security levels work in RecoverPoint.
b. Change Password on page 236

228

To modify the users username:

RecoverPoint 4.4 Administrator's Guide

Admin

a. Remove the user according to Remove users on page 229.


b. Add new users on page 226 with a new username.
l

To modify the users role:


a. Select a user.
b. Click Edit.
The Edit Local User dialog box is displayed. The name of this dialog box
depends on the user type (Local, LDAP User, LDAP Group or LDAP User and
Group), but all of these dialog boxes contain the same options except for the
Edit Local User dialog, which additionally allows you to change the user and
password of the local user.

c. Select one of the pre-defined roles on page 230 from the drop-down in the
User Settings section.
d. If required, modify the access to specific consistency groups by selecting Only
for selected groups.
e. Click OK.

Remove users
Note

Only users with security permissions on page 231 can remove users and you cannot
remove pre-defined users on page 226.
Procedure
1. Select Admin > Users and > Roles > Manage Users.
2. Select a user.
3. Click Remove, and click yes in the confirmation dialog box that appears.

Manage Users

229

Admin

Manage Roles
Select Admin > Users and Roles > Manage Roles to view, add, remove and modify
RecoverPoint roles.

How roles work


Each RecoverPoint user is defined by a user name, a password, and a role. A role is a
named set of access permissions. By assigning a role to users, the users receive all the
access permissions defined by the role.
Note

User authorization grants or denies users access to resources managed by RecoverPoint.


User authorization is identical, regardless of whether the user was authenticated by
RecoverPoint or by a Microsoft Windows Active Directory LDAP server. User authorization
can be limited to specific consistency groups.
The following pre-defined roles exist in RecoverPoint:
Table 51 Pre-defined roles

Role

Permissions

security

Security

Read only

System configuration

Storage management

Splitter configuration

Upgrade

Data transfer

Failover

Boxmgmt

Group configuration

Target image

Read only

admin

230

boxmgmt

Boxmgmt

monitor

Read only

webdownload

Web download

RecoverPoint 4.4 Administrator's Guide

Admin

Roles define the following permissions:


Table 52 Permissions

Permission

Description

Splitter
Configuration

Add or remove splitters, and attach or detach splitters to volumes.

Group
Configuration

Create and remove consistency groups, and modify all group settings except
those that are included in the data transfer, target image, and failover
permissions, may bookmark images, and resolve settings conflict.

Storage
Management

Register arrays and vCenter Servers, and update storage registration details for
arrays and vCenter Servers.

Data Transfer

Enable and disable access to images, and undo writes to the image access log.

Target Image

Enable and disable access to images, resume distribution, and undo writes to
the image access log.

Failover

Modify replication direction (in temporary failover and recover production),


initiate failover, verify failover.

System
Configuration

Configure and manage e-mail alerts, SNMP, System notifications, alert rules,
licenses, syslog, and other system configuration settings.

Security

All commands dealing with roles, users, LDAP configuration, and security level.

Upgrade

Installing RPA software, RPA maintenance, including upgrading to a minor


RecoverPoint release, upgrading to a major RecoverPoint release, replacing an
RPA, adding new RPAs, converting an environment to a RecoverPoint/SE or
RecoverPoint configuration, or modifying system settings.

Web
download

Download RecoverPoint installation packages from the EMC web site.

You add roles, edit roles and remove roles by selecting Admin > Users and Roles > Manage
Roles.

Add a new role


Before you begin
l

Only users with security permissions on page 231 can add, edit, or remove roles
added by the user.

You cannot edit or remove pre-defined roles on page 230.

Procedure
1. Select Admin > Users and > Roles > Manage Roles.
2. Click Add. The Add Role dialog box is displayed.

Manage Roles

231

Admin

3. Enter a new role name and set the required access permissions in the role settings on
page 232.
Table 53 Role settings

Setting

Description

Role

Name of the new RecoverPoint role to add.

Permissions

Select the access permissions on page 231 to be granted to all persons who are
assigned to this role.

4. Click OK.

Edit an existing role


Use the following procedure to change the name or permissions of a role.
Note

Only pre-defined users on page 226 with security permissions on page 231 can edit roles.
You cannot edit pre-defined roles on page 230.
Procedure
1. Select Admin > Users and Roles > Manage Roles.
2. Select a role.
3. Click Edit. The Edit Role dialog box is displayed.

232

RecoverPoint 4.4 Administrator's Guide

Admin

4. Change the role name and/or select new permissions on page 231.

Remove a role
Note

Only pre-defined users on page 226 with security permissions on page 231 can remove
roles. You cannot removepre-defined roles on page 230.
Procedure
1. Select Admin > Users and > Roles > Manage Roles.
2. Select a role.
3. Click Remove, and click yes in the confirmation dialog box that appears.

Manage User Authentication


Select Admin > Users and Roles > Manage user Authentication to configure access to an
LDAP Active Directory for user authentication.

How user authentication works


User authentication configures RecoverPoint to use the organizations Microsoft Windows
Active Directory LDAP server for authentication.
RecoverPoint provides two independent mechanisms for authenticating users: appliancebased authentication and authentication via the organizations Microsoft Windows Active
Directory LDAP (Lightweight Directory Access Protocol) server. The two authentication
mechanisms can be used simultaneously, LDAP may be used exclusively, or appliancebased authentication can be used exclusively.
Manage User Authentication

233

Admin

Authenticate users using LDAP


Before you begin
l

Only users with security permissions on page 231 can configure user authentication
through an organizations Microsoft Windows Active Directory LDAP server.

Although optional, it is strongly recommended to have secondary LDAP servers at


both the production and at the copy sites. If no LDAP server is available, users may
not be able access RPAs or use RecoverPoint.

RecoverPoint only accepts LDAP certificates in PEM format. To format the certificate
created on the LDAP server in PEM format, use the following procedure:
1. On the LDAP server, export a copy of the server certificate from the Active Directory
server. Use the Certification Authority application's Copy File to option to export
the certificate in Base-64 Encoded X.509 (.cer) format.
2. Copy the server certificate to a system with OpenSSL Certificate Authority software
installed. You can use any Linux or Windows system.
3. Log into the system where you copied the certificate, and run the following
command to create the PEM certificate:
n

On Linux:
> /opt/symas/bin/openssl x509 -in AD_certificate_name -out
OpenLDAP_certificate name

On Windows:
> openssl x509 -in drive:/path/AD_file.cer -inform d -out drive:/path/
OpenLDAP_file.pem

Procedure
1. Select Admin > Users and Roles > Manage User Authentication. The Manage User
Authentication screen is displayed.

2. Configure the LDAP settings:


234

RecoverPoint 4.4 Administrator's Guide

Admin

Table 54 LDAP Configuration settings

Settings

Description

Enable Active
Directory Support

Check to activate RecoverPoint authentication and authorization using a


Microsoft Windows Active Directory LDAP server.

LDAP Configuration
Primary LDAP
server

IP address of the primary LDAP server

Secondary LDAP
server

(optional) IP address of the secondary LDAP server.


Although optional, it is strongly recommended to have secondary LDAP
servers at both the production and at the copy sites. If no LDAP server is
available, users may not be able access RPAs or use RecoverPoint.

Base Distinguished Node in the LDAP tree from which to start a search for users:
Name
dc=Klaba,dc=COM
Search Base
Distinguished
Name

Root of the LDAP user search tree. The suffix of the Search Base
Distinguished Name must be the Base Distinguished Name. The format will
be similar to the following:
cn=Users,dc=Klaba,dc=COM

Binding Type:
To specify the type of binding (authentication against the Active Directory).
Bind Distinguished
Name

Distinguished name to use for initial binding when querying the LDAP
server. The format of the Bind Distinguished Name will be similar to the
following:
cn=Administrator,cn=Users,dc=Klaba,dc=COM
The bind distinguished name can be any user on the LDAP server who has
read permission for the directory in the defined search base.

Password

Password of the bind distinguished name to use for initial binding when
querying LDAP server

Directory Access Protocol


LDAP

To send LDAP query over non-secure connection.

LDAP over SSL

To send LDAP query over a secure connection.

Upload certificate
from file

Select if you want to enter a path to an Active Directory certificate to use for
secure communication with LDAP server.
To export the certificate and format it in PEM format:
On the LDAP server, export a copy of the server certificate from the Active
Directory server. Use the Certification Authority application's Copy File to
option to export the certificate in Base-64 Encoded X.509 (.cer) format.
Copy the server certificate to a system with OpenSSL Certificate Authority
software installed. You can use any Linux or Windows system.
Log into the system to which you copied the certificate, and run the
following command:
On Linux:
> /opt/symas/bin/openssl x509 -in AD_certificate_name -out
OpenLDAP_certificate name
Manage User Authentication

235

Admin

Table 54 LDAP Configuration settings (continued)

Settings

Description
On Windows:
> openssl x509 -in drive:/path/AD_file.cer -inform d -out drive:/path/
OpenLDAP_file.pem

Advanced settings
The advanced settings are optional and can be left at their default values.
Search time limit

Default 30 sec.

3. Click Apply.
4. Test LDAP settings on page 236 to verify that you have configured LDAP correctly.

Test LDAP settings


Procedure
1. Select Admin > Users and > Roles > Manage User > Authentication.
2. Enter the LDAP Configuration settings on page 235.
3. Click Apply.
4. Click Test configuration.
Note

An error is displayed if the system is unable to connect to the primary or secondary


LDAP servers.

Modify LDAP settings


Procedure
1. Select Admin > Users and Roles > Manage User > Authentication.
2. Click Clear LDAP Settings, and click yes in the confirmation dialog box.
3. Modify the LDAP Configuration settings on page 235.
4. Click Apply.
5. Test LDAP settings on page 236 to verify that you have configured LDAP correctly.

Change Password
Select Admin > Users and Roles > Change Password to change the password of the
currently logged-in user.

236

RecoverPoint 4.4 Administrator's Guide

Admin

Procedure
1. Enter your current password.

2. Enter the new password in the New Password and Confirm Password fields according
to the password rules of your user's security level.
3. Click OK.

Switch User
Use the following procedure to log off, and log back in as a different RecoverPoint user.
Select Admin > Users and Roles > Switch User to log off, and log back in as a different
RecoverPoint user.

Procedure
1. Enter the username and password of the user as whom you want to log into
RecoverPoint.

Switch User

237

Admin

2. Click Login.

Change Security Level


Use Admin > Users and Roles > Change Security Level to set the password strength and
security rules for the currently logged-in user.

Change a users security level


Sets the security level of user passwords that are required to access the RPAs in your
system.
Before you begin
l

Only pre-defined users on page 226 with security permissions on page 231 can
change the security level of a user.

Regardless of the security level, any user who tries unsuccessfully three times to log
in will be locked out. To unlock the user, use the CLI command unlock_user. Only
users with security permission can unlock a user.

The default security level is set to Basic. It is recommended that RecoverPoint


administrators set the level to High to meet relevant security standards, such as those
of the USA Department of Defense Security Technical Implementation Guides (DoD STIG).

When the security level is set to High, user passwords:

must contain a minimum of fourteen characters; at least two characters that are
lower-case, at least two characters that are upper-case, and at least two
characters that are non-alphabetical (either digits or special characters).

can only be reset once every 24 hours.

cannot be reused.

expire in 60 days.

When the security level is set to Basic, user passwords must have a minimum of five
characters.

Procedure
1. Select Admin > Users and Roles > Change Security Level.
2. Select either High or Basic.
3. Click OK.

238

RecoverPoint 4.4 Administrator's Guide

Admin

Change Idle Logout Time


Use Admin > Users and Roles > Change Idle Logout Time to specify after how much time
without user input to automatically log off the user.

System Analysis Tools


Use the System Analysis Tools menu to detect bottlenecks in replication, analyze the
load on each RPA, and collect system information for support purposes.

Note

You can also access additional system analysis tools through the RecoverPoint CLI. Use
the RecoverPoint CLI to export statistics , export consolidated statistics , and load the
statistics analysis tool.

Detect Bottlenecks
Use Admin > System Analysis Tools > Detect Bottlenecks to display statistics about
system performance by group, RPA, or RPA cluster, and use the statistics to identify and
mitigate bottlenecks in transfer.

Procedure
1. Define the bottleneck detection settings on page 240. Note that in bottleneck
detection, the entered times are according to the local time and the analysis output is
according to GMT.

Change Idle Logout Time

239

Admin

Table 55 Bottleneck detection settings

Setting

Description

Detection
Mode

Default=System overview and bottleneck analysis


Type of analysis to perform.
Possible values are System overview and bottleneck analysis, Initialization periods,
High load periods, Initialization, high load, and peak analysis, Advanced analysis,
and Peak analysis.

Start

Default=Earliest available time


Start time for analysis. Possible values are:
l

Earliest available time

A user-specified date and time, where the date and time, if not specified, is
the current local time today.
Note

Enter times according to local time. Output is according to GMT.


End

Default=The current date and time


End time for analysis, where the date and time, if not specified, is the current
local time today.
Note

Enter times according to local time. Output is according to GMT.

240

RecoverPoint 4.4 Administrator's Guide

Admin

Table 55 Bottleneck detection settings (continued)

Setting

Description

Advanced
Overview

Default=no
Whether the analysis covers a broader set of system performance indicators (yes)
or not (no).

Detailed
Overview

Default=no
Whether the analysis is confined to the RPA cluster only (no) or also includes
analysis of the groups and RPAs (yes).

Peak Duration

Default=5 minutes
Includes an analysis of the periods in which the write peaks were of the specified
time duration.

Groups to
Include

Default=all
The consistency groups on which to perform the analysis. To analyze only global
statistics, without statistics on individual groups, clear all of the group
checkboxes.

2. Click Start to start the analysis. A progress bar is displayed.

[Wait for the process to complete]


3. During analysis, you can use any of the bottleneck detection commands on page 241.
Table 56 Bottleneck detection commands

Action

Description

Start New
Analysis

To stop the current analysis and display the screen containing the bottleneck
analysis settings.
Note

Only available when analysis is in progress.


Stop Analysis

To stop bottleneck analysis.


Note

Only available when analysis is in progress.

Detect Bottlenecks

241

Admin

Table 56 Bottleneck detection commands (continued)

Action

Description

Display
Results

To open a window containing the bottleneck analysis results.


Note

Only available when analysis is complete.

4. Click Display Results. The bottleneck detection results are displayed.

5. Copy the results and paste them into a text file.


6. Save the text file for later reference.

Balance Load
Use this procedure to analyze the load on each RPA and display a load balancing
strategy.
Before you begin

242

Load balancing is only available on the RPA running the cluster control, or through
the floating Cluster Management IP. Load balancing will not work if the RPA running
the cluster control is down. If this RPA goes down, and then back up again, five
minutes must pass before load balancing can be performed again.

For best results, groups should be configured and replicating for seven days before
performing load balancing, so that a long enough traffic history is available for the
load balancing analysis.

The load balancing analysis is performed on all RPAs at all sites.

Groups can be excluded from the load balancing recommendation. However, even
when excluded from the recommendation, all groups are included in the analysis.

RecoverPoint 4.4 Administrator's Guide

Admin

The load balancing analysis is performed on all RPAs in the environment. If there are
three RPAs in total, and one of them goes down for a week, and it is the primary RPA
of a group included in the analysis, the load balancing recommendation calculations
are still based on the total number of RPAs, as if they were all working. In other
words, the load balancing mechanism accumulates group statistics for its
calculations, as if groups were always running on their preferred RPAs, even if
flipovers happened during the week.

Load balancing can identify cases in which distributed group segments do not
actually require four separate RPAs to run on, and recommend RPA assignments
accordingly. Distributed group segments can be run on different RPAs, or the same
RPA, and each group segment is treated as a separate group in the load balancing
analysis and recommendation. If you are not well acquainted with distributed
consistency groups, it is recommended that you become acquainted with how they
work before using the load balancing tool (see the EMC RecoverPoint Glossary of Terms
for more information on distributed consistency groups).

To ensure consistency, consistency groups are initialized when moving from RPA to
RPA. During switchover, all groups running on the preferred RPA are initialized once
when they move to the non-preferred RPA, and then another time when they switch
back to their preferred RPAs. Therefore, any re-assignment of RPAs during replication
should be carefully planned out, as not to affect the performance of the production
host applications.

Since the modification of preferred RPA assignments causes flipovers that cause
initializations, if the load balancing analysis finds that no significant changes in
workload or number of consistency groups have occurred, the mechanism will not
recommend preferred RPA re-assignment.

The load balancing mechanism smooths-out I/O statistics, so that any sudden peaks
or lapses in traffic do not cause it to recommend unnecessary changes.

Procedure
1. Select Admin > System Analysis Tools > Balance Load.
2. Select the consistency groups to exclude from the analysis.

Note

The groups that you select will be preceded by an asterisk in the load balancing
recommendation, and their traffic statistics will be included in the analysis upon
which the recommendation is based.
3. Click Start to start the analysis. A progress bar is displayed.
Balance Load

243

Admin

[Wait for the process to complete]


4. During analysis, you can use any of the load balancing commands on page 244.
Table 57 Load balancing commands

Action

Description

Start New
Analysis

To stop the current analysis and display the screen containing the load
balancing settings.
Note

This option is only enabled when analysis is in progress.


Stop Analysis

To stop the load balancing analysis.


Note

This option is only enabled when analysis is in progress.


Display Results

To open a window containing the load balancing analysis results.


Note

This option is only enabled when analysis is complete.

5. When the analysis is complete:


l

244

Click Display Results. The load balancing recommendation is displayed.

RecoverPoint 4.4 Administrator's Guide

Admin

A normal scope event is logged indicating that the load balancing process has
ended. The event contains a copy of the load balancing recommendation output.
6. Understand the load balancing results:
l
If the load balancing recommendation is based on less than 7 days-worth of data,
a note is displayed indicating so.
l
One or more of the following load balancing recommendations are displayed:
n
No action is necessary. The environment is stable, and groups are evenly
distributed across all RPAs.
n
Action is necessary. The environment is not stable because groups are not
evenly distributed across all RPAs. To correct this, manually modify the
Preferred RPA of each group.
n
Action may be necessary. The environment is stable, although groups are not
evenly distributed across all RPAs, and this may affect future performance. To
distribute groups evenly across all RPAs, manually modify the Preferred RPA of
each group.
l
The current preferred and recommended RPA assignments are displayed per
consistency group, along with each RPAs average throughput and incoming writerate, in a Recommended Load Distribution table.
l

Note

For distributed groups, each consistency group segment is treated as a separate


group, and the performance statistics are displayed per group segment.

Balance Load

245

Admin

Two tables, Traffic per RPA before Application of Recommendation and Traffic per
RPA after Application of Recommendation, are displayed in order to aid you in the
process of understanding the performance implications on all RPAs, if you do
choose to apply the recommendation. Each RPAs average throughput and IOPS are
displayed, and the RPA with the least amount of traffic is highlighted, in each of
the tables.

If any non-distributed (regular) consistency groups have experienced a minute of


throughput that exceeded the maximum throughput rate of a single RPA (as
described in EMC RecoverPoint Release Notes) at least 50 times during the week,
the load balancing recommendation indicates that these groups should be set as
distributed. If any distributed consistency groups have experienced less than 40
minutes of throughput at a rate of over 60 MB/sec during the week, the load
balancing recommendation indicates that these groups should be set as nondistributed.

If the total group throughput is exceeded, a message is displayed indicating that


you should add additional RPAs to your RPA cluster, and run load balancing
analysis again after 7 days. See the EMC RecoverPoint Release Notes for the total
throughput per distributed consistency group and total throughput per RPA
limitations.

7. Retrieve the load balancing results:


a. Open a browser window and type the following path into your browsers address
bar:
<Cluster_Management_IP>/info/load_balancing/
b. Log in as one of the pre-defined users on page 226 with webdownload
permissions.

246

RecoverPoint 4.4 Administrator's Guide

Admin

c. Download the relevant text file(s) and store them for later reference. The load
balancing output is saved to text files with the name:
balance_load_yyyy-mm-dd_hh-mm.txt
Note

Ten load balancing output files are stored in this location at a time. When ten files
have already been stored, the older files are removed to make way for newer files.
The date and time of the output file is indicated in the filename.
d. Save the text files to a local directory for later reference.
8. Change the preferred RPAs of consistency groups based on the load balancing
recommendation:
a. Select Protection > Manage Protection.
b. In the left pane, select the group whose preferred RPA you want to change.
c. In the right pane, select the Group Policy tab.
d. Change the value of Preferred RPA according to the load balancing
recommendation. If the group is a distributed group and the load balancing
recommendation indicates so, change the value of Secondary RPAs too.
e. Click Apply.
Each time a consistency groups preferred or secondary RPAs are modified, the
group undergoes an initialization to ensure consistency. For distributed
consistency groups, if one group segment is initialized, all group segments are
initialized.
9. Verify that load balancing improved performance on page 247

Verify that load balancing improved performance


After running load balancing, use the following procedure to check the implications on
performance.
Procedure
1. Wait 7 days, for a new traffic history to be available.
2. Balance Load on page 242.
3. Compare the old load balancing results and performance statistics to the new load
balancing results and performance statistics.

Collect System Information


Use Admin > System Analysis Tools > Collect System Information to retrieve system log
files and environment information for support purposes.

Collect System Information

247

Admin

How system information collection works


To analyze and resolve support cases, you can Collect system information on page 248
for the past thirty days or you can Collect system information around a specific event on
page 50. System information is collected using the Collect System Information wizard.

Collect system information


Before you begin
l

In order to collect system information for VNX and VPLEX splitters, you must first
Attach volumes to the splitter on page 202.
Errors will occur in the following cases:
If connection with an RPA is lost while info collection is in process, no information
is collected.
n
In this case, run the process again. If the collection from the remote site failed
because of a WAN failure, run the process locally at the remote site.
n
If a simultaneous info collection process is being performed on the same RPA,
only the collector that established the first connection can succeed.
n
If an FTP failure occurs, the entire process fails.
The system information collected from RPAs also includes Dell OMSA hardware
configuration information, provided the RPAs are running on Dell PowerEdge
platforms.
To retrieve the output file containing the system information, you must log in as one
of the pre-defined users on page 226 with webdownload permissions on page 231.
It is not possible to collect VNXe system information remotely through the RPA.
Collect VNXe and RecoverPoint VNXe splitter system information manually using the
VNXe Unisphere. VNXe credentials are required for system information collection.

Procedure
1. Read Attach volumes to the splitter on page 202 for all splitters in the system.
2. Select Admin > System Analysis Tools > Collect System Information.
3. If this is your first time collecting system information, click Start. Otherwise, click
Start New Collection Process.
The Collect System Information wizard is displayed, to guide you through the
collection process.

248

RecoverPoint 4.4 Administrator's Guide

Admin

NOTICE

You can also collect system information around a specific event by selecting
Dashboard > System Events. Then select an event in the event log and click Collect
System Information. The Collect System Information wizard is displayed and the
collection start and end fields are automatically set to 10 minutes before and after the
selected event.
4. Define the system information collection settings on page 249.
Table 58 System information collection settings

Setting

Description

From

Default=yesterday, the current hour


Specify the date and time (in GMT or your local time) of the first system
information that you want to include in the output.
Although the system information of the past thirty days is available for
collection, only three days of system information can be collected at a
time.
Note

GMT is not adjusted for daylight savings time.


To

Default=today, the current hour


Specify the date and time (in GMT or your local time) of the latest system
information that you want to include in the output.
Although the system information of the past thirty days is available for
collection, only three days of system information can be collected at a
time.
Note

GMT is not adjusted for daylight savings time.


Include Information
from RPA Clusters

Optional
Default=selected
When selected, enables the collection of system information from RPA
clusters.

RPA clusters

Optional
Default=all RPA clusters
The RPA clusters from which to collect system information. If none of the
checkboxes are selected, only the system information of the RPA cluster
from which the collection process is triggered is collected.
Note

Only relevant when Include Information from RPA Clusters is selected.


Include core files

Optional
Default=disabled

Collect System Information

249

Admin

Table 58 System information collection settings (continued)

Setting

Description
Whether or not to include core files in the output. Core files may be large.
Subsequently, including these files in the collection process may
substantially increase collection time.
Note

Only relevant when Include Information from RPA Clusters is selected.


Include Information
from Splitters

Optional
Default=selected
When selected, enables the collection of system information from the
selected splitters.
Note

Attach volumes to the splitter on page 202 before system information


can be collected.
Note

VNXe splitter system information must be collected manually from the


array.
Place the output file(s) Optional
on my local RPA
Default=enabled
When selected, copies the collection process output file(s) to the RPA
from which the collection process was triggered.
The output file(s) are placed in: http://[RPA IP address]/info or https://
[RPA IP address]/info.
Copy the output file(s) Optional
to an FTP server
Default=disabled
When selected, copies the collection process output file to the specified
FTP server.
When Copy the output file(s) to an FTP server is selected
FTP Server Address

The IP or DNS address of the FTP server to which to upload the collection
process output file.
For example: 10.10.180.10 or ftp.EMC.com

Port

The port through which to access the specified FTP server.

User Name

The username to use when logging into the specified FTP server.

Password

The password to use when logging into the specified FTP server.

Remote Path

Optional
The path to the copy of the output file stored on the specified FTP server.
For example: / (to access the rootdir)

Override default
filename (Not
recommended)
250

RecoverPoint 4.4 Administrator's Guide

Optional
Default=disabled

Admin

Table 58 System information collection settings (continued)

Setting

Description
Whether to override the default file name of the output file placed on the
FTP server.
Note

It is recommended to keep the default file name.


When selected, RecoverPoint renames the output file uploaded to the
FTP server according to the new file name specified in the Filename field.
New File Name

Optional
The new file name for the output file placed on the FTP server.
Note

Only relevant when Override default filename is selected. It is


recommended to clear the Override default filename checkbox, and keep
the default file name as is.

5. Click Next Test Connectivity. The Test Connectivity screen is displayed.

6. In The Test Connectivity screen, review the displayed information to verify that your
configuration settings are correct, and the system can access all RPAs and splitters:
l

Verify that the Collection Settings are configured as intended. If required, click
Back to Define Settings to edit your settings.

Verify that a green check mark exists in the Status column of all listed splitters and
RPAs, and that the text Action succeeded exists in the Details column of all listed
splitters and RPAs. If all splitters and RPAs do not have green check marks in their
Status columns, you can collect the information directly from individual hosts on
which the feature is enabled.
n

for Windows-based hosts; from the Program Files\KDriver\hic


directory on the host, run:
host_info_collector
Collect System Information

251

Admin

for Solaris or AIX-based hosts; from the kdriver/info_collector


directory on the host, run:
info_collect.sh

7. Click the Finish button to initiate the collection process. A progress bar is displayed,
and the collection process begins.

Note

Be patient. The collection process can take awhile, depending on the time span and
the amount of data being collected.
8. During the collection process, you have the following options:
Table 59 Collect system information options

Option

Description

Start New Collection


Process

To stop the current collection process and display the screen


containing the collection settings.
Note

Only available while collection is in progress.


Stop Collecting System
Information

To stop the current collection process.


Note

Only available while collection is in progress.


Display Results

To open a window containing the collected system information.


Note

Only available when collection is complete.

9. After the collection process is complete, click Display Results.


252

RecoverPoint 4.4 Administrator's Guide

Admin

10.In the Collection Results dialog box:


a. Verify that the specified system information has been successfully collected. Verify
that a green check mark exists in the Status column of all listed splitters and RPAs,
and that the text Action succeeded exists in the Details column of all splitters and
RPAs. If all splitters and RPAs do not have green check marks in their Status
columns, you can collect the information directly from individual hosts on which
the feature is enabled.
l

for Windows-based hosts; from the


Program Files\KDriver\hic directory on the host, run:
host_info_collector

for Solaris or AIX-based hosts; from the kdriver/info_collector


directory on the host, run:
info_collect.sh

b. Retrieve the system information. Click the Output file (HTTP) link, or Output file
(HTTPS) link, and enter the username and password of a pre-defined users on page
226 with webdownload permissions on page 231, and click Close.
Upon completion of the collection process, the output file is placed in: http://[RPA
IP address]/info or https://[RPA IP address]/info.
11.If you enabled the Copy output file to an FTP server option in step 4 on page 249, you
can now retrieve the remote copy of the output file from the FTP server:
a. Open a Web browser window.
b. Enter the FTP Server Address or DNS name of the FTP server you specified in
step 4 on page 249 into the address bar.
c. At the login prompt, enter the User Name and Password you specified in step 4 on
page 249.
d. Browse to the Remote Path that you specified in step 4 on page 249.

System Notifications
Use the System Notifications menu to manage event filters, configure system reports and
alerts and manage call home events.
System Notifications

253

Admin

Manage Event Filters


Use Admin > System Notifications > Manage Event Filters to define and modify the
settings for the sending of outbound system events through email, SNMP, and/or Syslog.

Display all event filters


To display all of the event filters currently defined in the system, select Admin > System
Notifications > Manage Event Filters.

254

RecoverPoint 4.4 Administrator's Guide

Admin

Create an event filter


Procedure
1. Select Admin > System Notifications > Manage Event > Filters.
2. Click Add. The Event Filter wizard is displayed.
3. In the Define Filter screen:

a. Define the event filter settings according to the instructions in the following table.
Table 60 Event Filter settings

Setting

Description

Filter Name

Enter a unique name of the event filter to be created.

Topic

Select which events to report, according to the component of the


RecoverPoint system on which the events occur:

Groups

Level

All Topics

Site

RPA

Consistency Group

Splitter

Only relevant when Topic is set to All Topics or Consistency Group.


Select the names of the groups that are to be subject to this filter. There is
no default value for this parameter.
l

Info: Event is informative in nature, usually referring to changes in the


configuration or normal system state. When Level=Info, events of all
levels are returned; there is no filtering of events as a result of the
Level setting.

Warning: Event indicates a warning, usually referring to a transient


state or an abnormal condition that does not degrade system
performance.When Level=Warning, all events with level of Warning or
Manage Event Filters

255

Admin

Table 60 Event Filter settings (continued)

Setting

Description
Error are returned; Info-level events are filtered. The Warning level
includes Warning Off events.

Scope

Event IDs to
Exclude

Error: Event indicates an important event that is likely to disrupt normal


system behavior and/or performance. When Level=Error, only events at
the Error level are returned; Warning-level and Info-level events are
filtered. The Error level includes Error Off and Brief Error events.

Normal: To report selected basic events and root-cause events. A rootcause event returns a single description for a system event that can
spawn an entire set of detailed and advanced events. When
Scope=Normal, all of those detailed and advanced events are filtered
out, leaving only the normal events. This is the default setting, and in
most cases, it is sufficient for effective monitoring of system behavior.

Detailed: This category includes all events (that is, detailed and
normal), with respect to all components, that are generated for use by
users.

Advanced: In specific cases (for instance, for troubleshooting a


problem) EMC Customer Support may ask you to retrieve information
from the advanced log events. These events contain information that is
intended primarily for the technical support engineers.

The IDs of the Events that are not subject to this filter.
Note

Use a comma to separate multiple event IDs.

b. Click Next Define Method of Transfer.


4. In the Define Method of Transfer screen:

256

RecoverPoint 4.4 Administrator's Guide

Admin

a. Select one or more methods of transfer.


b. If you selected Email:
l

Select the frequency with which the events will be sent.

Enter the email addresses of all recipients in a comma-separated list.

c. Click Finish to save the event filter and exit the wizard.
5. Configure the transfer method on page 257 for each method of transfer that you
defined.

Configure the transfer method


Events can be transferred to the specified servers and/or email addresses by email,
syslog or SNMP.

Configure Email event notification


Use the following table to enable the transfer of events by email.
Procedure
1. Select Admin > System Notifications > Manage Event > Filters.
2. In the bottom pane, select the Email tab.

3. Enter the values for the fields in the Email tab according to the instructions in the
following table.
Table 61 Email settings

Setting

Description

Enable Email
Notifications

Select to enable the events defined in any event filters whose method of transfer
is set to Email to be transferred to the specified server from the specified sender.
Note

Selecting this checkbox also enables the sending of email alerts to the specified
server address if you enabled both reports and alerts in the Configure System
Reports and Alerts on page 264 screen.
SMTP server
address

Enter the IP address or DNS name of the server to which to send the emails.

Manage Event Filters

257

Admin

Table 61 Email settings (continued)

Setting

Description
Note

When you click Apply, any change to this field automatically updates the SMTP
server address field in the Configure System Reports and Alerts on page 264
screen.
Sender
address

Enter the email address that should appear in the sender field of all emails
concerning sent to the specified server regarding event notifications.

4. Click Apply.
Results
All of the event filters whose Method of Transfer includes Email will be transferred to the
recipients defined in the filter (at the frequency defined in the filter) using the specified
server address and sender.

Configure Syslog event notification


Use the following procedure to enable the transfer of events by syslog.
Procedure
1. Select Admin > System Notifications > Manage Event Filters.
2. In the bottom pane, select the Syslog tab.

3. Enter the values for the fields in the Syslog tab according to the following table.

258

RecoverPoint 4.4 Administrator's Guide

Admin

Table 62 Syslog settings

Setting

Description

Enable Syslog
Notifications

Select to enable the events defined in any event filters whose method of transfer
is set to Syslog to be transferred to the specified servers with the specified label.

Facility

Select one of the available labels to be attached to all messages.

Specify target
host for

(Optional) Specify the syslog server at each RPA cluster to which you wish to
deliver notifications. The address may be either in IP or DNS format. A DNS
address will work only if a DNS server is configured in the RecoverPoint system.

4. Click Apply.
Results
All of the event filters whose Method of Transfer includes Syslog will be transferred by
Syslog using the specified target host and facilities.

Configure SNMP event notification


Use the following procedure to enable the transfer of events by SNMP.
RecoverPoint supports the standard Simple Network Management Protocol (SNMP),
including support for SNMP version 3 (SNMPv3). RecoverPoint supports various SNMP
queries and can be configured to generate SNMP traps (notification events), which are
sent to designated network management systems.
Procedure
1. Select Admin > System Notifications > Manage Event Filters.
2. In the bottom pane, select the SNMP tab.

3. Enter the values in the fields of the SNMP tab according to the following table.
Table 63 SNMP settings

Setting

Description

Enable SNMP
Agent

Select to enable the RecoverPoint SNMP agent to send all of the events defined
in any of the event filters whose method of transfer is set to SNMP to the
specified servers with the specified label.
Manage Event Filters

259

Admin

Table 63 SNMP settings (continued)

Setting

Description
The RecoverPoint SNMP agent must be enabled in order to send SNMP traps
(notification events) and to respond to host-initiated SNMP queries.

Allow secure
transports
only

When selected, only encrypted SNMP queries over secure transport will be
enabled; use of community strings and SNMP over an unsecured port will be
disabled.

Send event
traps

When selected, sends SNMP traps (notification events) to the RecoverPoint


SNMP agent according to your SNMP trap configuration on page 261. Does not
affect host queries.

Specify trap
(Optional) The network management server to which you wish to deliver
destination for notifications. The address may be either in IP or DNS format. A DNS address will
work only if a DNS server is configured in the RecoverPoint system.
SNMP v1 RO
community

If you are using SNMP version 1, enter an SNMPv1 read-only community string to
enable the use of SNMPv1 community strings. If not enabled, only SNMPv3 user
names with passwords or certificates are enabled.
Note

The read-only community string is transmitted in cleartext.

4. To add an SNMPv3 user:


a. Click Add in the SNMPv3 Users area. The Add SNMPv3 User dialog box is
displayed.

b. Enter the required information.


c. To add a certificate, click Browse and select the certificate. The certificate file can
contain more than one certificate; select one. All certificates in the file are equally
valid. For optimal security, specify a user certificate and do not specify a password.
5. To remove an SNMPv3 user, select the user in the SNMPv3 Users table and click
Remove.
6. Click Apply to save your changes.
7. Perform SNMP trap configuration on page 261.

260

RecoverPoint 4.4 Administrator's Guide

Admin

SNMP trap configuration


RecoverPoint supports the default MIB-II. The RecoverPoint MIB can be downloaded from
powerlink.emc.com at the following location:
Home > Support > Software Downloads and Licensing > Downloads PR > RecoverPoint
The application MIB OID is:
1.3.6.1.4.1.21658
The trap identifiers for RecoverPoint traps are as follows:
l

Info

Warning

Error

The RecoverPoint SNMP trap variables and their possible values are listed in the
following table.
Table 64 RecoverPoint SNMP trap variables

Variable

OID

Description

dateAndTime

3.1.1.1

Date and time that trap was sent

eventID

3.1.1.2

Unique event identifier; the values are listed in the Events chapter
of this Administrators Guide.

siteName

3.1.1.3

Name of site where event occurred

eventLevel

3.1.1.4

Valid values are:

eventTopic

3.1.1.5

info

warning

warning off

error

error off

Valid values are:


l

site

K-Box

group

splitter

management

hostName

3.1.1.6

Name of host

kboxName

3.1.1.7

Name of K Box

volumeName

3.1.1.8

Name of volume

groupName

3.1.1.9

Name of group

eventSummary

3.1.1.10 Short description of event

eventDescription 3.1.1.11 More detailed description of event

Manage Event Filters

261

Admin

Note

OpenManage Server Administrator (OMSA) support is available only for systems in which
the RPAs are running on Dell PowerEdge platforms.
OMSA support provides RecoverPoint customers with the ability to:
l

Display Dell hardware event notifications, together with RecoverPoint event


notifications, in real-time, through the same management console. The instructions
for this procedure follow.

Collect System Information on page 247 that includes Dell hardware configuration
information.

RecoverPoint generates events that result in Simple Network Management Protocol


(SNMP) traps. When an event with predefined characteristics (defined in RecoverPoint
and OMSA MIBs) occurs on your system, the SNMP subagent sends information about the
event, along with trap variables, to a specified management console.
Use the following procedure to view OMSA and RecoverPoint events in real-time.
Procedure
1. Configure SNMP event notification on page 259 to send event traps to your event
management console. Ensure that the Enable SNMP Agent and Send event traps
checkboxes are checked and that the IP address of the machine that has been
dedicated as the event management console is defined as a Trap destination.
2. Install a MIB browser on the machine dedicated as the event management console.
Note

MIB browsers are used to manage SNMP-enabled devices and applications in a


network. MIB browsers enable users to load MIBs, issue SNMP requests, and receive
SNMP traps.
3. Open the MIB Browser on your management console, and
a. Enter the cluster management IP of the RPA cluster into the address bar. The
cluster management IP is a virtual IP address assigned to the RPA that is currently
active in the cluster. In the event that this RPA fails, the cluster management IP
address dynamically switches to the RPA that assumes operation.
b. Enable the MIB browsers Trap Receiver. In the Trap Receiver, both the
RecoverPoint and OMSA events are displayed in real-time. RecoverPoint events are
preceded with their severity level (Info, Warning, or Error), the Dell OMSA event
OIDs have the prefix 1.3.6.1.4.1.674.
After you finish
See the Dell OpenManage Server Administrator Version 1.9 SNMP Reference Guide,
which can be found at the Dell website:
http://support.dell.com/support/edocs/software/svradmin/1.9.2/en/SNMP/

Modify an existing event filter


Procedure
1. Select Admin > System Notifications > Manage Event Filters.

262

RecoverPoint 4.4 Administrator's Guide

Admin

2. Select an event filter in the table.


3. Click Edit. The Event Filter wizard is displayed, with the current settings for the
selected event filter.

4. Modify the event filter settings according to the instructions in Create an event
filter on page 255.
5. Click Next Define Method of Transfer.
6. If necessary, in the Define Method of Transfer screen, modify the methods of transfer.
For each new method of transfer that you define, Configure the transfer method on
page 257.
7. Click Finish to save the new event filter settings and exit the wizard.

Remove an existing event filter


Procedure
1. Select Admin > System Notifications > Manage Event > Filters.
Manage Event Filters

263

Admin

2. Select an event filter in the table.


3. Click Remove, and then, click Yes in the confirmation dialog box.

Configure System Reports and Alerts


Use Admin > System Notifications > Configure System Reports and Alerts to modify the
system report and alert settings, define or edit a transfer method, and test your settings
to ensure that they work. You can also use this screen to configure notification of new
RecoverPoint software releases.

Depending on the severity of the event, system alerts will automatically open a service
request with EMC Customer Support and will attach the latest system report to the service
request. System alerts are sent in real-time (at the time that the event occurs), allowing
EMC to provide preemptive support for RecoverPoint issues.
System reports contain the configuration and state of the RecoverPoint system, and they
are sent to the EMC system report (SyR) database every Sunday. Only one system report
is sent at a time (containing the information of all RPA clusters) unless there are
connectivity issues between RPA clusters, or an RPA cluster in the RecoverPoint system is
unable to send a system report. In this case, the problematic RPA cluster will send its own
system report. All system reports contain the name of the RPA cluster that they are sent
from.
The following parameters are sent to the EMC system report (SyR) database and included
in any opened service requests:

264

RecoverPoint 4.4 Administrator's Guide

Admin

Table 65 System parameters sent to SyR

Parameter

Description

System ID

The unique identifier of all activated RPA clusters in a RecoverPoint system. The
System ID:
l

Is generated for the first RPA cluster that is added to the system during
RecoverPoint deployment. When adding additional RPA clusters to the
system, the additional RPA clusters receive the System ID of the first RPA
cluster.

Does not change with upgrades. When upgrading to RecoverPoint 4.0 from
previous RecoverPoint versions, the System ID value is taken from the value
of Account ID in the previous RecoverPoint version.

Can be displayed by running the get_account_settings CLI command,


and can only be modified by EMC Customer Support.

Also known as: Account ID


SSID

The Software Serial ID. The unique identifier of an RPA cluster in the RecoverPoint
system. The SSID is the identification used by the install base to support EMC
equipment installed at customer sites, through the system reporting and ESRS
mechanisms.
Note

The format and function of the SSID has changed from previous RecoverPoint
versions.
The SSID is in the format <SN>EID<RPA cluster ID>, where <SN> and <RPA cluster ID>
are described in the licensing section of the EMC RecoverPoint Product Guide. Its
length can be no more than 22 characters, and therefore, the RPA cluster ID will
be truncated as required. The SSID:

Site ID

Is generated for each RPA cluster when the first license related to the RPA
cluster is added to the system. When additional licenses are added to the
system, the SSID of the RPA cluster does not change.

If no license was added for an RPA cluster, the SSID of that RPA cluster will
be empty.

Can be displayed by running the get_system_settings CLI command, and can


only be modified by EMC Customer Support.

Does not change with upgrades. When upgrading to RecoverPoint 4.0 from
previous RecoverPoint versions, the SSID value is taken from the value of the
SSIDs in the previous RecoverPoint version.

Does not change even if the SSID SN from which it was generated no longer
exists in any of the installed licenses.

The ID used in various EMC global processes as a unique identifier of the


physical location of an RPA cluster.
The Site ID:
l

Is obtained from the value of the <Site_Info> parameter in the license file
variables. Its value is set by the first installed license for an RPA cluster.

Configure System Reports and Alerts

265

Admin

Table 65 System parameters sent to SyR (continued)

Parameter

Description
Note

If no license was installed for an RPA cluster, the Site ID of that RPA cluster
will be empty.
l

Can be displayed by running the get_system_settings CLI command,


and can only be modified by EMC Customer Support.

Does not change even if the <Site_Info> from which it was generated no longer
exists in any of the installed licenses.

Configure system reports and alerts


Before you begin
l

Best practice is to keep both system reports and alerts, and compression and
encryption enabled.

System reports and alerts will only be sent provided a method of transfer (SMTP,
ESRS, or FTPS) is configured. You can configure the method of transfer using the
Getting Started Wizard or by selecting Admin > System Notifications > Configure
System Reports and Alerts.

To transfer system reports and alerts using SMTP or ESRS, ensure that port 25 is open
and available for SMTP traffic.

To transfer system reports and alerts using FTPS, ensure that ports 990 and 989 are
open and available for FTPS traffic.

To send system reports and alerts, a permanent RecoverPoint license must exist in
the system; system reports and alerts will not work with a temporary license.

By default:
l

Both system reports and system alerts are enabled.

System reports and alerts are sent through SMTP, but they can be configured for
transfer through a customer's ESRS server, or RecoverPoints built-in FTPS server.

System reports and alerts are compressed and encrypted with RSA encryption using a
256-bit key before they are sent.

Procedure
1. Select Admin > System Notifications > Configure System Reports and Alerts.

266

RecoverPoint 4.4 Administrator's Guide

Admin

2. Ensure that the Enable System Reports and Alerts checkbox is selected, and the radio
button is set to System reports and system alerts .
3. Ensure that the Compression and Encryption checkboxes are selected.
4. Define the transfer method.
l

To transfer system notifications through an SMTP server:


a. In the Transfer Method section, ensure the SMTP radio button is selected.
b. In the SMTP server address field, specify the IP address or DNS name of your
dedicated SMTP server, in IPv4 or IPv6 format.
c. In the Sender email field, specify the email address from which the system
notifications will be sent.
d. Click Apply.

To transfer system notifications through RecoverPoints built-in FTPS server:


a. In the Transfer Method section, select the FTPS radio button.
b. Click Apply.

To transfer system notifications through an ESRS gateway:


a. In the Transfer Method section, select the ESRS radio button.
b. In the ESRS gateway IP address field, specify the IP address of your ESRS
gateway in IPv4 format.
c. Click Apply.

Configure notification of new RecoverPoint software releases


This feature is enabled by default.
Procedure
1. Select Admin > System Notifications > Configure System Reports and Alerts.

Configure System Reports and Alerts

267

Admin

2. Select or clear the Enable New Release Notification checkbox.

Ensure that system reports are configured correctly


Use the following procedure to ensure that system reports can be sent using the
configured transfer method.
Procedure
1. Configure system reports and alerts on page 266.
2. Click Test Connectivity.
3. In the Test Connectivity dialog box, enter the email address to which EMC Customer
Support can send a verification email.

Note

The connectivity test, if successful, opens a service request containing call home
event number 30999 and sends an email to the specified verification email address
from EMC Customer Support to verify that the system reports mechanism (SyR) has
been successfully configured.
4. Wait ten minutes.
5. Select Dashboard > System Events and look for event 1020 Failed to send system
report.

268

If this event does not appear in your event logs, the system notifications
mechanism has been properly configured.

If you do receive event 1020; check whether there is an issue with your selected
method of transfer. If a problem exists, fix it, Configure system reports and
alerts on page 266 and click Test Connectivity again. If problem persists contact
EMC Customer Support.

RecoverPoint 4.4 Administrator's Guide

Admin

Display the current system report


To view the current system report information, run the get_system_report CLI
command.

Send the current system report to an email address


To send the current system report information to a specified e-mail address, run the
get_system_report CLI command.

Display the current system report settings


To display the current system notification settings, run the
get_system_report_settings CLI command.

Manage Call Home Events


Use Admin > System Notifications > Manage Call Home Events to view and suppress the
events upon which a service request is automatically opened with EMC Customer
Support.

RecoverPoints call home feature is a proactive online service capability of RPAs that
enables RPAs to continuously monitor their own health, and the health of the
RecoverPoint system, using a pre-defined set of event-filtering rules. The call home event
mechanism automatically notifies EMC Customer Support of potential issues, using
secure transmissions. If a serious problem arises, the call home mechanism
automatically detects it and opens a service request with EMC Customer Support. When
the service request is received, EMC Customer Support pro-actively engages the relevant
EMC personnel to start working on the issues resolve, and immediately start working with
the relevant customer parties, or uses a configured ESRS gateway, to resolve the issue.
The call home event mechanism is customizable so that any specific environment or
specific customer requirement can be accommodated. EMC Customer Support have the
ability to create additional call home events, as required, to meet a specific customer's
needs. They can also permanently disable or temporarily suppress call home events. For
example, if an RPA cluster keeps reporting a problem that EMC Customer Support are
already aware of and have scheduled a customer visit to fix next week, the event
reporting the issue can be suppressed for a period of 2 weeks.

How call home events work


The call home events mechanism works in the following way, whenever the logic in the
call home event script is fulfilled:
1. RecoverPoint event number 8000 "Calling home" is displayed in the event log,
and in parallel; a call home event is sent to SyR.
2. SyR opens an SR (Service Request) with EMC Customer Support, and relays the
following information:
Manage Call Home Events

269

Admin

The ID of the call home event is sent as the symptom code.

The event's Severity.

A description of the event scripts logic.

The email address and telephone number defined in the contact_info


parameter of the config_user_account CLI command.
Note

If contact information is not specified, and a call home event is triggered, a


Service Request is still opened, but it will not contain this information, which will
elongate the cases time to resolve.
3. EMC Customer Support engages the customer (or uses an ESRS Gateway) to proactively resolve the issue.
Note

Whenever the logic of a call home event script cannot be fulfilled, RecoverPoint event
8001 is generated in the event logs.

Before you begin

270

RecoverPoint ships with a set of pre-defined call home events that are enabled by
default.

The call home events mechanism is not available when the RPA running the Cluster
control is down.

The call home events mechanism is disabled during upgrade and while RecoverPoint
is in maintenance mode. The call home events mechanism discards any events
issued while the system is in maintenance mode.

The call home events, and the last call home time for each event, will survive RPA
failure. In the case of a disaster, all of the user-defined call home events, and any call
home events that may have been logged up until the time of the disaster, are
retrievable. This information will not, however, survive repository volume format.

When upgrading to a new RecoverPoint version that introduces no change in the


system-defined call home events, the existing system-defined rules are preserved.
When upgrading to a new RecoverPoint version that does introduce a change in the
system-defined call home events, the new system-defined events will override the old
ones.

The call home event mechanism is based upon with the most current system settings.
Therefore, if one of the rules in a call home event references a system property (e.g.
replication type), the value of the property must be the current value of the property
in the system settings. The call home mechanism does not save previous settings. If a
rule refers to a system entity (e.g. group, splitter, etc), and the entity has been
deleted from the system (and therefore, the system settings), no call home event is
issued.

The call home events mechanism automatically includes the contact information
defined in the contact_info parameter of the config_user_account CLI
command in the Service Request which is opened with EMC Customer Support. In
order to ensure a swift turnaround time, ensure that the contact_info parameter
contains both the email address and the telephone number of the person at your
company that is designated to mitigate RecoverPoint support situations. If contact
information is not specified and a call home event is triggered, a Service Request is

RecoverPoint 4.4 Administrator's Guide

Admin

still opened, but it will not contain this information, which will elongate the cases
time to resolve.

Display all call home events


To display all of the call home events in the system, select Admin > System Notifications >
Manage Call Home Events and select an event in the table to display the call home event
script, description and status in the bottom pane.

Suppress one call home event until a specific date and time
Use the following procedure to suppress one call home event for a specific period of time.
Procedure
1. Select Admin > System Notifications > Manage Call Home Events
2. Select the event.
3. In the Status area of the bottom pane, select the Suppress Until radio button, and set
the date and time fields.
4. Click Apply.
NOTICE

Note the change in the event status in the Status column of the event table. To enable
suppressed events, select the event in the table, change the date and time fields to
the current date and time, and click Apply.

Suppress all call home events until a specific date and time
Use the following procedure to suppress all call home events for a specific period of time.
Procedure
1. Select Admin > System Notifications > Manage Call Home Events
Manage Call Home Events

271

Admin

2. Select the event.


3. Click the Suppress All button under the event list. The Suppress All dialog box is
displayed.

4. In the Suppress All dialog box, set the date and time fields, and click OK.
NOTICE

Note the change in the status of all of the events in the Status column of the event
table. To enable all of the suppressed events, click Suppress All, set the date and
time fields to the current date and time, and click OK.

Modify call home events


To modify call home events, contact EMC Customer Support.

272

RecoverPoint 4.4 Administrator's Guide

CHAPTER 7
Support

This section describes all of the RecoverPoint support options and how to quickly access
your Recoverpoint documentation, product support, or online community of users from
within Unisphere for RecoverPoint.
l

RecoverPoint Support..........................................................................................274

Support

273

Support

RecoverPoint Support
Unisphere for RecoverPoint product support is accessed through the Support menu.

Table 66 Support options

Option

When to Use

RecoverPoint
Documentation

Display the RecoverPoint documentation on EMC Online Support.

EMC Online
Support

Download the latest EMC products, get and manage licenses, open and track
service requests, chat with EMC technical support, view and manage your
company information, or contact EMC.

The
RecoverPoint
Community

Interact with other RecoverPoint users and share best practices and tips, or
communicate directly with the experts responsible for the EMC RecoverPoint
family.

RecoverPoint Documentation
Click Support > RecoverPoint Documentation to display the RecoverPoint documentation
on EMC Online Support.

When clicked, the RecoverPoint documentation page is displayed, enabling you to search
for the RecoverPoint documentation that you require by RecoverPoint version, content
type, support task or OS.

274

RecoverPoint 4.4 Administrator's Guide

Support

Figure 29 RecoverPoint Documentation page on EMC Online Support

EMC Online Support


Click Group > EMC Online Support to download EMC products, manage licenses, open
service requests, chat with EMC technical support, or contact EMC.

When clicked, the EMC Online Support page is displayed, enabling you to:

EMC Online Support

275

Support

Figure 30 EMC Online Support page


l

Register your product to access software downloads or obtain and manage license
keys.

Track, view, manage and create service requests for yourself and your company.

Engage with EMC Technical Support for answers to your questions.

View and manage your company information - Sites, products, contacts and
contracts.

Manage site preferences to guide your service center experience.

Download the new Mobile App to manage and track your Service Requests anytime,
anywhere.

Contact EMC

Learn about EMC's Maintenance, Warranty Information and Parts Return Site.

Learn how to fully leverage the capabilities of the new EMC experience.

Gain one-stop access to technical and support-related information by EMC product.

Download the latest EMC products and patches.

Connect with EMC customers and technical experts in this worldwide community of
your peers.

Submit your ideas for improving serviceability on EMC products. SER is transparent
data repository where all requests can be submitted, managed and tracked.

The RecoverPoint Community


Click Support > The RecoverPoint Community to interact with other RecoverPoint users
and share best practices and tips, or communicate directly with the experts responsible
for the EMC RecoverPoint family.

276

RecoverPoint 4.4 Administrator's Guide

Support

When clicked, the RecoverPoint community page is displayed, enabling you to join the
community, start a discussion or view feeds.

Figure 31 The RecoverPoint Community page

The RecoverPoint Community

277

Support

278

RecoverPoint 4.4 Administrator's Guide

CHAPTER 8
Troubleshooting

This section describes occurrences in RecoverPoint, how to identify them, and how to
mitigate them from a users perspective.
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l

First-time initialization is taking too long............................................................. 280


Host applications are hanging.............................................................................281
One-phase distribution workarounds.................................................................. 282
Copy is being regulated.......................................................................................282
Copy is experiencing a high load......................................................................... 283
RPA keeps rebooting........................................................................................... 286
Host applications crash during initialization........................................................287
Add more LUNs than allowed per VNX storage group........................................... 290
Swap the LUN numbers of journal volumes..........................................................291
Find your array serial numbers.............................................................................291
Find your RPA cluster IDs..................................................................................... 293
Find your XtremIO cluster IDs...............................................................................293
Understanding the RecoverPoint license............................................................. 294
Calculate your required journal size.....................................................................297
Validate replication integrity............................................................................... 304
MetroPoint group conversion.............................................................................. 308

Troubleshooting

279

Troubleshooting

First-time initialization is taking too long


You can save both time and bandwidth by initializing copy volumes from a backup, when
they are first added to the system.

How does it work


When a consistency group is initialized for the first time, the RecoverPoint system cannot
identify which blocks are identical between the production and copy volumes, and must
therefore mark all blocks for that volume. This is true both following the creation of a new
consistency group and following the addition of a volume (or volumes) to an existing
group. While initializing, the RecoverPoint system efficiently determines which blocks are
actually different between the production and copies, and sends only the data for those
blocks to the copy storage, as the initialization snapshot.
The volumes at the local and remote RPA clusters can be initialized while the host
applications are either running or not. Initialization of one consistency group does not
interfere with the operation of other consistency groups. Initialization can be carried out
automatically over IP or Fibre Channel.
Alternatively, you can back up the current production data, manually transfer it to the
remote cluster, and then copy the production image to the copy volumes, where it
becomes the pre-replication image.
The state of data transfer prior to any type of initialization is always paused.
NOTICE

First-time initialization can cause changes in the partition table of the copy volume or
volumes. In a Windows environment, you must clear the OS cache before changing the
partition table of a copy volume. To clear the OS cache, disable the LUN or LUNs on which
the volume resides, and re-enable it. You can do so from the Disk drives interface of the
Windows Device Manager.
You can initialize new consistency groups by creating a backup of your production
volumes, physically transferring them to the remote site, and copying the backup images
onto the remote storage volumes.
During this process, applications at the production site can be running or not. Although
transfer is paused during this process, unless the production host is completely shut
down during the creation and physical transfer of the production image to the remote
site, the host applications keep writing to storage. In this time, the production and copy
volumes become inconsistent, and any writes to production volumes during this process
are subsequently synchronized, upon completion of the process, and start of transfer.
Only synchronizing the changes made to the production volumes during this time results
in a relatively small amount of additional traffic, and takes substantially less time than to
synchronize over IP or Fibre Channel.

Initialize from backup


Use the following procedure to initialize from a backup image, when the production and
copy volumes are not consistent.
Procedure
1. At the production RPA cluster Attach volumes to the splitter on page 202 and ensure
that all splitters that have access to replication volumes in the group are attached to
those volumes.
280

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

2. Create a new consistency group, and define replication and journal volumes using
Protect Volumes on page 53 or Policy Templates on page 68.
3. Pause transfer on page 128 for the group, to stop the replication of data from
production to the copy.
4. In the RecoverPoint CLI, run the clear_markers command to inform the system
that the copy volume at the remote RPA cluster is known to be identical to its
corresponding production volume, and a full volume sweep synchronization is not
required.
NOTICE

For the selected consistency groups, clearing markers clears all markers of the
selected copy from the production journal volume; that is, it treats the selected copy
as identical to the production source. Caution: To clear markers, the production
source and the selected copy volume must be absolutely identical. If they are not, the
inconsistencies will remain. The clear markers command should be used only with
extreme caution. It is useful when a production source and a copy have been
synchronized manually by initializing from backup and adequate bandwidth is not
available to synchronize using the storage network. Otherwise, the best practice is not
to use this command.
5. Create a block-based backup of the production volumes.
6. Physically transfer the backup to the remote cluster.
7. At the remote RPA cluster, in Unisphere for RecoverPoint:
a. Test a Copy on page 147 to enable image access to the image at the remote RPA
cluster in Logged Accessmode, and enable direct access. Do not select the prereplication image.
If your consistency group contains more than one non-production copy, for all
other copies, the journal is erased and all journal history is lost.
b. Click Finish to close the wizard without disabling image access.
8. Restore the backup onto the remote copy volumes. The backup image becomes the
pre-replication image.
9. In Unisphere for RecoverPoint:
a. Select Recovery > Manage Recovery.
b. Select Finish Testing from the Select Next Action combo box of the accessed copy
at the remote RPA cluster, to disable image access and start transfer to the copy.
Upon start of transfer, the system synchronizes the data of the copy volume with
the corresponding production data.
10.At the remote RPA cluster, if the group contains more than one non-production copy,
Start transfer on page 131 to all of the other copies.

Host applications are hanging


Some RecoverPoint users set a policy that enables RecoverPoint to control the
acknowledgment of writes back to the host in the case of bottlenecks or insufficient
resources that would otherwise prevent RecoverPoint from replicating the data.

Host applications are hanging

281

Troubleshooting

If your host applications experience delays, loss of client connectivity, or slow response
times, check whether the value of the regulate_application parameter in the
config_link_policy CLI command is set to yes (default is no).

When does application regulation happen?


Application regulation happens when a user enables the regulate_application
parameter in the config_link_policy CLI command.

How does application regulation work?


The system slows host applications when approaching the RPO policy limit. When the
system cannot replicate the current incoming write-rate while guaranteeing the RPO on
page 84 setting, the system slows host applications to guarantee that the RPO is always
enforced. Additionally, if there is a bottleneck in the system, the system will regulate the
host applications instead of entering a High load state.

How do I know application regulation is happening?


If your host applications experience delays, loss of client connectivity, or slow response
times, check whether the value of the regulate_application parameter in the
config_link_policy CLI command is set to yes. If it is, your host applications are
being regulated to ensure your RPO.

What can I do to stop my group from being regulated?


To come out of this state, set the value of the regulate_application parameter in
the config_link_policy CLI command to no.
Note

Before disabling application regulation, make sure you are familiar with replication
modes and understand the implications of doing so.

One-phase distribution workarounds


To avoid long initializations, perform one of the following procedures in the RecoverPoint
Administrator's Guide:
l

Use the initialization from backup procedure First-time initialization is taking too
long on page 280.

Set the value of the allow_long_resync parameter of the


config_copy_policy CLI command to no(default is yes).

Add volumes to the copy journal. Note that the additional journal volumes must be
permanent or a full sweep will occur when the volumes are removed.

Copy is being regulated


RecoverPoint includes a smart mechanism that protects the system from adverse affects
and over-consumption of system resources, when a system component is operating
improperly in the system. This mechanism is referred to as Control action regulation.
This section answers the questions:
282

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

When does control action regulation happen?


Control action regulation happens when a system component is operating improperly in
the system, and jittering (quickly changing) between two states for a set period of time.

How do I know control action regulation is happening?


You know the control action regulation mechanism has been enabled, when:
l

Event number 4133 (Copy regulation has started)is displayed in the event log.

In the consistency group Status tab of the GUI, the Role of a copy becomes Regulated
and is displayed in red.

How does control action regulation work?


When control action regulation happens, to allow the environment to stabilize, the
control action regulation mechanism places the copy in a Regulated state, in which the
system will protect itself by closing the link to the copy, limiting any adverse affects and
over-consumption of system resources. The copy stays in the state it was in before
regulation began for 30 minutes or until corrective action is taken.

How do I release a copy from control action regulation?


When control action regulation happens, you can:
l

Release all groups at all copies from this state by running the
unregulate_all_copies command from the CLI.

Check previous event logs. Look for repetitive errors that may indicate a specific
problem in the system.

Check SAN/IP events outside of RecoverPoint, as instabilities may not originate from
RecoverPoint.

If regulation persists, Collect System Information on page 247 and contact EMC
Customer Support for further instruction.

How do I verify that regulation is over?


The following indicators can help you verify that your copy is no longer being regulated:
l

Event 4132 (Copy regulation has ended due to a user action or internal timeout.) is
displayed in the event log.

The Role of a copy is no longer displayed in red in the consistency group Status tab.

Copy is experiencing a high load


High load is a system state that indicates resource depletion during replication.
There are two kinds of high loads in RecoverPoint:
l

Permanent High-loads - In these cases, RecoverPoint stops and waits for a user action
in order to come out of high load.

Temporary High-loads - In these cases, RecoverPoint tries to recover from the high
load and will keep trying until the condition that triggered the high load changes.

When does control action regulation happen?

283

Troubleshooting

How do I know a copy is experiencing a high load?


You know that a copy has entered a high load state, when:
l

Warning events are logged specifying that the copy is experiencing high load.

In Unisphere for RecoverPoint:


n

In the consistency group or copy Status tab, the transfer state on page 43 of a
copy becomes High load.

In the Consistency Group Transfer State widget in the Dashboard, a state of Init
(orange) is displayed for the group.

What is a permanent high load?


A permanent high load is a system state that happens during replication, when the size
of the journal, or the queue of snapshots waiting for distribution of the journal at the
copy, is insufficient.

When do permanent high loads occur?


A permanent high load generally happens in one of two cases:
l

During copy testing in RecoverPoint products with a splitter, if a user accesses a copy
in logged or virtual access modes for a long time and the queue of snapshots waiting
for distribution of the journal reaches its maximum capacity, in this case the system
will Pause Transfer and wait for user input.

When the system is in initialization mode, and the journal volume has reached its
maximum capacity, while the value of the allow_long_resync parameter of the
config_copy_policy CLI command is set to no.

How do permanent high loads work?


When any of the events in When do permanent high loads occur? on page 284 occurs, the
system stops transfer and waits for user input.

How can I tell a copy is under permanent high load?


The following indicators are displayed when your copy is experiencing a permanent high
load:
l

Warning events are logged specifying that the copy is experiencing high load.

The transfer state on page 43 is displayed as High load. You can display the transfer
state:
n

In the RecoverPoint CLI, by running the get_group_states command.

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard of the
Overall Health tab, groups undergoing permanent high load are labeled Init
(orange).

What can I do to come out of permanent high load?


To release a copy from a permanent high load:
284

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

During copy testing in RecoverPoint products with a splitter, if the user accessed a
copy in logged or virtual access mode for a long time and the queue of snapshots
waiting for distribution of the journal reached its maximum capacity, you should
disable image access or enable direct access according to image access modes.

If the system was in initialization mode, and the journal volume became full, while
the value of the allow_long_resync parameter of the config_copy_policy
CLI command is set to no.

How do I verify that a permanent high load is over?


The following indicators can help you verify that your copy is no longer experiencing a
permanent high load:
l

Warning events are logged specifying that the copy is no longer experiencing high
load.

The transfer state on page 43 is no longer displayed as High load. You can display the
transfer state:
n

In the RecoverPoint CLI, by running the get_group_states command.

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard > Overall
Health tab.

What is a temporary high load?


A temporary high load is a system state that happens during replication, when the RPA
resources at the production site are insufficient.

When do temporary high loads occur?


Temporary high loads occur:
l

In extreme cases, during uncommonly great durations of uncommonly extreme write


loads

When the copy or journal volumes are not fast enough to handle distribution

If the WAN is too slow

If the compression level is too high

How do temporary high loads work?


When any of the events in When do temporary high loads occur? on page 285 occurs,
traffic is paused and started immediately. If resources are still low, the system waits five
minutes and then tries to pause and start transfer again until the required resources are
available.
NOTICE

Upon every start of transfer, a short initialization occurs.

How can I tell a copy is under temporary high load?


The following indicators are displayed when your copy is experiencing a temporary high
load:
How do I verify that a permanent high load is over?

285

Troubleshooting

Warning events are logged specifying that the copy is experiencing high load.

The transfer state on page 43 is displayed as high load, followed by a progress status.
You can display the transfer state:
n

In the RecoverPoint CLI, by running the get_group_states command.

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard of the
Overall Health tab, groups undergoing temporary high load are labeled Init
(orange).

What should I know about temporary high loads?


Temporary high loads are a common occurrence and expected to happen from time to
time.
If the high load lasts for an extreme period of time or occurs too frequently (and will
eventually impact the business RPO) contact EMC Customer Support for a mitigation
plan.

How do I verify that a temporary high load is over?


The following indicators can help you verify that your copy is no longer experiencing a
temporary high load:
l

Warning events are logged specifying that the copy is no longer experiencing high
load.

The transfer state on page 43 is no longer displayed as High load. You can display the
transfer state:
n

In the RecoverPoint CLI, by running the get_group_states command.

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard > Overall
Health tab.

RPA keeps rebooting


Reboot regulation is a state of regulation that allows the system to detach an RPA from its
RPA cluster in the event of frequent unexplained reboots or internal failures.

When does reboot regulation happen?


Reboot regulation happens when an RPA is frequently and unexpectedly rebooting, or
undergoing a repeated internal failure.

How does reboot regulation work?


When an RPA behaves according to When does reboot regulation happen? on page 286,
the system detaches the RPA from the RPA cluster.

286

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

How do I know reboot regulation is happening?


Reboot regulation is happening when the user receives the following message when
logging into the RPA as a boxmgmt user:

What should I do to stop reboot regulation?


To stop reboot regulation, contact EMC Customer Support for further instructions.

Host applications crash during initialization


During initialization RecoverPoint reads the data at the production storage in order to
synchronize the data of the copy volumes with the production volumes. If the production
storage cannot handle the read-rate it will hang, causing RecoverPoint and the
production host to timeout, and the user's applications to crash, lose access to the
storage, or experience I/O or path failures.
RecoverPoints I/O throttling mechanism enables users to limit the storage read-rate of
each array or vCenter Server seen by an RPA cluster or all arrays and vCenter Servers seen
by an RPA cluster, allowing the storage to handle the I/O rate during initialization.
When the I/O throttling mechanism is enabled (Default is disabled) RecoverPoint limits
the I/O rate that it sends to the storage according to the throttling level defined by the
user.

What is my storage read-rate during initialization?


Use the following procedure to check your storage read-rate during initialization.
Procedure
1. Connect to the RecoverPoint CLI at the relevant site and run the
detect_bottlenecks command.
2. Note the relevant read-rates:
l

If you are connected to an RPA cluster at the production site, note the value of the
Speed of reading from local user volume during
initialization parameter.

If you are connected to a remote RPA cluster, note the value of the Speed of
reading from remote user volume during initialization
parameter.

How does I/O throttling work?


In RecoverPoint, I/O throttling works in the following way:
1. The user sets the I/O throttling level to limit the storage read-rate during initialization.
2. During initialization, I/Os are delayed, so they do not exceed the value of the
throttling level.
NOTICE

The higher the throttling level, the longer initialization will take.
How do I know reboot regulation is happening?

287

Troubleshooting

When should I enable I/O throttling?


Enable I/O throttling if your host applications experience I/O or path failures, crash, or
lose access to the storage during initialization.

How do I enable I/O throttling?


Before you begin
The following limitations apply to the I/O throttling feature:
l

I/O throttling can only be configured using the RecoverPoint CLI or REST API. You
cannot configure I/O throttling using the Unisphere for RecoverPoint GUI.

I/O throttling can only be configured for arrays and vCenter Servers that have been
mapped to, and are therefore seen by, the RPA cluster.

When selecting to configure I/O throttling for all arrays or vCenter Servers seen by an
RPA cluster, the performance of all consistency groups residing on the same array or
vCenter Server is affected. For example, setting the throttling level to high limits the
I/O rate of all RPAs in an RPA cluster to a maximum read-rate of 50 MBps, per array or
vCenter Server.

The higher the throttling level, the longer initialization will take. Ensure that a low
throttling level is insufficient before setting the throttling level to high.

You must be logged into RecoverPoint as an admin user to configure I/O throttling.

To ensure efficiency, RecoverPoint sends I/Os to storage in parallel. The I/O throttling
mechanism regulates parallel I/Os in the following way:

When set to low, the I/O throttling mechanism allows for a maximum of 10 MB of
parallel I/Os per second.

When set to high, the I/O throttling mechanism allows for a maximum of 1 MB of
parallel I/Os per second.

When set to custom, the I/O throttling mechanism allows for a maximum of
<CustomValue>/20 of parallel I/Os. For example, if the custom value is 40 MBps,
the I/O throttling mechanism will allow for 40/20=2 MB of parallel I/Os per
second.

In previous RecoverPoint versions, I/O throttling was only available per RPA cluster.
Therefore:
n

When upgrading to RecoverPoint 4.1 from a previous RecoverPoint version, the


I/O throttling settings in the previous RecoverPoint version are applied per array
and vCenter Server that are seen by the RPA cluster at the time of the upgrade.
Any arrays or vCenter Servers mapped to the RPA cluster after upgrade will require
additional I/O throttling configuration.

If your RecoverPoint system contains at least one RPA cluster running a


RecoverPoint version prior to RecoverPoint 4.1:
you will not be able to set the I/O throttling level per array or vCenter Server.
the I/O throttling settings of all RPA clusters will be the RecoverPoint 4.0 and
earlier version.

288

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

Note

See the RecoverPoint 4.0 version of the EMC RecoverPoint Administrators Guide
and the EMC RecoverPoint CLI Guide for the I/O throttling settings if at least one
of the RPA clusters in your RecoverPoint system is running a RecoverPoint
version prior to 4.1.
Use the following procedure to enable I/O throttling for a specific array or vCenter Server.
Procedure
1. Run the config_io_throttling command in the RecoverPoint CLI.
2. Select the RPA cluster to which the array or vCenter Server is mapped.
3. Select the array or vCenter Server from the list.
4. Set the throttling level:Set the throttling level to low to limit the I/O rate of the
specified array or vCenter Server to a maximum read-rate of 200 MBps.
l

Set the throttling level to high to limit the I/O rate of the specified array or vCenter
Server to a maximum read-rate of 50 MBps.

Set the throttling level to custom, and specify the custom value in MBps to limit
the I/O rate of the specified array or vCenter Server to any other read-rate.

Set the throttling level to none to disable I/O throttling on the specified array or
vCenter Server.

1. To enable I/O throttling for all arrays or vCenter Servers mapped to an RPA cluster:
5. Run the config_io_throttling command in the RecoverPoint CLI.
6. Select the RPA cluster to which the arrays and/or vCenter Servers are mapped.
7. Scroll down to the bottom of the list of seen arrays and vCenter Servers, and select the
last option in the list.
8. Set the throttling level:
l

Set the throttling level to low to limit the I/O rate of all RPAs in an RPA cluster to a
maximum read-rate of 200 MBps, per array.

Set the throttling level to high to limit the I/O rate of all RPAs in an RPA cluster to a
maximum read-rate of 50 MBps, per array.

Set the throttling level to custom, and specify the custom value in MBps to limit
the I/O rate of all RPAs in an RPA cluster to any other read-rate, per array.

Set the throttling level to none to disable I/O throttling of all RPAs in the specified
RPA cluster.

Results
The read-rate of the RPA on which the consistency group is running is limited to 50 MBps,
200 MBps, or the specified custom setting divided by the number of RPAs in the cluster.

How do I check how throttling affects my system performance?


To check the effect that I/O throttling has on your system performance, perform the steps
outlined in How do I check how throttling affects my system performance? on page 289
before and after enabling I/O throttling.

How can I tell whether or not I/O throttling is enabled?


When I/O throttling is enabled, initializations take longer to complete. The higher the
throttling level, the longer initialization will take to complete.
How do I check how throttling affects my system performance?

289

Troubleshooting

To verify whether I/O throttling is enabled, run the get_system_settings CLI


command and note the state of the I/O Throttling section per RPA cluster.

Add more LUNs than allowed per VNX storage group


Note

This section is not relevant to VNXe.


RecoverPoint requires the following types of volumes at both production and remote
copies:
l

Replication

Journal

In VNX storage systems, the number of LUNs per storage group is limited. This limit is
typically 256, 512, or 1024, depending on the specific VNX model, as presented in the
EMC RecoverPoint Release Notes.

How do I know I need more VNX LUNs?


You know that you need more VNX LUNs when:
l

The following message is displayed in the RecoverPoint or Unisphere UI:


The maximum number of supported LUNs per storage group has
been reached. Either delete existing consistency
groups or contact Customer
Support for further assistance.

You calculate in advance the number of LUNs that you expect to need to support the
replication, journal, and repository volumes for your RecoverPoint configuration, and
that number exceeds the number of LUNs currently available.

What can I do if I need more VNX LUNs?


To overcome this limit, you can create additional storage groups, up to 16 per VNX
storage system. An equal number of virtual WWNs is created for each RPA Fibre Channel
port, and one virtual WWN from each RPA Fibre Channel port is assigned to each
additional storage group. Use the following procedure to create additional VNX storage
groups.
Procedure
1. Calculate the total number of storage groups to create, by dividing the estimated total
number of LUNs you need by the limit for your VNX model.
2. Log in to RecoverPoint as user boxmgmt, to open the Installation Manager.
3. In the Installation Manager, use the Number of virtual ports per port option in the
Installation Manager to create the virtual WWNs at each site, as follows:
a. Open theSetup > Modify settings > Site-1 details menu.
b. Use the Number of virtual ports per port option to set the desired number of virtual
WWNs.
c. Apply the modified settings.
4. Zone the newly created WWNs:
a. For Brocade switches that support NPIV, enable NPIV and set ports as F_Ports;
otherwise, set ports as L_Ports. For all other switches, every switch port to which a
290

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

RecoverPoint Fiber Channel port that uses virtual ports is attached must be
configured as a Fabric Loop port.
b. Create RPA-to-Storage zones with the new WWNs in the same way that existing
WWNs are already zoned.
5. Repeat step 3 on page 290 and step 4 on page 290 for each RPA cluster in your
RecoverPoint system.
Results
The RPAs automatically create new storage groups, and register the new WWNs to those
storage groups.

How can I verify that the storage groups were added successfully?
Upon successful completion of registration (which may take between several minutes
and an hour), the event, Registration of all the site RPAs initiators with the CLARiiON array
is complete., is generated for each site.

Swap the LUN numbers of journal volumes


Swapping LUN numbers for LUNs that have already been exposed to an RPA cluster
should be avoided when possible. When it cannot be avoided, it should be done
according to the following procedure.
Use the following procedure to swap LUN numbers on LUNs that have already been
exposed to an RPA cluster.
Procedure
1. Disable the consistency group.
Disabling the consistency group causes a full sweep of the consistency group when it
is enabled. This procedure will also cause journal loss.
2. Remove journals from the consistency group.
3. Swap the LUNs (on the storage array).
4. Add LUNs as journals.
5. Enable the consistency group.
NOTICE

The journal is deleted and a full sweep of the consistency group occurs.

Find your array serial numbers


This section will help you find the serial numbers of your storage arrays, when activating
RecoverPoint licenses.

Find the serial numbers of VNX arrays


For VNX Series arrays, use the Naviseccli getagent command to find the array serial
number.
For example:
naviseccli -Scope 0 -user <username> -password <password> -address
<spaddress> getagent

How can I verify that the storage groups were added successfully?

291

Troubleshooting

Agent Rev:
Name:
Desc:
Node:
Physical Node:
Signature:
Peer Signature:
Revision:
SCSI Id:
Model:
Model Type:
Prom Rev:
SP Memory:
Serial No:
SP Identifier:
Cabinet:

7.32.0 (5.2)
K10
A-APM00120701240
K10
2998645
2998612
05.32.000.4.007
0
VNX7500
Rackmount
8.10.00
24576
APM00120701240
A
SPE5

NOTICE

spaddress is the address of SPA or SPB, not the control station.

Find the serial numbers of Symmetrix arrays


For Symmetrix arrays, you can display the array serial number on the Symmetrix array or
through RecoverPoint.
On the Symmetrix array, run the SYMCLI command to find the array serial number.
For example:
# symcfg list
SYMMETRIX
Mcode Cache Num Phys Num Symm
SymmID Attachment Model Version Size (MB) Devices Devices
000194900874 Local VMAX-1SE 5876 28672 32 695

In RecoverPoint, run the get_splitter_states CLI command to display the array


serial number.
For example:
SiteC> get_splitter_states cluster=SiteC
Cluster:
SiteC:
Splitters:
SYMM- 000194900874
Status: OK

Find the serial numbers of VPLEX arrays


For VPLEX arrays, you can display the array serial number on the VPLEX array or through
RecoverPoint.
For example:
VPlexcli:/> ls -t /engines/*::top-level-assembly
/engines/engine-2-1:
Name
Value

292

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

-----------------top-level-assembly

-------------FNM00104500511

/engines/engine-1-1:
Name
Value
------------------ -------------top-level-assembly FNM00104500511

In RecoverPoint, run the get_splitter_states CLI command to display the array


serial number.
ClusterA> get_splitter_states cluster=ClusterA
Cluster:
ClusterA:
Splitters:
FNM00104500511
Status: OK

Find your RPA cluster IDs


Procedure
1. Run the CLI command get_internal_cluster_uids.
2. Enter the cluster name, or alternately, press Enter to see a list of all of the RPA cluster
IDs in the system.
3. Note the output:
Internal cluster uids:
Cluster1: 123123123123123123
Cluster2: 345345345345345345
Cluster3: 456456456456456456

Find your XtremIO cluster IDs


The licenses for XtremIO are enforced to the XtremIO cluster ID via the license Locking ID.
The XtremIO cluster ID (called a PSNT, or Product PSNT Serial Number in XtremIO)
contains 5 alphanumeric characters followed by 9 numeric characters.
For example: XIO00123456789

Find your XtremIO cluster IDs (PSNTs) in XtremIO


Your XtremIO PSNTs can be displayed:
l

In the Inventory of the XtremIO GUI, for example:

Find your RPA cluster IDs

293

Troubleshooting

By running the show-clusters-info command in the XtremIO CLI, for example:


xmcli (admin)> show-clusters-info
ClusterName

Index
State

ConnState

ActivationTime

Start-Time

SW-Version PSNT

EncryptionMode

xbrick414

active

connected
Wed Jun 10
17:30:15
2015

Wed Jun 10
17:30:15
2015

4.0.0-54
XIO00150201280

disabled

Find your XtremIO cluster IDs (PSNTs) in RecoverPoint


RecoverPoint discovers XtremIO PSNTs automatically and they are displayed when you
Register storage on page 187. When multiple PSNTs exist the user should register each
PSNT, one at a time.

Understanding the RecoverPoint license


The license file format illustrated below is described in the following table.
Figure 32 License file format

Figure 33 on page 295 contains some common examples of RecoverPoint licenses. One
license file can include multiple licenses. Each license starts with the word INCREMENT.

294

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

Figure 33 Example licenses

The following table described the variables in the license file, and their possible values.
Table 67 License file variables

Variable

Description

Examples

{Replication} The RecoverPoint license model.


Valid values are:

{Type}

{Prod}

LOC for local replication

REM for remote replication

RP.xx.REM.xx
RP.xx.LOC.xx

The RecoverPoint license model.


Valid values are:

RP.CL.xxx.xx
RP.SE.xxx.xx

CL for RecoverPoint/CL

RP.EX.xxx.xx

EX for RecoverPoint/EX

SE for RecoverPoint/SE

The array model. Valid values are:


l

00 - for RecoverPoint/CL licenses

ELA - for licenses based on Enterprise


License Agreements

VMAX

VMAX10K, VMAX20K, VMAX40K

VPLEX

RP.xx.xx.VNX5700

Understanding the RecoverPoint license

295

Troubleshooting

Table 67 License file variables (continued)

Variable

Description
l

VNX5100, VNX5300, VNX5500,


VNX5700, VNX7500

CX, CX4-120, CX4-240, CX4-480,


CX4-960, AX4-5

{Version}

The license schema version.

{Expiration}

The license expiration date.


Licenses can be:
l

Permanent (for an unlimited time)

Expiring:
- Evaluation: Up to 90 days

Examples

1.0
l

15-dec-2014

permanent

VNX array serial numbers are in


the format: APM00120801240

VPLEX array serial numbers are


in the format:
FNM00105500511

VMAX array serial numbers are


in the format: 000194910874

RPA Cluster IDs are in the


format:
7754643987555540854

12 (for 12 TB)

0 (for unlimited capacity)

- Temporary: Up to 365 days


- Emergency: Up to a few days
Note

Warnings will appear 90 days before


license expiration.
Valid values are:

{LockingID}

{Capacity}

{Unit}
296

RecoverPoint 4.4 Administrator's Guide

permanent

A date in the format day-month-year

The unique identifier of the entity that the


license is tied to.
A Locking ID can be:
l

The serial number of a VNX Series array.

The serial number of a VPLEX array.

The SYMM ID of a Symmetrix array.

An RPA Cluster ID.

A System ID

The amount of data licensed for replication at


a given site.
l

In RecoverPoint/CL: as purchased, per


RPA cluster.

In RecoverPoint/SE: the entire array


capacity (unlimited)

In RecoverPoint/EX: VNX; the entire array


capacity. All other arrays; as purchased
per array.
A value of zero means that the
replication capacity is unlimited.

The unit of data being replicated.

TB

Troubleshooting

Table 67 License file variables (continued)

Variable

Description

Examples

{Customer}

The customers name and physical address,


as entered in the RecoverPoint sales order.

EMC, MyStreet, Hopkington, MA

{Issued}

The license creation date, in the format daymonth-year.

15-Dec-2014

{Site_info}

The Site ID is a global EMC setting that


identifies the geographical location of the
customer site.
In RecoverPoint, the EMC site ID is called a
Location ID.

ACTIVATED TO SiteID: 112207055

The site ID is received by the RecoverPoint


system from ELMS and is sent to the system
report mechanism (SyR) and used (in
RecoverPoint/CL licensing) to associate an
RPA cluster to a geographical location.
{SN}

The Activation ID, a uniquely generated value 2055509


specific to the activation. Every time a license
is activated, the SN changes. This value is
sent to the system report mechanism (SyR)
for support purposes.

{Sign}

An encrypted hash segment containing the


license information which is used by
RecoverPoint to validate the license and
ensure its uniqueness.

0004 0EAA 52A0 F74E 6EG4 BE6A


6CF0 1400 \
10E3 A333 704C 11E1 9DAF DA5B
1C0B

Calculate your required journal size


The calculation of journals depends on whether or not snapshot consolidation is
enabled.
The recommended minimum journal size when not utilizing snapshot consolidation is:
MinJournalSize = 1.05 * [(D data per second)*(required rollback time
in seconds)/(1 image access log size)] + (reserved for marking)

For example, if:


D data per second = 5 Mb/s
required rollback time = 24 hr = 86 400 s
image access log = 0.20
reserved for marking = 1.5 GB
Then the minimum journal size would be:
1.05 * 5 Mb/s * 86 400 s / (1 - 0.20) + 1.5 GB = 579 000 Mb
579 000 Mb = 579 000/8 MB = 72 375 MB = 72.4 GB

You can use iostat (UNIX) or perfmon (Windows) to determine the value for D data per
second.
Calculate your required journal size

297

Troubleshooting

The default image access log size is 20%, and it can only be changed using the
RecoverPoint CLI.
See the EMC RecoverPoint Release Notes for the minimum and maximum journal size
limitations.

Minimum journal size with snapshot consolidation


Journal sizing with snapshot consolidation must take into account the incremental
change of data over the period of consolidation. Therefore, when snapshot consolidation
is enabled, the formula for estimating the minimum required journal size is:
MinJournalSize =
SpaceForMarkingAndReplication
+ AvgHourlyChangeRate
* (UnconsolidatedDuration
+ DailyConsolidations
+ WeeklyConsolidations
+ MonthlyConsolidations)
/ spaceAfterConsolidationProcess
/ spaceAfterImageAccessProcesses
* spaceAfterCompression
Note

See the EMC RecoverPoint Release Notes for the minimum and maximum journal size
limitations, when snapshot consolidation is enabled.
For example, if an application has the following requirements:
l

An average data change rate of 0.5MBps

An image access log size that is 20% of the total journal size

A distributed consistency group

A 12 hour unconsolidated duration (i.e. protection window)

10 daily snapshots

2 weekly snapshots

3 monthly snapshots

Journal compression is disabled

The minimum size for the journal at this copy would be calculated as (if performance data
was unavailable for the localities of reference):
MinJournalSize = 6
+ { [0.5*(60*60/1024)]
* 12
+ [24 + (10-1)*24*(1-0.20)]
+ [7*24*(1-0.20)+ (2-1)*7*24*(1-0.20)*(1-0.20)]
+ [7*24*4*(1-0.20)*(1-0.20)+(3-1)*28*24*(1-0.20)*
(1-0.20)*(1-0.20)]
/ (1-0.25)
/ [1-(0.20 + 0.10)]
* (1-0)}

Where:

298

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

Table 68 Journal size with snapshot consolidation equation legend

Field

Description

MinJournalSize

The minimum journal size required to sustain the specified


journal configuration, in GB.

SpaceForMarkingAndReplication

The journal space required (mainly) for the storage of the


delta marking information, and other replication processes, in
GB.
Can be one of two constant values:
l

If the group is a regular (non-distributed) consistency


group, set this value to 1.5.

If the group is a distributed consistency group, set this


value to 6.

Note

The space required for marking and replication processes is


four times larger for distributed consistency groups,
regardless of the number of RPAs configured. This is due to
the four consistency group segments that are created when
you use distributed consistency groups.
AvgHourlyChangeRate

The average change rate of the replicated data based on


iostat.a, in GB per hour.
For example:
If the average data change rate of the replicated data is
0.5MBps, the AvgHourlyChangeRate is:
0.5MBps * (60Sec * 60Min / 1024forGB) =
1.75 GBps

Note

If this value is unknown, create the consistency group and


allow it to replicate during a period in time that adequately
represents your average change rate. Then, Detect
Bottlenecks on page 239 for this period of time and note the
Incoming writes rate for link.
UnconsolidatedDuration

The period of time during which snapshot data is not to be


consolidated, in hours. The periods start time is always
today, and the periods end time is n hoursago.Also referred to
as the protection window, or the continuous protection period.
For example:
If the past two days-worth of unconsolidated data must
always be available, the value of UnconsolidatedDuration is:
2 * 24Hrs =

48

Minimum journal size with snapshot consolidation

299

Troubleshooting

Table 68 Journal size with snapshot consolidation equation legend (continued)

Field

Description
Note

Value must be at least 12 hours. This value is set and can be


modified through the Do not consolidate snapshots for at
least setting in the journal policies on page 87.
DailyConsolidations

The required number of daily snapshots less the journal


space saved during daily consolidations (i.e. the
%DailyLoR), in hours.
This value is computed as:
24 + (NumberOfDailyConsolidations - 1) * 24
* (1-%DailyLoR)
For example:

If three daily snapshots are required, the value of


DailyConsolidations is:
24 + (3-1) * 24Hrs * (1-%DailyLoR)

Note

This value is set, and can be modified, through the one


snapshot per day for x days setting in the journal policies on
page 87. There must be 24 hours-worth of unconsolidated
data available in order to create the first daily snapshot, and
therefore, the first 24 hours are not multiplied by the
%DailyLoR.
%DailyLoR

The percent of journal space saved during each daily


consolidation due to the data that is repeated to the same
location, and therefore, write-folded upon consolidation.
Note

The locality of reference depends on each consistency groups


(i.e. applications) specific I/O pattern, how many blocks get
re-written, and other factors. If you do not know this value,
you can use a value of 0.2 (i.e. 20%) for a general estimation.
For a more precise value based on your system performance,
Collect System Information on page 247 and send it to EMC
Customer Support.
To calculate the journal space left after daily consolidations,
use a value of:
1 - %DailyLoR, For example: 1 - 0.20 (if performance
data is not available).

300

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

Table 68 Journal size with snapshot consolidation equation legend (continued)

Field

Description
Note

The %DailyLoR is not applied to the first daily snapshot, as


there must be 24 hours-worth of unconsolidated data
available in order to create the first daily snapshot. See the
DailyConsolidations parameter for a detailed
description.
WeeklyConsolidations

The required number of weekly snapshots, less the journal


space saved during daily and weekly consolidations (i.e. the
%DailyLoR and the %WeeklyLoR), in hours.
This value is computed as:
7 * 24 * (1-%DailyLoR) +
(NumberOfWeeklyConsolidations - 1) * 7 * 24
*
(1 - %DailyLoR ) * (1 - %WeeklyLoR )
For example:

If four weekly snapshots are required, the value of


WeeklyConsolidations is:
7 * 24 * (1-0.20) + (4 - 1) * 7 * 24 * (1 0.20 ) * (1 - 0.20 )
Note

This value is set, and can be modified, through the one


snapshot per week for y weeks setting in the journal
policies on page 87. There must be 7 daily snapshots
available in order to create the first weekly snapshot, and
therefore, the first 7 days (7 * 24Hrs) are only multiplied by
the %DailyLoR (as opposed to the %DailyLoR and
%WeeklyLoR).
%WeeklyLoR

The percent of journal space saved during each weekly


consolidation due to the data that is repeated to the same
location, and therefore, write-folded upon consolidation.
Note

The locality of reference depends on each consistency groups


(i.e. applications) specific I/O pattern, how many blocks get
re-written, and other factors. If you do not know this value,
you can use a value of 0.2 (i.e. 20%) for a general estimation.
For a more precise value based on your system performance,
Collect System Information on page 247 and send it to EMC
Customer Support.
To calculate the journal space left after weekly
consolidations, use a value of:
1 - %WeeklyLoR, For example: 1 - 0.20 (if performance
data is not available).

Minimum journal size with snapshot consolidation

301

Troubleshooting

Table 68 Journal size with snapshot consolidation equation legend (continued)

Field

Description
Note

The %WeeklyLoR is not applied to the first weekly snapshot,


as there must be 7 daily snapshots available in order to
create the first weekly snapshot. See the
WeeklyConsolidations parameter for a detailed description.
MonthlyConsolidations

The required number of monthly snapshots, less the journal


space saved during daily, weekly, and monthly consolidations
(i.e. the %DailyLoR, the %WeeklyLoR, and the
%MonthlyLoR), in hours.
This value is computed as:
7 * 24 * 4 * (1 - %DailyLoR) * (1 %WeeklyLoR) +
(NumberOfMonthlyConsolidations - 1 ) * 28 *
24 * (1 - %DailyLoR) *
(1 - %WeeklyLoR) * (1 - %MonthlyLoR)
For example:

If two monthly snapshots are required, the value of


MonthlyConsolidations is:
7 * 24 * 4 * (1 - 0.20) * (1 - 0.20) + (2 1 ) * 28 * 24 * (1 - 0.20) * (1 - 0.20) *
(1 - 0.20)
Note

This value is set, and can be modified, through the journal


policies on page 87. In the snapshot consolidation process, a
month is represented as 28 days. There must be 28 daily
snapshots or 4 weekly snapshots available in order to create
the first weekly snapshot, and therefore, the first 28 days (7
* 24Hrs * 4Days) are only multiplied by the %DailyLoR
and the %WeeklyLoR (as opposed to the %DailyLoR, the
%WeeklyLoR and the %MonthlyLoR.
%MonthlyLoR

The percent of journal space saved during each monthly


consolidation due to the data that is repeated to the same
location, and therefore, write-folded upon consolidation.
Note

The locality of reference depends on each consistency groups


(i.e. applications) specific I/O pattern, how many blocks get
re-written, and other factors. If you do not know this value,
you can use a value of 0.2 (i.e. 20%) for a general estimation.
For a more precise value based on your system performance,
Collect System Information on page 247 and send it to EMC
Customer Support.
To calculate the journal space left after monthly
consolidations, use a value of:
302

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

Table 68 Journal size with snapshot consolidation equation legend (continued)

Field

Description
1 - %MonthlyLoR, For example: 1 - 0.20 (if
performance data is not available).
Note

The %MonthlyLoR is not applied to the first monthly


snapshot, as the data of the first daily snapshot is not
repeated to the same location. See the
MonthlyConsolidations parameter for a detailed description.
%LAProcess

The percentage of the journal space that is reserved for


logged access processes.
Default value=20%
Note

This value is set, and can be changed through the


image_access_log_size parameter of the
config_copy_policy CLI command.
%VAProcess

The percentage of the journal space that is reserved for virtual


access processes.
Constant value=10%
This value is constant, and cannot be changed.

spaceAfterImageAccessProcesses The percentage of the journal space that is left after the space
reserved for the logged (%LAProcess) and virtual
(%VAProcess) image access processes is subtracted.
Note

This value is calculated as: 1 - %LAProcess +


%VAProcess, for example: 1 - 0.20 + 0.10
spaceAfterConsolidationProcess

The percentage of the journal space that is left after the space
reserved for the snapshot consolidation process is
subtracted, where the snapshot consolidation process uses
25% of the total journal space.
This value is constant, and is calculated as: 1 - 0.25

spaceAfterCompression
Note

Journal compression is not available in RecoverPoint/SE.


The percentage of the journal space that is left after the space
saved by journal compression is subtracted:
l

If the data contained in the journal is not compressible,


use a value of: 1 - 0.

If the data contained in the journal is somewhat


compressible, use a value of: 1 - 0.20.

Minimum journal size with snapshot consolidation

303

Troubleshooting

Table 68 Journal size with snapshot consolidation equation legend (continued)

Field

Description
l

If the data contained in the journal is highly


compressible, use a value of: 1 - 0.40.

This value is set and can be modified through the journal


compression setting in the Journal policies on page 86.
Value depends highly upon the compressibility of the data
being stored in the journal, which will vary per consistency
group. The specified values are general estimations, and if
journal compression is used, they should be replaced with
actual values. To obtain actual values, enable the journal
compression setting for a representative period of time
(Warning: both enabling and disabling this setting causes
journal loss), and then Collect System Information on page
247. Send the collected system information to EMC Customer
Support to obtain journal compression values based on the
performance data of the group being replicated.
Note

Enabling journal compression is not recommended, as this


process requires RPA CPU resources, and enabling it has
adverse effects on the distribution process, and by
association, the RPA throughput. See the EMC RecoverPoint
Release Notes for performance statistics of consistency
groups, with and without journal compression.
a.

You can use iostat (UNIX) or perfmon (Windows) to determine the value for data per second.

Validate replication integrity


Integrity validation ensures replication integrity between the production and copy
volumes of a specific consistency group, or all consistency groups in a RecoverPoint
system, without necessitating the stopping of I/Os at the production.

How can replication integrity be compromised?


Replication integrity can be compromised, for example, due to:
l

Configuration issues. If the host replicates data without going through a RecoverPoint
write splitter, as in ESX environments, for example.

Faulty procedures. Due to incorrect usage of the clear_markers command or


incorrectly performing a LUN resize, for example.

When should I validate replication integrity?


You should validate replication integrity:
l

304

If image access fails


RecoverPoint protects you from possible integrity issues by not allowing you to test,
fail over to, or recover production from, an image whose replication integrity has been
compromised. During copy testing, failover, and production recovery; when accessing
an image in logged access mode, image access will fail if the selected image is not

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

consistent with its production source. In such cases, you should check the integrity of
the group Using the integrity check tool on page 306 to ensure data consistency
between the groups production and copy volumes.
l

Periodically
You can validate replication integrity periodically Using the periodic integrity
validation tool on page 307 to ensure data consistency between all of the production
and copy volumes in your RecoverPoint system on an ongoing basis.

How the integrity check tool works


The integrity check tool enables users to perform an integrity check of all of the volumes
of a consistency group copy to ensure consistency between all of the copy volumes and
their production sources.

Before you begin


Before you use the integrity check tool, note the following limitations:
l

The integrity check process only runs on the groups primary RPA.

The integrity check process reads all of the data of all of the production and copy
volumes of the group, which can put extra load on the production storage, and then
compares their signatures, which can put extra load on the WAN. The extra load
placed on the WAN may lead to high load or WAN disconnection, which will cause the
integrity check to stop before completing the process on all consistency group
volumes.

Integrity check is not supported during snap-based replication.

How it works
When enabled, the integrity check tool:
1. Reads a section of the groups production and copy volumes.
2. Creates hash signatures of that section of the production and copy volumes.
3. Compares the two signatures.
4. Repeats this process as many times as required across all group volumes.
5. If no inconsistency is found, event 4107 - Integrity check completed successfully, is
logged.
NOTICE

If at any time during this process a possible inconsistency is found, event 4108 - A
possible replication integrity issue has been identified is logged.

How the periodic integrity validation tool works


The periodic integrity validation tool enables users to perform an integrity check on all of
the volumes of a consistency group on an ongoing bases, to ensure consistency between
all production and copy volumes of all groups in a RecoverPoint system, at all times.

Before you begin


Before you use the periodic integrity validation tool, note the following limitations:
l

By default, periodic integrity validation is disabled.

Integrity validation during snap-based replication is not supported.

The periodic integrity validation tool is only enabled if all RPA clusters in your
RecoverPoint system are running RecoverPoint 4.1 SP1 or later.
How the integrity check tool works

305

Troubleshooting

To temporarily disable or enable all of the link settings in your entire RecoverPoint
system that were set Using the periodic integrity validation tool on page 307 see Can
I disable or enable integrity validation globally? on page 307.

How it works
For each copy whose integrity you want to periodically validate, you will need to run the
config_periodic_integrity_validation CLI command, specify the copy name,
and select a mode of operation.
The following modes are available:
l

Light: In this mode, each RPA will read 1MB every 5 seconds.

Heavy: In this mode, each RPA will read 10MB every 0.5 second.
Note

When using this mode, the maximum degradation in total performance (throughput
and IOPs) should not exceed 10%. However, using this mode is not recommended,
and this mode should not be used without explicit instruction from EMC Customer
Support.
l

Disabled

When enabled, the tool randomly selects an area of the production storage protected by
each consistency group:
1. Each RPA sequentially scans the volumes of the consistency groups that are running
on it (selecting a different group each time) and randomly selects an area of the
storage protected by the group. Each volumes beginning and end receive priority in
the random selection process.
2. If the randomly selected area of a production volume has already been replicated but
has not been written to the target storage yet, this area is not scanned and is
considered consistent.
3. If a possible replication inconsistency is detected, the tool will re-scan the same area
of the production volume to ensure that the results of the first scan are correct.
4. If the second scan detects a possible replication integrity issue, event 4165 is
logged, and a call home event will automatically open a service request with EMC
Customer Support with the relevant consistency group name, volume name and
metadata, while the data itself is written in the RPA logs.
5. If no replication integrity issues are detected no events are logged.

How do I configure integrity validation?


Replication integrity is enabled separately per tool:
l

How the integrity check tool works on page 305

How the periodic integrity validation tool works on page 305

Note

Both integrity validation tools can only be accessed using the RecoverPoint CLI.

Using the integrity check tool


All of the settings of the integrity check tool are configured per link of a consistency
group.
You can:
306

RecoverPoint 4.4 Administrator's Guide

Troubleshooting

Start a new integrity check by running the start_integrity_check CLI


command.

Stop an integrity check that is in progress by running the stop_integrity_check


CLI command.

Display the progress of an integrity check by running the


show_integrity_check_status CLI command.

Using the periodic integrity validation tool


All of the settings of the periodic integrity validation tool are configured per link of a
consistency group.
You can:
l

Enable periodic integrity validation by running the


config_periodic_integrity_validation CLI command, and setting the
mode to light.
NOTICE

Setting the integrity validation mode to heavy might affect performance. Consult EMC
Customer Support before setting the mode to heavy.
l

Disable periodic integrity validation by running the


config_periodic_integrity_validation CLI command, and setting the
mode to disabled.

Display the current integrity validation settings by running the


get_group_settings command and noting the value of the Integrity validation
policy field.

How do I know whether or not integrity validation was successful?


The output of the integrity validation process differs between the integrity check tool and
the periodic integrity validation tool.

The integrity check tool output


After Using the integrity check tool on page 306 one of the following events is logged to
indicate whether or not a possible integrity issue has been detected:
l

Event 4107 - Integrity check completed successfully.

Event 4108 - A possible replication integrity issue has been identified.

The periodic integrity validation tool output


After Using the periodic integrity validation tool on page 307, if a possible replication
integrity issue is detected, event 4165 is logged and a call home event automatically
opens a service request with EMC Customer Support containing the name of the relevant
group and volume, and the relevant metadata. After receiving the service request, EMC
Customer Support will pro-actively contact you to help resolve any possible integrity
issues.

Can I disable or enable integrity validation globally?


You can temporarily disable or enable all of the link settings in your entire RecoverPoint
system that were set Using the periodic integrity validation tool on page 307, without
deleting the link settings by running the
config_global_periodic_integrity_validation CLI command. Running this
How do I know whether or not integrity validation was successful?

307

Troubleshooting

command and setting enabled to no, will override all of link-level settings. Running this
command and setting enabled to yes, will use the configured link-level settings.
To display whether or not global periodic integrity validation is enabled, run the
get_global_policy CLI command.

What should I do if an issue is detected?


If a possible replication integrity issue is detected, use the clear_markers CLI command
to trigger a full sweep. If problem persists, contact EMC Customer Support.

MetroPoint group conversion


Use the following procedure to convert a regular consistency group into a MetroPoint
group, or a MetroPoint group into a regular consistency group.
Before you begin
l

To become a MetroPoint group, the production volume must be a VPLEX distributed


(DR1) volume, and it may not have more than one remote copy.

To protect a VPLEX Metro distributed volume at both VPLEX clusters simultaneously,


use MetroPoint topology.

Procedure
1. Select Protection > Manage Protection
2. Select a group
3. Click Add Copy and follow the instructions in Add a copy on page 116.
4. Add a standby production copy
Results
The group becomes a MetroPoint group.
Note

Follow the instructions in Remove a copy on page 123 and remove the standby
production copy to turn a MetroPoint group into a regular group.

308

RecoverPoint 4.4 Administrator's Guide

CHAPTER 9
Events

This section describes all of the events issued by the RecoverPoint system.
l
l
l

Normal Events.....................................................................................................310
Detailed events................................................................................................... 320
Advanced events.................................................................................................360

Events

309

Events

Normal Events
The following table contains the Normal scope events that are issued by the RecoverPoint
system.
Table 69 Normal scope events

ID

Topic

Level

Summary

More Details

User Action

1002

Mgmt

Info

User logged out

None

1010

Mgmt

Warning

License grace period


expires in one day

Grace period expires in 1 day. You


must activate each license and
enter your licenses into
RecoverPoint.

1012

Mgmt

Warning

License grace period


about to expire

License grace period expires in less than <expiration interval> days. You
must obtain a new RecoverPoint
license.

1013

Mgmt

Error

License expired

None

Obtain a new RecoverPoint license.

1021

RPA

Error

HBA firmware error

None

2000

RPA
Cluster

Info

Cluster management
running on RPA

None

3002

RPA

Warning

Cluster management
switched over to RPA

None

3007

RPA

Warning
off

RPA is up

None

3014

RPA

Warning

RPA unable to access


volume

None

3022

RPA

Warning
off

LAN link to RPA restored

None

3023

RPA

Warning

LAN link to RPA down

None

3155

Group

Info

Snap interfaces created


on storage

None

3156

Group

Info

Snap interfaces deleted


from storage

None

3170

Mgmt

Info

New site keys and


certificates are
generated and being
synchronized between
clusters.

None

3171

Mgmt

Info

New site keys and


certificates are
synchronized between
clusters.

None

4000

Group

Info

Group capabilities OK

None

310

RecoverPoint 4.4 Administrator's Guide

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4001

Group

Warning

Minor problem in group


capabilities

None

4003

Group

Error

Group capabilities
problem

None

4007

Group

Info

Pausing data transfer for


group

None

4008

Group

Warning

Pausing data transfer to


copy

1. To determine the cause of this


event, check adjacent events for
this group, both before and after
this event.
2. Verify that transfer resumes in a
timely manner. To do so, check the
transfer state in the RecoverPoint
GUI or CLI, or check subsequent
events for this group.
3. If transfer does not resume,
collect logs for the period when the
event occurred, and contact EMC
Customer Support

4009

Group

Error

Pausing data transfer for


group

None

4010

Group

Info

Starting data transfer to


copy

None

4015

Group

Info

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

4016

Group

Warning

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

4017

Group

Error

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

4018

Group

Warning

Transfer of latest
snapshot complete

Transfer of latest snapshot from


source complete (no data loss)

4019

Group

Warning

Group under high load

Group in high load -- transfer to be


paused temporarily

4020

Group

Warning
off

Group high load over

None

4021

Group

Error

Initialization paused

Initialization has been paused


because the journal is full.

To complete initialization, add


journal volumes to the journal or
enable long initialization.

4022

Group

Error off

Initialization resumed

None

4023

Group

Error

Transfer paused

Transfer has been paused because


the journal is full.

To restart transfer, first disable


access to the accessed image.

4024

Group

Error off

Transfer restarted

None

Normal Events

311

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4025

Group

Warning

Group under high load

Group is undergoing a high load so


initialization will be restarted.

4026

Group

Warning
off

Group high load is over

None

4027

Group

Error

Group under high load

Roll to the physical image will be


paused and transfer will be paused.

4028

Group

Error off

Group high load is over

None

4040

Group

Error

Journal error

Full sweep to be performed.

4041

Group

Info

Group side activated

None

4042

Group

Info

Group side deactivated

None

4043

Group

Warning

Group side deactivated

None

4044

Group

Error

Group side deactivated

None

4051

Group

Info

Resuming distribution

Disabling access to image and


resuming distribution.

4054

Group

Error

Enabling access to image None

4057

Group

Error

Specified image removed The specified image has been


from journal
removed from the journal.

4062

Group

Info

Access enabled to latest


image

None

4063

Group

Warning

Access enabled to latest


image

None

4064

Group

Error

Access enabled to latest


image

None

4080

Group

Warning

Current lag exceeds


maximum lag

None

4081

Group

Warning
off

Current lag within policy


limit

None

4084

Group

Info

Markers cleared

None

4085

Group

Warning

Unable to clear markers

None

4086

Group

Info

Synchronization started

None

4087

Group

Info

Synchronization
completed

None

4091

Group

Error

Image access log is full

Access to copy volumes for this


group at this RPA cluster has been
disabled. If you do not wish to
retain the writes that were added to
the copy image while image access
was enabled, disable image
access; the relevant writes are

312

RecoverPoint 4.4 Administrator's Guide

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

automatically undone.
Alternatively, use the undo writes
command to undo the same writes;
image access remains enabled.
- If, however, you wish to retain the
writes that were added to the copy
image while image access has been
enabled, you must increase the size
of the copy journal by adding one or
more volumes to the journal.
Alternatively, you can set the image
access mode to direct; however, in
that case, the copy journal is
erased, and a full sweep
resynchronization will be required
to restore consistency between the
production and copy images.
- To help prevent this problem from
recurring, increase the size of your
image access log, using one or both
of the following:
- - Add volumes to the journal.
- - Modify the policy setting for the
portion of the journal allocated to
the image access log. This is
permitted only when image access
is disabled.
4095

Group

Info

Writes to image access


log cannot be undone

The image access log is being


written to storage, so the writes to
the image access log cannot be
undone.

4097

Group

Warning

Starting three-phase
distribution

Max journal lag exceeded or


insufficient journal

Check the maximum journal lag


setting value, and verify that the
current value is indeed required. If
the maximum journal lag value has
not been defined or the current
journal lag is lower than the
maximum journal lag value, this is
probably a journal space or
performance issue. In this case,
enlarge your journal or increase the
speed of the journal by placing it in
a faster array, on faster media, or
any other procedure that can
increase the journal write-rate.

4098

Group

Warning
off

Maximum journal lag


within limit

Distribution normal -- rollback


information retained.

4099

Group

Warning

Starting long
initialization

None

Normal Events

313

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4110

Group

Info

Enabling virtual access


to image

None

4111

Group

Info

Virtual access to image


enabled

None

4112

Group

Info

Rolling to physical image None

4113

Group

Info

Roll to physical image


stopped

None

4114

Group

Info

Roll to physical image


complete

Logged access to physical image


enabled.

4115

Group

Error

Cannot enable access to


virtual image

Unable to enable access to virtual


image due to a partition table error.

4116

Group

Error

Virtual writes buffer full

Virtual access is disabled.

4118

Group

Error

Cannot enable virtual


access

None

4119

RPA

Error

Initiator issued out of


bounds I/O

None

Contact EMC Customer Support

4120

Group

Warning

Journal usage exceeding


threshold

Because you are accessing a copy


image in logged access mode for an
unusually long time, the copy
journal usage is exceeding the
normal threshold, and this may
affect the image access log
capacity.

4122

Group

Error off

Trying to regain write


permissions

Trying to regain write permissions


to non-production copy.

4124

Group

Error off

Read and write to volume None


restored

4129

Group

Warning

Logged access enabled


for unusually long time

Logged access has been enabled


on a group copy for an unusually
long time.

4130

Group

Warning
off

Logged access no longer


enabled for unusually
long time

Logged access on group copy is


now disabled or it has been
enabled to a different image.

4135

Group

Info

Copy entered
maintenance mode

All copy activities have been


suspended.

4136

Group

Info

Copy exited maintenance All copy activities have resumed.


mode

4137

Group

Info

Snapshot consolidation
has been successful

None

4138

Group

Warning

Snapshot consolidation
failed

None

314

RecoverPoint 4.4 Administrator's Guide

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4140

Group

Info

Change pipe mode


because of sync/async
dynamic settings

None

4141

Group

Info

Snapshot consolidation
policy change

The user has changed the snapshot


consolidation policy setting(s).

4142

Group

Info

Snapshot consolidation
request registered

None

4150

Group

Error

Phoenix System RP
communication error

None

4151

Group

Info

Phoenix System RP
settings change

None

4152

Group

Info

Phoenix System RP using Phoenix System RP switched back


allocated disk space
to using only allocated disk space
to protect consistency group.

4153

Group

Info

Phoenix System RP using Phoenix System RP using


unallocated disk space
unallocated disk space to protect
consistency group.

4154

Group

Warning

Phoenix System RP ran


out of disk space

None

4155

Group

Info

Initializing Phoenix
System RP

None

4156

Group

Warning

Maximum lag exceeded

Maximum lag between source and


target cluster exceeded.

4157

Group

Error

Phoenix System RP failed Phoenix System RP failed to


to allocate disk space
allocate disk space to protect
consistency group due to
insufficient disk space.

4159

Group

Info

Volume was resized

Replication set will be resized


automatically.

4160

Group

Info

Volumes were resized

None

4161

Group

Info

Short initialization
started

Short initialization started because


of resize

4162

Group

Info

Replication set resize


enabled

You may now resize the replication


set capacity in RecoverPoint.

4174

RPA
Cluster

Info

volume migration started Migration of volume configuration


data has started

4210

RPA
Cluster

Warning

VM no longer being
replicated

None

4211

RPA
Cluster

Warning

VM partially replicated

None

Normal Events

315

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

4212

RPA
Cluster

Info

VM is fully replicated

None

4213

RPA
Cluster

Error

vCenter Server is not


accessible

None

4300

RPA
Cluster

Warning

Writing rate to the


production journal is
slow

If bottleneck detection shows a


delay factor >> 1 for this journal
replication of the group may be
disrupted.

4301

RPA

Warning

RPA is unable to handle


incoming data rate due
to high compression
level

None

Try to reduce the load on the RPA by


lowering the compression level.
Alternatively

4302

Group

Warning

Journal is unable to
handle incoming data
rate

None

If the event recurs, try to configure


journal volumes for better
performance.

4303

Group

Warning

Journal and replication


volumes at copy are
unable to handle
incoming data rate

None

If the event recurs, try to configure


journal volumes for better
performance.

4304

Group

Warning

Remote storage is
unable to handle
incoming data rate;
regulating distribution

In distribution, there are reads and If the event recurs, try to configure
writes to the journal and replication remote storage for better
volumes on the remote storage. If
performance.
I/O access to this storage is too
slow, high load states may occur.

4305

Group

Warning

Remote cluster is unable


to handle incoming data
rate

This may cause high load states at


the local cluster.

4306

RPA
Cluster

Warning

Data transfer rate


between clusters is slow

This may increase RPO, and cause a If this event recurs, examine the
high-load state.
WAN/FC link for possible high
latency or packet loss. Consult your
network administrator if needed.

4307

RPA
Cluster

Warning

Reading rate from local


replication volumes
during resynchronization
is slow

A slow reading rate during


resynchronization (which involves
reading a very large volume of data
from the local replication volumes)
may increase RPO.

If the event recurs, try to configure


local replication volumes for better
performance.

4308

RPA
Cluster

Warning

Reading rate from


replication volumes at
copy during
resynchronization is slow

A slow reading rate during


resynchronization (which involves
reading a very large volume of data
from replication volumes at the
copy) may increase RPO.

If the event recurs, try to configure


replication volumes at the copy for
better performance.

4309

RPA

Warning

RPA utilization reached


80%

High utilization of RPA resources


may be due to high volume of
incoming data, high volume of

If bottleneck detection shows that


compression is using 80% or more
of CPU resources, reduce the
compression setting. If incoming

316

RecoverPoint 4.4 Administrator's Guide

User Action

If the event recurs, try to configure


remote storage for better
performance.

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

incoming IOPS, or high


compression level.

data is near the limit, consider


adding an RPA or moving groups to
other RPAs .
If bottleneck detection shows that
compression for this link is using
80% or more of CPU resources,
reduce the compression setting. If
incoming data is near the limit,
consider splitting the existing group
onto two or more RPAs .

4310

Group

Warning

Link utilization reached


80%

High utilization of the link may be


due to a high volume of incoming
data, high volume of incoming
IOPS, or high compression level.

4312

Group

Error

Switching to virtual
access

After resizing, virtual image access


with roll is not allowed to a point-intime before the volume resize.
Virtual image access was
performed instead.

4400

RPA
Cluster

Warning
off

Writing rate to the


production journal is no
longer slow

None

4401

RPA

Warning
off

RPA handles the


incoming data rate
successfully

None

4402

Group

Warning
off

Journal handles
incoming data rate
successfully

None

4403

Group

Warning
off

Journal and replication


volumes at copy handle
incoming data rate
successfully

None

4404

Group

Warning
off

Remote storage is able to None


handle incoming data
rate and no longer
regulates distribution

4405

Group

Warning
off

Remote cluster handles


incoming data rate
successfully

None

4406

RPA
Cluster

Warning
off

Data transfer rate


between clusters is no
longer slow

None

4407

RPA
Cluster

Warning
off

Reading rate from local


None
replication volumes
during resynchronization
is normal

4408

RPA
Cluster

Warning
off

Reading rate from


replication volumes at
copy during

None

Normal Events

317

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

resynchronization is
normal
4409

RPA

Warning
off

RPA utilization is back to


normal

4410

Group

Warning
off

Link utilization is back to None


normal

4412

Group

Error off

Able to roll to physical


image again

None

5013

Splitter

Error

Splitter is down

None

5014

Splitter

Info

Writes to replication
volumes disabled, reads
are allowed

None

5015

Splitter

Error off

Splitter is up

None

5016

Splitter

Warning

Splitter has restarted

None

5017

Splitter

Error off

Splitter version is
supported

None

5018

Splitter

Error

Splitter version is not


supported

None

5019

Splitter

Warning

Writes to replication
volumes disabled, reads
are allowed

None

5020

Splitter

Error

Writes to replication
volumes disabled, reads
are allowed

None

5029

Splitter

Error

LUN mismatch detected

Volume sweep will follow.

5030

Splitter

Error

Possible splitter write


failure

None

5031

Splitter

Warning

Splitter not splitting to


replication volumes

Volume sweep(s) required.

5032

Splitter

Info

Splitter splitting to
replication volumes

None

5035

Splitter

Info

Writes to replication
volumes disabled

None

5036

Splitter

Warning

Writes to replication
volumes disabled

None

5037

Splitter

Error

Writes to replication
volumes disabled

None

5038

Splitter

Info

Splitter delaying writes

None

5039

Splitter

Warning

Splitter delaying writes

None

318

RecoverPoint 4.4 Administrator's Guide

None

From the relevant hosts, rescan the


SAN.

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

5040

Splitter

Error

Splitter delaying writes

None

5041

Splitter

Info

Splitter not splitting to


replication volumes

None

5042

Splitter

Warning

Splitter not splitting to


replication volumes

None

5043

Splitter

Error

Splitter not splitting to


replication volumes

None

5045

Group

Warning

Splitter and RPA problem Simultaneous problems reported in


detected
splitter and RPA. Full-sweep
resynchronization will be required
upon restarting data transfer.

5046

Splitter

Warning

Splitter write re-issued to RPA did not acknowledge an I/O in


RPA
a timely manner. As a result, the
splitter has re-issued the write.

Following this event, it is


recommended to:
- Monitor subsequent events, to
determine if the problem recurs.
- If the problem persists, or
increases in frequency, contact
EMC Customer Support .

5052

Splitter

Error

Splitter does not support Group was successfully configured


synchronous replication to replicate synchronously but
since then the splitter has been
replaced. The new splitter does not
support synchronous replication
and consequentially replication
mode has been changed to
asynchronous.

Replace your current splitter with


one that supports synchronous
replication. If problem persists
contact EMC Customer Support.

5053

Group

Error off

Splitter now supports


synchronous replication

Group is configured to replicate


synchronously and the splitter now
supports synchronous replication.

5059

Splitter

Info

Array roll started

An array roll has been triggered.


RecoverPoint transfer will be
paused until roll is completed.

No user action is required.

5060

Splitter

Info

VPLEX inter-cluster link


recovery started

VPLEX inter-cluster link recovery


started, RecoverPoint transfer will
be paused until recovery is
completed.

No user action is required.

5061

Splitter

Info

VNX roll started

Failover started at another RP


system attached to VPLEX, no
snapshots can be created by
RecoverPoint.

No user action is required.

5079

RPA

Error

Phoenix System RP
cluster error

None

5080

RPA

Error off

Phoenix System RP
cluster is OK

Phoenix System RP cluster is OK


and responding

Normal Events

319

Events

Table 69 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

5081

RPA

Error

Phoenix System RP
None
cluster not responding to
requests

5082

RPA

Info

Phoenix System RP
None
cluster configuration has
changed

7111

RPA
Cluster

Error

Volumes switched on
storage

System has detected that volumes


were switched in the storage
configuration without following
proper procedures.

7112

Splitter

Error

Volumes switched on
storage

System has detected that volumes


were switched in the storage
configuration without following
proper procedures.

9999

RPA

Info

Test event

None

18036

RPA

Error

Negotiation of transfer
protocol failed again

Negotiation of transfer protocol


failed due to network issues in TCP
layer between clusters. Problem
was temporarily corrected, but has
since returned.

User Action

If problem persists, check your TCP


network.

Detailed events
The following table contains the Detailed scope events that are issued by the
RecoverPoint system.
Table 70 Detailed scope events

ID

Topic

Level

Summary

More Details

1000

Mgmt

Info

User logged in

None

1001

Mgmt

Warning

User login failed

None

1003

Mgmt

Warning

Failed to generate SNMP


trap

None

1004

Mgmt

Warning

Failed to send email alert Failed to send email alert to the


specified address.

1005

Mgmt

Warning

Failed to update local file None

1006

Mgmt

Info

Settings changed

320

RecoverPoint 4.4 Administrator's Guide

None

User Action

Check whether there is an issue


with your selected method of
transfer. If a problem exists, fix it
and then, perform a connectivity
test by running the
test_syr_connectivity CLI
command. If problem persists
contact EMC Customer Support.

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

1007

Mgmt

Info

Settings change failed

None

1008

Mgmt

Info

User action succeeded

None

1009

Mgmt

Info

User action failed

None

1011

Mgmt

Error

License grace period


expired

You must obtain a new


RecoverPoint license and then
activate each license and enter your
licenses into RecoverPoint.

1014

Mgmt

Info

User bookmarked an
image

None

1017

Mgmt

Warning

RPA to host multipathing None


problem

1018

Mgmt

Warning
off

RPA multipathing
problem fixed

None

1019

Mgmt

Warning

User action performed


successfully

None

1020

Mgmt

Warning

Failed to send system


report

None

1031

Mgmt

Warning

System limit exceeded


low watermark

None

1032

Mgmt

Info

System limit passed


below low watermark

None

1033

Mgmt

Warning

System limit exceeded


high watermark

None

1034

Mgmt

Info

System limit passed


below high watermark

None

1035

Mgmt

Error

System limit exceeded

None

1036

Mgmt

Info

System limit passed


below limit

None

3000

RPA

Warning
off

RPA successfully
communicating with
cluster

None

3001

RPA

Warning

RPA problem
communicating with
cluster

RPA is currently experiencing a


problem communicating with its
cluster.

User Action

Check whether there is an issue


with your selected method of
transfer. If a problem exists, fix it
and then, perform a connectivity
test by running the
test_syr_connectivity CLI
command. If problem persists
contact EMC Customer Support.

Detailed events

321

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

3005

RPA
Cluster

Error

Settings conflict between A settings conflict has been


clusters
discovered between two RPA
clusters.

3006

RPA
Cluster

Error off

Settings conflict between The settings conflict between RPA


clusters resolved
clusters has been resolved by a
user action.

3008

RPA

Warning

RPA appears to be down

None

3030

RPA

Warning

RPA switched path to


storage

None

3036

RPA

Error

RPA clock out of sync

None

3037

RPA

Error off

RPA clock is back in sync None

3038

RPA
Cluster

Info

Director automatically
added

None

3040

RPA
Cluster

Info

RPA cluster added

None

3041

RPA
Cluster

Info

RPA cluster removed

None

3157

Group

Error

Snap based replication


stopped

Array resources usage exceeded


maximum threshold

3158

Group

Error off

Snap based replication


resumed

Array resources usage is back to


normal

3159

RPA
Cluster

Error

System was unable to


None
remove snap interface(s)
from the array

Remove the snap interface(s)


manually

3160

Group

Error

System was unable to


remove a consistency
group from the array

Remove the array consistency


group manually

3161

Group

Error

System was unable to


None
remove consistency snap
from array

Remove array consistency snap


manually

3162

Group

Error

System was unable to


promote consistency
snap on array

None

Remove the array consistency


group manually

3163

Group

Error

Array operation took too


long

None

Array may be loaded. Increase snap


based replication interval.

4056

Group

Error

No image in journal
matches query

None

4078

Group

Info

Replication set resized

None

4082

Group

Warning

Next synchronization will The copy marked all of its


be a full sweep
replication volumes as dirty.

322

RecoverPoint 4.4 Administrator's Guide

More Details

None

User Action

Contact EMC Customer Support.

Clear array resources

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

4083

Group

Warning

Next synchronization will The copy marked some of its


be a volume sweep
replication volumes as dirty. Next
synchronization will be a volume
sweep on these volumes.

4090

Group

Warning

Image access log 80%


full

Image access is enabled to a nonproduction copy.


- The capacity of the copy journals
image access log has nearly been
reached; only limited further
information can be written to it. If
the log becomes completely full,
the system will block access by
host applications to the replication
volumes that belong to that group
(and the application will receive I/O
errors).
- When the system is in this state, it
is recommended that immediate
action be taken to prevent the log
from becoming 100% full.

4106

Group

Warning

Starting full sweep

Cannot write additional replication


data for this group to volume
because its capacity has been
reached. Starting full sweep.

4107

Group

Info

Replication integrity
check successful

None

4108

Group

Error

Replication integrity
issue detected

A possible replication integrity


issue has been detected.

User Action

If you do not wish to retain the


writes that were added to the copy
image while image access was
enabled, disable image access; the
relevant writes are automatically
undone. Alternatively, use the undo
writes command to undo the same
writes; image access remains
enabled.
- If, however, you wish to retain the
writes that were added to the copy
image while image access has been
enabled, you can do one of the
following:
- - Increase the size of the copy
journal by adding one or more
volumes to the journal.
- - Fail over to the copy.
- Note: The copy journal is erased.
- - Set the image access mode to
direct.
- Note: The copy journal is erased,
and a full sweep resynchronization
will be required to restore
consistency between the
production and copy images.
- To help prevent this problem from
recurring, increase the size of your
image access log, using one or both
of the following:
- - Add volumes to the journal.
- - Modify the policy setting for the
portion of the journal allocated to
the image access log. This is
permitted only when image access
is disabled.

Contact EMC Support

Detailed events

323

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4109

Group

Error

Replication integrity
check stopped before it
finished

Replication integrity check stopped


before it finished because transfer
has been paused, and no
inconsistency has been found.

Check your system events to better


understand the root cause of the
pause in transfer. Run the
replication integrity check again
after restarting transfer.

4117

Group

Warning

Virtual writes buffer is


90% full

When virtual writes buffer becomes


full virtual access is disabled.

4121

Group

Error

Error writing to volume

Ensure that RecoverPoint has write


permission to all volumes. System
will retry automatically. If the
problem recurs repeatedly,
possible causes may be: volume
contains bad sector; volume is thin
device not attached to any storage
pool; volume is attached to storage
pool that is full.

4123

Group

Error

Read and write to volume Read from and write to volume


failed
failed. System retried and failed
again. System will retry
automatically. If the problem recurs
repeatedly, contact EMC Customer
Support.

4125

Group

Error

Journal capacity
insufficient

Journal capacity is currently


insufficient for the required
protection window.

4126

Group

Error off

Journal capacity
sufficient

Journal capacity is currently


sufficient for the required
protection window.

4127

Group

Warning

Predicted journal
capacity insufficient

Journal capacity is predicted to be


insufficient for the required
protection window.

4128

Group

Warning
off

Predicted journal
capacity sufficient

None

4131

Group

Warning

Transfer paused or
synchronizing for
unusually long time

None

4132

Group

Warning
off

Transfer resumed

Transfer has resumed (following


long pause or synchronization)

4133

Group

Error

Starting copy regulation

The system has detected an


unstable state in this copy and
therefore the copy is being
regulated for a period of <x> minutes
in order to allow for the
environment to stabilize. See
Details for cause of regulation.
Event logs can also be a good

324

RecoverPoint 4.4 Administrator's Guide

If you extended a storage pool or


attached a device to a storage pool,
run the CLI command
rescan_san If problem persists,
contact EMC Customer Support.

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

source for the detection of system


issues. Look for repetitive errors,
that may indicate a specific
problem in the system. Also, please
check SAN/IP events outside of
RecoverPoint, as instabilities may
not originate from RecoverPoint. If
regulation persists, collect all
system information, and contact
EMC Customer Support
4134

Group

Error off

Copy regulation has


ended

Copy regulation has ended due to a


user action or internal timeout.

4139

Splitter

Warning

Storage unreachable

The configured login credentials are Ensure that there is connectivity


incorrect or the storage is
between the RPA cluster and the
unreachable.
storage, and that the entered
credentials are correct, and try
again.

4158

Group

Warning

Journal capacity
extended

Journal capacity is currently


extended for the required
protection window

4163

Group

Error

Snap-based replication
has been disabled

The storage cannot perform the


operations necessary for snapbased replication. Snap-based
replication has been paused for a
period of %
%t_SC_ArrayOperationsFreezeInter
val%% minutes in order to allow
you to fix any issues with the
storage.

4164

Group

Error off

Snap-based replication
has been restored

Snap-based replication error has


ended due to user action or internal
system timeout.

4165

Group

Error

Possible integrity issue


detected

A possible replication integrity


issue has been detected when
using the periodic integrity check.

4166

Group

Error

Replication integrity
check failed to run

Replication integrity check failed to Wait for the current integrity check
run because another integrity check to end, or stop the current process.
is already running on the same RPA.

4167

Group

Error

Replication integrity
check stopped before it
finished

Replication integrity check stopped


before it finished, and no
inconsistency has been found.

Run the replication integrity check


again.

4175

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is adding


an RPA to the cluster.

No action required.

See the More Information area for a


description of the exact problem.
After fixing the problem, run the
clear_all_storage_errors
CLI command to restart snap-based
replication. If problem persists,
collect system information, and
contact EMC Customer Support.

Use the set_markers CLI


command to trigger a full sweep. If
problem persists, contact EMC
Customer Support.

Detailed events

325

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4176

RPA
Cluster

Info

System has entered


maintenance mode

The RecoverPoint system is


performing a major version
upgrade.

No action required.

4177

RPA
Cluster

Info

System has entered


maintenance mode

The RecoverPoint system is


performing a minor version
upgrade.

No action required.

4178

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is


performing an RPA replacement.

No action required.

4179

RPA

Info

System has exited


maintenance mode

The RecoverPoint system is no


No action required.
longer adding an RPA to the cluster.

4180

RPA
Cluster

Info

System has exited


maintenance mode

The RecoverPoint system is no


longer performing a major version
upgrade.

No action required.

4181

RPA
Cluster

Info

System has exited


maintenance mode

The RecoverPoint system is no


longer performing a minor version
upgrade.

No action required.

4182

RPA

Info

System has exited


maintenance mode

The RecoverPoint system is no


longer performing an RPA
replacement.

No action required.

4183

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is


performing an RPA conversion from
class to RPSE.

No action required.

4184

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is no


longer performing an RPA
conversion from class to RPSE.

No action required.

4185

RPA
Cluster

Info

System has entered


maintenance mode

System has entered user-initiated


maintenance mode

No action required.

4186

RPA
Cluster

Info

System has exited


maintenance mode

System has exited user-initiated


maintenance mode

No action required.

4230

RPA
Cluster

Info

All initiators registered


with storage

All RPAs initiators were successfully


registered with the storage system.

4231

RPA

Info

RPA initiators registered


with storage

RPA initiators were successfully


registered with the storage system.

4232

RPA
Cluster

Info

Storage group created on Storage group was successfully


storage
created on the storage system

4233

RPA
Cluster

Info

All storage groups


created on storage

All storage groups were


successfully created on the storage
system

4311

Group

Info

Load balancing
recommendation

None

4501

Mgmt

Warning

License invalid in
RecoverPoint

You have installed a


RecoverPoint/SE license although

326

RecoverPoint 4.4 Administrator's Guide

To benefit from RecoverPoint/SE


simplifications and the Unisphere

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

the system was installed with the


Deployment Manager RecoverPoint
Installer Wizard.

RecoverPoint/SE plug-in, you must


perform one of the following
actions:
- - If using a release earlier than 3.4
SP1: 1) upgrade the system to
release 3.4 SP1 or later using the
Deployment Manager Upgrade RPAs
Wizard, 2) convert the environment
to RecoverPoint/SE using the
Deployment Manager Converter
Wizard.
- - If using release 3.4 SP1 or later,
convert the environment to
RecoverPoint/SE using the
Deployment Manager Converter
Wizard.
To benefit from full RecoverPoint
functionality, such as bandwidth
reduction and support for
additional capacity, you must
perform the following actions:
- - If using a release earlier than 3.4
SP1: 1) upgrade the system to
release 3.4 SP1 or later using the
Deployment Manager Upgrade RPAs
Wizard, 2) convert the environment
to RecoverPoint using the
Deployment Manager Converter
Wizard.
- - If using release 3.4 SP1 or later:
1) convert the environment to
RecoverPoint using the Deployment
Manager Converter Wizard, 2)
upgrade the existing
RecoverPoint/SE license to a
RecoverPoint license.

4502

Mgmt

Warning

License invalid in
RecoverPoint/SE

You have installed a RecoverPoint


license although the system was
installed with the Deployment
Manager RecoverPoint/SE Installer
Wizard.

5008

Splitter

Warning

Host shut down

None

5010

Splitter

Warning

Splitter stopped

Depending on the policy, writing by


host may be disabled for some
groups, and a full sweep may be
required for other groups.

5011

Splitter

Info

Splitter stopped

Full sweep will be required.

5012

Splitter

Warning

Splitter stopped

Writes to replication volumes


disabled

5050

Splitter

Warning

Failed to collect system


info

None

Verify that all arrays and vCenter


Severs have been registered with
the correct credentials.

Detailed events

327

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

5051

Splitter

Warning

Unregistered array or
vCenter Server found

No login credentials have been


defined for this array or vCenter
Server. Define login credentials to
extend the period in which system
info is saved, from three days to
thirty days. It is recommended that
you do so as soon as possible.

5054

Group

Warning

Consistency group is
configured with a LUN
greater than 2 TB and a
VNX/CLARiiON splitter
version that does not
support the LUN.

None

5055

Group

Warning
off

Consistency group was


configured with a LUN
greater than 2 TB and a
VNX/CLARiiON splitter
version that did not
support the LUN. Now
the splitter version
supports the LUN.

None

5056

Splitter

Warning

SCSI errors detected in


SANTap based
replication short
synchronization might
occur.

None

5057

Group

Warning

Volume configuration not Consistency group contains a


supported by VNX/
replication set with volumes that
CLARiiON splitter version are not equal in size, and this is not
supported by the VNX/CLARiiON
splitter version currently installed
on the system.

5058

Group

Warning
off

Volume configuration
Replication set volume
supported by VNX/
configuration is now supported by
CLARiiON splitter version the VNX/CLARiiON splitter version
installed on the system.

6000

Group

Error

Image cannot be
accessed

An unrecognised error has


occurred. The specified image of
your distributed group cannot be
accessed.

Try accessing a different image. If


you cannot access any other
images, contact EMC Customer
Service.

6001

Group

Error off

Stopped trying to access


inaccessible image

The system has stopped trying to


access an inaccessible image of a
distributed group.

No user action is required.

8000

Mgmt

Error

Calling home

None

A service request is being opened


for you with EMC Customer Service.

8001

Mgmt

Error

Call home event failure

None

Contact EMC Customer Support.

328

RecoverPoint 4.4 Administrator's Guide

User Action

No action required.

Reconfigure the replication set so


that all of its volumes are of the
same size. Note: Modifying the
volumes in a replication set causes
a volume sweep on all of the
volumes in the replication set.

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

8002

RPA
Cluster

Info

VM created

None

8003

RPA
Cluster

Info

VM network reconfigured None

8004

RPA
Cluster

Info

Private network created


on ESX

None

8005

RPA
Cluster

Info

Private network on ESX


erased

None

8101

RPA
Cluster

Info

Array provider registered

None

8102

RPA
Cluster

Info

Array provider
unregistered

None

8103

RPA
Cluster

Info

Storage was registered

None

8104

RPA
Cluster

Info

Storage was
unregistered

None

8105

RPA
Cluster

Warning

Lost connectivity to
storage

None

8106

RPA
Cluster

Info

Restored connectivity to
storage

None

8107

RPA
Cluster

Info

Resource pool was


registered

None

8108

RPA
Cluster

Info

Resource pool was


unregistered

None

8200

Group

Info

Consistency group was


created on an array

None

8201

Group

Info

Snap-based replication
is done

None

8202

Group

Info

Bookmark has been


applied

None

8203

Group

Error

Failed to create snap on


storage

None

8204

Group

Error

Failed to replicate snap


on time

None

8205

Group

Error

Failed to attach snapshot None


to snap interface

9998

RPA

Info

Cannot identify event

None

10203

RPA

Error off

RPA cluster was down

None

User Action

Ensure that there is connectivity


with the storage and that the
entered credentials are correct, and
try again. If problem persists,
contact EMC Customer Support.

Detailed events

329

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

10206

RPA

Brief
error

Internal process
restarted

None

10207

RPA

Brief
error

Internal process
restarted

None

10210

RPA

Error

High load occurring


during group
initialization

High load situation is ongoing.

Run detect_bottlenecks for


the consistency group in which the
high load condition has been
detected.Consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

10211

RPA

Error

SAN communication
problem between
splitters and RPAs
existed but has resolved
itself

None

Temporary issue on the SAN caused


communication problems between
splitters and RPAs at the specified
cluster. The problem resolved itself.

10212

RPA

Error off

Fibre Channel is back up

Temporary problem in Fibre


Channel link between splitters and
RPAs has been resolved.

10213

RPA

Error off

High load occurring


during group
initialization

High load situation is ongoing.

10216

RPA

Brief
error

Internal process
restarted

None

10501

RPA

Info

Synchronization
complete

None

10502

RPA

Info

Access to image enabled Access to an image at the copy has


been enabled.

10503

RPA

Error

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

10504

RPA

Info

Access to image
disabled

Access to an image at the copy has


been disabled.

10505

RPA

Info

Image access log writes


have been undone

None

10506

RPA

Info

Roll to physical image


complete

Logged access to physical image


now enabled.

10507

RPA

Info

Journal available again

Due to system changes the journal


was temporarily out of service, but
it is now available again.

330

RecoverPoint 4.4 Administrator's Guide

User Action

Run detect_bottlenecks for


the consistency group in which the
high load condition has been
detected. Consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

10508

RPA

Info

Data flushed from source All data has been flushed from the
RPA
source RPA. Failover will proceed.

10509

RPA

Info

Initial long initialization


complete

10510

RPA

Info

Following a pause
None
transfer system now
cleared to restart transfer

10511

RPA

Info

Finished recovering
replication backlog

10512

RPA

Info

Following a pause
None
transfer system now
cleared to restart transfer

12000

RPA

Error

Stable root cause

None

12001

RPA

Error

Splitter is down

None

12002

RPA

Error

Error in links to all other


clusters

None

12003

RPA

Error

Error in link to an RPA at


other cluster(s)

None

12004

RPA

Error

Error in data link to other Error in data link. All RPAs at cluster
cluster(s)
are unable to transfer replicated
data to other cluster(s) due to
communication failure over the
WAN link. Data transfer between
RPA clusters has stopped.

User Action

None

None

- Check if other cluster is down.


- - In case of IP connectivity:
- o Check RecoverPoint gateways
and IPs configuration.
- o Check if there are any ISPrelated issues.
- o Check RecoverPoint firewall
configuration.
- - In case of FC connectivity, check
RecoverPoint zoning.
- - If you recently changed the
network configuration, please verify
that RecoverPoint configuration
matches the new network
configuration.
- - If none of the above instructions
resolve this issue, collect system
information from all clusters
experiencing connectivity issues,
and contact EMC Customer Service.
-

1. Check if replication process at


other cluster is down.
- 2. In case of IP connectivity:
- a. check RP gateways/IPs
configuration.
- b. Check firewall configuration.
- 3. If you recently changed network
configuration, please verify that RP
configuration matches the new

Detailed events

331

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
network configuration.
-

12005

RPA

Error

Error in data link over


WAN. RPA is unable to
transfer replicated data
to other cluster

None

12006

RPA

Error

RPA is disconnected
from the RPA cluster

None

12007

RPA

Error

All RPAs are


disconnected from the
RPA cluster

None

12008

RPA

Error

RPA is down

None

12009

RPA

Error

Link entered high load

Due to heavy I/O activity a link


entered a high load state in order to
prevent I/O failures on that link.
The following are among the
possible causes of the high load:RPA is unable to handle the large
volume of incoming data. (RPA
performance statistics are
presented in the Release Notes that
accompany each RecoverPoint
product release.) - Journal reaches
capacity, because the rate of the
distribution process consistently
lags behind the rate of incoming
data to the copy journal.
- - WAN is too slow to handle the
data rate. - Compression for WAN
optimization is too high, such that
the RPA is unable to handle the
volume of incoming data. Peak I/O
activity in the SAN causes a
temporary bottleneck in the
environment. This can be
considered normal behavior, and
does not necessarily require user
action.

12010

RPA

Error

Journal error -- full sweep None


to be performed after
error is corrected

332

RecoverPoint 4.4 Administrator's Guide

1. Check if replication process at


other cluster is down.
- 2. In case of IP connectivity:
- a. check RP gateways/IPs
configuration.
- b. Check firewall configuration.
- 3. If you recently changed network
configuration, please verify that RP
configuration matches the new
network configuration.
-

If high load persists, consider


running the balance_load
command and applying the load
balancing recommendation, or
manually modifying the preferred
RPA of each group according to the
recommendation. Check for
scheduled activities in your
environment. When schedule
permits, run
detect_bottlenecks for the
consistency group in which the high
load condition was detected. If
relevant, consider enabling fast
first-time initialization. (For details,
see the RecoverPoint
Administrators Guide).

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

12011

RPA

Error

Image access log or


virtual buffer is full

Access to copy volumes for this


group at this RPA cluster has been
disabled.

If you do not wish to retain the


writes that were added to the copy
image while image access was
enabled, disable image access; the
relevant writes are automatically
undone. Alternatively, use the undo
writes command to undo the same
writes; image access remains
enabled.
- If, however, you wish to retain the
writes that were added to the copy
image while image access has been
enabled, you must increase the size
of the copy journal by adding one or
more volumes to the journal.
Alternatively, you can set the image
access mode to direct; however, in
that case, the copy journal is
erased, and a full sweep
resynchronization will be required
to restore consistency between the
production and copy images.
- To help prevent this problem from
recurring, increase the size of your
image access log, using one or both
of the following:
- - Add volumes to the journal.
- - Modify the policy setting for the
portion of the journal allocated to
the image access log. This is
permitted only when image access
is disabled.

12012

RPA

Error

Cannot enable virtual


access

None

12013

RPA

Error

Cannot enable image


access

Cannot enable image access to the


specified image.

12014

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and all splitters and storage is
down.

12016

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and all storage is down.

12022

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


splitters or storage volumes (or
both) is down.

12023

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


all splitters and storage is down.

Detailed events

333

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

12024

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


all splitters is down.

12025

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


all storage is down.

12026

RPA

Error

Error occurred in link to


None
an RPA at other cluster(s)

12027

RPA

Error

RPAs are unable to


access any volume at
this cluster that belongs
to this consistency group

RPAs cannot complete I/O


operations at this cluster to any
volume -- replication or journal -- in
the consistency group.

12029

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and one or more replication
volumes is down.

12030

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and one or more journal volumes is
down.

12031

RPA

Error

Error occurred in link


between one or more
RPAs at cluster and all
RPAs at other cluster(s)

None

Ensure that at RPA(s) cluster are


connected properly to the other
cluster(s).

12032

RPA
Cluster

Error

Error occurred in link


between all RPAs at
cluster and all RPAs at
other cluster(s)

None

Ensure that all RPAs at cluster are


connected properly to the other
cluster(s)

12033

RPA

Error

volume is not accessible; None


repository data may be
lost

12034

RPA

Error

Writes to storage may


have occurred without
corresponding writes to
RPA

None

12035

RPA

Error

Cluster control at other


cluster(s) unavailable

Error occurred in link to cluster


leader RPA at other cluster(s).

12036

RPA

Error

Negotiation of transfer
protocol failed

Negotiation of transfer protocol


failed due to network issues in TCP
layer between clusters

12037

RPA

Error

All replication volumes in None


the consistency group (or

334

RecoverPoint 4.4 Administrator's Guide

User Action

Suggestions: Check your SAN and


storage configurations for any
irregularities; for example, incorrect
LUN masking or zoning. Ensure that
the Reservations Support policy
parameter setting (enabled/not
enabled) is appropriate for your
environment, as described in the
RecoverPoint Administrators
Guide.

If problem persists, check your TCP


network.

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

groups) are not


accessible
12038

RPA

Error

All journal volumes in


None
the consistency group (or
groups) are not
accessible

12039

RPA

Warning

Long initialization
started

None

12040

RPA

Error

System detected bad


sectors in volume

None

12041

RPA

Error

Error writing to volume

Ensure that RecoverPoint has write


permission to all volumes. Possible
causes: volume is read-only;
volume contains bad sector;
volume is thin device not attached
to any storage pool; volume is
attached to storage pool that is full.

12043

RPA

Error

Splitter writes may have


failed

None

12044

RPA

Error

Problem with IP link


None
between RPAs (in at least
in one direction)

12045

RPA

Error

Problem with all IP links


between RPAs

None

12046

RPA

Error

Problem with IP link


between RPAs

None

12047

RPA

Error

RPA network interface


card (NIC) problem

None

12048

RPA

Error

Unsupported splitter
version

None

12049

RPA

Info

RPA has entered


maintenance mode

None

12050

RPA

Error

RecoverPoint has
None
dynamically started
replicating
asynchronously to one of
the replicas of this
group. The group will
now be initialized.
During initialization data
is not transferred
synchronously

12051

RPA
Cluster

Error

VNX/CLARiiON storage
disaster

If you extended a storage pool or


attached a device to a storage pool,
run the rescan_san CLI
command. If problem cannot be
solved, contact EMC Customer
Support.

A VNX/CLARiiON storage (and


optionally VNX/CLARiiON splitter)
Detailed events

335

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

disaster has occurred. All of the


RPAs at the cluster are not able to
connect to all of the VNX/CLARiiON
storage arrays and splitters (if any).
12052

RPA
Cluster

Error

Storage disaster

A storage disaster has occurred. All


of the RPAs at the cluster are not
able to connect to all of the arrays
or vCenter Servers.

Contact your SAN administrator and


identify connectivity issues:
- 1. Check for recent changes in the
fibre channel configuration. Check
the fibre channel switch
configuration (VSAN and active
zoning) and communication status

12053

RPA
Cluster

Error

VNX/CLARiiON storage
disaster with working
splitter

A VNX/CLARiiON storage disaster


has occurred, but at least one of
the VNX/CLARiiON splitters is up.
However, all of the RPAs at the
cluster are not able to connect to all
of the VNX/CLARiiON storage
arrays.

Contact your SAN administrator to


identify connectivity issues: Check
for recent changes in the fibre
channel configuration. Check the
fibre channel switch configuration
(VSAN and active zoning) and
communication status, and confirm
that there is a stable fibre channel
connection between the RPA, the
switch, the storage devices, and the
splitters. Also check the name
server and confirm that the RPAs,
storage and splitters are loggedinto the fabric. Run fibre channel
connectivity tests (through the
boxmgmt diagnostics menu) on the
RPAs at the cluster where the error
was reported. Confirm that the
storage device is functioning
properly. Check the storage LUN
masking and mapping, and confirm
that the RPAs are registered and
configured to access the storage. If
none of the above instructions
resolve this issue, collect system
information from the cluster
experiencing connectivity issues,
and contact EMC Customer Service.

12054

RPA

Error

RPA to storage
connectivity failure

This RPA cannot connect to the


storage. This may be because the
RPA and the storage are not
connected in the SAN.

Contact your SAN administrator and


identify connectivity issues: Check
for recent changes in the fibre
channel configuration. Check the
fibre channel switch configuration
(VSAN and active zoning) and
communication status, and confirm
that there is a stable fibre channel
connection between this RPA, the
switch, the storage devices, and the
splitters. Also check the name

336

RecoverPoint 4.4 Administrator's Guide

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
server and confirm that the RPA is
logged-into the fabric. Run fibre
channel connectivity tests (through
the boxmgmt diagnostics menu) on
the RPA for which this error was
reported. Check the storage LUN
masking and mapping, and confirm
that the RPA is registered and
configured to access the storage. If
none of the above instructions
resolve this issue, collect system
information from the cluster
experiencing connectivity issues,
and contact EMC Customer Service.

12055

RPA

Error

RPA to VNX/CLARiiON
storage connectivity
failure

This RPA cannot connect to the


VNX/CLARiiON storage. This may be
because the RPA and the VNX/
CLARiiON storage are not connected
in the SAN.

Contact your SAN administrator and


identify connectivity issues: Check
for recent changes in the fibre
channel configuration. Check the
fibre channel switch configuration
(VSAN and active zoning) and
communication status, and confirm
that there is a stable fibre channel
connection between this RPA, the
switch, the storage devices, and the
splitters. Also check the name
server and confirm that the RPA is
logged-into the fabric. Run fibre
channel connectivity tests (through
the boxmgmt diagnostics menu) on
the RPA for which this error was
reported. Check the storage LUN
masking and mapping, and confirm
that the RPA is registered and
configured to access the storage. If
none of the above instructions
resolve this issue, collect system
information from the cluster
experiencing connectivity issues,
and contact EMC Customer Service.

12056

RPA

Error

RPA to VNX/CLARiiON
storage/splitter
connectivity failure

This RPA cannot connect to the


VNX/CLARiiON storage (and the
VNX/CLARiiON splitter, if any). This
may be because the RPA and the
VNX/CLARiiON storage are not
connected in the SAN.

Contact your SAN administrator and


identify connectivity issues: Check
for recent changes in the fibre
channel configuration. Check the
fibre channel switch configuration
(VSAN and active zoning) and
communication status, and confirm
that there is a stable fibre channel
connection between this RPA, the
switch, the storage devices, and the
splitters. Also check the name
Detailed events

337

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
server and confirm that the RPA is
logged-into the fabric. Run fibre
channel connectivity tests (through
the boxmgmt diagnostics menu) on
the RPA for which this error was
reported. If none of the above
instructions resolve this issue,
collect system information from the
cluster experiencing connectivity
issues, and contact EMC Customer
Service.

12057

RPA
Cluster

Info

Replication volume
reservation conflict

This is OK for disabled groups.

12058

RPA

Error

RPA to splitter
communication error

None

12059

RPA

Error

RPA communication
problem with volumes

RPA has encountered a


communication problem with all
volumes

12060

RPA

Error

RPA communication
problem with journal
volumes

None

12061

RPA

Error

RPA communication
problem with replication
volumes

None

12062

RPA

Error

RPA communication
problem with replication
volume

None

12063

RPA

Error

RPA communication
problem with journal
volume

None

12070

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and Phoenix System RP is down.

12071

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


Phoenix System RP is down.

12072

RPA

Error

Fibre Channel is down

FC link between RPAs down.

12073

RPA

Error

RPA communication
None
problems with repository
volume

12074

RPA

Error

FC link between RPAs


and repository volume
down

None

12075

RPA
Cluster

Error

Splitter errors while


controlling volume

None

338

RecoverPoint 4.4 Administrator's Guide

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

12076

RPA
Cluster

Error

Splitter errors while


controlling group

None

12077

RPA
Cluster

Error

Splitter errors while


controlling storage

None

12078

RPA

Error

Splitter communication
error

Although the RPA sees the splitter,


the cluster control has been unable
to communicate with it for the past
five minutes, and therefore, cannot
detect whether or not any writes
have passed through it.
- To ensure consistency between
the copy and production, transfer
will be paused. When the system
re-starts transfer, an initialization or
full sweep will follow.

To ensure data integrity, it is highly


recommended that any snapshots
created in the past five minutes are
not selected for failover.

12079

RPA

Error

VNX/CLARiiON splitter
communication error

Although the RPA sees the splitter,


the cluster control has not been
able to communicate with the
splitter for the past five minutes.
- If one of the storage processors
(SP-A or SP-B) is available and is
able to communicate with the other
storage processor, RecoverPoint
will momentarily detect whether or
not writes have passed through the
splitter.
- If writes have passed through the
splitter, no storage processors are
available and communicating with
the other storage processor, or the
splitter has rebooted, transfer will
be paused To ensure consistency
between the copy and production,
when the system re-starts transfer,
an initialization or full sweep will
follow.
- If no writes have passed through
the splitter, one of the storage
processors are available and
communicating with the other, and
the splitter has not rebooted,
transfer will not be paused, and no
initialization or full sweep will be
necessary.

To ensure data integrity, it is highly


recommended that any snapshots
created in the past five minutes are
not selected for failover.

12080

RPA

Error

Communications
problem in internal
process

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes, To
determine the cause of this
Detailed events

339

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

problem, collect logs for the period


when the event occurred, and
contact EMC Customer Support.
12081

RPA

Error

Communications
problem in internal
process

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes, To
determine the cause of this
problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

12082

RPA

Error off

Communications
problem in internal
process

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes, To
determine the cause of this
problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

12083

RPA

Error

Communications with
Communications with Symmetrix
Symmetrix director failed director has failed repeatedly.

12085

Mgmt

Info

User logged in

None

12086

RPA

Warning

RPA disk drive failure


detected

None

Contact EMC Customer Support.

12087

RPA

Warning

RPA fan failure detected

None

Contact EMC Customer Support.

14001

RPA

Error off

Splitter is up and version None


is supported

14002

RPA

Error off

All links to all other


clusters have been
restored

None

14003

RPA

Error off

Link to an RPA at other


cluster(s) has been
restored

None

14004

RPA

Error off

Data link has been


restored. All RPAs at
cluster are able to
transfer replicated data
to other cluster(s).

None

14005

RPA

Error off

Data link over WAN has


been restored. RPA is
able to transfer
replicated data to other
cluster

None

340

RecoverPoint 4.4 Administrator's Guide

Check for communication failure,


port failure, faulty zoning, or
configuration errors.

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

14006

RPA

Error off

Connection of RPA to the


RPA cluster is restored

None

14007

RPA

Error off

Connection of all RPAs to None


the RPA cluster is
restored

14008

RPA

Error off

RPA is up

None

14009

RPA

Error off

Link was in high load,


but has now retuned to
normal operation

Due to heavy I/O activity, the link


entered a high load state in order to
prevent I/O failures on that link.
The link has since exited the high
load state, and has returned to
normal operation. The following are
among the possible causes of the
high load:
- 1. RPA is unable to handle the
large volume of incoming data.
(RPA performance statistics are
presented in the Release Notes that
accompany each RecoverPoint
product release.)
- 2. Journal reaches capacity,
because the rate of the distribution
process consistently lags behind
the rate of incoming data to the
copy journal.
- 3. WAN is too slow to handle the
data rate. -Compression for WAN
optimization is too high, such that
the RPA is unable to handle the
volume of incoming data.
- 4. Peak I/O activity in the SAN
causes a temporary bottleneck in
the environment. This can be
considered normal behavior, and
does not necessarily require user
action.

14010

RPA

Error off

Journal error corrected -full sweep required

None

14011

RPA

Error off

Image access log or


virtual buffer no longer
full

None

14012

RPA

Error off

Virtual access to image


enabled

None

14013

RPA

Error off

No longer trying to
access a diluted image

None

User Action

If high load persists, consider


running the balance_load
command and applying the load
balancing recommendation, or
manually modifying the preferred
RPA of each group according to the
recommendation. Check for
scheduled activities in your
environment. When schedule
permits, run
detect_bottlenecks for the
consistency group in which the high
load condition was detected. If
relevant, consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

Detailed events

341

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

14014

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and all splitters and storage is
restored.

14016

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and all storage is restored.

14022

RPA

Error off

Fibre Channel is back up

Fibre Channel link that was down


between RPA and splitters is
restored.

14023

RPA

Error off

Fibre Channel is back up

Fibre Channel link between RPA and


all splitters and storage is restored.

14024

RPA

Error off

Fibre Channel is back up

Fibre Channel link between RPA and


all splitters is restored

14025

RPA

Error off

Fibre Channel is back up

Fibre Channel link between RPA and


all storage is restored

14026

RPA

Error off

The link to an RPA at


None
other cluster(s) has been
restored

14027

RPA

Error off

Access to all volumes in None


the consistency group (or
groups) is restored

14029

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and one or more user volumes is
restored.

14030

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and one or more journal volumes is
restored.

14031

RPA

Error off

Link between one or


None
more RPAs at cluster and
all RPAs at the other
cluster(s) has been
restored

14032

RPA
Cluster

Error off

Link between all RPAs at


cluster and all RPAs at
the other cluster(s) has
been restored

None

14033

RPA

Error off

Access to volume
restored

Restored repository data integrity.

14034

RPA

Error off

Replication consistency
to storage restored

Replication consistency in writes to


storage has been restored.

14035

RPA

Error off

Cluster control at other


cluster(s) available

The link to cluster leader RPA at


other cluster(s) has been restored.

342

RecoverPoint 4.4 Administrator's Guide

User Action

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

14036

RPA

Error off

Negotiation of transfer
protocol successful
again

Negotiation of transfer protocol,


which failed to due to network
issues in TCP layer between
clusters, has been restored.

14037

RPA

Error off

Access to all replication


volumes in the
consistency group (or
groups) has been
restored

None

14038

RPA

Error off

Access to all journal


volumes in the
consistency group (or
groups) is restored

None

14039

RPA

Warning
off

Long initialization
completed

None

14040

RPA

Error off

Bad sectors in volume


have been corrected

None

14041

RPA

Error off

Write access to volume


restored

None

14042

RPA

Error off

Synchronization in
progress to restore any
failed writes in group

None

14043

RPA

Error off

Synchronization in
progress to restore any
failed writes

None

14044

RPA

Error off

Problem with IP link


None
between RPAs (in at least
in one direction)
corrected

14045

RPA

Error off

All IP links between RPAs None


restored

14046

RPA

Error off

IP link between RPAs


restored

None

14047

RPA

Error off

RPA network interface


card (NIC) problem
corrected

None

14049

RPA

Info

RPA is out of
maintenance mode

None

14050

RPA

Error off

RecoverPoint has
None
dynamically resumed
synchronous replication
to one of the replicas of
this group. The group will
now be initialized.

User Action

Detailed events

343

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

During initialization data


is not transferred
synchronously
14051

RPA
Cluster

Error off

End of VNX/CLARiiON
storage disaster

The VNX/CLARiiON storage (and


optionally VNX/CLARiiON splitter)
disaster is now over. All of the RPAs
at the cluster are now able to
connect to all of the VNX/CLARiiON
storage arrays and VNX/CLARiiON
splitters (if any).

14052

RPA
Cluster

Error off

End of storage disaster

The storage disaster is now over. All No user action is required.


of the RPAs at the cluster are now
able to connect to all of the arrays
or vCenter Servers.

14053

RPA
Cluster

Error off

End of VNX/CLARiiON
storage disaster with
working splitter

The VNX/CLARiiON storage disaster, No user action is required.


in which at least one of the VNX/
CLARiiON splitters was working, is
now over. All of the RPAs at the
cluster are now able to connect to
all of the storage arrays.

14054

RPA

Error off

End of RPA to storage


connectivity failure

This RPA was not able to connect to


the storage, but since then,
connectivity has been restored.

No user action is required.

14055

RPA

Error off

End of RPA to VNX/


CLARiiON storage
connectivity failure

This RPA was not able to connect to


the VNX/CLARiiON storage, but
since then, connectivity has been
restored.

No user action is required.

14056

RPA

Error off

End of RPA to VNX/


CLARiiON storage/
splitter connectivity
failure

This RPA was not able to connect to


the VNX/CLARiiON storage (and the
VNX/CLARiiON splitter, if any), but
since then, all connectivity has
been restored.

No user action is required.

14057

RPA
Cluster

Info

Group volumes
reservation conflicts
were fixed

None

14058

RPA

Error off

End of splitter to RPA


connectivity error

None

14059

RPA

Error off

End of connectivity
problems between all
group volumes and RPA

None

14060

RPA

Error off

End of connectivity
None
problems between group
journal volumes and RPA

14061

RPA

Error off

End of connectivity
None
problems between group

344

RecoverPoint 4.4 Administrator's Guide

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

replication volumes and


RPA
14062

RPA

Error off

End of connectivity
None
problems between group
replication volume and
RPA

14063

RPA

Error off

Connectivity problems
with group journal
volume and RPA were
fixed

14070

RPA

Error off

Fibre Channel is up again Fibre Channel link between all RPAs


and Phoenix System RP is restored.

14071

RPA

Error off

Fibre Channel is up again Fibre Channel link between RPA and


Phoenix System RP is restored.

14072

RPA

Error off

Fibre Channel is up again FC link between RPAs restored

14073

RPA

Error off

Connectivity between
repository volume and
RPA restored

None

14074

RPA

Error off

FC link between all RPAs


and repository volume
restored

None

14075

RPA
Cluster

Error off

Control issues between


volume and splitter
corrected

None

14076

RPA
Cluster

Error off

Control issues between


group and splitter
corrected

None

14077

RPA
Cluster

Error off

Control issues between


storage and splitter
corrected

None

14078

RPA

Error off

End of splitter
communication error

The cluster control is able to


communicate with the splitter
again. To ensure consistency
between the copy and production,
initialization or full sweep will
follow.

No user action is required.

14079

RPA

Error off

End of VNX/CLARiiON
splitter communication
error

The cluster control is able to


communicate with the VNX/
CLARiiON splitter again.
- If any writes have passed through
the splitter, initialization or full
sweep will follow to ensure
consistency between the copy and
production.
- If no writes have passed through

No user action is required.

None

Detailed events

345

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details
the splitter, transfer will not be
paused and initialization or full
sweep will not be necessary.

14080

RPA

Error off

Fix of communications
problem in internal
process

None

14081

RPA

Error off

Internal process
communication problem
has been fixed

None

14082

RPA

Error off

Fix of communications
problem in internal
process

None

14083

RPA

Error off

Communication with
Symmetrix director
restored

After a failure, communications


with a Symmetrix director has been
restored successfully.

14086

RPA

Warning
off

RPA disk drive failure


resolved

None

14087

RPA

Warning
off

RPA fan failure resolved

None

16000

RPA

Brief
error

Transient root cause

None

16001

RPA

Brief
error

Brief splitter error


corrected. Problem has
been corrected.

Splitter was down. Problem has


been corrected.

16002

RPA

Brief
error

Brief error in links to all


other clusters corrected

Error occurred in all links to all


other clusters. Problem has been
corrected.

16003

RPA

Brief
error

Brief error in link to an


RPA at other cluster(s)
corrected

Error occurred in link to an RPA at


other cluster(s). Problem has been
corrected.

16004

RPA

Brief
error

Brief error in data link to Error in data link. All RPAs at cluster
other cluster(s) corrected are unable to transfer replicated
data to other cluster(s). Problem
has been corrected.

16005

RPA

Brief
error

Error occurred in data


link over WAN. RPA is
unable to transfer
replicated data to other
cluster. Problem has
been corrected

None

16006

RPA

Brief
error

RPA was disconnected


from the RPA cluster.
Connection has been
restored

None

346

RecoverPoint 4.4 Administrator's Guide

User Action

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

16007

RPA

Brief
error

All RPAs were


None
disconnected from the
RPA cluster. Problem has
been corrected

16008

RPA

Brief
error

RPA was down. Problem


has been corrected

16009

RPA

Brief
error

Link was in high load but Due to heavy I/O activity, a link
has now returned to
entered a high load state in order to
normal operation
prevent I/O failures on that link.
The link has since exited the high
load state, and has returned to
normal operation. The following are
among the possible causes of the
high load: RPA is unable to handle
the large volume of incoming data.
(RPA performance statistics are
presented in the Release Notes that
accompany each RecoverPoint
product release.) Journal reaches
capacity, because the rate of the
distribution process consistently
lags behind the rate of incoming
data to the copy journal. WAN is too
slow to handle the data rate.
Compression for WAN optimization
is too high, such that the RPA is
unable to handle the volume of
incoming data. Peak I/O activity in
the SAN causes a temporary
bottleneck in the environment. This
can be considered normal behavior,
and does not necessarily require
user action.

16010

RPA

Brief
error

Journal error occurred.


Problem has been
corrected -- full sweep
required

None

16011

RPA

Brief
error

Briefly image access log


or virtual buffer was full

Problem has been corrected.

16012

RPA

Brief
error

Briefly unable to enable


virtual access to image

Problem has been corrected.

16013

RPA

Brief
error

Was unable to enable


access to specified
image. Problem has
been corrected

None

User Action

None
If high load persists, consider
running the balance_load
command and applying the load
balancing recommendation, or
manually modifying the preferred
RPA of each group according to the
recommendation. Check for
scheduled activities in your
environment. When schedule
permits, run
detect_bottlenecks for the
consistency group in which the high
load condition was detected. If
relevant, consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

Detailed events

347

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

16014

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and all splitters and storage was
down. Problem has been corrected.

16016

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and all storage was down. Problem
has been corrected.

16022

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between RPA and


splitters was briefly down. Problem
has been corrected.

16023

RPA

Brief
error

Fibre Channel is back up


again

None

16024

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between RPA and


all splitters was down. Problem has
been corrected.

16025

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between RPA and


all storage was down. Problem has
been corrected.

16026

RPA

Brief
error

Brief RPA and all storage


error corrected

Error occurred in link to an RPA at


other cluster(s). Problem has been
corrected.

16027

RPA

Brief
error

Brief group(s) volumes


accessibility error
corrected

All volumes in the consistency


group (or groups) were not
accessible. Problem has been
corrected.

16029

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and one or more replication
volumes was down. Problem has
been corrected.

16030

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and one or more journal volumes
was down. Problem has been
corrected.

16031

RPA

Brief
error

Brief link error between


RPA(s) at cluster and all
RPAs at other cluster(s)
corrected

Error occurred in link between one


or more RPAs at cluster and all RPAs
at the other cluster(s). Problem has
been corrected.

16032

RPA
Cluster

Brief
error

Brief link error between


all RPAs at cluster and all
RPAs at other cluster(s)
corrected

Error occurred in link between all


RPAs at cluster and all RPAs at
other cluster(s). Problem has been
corrected.

16033

RPA

Brief
error

Brief volume
accessibility error
corrected

volume was not accessible.


Problem has been corrected.

348

RecoverPoint 4.4 Administrator's Guide

User Action

Fibre Channel link between RPA and


all splitters and storage was down.
Problem has been corrected.

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

16034

RPA

Brief
error

Brief write consistency


error corrected

Writes to storage may have


occurred without corresponding
writes to RPA. Problem has been
corrected.

16035

RPA

Brief
error

Cluster control at other


Error occurred in link to cluster
cluster(s) available again leader RPA at other cluster(s).
Problem has been corrected.

16036

RPA

Brief
error

Negotiation of transfer
protocol successful
again

16037

RPA

Brief
error

Brief group(s) replication All replication volumes in the


volumes accessibility
consistency group (or groups) were
error corrected
not accessible. Problem has been
corrected.

16038

RPA

Brief
error

Brief group(s) journal


volumes accessibility
error corrected

All journal volumes in the


consistency group (or groups) were
not accessible. Problem has been
corrected.

16039

RPA

Info

System ran long resync

None

16040

RPA

Brief
error

System had detected


bad sectors in volume.
Problem has been
corrected

None

16041

RPA

Brief
error

There was a problem


writing to volume.
Problem has been
corrected

None

16042

RPA

Brief
error

Splitter write may have


failed (while group was
transferring data).
Problem has been
corrected

None

16043

RPA

Brief
error

Splitter writes may have


failed

None

16044

RPA

Brief
error

There was a problem


with an IP link between
RPAs (in at least in one
direction). Problem has
been corrected

None

16045

RPA

Brief
error

There was a problem


with all IP links between
RPAs. Problem has been
corrected

None

User Action

Negotiation of transfer protocol


failed due to network issues in TCP
layer between clusters. Problem
has been corrected.

Detailed events

349

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

16046

RPA

Brief
error

There was a problem


with an IP link between
RPAs. Problem has been
corrected

None

16047

RPA

Brief
error

There was a RPA network None


interface card (NIC)
problem. Problem has
been corrected

16048

RPA

Brief
error

Brief unsupported
splitter version error
corrected

Splitter version was not supported.


Problem has been corrected.

16049

RPA

Info

RPA temporarily entered


maintenance mode but
has since exited

None

16050

RPA

Error

RecoverPoint had
dynamically resumed
synchronous replication
to one of the replicas of
this group but has since
started replicating
asynchronously again.
Consequentially the
group has been
initialized twice. During
initialization data was
not transferred
synchronously. If this is
not the expected
behavior contact EMC
Customer Support

None

16051

RPA
Cluster

Brief
error

End of brief VNX/


CLARiiON storage
disaster

The brief VNX/CLARiiON storage


(and optionally, VNX/CLARiiON
splitter) disaster is now over. For a
brief time, all of the RPAs at the
cluster were not able to connect to
all of the VNX/CLARiiON storage
arrays and splitters (if any), but
since then, all connectivity has
been restored.

16052

RPA
Cluster

Brief
error

End of brief storage


disaster

The brief storage disaster is now


No user action is required.
over. For a brief time, all of the RPAs
at the cluster were not able to
connect to all of the arrays or
vCenter Servers, but since then, all
connectivity has been restored.

350

RecoverPoint 4.4 Administrator's Guide

User Action

No user action is required.

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

16053

RPA
Cluster

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and all CX storage was down.
Problem has been corrected.

16054

RPA

Brief
error

End of brief RPA to


storage connectivity
failure

For a brief time, this RPA was not


able to connect to the storage, but
since then, connectivity has been
restored.

No user action is required.

16055

RPA

Brief
error

End of brief RPA to VNX/


CLARiiON storage
connectivity failure

For a brief time, this RPA was not


able to connect to the VNX/
CLARiiON storage, but since then,
connectivity has been restored.

No user action is required.

16056

RPA

Brief
error

Brief RPA to VNX/


CLARiiON storage/
splitter connectivity error
corrected

For a brief time, this RPA was not


able to connect to the VNX/
CLARiiON storage (and the VNX/
CLARiiON splitter, if any), but since
then, connectivity has been
restored.

No user action is required.

16057

RPA
Cluster

Info

Brief reservation
conflicts on group
volumes

None

16058

RPA

Brief
error

Brief connectivity
None
problem between splitter
and RPA

16059

RPA

Brief
error

Brief connectivity
problem between all
group volumes and RPA

16060

RPA

Brief
error

Brief connectivity
None
problem between group
journal volumes and RPA

16061

RPA

Brief
error

Brief connectivity
problem between group
replication volumes and
RPA

None

16062

RPA

Brief
error

Brief connectivity
problem between group
replication volume and
RPA

None

16063

RPA

Brief
error

Brief connectivity
problem between group
journal volume and RPA

None

16070

RPA

Brief
error

Fibre Channel is up again Fibre Channel link between all RPAs


and Phoenix System RP was down.
Problem has been corrected.

None

Detailed events

351

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

16071

RPA

Brief
error

Fibre Channel is up again Fibre Channel link between RPA and


Phoenix System RP was down.
Problem has been corrected.

16072

RPA

Brief
error

Fibre Channel is back up


again

Fibre channel link between RPAs


was down but problem has been
corrected and the link is back up
again.

16073

RPA

Brief
error

Brief connectivity
problem between
repository volume and
RPA

None

16074

RPA

Brief
error

FC link between RPAs


and repository volume
restored

None

16075

RPA
Cluster

Brief
error

Brief control issues


between volume and
splitter corrected

None

16076

RPA
Cluster

Brief
error

Brief control issues


between group and
splitter corrected

None

16077

RPA
Cluster

Brief
error

Brief control issues


between storage and
splitter corrected

None

16078

RPA

Brief
error

Brief splitter
communication error

A temporary splitter communication No user action is required.


error occurred, but the problem has
since been corrected. To ensure
consistency between the copy and
production, initialization or full
sweep will follow.

16079

RPA

Brief
error

Brief VNX/CLARiiON
splitter communication
error

A temporary VNX/CLARiiON splitter


communication error occurred, but
the problem has since been
corrected.
- If any writes have passed through
the splitter, initialization or full
sweep will follow to ensure
consistency between the copy and
production.
- If no writes have passed through
the splitter, transfer will not be
paused and initialization or full
sweep will not be necessary.

No user action is required.

16080

RPA

Brief
error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

352

RecoverPoint 4.4 Administrator's Guide

More Details

User Action

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

16081

RPA

Brief
error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

16082

RPA

Brief
error

Internal process
communication problem
briefly resolved

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

16083

RPA

Brief
error

Communications with
Symmetrix director
restored again

Communications with a Symmetrix


director failed briefly and has now
been restored

If this recurs frequently, contact


EMC Customer Support.

16086

RPA

Warning
off

Brief RPA power supply


failure resolved

None

Check electrical power to both


power supplies. If problem persists,
contact EMC Customer Support.

16087

RPA

Warning
off

Brief generic hardware


failure resolved

None

Contact EMC Customer Support.

18001

RPA

Error

Splitter problem, which


was considered to have
been corrected, has
reoccurred

None

18002

RPA

Error

Error occurred in all links None


to all other clusters.
Problem was temporarily
corrected, but has since
returned

18003

RPA

Error

Error occurred in link to


an RPA at other
cluster(s). Problem was
temporarily corrected,
but has since returned

None

18004

RPA

Error

Error in data link. All


RPAs at cluster are
unable to transfer
replicated data to other
cluster(s). Problem was
temporarily corrected,
but has since returned

Due to communication failure over


the WAN link, data transfer between
the clusters has stopped.

18005

RPA

Error

RPA is unable to transfer None


replicated data to other
cluster. Data link was
temporarily restored, but
problem has returned

18006

RPA

Error

Connection of RPA to the


RPA cluster was
temporarily restored but
problem has returned

None

Detailed events

353

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18007

RPA

Error

All RPAs were


temporarily restored to
the RPA cluster but
problem has returned

None

18008

RPA

Error

RPA was temporarily up


but problem has
returned -- RPA is down

None

18009

RPA

Error

Group temporarily exited None


high load but problem
has returned

18010

RPA

Error

Journal error was


None
temporarily corrected but
problem has returned

18011

RPA

Error

Image access log or


None
virtual buffer was
temporarily no longer full
and writing by hosts at
copy was re-enabled -but problem has
returned

18012

RPA

Error

Virtual access to image


was temporarily enabled
but problem has
returned

None

18013

RPA

Error

Access to image was


temporarily enabled but
problem has returned

None

18014

RPA

Error

Fibre Channel is down


again

Fibre Channel link between all RPAs


and all splitters and storage was
temporarily restored but problem
has returned

18016

RPA

Error

Fibre Channel is down


again

Fibre Channel link between all


splitters and all storage was
temporarily restored but problem
has returned.

18022

RPA

Error

Fibre Channel is down


again

Fibre Channel link that was down


between RPA and splitters or
storage volumes (or both) was
temporarily restored but problem
has returned.

18023

RPA

Error

Fibre Channel is down


again

Fibre Channel link between RPA and


all storage was temporarily restored
but problem has returned.

354

RecoverPoint 4.4 Administrator's Guide

User Action

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18024

RPA

Error

Fibre Channel is down


again

Fibre Channel link between RPA and


all splitters was temporarily
restored but problem has returned.

18025

RPA

Error

Fibre Channel is back


down again

Fibre Channel link between RPA and


all storage was temporarily restored
but problem has returned.

18026

RPA

Error

Error occurred in link to


an RPA at other
cluster(s). Problem was
temporarily corrected,
but has since returned

None

18027

RPA

Error

Access to all volumes in None


the consistency group (or
groups) was temporarily
restored but problem has
returned

18029

RPA

Error

Fibre Channel is back


down again

Fibre Channel link between all RPAs


and one or more user volumes was
temporarily restored but problem
has returned

18030

RPA

Error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and one or more journal volumes
was temporarily restored but
problem has returned

18031

RPA

Error

Error occurred in link


between one or more
RPAs at cluster and all
RPAs at the other
cluster(s). Problem was
temporarily corrected,
but has since returned

None

18032

RPA
Cluster

Error

Error occurred in link


between all RPAs at
cluster and all RPAs at
other cluster(s). Problem
was temporarily
corrected, but has since
returned

None

18033

RPA

Error

Access to volume was


temporarily restored but
problem has returned

None

18034

RPA

Error

Replication consistency
in writes to storage and
writes to RPAs was
temporarily restored but

None

User Action

Detailed events

355

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

problem may have


returned
18035

RPA

Error

Cluster control at other


cluster(s) unavailable
again

18037

RPA

Error

Access to all replication None


volumes in the
consistency group (or
groups) has been
temporarily restored, but
problem has returned

18038

RPA

Error

Access to all journal


None
volumes in the
consistency group (or
groups) was temporarily
restored but problem has
returned

18039

RPA

Warning

Long resync was


completed but has now
restarted

None

18040

RPA

Error

User marked volume as


OK but bad sectors
problem persists

None

18041

RPA

Error

Problem writing to
volume was corrected,
but has recurred

None

18042

RPA

Error

Synchronization had
None
restored any failed writes
in group but problem has
returned

18043

RPA

Error

Internal problem

18044

RPA

Error

Problem with IP link


None
between RPAs (in at least
in one direction) was
corrected but problem
has returned

18045

RPA

Error

Problem with IP link


None
between RPAs (in at least
in one direction) was
corrected but problem
has returned

18046

RPA

Error

Problem with IP link


between RPAs was

356

RecoverPoint 4.4 Administrator's Guide

Error occurred in link to cluster


leader RPA at other cluster(s).
Problem was temporarily corrected,
but has since returned.

None

None

User Action

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

corrected but problem


has returned
18047

RPA

Error

RPA network interface


card (NIC) problem was
corrected but problem
has returned

None

18049

RPA

Info

RPA temporarily exited


maintenance mode but
has since re-entered

None

18050

RPA

Error off

RecoverPoint had
dynamically resumed
synchronous replication
to one of the replicas of
this group but has since
started replicating
asynchronously again.
Consequentially the
group has been
initialized twice. During
initialization data was
not transferred
synchronously. If this is
not the expected
behavior contact EMC
Customer Support

None

18051

RPA
Cluster

Error

Recurring VNX/CLARiiON
storage disaster

None

18052

RPA

Error

Recurring storage
disaster

None

18053

RPA

Error

Recurring VNX/CLARiiON
storage disaster with
working splitter

None

18054

RPA

Error

Recurring RPA to storage


connectivity failure

None

18055

RPA

Error

Recurring RPA to VNX/


CLARiiON storage
connectivity failure

None

18056

RPA

Error

Recurring RPA to VNX/


CLARiiON storage/
splitter connectivity
failure

None

18057

RPA
Cluster

Info

Reservation conflicts
from group replication
volumes were briefly
fixed

None

Detailed events

357

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18058

RPA

Error

Splitter to RPA
connectivity error has
returned

None

18059

RPA

Error

Connectivity problems
between all group
volumes and RPA were
briefly corrected

None

18060

RPA

Error

Connectivity problems
between group journal
volumes and RPA were
briefly corrected

None

18061

RPA

Error

Connectivity problems
between group user
volumes and RPA were
briefly corrected

None

18062

RPA

Error

Connectivity problems
between group user
volume and RPA were
briefly corrected

None

18063

RPA

Error

Connectivity problems
between group journal
volume and RPA were
briefly corrected

None

18070

RPA

Error off

Fibre Channel link is


down again

Fibre Channel link between all RPAs


and Phoenix System RP was
temporarily restored but problem
has returned.

18071

RPA

Error off

Fibre Channel is down


again

Fibre Channel link between RPA and


Phoenix System RP was temporarily
restored but problem has returned.

18072

RPA

Error off

Fibre Channel is down


again

Fibre channel link between RPAs


was temporarily restored but the
problem has returned and the link
is back down again.

18073

RPA

Error

Connectivity between
repository volume and
RPA restored

None

18074

RPA

Error

Fibre Channel is down


again

Fibre Channel link between all RPAs


and repository volume was
temporarily restored but problem
has returned.

18075

RPA
Cluster

Error

Control issues between


volume and splitter
corrected

None

358

RecoverPoint 4.4 Administrator's Guide

User Action

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

18076

RPA
Cluster

Error

Control issues between


group and splitter
corrected

None

18077

RPA
Cluster

Error

Control issues between


storage and splitter
corrected

None

18078

RPA

Error

Splitter communication
error returned

The cluster control was temporarily


able to communicate with the
splitter, but since then, it has lost
communication with the splitter
again.
- To ensure consistency between
the copy and production, transfer
will be paused. When the system
re-starts transfer, an initialization or
full sweep will follow.

To ensure data integrity, it is highly


recommended that any snapshots
created in the past five minutes are
not selected for failover.

18079

RPA

Error

VNX/CLARiiON splitter
communication error
returned

The cluster control was temporarily


able to communicate with the VNX/
CLARiiON splitter, but since then, it
has lost communication with the
splitter again.
- If any writes have passed through
the splitter, initialization or full
sweep will follow to ensure
consistency between the copy and
production.
- If no writes have passed through
the splitter, transfer will not be
paused and initialization or full
sweep will not be necessary.

To ensure data integrity, it is highly


recommended that any snapshots
created in the past five minutes are
not selected for failover.

18080

RPA

Error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

18081

RPA

Error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

18082

RPA

Error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

18083

RPA

Error

Communication with
After a brief recovery,
Symmetrix director failed communications with a Symmetrix
again
director has failed again.

18086

RPA

Brief
warning

RPA power supply failure None


is back

Check for communication failure,


port failure, faulty zoning, or
configuration errors.

Detailed events

359

Events

Table 70 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18087

RPA

Brief
warning

Generic hardware failure


is back again

None

User Action

Advanced events
The following table contains the Normal scope events that are issued by the RecoverPoint
system.
Table 71 Advanced scope events

ID

Topic

Level

Summary

More Details

2004

RPA
Cluster

Info

Repository volume
created

None

2005

RPA
Cluster

Info

Repository volume
removed

None

2006

RPA
Cluster

Info

Splitter(s) added

None

2007

RPA
Cluster

Info

Splitter(s) removed

None

3003

RPA

Warning

Stopping a system
process

None

3004

RPA

Info

Starting a system
process

None

3009

RPA

Warning

A system process
stopped

None

3010

RPA

Warning

A system process
stopped

None

3031

RPA

Warning

An internal process
restarted

None

3032

RPA

Info

An internal process
restarted

None

3034

RPA

Info

Negotiating transfer
protocol

None

3035

RPA

Info

An internal process
restarted

None

4005

Group

Info

Negotiating transfer
protocol

None

4013

RPA

Info

Negotiating transfer
protocol

None

4013

RPA

Info

Negotiating transfer
protocol

None

360

RecoverPoint 4.4 Administrator's Guide

User Action

Events

Table 71 Advanced scope events (continued)

ID

Topic

Level

Summary

More Details

4014

Group

Info

Negotiating transfer
protocol

None

4050

Group

Info

Enabling access to image None

4052

Group

Warning

Enabling access to image None

4072

Group

Info

Journal cleared

None

4073

Group

Warning

Journal cleared

None

4074

Group

Error

Journal cleared

None

4075

Group

Info

Journal available

None

4076

Group

Info

Adding volume(s)

None

4077

Group

Info

Removing volume(s)

None

4092

Group

Info

Accessing image

Writing by hosts to the accessed


image has been enabled.

4093

Group

Info

Undoing writes to image


access log

None

4094

Group

Info

Writes to image access


log have been undone

None

4096

Group

Info

Image access log has


been written to storage

None

4100

RPA
Cluster

Info

Group created

Creating a new group modifies the


load distribution across RPAs.

4101

RPA
Cluster

Info

Group removed

None

4102

Group

Info

Journal volume(s)
removed

None

4103

Group

Info

Journal volume(s) added

None

4104

Group

Info

Group started accepting


writes

None

4105

Group

Info

Group stopped accepting None


writes

5000

Splitter

Info

Volume(s) attached to
splitter(s)

5001

Splitter

Info

Volume(s) detached from None


splitter(s)

5049

Splitter

Warning

Splitter write to RPA


failed

None

10000

RPA

Info

Analyzing changes in
system

None

User Action

To balance the write load across all


RPAs run the balance_load CLI
command in seven days and apply
the recommendation.

None

Advanced events

361

Events

Table 71 Advanced scope events (continued)

ID

Topic

Level

Summary

More Details

10001

RPA

Info

System is stable

System changes have occurred.


System is now stable.

10002

RPA

Info

Issuing intermediate
report

System activity has not stabilized


and therefore, an intermediate
report is being issued.

10101

RPA

Error

Unrecognised system
activity

Cause of system activity unclear.

10102

RPA

Info

Internal changes
occurred

Cluster control recorded internal


changes that do not impact system
operation.

10201

RPA

Info

Settings have changed

None

10202

RPA

Info

System changes at other


cluster

System changes have occurred at


the another cluster.

12042

RPA

Error

Splitter write may have


failed during transfer

Splitter write may have failed (while


group was transferring data).
Synchronization will be required.

362

RecoverPoint 4.4 Administrator's Guide

User Action

To obtain more information filter


events using Detailed scope.

For details of the system changes,


refer to the event log at the other
cluster(s).

You might also like