Avamar Staging Tape Out Outputting Avamar Backups To Tape Technical Note PDF

AVAMAR STAGING TAPE OUT: OUTPUTTING AVAMAR BACKUPS TO TAPE
TECHNICAL NOTES AVAMAR 4.X AND LATER SOFTWARE
TECHNICAL NOTE
P/N 300-007-419 REV A04 MAY 7, 2012
Table of Contents
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Applications Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Tape Out Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Monitoring Tape Out Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Crontab Automation of selectfortape.pl Application . . . . . . . . . . . . . . . . . . 31 Appendix A selectfortape.pl Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Appendix B tapeout.pl Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Overview
Overview
This document describes how to select specific Avamar backups and export them to tape using a redirected restore or "staging" server.
Exporting to tape consists of the following steps: 1. Run the selectfortape.pl script to select backups that you want to export to tape from the Avamar server database. This script generates an output list of selected backups on the Avamar server. The output list is based on user-defined criteria. 2. Run the tapeout.pl script on the staging server to restore the selected data to the local hard drive and then export it to tape. This script uses the output list generated by selectfortape.pl to perform a restore of the data to the staging server. This occurs until it reaches the end of the output list or when the hard drive fills up on the staging server. The installation location of the two main applications (selectfortape.pl and tapeout.pl) determines on which servers each process occurs.
AVAMAR STAGING TAPE OUT: OUTPUTTING AVAMAR BACKUPS TO TAPE TECHNICAL NOTE
Overview
tapeout.pl Modes
When tapeout.pl begins exporting to tape, it runs in one of two sets of modes. You configure these modes through tapeout.pl command line options.
Scripted, Manual Modes

MODE DESCRIPTION
Designated tape dump script
Uses a custom script for exporting backups in the staging directory to tape. Template scripts are available to modify for specific systems (for example, NetWorker). This is the default mode. Provides the means for you to manually backup the staging directory to tape.
Manual
Nonincremental, Incremental Modes

MODE DESCRIPTION
Nonincremental mode Incremental mode
After the complete contents of the staging area is exported to tape, the data is deleted. Additively restores to the destination incrementally, thus reducing restoration times and allowing more frequent staging. When restoring, it creates an /INCREMENTAL directory in stagingdir. IMPORTANT: "Additive" means that incremental restores export only new and changed files from previous backups. Incremental backups do not delete files from the staging server and tape that have been deleted from clients.
The remainder of this document is organized as follows: Hardware and software requirements Application setup Tape out procedures Monitoring and automation instructions Script reference information
Requirements
Requirements
This section outlines the hardware and software requirements that servers in a tape export setup must meet.
Source Avamar Server

All backups chosen for tape export are initially stored on the Avamar node (singlenode servers) or utility node (multi-node servers).
Hardware Requirements
There are no specific hardware requirements beyond the standard bill of materials for an Avamar server.
Software Requirements
Avamar server version 4.x or later This software must already be installed and running for valid backups to exist. Perl 5.8.0 or later This software is installed with the standard operating system. Perl-DBI module This application is installed with the standard operating system. Perl-DBD-PgPP module Refer to Applications Setup (page 7) for instructions about this module. selectfortape.pl and selectfortape_interactive.pl These are Perl scripts.
Environment Requirements
Replication or source target The Avamar server can be either a source or, preferably, a replication target. If it is a source target, EMC strongly recommends, for performance reasons, that Avamar capacity not exceed 40% or that incremental mode be utilized. Retention policies If a specific kind of backup is being archived (for example, a monthly backup), you should use advanced retention policies. To reliably select monthly backups, monthly retention policies should be different from daily or weekly ones.
Requirements
Staging Server
EMC recommends that the storage utilized for staging activity is dedicated for staging purposes during operation and is on a separate storage space outside of the local operating system files. When running in nonincremental mode, the staging directory will be deleted by tapeout.pl upon completion as described in Overview (page 2) and Requirements (page 4).
Hardware Requirements
Nonincremental mode. A staging server can be any machine similar to any Avamar client. Although not preferred, it can even be an Avamar server (for instance, a spare node). Besides hardware specifications similar to any Avamar client, an Avamar staging server must have at least 20% more storage available than the largest backup chosen (as reported by Avamar Administrator). This requirement conservatively handles the likelihood that the data is larger on an actual file system due to file system block sizes and so forth. Incremental mode. In incremental mode, the Avamar staging server should have enough backend storage to accommodate all backups to be exported to tape at the time of export. Unlike nonincremental mode, incremental mode does not delete data and manage the storage of the staging server. The storage requirements increase as more data is added to the Avamar server.
Software Requirements
Software requirements for a staging server are different from other nodes because cross-platform restores may not be supported depending on the backup product used. Both Microsoft Windows and Red Hat Enterprise Linux are supported. Microsoft Windows Requirements Microsoft Windows 2000, XP, 2003 or 2008 Avamar client of the same version as the Avamar server. Refer to the Avamar Backup Clients Installation Guide and Reference Manual for instructions, which is available on the Avamar server Documents and Downloads web page. ActivePerl 5.8.8 or later ActivePerl is available at: http://www.activeperl.com. tapeout.pl program avtar.cmd file Supported version of tape backup software (for instance, the Networker agent)
Requirements Linux Requirements Red Hat Enterprise Linux v3u8 or v4u6 Avamar client of the same version as the Avamar server. Refer to the Avamar Backup Clients Installation Guide and Reference Manual for instructions, which is available on the Avamar server Documents and Downloads web page. Perl 5.8 tapeout.pl program avtar.cmd Supported version of tape backup software (for instance, the Networker agent)
Limitations
Although supported by Avamar and certain tape backup programs, cross-platform restores are not recommended for exporting to tape. When performing restores, Windows backups should return to Windows staging servers and the same with non-Windows backups. To maintain the same operating system, use the --osinclude or --osexclude options when running the tapeout.pl script. If a single platform is used as the staging server to output to tape backups for multi-platform clients (for example, a Linux staging server is used to stage data for both Linux and Windows clients), the following should be noted: In nonincremental mode, deleting exported files and directories might be difficult after backup to tape. In that case, tapeout.pl attempts to move the directory structures that cannot be fully deleted to a renamed directory called [STAGING_DIRECTORY]_could_not_delete. Also, although data will be properly exported to tape, ACLs and permissions most likely will not be properly exported for the data from clients that do not match the operating system of the staging server.
Applications Setup
Applications Setup
Before performing export to tape procedures, you must set up the applications on the correct Avamar servers. The selectfortape.pl application must be set up on the source Avamar server and the tapeout.pl application and, if necessary, the tape dump script must be set up on the Linux or Windows staging server.
selectfortape.pl on Source Avamar Server

selectfortape.pl is an application that queries the Avamar server's backup database and selects backups that meet certain user-defined restrictions, including the following:
RESTRICTION DESCRIPTION
Start date End date Minimum retention Include Exclude
Two parameters that form a date range between which all successful backups must have been started and completed. The minimum retention on the backup. Values that exceed this minimum value are selected. The inclusion of certain domains or clients. /REPLICATE is included by default. The exclusion of domains or clients. /MC_BACKUPS is excluded by default. To do a mutual exclusion, exclude "/" before doing includes.
For multi-node servers, selectfortape.pl must be run on the Avamar utility node, and the following option is required: --snapupoutput=PATH Where PATH is the name of an Avamar client on the Avamar server that can house the newly created backup lists. The machine should be a new one created through the Avamar Management Console. Refer to Appendix A selectfortape.pl Script (page 32) for complete command instructions. The following files are required for running selectfortape.pl on the source Avamar server: Perl-DBD-PgPP.rpm selectfortape.pl selectfortape_interactive.pl (only required for interactive mode)
Applications Setup
Setting up selectfortape.pl
The following instructions assume that the source Avamar server has been configured properly according to Avamar software installation guidelines. Using winscp, ftp or other transfer mechanism, place files and applications of importance in some directory. Instructions that follow use the sample directory /tmp. IMPORTANT: All commands must be run as the root user. 1. Install the perl-DBD-PgPP module rpm by entering: rpm -ivh perl-DBD-PgPP-VERSION.rpm Where VERSION is the latest version of the module. 2. Create a symlink from /usr/lib/perl5/site_perl/5.6.1/DBD. For Red Hat 3, enter: cd /usr/lib/perl5/5.8.0/i386-linux-thread-multi ln -s /usr/lib/perl5/site_perl/5.6.1/DBD . For Red Hat 4, enter: cd /usr/lib/perl5/5.8.5/ ln -s /usr/lib/perl5/site_perl/5.6.1/DBD . 3. Move selectfortape.pl to /usr/local/avamar/bin by entering: mv selectfortape.pl /usr/local/avamar/bin/ 4. Make selectfortape.pl executable (change its permissions) by entering: chmod a+rx /usr/local/avamar/bin/selectfortape.pl 5. For selectfortape_interactive.pl, move and change permissions by entering: mv selectfortape_interactive.pl /usr/local/avamar/bin/ chmod a+rx /usr/local/avamar/bin/selectfortape_interactive.pl
Applications Setup
tapeout.pl on Avamar Staging Server

The tapeout.pl application receives output generated by selectfortape.pl. The mode of export to tape you used determines the options required for tapeout.pl. When running the tapeout.pl application, the following arguments are required:
ARGUMENT DESCRIPTION
--stage --stagingdir=[DIR] --tapescript=[SCRIPTPATHNAME]
Instructs tapeout.pl to enter staging mode. The directory to be used for the redirected restores. The path to the tape script that is called to export the backups to tape when the backups have all been restored to the staging server. Prompts you to export backups to tape manually before proceeding with additional restores.
--manual
NOTE: Either the manual or tapescript option is required. Refer to Appendix B tapeout.pl Script (page 36) for complete command instructions.
Setting Up tapeout.pl on Linux Staging Server

The Linux staging server should be set up like an Avamar data or spare node with enough external storage to hold approximately 20% more space than the largest backup on the Avamar server. The additional space must exist on a single partition or mount point. Using winscp, ftp or other transfer mechanism, place the files and applications of importance in some directory. Instructions that follow use the sample directory /tmp. IMPORTANT: All commands must be run as the root user, or the user should have root level privileges. The following files are required for running tapeout.pl on a Linux staging server: AvamarClient-VERSION.rpm tapeout.pl Because tapeout.pl does not perform database queries, no additional Perl modules are required.
Applications Setup The following instructions describe how to ensure that the export to tape process runs successfully: 1. Place tapeout.pl in the Avamar software path. (a) Move the program tapeout.pl to /usr/local/avamar/bin and change its permissions so that it is executable by entering: mv tapeout.pl /usr/local/avamar/bin chmod a+rx /usr/local/avamar/bin/tapeout.pl IMPORTANT: The tapeout.pl program creates files such as logs and staging files in the directory the user runs tapeout.pl. This also applies to when tapeout.pl is run from an automated script. 2. Perform a test avtar command to ensure that authentication is working properly by entering: avtar --archives --path=/MC_BACKUPS If the avtar.cmd file has been configured correctly, the command should return a list of the backups under /MC_BACKUPS. Every Avamar server has a /MC_BACKUPS directory that contains Enterprise Management Console flushes.
Setting Up tapeout.pl on a Windows Avamar Staging Server

In nonincremental mode, the following describes how to configure a Windowsbased Avamar staging server. The server should have 20% more capacity than the largest backup on the Avamar server. The additional space must reside on a single partition or mount point. Using winscp, ftp or other transfer mechanism, place the files and applications of importance in some directory. Instructions that follow use the sample directory /tmp. IMPORTANT: All commands must be run as domain-level administrator. The following files are required for running tapeout.pl on a Windows staging server: ActivePerl5.8 AvamarClient-VERSION.exe tapeout.pl avtar.cmd The following instructions describe how to ensure that the export to tape process runs successfully: 1. Create a working directory and place tapeout.pl in that directory. 2. Place avtar.cmd in c:\Program Files\avs\var.
10
Applications Setup 3. Edit avtar.cmd with the following user-specific information: --server=SERVERNAME Where SERVERNAME is the name or IP address of the Avamar server that contains the backups to be exported. --id=USER Where USER is the user name of an Avamar root-level administrator. --password=PASSWORD Where PASSWORD is the password of the previously-mentioned Avamar user. 4. From a command prompt, perform a test avtar command to ensure that authentication is working properly by entering: c:\Program Files\avs\bin\avtar.exe --archives --path=/MC_BACKUPS If the avtar.cmd file has been configured correctly, the command should return a list of the backups under /MC_BACKUPS. Every Avamar server has a /MC_BACKUPS directory that contains Enterprise Management Console flushes.
Tape Dump Script

Except when manual mode is chosen, tapeout.pl calls an additional script that exports to tape the backups that have been restored to the staging server. The following template scripts are available and can be used or modified to do tape backups: tapedump_networker.pl For NetWorker (7.3.2). tapedump_netbackup.pl For NetBackup (5.0, 5.1 or 6.0). tapedump_backupexec.pl For Backup Exec (11d). tapedump_tsm.pl For TSM. Refer to Additional Instructions for Avamar Tape Out Staging Method for TSM (page 13) for additional information. These templates are very simple, and the degree of modification varies with respect to environmental requirements. They are written in Perl since its interpreter is always resident on the server that tapeout.pl is run on (tapeout.pl is a Perl program). Depending on the backup policy configuration for the tape backup program, information including name or incremental characteristics may be necessary to perform a command line backup.
11
Applications Setup Scripts can be written in any language as long as these requirements are met: No command line parameters allowed The reason for this is tapeout.pl cannot pass through parameters to the tape dump script. As a result, directories must be modified in the tape dump script. Return code 0 for success Return non-0 (1 by default) for failure NOTE: Before backing up, a pre-defined tape backup policy for that particular product must exist for the backups when they go to tape.
Example
The following is the content of the NetBackup template script including comments: #!/usr/bin/perl # $Id: tapedump_netbackup.pl,v 1.1 2007/05/03 16:54:12 ekim Exp $ # This is a simple script that performs a backup to tape on a # specified staging directory for Veritas NetBackup and returns the # exit codes. If the user wishes to have a different staging script, # the requirement is that the return code either be 0 for pass and # greater than 0 for not pass. ## BEGIN USER VARIABLES # location of restored data during staging. passed into tapeout.pl as # --stagingdir # e.g. $stagedir = 'e:\stagedir'; $stagedir = 'STAGEDIR'; # NetBackup server # e.g. $server = 'nbu1.avamar.com'; $server = 'SERVER'; # path of NetBackup bpbackup command # e.g. $savecmd = '"c:\Program Files\VERITAS\NetBackup\bin\bpbackup.exe"'; $savecmd = '"c:\Program Files\VERITAS\NetBackup\bin\bpbackup.exe"'; # Netbackup policy to use (must be pre-defined) # e.g. $policy = "tapeout"; $policy = "POLICY"; ## END USER VARIABLES print "Saving $stagedir to $server...\n"; # -w flag tells bpbackup to wait for error code `$savecmd -w -S $server -p $policy $stagedir`; if ( $? == 0 ){ print "Backup to tape completed successfully.\n"; exit (0); } else { print "Backup to tape failed.\n"; exit (1); }
12
Tape Out Procedures
NOTE: The complexity and functionality of a tape dump script depends on environmental constraints.
Additional Instructions for Avamar Tape Out Staging Method for TSM
The tapedump_tsm.pl script uses the tsm command line client (dsmc) to invoke a TSM client "archive" function. This function performs tape out operations to the TSM server using the Avamar tape out staging method. Several variables embedded in the tapedump_tsm.pl script must be adjusted for any one particular implementation. Use the following general instructions to prepare for TSM tape out: 1. On the TSM Server, perform the following: (a) Register the staging server as a new client node. (b) If necessary, create new management classes and archive copy groups. 2. On the Avamar staging server, do the following: (a) Install the TSM client software. (b) Configure the dsm.opt and dsm.sys files with the following information: nodename passwordaccess generate tcpserveraddress 3. To generate the TSM client password, enter the following from an operating system command prompt: dsmc q sess
Tape Out Procedures

Running selectfortape.pl
When running the selectfortape.pl application, the following considerations should be noted. Besides required arguments, the following arguments are useful: --nah --cod --mod --include --exclude Refer to Appendix A selectfortape.pl Script (page 32) for complete descriptions of command line parameters.
13
Tape Out Procedures
Example 1
selectfortape.pl --start=2007-02-02 --end=2007-02-02 --retention=1 --utility=avamar01 --snapupoutput=/clients/snapupholder This command produces the following results: All scheduled backups performed on 2007-02-02. Backups with no minimum retention. When specifying --retention=1, this means that backups with no end date are counted. The backups to be searched reside on avamar01. The backup list is stored in the client name /clients/snapupholder on the server avamar01.
Example 2
selectfortape.pl --start=2006-12-01 --end=2007-06-04 --retention=30 --utility=avamar01 --snapupoutput=/clients/snapupholder --cod --mod This command produces the following results: All scheduled, client initiated (cod) and Management Console initiated (mod) backups occurring on any date from 2006-12-01 to 2007-06-04 inclusive. Backups are of a minimum retention of 30 days. The backups reside on the Avamar server named avamar01. The backup list is saved in the client named /clients/snapupholder.
Example 3
selectfortape.pl --start=2006-12-01 --end=2007-06-04 --retention=30 --utility=avamar01 --cod --mod --nah --exclude=/REPLICATE --output=snapups.out This command produces the following results: All scheduled, client initiated (cod), Management Console initiated (mod) and naked ad hoc (nah, command line avtar backups) backups except backups in the /REPLICATE domain performed from 2006-12-01 to 2007-06-04 inclusive. Backups are of a minimum retention of 30 days. The backups reside on the Avamar server named avamar01. The backup list is saved to a local file named snapups.out.
14
Tape Out Procedures
Example 4
perl selectfortape.pl --start=2009-01-01 --end=2009-06-30 -retention=30 --utility=avamar01 --snapupoutput=/clients/snapupholder --cod --mod --firstday This command produces the following results: All scheduled, client initiated (cod) and Management Console initiated (mod) backups occurring on any date from 2009-01-01 to 2009-06-30 inclusive. Backups are of a minimum retention of 30 days. The backups reside on the Avamar server named avamar01. The backup list is saved in the client named /clients/snapupholder. Returns the first (earliest) day's worth of backups (for each client) within 2009-01-01 and 2009-06-30.
Example 5
perl selectfortape.pl --start=2009-01-01 --end=2009-06-30 -retention=30 --utility=avamar01 --snapupoutput=/clients/snapupholder --cod --mod --firstbackup --lastbackup This command produces the following results: All scheduled, client initiated (cod) and Management Console initiated (mod) backups occurring on any date from 2009-01-01 to 2009-06-30 inclusive. Backups are of a minimum retention of 30 days. The backups reside on the Avamar server named avamar01. The backup list is saved in the client named /clients/snapupholder. Returns the first backup and last backup for each client within 2009-01-01 and 2009-06-30.
Selectfortape Interactive
The selectfortape.pl script can be run in interactive mode. In this mode, you are prompted each step of the selection process. To run selectfortape.pl in interactive mode, type: root@aeesx3:~/#: selectfortape.pl --interactive NOTE: No other options can be used with --interactive.
Interactive mode presents all prompt options in each dialog box. If you dont use interactive mode, you must refer to Appendix A selectfortape.pl Script (page 32) or use the --help parameter for a list of options. Also, use interactive mode if you feel more comfortable in a graphical user interface than at the command line. Interactive mode displays windows described in the following topics, in the order shown. Use Tab to navigate around the interface and Enter to execute highlighted options. In windows that display a list, use up and down arrows to
15
Tape Out Procedures highlight items and press the space bar to toggle selection (marked by an X) and deselection. Upon completion of the following steps, a command line equivalent to a manual operation is displayed. This information is useful it you want to automate the selectfortape procedure for future tape out operations.
Selection - Utility Node

Enter the hostname of the utility node (or Avamar server, if a single node server). Display default is defined by --server=HOSTNAME in the usersettings.cfg file in the following path: /usr/local/avamar/etc.
Selection - Client List Options Filter

Select whether you initially want all clients or no clients (None) selected in the window that follows. For example, if you have 100 clients but want to export to tape just three of them, select None so that all clients appear deselected and you can then select the desired three clients. Likewise, if you want to export to tape 97 of 100 clients, select All and manually deselect the unwanted clients in the client list window.
16
Tape Out Procedures
Selection - Client List Options

Select the clients you want to export to tape. X indicates the client is selected. To toggle the selection, press the space bar on the highlighted client.
17
Tape Out Procedures
Selection - Start Date

Type the start date of the window for eligible backups. Required format: YYYY-MM-DD. Default is 30 days before current date.
Selection - End Date

Type the end date of the window for eligible backups. Required format: YYYY-MM-DD. Default is current date.
18
Tape Out Procedures
Selection - Retention
Type the minimum retention period (in days) for eligible backups. Default is 0.
Selection - OS Includes
Select which operating systems should be included in the selection process. X indicates the OS is selected. To toggle the selection, press the space bar on the highlighted OS. Default is all selected.
19
Tape Out Procedures
Selection - Additional Commands

The commands in the following window provide more granular selections.
COMMAND
DESCRIPTION
nah cod mod firstday firstbackup lastday lastbackup sun, mon, tue, wed, thu, fri, sat
Includes "naked adhoc" backups (backups done by command line). Includes "client on demand" backups through the client interface. Includes "client on demand" backups through the Avamar Administrator. Includes the entire first day of backups (for each client) within the selection criteria provided. Includes the first backup (for each client) within the selection criteria provided. Includes the entire last day of backups (for each client) within the selection criteria provided. Includes the last backup (for each client) within the selection criteria provided. Includes backups that occurred on the days selected.
NOTE: Scheduled backups are always included
20
Tape Out Procedures
Selection - Running Selection

The following window is displayed as the selectfortape.pl script gathers all backups that meet the selection criteria. To facilitate further investigation, if required, the debug log file location is displayed in the box.
Selection - selectfortape.pl Debug Log

This window contains the debug log file. Use the up and down arrow keys to scroll down and review the log. If all is satisfactory, click OK.
21
Tape Out Procedures
Selection - Query Results

This window displays all of the backups as determined by your selection criteria. You can exclude backups by deselecting them. This window displays the following: Column 1 - determines whether a backup is selected Column 2 - line number Column 3 - client Column 4 - date of backup Column 5 - client operating system Column 6 - backup label number
Selection - Use Default User, Password

In this window, select whether to use default settings for user and password, which are displayed. If you click Accept Default, the following two windows do not appear.
22
Tape Out Procedures
Selection - Upload User ID

Specify the user ID the script will use for the upload of selected backup criteria to the Avamar server. This window does not appear if you accepted default settings in the Use Default User, Password window.
23
Tape Out Procedures
Selection - User Password

Type the password for the user specified in the previous window. This window does not appear if you accepted default settings in the Use Default User, Password window.
Selection - Avamar Path

Specify the path to an existing client on the Avamar server to which the output file (that contains backup selections) will be uploaded. IMPORTANT: You must have already created a "fake" (unassigned) client solely for the purpose of storing output files. For example, create a client named "snapupholder" in the /clients domain. Refer to the Avamar System Administration Manual for instructions on creating clients.
24
Tape Out Procedures
Selection - Upload Status

The following window provides upload status. Initially, refer to the header to determine if the upload succeeded or failed. Use the up and down arrows to review the log file and, if the upload failed, to determine why. If the upload failed, click OK to return to the previous window to reenter a valid path. If the upload succeeded, click OK to exit the interactive script. Following is a successful upload example (header displays "upload succeeded").
Selection - Exit Status

The following window provides exit codes and the manual command line equivalent to the operation the interactive mode performed. This information is useful for automating the selectfortape procedure as described in Crontab Automation of selectfortape.pl Application (page 31).
25
Tape Out Procedures
Running tapeout.pl
When running the tapeout.pl application, the following considerations should be noted. Besides required arguments, the following arguments are strongly recommended: --incremental --onloggedavtarerror=[retry|skip] --onnewavtarerror=[exit|skip|continue] --logdir=[PATH] --stagingfile=[PATHTOFILE] Refer to Appendix B tapeout.pl Script (page 36) for complete descriptions of command line parameters.
Example 1
tapeout.pl --stage --stagingdir=/data01/restore_data --tapescript= /tmp/export_to_tape.pl --snapupoutput=/clients/snapupholder This command produces the following results: Uses the staging mode. The directory /data01/restore_data is the target for the re-directed restores. Uses the script /tmp/export_to_tape.pl to export the backups to tape after tapeout.pl has finished staging data. Uses the list that was generated by selectfortape.pl that is stored on the Avamar server in /clients/snapupholder.
Example 2
tapeout.pl --stage --stagingdir=/data01/restore_data --tapescript= /tmp/export_to_tape.pl --snapupoutput=/clients/snapupholder --onnewavtarerror=skip --onloggedavtarerror=retry This command produces the following results: Uses the staging mode. The directory /data01/restore_data is the target for the redirected restores. Uses the script /tmp/export_to_tape.pl to export the backups to tape after tapeout.pl has finished staging data. Uses the list that was generated by selectfortape.pl that is stored on the Avamar server in /clients/snapupholder. Every time an error occurs with one of the Avamar restores, marks as failed and skipped and then proceeds to delete the data and continue to the next restore. Every time there is an existing failure from a previous run, it tries to re-restore and export to tape.
26
Tape Out Procedures
Example 3
tapeout.pl --stage --stagingdir=/data01/restore_data --manual --snapupoutput=/clients/snapupholder --onnewavtarerror=continue --onloggedavtarerror=skip --stagingfile=/tmp/staging_info --osinclude=windows This command produces the following results: Uses the staging mode. The directory /data01/restore_data is the target for the redirected restores. Instead of having a tape script, the user manually exports to tape the restored data. Uses the list that was generated by selectfortape.pl that is stored on the Avamar server in /clients/snapupholder. Every time an error occurs with one of the Avamar restores, it marks as failed and then proceeds to the next restore without deleting any data. Every time there is an existing failure from a previous run, it tries to re-restore and export to tape. Creates the file /tmp/staging_info instead of the default tapeout_staging.txt to contain all of the information about the status information of the restore and tape out. This file acts as a database for the tapeout.pl. Only restores Microsoft Windows backups to the staging server. Includes and excludes are mutually exclusive and can range from the following choices: unknownfilesystem, linux, solaris, hp-ux, aix, mac, unix, windows or ndmp.
27
Monitoring Tape Out Progress
Monitoring Tape Out Progress

The staging file is important to both the tapeout.pl application and the user for tape out progress monitoring purposes. It is used by the application as a status database which, since it is human-readable, can be used to accurately monitor tape out progress. The staging file is specified by --stagingfile=FILENAME (tapeout_staging.txt, by default). A description of the fields and possible status messages in the file are shown below.
Field Definitions
The fields of the staging file are formatted as: FULL CLIENT PATH WITH LABELNUM | CLIENT NAME | PLUGIN TYPE | LABEL NUMBER | BACKUP SIZE (BYTES) | BACKUP DATE | RESTORE STATUS | RESTORE DATE/TIME | TAPEOUT STATUS | TAPEOUT DATE/TIME FULL CLIENT PATH This field shows the full Avamar client path as shown in the Avamar Administrator. This is also the path that is created in the staging directory. CLIENT NAME This field shows the name of the client.
OS This field shows the operating system of the client. Valid OS types are the following: unknownfilesystem linux solaris hp-ux aix mac unix windows ndmp LABEL NUM This field shows the backup label number as it is recorded in the Avamar server. BACKUP SIZE BACKUP DATE This field shows the size of the backup. This field shows the date when the backup was performed.
28
Monitoring Tape Out Progress AVTAR STATUS This field shows the status of the restore to disk. Valid states are the following:
STATE DESCRIPTION
avtar none avtar in progress avtar complete avtar failed continued
The restore to disk has not run yet. The restore to disk is in progress. The restore to disk is complete. The restore failed but tape out continued. Occurs only when--onnewavtarerror is set to continue. The restore failed and tape out skipped. Occurs only when --onnewavtarerror or --onloggedavtarerror is set to skip. The restore failed and tape out exited. Occurs only when --onnewavtarerror is set to exit. The restore failed and tape out skipped. Occurs only when --onnewavtarerror is set to skip. The restore failed because the backup size was too large for the disk. Occurs only when --onnewavtarerror is set to continue. The restore failed and exited because the backup size was too large for the disk. Occurs only when --onnewavtarerror is set to exit.
avtar failed skipped
avtar failed exit
avtar failed skipped (too large for FS) avtar failed (too large for FS)
avtar failed exit (too large for FS)
When avtar fails, an error code and short summary are received. Error codes include the following: 151:EXIT_AUTH_FAIL 152:EXIT_BAD_FLAGS 153:EXIT_CACHE_OR_MUTEX_LOCKED 154:EXIT_LOCAL_PERMISSIONS 155:EXIT_EXCLUDELIST 156:EXIT_MOUNTTABLE 157:EXIT_MISC 158:EXIT_CONNECT 159:EXIT_INCLUDELIST 160:EXIT_OK_WITH_OPENFILES 161:EXIT_RESTORE_FAIL 162:EXIT_TARGETLIST
29
Monitoring Tape Out Progress 163:EXIT_CANCELED 164:EXIT_CRASHED 165:EXIT_INVALIDTICKET 166:EXIT_KILLED 167:EXIT_SSL_ERROR 168:EXIT_AUTH_NO_MACHINE 169:EXIT_MCSCANCEL 170:EXIT_OK_WITH_ERRORS 171:EXIT_ABORT 172:EXIT_SIGINT 173:EXIT_SIGTERM 174:EXIT_DUMPSTREAM 175:EXIT_UNHANDLEDEXCEPTION 176:EXIT_FATALSIGNAL 177:EXIT_NO_ARCHIVES AVTAR DATE This field shows the time the restore to disk last reported status (format YYYY/MM/DD-HH:MM:SS). TAPEOUT STATUS This field shows the status of the tape operations. Valid states are the following:
STATE DESCRIPTION
tapeout none tapeout in progress tapeout complete tapeout complete (manual) tapeout failed
No tape activity has occurred yet. Tape backup is in progress. Tape backup has completed using the tape script. Tape backup has completed using the --manual method. Tape backup script returned an exit code that was not 0.
TAPEOUT DATE This field shows the time the tape backup last reported status (format YYYY/MM/DD-HH:MM:SS).
30
Crontab Automation of selectfortape.pl Application
Example Staging File Contents

/REPLICATE/server/clients/tester.example.com/0000032/ | tester.example.com | Linux | 32 | 55255040 | avtar complete | 2007/05/08-11:32:55 | tapeout complete | 2007/05/1011:32:55 /REPLICATE/server/clients/tester.example.com/0000033/ | tester.example.com | Linux | 33 | 55113728 | avtar failed continued | 2007/05/09-11:32:55 | tapeout complete | 2007/ 05/10-11:32:55 /REPLICATE/server/clients/tester.example.com/0000034/ | tester.example.com | Linux | 34 | 40440832 | avtar complete | 2007/05/10-11:32:55 | tapeout complete | 2007/05/1011:32:55 /REPLICATE/server/clients/tester.example.com/0000035/ | tester.example.com | Linux | 35 | 40440956 | avtar in progress | 2007/05/12-11:32:55 | tapeout none | 2007/05/0811:32:55 /REPLICATE/trne12/clients/tester2.example.com/0000028/ | tester2.example.com | Linux | 28 | 40097792 | avtar none | 2007/05/08-11:32:56 | tapeout none | 2007/05/08-11:32:56 /REPLICATE/trne12/clients/tester3.example.com/0000027/ | tester3.example.com | Linux | 27 | 39536640 | avtar none | 2007/05/08-11:32:56 | tapeout none | 2007/05/08-11:32:56
Crontab Automation of selectfortape.pl Application

Use the following information to automate running the selectfortape.pl application. Display the current cron schedule for the utility node by entering the following (as user root): crontab -u dpn -l For a utility node, the default cron output looks similar to the following:
# 0 0 # <<< BEGIN AXION ADMINISTRATOR MANAGED ENTRIES -- DO NOT MANUALLY MODIFY >>> 6 * * * /usr/local/avamar/bin/cron_env_wrapper morning_cron_run 18 * * * /usr/local/avamar/bin/cron_env_wrapper evening_cron_run <<< END AXION ADMINISTRATOR MANAGED ENTRIES >>>
Modify the crontab directly with the automated selectfortape.pl line by entering the following (as user root): crontab -u dpn -e
Example
To run the selectfortape.pl at 6 a.m. on the third day of every month, specifying two days prior as the start date and the current day as the end date, add the following to the crontab output: 0 6 3 * * /root/selectfortape.pl --start=`date -d '2 days ago' +\%Y\%m-\%d` --end=`date -d 'today' +\%Y-\%m-\%d` --retention=61 --snapupoutput=/clients/snapupholder --utility=[UTILITY NODE] --exclude=/clients/clientx NOTE: The start and end dates should be modified according to site-specific requirements.
31
Appendix A selectfortape.pl Script The resulting cron would look similar to the following:
0 6 3 * * /root/selectfortape.pl --start=`date -d '2 days ago' +\%Y-\%m-\%d` -end=`date -d 'today' +\%Y-\%m-\%d` --retention=61 --snapupoutput=/clients/snapupholder --utility=[UTILITY NODE] --exclude=/clients/clientx # <<< BEGIN AXION ADMINISTRATOR MANAGED ENTRIES -- DO NOT MANUALLY MODIFY >>> 0 6 * * * /usr/local/avamar/bin/cron_env_wrapper morning_cron_run 0 18 * * * /usr/local/avamar/bin/cron_env_wrapper evening_cron_run # <<< END AXION ADMINISTRATOR MANAGED ENTRIES >>>
Appendix A selectfortape.pl Script

selectfortape.pl is a Perl script that queries the Avamar server, selects backups according to user-defined criteria and restores them to a specific location (usually on the staging server), where they can be processed by tapeout.pl (page 36). Backups can be filtered according the the following criteria: Start date and end date A range of dates for determining backups of interest. Minimum retention A minimum retention setting in days for determining backups of interest; backups with a retention setting of this number of days or higher will be included in the archive to tape operation. Include A list of explicitly included clients and domains; for example, /REPLICATE domain is included in tape archive operations by default. Exclude A list of explicitly excluded clients and domains; for example /MC_BACKUPS is excluded in tape archive operations by default.
Synopsis
selectfortape.pl --start=STARTDATE --end=ENDDATE --retention=MINRETENTION --utility=UTILITYNODE [ --avtar=AVTARPATH ] [--root=ROOTDIR] [--output=OUTFILE] [--append] [--exclude=EXCLUDECLIENTS] [--include=INCLUDECLIENTS] [--cod] [--mod] [--nah] [--quiet] [--debug=DEBUGFILE] [--snapupoutput=SNAPUPPATH] [--osinclude=OS,OS] [--osexclude=OS,OS] [--firstday] [--firstbackup] [--lastday] [--lastbackup] [--dayofweek=DAYOFWEEK(S)] [--interactive=PATH]
32
Appendix A selectfortape.pl Script
Required Arguments
All of the following arguments are required in order for selectfortape.pl to successfully complete. --start=DATE Used with --end=DATE to specify a range of dates for determining backups of interest. Backups starting on or after this date will be included in this restore operation. Valid format is YYYY-MM-DD, where YYYY is a four-digit year, MM is a two-digit month and DD is a two-digit day of the month. --end=DATE Used with --start=DATE to specify a range of dates for determining backups of interest. Backups ending on or before this date will be included in this restore operation. Valid format is YYYY-MM-DD, where YYYY is a four-digit year, MM is a two-digit month and DD is a two-digit day of the month. --retention=MINRETENTION Specifies a minimum retention setting in DAYS for determining backups of interest. Backups with a retention setting of this number of DAYS or higher will be included in this restore operation. Host name or IP address of the Avamar server utility node that contains the backups to be exported to tape.
--utility=UTILITYNODE
Options
--append --avtar=AVTARPATH --cod Appends backup list to --output=FILE. Default setting overwrites OUTFILE. Specifies full PATH to the avtar command. Default setting is /usr/local/avamar/bin. Includes client on-demand backups found within the specified date range are included in this restore operation. If not supplied, default behavior is to exclude client on-demand backups from restore operations. --dayofweek=DAYOFWEEK(S) Can specify three-letter day(s) of week that are to be returned within the given start/end dates. Valid days: sun, mon, tue, wed, thu, fri, sat. Enables writing of extended debugging messages to the speficified FILE. FILE is always overwritten.
--debug=FILE
33
Appendix A selectfortape.pl Script --exclude=EXCLUDECLIENTS Excludes these domains and clients (EXCLUDECLIENT list) from this restore operation. This option accepts both valid client names or domain names. Use a new --exclude argument for each exclusion you want to add. Default exlusions, which need not be explicitly added, are: /MC_BACKUPS, /mc_backups, /MC_DELETED, /MC_RETIRED and /REPLICATE. --firstbackup --firstday Returns the first (earliest) backup per client returned by the selection criteria. Returns the first (earliest) day's worth of backups per client returned by the selection criteria. Shows help, then exits. Adds back specific clients and domains after --exclude=CLIENT-LIST is processed. Use a new --include argument for each exclusion you want to add. --interactive=PATH Calls selectfortape.pl in interactive mode (GUI). Script automatically searches for selectfortape_interactive.pl in /usr/local/avamar/bin. This option must not be used with any other command line option. You can specify an alternate path to selectfortape_interactive.pl with the =PATH suffix. --lastbackup --lastday Returns the last (most recent) backup per client returned by the selection criteria. Returns the last (most recent) day's worth of backups per client returned by the selection criteria. Immediately restores backups listed in --snapupoutput=LOCATION to local hard disk drive. Includes administrator-initiated on-demand backups in this restore operation. If not supplied, default behavior is to exclude administrator-initiated on-demand backups from restore operations.
--help | --usage --include=INCLUDECLIENTS
--local
--mod
34
Appendix A selectfortape.pl Script --nah Includes "naked ah-hoc" backups in this restore operation. If not supplied, default behavior is to exclude naked ah-hoc backups from restore operations. --output=OUTFILE Specifies full path of the an output file, which lists all backups that will be restored in this operation. Default name is snapups2tape.txt. --quiet --root=ROOTDIR Runs quietly (suppresses screen messages). Specifies top-level (root) directory (DIR) where backups will be staged for eventual archive to tape. Default is Dackupdir. --snapupoutput=SNAPUPPATH Upon completion, backs up the --output=OUTFILE to this domain /client SNAPUPPATH on the Avamar server.
Notes
Exit Codes:
0 1 Success Failure
35
Appendix B tapeout.pl Script

tapeout.pl is a Perl script that receives output generated by selectfortape.pl (page 32). The method of tape out used determines required arguments for tapeout.pl.
Synopsis
tapeout.pl {--manual | --tapescript=SCRIPT} --stage --stagingdir=DIR
[ --avtar=AVTARPATH ] [--root=ROOTDIR] [--quiet] [--log=LOGFILE] [--snapupoutput=SNAPUPPATH] [--output=FILENAME] [--id=ID] [--password=PASSWORD] [--clientdump] [--stage] [--stagingdir=DIR] [--tapescript=PATHTOSCRIPT] [--stagingfile=PATHTOFILE] [--logdir=DIR] [--osinclude=OS,OS] [--osexclude=OS,OS] [--manual] [--incremental] [--onloggedavtarerror=retry|skip] [--onnewavtarerror=exit|skip|continue] [--avtarflags="--flag1,--flag2"] [--help | --usage] Use --usage or --help options for extended usage information.
Mode Control Arguments

tapeout.pl can operate in two modes and the arguments controlling the operational mode are mutually exclusive: --manual If supplied, tapeout.pl operates in manual mode. Following succesful staging of the backups, user is prompted to manually archive backups to tape using the tape backup software installed on the staging server. If supplied, tapeout.pl automatically invokes this tape dump script, which exports the tape backups that have been restored to the staging server. SCRIPT value is path and script file name. Refer to 1 Failure (page 38) for additional information.
--tapescript=SCRIPT
Other Required Arguments

All of the following arguments are required in order for tapeout.pl to successfully complete. --stage This argument must be supplied. Otherwise, tapeout.pl will assume that an obsolete method for exporting backups has been used. Specifies target directory (DIR) that is the target for the redirected restores during the staging process
--stagingdir=DIR
36
Options
--avtar=AVTARPATH Specifies full PATH to the avtar command. Default setting is the default OS-specific location of the avtar binary on the staging server. When performing the avtar restores, if additional flags are to be passed in through the commandline and an avtar.cmd is not enough, then the user can specify additional avtar flags in this flag. Be sure to encapsulate all of the additional avtar commands in quotes. Lists out the clients only from the snapupoutput to STDOUT. Can be used in conjunction with the --output or --snapupoutput option depending on the file to be used. Shows help, then exits. Authenticate as this Avamar user ID (account name). Default user ID is root. Incremental mode is a restore in which the destination is incrementally restored, thus reducing restoration times and allowing for more frequent staging. When restoring, an "INCREMENTAL" directory is created in the stagingdir. Enables writing of logging messages to the speficified FILE. Default FILE name is tapeout.log. Specifies directory (DIR) containing all of the avtar restore logs as well as tape logs. The default directory is tapeout_logs. When encountering a previously failed backup from the stagingfile, the behaviour can be controlled. retry means to retry restoring it, while skip means do not do anything with it and leave it alone. skip is the default When encoutering an error on an avtar, the behaviour can be controlled. exit means to exit on a failed restore. skip means to delete all data that was restored and mark as failed and continue on. continue means to mark as failed but do not delete the data and continue on. exit is the default.
--avtarflags="--FLAG1, --FLAG2"
--clientdump
--help | --usage --id=USER --incremental [default: disabled]
--log=LOGFILE
--logdir=DIR
--onloggedavtarerror= {retry | skip}
--onnewavtarerror= {exit | skip | continue}
37
Appendix B tapeout.pl Script --osexclude=OS,OS When specifying an exclude(s), any OS that was not excluded is automatically be included. The valid OS types are: unknownfilesystem, linux, solaris, windows, hp-ux, aix, mac, unix, ndmp. When specifying an include(s), any OS that was not included will automatically be excluded. The valid OS types are: unknownfilesystem, linux, solaris, windows, hp-ux, aix, mac, unix, ndmp. This must be the same output file path that was specified by selectfortape.pl (page 32). Specifies PASSWORD for the --id=USER account. Specifies the POLICY base name to be used. Runs quietly (suppresses screen messages). TBD - Eugene, Gary ??? This must be the same domain/client LOCATION on the Avamar server that was specified by selectfortape.pl (page 32).
--osinclude=OS,OS
--output=FILE --password=PASSWORD --policy=POLICY --quiet --root=ROOTDIR --snapupoutput=LOCATION
Notes
Exit Codes:
0 1 Success Failure
38
Copyright 2002 - 2011 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date regulatory document for your product line, go to the Technical Documentation and Advisories section on EMC Powerlink. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
39

Avamar Staging Tape Out Outputting Avamar Backups To Tape Technical Note PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Avamar Staging Tape Out Outputting Avamar Backups To Tape Technical Note PDF

Uploaded by

Copyright:

Available Formats

AVAMAR STAGING TAPE OUT: OUTPUTTING AVAMAR BACKUPS TO TAPE

TECHNICAL NOTES AVAMAR 4.X AND LATER SOFTWARE

Scripted, Manual Modes

Designated tape dump script

Nonincremental, Incremental Modes

Nonincremental mode Incremental mode

Source Avamar Server

selectfortape.pl on Source Avamar Server

Start date End date Minimum retention Include Exclude

tapeout.pl on Avamar Staging Server

--stage --stagingdir=[DIR] --tapescript=[SCRIPTPATHNAME]

Setting Up tapeout.pl on Linux Staging Server

Setting Up tapeout.pl on a Windows Avamar Staging Server

Tape Dump Script

Tape Out Procedures

Tape Out Procedures

Tape Out Procedures

Tape Out Procedures

Selection - Utility Node

Selection - Client List Options Filter

Tape Out Procedures

Selection - Client List Options

Tape Out Procedures

Selection - Start Date

Selection - End Date

Tape Out Procedures

Tape Out Procedures

Selection - Additional Commands

NOTE: Scheduled backups are always included

Tape Out Procedures

Selection - Running Selection

Selection - selectfortape.pl Debug Log

Tape Out Procedures

Selection - Query Results

Selection - Use Default User, Password

Tape Out Procedures

Selection - Upload User ID

Tape Out Procedures

Selection - User Password

Selection - Avamar Path

Tape Out Procedures

Selection - Upload Status

Selection - Exit Status

Tape Out Procedures

Tape Out Procedures

Monitoring Tape Out Progress

Monitoring Tape Out Progress

avtar none avtar in progress avtar complete avtar failed continued

avtar failed skipped

avtar failed exit

avtar failed exit (too large for FS)

Crontab Automation of selectfortape.pl Application

Example Staging File Contents

Crontab Automation of selectfortape.pl Application

Appendix A selectfortape.pl Script

Appendix A selectfortape.pl Script

--help | --usage --include=INCLUDECLIENTS

Appendix B tapeout.pl Script

Appendix B tapeout.pl Script

Mode Control Arguments

Other Required Arguments

Appendix B tapeout.pl Script

--help | --usage --id=USER --incremental [default: disabled]