Sfrac5 Install

Storage Foundation for Oracle RAC
Symantec
Document Title Subject Version Date Author Filename Comments
Storage Foundation for RAC
2709.1 2008/8/22 Symantec Sfrac5-cpic-install-report_20080823.doc
Id
Title
Organisation
Author
Version
Page 2 of 134
1 2 3 4 ......................................................................................2 .............................................................................................3 .......................................................................4 .......................................................................5

4.1 .................................................................................................................................................5 4.2 .................................................................................................................................................8 4.3 .......................................................................................................................................12
5 Veritas SFRAC ............................................................. 14

5.1 ...............................................................................................................................................14 5.2 ...............................................................................................................................................23
6 Oracle RAC ........................................................... 46

6.1 Oracle 10gr2 RAC .................................................................................................................46
7 Oracle RAC ................................................................. 81

7.1 ........................................................................................81 7.2 dbca ...................................................................................................................................81
8 Veritas Cluster ............................................................ 94

8.1 CLUSTER .........................................................................................................................94 8.2 ORACLE CRS ..........................................................103 8.2 Oracle ................................................................................................104 8.3 Oracle Agent .........................................................................................105 8.4 ..........................................................................................................................105
9 Veritas Cluster .............................................................106

9.1 ....................................................................................................................106 9.2 Cluster .................................................................................................................106
10 Cluster Oracle RAC ...................................... 125

10.1 Oracle .....................................................................................................................125 10.2 Oracle CRS push .......................................................................................125
Page 3 of 134
AIX5.3 Veritas Storage Foundation for RAC 5.0 Mp1 Oracle 10gR2 RAC 10.2.0.2 Veritas HA Oracle
Page 4 of 134
4.1
P595 5cpu 20G
4.1.1
1salesdb1 Aix 5300-05 csp 2salesdb2 P595 5cpu 20G
Aix 5300-05 csp
4.1.2
Symantec 1. 2. 3. 4. 5. 6. IP IP( ether channel ) 2 VLAN VALN VLAN Cluster VALN Cluster VALN Cluster IP IP
1salesdb1 10.203.100.80/255.255.255.0 10.203.38.5/255.255.255.0 en9(en0, en4 ) en6(en1, en5 )
en2 en8 SFRAC 10.203.100.82 ORACLE RAC IP 2salesdb2 10.203.100.81/255.255.255.0 10.203.38.7/255.255.255.0 en9(en0, en4 ) en6(en1, en5 )
en2 en8 SFRAC 10.203.100.83 ORACLE RAC IP /etc/hosts 127.0.0.1 loopback localhost # loopback (lo0) name/address 10.203.100.80 salesdb1 10.203.100.81 salesdb2 10.203.100.82 salesdb1-vip 10.203.100.83 salesdb2-vip Page 5 of 134
10.203.38.5 10.203.38.7
salesdb1-priv salesdb2-priv
VLAN 4 ,2 oracle crs 2 symantec rac
4.1.3
symantec I/O fencing 3 200MB oracle crs vote 4 1GB vote disk
: HP 24000/12000 AIX Hostmode:0F(AIX) System option:02(veritas SFRAC) AIX HP ODM HP Hostmode:08(HP-UX) System option:02(veritas SFRAC) EMC 3-80: AIX FLARE 19 Initiator type: CLARiiON open Failover mode:3 AIX EMC ODM HP FLARE 19 Initiator type: HP NO AUTOTresspass Failover mode:2 HBA dyntrk fc_err_recov salesdb1 HBA salesdb1# lsattr -El fscsi0 attach dyntrk switch no How this adapter is CONNECTED Dynamic Tracking of FC Devices False True
Page 6 of 134
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True scsi_id 0x157a00 Adapter SCSI ID FC Class for Fabric False True sw_fc_class 3
salesdb1# lsattr -El fscsi1 attach dyntrk scsi_id switch no 0x157a00 How this adapter is CONNECTED Dynamic Tracking of FC Devices Adapter SCSI ID FC Class for Fabric False True False True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True sw_fc_class 3
HBA salesdb1# chdev -l fscsi0 -a fc_err_recov=fast_fail -P fscsi0 changed salesdb1# chdev -l fscsi1 -a fc_err_recov=fast_fail -P fscsi1 changed salesdb1# chdev -l fscsi0 -a dyntrk=yes -P fscsi0 changed salesdb1# chdev -l fscsi1 -a dyntrk=yes -P fscsi1 changed salesdb1# lsattr -El fscsi0 attach dyntrk scsi_id switch yes How this adapter is CONNECTED Dynamic Tracking of FC Devices False True False True False True False True
fc_err_recov fast_fail FC Fabric Event Error RECOVERY Policy True 0x157a00 Adapter SCSI ID FC Class for Fabric sw_fc_class 3 attach dyntrk scsi_id switch yes
salesdb1# lsattr -El fscsi1 How this adapter is CONNECTED Dynamic Tracking of FC Devices
fc_err_recov fast_fail FC Fabric Event Error RECOVERY Policy True 0x157a00 Adapter SCSI ID FC Class for Fabric sw_fc_class 3
4.1.4
oracle oinstall dba
Page 7 of 134
4.2
4.2.1
lslpp -l|grep bos.adt.libm lslpp -l|grep perfagent.tools lslpp -l|grep bos.perf lslpp -l|grep Java14 lslpp -l|grep rsct.core lslpp -l|grep rsct.compat.basic lslpp -l|grep rsct.compat.clients lslpp -l|grep rsct.basic lslpp -l|grep rsct.clients lslpp -l|grep xlC : SF Oracle xlC 8.0 * xlC.aix50.rte: * xlC.rte: # lslpp -l DLManager* ODM # lsdev -Cc disk hdisk2 Available 03-08-02 XP Disk Array (Fibre) ODM SF Oracle RAC / 1G /opt2G /usr5 G /tmp5 G /var5 G swap # lsps -a swap 75% XP Disk Array (Fibre)
Page 8 of 134
4.2.2 PATH
oracle /etc/hosts.equiv salesdb1 oracle salesdb2 oracle salesdb1-priv oracle salesdb2-priv oracle root /.rhosts salesdb1 root salesdb2 root salesdb1-priv root salesdb2-priv root root oracle rsh <peer hostname> date, rlogin <peer hostname, rlogin 1. ip rlogin /.rhosts ip 2.Oracle
root PATH .profile # vi .profile PS1=`hostname`@'$PWD'"# " set -o vi PATH=/usr/sbin:/sbin:/usr/bin:/usr/lib/vxvm/bin:/opt/VRTSvxfs/sbin:/opt/VRTSvcs/bin:/opt/VRTS/ bin:/opt/VRTSvcs/rac/bin:/opt/VRTSob/bin:$PATH ORACLE_BASE=/u01/oracle ORACLE_HOME=$ORACLE_BASE/product/db10gr2 ORA_CRS_HOME=$ORACLE_BASE/product/crs10gr2 PATH=$ORA_CRS_HOME/bin:$ORACLE_HOME/OPatch:$PATH
4.2.3 ORACLE
oracle .profile ORACLE_SID 2 chnldb2
Oracle RAC DB 10g Environment
port ORACLE_BASE=/u01/oracle
port ORACLE_HOME=$ORACLE_BASE/product/db10gr2
port ORA_CRS_HOME=$ORACLE_BASE/product/crs10gr2
port ORACLE_SID=chnldb1
Page 9 of 134
port ORACLE_TERM=xterm
port NLS_LANG='AMERICAN_AMERICA.ZHS16GBK'
export NLS_LANG='SIMPLIFIED CHINESE_CHINA.ZHS16GBK'
port ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
port LIBPATH=$ORACLE_HOME/lib:$ORACLE_HOME/ctx/lib
port LD_LIBRARY_PATH=$ORA_CRS_HOME/lib:$ORACLE_HOME/lib32:$ORACLE_HOME/lib: RACLE_HOME/ctx/lib:$ORACLE_HOME/RDBMS/lib:/lib:/usr/lib
port ORACLE_DOC=$ORACLE_HOME/doc
port PATH=$ORA_CRS_HOME/bin:$ORACLE_HOME/OPatch:$ORACLE_HOME/bin:$ORACLE_HOME/jre/1.4.2/bin: ATH
port CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib: RACLE_HOME/network/jlib
port TMP=/tmp
port TMPDIR=$TMP
port NLS_DATE_FORMAT='yyyy-mm-dd hh24:mi:ss'
port NLS_TIMESTAMP_TZ_FORMAT='yyyy-mm-dd hh24:mi:ss.ff'
port EDITOR=vi
port AIXTHREAD_SCOPE=S
acle BASE /u01/oracle
acle CRS /u01/oracle/product/crs10gr2
acle Database /u01/oracle/product/db10gr2
acle chnldb1 1 chnldb1 2 chnldb2
4.2.4
1. smitty # smitty tuning => tuning network option parameter sb_max udp_recvspace rfc1323 Ipqmaxlen udp_sendspace tcp_sendspace tcp_recvspace # no -p -o rfc1323=1 # no -p -o tcp_sendspace=262144 # no -p -o tcp_recvspace=655360 # no -p -o udp_sendspace=65536 1310720 262144 1 512 65536 262144 655360
Page 10 of 134
# no -p -o udp_recvspace=262144 # no -p -o sb_max=1310720 # no -r -o ipqmaxlen=512 2. smitty # smitty tuning => tuning virtual Memory management parameter MINPERM% MAXPERM% MAXCLIENT% Lru_file_repage Maxpin% strict_maxclient strict_maxperm Minfree Maxfree # vmo -p -o minperm%=5 # vmo -p -o maxclient%=15 # vmo -p -o maxperm%=15 # vmo -p -o strict_maxperm=0 # vmo -p -o strict_maxclient=1 # vmo -L lru_file_repage # vmo -p -o lru_file_repage=0 # vmo -p -o maxfree=4000 # vmo -p -o minfree=3000 # vmo -p -o maxpin%=80 shutdown Fr Maximum number of PROCESSES allowed for each user 3. limits # vi /etc/security/limits default: rss = -1 nofiles = -1 root: fsize = -1 data = -1 stack = -1 core = -1 fsize_hard = -1 Default 512 >= 2048 Default 5% 15% 15% 0 80 1 0 3000 4000
Page 11 of 134
cpu_hard = -1 data_hard = -1 stack_hard = -1 core_hard = -1 oracle: fsize = -1 data = -1 stack = -1 core = -1 fsize_hard = -1 cpu_hard = -1 data_hard = -1 stack_hard = -1 core_hard = -1 4. AIO # smitty aio
Change / Show Characteristics of Asynchronous I/O MINIMUM number of servers MAXIMUM number of servers per cpu [150] [300]
4.3
4.3.1
1. Veritas Storage Foundation 5 for aix software 2. Veritas Storage Foundation 5 for aix MP1 software 3. Oracle 10g release 2 for Aix 5L Cluster software 4. Oracle 10g release 2 for Aix 5L Database software 5. Oracle 10g release 2 patchset 10.2.0.2 metalink.oracle.com 6. Oracle small patch 4637591 4690794 metalink.oracle.com /software/Symantec/SF5/sf5-aix /software/Symantec/SF5/sf5-aix-mp1 /software/db/cluster /software/db/database /software/db/patch10202
Page 12 of 134
/software/Symantec/oracle_patch
Page 13 of 134
Veritas SFRAC
5.1
5.1.1
Salesdb1
# chdev -P -l ent2 -a media_speed=1000_Full_Duplex # chdev -P -l ent8 -a media_speed=1000_Full_Duplex
Salesdb2
# chdev -P -l ent2 -a media_speed=1000_Full_Duplex # chdev -P -l ent8 -a media_speed=1000_Full_Duplex
5.1.2 Storage Foundation for RAC 5.0 for aix

salesdb1@/software/symantec/SF5/sf5-aix/sfrac# ./installsfrac -rsh -installonly [Page 1] Veritas Storage Foundation for Oracle RAC 5.0 Installation Program Copyright (c) 2006 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The Licensed Software and Documentation are deemed to be "commercial computer software" and "commercial computer software documentation" as defined in FAR Sections 12.212 and DFARS Section 227.7202. Logs for installsfrac are being created in /var/tmp/installsfrac-LXoxOx. Enter the system names separated by spaces on which to install SFRAC:salesdb1 salesdb2 [Page 2] Initial system check: Checking rsh communication with salesdb2 .................................................................. AIX 5.3 Checking SFRAC installation on salesdb1 ............................................................. not installed Checking SFRAC installation on salesdb2 ............................................................. not installed [Page 3] Veritas Storage Foundation for Oracle RAC 5.0 Installation Program
Page 14 of 134
Checking system licensing Installing licensing filesets SFRAC is not licensed on salesdb1 Enter a SFRAC license key for salesdb1: XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX XXXXXXXXXXXXXXXX successfully registered on salesdb1 Demo SFRAC license registered on salesdb1 Do you want to enter another license key for salesdb1? [y,n,q] (n) XXXXXXXXXXXXXXXX successfully registered on salesdb2 Demo SFRAC license registered on salesdb2 Do you want to enter another license key for salesdb2? [y,n,q] (n) [Page 4] Veritas Storage Foundation for Oracle RAC 5.0 Installation Program Checking installed filesets on salesdb1 Checking installed filesets on salesdb2
[Page 5] Veritas Storage Foundation for Oracle RAC 5.0 Installation Program The following SFRAC filesets will be installed: VRTSperl.rte VRTSvlic VRTSicsco VRTSpbx VRTSsmf VRTSat VRTSobc33 VRTSob VRTSobgui VRTSccg VRTSmh VRTSaa Veritas Perl 5.8.8 Redistribution Veritas Licensing Symantec Common Infrastructure Symantec Private Branch Exchange Symantec Service Management Framework Symantec Product Authentication Service Veritas Enterprise Administrator Core Service Veritas Enterprise Administrator Service Veritas Enterprise Administrator Veritas Enterprise Administrator Central Control Grid Veritas Storage Foundation Managed Host by Symantec Veritas Enterprise Administrator Action Agent Page 15 of 134
VRTSspt SYMClma VRTSveki VRTSllt.rte VRTSgab.rte VRTSvxfen.rte VRTSvcs.rte
Veritas Software Support Tools Symantec License Inventory Agent Veritas Kernel Interface Veritas Low Latency Transport Veritas Group Membership and Atomic Broadcast Veritas I/O Fencing Veritas Cluster Server
Press [Return] to continue: ...continued: VRTSvcsag.rte VRTSvcs.man VRTSvcs.doc VRTSjre15.rte VRTScutil.rte VRTScssim.rte VRTScscw.rte VRTSweb.rte VRTScscm.rte VRTScmcs.rte VRTScmccc.rte VRTSvxvm VRTSdsa VRTSfspro VRTSvmman VRTSvmdoc VRTSdcli VRTSvmpro Veritas Cluster Server Bundled Agents Veritas Cluster Server Man Pages Veritas Cluster Server Documentation Veritas Java Runtime Environment Redistribution Veritas Cluster Utilities Veritas Cluster Server Simulator Veritas Cluster Server Configuration Wizards Veritas Java Web Server Veritas Cluster Server Cluster Manager Veritas Cluster Management Console for single cluster environments Veritas Cluster Management Console cluster connector Veritas Volume Manager Binaries Veritas Datacenter Storage Agent Veritas File System Management Services Provider Veritas Volume Manager Manual Pages Veritas Volume Manager Documentation Veritas Distributed Command Line Interface Veritas Volume Manager Management Services Provider VRTSvcs.msg.en_US Veritas Cluster Server Message Catalogs
VRTSacclib.rte Veritas ACC Library
Press [Return] to continue: ...continued: VRTSvsvc VRTSalloc VRTSvdid.rte VRTSddlpr VRTSvrpro VRTSvcsvr VRTSvrw Veritas Volume Server and Client Provider Veritas Volume Manager Intelligent Storage Provisioning Veritas Device Identification API Veritas Device Discovery Layer Services Provider Veritas Volume Replicator Client Extension and Provider for Veritas Cluster Server Agents for VVR Veritas Volume Replicator Web Console Page 16 of 134
Veritas Enterprise Administrator
VRTSvrdoc VRTSvxfs VRTSfsman VRTSfsdoc VRTSfssdk VRTSfsmnd VRTScavf VRTSglm VRTScfsdc VRTSdbms3 VRTSdbac.rte VRTSvcsor.rte
Veritas Volume Replicator Documentation Veritas File System Veritas File System Manual Pages Veritas File System Documentation Veritas File System Software Developer Kit Veritas File System Software Developer Kit Manual Pages Veritas Cluster Server Agents for Cluster File System Veritas Group Lock Manager Veritas Cluster File System Documentation Symantec Shared DBMS Veritas Oracle Real Application Cluster Support Package Veritas High Availability Agent for Oracle
VRTSvail.VRTSvail Veritas Array Integration Layer
Press [Return] to continue: ...continued: VRTScsocw.rte VRTSdbcom VRTSdbed VRTSgms VRTSodm VRTSvxmsa Veritas Cluster Server Configuration Wizard for Oracle and RAC Veritas Storage Foundation Common Utilities for Databases Veritas Storage Foundation for Oracle Veritas Group Messaging Services Veritas Oracle Disk Manager Veritas Mapping Service, Application Libraries
VRTSgapms.VRTSgapms Veritas Generic Array Plugin
Press [Return] to continue: Before Veritas Storage Foundation for Oracle RAC is installed, be sure you have reviewed the Veritas Storage Foundation 5.0 for Oracle RAC Installation and Configuration Guide [Page 6] Veritas Storage Foundation for Oracle RAC 5.0 Installation Program Installing SFRAC: 100% Installation completed successfully on all systems Installation log files, summary file, and response file are saved at: /opt/VRTS/install/logs/installsfrac-LXoxOx See the Veritas Storage Foundation for Oracle RAC Installation and Configuration Guide for more information.
Page 17 of 134
When installsfrac installs software, some software may be applied rather than committed. It is the responsibility of the system administrator to commit the software, which can be performed later with the -c option of the installp command. shutdown Fr
5.1.3 Storage Foundation 5.0MP1 for AIX

salesdb1@/software/symantec/sf5mp1/sf_v5.0_for_aix_sp1# cd sf_v5.0_for_aix_sp1 salesdb1@/software/symantec/sf5mp1/sf_v5.0_for_aix_sp1# ./installmp -rsh salesdb1 salesdb2 [Page 1] Veritas Maintenance Pack 5.0MP1 Installation Program Copyright (c) 2007 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The Licensed Software and Documentation are deemed to be "commercial computer software" and "commercial computer software documentation" as defined in FAR Sections 12.212 and DFARS Section 227.7202. Logs for installmp are being created in /var/tmp/installmp-Vh722D. Initial system check: Checking rsh communication with salesdb2 .................................................................. AIX 5.3 [Page 2] Veritas Maintenance Pack 5.0MP1 Installation Program Checking installed filesets on salesdb1 Checking installed filesets on salesdb2 The following patches will be installed on salesdb1: VRTSsmf VRTSat VRTSobc33 VRTSob VRTSobgui VRTSccg VRTSmh VRTSaa Symantec Service Management Framework Symantec Product Authentication Service Veritas Enterprise Administrator Core Service Veritas Enterprise Administrator Service Veritas Enterprise Administrator Veritas Enterprise Administrator Central Control Grid Veritas Storage Foundation Managed Host by Symantec Veritas Enterprise Administrator Action Agent Page 18 of 134
SYMClma VRTSdbms3 VRTSllt.rte VRTSgab.rte VRTSvxfen.rte VRTSvcs.rte VRTSvcsag.rte VRTScssim.rte VRTScscm.rte VRTScmcs.rte VRTScmccc.rte
Symantec License Inventory Agent Symantec Shared DBMS Veritas Low Latency Transport Veritas Group Membership and Atomic Broadcast Veritas I/O Fencing Veritas Cluster Server Veritas Cluster Server Bundled Agents Veritas Cluster Server Simulator Veritas Cluster Server Cluster Manager Veritas Cluster Management Console for single cluster environments Veritas Cluster Management Console cluster connector
Press [Return] to continue: ...continued: VRTSfspro VRTSvxfs VRTSfsman VRTSvxvm VRTSvmman VRTSdcli VRTSvmpro VRTSvsvc VRTSalloc VRTSddlpr VRTSvrpro VRTSvrw VRTSdbcom VRTScavf VRTSglm VRTSdbac.rte VRTSvcsor.rte VRTScsocw.rte VRTSdbed VRTSodm Veritas File System Management Services Provider Veritas File System Veritas File System Manual Pages Veritas Volume Manager Binaries Veritas Volume Manager Manual Pages Veritas Distributed Command Line Interface Veritas Volume Manager Management Services Provider Veritas Volume Server and Client Provider Veritas Volume Manager Intelligent Storage Provisioning Veritas Device Discovery Layer Services Provider Veritas Volume Replicator Client Extension and Provider for Veritas Volume Replicator Web Console Veritas Storage Foundation Common Utilities for Databases Veritas Cluster Server Agents for Cluster File System Veritas Group Lock Manager Veritas Oracle Real Application Cluster Support Package Veritas High Availability Agent for Oracle Veritas Cluster Server Configuration Wizard for Oracle and RAC Veritas Storage Foundation for Oracle Veritas Oracle Disk Manager
The following patches will be installed on salesdb2: Press [Return] to continue: ...continued: VRTSsmf Symantec Service Management Framework Page 19 of 134
VRTSat VRTSobc33 VRTSob VRTSobgui VRTSccg VRTSmh VRTSaa SYMClma VRTSdbms3 VRTSllt.rte VRTSgab.rte
Symantec Product Authentication Service Veritas Enterprise Administrator Core Service Veritas Enterprise Administrator Service Veritas Enterprise Administrator Veritas Enterprise Administrator Central Control Grid Veritas Storage Foundation Managed Host by Symantec Veritas Enterprise Administrator Action Agent Symantec License Inventory Agent Symantec Shared DBMS Veritas Low Latency Transport Veritas Group Membership and Atomic Broadcast Veritas I/O Fencing Veritas Cluster Server Veritas Cluster Server Bundled Agents Veritas Cluster Server Simulator Veritas Cluster Server Cluster Manager Veritas Cluster Management Console for single cluster environments Veritas Cluster Management Console cluster connector Veritas File System Management Services Provider
VRTSvxfen.rte VRTSvcs.rte VRTSvcsag.rte VRTScssim.rte VRTScscm.rte VRTScmcs.rte VRTScmccc.rte VRTSfspro
Press [Return] to continue: ...continued: VRTSvxfs VRTSfsman VRTSvxvm VRTSvmman VRTSdcli VRTSvmpro VRTSvsvc VRTSalloc VRTSddlpr VRTSvrpro VRTSvrw VRTSdbcom VRTScavf VRTSglm VRTSdbac.rte VRTSvcsor.rte VRTScsocw.rte VRTSdbed Veritas File System Veritas File System Manual Pages Veritas Volume Manager Binaries Veritas Volume Manager Manual Pages Veritas Distributed Command Line Interface Veritas Volume Manager Management Services Provider Veritas Volume Server and Client Provider Veritas Volume Manager Intelligent Storage Provisioning Veritas Device Discovery Layer Services Provider Veritas Volume Replicator Client Extension and Provider for Veritas Volume Replicator Web Console Veritas Storage Foundation Common Utilities for Databases Veritas Cluster Server Agents for Cluster File System Veritas Group Lock Manager Veritas Oracle Real Application Cluster Support Package Veritas High Availability Agent for Oracle Veritas Cluster Server Configuration Wizard for Oracle and RAC Veritas Storage Foundation for Oracle Page 20 of 134
VRTSodm
Veritas Oracle Disk Manager
Press [Return] to continue: [Page 3] Installmp is now ready to install MP1. Required 5.0 processes that are currently running will be stopped. [Page 4] Are you sure you want to install MP1? [y,n,q] (y) [Page 5] Veritas Maintenance Pack 5.0MP1 Installation Program Stopping processes: 100% Shutdown completed successfully on all systems [Page 6] Veritas Maintenance Pack 5.0MP1 Installation Program Installing MP1: 100% Installation completed successfully on all systems Installation log files, summary file, and response file are saved at: /opt/VRTS/install/logs/installmp-Vh722D See the Veritas Storage Foundation for Oracle RAC Installation and Configuration Guide for more information. When installsfrac installs software, some software may be applied rather than committed. It is the responsibility of the system administrator to commit the software, which can be performed later with the -c option of the installp command. Please run shutdown r to reboot machine. shutdown Fr
5.1.4 VRTSvsvc package

salesdb1 Salesdb1@/opt/VRTS/install# lslpp -l VRTSvsvc Salesdb1@/opt/VRTS/install# installp -u VRTSvsvc
Page 21 of 134
5.1.5 Storage Foudation

vxsvc # ps -ef|grep vxsvc # /opt/VRTSob/bin/vxsvcctrl stop AIX_Storage_Foundation_5.0MP1RP3-FS.tar # tar xvf AIX_Storage_Foundation_5.0MP1RP3-FS.tar # cd patches # smitty install AIX_Storage_Foundation_5.0MP1RP3-VM.tar # tar xvf AIX_Storage_Foundation_5.0MP1RP3-VM.tar # smitty install rp1.tar # tar xvf rp1.tar # smitty install VRTSaa.bff VRTSccg.bff VRTSob.bff VRTSobgui.bff VRTSobc33.bff VRTSHPXP12k_292440.tar # tar xvf VRTSHPXP12k_292440.tar # smitty install ODM # cd /software/symantec/patch/odm # smitty install ODM # cd /software/symantec/patch/10204 # unzip oracle_10.2.0.4.zip # cd oracle_10.2.0.4/10.2.0.4/e1049285 Page 22 of 134
1. Backup the SKGXP libraries under /opt/VRTSvcs/ops/lib # cd /opt/VRTSvcs/ops/lib # for file in libskgxp* > do > cp $file $file.bak > done 2. Change directory to the patch location. # cd <patch_location>/lib 3. Copy all the libraries to /opt/VRTSvcs/ops/lib # cp * /opt/VRTSvcs/ops/lib # cd oracle_10.2.0.4/10.2.0.4/e1243787a 1. # cd /opt/VRTSvcs/rac/bin # cp cssd-monitor cssd-monitor.orig 2. On each node, change directory to the patch location: # cd <patch_location> 3. On each node, copy the new cssd-monitor file to /opt/VRTSvcs/rac/bin: # cp cssd-monitor /opt/VRTSvcs/rac/bin # chmod 744 /opt/VRTSvcs/rac/bin/cssd-monitor
5.2
5.2.1 Storage Foundation for RAC
salesdb1@/opt/VRTS/install# ./installsfrac -rsh -configure [Page 1] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Copyright (c) 2006 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The Licensed Software and Documentation are deemed to be "commercial computer software" and "commercial computer software documentation" as defined in FAR Sections 12.212 and DFARS Section 227.7202. Logs for installsfrac are being created in /var/tmp/installsfrac-KrjD7l. Enter the system names separated by spaces on which to configure SFRAC: salesdb1 salesdb2
Page 23 of 134
Initial system check: Checking rsh communication with salesdb2 .................................................................. AIX 5.3 Checking SFRAC installation on salesdb1 ................................................................... 5.0.1.0 Checking SFRAC installation on salesdb2 ................................................................... 5.0.1.0 [Page 2] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Checking system licensing [Page 3] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The SFRAC installer will help you to: . .............................................................................................................. . * Complete system checks * Configure SFRAC stack * Complete steps required before Oracle installation * Launch Oracle Installer from within this SFRAC installer . .............................................................................................................. . Press [Return] to continue: [Page 4] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Some steps are marked **INSTRUCTIONS ONLY**. These steps: . .............................................................................................................. . * Do not perform any action * Require you to understand the instructions and act * May require you to refer to documentation . .............................................................................................................. . Press [Return] to continue: [Page 5] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu . .............................................................................................................. . * Keep root shells open on all systems to run INSTRUCTIONS ONLY steps * Follow the steps in order
Page 24 of 134
* Navigation keys for all screens: q to quit, b for previous menu . .............................................................................................................. . 1) Check systems for SFRAC ** INSTRUCTIONS ONLY ** 2) Configure SFRAC 3) Prepare to install Oracle 4) Install or relink Oracle Select an option [1-4,q]2 2 SFRAC [Page 8] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Configure SFRAC . .............................................................................................................. . This step will help you configure components of SFRAC stack . .............................................................................................................. . 1) Configure VCS, CVM and CFS 2) Configure I/O fencing ** INSTRUCTIONS ONLY ** 3) [Go to previous menu] Select an option [1-2,b,q]1 [Page 9] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Configure SFRAC > Configure VCS, CVM and CFS . .............................................................................................................. . After completing VCS, CVM and CFS configuration, the installer will exit. You will need to restart the SFRAC installer with -configure option to access the Main Menu. . .............................................................................................................. . Continue with configuration? [y,n,q,b] (n) y [Page 10] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Stopping SFRAC processes. Please wait... SFRAC processes are stopped
Page 25 of 134
Press [Return] to continue: [Page 11] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program To configure SFRAC, please answer the following sets of questions. When a [b] is presented after a question, 'b' may be entered to go back to the first question of the configuration set. When a [?] is presented after a question, '?' may be entered for help or additional information about the question. Following each set of questions, the information you have entered will be presented for confirmation. To repeat the set of questions and correct any previous errors, enter 'n' at the confirmation prompt. No configuration changes are made to the systems until all configuration questions are completed and confirmed. Press [Return] to continue: [Page 12] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program To configure VCS for SFRAC the following information is required: A unique Cluster name A unique Cluster ID number between 0-65535 Two or more NIC cards per system used for heartbeat links All systems are being configured to create one cluster Enter the unique cluster name: [?]rac1 Enter the unique Cluster ID number between 0-65535: [b,?]1 Discovering NICs on salesdb1 ................................................................... discovered en0 en1 en2 en3 en4 en5 en6 en7 en8 Enter the NIC for the first private heartbeat link on salesdb1: [b,?] en2( ) En1 has an IP address configured on it. It could be a public NIC on salesdb1. Are you sure you want to use en2 for the first private heartbeat link? [y,n,q,b,?] (n) y
Page 26 of 134
Would you like to configure a second private heartbeat link? [y,n,q,b,?] (y) y Enter the NIC for the second private heartbeat link on salesdb1: [b,?] en8( ) Would you like to configure a third private heartbeat link? [y,n,q,b,?] (n) n Are you using the same NICs for private heartbeat links on all systems? [y,n,q,b,?] (y) y Checking Media Speed for en2 on salesdb1 ......................................................... 1000 Mbps Full Duplex Checking Media Speed for en6 on salesdb1 .............................................................. Auto negotiation Checking Media Speed for en2 on salesdb2 ......................................................... 1000 Mbps Full Duplex Checking Media Speed for en6 on salesdb2 .............................................................. Auto negotiation The Private NICs do not have the same Media Speed. It is recommended that the Media Speed should be same for all the private NICs. Without this, LLT may not function properly. Consult your Operating System manual for information on how to set the Media Speed. Do you want to continue with the installation? [y,n,q,b] (n) y [Page 13] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Cluster information verification: Cluster Name: rac1 Cluster ID Number: 1 Private Heartbeat NICs for salesdb1: link1=en2 link2=en6 Private Heartbeat NICs for salesdb2: link1=en2 link2=en6 Is this information correct? [y,n,q] (y) y [Page 14] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Veritas Storage Foundation for Oracle RAC can be configured to utilize Symantec Security Services. Running SFRAC in Secure Mode guarantees that all inter-system communication is encrypted and that users are verified with security credentials. When running SFRAC in Secure Mode, NIS and system usernames and passwords are used to verify identity. SFRAC usernames and passwords are no longer utilized when a cluster is running in Secure Mode. Before configuring a cluster to operate using Symantec Security Services, another system must already have Symantec Security Services installed and be operating as a Root Broker. Refer to
Page 27 of 134
the Veritas Cluster Server Installation Guide for more information on configuring a Symantec Product Authentication Service Root Broker. Would you like to configure SFRAC to use Symantec Security Services? [y,n,q] (n) n [Page 15] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The following information is required to add SFRAC users: A user name A password for the user User privileges (Administrator, Operator, or Guest) Do you want to set the username and/or password for the Admin user (default username = 'admin', password='password')? [y,n,q] (n) n Do you want to add another user to the cluster? [y,n,q] (y) n [page 16] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program SFRAC User verification: User: admin Privilege: Administrators
Passwords are not displayed Is this information correct? [y,n,q] (y) [Page 17] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Setup will now configure Veritas Cluster Management Console. If you have configured a management server, you can configure the cluster to be managed by the management server. You can also configure Veritas Cluster Management Console on cluster nodes to manage the local cluster. Do you want this cluster to be managed by a management server? Enter 'y' if you have set up a management server. [y,n,q] (y) n [Page 18] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program
Page 28 of 134
The following information is required to configure the Cluster Management Console: A public NIC used by each system in the cluster A Virtual IP address and netmask for the Cluster Management Console Do you want to configure the Cluster Management Console [y,n,q] (y) n [Page 19] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The following information is required to configure SMTP notification: The domain-based hostname of the SMTP server The email address of each SMTP recipient A minimum severity level of messages to send to each recipient Do you want to configure SMTP notification? [y,n,q] (y) n [Page 20] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The following information is required to configure SNMP notification: System names of SNMP consoles to receive VCS trap messages SNMP trap daemon port numbers for each console A minimum severity level of messages to send to each console Do you want to configure SNMP notification? [y,n,q] (y) n [Page 21] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The following is required to configure the Global Cluster Option: A public NIC used by each system in the cluster A Virtual IP address and netmask The Virtual IP address and NIC may be the same as those configured for Cluster Management Console Do you want to configure the Global Cluster Option? [y,n,q] (y) n
Page 29 of 134
[Page 22] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The following information is required to setup the permission for Veritas Storage Foundation for Oracle RAC Utility to support Oracle Database: The Veritas Storage Foundation for Oracle RAC utility for database administration requires permission changes to allow database administrators (DBAs) access to the tools. The default settings at installation time for the /opt/VRTSdbed directory allows only the root user access to the directory. You need to change permissions to allow a DBA or a group of DBAs access to the Veritas Storage Foundation for Oracle RAC tools Do you want to add single user access on salesdb1 [y,n,q,?] (y) n Do you want to add group access on salesdb1 [y,n,q,?] (y) n You chose 'no' to set permission for Oracle DBA access to /opt/VRTSdbed directory on salesdb1. You must manually set permission of the directory to 750, owner to Oracle DBA user, and group to Oracle DBA group in order to run Veritas Storage Foundation for Oracle RAC utilities.
Press [Return] to continue: [Page 23] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The following information is required to setup the permission for Veritas Storage Foundation for Oracle RAC Utility to support Oracle Database: The Veritas Storage Foundation for Oracle RAC utility for database administration requires permission changes to allow database administrators (DBAs) access to the tools. The default settings at installation time for the /opt/VRTSdbed directory allows only the root user access to the directory. You need to change permissions to allow a DBA or a group of DBAs access to the Veritas Storage Foundation for Oracle RAC tools Do you want to add single user access on salesdb2 [y,n,q,?] (y) n Do you want to add group access on salesdb2 [y,n,q,?] (y) n You chose 'no' to set permission for Oracle DBA access to /opt/VRTSdbed directory on salesdb2. You must manually set permission of the directory to 750, owner to Oracle DBA user, and group to Oracle DBA group in order to run Veritas Storage Foundation for Oracle RAC utilities.
Press [Return] to continue:
Page 30 of 134
[Page 24] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Veritas Volume Manager default disk group name configuration:
Many Veritas Volume Manager commands affect the contents or configuration of a disk group. Such commands require that the user specify a disk group. This is accomplished by using the -g option of a command or setting the VXVM_DEFAULTDG environment variable. An alternative to these two methods for some commands is to configure the name of the default disk group of a system. Note: The default disk group feature is not available with the DCLI vxadm(1M) and vxquery(1M) commands. When using these commands, a required disk group must be explicitly identified using the -g option. Do you want to set up a default disk group for each system? [y,n,q,?] (y) n Creating Veritas Storage Foundation for Oracle RAC configuration files ............................................... Done Copying configuration files to salesdb1 ........................................................................... Done Updating pse.conf on salesdb1 .......................................................... pse.conf correct without update Copying configuration files to salesdb2 ........................................................................... Done Updating pse.conf on salesdb2 .......................................................... pse.conf correct without update [Page 25] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Verify the install systems Fully Qualified Hostnames.
Querying fully qualified domain name of host "salesdb1" ...................................................... fail Press [Return] to continue: [Page 26] Unable to find FQHN for salesdb1!
Enter the fully qualified hostname of "salesdb1" or 'QUIT' to quit install. Enter the fully qualified host name:salesdb1
Page 31 of 134
Querying fully qualified domain name of host "salesdb2" ...................................................... fail Press [Return] to continue: [Page 27] Unable to find FQHN for salesdb2!
Enter the fully qualified hostname of "salesdb2" or 'QUIT' to quit install. Enter the fully qualified host name:salesdb2 [Page 28] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Choose how this install will be managed. This product can configured one of the two following ways: * Storage Foundation Management Server managed host (Recommended) * Standalone host
Pre-requisites for Storage Foundation Management Server managed host: * A Storage Foundation Management Server (SFMS) is configured * SFMS should be currently running and fully operational. * Authentication Broker that is used by SFMS should be running. The following information would be requested during configuration: * The host name of the SFMS * Previously set SFMS agent account password There are no pre-requisites for deploying as a standalone host. Enabling Storage Foundation Management Server management simplifies and improves management of the complex data center resources, reducing planned and unplanned down time. Enable Storage Foundation Management Server Management? [y,n,q] (y) n [Page 29] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Verify the install systems Fully Qualified Hostnames. Page 32 of 134
Configuring VEA in STANDALONE mode on "salesdb1" ............................................................... ok Configuring gridnode on "salesdb1" ............................................................................. ok Registering gridnode on "salesdb1" ............................................................................. ok Configuring actionagent on "salesdb1" .......................................................................... ok Registering actionagent on "salesdb1" .......................................................................... ok Stopping StorageAgent on "salesdb1" ............................................................................ ok Registering StorageAgent on "salesdb1" ......................................................................... ok Registering VAILAgent on "salesdb1" ............................................................................ ok [Page 30] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Verify the install systems Fully Qualified Hostnames. Configuring VEA in STANDALONE mode on "salesdb2" ............................................................... ok Configuring gridnode on "salesdb2" ............................................................................. ok Registering gridnode on "salesdb2" ............................................................................. ok Configuring actionagent on "salesdb2" .......................................................................... ok Registering actionagent on "salesdb2" .......................................................................... ok Stopping StorageAgent on "salesdb2" ............................................................................ ok Registering StorageAgent on "salesdb2" ......................................................................... ok Registering VAILAgent on "salesdb2" ............................................................................ ok Do you want to start Veritas Storage Foundation for Oracle RAC processes now? [y,n,q] (y) y [Page 31] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Starting SFRAC: 100% The following processes or drivers failed to start on salesdb1: odm The following processes or drivers failed to start on salesdb2: vxcached vxconfigbackupd vxconfigd
Page 33 of 134
vxesd vxrelocd odm Configuring CFS agents: Confirming VxVM configuration daemons startup ........................................................... Disabled Unable to complete SFCFS configuration at this time. <cfscluster config> must be run after system reboot. You declined to set up the name of the default disk group for salesdb1. vxconfigd on salesdb2 is not available; installsfrac cannot set the default disk group. Configuration log files, summary file, and response file are saved at: /opt/VRTS/install/logs/installsfrac-GtBg79 SFCFS configuration could not be completed during the install. <cfscluster config> must be run after the system is rebooted. See the Veritas Storage Foundation for Oracle RAC Installation and Configuration Guide for more information. To continue SFRAC configuration, invoke the installer using -configure flag.
5.1.6 salesdb2 volume manager

The following processes or drivers failed to start on salesdb2: vxcached vxconfigbackupd vxconfigd vxesd vxrelocd odm vxinstall root vxinstall volume manager salesdb2@/opt/VRTS/install# vxinstall
Page 34 of 134
VxVM vxinstall INFO V-5-2-2541 Licensing information: System host ID: 0xacb0e79 Host type: powerpc Some licenses are already installed. Do you wish to review them [y,n,q,?] (default: y) n Do you wish to enter another license key [y,n,q,?] (default: n) VxVM vxinstall INFO V-5-2-1807 Populating VxVM DMP device directories ....
Do you want to setup a system wide default disk group? [y,n,q,?] (default: y) n Starting the relocation daemon, vxrelocd. Starting the site auto reattach daemon, vxsited. Starting the cache daemon, vxcached. Starting the diskgroup config backup daemon, vxconfigbackupd. Starting the dg monitoring deamon, vxvvrsecdgd. /usr/sbin/vxstart_vvr[21]: test: 0403-004 Specify a parameter with this command.
VxVM vxinstall INFO V-5-2-2291 The installation is successfully completed.
5.2.2 salesdb1 salesdb2

# shutdown -Fr
5.2.3 salesdb1 cluster

CVM (gabconfig a ) salesdb1@/# cfscluster config The cluster configuration information as read from cluster configuration file is as follows. Cluster : rac1 Nodes : salesdb1 salesdb2 You will now be prompted to enter the information pertaining
Page 35 of 134
to the cluster and the individual nodes.
Specify whether you would like to use GAB messaging or TCP/UDP messaging. If you choose gab messaging then you will not have to configure IP addresses. Otherwise you will have to provide IP addresses for all the nodes in the cluster.
------- Following is the summary of the information: -----Cluster Nodes Transport : rac1 : salesdb1 salesdb2 : gab
-----------------------------------------------------------
Waiting for the new configuration to be added. ======================================================== Cluster File System Configuration is in progress... cfscluster: CFS Cluster Configured Successfully
5.2.4
salesdb1@/# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 575904 membership 01 Port b gen 575903 membership 01 Port d gen 575901 membership 01 Port f gen 57590c membership 01 Port h gen 575906 membership 01 Port o gen 575902 membership 01 Port v gen 575908 membership 01 Port w gen 57590a membership 01 8 a GAB internal use b I/O Fencing d ODM (Oracle Disk Manager) f CFS (VxFS cluster feature) Page 36 of 134
h j k l m o q s t u v w x z
VCS engine (HAD) vxclk monitor port vxclk synch port vxtd (SCRP) port vxtd replication port vcsmm (Oracle RAC/OPS membership module) qlog (VxFS QuickLog) Storage Appliance Storage Appliance CVM (Volume Manager cluster feature) CVM CVM GAB test user client GAB test kernel client
5.2.5 salesdb1 vxfen

HP vxdisk list HA vxdisksetup salesdb1:/etc/VRTSvcs/conf/config# for i in `vxdisk list|grep XP12K|awk { print $1 }` > do > vxdisksetup -i $i format=cdsdisk > done vxdisk list <diskname> 3 159mB XP12K0_70XP12K0_71XP12K0_72 salesdb1@/tmp# salesdb2@/tmp# /etc/vx/diag.d/vxdmpinq /dev/rhdisk144 Inquiry for /dev/rhdisk144, evpd 0x0, page code 0x0 Vendor id : HP Product id : OPEN-V Revision : 6001 Serial Number : 50 0B06C0903 salesdb2@/tmp# /etc/vx/diag.d/vxdmpinq /dev/rhdisk143 Inquiry for /dev/rhdisk143, evpd 0x0, page code 0x0 Vendor id : HP Product id : OPEN-V Revision : 6001 Serial Number : 50 0B06C0902 salesdb2@/tmp# /etc/vx/diag.d/vxdmpinq /dev/rhdisk142 Inquiry for /dev/rhdisk142, evpd 0x0, page code 0x0 Vendor id : HP Product id : OPEN-V Revision : 6001 Serial Number : 50 0B06C0901 salesdb2 OS VM # vxdisk -e list salesdb1 salesdb2 Page 37 of 134
XP12K0_0 XP12K0_1 XP12K0_3
hdisk74 hdisk73 hdisk72
XP12K0_2 XP12K0_0 XP12K0_1
fencing salesdb1@/# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n VERITAS vxfentsthdw version 5.0 AIX
The utility vxfentsthdw works on the two nodes of the cluster. The utility verifies that the shared storage one intends to use is configured to support I/O fencing. It issues a series of vxfenadm commands to setup SCSI-3 registrations on the disk, verifies the registrations on the disk, and removes the registrations from the disk.
******** WARNING!!!!!!!! ******** THIS UTILITY WILL DESTROY THE DATA ON THE DISK!! Do you still want to continue : [y/n] (default: n) y Enter the first node of the cluster: salesdb1 Enter the second node of the cluster: salesdb2 Enter the disk name to be checked for SCSI-3 PGR on node salesdb1 in the format: for raw: /dev/rhdisk74 Enter the disk name to be checked for SCSI-3 PGR on node salesdb2 in the format: for raw: /dev/rhdisk74 /dev/rhdisk73/dev/rhdisk73 salesdb1@/# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n Enter the disk name to be checked for SCSI-3 PGR on node salesdb1 in the format: for raw: /dev/rhdisk73
Page 38 of 134
Enter the disk name to be checked for SCSI-3 PGR on node salesdb2 in the format: for raw: /dev/rhdisk73 /dev/rhdisk72/dev/rhdisk72 salesdb1@/# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n Enter the disk name to be checked for SCSI-3 PGR on node salesdb1 in the format: for raw: /dev/rhdisk72 Enter the disk name to be checked for SCSI-3 PGR on node salesdb2 in the format: for raw: /dev/rhdisk72 fencing fencing key echo /dev/vx/rdmp/XP12K0_134 > /tmp/1.tmp vxfenadm -a -k TMP -f /tmp/1.tmp vxfenadm -c -k TMP -f /tmp/1.tmp fencing dg salesdb1@/# vxdg init vxfencoorddg vxfencoorddg01=XP12K0_70 salesdb1@/# vxdg -g vxfencoorddg adddisk vxfencoorddg02=XP12K0_71 salesdb1@/# vxdg -g vxfencoorddg adddisk vxfencoorddg03=XP12K0_72 salesdb1@/# vxdg -g vxfencoorddg set coordinator=on (,) salesdb1@/etc/VRTSvcs/conf/config# vxdg deport vxfencoorddg vxfentsthdw fencing dg SCSI-3 salesdb11@/# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n -c vxfencoorddg PASSED Evaluate the disk before testing ........................ No Pre-existing keys salesdb1@/etc/VRTSvcs/conf/config# vxdg -t import vxfencoorddg salesdb1@/etc/VRTSvcs/conf/config# vxdg deport vxfencoorddg salesdb1@/etc/VRTSvcs/conf/config# echo "vxfencoorddg" > /etc/vxfendg salesdb1@/etc/VRTSvcs/conf/config# rcp /etc/vxfendg salesdb2:/etc fencing daemon cluster salesdb1@/# haconf -dump -makero VCS WARNING V-16-1-10369 Cluster not writable. salesdb1@/# hastop all salesdb1@/# hastatus -sum HA Page 39 of 134
salesdb1@/# cd /etc/init.d Salesdb1@/etc/init.d# ./vxfen.rc stop Stopping vxfen.. Stopping vxfen.. Done Salesdb2@/etc/init.d# ./vxfen.rc stop Stopping vxfen.. Stopping vxfen.. Done cluster , UseFence = SCSI3 salesdb1@/etc/VRTSvcs/conf/config# cp main.cf main.orig salesdb1@/etc/VRTSvcs/conf/config# vi main.cf cluster rac_cluster1 ( UserNames = { admin = "cDRpdxPmHpzS." } Administrators = { admin } UseFence = SCSI3 HacliUserLevel = COMMANDROOT ) /etc/VRTSvcs/conf/config/main.cf salesdb1@/etc/VRTSvcs/conf/config# hacf -verify /etc/VRTSvcs/conf/config salesdb1@/etc/VRTSvcs/conf/config# rcp main.cf salesdb2:/etc/VRTSvcs/conf/config/ salesdb1@/etc/VRTSvcs/conf/config# cp /etc/vxfen.d/vxfenmode_scsi3_dmp /etc/vxfenmode salesdb2@/etc/VRTSvcs/conf/config# cp /etc/vxfen.d/vxfenmode_scsi3_dmp /etc/vxfenmode : raw: # cp /etc/vxfen.d/vxfenmode_scsi3_raw /etc/vxfenmode
5.2.6 salesdb1 salesdb2 cluster

salesdb1@/etc/VRTSvcs/conf/config# /etc/init.d/vxfen.rc start Starting vxfen.. Checking for /etc/vxfendg Starting vxfen.. Done salesdb2@/etc/VRTSvcs/conf/config# /etc/init.d/vxfen.rc start Starting vxfen.. Checking for /etc/vxfendg Starting vxfen.. Done salesdb1@/etc/VRTSvcs/conf/config# hastart salesdb2@/etc/VRTSvcs/conf/config# hastart HA salesdb1@/opt/VRTS/messages# hastatus -sum
Page 40 of 134
-- SYSTEM STATE -- System A salesdb1 A salesdb2 -- GROUP STATE -- Group B cvm B cvm System salesdb1 salesdb2 Probed Y Y N N AutoDisabled State State RUNNING RUNNING Frozen 0 0
ONLINE ONLINE
salesdb1@/etc/VRTSvcs/conf/config# hastatus -sum -- SYSTEM STATE -- System A salesdb1 A salesdb2 -- GROUP STATE -- Group B cvm B cvm System salesdb1 salesdb2 Probed Y Y N N AutoDisabled State State RUNNING RUNNING Frozen 0 0
ONLINE ONLINE
vxfen salesdb1@/# /sbin/vxfenadm -d I/O Fencing Cluster Information: ================================ Fencing Protocol Version: 201 Fencing Mode: SCSI3 Fencing SCSI3 Disk Policy: dmp Cluster Members: * 0 (salesdb1) 1 (salesdb2) RFSM State Information: node 0 in state 8 (running) Page 41 of 134
node 1 in state 8 (running)
5.2.7 salesdb1
DG DG name Chnldbocrvoted g Chnldbdatadg disk 2 4 vol name chnldbcrsvol chnldbsysvol chnldbredovol 4 4 chnldbundovol chnldbtempvol size 2G 20G 100G 120G 120G 720G 300G 600G mount point /oracrs /chnldb/chnldbsys_u01 / chnldb/chnldbredo_u01 / chnldb/chnldbundo_u0 1 / chnldb/chnldbtemp_u0 1 / chnldb/chnldbdata_u01 /chnldb/chnldbidx_u01 /oraflashback layout Conc Mirror stripe col 4 stripe col 4 stripe col 4 stripe col 4 stripe col 4 stripe col 4 Conc stripe unitsize bsize 8k 8k 8k 8k 8k 8k 8k 8k
512 512 512 512 512 512
24 chnldbdatavol 12 chnldbidxvol Chnldblogdg 20 chnldbfbvol 30GB
disk group Cluster Group
Volume:
Page 42 of 134
volume stripe volume:
Page 43 of 134
volume Columns Stripe unit size
Page 44 of 134
block size 8KAdd to file system table # umount /oracrs # mount -V vxfs -o rw,cluster,crw,delaylog,suid,ioerror=mdisable,qio,largefiles /dev/vx/dsk/chnldbocrvotedg/chnldbcrsvol /oracrs # chown -R oracle:oinstall /chnldb/chnldbocrvote
Page 45 of 134
Oracle RAC
6.1 Oracle 10gr2 RAC
6.1.1 Oracle CRS 10.2.0.1
salesdb1@/opt/VRTS/install#./installsfrac -rsh -configure [Page 1] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Copyright (c) 2006 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The Licensed Software and Documentation are deemed to be "commercial computer software" and "commercial computer software documentation" as defined in FAR Sections 12.212 and DFARS Section 227.7202. Logs for installsfrac are being created in /var/tmp/installsfrac-mUS8zG. Enter the system names separated by spaces on which to configure SFRAC: salesdb1 salesdb2 Initial system check: Checking rsh communication with salesdb2 .................................................................. AIX 5.3 Checking SFRAC installation on salesdb1 ................................................................... 5.0.1.0 Checking SFRAC installation on salesdb2 ................................................................... 5.0.1.0 [Page 2] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Checking system licensing Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The SFRAC installer will help you to: . .............................................................................................................. . * Complete system checks * Configure SFRAC stack * Complete steps required before Oracle installation * Launch Oracle Installer from within this SFRAC installer . .............................................................................................................. . Press [Return] to continue: [Page 3]
Page 46 of 134
Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Some steps are marked **INSTRUCTIONS ONLY**. These steps: . .............................................................................................................. . * Do not perform any action * Require you to understand the instructions and act * May require you to refer to documentation . .............................................................................................................. . Press [Return] to continue: [Page 4] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu . .............................................................................................................. . * Keep root shells open on all systems to run INSTRUCTIONS ONLY steps * Follow the steps in order * Navigation keys for all screens: q to quit, b for previous menu . .............................................................................................................. . 1) Check systems for SFRAC ** INSTRUCTIONS ONLY ** 2) Configure SFRAC 3) Prepare to install Oracle 4) Install or relink Oracle Select an option [1-4,q,?]4 [Page 5] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Check supported matrix ** INSTRUCTIONS ONLY ** . .............................................................................................................. . Before continuing, refer to <http://support.veritas.com/docs/274421> for the latest SFRAC support matrix of all qualified Oracle releases and patches. . .............................................................................................................. . Press [Return] to continue: [Page 6] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Choose Oracle version . .............................................................................................................. . * Choose from the following supported versions
Page 47 of 134
. .............................................................................................................. .
1) Oracle 9iR2 2) Oracle 10gR1 3) Oracle 10gR2 Select an option [1-3,q] 3
[Page 7] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Choose task . .............................................................................................................. . * Oracle Clusterware (CRS) must be installed before RDBMS server * Relinking is required if you installed Oracle software or patches from outside SFRAC installer . .............................................................................................................. .
1) Install Oracle Clusterware (CRS) 2) Install Oracle RDBMS server 3) Relink Oracle 4) [Go to previous menu] Select an option [1-3,b,q]1 [Page 8] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle Clusterware (CRS) > Set DISPLAY . .............................................................................................................. . Specify the address of the X11-server on the machine where you want Oracle installer displayed. Example: 10.20.12.150:0.0 . .............................................................................................................. .
Enter DISPLAY environment variable: [b] (10.222.18.89:0) [Page 9] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program
Page 48 of 134
Main Menu > Install or relink Oracle > Install Oracle Clusterware (CRS) > Oracle Username . .............................................................................................................. . Oracle is installed and run as special Oracle userid. You should have created such a user id on all systems using the name and numerical uid & gid . .............................................................................................................. .
Enter Oracle UNIX user name: [b] (oracle) [Page 10] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle Clusterware (CRS) > Oracle Groupname . .............................................................................................................. . Oracle database home will belong to this group . .............................................................................................................. . Enter Oracle UNIX group name: [b] (oinstall) [Page 11] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle Clusterware (CRS) > CRS Install Image . .............................................................................................................. . Specifies the location of Oracle CRS install image . .............................................................................................................. . Enter absolute path of CRS install image: [b] /software/db/crs [Page 12] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle Clusterware (CRS) > Oracle base directory . .............................................................................................................. . Specifies the base of the Oracle directory structure for OFA compliant databases. . .............................................................................................................. .
Enter Oracle base directory: [b] /u01/oracle [Page 13] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program
Page 49 of 134
Main Menu > Install or relink Oracle > Install Oracle Clusterware (CRS) > CRS Home directory . .............................................................................................................. . Specifies the directory containing Oracle CRS Software. . .............................................................................................................. .
Enter absolute path of CRS Home directory: [b] /u01/oracle/product/crs10gr2 /u01/oracle/product/crs10gr2: no such directory on fftest1 Do you wish to create this directory? [y,n,q,b] (n)y mkdir -p /u01/oracle/product/crs10gr2 chown -R oracle:oinstall /u01/oracle/product/crs10gr2 chmod -R 744 /u01/oracle/product/crs10gr2 Run above command on all systems? [y,n,q,b] (n) y [Page 14] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle Clusterware (CRS) > Patch Level[m(B . .............................................................................................................. . Specifies the patch level of Oracle CRS/Database Software being installed or relinked. For example: Enter 0.2 for 10.2.0.2. Enter 0.7 for 9.2.0.7 . .............................................................................................................. .
Enter Oracle Patch level for 10.2: [b] (0.1) [Page 15] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Oracle environment information verification Oracle Unix User: oracle Oracle Unix Group: oinstall Oracle Clusterware (CRS) Installation Path: /software/db/crs Oracle Clusterware (CRS) Home: /u01/oracle/product/crs10gr2 Oracle Release: 10.2 Oracle Patch Level: 0.1 Oracle Base: /u01/oracle Is this information correct? [y,n,q] (y) [Page 16]
Page 50 of 134
Verifying fileset prerequisites on salesdb1 .................................................................... ok Verifying fileset prerequisites on salesdb2 .................................................................... ok Comparing Oracle UNIX user id on all systems ................................................................ Done Comparing Oracle group id on all systems .................................................................... Done Archiving SFRAC libskgxn2.a on salesdb1 ........................................................................ ok Archiving SFRAC libskgxn2.a on salesdb2 ........................................................................ ok Copying libraries to /opt/ORCLcluster on salesdb1 .............................................................. ok Copying libraries to /opt/ORCLcluster on salesdb2 .............................................................. ok [Page 17] ORACLE CRS INSTALLATION For installation to proceed, Oracle requires you to run the rootpre.sh script located under /software/db/cluster/rootpre. Run the following commands on all nodes: # cd /software/db/cluster/rootpre #./rootpre.sh Has rootpre.sh been run by root? [y,n,q] y salesdb1 salesdb2 root rootpre.sh [Page 19] Invoking OUI to install Oracle CRS. Please wait... ............................................................ ok Now install Oracle CRS software from the OUI. Refer to Oracle documentation for installation instructions. NOTE: Near the end of the installation, Oracle CRS installer will prompt you to run the root.sh script. Press <RETURN> here when prompted to run root.sh: , CRS
Page 51 of 134
next
Home Name CRSHomeproduct language EnglishSimplified ChineseTraditional Chinese oknext Page 52 of 134
Oracle overall result passednext
/etc/hosts private node name _-next
Page 53 of 134
ip private public edit next: en6
normal redundancylocation /oracrs/ocr/ocr_01.dbf/oracrs/ocr/ocr_02.dbf
Page 54 of 134
Normal RedundancyVoting Disk Location /oracrs/css/vdisk_01.dbf
Page 55 of 134
rootinstRoot.sh root.sh 1 root.sh SFRAC Press <RETURN> here when prompted to run root.sh: Verifying binaries in /u01/oracle/product/crs on salesdb1 ................................................................ ok Verifying binaries in /u01/oracle/product/crs on salesdb2 ................................................................ ok Copying SFRAC libskgxn on salesdb1 ............................................................................. ok Copying SFRAC libskgxn on salesdb2 ............................................................................. ok Copying SFRAC libskgxp on salesdb1 ............................................................................. ok Copying SFRAC libskgxp on salesdb2 ............................................................................. ok Copying SFRAC ODM library on salesdb1 .......................................................................... ok Copying SFRAC ODM library on salesdb2 .......................................................................... ok CRS post-installation setup is now complete. You may now invoke root.sh to complete CRS installation. Press <RETURN> here after running root.sh on all nodes: rootinstRoot.sh root.sh
Page 56 of 134
1 root.sh
2 root.sh DISPLAY <local ip addr>:0 vipca vipca # cd /u01/oracle/product/crs10gr2/bin # ./vipca
Page 57 of 134
next
salesdb1-vipnext
Page 58 of 134
Vipca ok oracle crs ok oracle crs
Succeeded crs crs SFRAC
Page 59 of 134
Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Choose task . .............................................................................................................. . * Oracle Clusterware (CRS) must be installed before RDBMS server * Relinking is required if you installed Oracle software or patches from outside SFRAC installer . .............................................................................................................. .
1) Install Oracle Clusterware (CRS) 2) Install Oracle RDBMS server 3) Relink Oracle 4) [Go to previous menu] Select an option [1-3,b,q]q oracle crs
6.1.2 Oracle database 10.2.0.1

salesdb1@/opt/VRTS/install#./installsfrac -rsh -configure [Page 1] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Copyright (c) 2006 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The Licensed Software and Documentation are deemed to be "commercial computer software" and "commercial computer software documentation" as defined in FAR Sections 12.212 and DFARS Section 227.7202. Logs for installsfrac are being created in /var/tmp/installsfrac-z8CqPt. Enter the system names separated by spaces on which to configure SFRAC: salesdb1 salesdb2 Initial system check: Checking rsh communication with salesdb2 .................................................................. AIX 5.3 Checking SFRAC installation on salesdb1 ................................................................... 5.0.1.0 Checking SFRAC installation on salesdb2 ................................................................... 5.0.1.0
Page 60 of 134
[Page 2] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Checking system licensing [Page 3] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program The SFRAC installer will help you to: . .............................................................................................................. . * Complete system checks * Configure SFRAC stack * Complete steps required before Oracle installation * Launch Oracle Installer from within this SFRAC installer . .............................................................................................................. . Press [Return] to continue: [Page 4] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Some steps are marked **INSTRUCTIONS ONLY**. These steps: . .............................................................................................................. . * Do not perform any action * Require you to understand the instructions and act * May require you to refer to documentation . .............................................................................................................. .
Press [Return] to continue: [Page 5] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu . .............................................................................................................. . * Keep root shells open on all systems to run INSTRUCTIONS ONLY steps * Follow the steps in order * Navigation keys for all screens: q to quit, b for previous menu . .............................................................................................................. .
1) Check systems for SFRAC ** INSTRUCTIONS ONLY ** Page 61 of 134
2) Configure SFRAC 3) Prepare to install Oracle 4) Install or relink Oracle Select an option [1-4,q,?] 4 [Page 6] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Check supported matrix ** INSTRUCTIONS ONLY ** . .............................................................................................................. . Before continuing, refer to <http://support.veritas.com/docs/274421> for the latest SFRAC support matrix of all qualified Oracle releases and patches. . .............................................................................................................. .
Press [Return] to continue: [Page 7] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Choose Oracle version . .............................................................................................................. . * Choose from the following supported versions . .............................................................................................................. .
1) Oracle 9iR2 2) Oracle 10gR1 3) Oracle 10gR2 Select an option [1-3,q] 3 [Page 8] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Choose task . .............................................................................................................. . * Oracle Clusterware (CRS) must be installed before RDBMS server * Relinking is required if you installed Oracle software or patches from outside SFRAC installer
Page 62 of 134
. .............................................................................................................. .
1) Install Oracle Clusterware (CRS) 2) Install Oracle RDBMS server 3) Relink Oracle 4) [Go to previous menu] Select an option [1-3,b,q]2 [Page 9] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Set DISPLAY . .............................................................................................................. . Specify the address of the X11-server on the machine where you want Oracle installer displayed. Example: 10.20.12.150:0.0 . .............................................................................................................. . Enter DISPLAY environment variable: [b] (10.222.18.89:0) [Page 10] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Oracle Username . .............................................................................................................. . Oracle is installed and run as special Oracle userid. You should have created such a user id on all systems using the name and numerical uid & gid . .............................................................................................................. .
Enter Oracle UNIX user name: [b] (oracle) [Page 11] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Oracle Groupname . .............................................................................................................. . Oracle database home will belong to this group . .............................................................................................................. .
Page 63 of 134
Enter Oracle UNIX group name: [b] (oinstall) [Page 12] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Database Install Image . .............................................................................................................. . Specifies the location of Oracle Database Software install image. . .............................................................................................................. .
Enter absolute path of Database install image: [b] /software/db/database [Page 13] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Oracle base directory . .............................................................................................................. . Specifies the base of the Oracle directory structure for OFA compliant databases. . .............................................................................................................. .
Enter Oracle base directory: [b] /u01/oracle [Page 14] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > CRS Home directory . .............................................................................................................. . Specifies the directory containing Oracle CRS Software. . .............................................................................................................. .
Enter absolute path of CRS Home directory: [b] /u01/oracle/product/crs10gr2 [Page 15] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Database home directory . .............................................................................................................. . Specifies the directory containing Oracle Database Software. Page 64 of 134
. .............................................................................................................. .
Enter absolute path of Database Home directory: [b]/u01/oracle/product/db10gr2 /oracle10/app/oracle/product/10.2.0: no such directory on salesdb1 Do you wish to create this directory? [y,n,q,b] (n) y [Page 16] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Create directory . .............................................................................................................. . This step will create a directory on all cluster nodes . .............................................................................................................. .
mkdir p /u01/oracle/product/db10gr2 chown -R oracle:dba /u01/oracle/product/db10gr2 chmod -R 744 /u01/oracle/product/db10gr2 Run above command on all systems? [y,n,q,b] (n) y [Page 17] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Database home directory . .............................................................................................................. . Specifies the directory containing Oracle Database Software. . .............................................................................................................. .
Enter absolute path of Database Home directory: [b] (/u01/oracle/product/db10gr2) [Page 18] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Main Menu > Install or relink Oracle > Install Oracle RDBMS server > Patch Level . .............................................................................................................. . Specifies the patch level of Oracle CRS/Database Software being installed or relinked. For example: Enter 0.2 for 10.2.0.2. Enter 0.7 for 9.2.0.7 . .............................................................................................................. .
Page 65 of 134
Enter Oracle Patch level for 10.2: [b] (0.1) [Page 19] Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program Oracle environment information verification Oracle Unix User: oracle Oracle Unix Group: oinstall Oracle Clusterware (CRS) Home: /u01/oracle/product/crs10gr2 Oracle Release: 10.2 Oracle Patch Level: 0.1 Oracle Base: /u01/oracle Oracle Home: /u01/oracle/product/db10gr2 RDBMS Installation Path: /software/db/database Is this information correct? [y,n,q] (y) Verifying CRS on salesdb1 ................................................................................. Running Verifying CRS on salesdb2 ................................................................................. Running Comparing Oracle UNIX user id on all systems ................................................................ Done Comparing Oracle group id on all systems .................................................................... Done [Page 20] ORACLE DB INSTALLATION For installation to proceed, Oracle requires you to run the rootpre.sh script located under /software/symantec/db/database/rootpre. Run the following commands on all nodes: #cd /software/symantec/db/database #./rootpre.sh Has rootpre.sh been run by root? [y,n,q] (y) salesdb1,salesdb2 root rootpre.sh ok Invoking OUI to install Oracle RDBMS Software. Please wait... .................................................
Now install Oracle database software from the OUI. Refer to Oracle documentation for installation instructions. NOTE: Near the end of the installation, Oracle Database Software installer will prompt you to run the root.sh script. Press <RETURN> here when prompted to run root.sh: Page 66 of 134
oracle database 10.2.0.1
Enterprise Editionproduct language EnglishSimplified ChineseTraditional Chinese oknext
Page 67 of 134
OUIHome1 ORAHomenext
Cluster Installationnext
Page 68 of 134
Oracle overall result passednext
Install database Software only
Page 69 of 134
root.sh SFRAC Press <RETURN> here when prompted to run root.sh:
Page 70 of 134
Verifying binaries in /u01/oracle/product/db10gr2 on salesdb1 .......................................... ok Verifying binaries in /u01/oracle/product/db10gr2 on salesdb2 .......................................... ok Copying SFRAC libskgxn on salesdb1 ............................................................................. ok Copying SFRAC libskgxn on salesdb2 ............................................................................. ok Copying SFRAC ODM library on salesdb1 .......................................................................... ok Copying SFRAC ODM library on salesdb2 .......................................................................... ok Copying SFRAC libskgxp on salesdb1 ............................................................................. ok Copying SFRAC libskgxp on salesdb2 ............................................................................. ok Relinking Oracle on salesdb1 ................................................................................... ok Relinking Oracle on salesdb2 ................................................................................... ok Oracle post-installation setup is now complete. You may now invoke root.sh to complete Oracle installation Press <RETURN> here after running root.sh on all nodes: salesdb1,salesdb2 root root.sh oracle database
SFRAC Veritas Storage Foundation for Oracle RAC 5.0 Configuration Program
Page 71 of 134
Main Menu > Install or relink Oracle > Choose task . .............................................................................................................. . * Oracle Clusterware (CRS) must be installed before RDBMS server * Relinking is required if you installed Oracle software or patches from outside SFRAC installer . .............................................................................................................. .
1) Install Oracle Clusterware (CRS) 2) Install Oracle RDBMS server 3) Relink Oracle 4) [Go to previous menu] Select an option [1-3,b,q] q installsfrac log files are saved at /opt/VRTS/install/logs/installsfrac-z8CqPt oracle database
6.1.3 Oracle crs 10.2.0.2

salesdb1 salesdb2 oracle crs # /u01/oracle/product/crs10gr2/bin/crsctl stop crs salesdb1salesdb2rootslibclean # /usr/sbin/slibclean runInstaller CRSDISPLAY<local ip addr>:0 salesdb1@/home/oracle$ cd /software/db/database/patches salesdb1@/home/oracle$ ./runInstaller
Page 72 of 134
CRSHomeORAHomenext
Page 73 of 134
next
root102.sh:
Page 74 of 134
$ $ $ $ $ $ $ $ $ $
cd /u01/oracle/product/crs10gr2/install/patch102/lib mv libskgxp10.a libskgxp10.a.orig cp /opt/VRTSvcs/rac/lib/libskgxp10_ver25_64.a libskgxp10.a mv libskgxpg.a libskgxpg.a.orig cp /opt/VRTSvcs/rac/lib/libskgxp10_ver25_64.a libskgxpg.a cd /u01/oracle/product/crs10gr2/install/patch102/lib32 mv libskgxp10.a libskgxp10.a.orig cp /opt/VRTSvcs/rac/lib/libskgxp10_ver25_32.a libskgxp10.a mv libskgxpg.a libskgxpg.a.orig cp /opt/VRTSvcs/rac/lib/libskgxp10_ver25_32.a libskgxpg.a
root # /u01/oracle/product/crs10gr2/bin/crsctl stop crs # /u01/oracle/product/crs10gr2/install/root102.sh # /u01/oracle/product/crs10gr2/bin/crs_stat -t oracle crs salesdb2 crs_stat
6.1.4 Oracle database 10.2.0.4

runInstaller Oracle database salesdb1@/home/oracle$ cd /software/db/patch10202/ salesdb1@/home/oracle$ ./runInstaller
Page 75 of 134
ORAHome next
Page 76 of 134
next
Page 77 of 134
root.sh root root.sh ok oracle patch readme.html changePerm.sh salesdb1@/home/oracle$ cd $ORACLE_HOME/install salesdb1@/home/oracle$ ./changePerm.sh y
6.1.5 Oracle database ( 10.2.0.2 )

oracle libknlopt.a salesdb1@/home/oracle$ cksum $ORACLE_HOME/rdbms/lib/libknlopt.a salesdb2@/home/oracle$ cksum $ORACLE_HOME/rdbms/lib/libknlopt.a salesdb1 libknlopt.a salesdb2 $ rcp $ORACLE_HOME/rdbms/lib/libknlopt.a salesdb2:/u01/oracle/product/db10gr2/rdbms/lib opatch 4637591
oracle $ cd /software/Symantec/oracle_patch $ unzip p4637591_10202_AIX64-5L.zip $ cd 4637591 $ opatch apply -local opatch succeeded
opatch 4690794 patch 6646850

oracle $ cd /software/Symantec/oracle_patch $ unzip p4690794_10202_AIX64-5L.zip $ cd 4690794 $ opatch apply -local opatch succeeded
6.1.6 relink
relink $ cp $ORACLE_HOME/lib/libskgxn2.a $ORACLE_HOME/lib/libskgxn2.a.bak $ cp $ORACLE_HOME/lib/libskgxn2.so $ORACLE_HOME/lib/libskgxn2.so.bak $ cp $ORACLE_HOME/lib/libskgxp10.a $ORACLE_HOME/lib/libskgxp10.a.bak SFRAC relink salesdb1@/opt/VRTS/install# ./installsfrac -rsh -configure
Page 78 of 134
Main Menu > Install or relink Oracle > Choose task . .............................................................................................................. . * Oracle Clusterware (CRS) must be installed before RDBMS server * Relinking is required if you installed Oracle software or patches from outside SFRAC installer . .............................................................................................................. .
1) Install Oracle Clusterware (CRS) 2) Install Oracle RDBMS server 3) Relink Oracle 4) [Go to previous menu] Select an option [1-3,b,q] 3 Oracle UNIX user name Oracle UNIX group name Oracle Base Directory Oracle CRS Home Directory Oracle Database Directory Oracle Patch level for 10.2 oracle oinstall /u01/oracle /u01/oracle/product/crs /u01/oracle/product/db10gr2 0.4
Page 79 of 134
relink
6.1.7 Link Veritas extension for ODM(Oracle Disk Manager)

6.1.6 # cksum /opt/VRTSodm/lib/libodm64.so $ORACLE_HOME /lib/libodm10.so $ mv $ORACLE_HOME/lib/libodm10.so $ORACLE_HOME/lib/libodm10.so.bak $ ln -s /opt/VRTSodm/lib/libodm64.so $ORACLE_HOME/lib/libodm10.so
6.1.8 Oracle RAC cluster interconnect Veritas LLT UDP

$ mv $ORACLE_HOME/lib/libskgxp10.a $ORACLE_HOME/lib/libskgxp10.a.bak.1 $ cp $ORACLE_HOME/lib/ libskgxpg.a.orcl $ORACLE_HOME/lib/libskgxp10.a $ rcp $ORACLE_HOME/lib/ libskgxpg.a.orcl salesdb2:$ORACLE_HOME/lib/libskgxp10.a $ mv $ORACLE_HOME/lib/libskgxpg.a $ORACLE_HOME/lib/libskgxpg.a.bak $ cp $ORACLE_HOME/lib/libskgxpg.a.orcl $ORACLE_HOME/lib/libskgxpg.a $ rcp $ORACLE_HOME/lib/libskgxpg.a.20080506.ORCL salesdb2:$ORACLE_HOME/lib/libskgxpg.a $ cd $ORACLE_HOME/rdbms/lib $ make -f ins_rdbms.mk rac_on ioracle alter cluster interconnect IPC version:Oracle UDP/IP (generic) IPC Vendor 1 proto 2
Page 80 of 134
Oracle RAC
7.1 7.2 dbca
7.2.1 Listener
2 Litsner : listener, listener2: 1521, 1522
7.2.2 Oracle RAC

salesdb1 oracle dbca $ dbca
RAC databasenext
Page 81 of 134
create databasenext
Page 82 of 134
general purpose
chnldb
Page 83 of 134
Next
password oracleNext
Page 84 of 134
Cluster File System
/chnldb/chnldbsys_u01
Page 85 of 134
next
Page 86 of 134
next
sga pga 40%
Page 87 of 134
Processes 1000
AL43UTF8 UTF8default language Simplified Chinesedefault date format china all initial parameters
Page 88 of 134
db_writer_processes 1 4Close next
max log history 1 100 Maximum Redo Log Files: 255 max log members 3 5
Page 89 of 134
datafile <tablespace name>_01.dbf
redo log group 100M thread 1 5 group memberthread 2 5 group member thread 1 group
Page 90 of 134
thread 2 group
Next
Page 91 of 134
Finish
exit
Page 92 of 134
dbca cluster database instances
7.2.2
sqlplus CREATE SMALLFILE TABLESPACE PAETL_DATA LOGGING DATAFILE '/lifedb/oradata/paetl_data_01.dbf' SIZE 12M REUSE AUTOEXTEND ON NEXT 10M MAXSIZE 8192M EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO; ALTER TABLESPACE PAETL_DATA ADD DATAFILE '/lifedb/oradata/paetl_data_02.dbf' SIZE 12M REUSE AUTOEXTEND ON NEXT 10M MAXSIZE 8192M;
Page 93 of 134
Veritas Cluster
8.1 CLUSTER
8.1.1 CLUSTER CVM
1 X cluster salesdb1@/# hagui admin connect password
cvm resource: cvmvoldg_vote resource ocrvotedg ocrvotevol CVMDiskGroup Value: chnldbocrvotedg CVMVolumn Value: chnldbcrsvol CVMActivation: sw CVMVolDgCVMActivation swshared write Enabled
Page 94 of 134
link cvmvoldg_vote cvm_clus
cfs_ocrvote : cfs_ocrvote mount /oracrs CFSMountBlockDevice /dev/vx/dsk/<dg name>/<volume name> Enabled MountPoint Value: /oracrs BlockDevice Value: /dev/vx/dsk/chnldbocrvotedg/chnldbcrsvol
Page 95 of 134
link cfs_ocrvote cvmvoldg_votevxfsckd
8.1.2 CLUSTER oracle

1 X cluster salesdb1@/# hagui admin connect password Oracle
Page 96 of 134
Add Service Group
ora_grp service group system startupService Group Type parallelok
Page 97 of 134
ora_grp Resources Add Resource
cvmvoldg_datadg CVMDiskGroup dg : chnldbdatadgCVMVolume Volume chnldbsysvol,chnldbredovol,chnldbundovol,chnldbtempvol,chnldbdatavol,chnldbidxvolCVMAc tivation sw shared write Critial Enabled
Page 98 of 134
cvmvoldg_logdg CVMDiskGroup dg : chnldblogdgCVMVolume Volume chnldbfbvolCVMActivation sw shared write Critial Enabled
cfs_chnldbsys_u01 Resource Type: CFSMount Block Device /dev/vx/dsk/<dg name>/<vol name>/dev/vx/dsk/chnldbdatadg/chnldbsysvol Critial Enabled
Page 99 of 134
link cfs_chnldbsys_u01 link cvmvoldg_datadg cfs_chnldbsys_u01 cvmvoldg_datadg
cfs_chnldbsys_u01, cfs_chnldbredo_u01, cfs_chnldbundo_u01, cfs_chnldbtemp_u01,cfs_chnldbdata_u01,cfs_chnldbidx_u01,cfs_oraflashback resource resource group online at all systems resouce up
Page 100 of 134
cssd Application oracle crs ApplicetionStartProgram /opt/VRTSvcs/ops/bin/cssd-onlineStopProgram /opt/VRTSvcs/ops/bin/cssd-offlineClearProgram /opt/VRTSvcs/ops/bin/cssdcleanMonitorProgram /opt/VRTSvcs/ops/bin/cssd-monitor Critical Enabled
link cssd mount
Page 101 of 134
8.1.3 link ora_grp cvm group
cluster rac1 link ora_grp cvm
Page 102 of 134
8.2 ORACLE CRS

8.2.1 Oracle CRS
Oracle CRS CVM oracle crs salesdb1@/# /etc/init.crs disable Automatic startup disabled for system boot. salesdb1@/etc/rc.d/rc2.d# cd /etc/rc.d/rc2.d salesdb1@/etc/rc.d/rc2.d# mv K96init.crs k96init.crs 2 salesdb2@/# /etc/init.crs disable Automatic startup disabled for system boot. salesdb2@/etc/rc.d/rc2.d# mv K96init.crs k96init.crs : /etc/inittab oracle 3 h1:2:respawn:/etc/init.evmd run >/dev/null 2>&1 </dev/null h2:2:respawn:/etc/init.cssd fatal >/dev/null 2>&1 </dev/null h3:2:respawn:/etc/init.crsd run >/dev/null 2>&1 </dev/null
Page 103 of 134
8.2 Oracle
Oracle cvmvoldg_vote, cfs_ocrvote, cssd, cfs_chnldbdata_u01 Critial
Page 104 of 134
8.3 Oracle Agent
OnlineTimeOutOfflineTimeOut 300() 600()
8.4
# vi /opt/VRTSvcs/ops/bin/cssd-offline su - oracle -c " export ORACLE_SID=chnldb1 /u01/oracle/product/db10gr2/bin/isqlplusctl stop /u01/oracle/product/db10gr2/bin/emctl stop dbconsole /u01/oracle/product/crs10gr2/bin/srvctl stop instance -d chnldb -i chnldb1 -o immediate " $INITDIR/init.crs stop 2
Page 105 of 134
Veritas Cluster
9.1
9.1.1
4
9.1.2 Storage Foundation for RAC

salesdb3 salesdb4 5.1
9.1.3 VRTSvsvc package

5.2.2 salesdb3 salesdb3@/opt/VRTS/install# installp -u VRTSvsvc
9.2 Cluster
9.2.1 Cluster
LLT dlpi salesdb3# strload -q stddev: yes stddev: yes spx: yes sc: yes stdmod: yes dlpi: yes xtiso: yes xtiso: yes xtiso: yes xtiso: yes xtiso: yes xtiso: yes timod: yes tirdwr: yes salesdb3# rsh salesdb4 strload -q stddev: yes stddev: yes spx: yes sc: yes stdmod: yes dlpi: yes xtiso: yes xtiso: yes xtiso: yes xtiso: yes xtiso: yes xtiso: yes timod: yes tirdwr: yes Page 106 of 134
SF/RAC salesdb3# vxinstall VxVM vxinstall INFO V-5-2-2541 Licensing information: System host ID: 0xacb0e76 Host type: powerpc Some licenses are already installed. Do you wish to review them [y,n,q,?] (default: y) Symantec License Manager vxlicrep utility version 3.02.16.0 Copyright (C) 1996-2006 Symantec Corporation. All rights reserved. Creating a report on all VERITAS products installed on this system -----------------***********************----------------License Key Product Name License Type OEM ID Demo End Date Editions Product = R4CE-P3UR-V4TW-277H-P9K4-97DC-PC6R-DBP = VERITAS Storage Foundation for Oracle RAC = DEMO_EXTENSION = 4095 = Sat Jan 5 00:00:00 2008 (30.4 days from now). = YES
Features := OPERATING_SYSTEM = AIX CPU Count = Not Restricted ODM#VERITAS Storage Foundation for Oracle = Enabled DBAC_EDITION = Enabled VXFS#VERITAS File System = Enabled VXFDD#VERITAS File System = Enabled VXCFS#VERITAS File System = Enabled VxVM#VERITAS Volume Manager = Enabled CVM_FULL#VERITAS Volume Manager = Enabled PGR#VERITAS Volume Manager = Enabled FMR_DGSJ#VERITAS Volume Manager = Enabled DBED_ORA_TOOLS#VERITAS Storage Foundation for Oracle = Enabled Mode#VERITAS Cluster Server = VCS_RAC Platform = AIX Version = 5.0 Tier#VERITAS Cluster Server = Unused PGR_TRAINING#VERITAS Volume Manager = Enabled File Change Log#VERITAS File System = Enabled Cross-platform Data Sharing#VERITAS File System = Enabled Extra-Big File Systems#VERITAS File System = Enabled Multi-Volume Support#VERITAS File System = Enabled Quality of Storage Service#VERITAS File System = Enabled Dynamic Lun Expansion#VERITAS Volume Manager = Enabled Hardware assisted copy#VERITAS Volume Manager = Enabled Cross-platform Data Sharing#VERITAS Volume Manager = Enabled Found_Edi_map#VERITAS Mapping Services = Enabled VXCKPT#VERITAS File System = Enabled -----------------***********************----------------Product Name License Type = VERITAS Volume Manager = DEMO
Page 107 of 134
Demo End Date Features := VVS_CONFIG PGR PGR_TRAINING Site Awareness
= = Enabled = Enabled = Enabled = Enabled
-----------------***********************----------------License Key Product Name License Type OEM ID Demo End Date Point Product Features := OPERATING_SYSTEM CPU Count DBED_ORA_TOOLS ODM = 88EE-PRP4-3P3C-M36O-4KGO-RJPP-NC8P-PPRC = VERITAS Storage Foundation for Oracle = DEMO_EXTENSION = 4095 = Sat Jan 5 00:00:00 2008 (30.4 days from now). = YES =0 = Not Restricted = Enabled = Enabled = VCS_0 = Unused
Mode#VERITAS Cluster Server Platform = AIX Version = 5.0 Tier#VERITAS Cluster Server
-----------------***********************----------------License Key Product Name License Type OEM ID Demo End Date Point Product Features := VXFS VXFDD VXCKPT VXCFS = 3GEV-UNPI-EBON-7S94-OR23-42PX-3P8R = VERITAS File System = DEMO_EXTENSION = 4095 = Sat Jan 5 00:00:00 2008 (30.4 days from now). = YES = Enabled = Enabled = Enabled = Enabled
CPU Count = Not Restricted Platform = AIX Version = 5.0 File Change Log = Enabled Cross-platform Data Sharing = Enabled Extra-Big File Systems = Enabled Multi-Volume Support = Enabled Quality of Storage Service = Enabled Maximum number of file systems = Not Restricted -----------------***********************-----------------
Page 108 of 134
License Key Product Name License Type OEM ID Demo End Date Point Product Features := FMR_DGSJ CVM_FULL
= B4EE-3PAO-2ZX8-W4OG-WBPC-6B6O-8CPP-PXP3-WR = VERITAS Volume Manager = DEMO_EXTENSION = 4095 = Sat Jan 5 00:00:00 2008 (30.4 days from now). = YES = Enabled = Enabled
VxVM = Enabled CPU Count = Not Restricted PGR = Enabled PGR_TRAINING = Enabled Platform = AIX Version = 5.0 Dynamic Lun Expansion = Enabled Hardware assisted copy = Enabled Cross-platform Data Sharing = Enabled Maximum number of volumes = Not Restricted -----------------***********************----------------License Key Product Name License Type OEM ID Demo End Date Point Product Features := Platform Version Tier Reserved Mode = P4EU-WH8B-2GSH-DO43-B3RC-C8YP-P = VERITAS Cluster Server = DEMO_EXTENSION = 4095 = Sat Jan 5 00:00:00 2008 (30.4 days from now). = YES = AIX = 5.0 = Unused =0 = VCS_RAC
-----------------***********************----------------License Key Product Name License Type OEM ID Demo End Date Point Product Features := Found_Edi_map = P8EE-JNRW-T8AP-DO4O-43B3-RNOC = VERITAS Mapping Services = DEMO_EXTENSION = 4095 = Sat Jan 5 00:00:00 2008 (30.4 days from now). = YES = Enabled
Do you wish to enter another license key [y,n,q,?] (default: n) n VxVM vxinstall INFO V-5-2-1807 Populating VxVM DMP device directories ....
Page 109 of 134
\ Do you want to setup a system wide default disk group? [y,n,q,?] (default: y) n Starting the relocation daemon, vxrelocd. Starting the site auto reattach daemon, vxsited. Starting the cache daemon, vxcached. Starting the diskgroup config backup daemon, vxconfigbackupd. Starting the dg monitoring deamon, vxvvrsecdgd. /usr/sbin/vxstart_vvr[21]: test: argument expected VxVM vxinstall INFO V-5-2-2291 The installation is successfully completed.
LLT salesdb1@/# vi /etc/llthosts ========================================================= ================= 0 salesdb1 1 salesdb2 2 salesdb3 3 salesdb4 ========================================================= ================= "/etc/llthosts" 5 lines, 40 characters salesdb1@/# rcp /etc/llthosts salesdb2:/etc salesdb1@/# rcp /etc/llthosts salesdb3:/etc salesdb1@/# rcp /etc/llthosts salesdb4:/etc LLT salesdb1@/# vi /etc/gabtab ========================================================= ================= /sbin/gabconfig -c -n4 ========================================================= ================= "/etc/gabtab" 1 line, 23 characters salesdb1@/# rcp /etc/gabtab salesdb2:/etc salesdb1@/# rcp /etc/gabtab salesdb3:/etc salesdb1@/# rcp /etc/gabtab salesdb4:/etc salesdb1@/# cat /etc/vcsmmtab ========================================================= ================= /sbin/vcsmmconfig -c & ========================================================= ================= salesdb1@/# rcp /etc/vcsmmtab salesdb3:/etc salesdb1@/# rcp /etc/vcsmmtab salesdb4:/etc salesdb1@/# cat /etc/vxfendg ========================================================= ================= vxfencoorddg
Page 110 of 134
========================================================= ================= salesdb1@/# rcp /etc/vxfendg salesdb3:/etc salesdb1@/# rcp /etc/vxfendg salesdb4:/etc salesdb1@/# ls /etc/vxf* /etc/vxfendg /etc/vxfenmode /etc/vxfentab /etc/vxfen.d: README vxfenmode_scsi3_dmp vxfenmode_disabled vxfenmode_scsi3_raw salesdb1@/# cat /etc/vxfenmode # # vxfen_mode determines in what mode VCS I/O Fencing should work. # # available options: # scsi3 - use scsi3 persistent reservation disks # customized - use script based customized fencing # disabled - run the driver but don't do any actual fencing # vxfen_mode=scsi3 # # scsi3_disk_policy determines the way in which I/O Fencing communicates with # the coordination disks. # # available options: # dmp - use dynamic multipathing # raw - connect to disks using the native interface # scsi3_disk_policy=dmp salesdb1@/# rcp /etc/vxfenmode salesdb3:/etc salesdb1@/# rcp /etc/vxfenmode salesdb4:/etc salesdb1@/# rcp /etc/llttab salesdb3:/etc salesdb1@/# rcp /etc/llttab salesdb4:/etc salesdb1@/# exit Connection closed. salesdb3# vi /etc/llttab ========================================================= ================= set-node salesdb3 set-cluster 1 link en2 /dev/dlpi/en:2 - ether - link en6 /dev/dlpi/en:6 - ether - ========================================================= ================= "/etc/llttab" 5 lines, 104 characters salesdb4 salesdb4@/# vi /etc/llttab ========================================================= ================= set-node salesdb4
Page 111 of 134
set-cluster 1 link en2 /dev/dlpi/en:2 - ether - link en6 /dev/dlpi/en:6 - ether - ========================================================= ================= "/etc/llttab" 5 lines, 104 characters salesdb3# /etc/init.d/llt.rc start Starting LLT... Starting LLT done. salesdb3# rsh salesdb4 /etc/init.d/llt.rc start Starting LLT... Starting LLT done. salesdb3# /etc/init.d/gab.rc start GAB: Starting GAB: Starting Done salesdb3# rsh salesdb4 /etc/init.d/gab.rc start GAB: Starting GAB: Starting Done salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00a membership 012 salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port a gen 83b00b jeopardy ; 3 salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 salesdb3# rsh salesdb1 gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00c membership 01 Port b gen 83b00c visible ; 23 Port d gen 83b00e membership 01 Port d gen 83b00e visible ; 23 Port f gen 83b011 membership 01 Port f gen 83b011 visible ; 23 Port h gen 83b00e membership 01 Port h gen 83b00e visible ; 23 Port o gen 83b00d membership 01 Port o gen 83b00d visible ; 23 Port v gen 83b025 membership 01 Port v gen 83b025 visible ; 23 Port w gen 83b029 membership 01 Port w gen 83b029 visible ; 23 salesdb3# /etc/init.d/vxfen.rc start Page 112 of 134
Starting vxfen.. Checking for /etc/vxfendg Starting vxfen.. Done salesdb3# rsh salesdb4 /etc/init.d/vxfen.rc start Starting vxfen.. Checking for /etc/vxfendg Starting vxfen.. Done salesdb3# /etc/init.d/vcsmm.rc start Starting VCSMM... Done configuring VCSMM. salesdb3# rsh salesdb4 /etc/init.d/vcsmm.rc start Starting VCSMM... Done configuring VCSMM. salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port o gen 83b00f membership 0123 salesdb3# rsh salesdb1 gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port d gen 83b00e membership 01 Port d gen 83b00e visible ; 23 Port f gen 83b011 membership 01 Port f gen 83b011 visible ; 23 Port h gen 83b00e membership 01 Port h gen 83b00e visible ; 23 Port o gen 83b00f membership 0123 Port v gen 83b025 membership 01 Port v gen 83b025 visible ; 23 Port w gen 83b029 membership 01 Port w gen 83b029 visible ; 23 salesdb3# /etc/init.d/lmx.rc start Starting LMX salesdb3# rsh salesdb4 /etc/init.d/lmx.rc start Starting LMX salesdb3# ls /etc/rc.d/rc2.d K45vxpbx_exchanged S34sfua_rep_mount S92gab K50isisd S45vxpbx_exchanged S94vxnm-vxnetd K50vxvail S48vxdbms3 S95vxvm-recover K75vxsmfd S50isisd S96init.crs K900.vxvm-discovery S50vxdbdctrl S96vradmind K96init.crs S50vxvail S96vxrsyncd K97vcs S70llt S97vxfen K98vcsmm S70vxatd S98rmxtab K98vxfen S71lmx S98vcsmm K99gab S75vxpal.StorageAgent S99odm K99lmx S75vxsmfd S99vcs K99vxatd S900.vxvm-discovery isisd salesdb3# /etc/rc.d/rc2.d/S99odm start Starting ODM... Device busy: /dev/odm salesdb3# rsh salesdb4 /etc/rc.d/rc2.d/S99odm start
Page 113 of 134
Starting ODM... Device busy: /dev/odm salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port o gen 83b00f membership 0123 salesdb3# /etc/rc.d/rc2.d/S99odm stop salesdb3# /etc/rc.d/rc2.d/S99odm start Starting ODM... salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port o gen 83b00f membership 0123 salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port d gen 83b00f membership 012 Port d gen 83b00f visible ; 3 Port o gen 83b00f membership 0123 salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port d gen 83b00f membership 012 Port d gen 83b00f visible ; 3 Port o gen 83b00f membership 0123 salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port d gen 83b00f membership 012 Port d gen 83b00f visible ; 3 Port o gen 83b00f membership 0123 salesdb3# rsh salesdb4 /etc/rc.d/rc2.d/S99odm stop salesdb3# rsh salesdb4 /etc/rc.d/rc2.d/S99odm start Starting ODM... salesdb3# gabconfig -a GAB Port Memberships ========================================================= ====== Port a gen 83b00b membership 0123 Port b gen 83b00e membership 0123 Port d gen 83b010 membership 0123 Port o gen 83b00f membership 0123 salesdb3#
Page 114 of 134
main.cf.HA salesdb1# cd /etc/*vcs/conf/config salesdb1# cp main.cf main.cf.2node salesdb1# haconf -makerw salesdb1# hagrp -display cvm #Group Attribute System Value cvm AdministratorGroups global cvm Administrators global cvm Authority global 0 cvm AutoFailOver global 0 cvm AutoRestart global 1 cvm AutoStart global 1 cvm AutoStartIfPartial global 1 cvm AutoStartList global salesdb1 salesdb2 cvm AutoStartPolicy global Order cvm ClusterFailOverPolicy global Manual cvm ClusterList global cvm Evacuate global 1 cvm ExtMonApp global cvm ExtMonArgs global cvm FailOverPolicy global Priority cvm FaultPropagation global 1 cvm Frozen global 0 cvm GroupOwner global cvm Guests global cvm Load global 0 cvm ManageFaults global ALL cvm ManualOps global 1 cvm OnlineRetryInterval global 0 cvm OnlineRetryLimit global 0 cvm OperatorGroups global cvm Operators global cvm Parallel global 1 cvm PreOnline global 0 cvm PreonlineTimeout global 300 cvm Prerequisites global cvm PrintTree global 1 cvm Priority global 0 cvm SourceFile global ./main.cf cvm SystemList global salesdb1 0 salesdb2 1 cvm SystemZones global cvm TFrozen global 0 cvm Tag global cvm TriggerEvent global 1 cvm TriggerResFault global 1 cvm TriggerResStateChange global 0 cvm TypeDependencies global cvm UserIntGlobal global 0 cvm UserStrGlobal global cvm AutoDisabled salesdb1 0 cvm AutoDisabled salesdb2 0 cvm Enabled salesdb1 1 cvm Enabled salesdb2 1 cvm IntentOnline salesdb1 1 cvm IntentOnline salesdb2 1 cvm NumRetries salesdb1 0 cvm NumRetries salesdb2 0 cvm PreOnlining salesdb1 0
Page 115 of 134
cvm PreOnlining salesdb2 0 cvm Probed salesdb1 1 cvm Probed salesdb2 1 cvm ProbesPending salesdb1 0 cvm ProbesPending salesdb2 0 cvm Restart salesdb1 0 cvm Restart salesdb2 0 cvm State salesdb1 |ONLINE| cvm State salesdb2 |ONLINE| cvm UserIntLocal salesdb1 0 cvm UserIntLocal salesdb2 0 cvm UserStrLocal salesdb1 cvm UserStrLocal salesdb2 cvm VCSi3Info salesdb1 cvm VCSi3Info salesdb2 salesdb1# hasys -add salesdb3 VCS NOTICE V-16-1-40128 System added; setting the localized attributes for resources in the ClusterService group recommended before proceeding. salesdb1# hasys -add salesdb4 VCS NOTICE V-16-1-40128 System added; setting the localized attributes for resources in the ClusterService group recommended before proceeding. salesdb1# hagrp -modify cvm SystemList -add salesdb3 2 salesdb1# hagrp -modify cvm SystemList -add salesdb4 3 salesdb1# hagrp -modify cvm AutoStartList -add salesdb3 salesdb1# hagrp -modify cvm AutoStartList -add salesdb4 salesdb1# hacf -verify . salesdb1# haconf -dump -makero salesdb1# pwd /etc/VRTSvcs/conf/config salesdb1# rcp main.cf types.cf salesdb3:/etc/VRTSvcs/conf/config salesdb1# rcp main.cf types.cf salesdb4:/etc/VRTSvcs/conf/config salesdb1# hastop -all -force salesdb1# hastatus -sum VCS ERROR V-16-1-10600 Cannot connect to VCS engine VCS WARNING V-16-1-11046 Local system not available salesdb1# vi main.cf include "types.cf" include "CFSTypes.cf" include "CVMTypes.cf" include "OracleTypes.cf" include "PrivNIC.cf" cluster rac1 ( UserNames = { admin = HMNfMHmJNiNNlVNhMK } ClusterAddress = "10.203.14.197" Administrators = { admin } UseFence = SCSI3 HacliUserLevel = COMMANDROOT ) system salesdb1 ( ) system salesdb2 ( ) system salesdb3 ( ) system salesdb4 (
Page 116 of 134
) group ClusterService ( SystemList = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 } Enabled @salesdb1 = 0 Enabled @salesdb2 = 0 AutoStart = 0 AutoStartList = { salesdb1, salesdb2, salesdb3, salesdb4 } OnlineRetryLimit = 3 OnlineRetryInterval = 120 ) IP webip ( Device = en8 Address = "10.203.13.222" NetMask = "255.255.255.0" ) NIC csgnic ( Device = en8 ) VRTSWebApp VCSweb ( Critical = 0 AppName = cmc InstallDir = "/opt/VRTSweb/VERITAS" TimeForOnline = 5 RestartLimit = 3 ) VCSweb requires webip webip requires csgnic // resource dependency tree // // group ClusterService // { // VRTSWebApp VCSweb // { // IP webip // { // NIC csgnic // } // } // } group cvm ( SystemList = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 } AutoFailOver = 0 Parallel = 1 AutoStartList = { salesdb1, salesdb2, salesdb3, salesdb4 } ) Application cssd ( Critical = 0 StartProgram = "/opt/VRTSvcs/ops/bin/cssd-online" StopProgram = "/opt/VRTSvcs/ops/bin/cssd-offline" CleanProgram = "/opt/VRTSvcs/ops/bin/cssd-clean" MonitorProgram = "/opt/VRTSvcs/ops/bin/cssd-monitor"
Page 117 of 134
) CFSMount cfs_ocrvote ( Critical = 0 MountPoint = "/oracrs" BlockDevice = "/dev/vx/dsk/oracrsdg/oracrsvol" ) CFSfsckd vxfsckd ( ActivationMode @salesdb1 = { ocrvotedg = sw, archdg = sw } ActivationMode @salesdb2 = { ocrvotedg = sw, archdg = sw } ) CVMCluster cvm_clus ( CVMClustName = rac1 CVMNodeId = { salesdb1 = 0, salesdb2 = 1 } CVMTransport = gab CVMTimeout = 200 ) CVMVolDg cvmvoldg_vote ( Critical = 0 CVMDiskGroup = ocrvotedg CVMVolume = { ocrvotevol } CVMActivation = sw ) CVMVxconfigd cvm_vxconfigd ( Critical = 0 CVMVxconfigdArgs = { syslog } ) cfs_ocrvote requires cvmvoldg_vote cfs_ocrvote requires vxfsckd cssd requires cfs_ocrvote cvm_clus requires cvm_vxconfigd
:q
salesdb1# cat main.cf include "types.cf" include "CFSTypes.cf" include "CVMTypes.cf" include "OracleTypes.cf" include "PrivNIC.cf" cluster rac1 ( UserNames = { admin = HMNfMHmJNiNNlVNhMK } ClusterAddress = "10.203.14.197" Administrators = { admin } UseFence = SCSI3 HacliUserLevel = COMMANDROOT ) system salesdb1 ( ) system salesdb2 ( ) system salesdb3 ( )
Page 118 of 134
system salesdb4 ( ) group ClusterService ( SystemList = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 } Enabled @salesdb1 = 0 Enabled @salesdb2 = 0 Enabled @salesdb3 = 0 Enabled @salesdb4 = 0 AutoStart = 0 AutoStartList = { salesdb1, salesdb2, salesdb3, salesdb4 } OnlineRetryLimit = 3 OnlineRetryInterval = 120 ) IP webip ( Device = en8 Address = "10.203.13.222" NetMask = "255.255.255.0" ) NIC csgnic ( Device = en8 ) VRTSWebApp VCSweb ( Critical = 0 AppName = cmc InstallDir = "/opt/VRTSweb/VERITAS" TimeForOnline = 5 RestartLimit = 3 ) VCSweb requires webip webip requires csgnic // resource dependency tree // // group ClusterService // { // VRTSWebApp VCSweb // { // IP webip // { // NIC csgnic // } // } // } group cvm ( SystemList = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 } AutoFailOver = 0 Parallel = 1 AutoStartList = { salesdb1, salesdb2, salesdb3, salesdb4 } ) Application cssd ( Critical = 0
Page 119 of 134
StartProgram = "/opt/VRTSvcs/ops/bin/cssd-online" StopProgram = "/opt/VRTSvcs/ops/bin/cssd-offline" CleanProgram = "/opt/VRTSvcs/ops/bin/cssd-clean" MonitorProgram = "/opt/VRTSvcs/ops/bin/cssd-monitor" ) CFSMount cfs_ocrvote ( Critical = 0 MountPoint = "/oracrs" BlockDevice = "/dev/vx/dsk/oracrsdg/oracrsvol" ) CFSfsckd vxfsckd ( ActivationMode ActivationMode ActivationMode ActivationMode ) @salesdb1 @salesdb2 @salesdb3 @salesdb4 = = = = { { { { ocrvotedg ocrvotedg ocrvotedg ocrvotedg = = = = sw, sw, sw, sw, archdg archdg archdg archdg = = = = sw sw sw sw } } } }
CVMCluster cvm_clus ( CVMClustName = rac1 CVMNodeId = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 } CVMTransport = gab CVMTimeout = 200 ) CVMVolDg cvmvoldg_vote ( Critical = 0 CVMDiskGroup = ocrvotedg CVMVolume = { ocrvotevol } CVMActivation = sw ) CVMVxconfigd cvm_vxconfigd ( Critical = 0 CVMVxconfigdArgs = { syslog } ) cfs_ocrvote requires cvmvoldg_vote cfs_ocrvote requires vxfsckd cssd requires cfs_ocrvote cvm_clus requires cvm_vxconfigd cvmvoldg_vote requires cvm_clus vxfsckd requires cvm_clus // resource dependency tree // // group cvm // { // Application cssd // { // CFSMount cfs_ocrvote // { // CVMVolDg cvmvoldg_vote // { // CVMCluster cvm_clus // { // CVMVxconfigd cvm_vxconfigd // } // }
Page 120 of 134
// // // // // // // // // //
} }
CFSfsckd vxfsckd { CVMCluster cvm_clus { CVMVxconfigd cvm_vxconfigd } } }
group ora_grp ( SystemList = { salesdb2 = 0, salesdb1 = 1, salesdb3 = 2, salesdb4 = 3 } Parallel = 1 AutoStartList = { salesdb2, salesdb1, salesdb3, salesdb4 } ) CFSMount cfs_oradata ( Critical = 0 MountPoint = "/lifedb/oradata" BlockDevice = "/dev/vx/dsk/oradg/oradata" ) CFSMount cfs_orasys ( Critical = 0 MountPoint = "/lifedb/orasys" BlockDevice = "/dev/vx/dsk/oradg/orasys" ) CFSMount cfsmount1 ( Critical = 0 MountPoint = "/archlog" BlockDevice = "/dev/vx/dsk/archdg/archlog" MountOpt @salesdb2 = "suid,rw" MountOpt @salesdb1 = "suid,rw" MountOpt @salesdb3 = "suid,rw" MountOpt @salesdb4 = "suid,rw" NodeList = { salesdb1, salesdb2, salesdb3, salesdb4 } Policy = { " salesdb1" } ) CVMVolDg cvmvoldg1 ( Critical = 0 CVMDiskGroup = archdg CVMActivation @salesdb2 CVMActivation @salesdb1 CVMActivation @salesdb3 CVMActivation @salesdb4 )
= = = =
sw sw sw sw
CVMVolDg cvmvoldg_ora ( Critical = 0 CVMDiskGroup = oradg CVMVolume = { orasys, oradata } CVMActivation = sw ) Oracle ora_lifedb ( Critical = 0 Sid @salesdb2 = lifedb2
Page 121 of 134
Sid @salesdb1 = lifedb1 Sid @salesdb3 = lifedb3 Sid @salesdb4 = lifedb4 Owner = oracle Home = "/u01/oracle/product/db10gr2" ) requires group cvm online local firm cfs_oradata requires cvmvoldg_ora cfs_orasys requires cvmvoldg_ora cfsmount1 requires cvmvoldg1 ora_lifedb requires cfs_oradata ora_lifedb requires cfs_orasys ora_lifedb requires cfsmount1 // resource dependency tree // // group ora_grp // { // Oracle ora_lifedb // { // CFSMount cfs_oradata // { // CVMVolDg cvmvoldg_ora // } // CFSMount cfs_orasys // { // CVMVolDg cvmvoldg_ora // } // CFSMount cfsmount1 // { // CVMVolDg cvmvoldg1 // } // } // } salesdb1# diff main.cf.2node main.cf 20a21,26 > system salesdb3 ( > ) > > system salesdb4 ( > ) > 22c28 < SystemList = { salesdb1 = 0, salesdb2 = 1 } --> SystemList = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 } 24a31,32 > Enabled @salesdb3 = 0 > Enabled @salesdb4 = 0 26c34 < AutoStartList = { salesdb1, salesdb2 } --> AutoStartList = { salesdb1, salesdb2, salesdb3, salesdb4 } 68c76 < SystemList = { salesdb1 = 0, salesdb2 = 1 } --> SystemList = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 }
Page 122 of 134
71c79 < AutoStartList = { salesdb1, salesdb2 } --> AutoStartList = { salesdb1, salesdb2, salesdb3, salesdb4 } 90a99,100 > ActivationMode @salesdb3 = { ocrvotedg = sw, archdg = sw } > ActivationMode @salesdb4 = { ocrvotedg = sw, archdg = sw } 95c105 < CVMNodeId = { salesdb1 = 0, salesdb2 = 1 } --> CVMNodeId = { salesdb1 = 0, salesdb2 = 1, salesdb3 = 2, salesdb4 = 3 } 148c158 < SystemList = { salesdb2 = 0, salesdb1 = 1 } --> SystemList = { salesdb2 = 0, salesdb1 = 1, salesdb3 = 2, salesdb4 = 3 } 150c160 < AutoStartList = { salesdb2, salesdb1 } --> AutoStartList = { salesdb2, salesdb1, salesdb3, salesdb4 } 171c181,183 < NodeList = { salesdb1, salesdb2 } --> MountOpt @salesdb3 = "suid,rw" > MountOpt @salesdb4 = "suid,rw" > NodeList = { salesdb1, salesdb2, salesdb3, salesdb4 } 179a192,193 > CVMActivation @salesdb3 = sw > CVMActivation @salesdb4 = sw 192a207,208 > Sid @salesdb3 = lifedb3 > Sid @salesdb4 = lifedb4 212,215d227 < // CFSMount cfsmount1 < // { < // CVMVolDg cvmvoldg1 < // } 222a235,238 > // } > // CFSMount cfsmount1 > // { > // CVMVolDg cvmvoldg1 salesdb1# hastart salesdb1# rcp main.cf types.cf salesdb4:/etc/VRTSvcs/conf/config salesdb1# rcp main.cf types.cf salesdb3:/etc/VRTSvcs/conf/config salesdb1# rcp main.cf types.cf salesdb2:/etc/VRTSvcs/conf/config salesdb1# hastatus attempting to connect....connected group resource system message --------------- -------------------- -------------------- -------------------salesdb1 RUNNING ClusterService salesdb1 OFFLINE cvm salesdb1 ONLINE ora_grp salesdb1 ONLINE ------------------------------------------------------------------------webip salesdb1 OFFLINE csgnic salesdb1 ONLINE VCSweb salesdb1 OFFLINE cssd salesdb1 ONLINE cfs_ocrvote salesdb1 ONLINE
Page 123 of 134
------------------------------------------------------------------------vxfsckd salesdb1 ONLINE cvm_clus salesdb1 ONLINE cvmvoldg_vote salesdb1 ONLINE cvm_vxconfigd salesdb1 ONLINE cfs_oradata salesdb1 ONLINE ------------------------------------------------------------------------cfs_orasys salesdb1 ONLINE cfsmount1 salesdb1 ONLINE cvmvoldg1 salesdb1 ONLINE cvmvoldg_ora salesdb1 ONLINE ora_lifedb salesdb1 ONLINE ^Csalesdb1# salesdb1# rlogin salesdb2 salesdb2# hastart salesdb2# /etc/vx/bin/vxclustadm -m vcs -t gab reinit salesdb2# rlogin salesdb3 salesdb3# hastart salesdb4# /etc/vx/bin/vxclustadm -m vcs -t gab reinit salesdb3# rlogin salesdb4 salesdb4# hastart salesdb4# /etc/vx/bin/vxclustadm -m vcs -t gab reinit
Page 124 of 134
10
Cluster Oracle RAC

10.1 Oracle
10.1.1 Oracle
dbca RAC
10.1.2 Oracle
4 Oracle root salesdb1 copy salesdb1# rcp /opt/ORCL/cluster/lib/* salesdb3:opt/ORCL/cluster/lib/ salesdb1# rcp /opt/ORCL/cluster/lib32/ salesdb3:/pt/ORCL/cluster/lib32/ salesdb4 Oracle RAC
10.2 Oracle CRS push

10.2.1 Oracle crs push
Oracle RAC salesdb1 oracle $ cd $ORA_CRS_HOME $ cd oui/bin $ ./addNode.sh oracle next
Page 125 of 134
hosts private node name virtual host name next
install crs salesdb4
Page 126 of 134
salesdb4 root orainstRoot.sh salesdb1 root rootaddnode.sh salesdb4 root.sh root.sh Network interface vipca ok push root salesdb1 # cd /u01/oracle/product/crs10gr2/bin/racgons add_config salesdb4:6200 6200 port $ORA_CRS_HOME/opmn/conf/ons.config remoteport $ORA_CRS_HOME/bin/crs_stat t crs resource online
10.2.2 Oracle database push

Oracle RAC salesdb1 oracle $ cd $ORACLE _HOME $ cd oui/bin $ ./addNode.sh oracle next
Page 127 of 134
next install database salesdb4
salesdb4 root root.sh ok
Page 128 of 134
10.2.3 Oracle RAC

salesdb4 oracle $ netca
cluster configuration
Page 129 of 134
listener configuration
reconfigure next finish listener salesdb1 oracle $ dbca
Page 130 of 134
Oracle real application cluster database
instance management
Page 131 of 134
add instance
sys oracleNext
Page 132 of 134
oracle next
next
Page 133 of 134
undo tablespace redo log finish ok
10.2.3 Veritas Cluster

root hagui resource hagui <>
Page 134 of 134

Sfrac5 Install

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sfrac5 Install

Uploaded by

Copyright:

Available Formats

Storage Foundation for Oracle RAC

Document Title Subject Version Date Author Filename Comments

Storage Foundation for RAC

2709.1 2008/8/22 Symantec Sfrac5-cpic-install-report_20080823.doc

5 Veritas SFRAC ............................................................. 14

6 Oracle RAC ........................................................... 46

7 Oracle RAC ................................................................. 81

8 Veritas Cluster ............................................................ 94

9 Veritas Cluster .............................................................106

10 Cluster Oracle RAC ...................................... 125

Aix 5300-05 csp

1salesdb1 10.203.100.80/255.255.255.0 10.203.38.5/255.255.255.0 en9(en0, en4 ) en6(en1, en5 )

VLAN 4 ,2 oracle crs 2 symantec rac

fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True sw_fc_class 3

Oracle RAC DB 10g Environment

export NLS_LANG='SIMPLIFIED CHINESE_CHINA.ZHS16GBK'

port LD_LIBRARY_PATH=$ORA_CRS_HOME/lib:$ORACLE_HOME/lib32:$ORACLE_HOME/lib: RACLE_HOME/ctx/lib:$ORACLE_HOME/RDBMS/lib:/lib:/usr/lib

port PATH=$ORA_CRS_HOME/bin:$ORACLE_HOME/OPatch:$ORACLE_HOME/bin:$ORACLE_HOME/jre/1.4.2/bin: ATH

port CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib: RACLE_HOME/network/jlib

port NLS_DATE_FORMAT='yyyy-mm-dd hh24:mi:ss'

port NLS_TIMESTAMP_TZ_FORMAT='yyyy-mm-dd hh24:mi:ss.ff'

acle BASE /u01/oracle

acle CRS /u01/oracle/product/crs10gr2

acle Database /u01/oracle/product/db10gr2

acle chnldb1 1 chnldb1 2 chnldb2

5.1.2 Storage Foundation for RAC 5.0 for aix

VRTSspt SYMClma VRTSveki VRTSllt.rte VRTSgab.rte VRTSvxfen.rte VRTSvcs.rte

VRTSacclib.rte Veritas ACC Library

Veritas Enterprise Administrator

VRTSvail.VRTSvail Veritas Array Integration Layer

VRTSgapms.VRTSgapms Veritas Generic Array Plugin

5.1.3 Storage Foundation 5.0MP1 for AIX

Veritas Enterprise Administrator

VRTSvxfen.rte VRTSvcs.rte VRTSvcsag.rte VRTScssim.rte VRTScscm.rte VRTScmcs.rte VRTScmccc.rte VRTSfspro

Veritas Enterprise Administrator

Veritas Oracle Disk Manager

5.1.4 VRTSvsvc package

5.1.5 Storage Foudation

Press [Return] to continue:

5.1.6 salesdb2 volume manager

VxVM vxinstall INFO V-5-2-2291 The installation is successfully completed.

5.2.2 salesdb1 salesdb2

5.2.3 salesdb1 cluster

to the cluster and the individual nodes.

5.2.5 salesdb1 vxfen

XP12K0_0 XP12K0_1 XP12K0_3

hdisk74 hdisk73 hdisk72

hdisk149 hdisk148 hdisk147

XP12K0_2 XP12K0_0 XP12K0_1

hdisk74 hdisk73 hdisk72

hdisk149 hdisk148 hdisk147

fencing salesdb1@/# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n VERITAS vxfentsthdw version 5.0 AIX

5.2.6 salesdb1 salesdb2 cluster

node 1 in state 8 (running)

512 512 512 512 512 512

24 chnldbdatavol 12 chnldbidxvol Chnldblogdg 20 chnldbfbvol 30GB

disk group Cluster Group

volume stripe volume:

volume Columns Stripe unit size

1) Oracle 9iR2 2) Oracle 10gR1 3) Oracle 10gR2 Select an option [1-3,q] 3

Oracle overall result passednext

/etc/hosts private node name _-next

ip private public edit next: en6

1) Check systems for SFRAC INSTRUCTIONS ONLY Page 61 of 134