Professional Documents
Culture Documents
<AL-ZAMIL INDUSTRY>
INSTALLATION ORACLE E-BUSINESSMULTINODE ON RAC
WITH PCP CONFIGURATION
Revision History
Description of Reviewer
Revision# Date Author Name
Revision Name
CONTENTS
1. Introduction .................................................................................................................................. 5
2. Environment Specification ......................................................................................................... 6
3. Installing Oracle E-Business (Database Tier) ............................................................................ 7
4. Installing Oracle E-Business (Apps Tier) .................................................................................. 12
5. Appling NLS Arabic Patch ....................................................................................................... 15
6. Upgrade Application from 12.1.1 to 12.3.1: .......................................................................... 16
7. Upgrade Database (11.1.0.7 to 11.2.0.3) .............................................................................. 17
8. Apply hrglobalization path as the following: ........................................................................ 30
9- Configure ASM Disks: ................................................................................................................ 31
10- Installing Oracle RAC 11gR2: ............................................................................................... 33
11- Create ASM Disk Group: ...................................................................................................... 56
12- Migrate Database to RAC................................................................................................... 60
13- Enable AutoConfig on Applications Database Tier ........................................................ 62
14- Establish Applications Environment for Oracle RAC ........................................................ 64
15- Set Up Load Balancing and PCP ........................................................................................ 66
16- Set Up Transaction Managers ............................................................................................. 67
17- Enable HTTP LoadBalancing and OC4J Cluster ............................................................... 68
18- Configure OCFS..................................................................................................................... 69
19- Table of Reference ............................................................................................................... 74
1. Introduction
This document is to highlight the installation and configuration steps that were performed
for the Oracle E-Bussiness "R12" Multinode on RAC DB 11gR2 with PCP implementations that
were performed at AL-Zamil. The document will go through the installation steps,
configuration and setup performed.
2. Environment Specification
2.1Hardware Description
2.1.1 Database Servers
Test Server Production Server
2* BL460c G7each with:2P/12Cores(Intel 2* BL620c G7 each with:2P/16Cores(Intel
Xeon X5675 (3.06GHz/6- Xeon X7560 (2.26GHz/8-
core/12MB/95W)32GB RAM (4x8GB 2Rx4 core/24MB/130W) 128GB RAM (16x8GB
PC3-10600R-9 Kit)2x 146GB 6G SAS 15K 2Rx4 PC3-10600R-9 Kit)2x 146GB 6G SAS
2.5in DP ENT HDD 15K 2.5in DP ENT HDD
RAM - 32 GB RAM - 16 GB
SWAP - 32 GB SWAP - 16 GB
OS - Oracle Enterprise Linux 5U5 OS - Oracle Enterprise Linux 5U5
RAM - 32 GB RAM - 16 GB
SWAP - 32 GB SWAP - 16 GB
OS - Oracle Enterprise Linux 5U5 OS - Oracle Enterprise Linux 5U5
Choose fresh database, Database SID, Hostname, Domain name, Database user with
which group and database base directory.
Note: This base directory will be a temp directory as we will migrate to RAC 11gR2.
You can choose suite licensing to get most of product licensed or you can choose
component licensing and choose your product individually as you see in next screen.
This screen means that all info you provide during installation process and all prerequisite
are correct and you can press next to proceed with Database installation on first Node.
Note: After finishing database installation transferconfiguration file that created by this
installation to the Application node and transfer stage software.
Point to the configuration file that has been created on the Database installation Phase.
Then Click Next
1- Apply Patch 8576725 to fix performance issue while applying NLS Arabic patch.
2- System administrator > OAM > License Manager > Language > then mark
3-Arabic language and submit
4-From AD Administration, run Maintain Multi-lingual Tables (AD
5-Administration Main Menu > Maintain Applications Database Entities Menu).
6-Apply Arabic Patch.
Open new terminal as user "root" and run the scripts one by one then return to this window
and click "OK" to continue
Oracle_Home set already to the new Oracle Database software that you installed in
previous step, press Next.....
This Screen Appears after all prerequisite Checked Successfully, Press Install ..
Project Management Plan Template Page 24 of 74
Document Identification
Log in to the system as the owner of the environment of the database being
upgraded.
Note: The Pre-Upgrade Information Tool must be copied to and must be run from
the environment of the database being upgraded
Start SQL*Plus.
Connect to the database instance as a user with SYSDBA privileges.
Set the system to spool results to a log file for later analysis:
SQL> SPOOL upgrade_info.log
Run the Pre-Upgrade Information Tool:
SQL> @$11gR2_ORACLE_HOME/rdbms/admin/utlu112i.sql
Turn off the spooling of script results to the log file:
SQL> SPOOL OFF
Check the output of the Pre-Upgrade Information Tool in upgrade_info.log.
Check the upgrade log file then fix the issues if founded then rerun utlu112i.sql
SQL> @catupgrd.sql
Note:The catupgrd.sql script determines which upgrade scripts must be run, runs
them, and then shuts down the database. You must run the script in the Oracle
Database 11g Release 2 (11.2) environment.
The upgrade script creates and alters certain data dictionary tables. It also upgrades
or installs the following database components in the new Oracle Database 11g
Release 2 (11.2) database:
SQLexec DBMS_DST.BEGIN_UPGRADE('14');
25-run post script :
DECLARE
acl_path VARCHAR2(4000);
BEGIN
SELECT acl INTO acl_path FROM dba_network_acls
WHERE host = '' AND lower_port IS NULL AND upper_port IS NULL;
IF DBMS_NETWORK_ACL_ADMIN.CHECK_PRIVILEGE(acl_path,
'user_name','connect') IS NULL THEN
DBMS_NETWORK_ACL_ADMIN.ADD_PRIVILEGE(acl_path,
'user_name', TRUE, 'connect');
END IF;
EXCEPTION
WHEN no_data_found THEN
DBMS_NETWORK_ACL_ADMIN.CREATE_ACL('ACL_name.xml',
'ACL description', 'user_name', TRUE, 'connect');
DBMS_NETWORK_ACL_ADMIN.ASSIGN_ACL('ACL_name.xml','host_name');
END;
/
NoteDon`t forget to replace user_name with apps and host_name with server hostname,
then commit
Execute $ORACLE_HOME/ctx/admin/ctxf102.sql as database user SYS, SYSTEM, or
CTXSYS
setting the compataible parameter :
SQL>shutdown immediate
Edit parameter COMPATIBLE =11.2.0
SQL>startup
Run $ORACLE_HOME/rdbms/admin/utlrp.sql to compile all invalid objects.
Note:Multi Org enabled be default in Oracle E-Bussines R12 so no need to enable it.
But to make you more sure do the following:
Run AD Configuration report to check that multi org enabled Use the following commands.
The report output file is written to adutconf.lst in the current working directory.
bash$ cd $APPL_TOP/admin/<SID>/out
bash$ sqlplus <APPS schema username>/<APPS schema password> \
@$AD_TOP/sql/adutconf.sql
Project Management Plan Template Page 29 of 74
Document Identification
Ref#HrGlobal (paymxearnexrules.ldt), Errors: ORA-1400: cannot insert NULL into ... [ID
1336293.1]
Latest Oracle HRMS Legislative Data Patch Available (HR Global / hrglobal)
[ID 145837.1]
Instructions for Running DataInstall/hrglobal.drv on R12 [ID 414434.1]
As you see in previous script ocrv1 represent to disk size 1 G and asmeb1 represent to Disk
size 200 G ... take care when you doing this steps.
2- Configure ASMLib using the following command:
bash# oracleasm configure -i
3- Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
Loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
7- Check that all disks appears on all nodes by doing the following command:
[root@dmmzidbts1 ~]# oracleasm listdisks
ASMEB1
ASMEB2
ASMEB3
ASMEB4
ASMEB5
OCRV1
OCRV2
OCRV3
Choose the place of OCR &Vote Disks and we are already configured OCR Disks before
with ASM so we will choose Oracle ASM.
In this screen write your Disk Group Name "OCRV" and make Redundancy as Normal then
choose only 3 Disks that you create them before for OCR and VOTE.
Oracle preforms prerequisite check if you found error in ASM like the following:
Verification result of failed node: racnode1
Note:ASM Errors will be found: safely ignore the below error as oracle ID 1210863.1
List of errors:
- PRVF-5150: Path ORCL:DISKCLU is not a valid path on all nodes
Operation Failed on Nodes: [racnode1]
List of errors:
- Could not get the type of storage
- Cause: Cause of Problem Not Available
- Action: User Action Not AvailablePlease Follow ref
In this screen follow the instruction in the box "Execute Configuratio Scripts"
By run first script on first node and then on the second node ..Second script run on first node
then on second node as you will see on next screen
Make Sure that root.sh completed successfully before run it on second node.
kmk
Press Create
1- Apply Application Patch #9926448 to fix known issue with FND_FS/SM alias generation
with SCAN enabled.
2- As the oracle user, navigate to the 11gR2 directory $11gR2_ORACLE_HOME/assistants/
rconfig/sampleXMLs, and open the sample file ConvertToRAC_AdminManaged.xml
using a text editor such as vi. This XML sample file contains comment lines that provide
instructions on how to edit the file to suit your site's specific needs.
3- Make a copy of the sample ConvertToRAC.xml file, and modify the foolwoing
parameter:
<n:Convert verify="ONLY">
Note: Convert verify="ONLY" rconfig only performs prerequisites check; it does not
start conversion after completing prerequisite checks. --------------------------
-------------------------
<!--Specify current OracleHome of non-rac database for SourceDBHome --
><n:SourceDBHome>/u02/app/oracle/product/11.2.0/dbhome_1</n:SourceDBHo
me>
<!--Specify OracleHome where the rac database should be configured. It can be
same as SourceDBHome -->
<n:TargetDBHome>/u01/app/oracle/product/11.2.0/db_1</n:TargetDBHome>
--------------------------------------------------------------------------
<!--Specify SID of non-rac database and credential. User with sysdba role is required
to perform conversion -->
<n:SourceDBInfo SID="ZITDB">
<n:Credentials>
<n:User>sys</n:User>
<n:Password>******</n:Password>
<n:Role>sysdba</n:Role>
-----------------------------------------------------------------------------
<n:NodeList>
<n:Node name="dmmzidbts1"/>
<n:Node name="dmmzidbts2"/>
</n:NodeList>
------------------------------------------------------------------------------
<n:InstancePrefix>ZITDB</n:InstancePrefix>
--------------------------------------------------------------------------------
<n:SharedStorage type="ASM">
---------------------------------------------------------------------------------
<n:TargetDatabaseArea>+EBDATA</n:TargetDatabaseArea>
----------------------------------------------------------------------------------
<n:TargetFlashRecoveryArea></n:TargetFlashRecoveryArea>
4- Run the following tool to convert single instance into Rac intsnace:
./rconfig<path to rconfig XML file created in Step 2 of this list>
5- Shutdown E-Bussiness Listener and create New listener on RAC using netca with
same port of E-Bussiness Listener.
6- Change the following parameter in script at step 2 and run rconfig to convert to
RAC:
<n:Convert verify="YES">
1- Ensure that you have applied the Oracle Applications patches listed in the prerequisites
section above.
3- Copy (e.g. via ftp) the appsutil.zip file to the database tier in the 11gR2_ORACLE_HOME.
4- Unzip the appsutil.zip file to create the appsutil directory in the 11gR2_ORACLE_HOME.
9- As the APPS user, run the following command on the primary node to de-register the
current configuration: SQL>exec fnd_conc_clone.clean_setup;
10- Set local_listener parameter to <SID>_local and verify that the instances are registered
in EBS database listener.
SQL> alter system set local_listener='<SID>_local' sid='<SID>';
s_db_listener to LISTENER_EBS
s_virtual_hostname to Your virtual hostname
s_clusterInterConnects to Your Private hostname
if you are going to use scane change the following parameter also:
<scanNameoa_var="s_scan_name">scan-zitst.ziic.net</scanName>
<scanPortoa_var="s_scan_port">1570</scanPort>
<scanUpdateFlagoa_var="s_update_scan">TRUE</scanUpdateFlag>
s_sqlnet_expire_time to 1
Note:
First time you will run autconfig on first node it will not generate the correct tnsnames
so ignore and run autoconfig on second node using the same above steps then
return to the first node and check all above parameter then rerun adconfig.pl script
second time will generate correct tnsnames entries
14- Now shutdown RAC listener that created before as you are going to use E-Business
Listener.
15- Stop all Database and listener processes then startup E-Bus listener and Database.
3- Edit SID=<Instance 1> and PORT=<New listener port > in $TNS_ADMIN/tnsnames.ora file,
to set up connection one of the instances in the Oracle RAC environment.
4- Confirm you are able to connect to one of the instances in the Oracle RAC
environment.
5- Edit the context variable jdbc_url, adding the instance name to the connect_data
parameter.
9- Verify the tnsnames.ora and listener.ora files. Copies of both are located in the
$INST_TOP/ora/10.1.2/network/admin directory and
$INST_TOP/ora/10.1.3/network/admin directory. In these files, ensure that the correct TNS
aliases have been generated for load balance and failover, and that all the aliases are
defined using the virtual hostnames.
10- Verify the dbc file located at $FND_SECURE. Ensure that the parameter APPS_JDBC_URL
is configured with all instances in the environment, and that load_balance is set to YES.
11- Open application and test that everything working fine (http,forms,report and
concurrent request)
12- Shutdown Application Process now and run preclone script then transfer application file
to second node and run adcfgclone on the second node.
6- Ensure that value of the profile option "Application Database ID" is set to dbc file
name generated in $FND_SECURE.
7- Log in to Oracle E-Business Suite Release 12 using the SYSADMIN account, and
choose the System Administrator Responsibility. Navigate to Install > Nodes
screen, and ensure that each node in the cluster is registered.
8- Verify that the Internal Monitor for each node is defined properly, with correct
primary and secondary node specification, and work shift details. For example,
Internal Monitor: Host2 must have primary node as host2 and secondary node as
host3. Also ensure that the Internal Monitor manager is activated: this can be
done from Concurrent > Manager > Administrator.
10- Navigate to system administrator > OAM > Site Map > Parallel Concurrent
Programming Setup and then add secondary node to all concurrent program
Project Management Plan Template Page 66 of 74
Document Identification
SQL>shutdown immediate;
_lm_global_posts=TRUE
_immediate_commit_propagation=TRUE
6-Log on to Oracle E-Business Suite Release 12 using the SYSADMIN account, and choose
the System Administrator responsibility. Navigate to Profile > System, change the profile
option Concurrent: TM Transport Type' to QUEUE', and verify that the transaction manager
works across the Oracle RAC instance.
Navigate to Concurrent > Manager > Define screen, and set up the primary and
secondary node names for transaction managers.
Restart the concurrent managers.
If any of the transaction managers are in deactivated status, activate them from
Concurrent > Manager > Administrator.
Connect as sysdba and make the following:
$FND_TOP/bin/txkrun.pl -script=SetAdvCfg \
-appsuser=apps -appspass=apps -enable=HTTPLoadBal \
-t_session_persistent=[y|n] \
-t_dns=[y|n] \
-s_webentryhost=<The HTTP load balancer host name> \
-s_webentrydomain=<The HTTP load balancer domain name> \
-s_webentryurlprotocol=<The HTTP load balancer's protocol> \
-s_active_webport=<The HTTP load-balancer's external port>
* Optional. If not specified, s_active_webport will be set to the value of the web port
(s_webport).
Note: s_active_webport : is the port between user and load balancer, We use port 80 in
test server
$ $FND_TOP/bin/txkrun.pl -script=SetAdvCfg \
-appsuser=apps -appspass=apps -enable=OC4JLdBal \
-s_oc4j_cluster_nodes=<hostName1:onsPort1,hostName2:onsPort2>
2. Download the appropriate version of the OCFS2 kernel module and tools :
ocfs2console
ocfs2-tools
ocfs2
4. Start the OCFS2 Console by issuing the following command as the root user.
# ocfs2console
The image below shows the console window.
5.Select the "Cluster > Configure Nodes..." menu option and ignore the subsequent
information message by clicking the "Close" button.
Enter the server name and IP address for the first node, then click the "OK" button.
6.Add the second node, then click the "Apply" button followed by the "Close" button on
the "Node Configuration" screen
7. Edit the "/etc/init.d/o2cb" script, removing the following entries "Take Backup from
script before edit":
### BEGIN INIT INFO
## Provides: o2cb
## Required-Start: $network
## Should-Start:
## Required-Stop:
## Default-Start: 2 3 5
## Default-Stop:
## Description: Load O2CB cluster services at system boot.
### END INIT INFO
The partition is displayed on the console screen once the format operation is complete.
Make sure the partition is highlighted, then click the mount button on the toolbar. Enter
the "/u02" mount point and any additional options, then click the "OK" button.
The partition is now listed along with its mount point in the console screen.
13.Only the mount operation is necessary on all other nodes, not the format operation.
To automount the filesystem, append the following entry to the "/etc/fstab" file on each
node "Take Backup Before append":
Number Reference
1 Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.1.1)
for Linux x86-64 [ID 761566.1]
2 Interoperability Notes EBS R12 with Database 11gR2 [ID 1058763.1]
3 Oracle Database Upgrade Guide 11g Release 2 (11.2) Chapters 3 and 4
4 Latest Oracle HRMS Legislative Data Patch Available (HR Global /
hrglobal) [ID 145837.1]
5 Instructions for Running DataInstall/hrglobal.drv on R12 [ID 414434.1]
6 Using Oracle 11g Release 2 Real Application Clusters with Oracle E-
Business Suite Release 12 [ID 823587.1]
7 Oracle HRMS Product Family - Release 11i and Release 12 Information [ID
135266.1]
8 JSP Pages Hanging in R12 After Removing Cached Class Files in _pages [ID
433386.1]
10 Device Checks for ASM Fails with PRVF-5150: Path ORCL: is not a valid path
[ID 1210863.1]
14
Running Configuration Wizards from the Command Line in Oracle E-
Business Suite Release 12 [ID 1364368.1]