Professional Documents
Culture Documents
1 Clustering
An Oracle JD Edwards EnterpriseOne Best Practices Paper July 2008
PURPOSE STATEMENT This document is intended to help you understand the features, functionality, and option available for EnterpriseOne HTML Server clustering in Oracle Application Server 10.1.3.1.0 conjunction with 8.97 Server Manager. There are many ways to configure multiple types of clustering and your results may differ depending on your custom setup. This document is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle. Updates and enhancements are provided in accordance with Oracles Technical Support Policies at: www.oracle.com/support/collateral/oracle-technical-support-policies.pdf
Page i
Clustering ___________________________________________________________________ 2
OAS 10.1.3.1.0 Cluster and 8.97 Server Manager ________________________________________ 2 Multiple JVMs ____________________________________________________________________ 2 Vertical Clustering _________________________________________________________________ 2 Horizontal Clustering _______________________________________________________________ 3 Application Server level clustering ____________________________________________________ 3 Application level clustering __________________________________________________________ 3
Page ii
Testing Second OC4J Install _________________________________________________________ 21 Configuring Vertical Cluster_________________________________________________________ 22 Configuring HTTP Server for load balancing ____________________________________________ 27 Testing the vertical cluster and request load balancing __________________________________ 28
Page iii
Overview
There are multiple types of clustering available in Oracle Application Server (OAS) 10.1.3.1. We will describe types of clustering that can be utilized in conjunction with E1 HTML Server. This paper will focus mainly on how to setup, interact, monitor, and manage vertical clustering, horizontal clustering and multiple JVM configurations in OAS 10.1.3.1 with 8.97 Server Manager.
suite and Oracle Containers for J2EE (OC4J). There are no Web Cache, Infrastructure, and Portal components in OAS 10.1.3.1.0. If you want these additional components to work with 10.1.3.1.0 you will have to obtain them from 10.1.2.0.2.
Clustering
Oracle defines clustering as a set of processes running on single or multiple computers that share the same workload. To simplify, we can classify this into two major types of clustering, vertical and horizontal (described below). OAS 10.1.3.1.0 provides multiple methods/mechanism to create these types of clustering, such as Application Server level clustering, and OC4J level clustering. In addition OAS 10.1.3.1.01 also provides option call Multiple JVMs, simple method for increasing performance and level of fault tolerance for deployed application on hardware with addition resources.
Multiple JVMs
By default, OC4J instance executes on and uses single Java Virtual Machine (JVM). However, in OAS, you can configure an OC4J instance to run on multiple JVMs. On a physical machine with additional hardware resources this will provide increase performance, by having single OC4J instance run on multiple processes, and provide a level of fault tolerance for deploy application via session information in HTTP. Any OC4J instance, except for OC4J instance that is hosting an active Application Server Control (ascontrol application), can be configured to use multiple JVMs whether or not it is part of a cluster or defined as a part of a group.
Vertical Clustering
In typical vertical clustering, multiple OC4J containers that are identically configured are grouped together in a single OAS 10.1.3.1.0 installation, single Application Server (single node), on one physical machine. This with or without using multiple JVM option will give user option to better optimize hardware resources.
Page 2
It is possible to have multiple installations of OAS 10.1.3.1.0, multiple Application Server (multiple nodes), on one physical machine and cluster can be achieved either at Application Server (node) level or at Application level. Step-by-step instruction on how to set this up will be discuss late in this paper.
Horizontal Clustering
In horizontal cluster, installation of OAS 10.1.3.1.0 is done on multiple physical machines. With multiple nodes in hand we can cluster these nodes at Application Server level and/or at Application level. We will first configure Dynamic Node Discovery so that ONS on each node can communicate with each other then we will create and/or add our horizontal instance (E1HTML Server application) to a cluster. One distinct advantage of horizontal cluster is better failover protection along with a better High Availability solution. In this scenario, even if hardware failures occur on one physical machine and that machine is unavailable other node residing on other physical machine will be able to accept and process user requests. Within vertical clustering, although you might be optimizing hardware resources, if that physical machine goes down it is single point of failure.
releases, where any configuration change made to individual instance get automatically propagated all other OC4J instances in that cluster, in OAS 10.1.3.1.0 changes can be made to a group and all of its OC4J instances via Group page in Application Server Control Console or to a single OC4J instance in a group. This is useful if you need to change JVM heapsize to match different hardware configuration or need to collect additional debug information by enabling verbose for garbage collection for specific instance in cluster.
Prerequisites
Following prerequisites must be completed in its entirety prior to proceeding with rest of this red paper. 1. Complete installation of JD Edwards EnterpriseOne Tools release 8.97 with latest update. This includes Deployment Server and Enterprise Server. a. Verify installation is functional by completing PORTTEST. NOTE: Please refer to 8.97 Tools Release installation guide for more detail on Porttest.(note: do you need to define or point to doc on Porttest?) 2. Complete installation of 8.97 Server Manager (To maximize usage, Oracle recommends that Server Manager be installed on EnterpriseOne Deployment Server). a. Make sure Server Manger is running and functional by logging into Server Management Console.
Page 4
3. Complete the installation of Server Management Agent on the machine you will be using for OAS install. a. Verify that Server Manager Agent is up and running and that it can communicate with Server Management Console. 4. Complete upload of EnterpriseOne 8.97 HTML code that matches Enterprise Server and Deployment Server to Server Manager via Server Management Console. 5. Complete JDBC driver installation via Server Management Console so that it is available during EnterpriseOne HTML server install. 6. Complete installation of Oracle Application Server 10g Release3 (OAS 10.1.3.1.0). a. Test to make sure that you have functioning OAS install Please refer to JD Edwards EnterpriseOne Documentation for detail step-by-step installation instruction for Server Manager, Deployment Server, Enterprise Server and OAS 10.1.3.1.0 install for JD Edwards EnterpriseOne software.
Architectural Diagram
Following naming convention will be used.
Page 5
Standalone Installation
Begin by logging into Server Manager Management Console installed on the Deployment Server. If Server Manger (SM) Agent is installed correctly on the OAS machine and is able to communicate to SM Management Console located on Deployment Server via JMX port it will start auto registration process of managed home.
1. Click on the link for OAS Server under Managed Home Location (DENGSSWIN01.mlab.jdedwards.com).
2. Create new managed instance by clicking Create New Managed Instance box.
Page 6
4. Complete the instance properties fields with correct information that correspond to your installation. For this installation we will use OAS10131 as our instance name.
Page 7
6. Click Create Instance button on Finish page to initiate creation of managed instance. Once managed instance have been created successfully you will see page similar to one shown below.
Page 8
1.
From Oracle Application Server OAS1 page locate J2EE Containers section enter OC4J container name (we are using EA_JS_81), select Group from drop down box if you have more than one group. NOTE: group can only be created from OAS ASC and must be created prior to creating OC4J container from SM) and click Create button.
2. Once OC4J creation process has completed, from the same page you will now see newly created OC4J container under J2EE Containers section. If State shows Undetermined refresh the screen.
Page 9
1. Click on the link for Managed Software under the Quick Link menu.
2. This brings up list of all managed software that has been uploaded to SM. NOTE: If you have not already uploaded E1HTML Server code to SM please review 8.97 SM guide for step-by-step instruction on how to complete this task.
3. Select the software to distribute (EnterpriseOne HTML Server Version 8.97.x.x) and click Distribute button.
Page 10
4. Select the server (DENGSSWIN01) from the list and click Distribute Software button to start the distribution of EnterpriseOne HTML Server code to OAS 10.1.3.1 Server (DENGSSWIN01). NOTE: In our case, software was previously distributed so you do have option to select the machine.
5. Behind the scene, the process takes E1HTML Server par file from SM and copy it to component directory under SM Agent home directory (ours is in C:\JDE_HOME\component)
Page 11
6. Once EnterpriseOne HTML Server software have been distributed go back to Managed Homes and Managed Instance page by clicking on Management Dashboard breadcrumb and clicking link for OAS server (DENGSSWIN01.mlab.jdedwards.com). From there you can verify whether software have been distributed properly under Managed Software Components section.
Page 12
1. From previous screen where we verified that EnterpriseOne HTML Server code has been properly distributed, click on Create New Managed Instance button under Managed Instance section.
2. Select EnterpriseOne HTML Server for Instance Type and click Continue button.
3. On the Instance Properties page fill-in all the required field and click Continue.
Server Group this is to group all the servers that SM is managing, for more detail please review 8.97 SM Guide. For this paper server group oasc was created prior. Instance Name this is what will show up as a link for Instance Name under Managed Home page, like OAS1. J2EE Server this is OC4J container that we created previously. If there is more than one OC4J container then you can select them from drop down box. Software Components This will list all software components that have been distributed to your machine under <JDE_HOME>\component directory.
Page 13
4. Enter valid entries for Path Code, Default Environment, Default Role, and Bootstrap Environment and click Continue.
5.
On the Finish page, click on Create Instance to start the EnterpriseOne HTML Server installation process.
This process can take up to few minutes. Once the installation is complete, SM management console will display a page similar to one shown below. NOTE: Status might show that E1HTML Server is currently running initially. Refresh the screen to get true status.
Page 14
1.
Click on link for JDBC Drivers under the Quick Links menu.
Page 15
2.
In this example we are using SQL Server as our database. Therefore we will be installing JDBC driver for SQL Server. Select the server that you want JDBC driver install to (DENGSSWIN01.mlab.jdedwards.com). Notice that this time JDBC driver is not installed under Status column. Click Install Driver button to start the installation process.
3. JDBC Installation process is quick so after clicking Install Driver button refresh the screen and verify the Status again.
Page 16
4. JDBC Driver install process copies the JDBC driver from SM to <ORACLE_HOME>\j2ee\<oc4j container name>\applib directory. NOTE: (troubleshooting) if you are having issue installing the driver first check the agent log. If agent log indicate issue with copying driver files check this directory. If you only see partial driver files delete them and re-install the driver by repeating steps above.
1. Click on Management Dashboard link then EnterpriseOne HTML Server link (EA_JS_81) under Managed Instance for OAS server (DENGSSWIN01).
2. Click Start under General Section and click OK on the popup box.
Page 17
3. Verify E1HTML Server has started. Status should show that it is running. NOTE: if Instance Information menu shows 0 Active JVM Count then hit F5 on the browser window to refresh the screen.
4. Open a new browser window and type in the URL for E1HTML Server or click on the link for the URL under Instance Properties (http://DENGSSWIN01.mlab.jdedwards.com:81/jde/owhtml), which will launch new Internet browser window with E1HTML login screen.
Page 18
7.
Verify user session from SM by clicking on User Sessions link under Runtime Metrics Menu on the left from EnterpriseOne HTML Server: EA_JS_81 page.
Page 19
Vertical Clustering
In typical vertical clustering, multiple OC4J containers that are identically configured are grouped together in a single OAS 10.1.3.1.0 installation, single Application Server (single node), on one physical machine. This with or without using multiple JVM option will give user option to better optimize hardware resources.
Following parameter was used for our second OC4J instance. 1. Under creating OC4J Container a. OC4J Container Name: VC_JS_82 2. Under Installing EnterpriseOne HTML Server a. Instance Properties Page
Page 20
Once install of second OC4J instance with E1HTML Server is completed you can go back to Managed Homes and Managed Instance page. There you should be able to see both instances (EA_JS_81 and VC_JS_82) for our Oracle Application Server node (DENGSSWIN01)
1. Same as before when we tested our first install, either click the link from EnterpriseOne HTML Server: VC_JS_82 page or open a new internet browser and enter URL to access the login screen. Make sure you are using port 82 since this is port we used for the second install. Enter valid EnterpriseOne userid / password and click Sign In.
Page 21
3. From SM -> EnterpriseOne HTML Server: VC_JS-82 page click on User Sessions link under Runtime Metrics menu to verify that user is accessing VC_JS-82 instance.
Page 22
1. Login to ASC on DENGSSWIN01. Click on the link to our first OC4J instance (EA_JS_81)
2.
From OC4J: EA_JS_81 page click on Application link either from top or bottom of the page
Page 23
4. Next, click on icon under Go to Task column next to Clustering Properties task name.
Page 24
5. Select Override parent application clustering settings and from drop down box next to Clustering select Enable. On Replication Protocol section select Multicast IP Replication and enter valid values for Multicast IP Address and Multicast Port. For this example, following values were used; Multicast IP Address 230.230.0.102 Multicast Port 45566 Click OK NOTE: The Multicast IP address must be in the range 224.0.1.0 239.255.255.255 and port must be a port that is not being used on this system.
Page 25
7. Behind scene, this process updates orion-application.xml file located in <ORACLE_HOME>\j2ee\<OC4J container (EA_JS_81)>\applicationdeployment\<Application Name (EA_JS_81)\ with cluster directive.
8. Repeat steps 1 thru 7 for second OC4J instance (VC_JS_82) NOTE: Multicast IP Address and Port must be identical to our first OC4J instance.
Page 26
1. Open up httpd.conf file using text editor. Httpd.conf file is located under <ORACLE_HOME>\Apache\Apache\conf directory. Append follow info to bottom of httpd.conf file and then save it. Listen 90 NameVirtualHost dengsswin01.mlab.jdedwards.com:90 <VirtualHost *:90> Oc4jMount /jde instance://E1JAS.DENGSSWIN01.mlab.jdedwards.com:EA_JS_81,E1JAS.DENGSSWIN 01.mlab.jdedwards.com:VC_JS_82 Oc4jMount /jde/* instance://E1JAS.DENGSSWIN01.mlab.jdedwards.com:EA_JS_81, E1JAS.DENGSSWIN01.mlab.jdedwards.com:VC_JS_82 </VirtualHost> NOTE: Listen and VirtualHost directive creates single entry point. Although we install our E1HTML Severs on port 81 and 82 we are setting single entry point to port 90. Oc4jMount point for /jde and /jde/* are then identified by point them to both OC4J instances (containers) preceded by node which they are installed on.
Page 27
2. Once you have saved httpd.conf file with modification from step 1 restart the HTTP Server.
Page 28
1. Open 2 internet browser windows and type in URL to our single entry point (port 90). Example (http://dengsswin01.mlab.jdedwards.com:90/jde/E1Menu.maf). If you get the login screen then our entry point was configured correctly. Login using valid EnterpriseOne user id and password and launch different application on each browser.
2. Next we need to make verify that requests are being balanced between 2 OC4J instances. To do that, login to SM and go to Managed Home and Managed Instance page. In the drop down box for Select View choose Search for User Resources.
3. From Search For User Activity page, type in user id that you used to access E1HTML and click Search.
Page 29
Horizontal Clustering
In a horizontal cluster, installation of OAS 10.1.3.1.0 is done on multiple physical machines. With multiple nodes in hand we can cluster these nodes at Application Server level and/or at Application level.
Page 30
1.
Once install have been successfully complete, you should see your new Oracle Application Server instance and E1HTML Server instance from Managed Homes and Managed Instances page in SM.
2.
Test to make sure you are able to login and that everything is working properly. Furthermore, verify via SM EnterpriseOne HTML Server: HC_JS_81 page that Instance Information menu show valid information. NOTE: At this point, from either Oracle Application Server OAS1 or Oracle Application Server OAS2 page in SM will only display J2EE containers that are installed within each physical server. Furthermore, you will have to access Oracle Application Control Console of each install separately (see screenshot below) since they are each standalone install of Oracle Application Server 10.1.3.1.0.
Page 31
Page 32
1. Login to ASC on DENGSSWIN01 and click the Topology Network Configuration link located on the bottom of the page.
Page 33
2. From Topology Network Configuration: E1JAS.DENGSWIN01.mlab.jdedwards.com page, select radio button for Configuring Dynamic Node Discovery Using Multicast and enter valid value in the Discover field and click Apply. For our example, follow value was used; 225.0.20.8001. NOTE: The Multicast IP address must be in the range 224.0.1.0 239.255.255.255 and port must be a port that is not being used on this system.
3. Once the change has been applied screen will refresh to display confirmation message.
4. Behind the scene, this process will <topology> directive in opmn.xml file located under <ORACLE_HOME>\opmn\conf directory.
5. To make sure that the latest change has been applied reload the opmn.xml file. Open a Command Prompt from DENGSSWIN01 and navigate to <ORACLE_HOME>\opmn\bin directory and issue following command; opmnctl reload
Page 34
6. Next, we need to configure our second Oracle Application Server (DENGSSWIN02). Open a Command Prompt from DENGSSWIN02 and navigate to <ORACLE_HOME>\opmn\bin directory.
7.
Issue following command to update the opmn.xml file on DENGSSWIN02 opmnctl config topology update discover=*225.0.0.20:8001
Page 35
opmnctl reload
8. Login to ASC on either DENGSSWIN01 or DENGSSWIN02 and you should be able see both Nodes under Members section. If you do not see both Nodes under Member section verify that you have entered same Multicast IP Address and Port for both Nodes in Topology Network Configuration page.
Page 36
9. Subsequently, now from either Oracle Application Server OAS1 or Oracle Application Server OAS2 page you can view all the OC4J containers on both node and you can manage them (stop, start, and/or restart) from single location within SM.
1. Login to ASC, it doesnt matter which one. You can use ASC on DENGSSWIN01 or DENGSSWIN02 since we have configured dynamic node discovery. Click on the link (HC_JS_81) for OC4J container that is hosting our E1HTML Server instance.
Page 37
Page 38
5. From Application: HC_JS_81 page, click on the icon under Go to Task column for Clustering Properties.
Page 39
6. Select Override parent application clustering settings and select Enable from drop down box for Clustering. Under Replication Protocol, select Multicast IP Replication and fill-in same IP and port information (IP address 230.230.0.102 and Port number 45566) that we used during our vertical cluster setup. Click OK to apply the change.
7. Your screen will refresh to display confirmation message when the process is completed.
Page 40
1. Open httpd.conf file from DENGSSWIN01 under <ORACLE_HOME>\Apache\Apache\conf and add our horizontal instance to the Oc4jMount point and Save the file. E1JASC.DENGSSWIN02.mlab.jdedwards.com:HC_JS_81
2. Restart HTTP Server on DENGSSWIN01. NOTE: we are only using HTTP Server from DENGSSWIN01. You can also use DENGSSWIN02 instead of DENGSSWIN0 if you like; just make same modification to the httpd.conf for the HTTP Server on DENGSSWIN02. Alternatively, if you are using 3rd party clustering product you can also cluster HTTP Server for further redundancy.
Page 41
Bottom half of the screenshot above shows 3 accesses to E1HTML server using 3 separate browser windows. Top half show user resource information and you can see that load is evenly distributed among the entire cluster members.
Page 42
Please refer to Chapter 5 of JD Edwards EnterpriseOne Tools Release 8.97 HTML Web Server Reference Guides for Oracle Application Server 10.1.3.1 (http://www4.peoplesoft.com/contdoc.nsf/0/96337DD5940C20DA8825738B001BB6E9?OpenD ocument) for detail instruction on how to install and configure Web Cache with JD Edwards EnterpriseOne HTML Server.
Login to ASC for OAS 10.1.2.x installation and verify that Web Cache component is enabled. (If not already enabled click on Enable/Disable Components button and move Web Cache component to enable side.) Next, click on link for Web Cache.
Page 43
Page 44
This will bring up Ports page. Click the Add a row button and add required information for new Listen Port. For our example we created new Listen Port, port 90 and used * for IP address.
To configure Origin Servers, from Administration page for Web Cache click the link for Origin Servers.
Copyright Oracle 2006. All rights reserved. EnterpriseOne 8.97 and Multiple JVMs on OAS 10.1.3.1 Page 45
From Origin Server page, click the Create button to add Origin Server.
Complete all require field. For our example we used following values which pointed to our first EnterpriseOne HTML Server, EA_JS_81. Host: dengsswin01.mlab.jdedwards.com Capacity: 100 Port: 81 Protocol: HTTP Failover Threshold: 5 Ping URL: /jde/E1Menu.maf Ping Frequency: 10 Click OK button to create your Origin Server.
Page 46
From Web Cache Administration page click on the link for Sites.
Page 47
Create new Named Sites Definitions by click on the Create button under Named Sites Definitions section.
Enter Host name and Port number you would like to use to define entry point to Web Cache Server. For our example, we created new site definition using port 90. This was added using Ports link under Web Cache Administration page.
Copyright Oracle 2006. All rights reserved. EnterpriseOne 8.97 and Multiple JVMs on OAS 10.1.3.1 Page 48
Next, select all the Origin Server that you would like to associate with this site. Web Cache will load balance incoming request among all Origin Servers associated to this site. NOTE: You have two options for Origin Server, Application Web Server and Proxy Web Server. Available Origin Server will be listed depending on how it was create during Origin Server section.
We added few additional settings. These Expires directives are not features of Web Cache but is highly recommended for EnterpriseOne HTML Server to increase performance.
Page 49
1.
From Managed Homes and Managed Instance page in SM click on the link for OAS Server Instance (OAS1).
Page 50
2. Locate OC4J instance name that you want to work with under J2EE Container section. Under the column labeled JVM Processes specify number of JVMs you would like to utilize for this OC4J instance and click Save. This process is dynamic and additional process will start immediately.
3. Refresh the screen to verify start of additional JVM process and Process ID that are associated to each of them.
Trobleshooting
Internal Server Error
Receive Internal Server error when accessing my vertical or horizontal cluster after configuring httpd.conf file for my single entry point without using Web Cache. Answer: Make sure that upper and lower case of the application server name matches between httpd.conf and ASC
Received Page cannot be displayed error message when accessing horizontal cluster using Web Cache Answer: First, check and see if you can get to that horizontal cluster instance from same browser outside of Web Cache by using URL.
Page 52
If you cannot access via URL and you are using non-default port, such as 90, you might have to add the port to httpd.conf file of that Oracle HTTP Server (OHS) and restart.
Considerations
Following task must be completed outside of Server Manager using Oracle Application Server Control console (ASC). Create new OC4J group Cluster Oracle Application Server install on multiple nodes Cluster E1 HTML server
Following task should be completed using Server Manager. All other functionalities Start and stop E1 HMTL Server Start and stop Oracle HTTP server Update SM Agent Update E1 HTML Server tools release Utilize SAW functionalities Make ini changes Set number of JVM process for OC4J container Use Multiple JVM option instead of Vertical Clustering
Limitation
List below describes current limitation within Server Manager. Cannot change Multiple JVM settings for horizontal instance from single Managed Instance After new E1 HTML tools release cluster is broken and will have to be manually configured again.
Page 53
Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A. Worldwide Inquiries: Phone: +1.650.506.7000 Fax: +1.650.506.7200 oracle.com Copyright 2006, Oracle. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle, JD Edwards, PeopleSoft, and Retek are registered trademarks of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. This document is for informational purposes only and may not be incorporated into a contract or agreement.