GSK_ERROR_BAD_CERT error configuring SSL between Plug-in and WebSphere Application Server V6.

1 Technote (troubleshooting) Problem(Abstract) When an HTTPS request is sent to a IBM WebSphere Application Server V6.1 server, from a web server, the web server plug-in log shows the error: ERROR: lib_stream: openStream: Failed in r_gsk_secure_soc_init: GSK_ERROR_BAD_CERT(gsk rc = 414) Cause WebSphere Application Server V6.1 has brand-new SSL management functions which need to be clearly understood to come to a proper configuration and a complete resolution. Although there are technotes for earlier Application Server versions related to this problem, there is no comprehensive document that describes the nature of this problem and how to make a valid SSL configuration between plug-in and WebSphere Application Server V6.1, Network Deployment (ND) and Stand Alone, to solve this problem. First look at the signs about the nature of the problem in the plug-in side: If trace is enabled for the web server plug-in, these lines are logged in the http_plugin.log file when a HTTPS request is tried:

DETAIL: ws_common: websphereFindTransport: Setting the transport(case 1): servis2 on port 9443 TRACE: ws_common: websphereExecute: Executing the transaction with the app server DEBUG: ws_common: websphereGetStream: Getting the stream to the app server TRACE: ws_transport: transportStreamDequeue: Checking for existing stream from the queue DEBUG: ws_common: websphereGetStream: socket 16 connected to servis2:9443 DEBUG: lib_stream: openStream: Opening the stream DEBUG: lib_stream: openStream: Stream is SSL ERROR: lib_stream: openStream: Failed in r_gsk_secure_soc_init: GSK_ERROR_BAD_CERT(gsk rc = 414)

As is apparent in the logs, the plug-in opens a stream through WebSphere Application Server HTTPS transport port (9443). However the certificate sent by WebSphere Application Server to the plug-in cannot be authenticated by the plug-in key file. The reason is that the plug-in key file does not have the adequate signer to authenticate the certificate sent by WebSphere Application Server. In earlier releases of WebSphere Application Server, it was suspected that the certificate might have become expired. However, WebSphere Application Server V6.1 has a new function called "certificate monitoring" which does the following:

Notifies the user in defined time intervals before a certificate expires Automatically replaces self-signed certificates that are about to expire Updates all trust stores in the cell

If you want to check it anyway, you can click on the "view certificate button" that will appear on your browser when you send the HTTPS request. Also, you can use the new WebSphere Application Server V6.1 embedded key management feature, or iKeyman, to view the certificate. Before attempting to describe how, which, and in which stores proper signer certificates must be added, WebSphere Application Server V6.1 SSL management enhancements must be closely examined. In WebSphere Application Server V6.1, each profile is created with its unique self signed certificates. All profiles have their own node level key and trust stores. For Network Deployment, there is also a cell level key store, CellDefaultKeyStore, and a cell level trust store, CellDefaultTrustStore, which are pointed out by all nodes in default cell settings. To establish a proper SSL configuration in which all nodes (including the dmgr node) can communicate with each other, their default certificates are added to the cell level trust store as signer certificates. In addition, for SSL to be properly configured between a web server and WebSphere Application Server V6.1, its plugin-key.kdb file must include all nodes' default certificates as signer certificates, and web server node's default certificate must exist in the CellDefaultTrustStore considering that a web server also works in a node. As the facts show, SSL configuration between plug-in and WebSphere Application Server is completely different for version 6.1. In earlier releases, default dummy certificates were exchanged between plug-in and WebSphere Application Server. These dummy certificates are also included in version 6.1, yet only for backwards capability issues. The rest of the document will show you how to establish proper configuration in WebSphere Application Server V6.1.

Resolving the problem For Network Deployment 1. In the administrative console, go to Security > SSL certificate and key management. Before doing any changes, put select Dynamically update the runtime when changes occur on this page. This option makes sure that changes are propagated to runtime immediately after they are saved. This option requires a restart to become active after it is selected. If this option is enabled, make sure that you make SSL configuration changes when the system does not have a high burden on it to prevent performance impacts. 2. Click the Manage endpoint security configurations link. 3. Expand Inbound or Outbound, expand the cell name to see the list of nodes. For all the nodes that appear in the list:

Opening an empty text file will help you through the process. 4. Go to Key stores and certificates which is under Related Items. 5. Click on the NodeDefaultKeyStore. Under Additional Properties, click on Personal Certificates. 6. Note down the serial number of the default certificate. Select the box near the default certificate. Click Extract. 7. Write the file name to be extracted with the full path, leave the data type as it is, note down the file path after the serial number. Click OK. If you chose to create a cell profile after your initial WebSphere Application Server installation, the cell manager node and the stand alone node you have created that time might have the same certificate with the same serial number. Do not let it confuse you. After the previous instructions are done for all nodes, continue with the following steps. 8. Come to the Manage endpoint security configurations page where you see the node list again (instructions 1-3). 9. Expand the node which includes the web server. 10. Click on the web server, then click on Key stores and certificates. 11. Click on the CMSKeyStore. 12. Click on the Signer certificates. You can either add here all the certificates you have extracted, or you can click on default certificates in this page, if there are any, and compare their serial numbers with the numbers that you have taken note of to determine which default certificates are missing. For all the certificates or just the missing ones apply the instructions below. 13. Click Add on the current page. 14. Enter the certificate file path, an alias as you wish, and leave the data type as it is. Click OK.

When you are sure that you have the complete set of default certificates added as signer certificates, save the changes, and synchronize. Verifying that changes are propagated Now, check if the changes are propagated to runtime. If you have enabled the option Dynamically update the runtime when changes occur on the SSL certificate and key management page, they are propagated automatically. Here is the checking procedure:

1. Open the httpd.conf file of your web server. Find the line that starts with "WebSpherePluginConfig". This line points to the path of your plugin-cfg.xmlfile that is used in runtime. Take note of the directory that this file resides. 2. Run iKeyman. There are copies of iKeyman you can use in the follownig locations: o plugin_root/bin o appserer_root/bin o httpserver_root/bin if you are using IBM HTTP Server 3. Click on the Key Database File menu and choose Open. Select CMS as the key database type, click on browse, and browse to the path that you have taken note of in step 1. Double-click plugin-key.kdb, and click ok to open it. The password is WebAS. 4. Click Personal Certificates. Select Signer Certificates from the drop down list. 5. Check to see that all the signer certificates that you have added in previous steps 12 14 are also added as signer certificates in this file. If they are not, add them by using the add button. Again you need to keep the data type as it is. You just need to browse to the key files and click "ok". 6. Restart the web servers.

For stand alone (Base and Express) The procedure is the same for these products. The only difference is the number of nodes. Note 1: If you still have the error after these changes make sure that: For Network Deployment, CellDefaultTrustStore has all nodes' the signer certs. For Stand Alone, NodeDefaultTrustStore has all nodes' the signer certs. You might also need to extract the web server's certificate and add it to the web server node's signer certificates, which can be done by using the administrative console's certificate management as in steps before verification, and plugin-key.kdb which can be done by using iKeyman as in the verification steps.

Note 2: From Fix Pack 5 (6.1.0.5) onwards, if a new node is federated after the web server definition has been created, the signer for the new node will be automatically updated to the plugin-key.kdb. It is recommended to apply this fix pack.