PSVR Linux IG

IBM Proventia Server Intrusion Prevention System 򔻐򗗠򙳰
Installation Guide for Proventia Server IPS

for Linux
Version 1.5
Copyright statement
© Copyright IBM Corporation 2006, 2009.

IBM Global Services
Route 100
Somers, NY 10589
U.S.A.
U.S. Government Users Restricted Rights — Use, duplication or disclosure restricted by GSA ADP Schedule Contract with
IBMCorp.
Publication Date: 30 September 2009

Trademarks and disclaimer
IBM® and the IBM logo are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both. Proventia®, SiteProtector™, X-Force® and
X-Press Update are trademarks or registered trademarks of Internet Security Systems™, Inc. in the United
States, other countries, or both. Internet Security systems, Inc. is a wholly-owned subsidiary of
International Business Machines Corporation.
Linux® is a registered trademark of Linus Torvalds in the United States, other countries, or both.
References in this publication to IBM products or services do not imply that IBM intends to make them
available in all countries in which IBM operates.
Disclaimer: The information contained in this document may change without notice, and may have been
altered or changed if you have received it from a source other than IBM Internet Security systems (IBM
ISS). Use of this information constitutes acceptance for use in an “AS IS” condition, without warranties of
any kind, and any use of this information is at the user’s own risk. IBM Internet Security Systems
disclaims all warranties, either expressed or implied, including the warranties of merchantability and
fitness for a particular purpose. In no event shall IBM ISS be liable for any damages whatsoever,
including direct, indirect, incidental, consequential or special damages, arising from the use or
dissemination hereof, even if IBM Internet Security systems has been advised of the possibility of such
damages. Some states do not allow the exclusion or limitation of liability for consequential or incidental
damages, so the foregoing limitation may not apply.
Reference herein to any specific commercial products, process, or service by trade name, trademark,
manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or
favoring by IBM Internet Security systems. The views and opinions of authors expressed herein do not
necessarily state or reflect those of IBM Internet Security systems, and shall not be used for advertising or
product endorsement purposes.
Links and addresses to Internet resources are inspected thoroughly prior to release, but the ever-changing
nature of the Internet prevents IBM Internet Security systems, Inc. from guaranteeing the content or
existence of the resource. When possible, the reference contains alternate sites or keywords that could be
used to acquire the information by other methods. If you find a broken or inappropriate link, please send
an e-mail with the topic name, link, and its behavior to mailto://support@iss.net.
© Copyright IBM Corp. 2006, 2009 iii

iv Proventia Server IPS for Linux: Installation Guide for Proventia Server IPS for Linux
Contents
Trademarks and disclaimer . . . . . . iii Use of iptables in the Proventia Server IPS for Linux
agent . . . . . . . . . . . . . . . . 15
About this publication . . . . . . . . vii Protecting an Apache Web Server . . . . . . . 15
How to use Proventia Server IPS documentation . . vii
Technical support . . . . . . . . . . . . vii Chapter 4. Installing Proventia Server
IPS . . . . . . . . . . . . . . . . 17
Chapter 1. Introduction to the Proventia Installation options . . . . . . . . . . . . 18
Server IPS Agent . . . . . . . . . . . 1 Typical installation . . . . . . . . . . . . 18
Custom installation . . . . . . . . . . . . 20
Introducing the IBM Proventia Server Intrusion
About automated installations . . . . . . . . 23
Prevention System (IPS) for Linux agent . . . . . 2
Automated installation with the IBM ISS response
About the Proventia Server IPS for Linux agent. . . 2
file . . . . . . . . . . . . . . . . . 23
Architecture . . . . . . . . . . . . . . 3
Installing an agent using the IBM ISS response
file . . . . . . . . . . . . . . . . 24
Chapter 2. Upgrading a Proventia Server Automated installation with a custom response file 24
IPS for Linux agent . . . . . . . . . . 5 Creating a response file without installing an
Upgrade options . . . . . . . . . . . . . 6 agent . . . . . . . . . . . . . . . 25
Upgrading an agent remotely. . . . . . . . . 6 Installing an agent and creating a response file 25
Installing a local X-Press Update Server . . . . 7 Automated installation with an existing response
Configuring the local x-press update server to file . . . . . . . . . . . . . . . . . 25
preserve update files. . . . . . . . . . . 8 Confirming the success of an automated installation 26
Making updates available from the local X-Press Uninstalling a Proventia Server IPS Agent . . . . 26
Update Server . . . . . . . . . . . . . 9
Upgrading an agent manually . . . . . . . . 9 Chapter 5. After You Install . . . . . . 27
After you upgrade . . . . . . . . . . . . 10 Defining SiteProtector connection settings . . . . 28
Defining the Agent Manager host name . . . . 28
Chapter 3. Before You Install . . . . . 11 Starting the SPA service . . . . . . . . . 28
Preinstallation checklist . . . . . . . . . . 12 Manually starting the Proventia Server IPS agent . . 28
Running a pilot program . . . . . . . . . . 12 Starting the agent . . . . . . . . . . . 29
OneTrust tokens and entitlements . . . . . . . 13 Starting the SPA service . . . . . . . . . 29
Agent naming conventions . . . . . . . . . 14 Monitoring SSL traffic on additional ports . . . . 29
Unhardening the operating system . . . . . . 14
Ensuring the agent host name can resolve . . . . 14 Index . . . . . . . . . . . . . . . 31
Configuring hosts file entries for each Network
Interface Card (NIC) . . . . . . . . . . . 14
Determining the SiteProtector management group
name . . . . . . . . . . . . . . . . 15
© Copyright IBM Corp. 2006, 2009 v

vi Proventia Server IPS for Linux: Installation Guide for Proventia Server IPS for Linux
About this publication
This guide provides the information you need to install an IBM Proventia Server Intrusion Prevention
System (IPS) for Linux (Proventia Server IPS for Linux). After you read this guide, you should be able to
install Proventia Server IPS for Linux agents on servers in your network.
Scope
This guide contains the prerequisites and the instructions for installing Proventia Server IPS for Linux
agents.
Audience
This guide is for any network or security administrator who is responsible for installing Proventia Server
IPS for Linux agents.
What’s New
This guide was updated to include the changes for the release of Proventia Server IPS for Linux, version
1.5.
v Version 1.5 adds support for OneTrust licensing. See “OneTrust tokens and entitlements” on page 13
for more information.
v Version 1.5 offers upgrade options to upgrade your version 1.0 agents to version 1.5 agents. See
Chapter 2, “Upgrading a Proventia Server IPS for Linux agent,” on page 5.
How to use Proventia Server IPS documentation

Read this guide before you install a Proventia Server IPS for Linux agent and then refer to this guide as
you complete the installation process.
Related publications
For additional information about agents or about SiteProtector, see the following publications:
v Administrator Guide for Proventia Server IPS for Linux
v SiteProtector Installation Guide
v SiteProtector Configuration Guide
v SiteProtector Policies and Responses Guide
License agreement
For licensing information on IBM Internet Security Systems products, download the IBM Licensing
Agreement at http://www.ibm.com/services/us/iss/html/contracts_landing.html.
Technical support
IBM Internet Security Systems (IBM ISS) provides technical support through its Web site and by e-mail or
telephone.
© Copyright IBM Corp. 2006, 2009 vii

The IBM ISS Web site
The Customer Support Web page (http://www.ibm.com/services/us/iss/support/) provides direct

access to online user documentation, current versions listings, detailed product literature, white papers,
and the Technical Support Knowledgebase.
Hours of support
The following table provides hours for Technical Support at the Americas and other locations:
Location Hours
Americas 24 hours a day
All other locations Monday through Friday, 9:00 a.m. to 6:00 p.m. during
their local time, excluding IBM ISS published holidays
Note: If your local support office is located outside the
Americas, you may call or send an e-mail to the
Americas office for help during off-hours.
Contact information
For contact information, go to the Contact us section of the Customer Support Web page at
http://www.ibm.com/services/us/iss/support/.
viii Proventia Server IPS for Linux: Installation Guide for Proventia Server IPS for Linux
Chapter 1. Introduction to the Proventia Server IPS Agent
This chapter describes the Proventia Server IPS for Linux agent, and contains information to help you as
you deploy agents.
Topics
“About the Proventia Server IPS for Linux agent” on page 2
“Architecture” on page 3
© Copyright IBM Corp. 2006, 2009 1

Introducing the IBM Proventia Server Intrusion Prevention System
(IPS) for Linux agent
The IBM Proventia Server Intrusion Prevention System (IPS) for Linux (Proventia Server IPS for Linux)
agent protects servers from the growing spectrum of threats while enabling the servers to keep data and
applications reliable, available, and confidential. This centrally managed enterprise protection agent
combines a proven intrusion prevention system with real-time monitoring and analysis of the operating
system, applications, and network activity to protect server environments from misuse and intrusions
with little impact on the performance of the system.
About the Proventia Server IPS for Linux agent

The Proventia Server IPS for Linux agent protects servers from the growing spectrum of threats while
enabling the servers to keep data and applications reliable, available, and confidential.
The agent combines a proven intrusion prevention system with real-time monitoring and analysis of the
server operating system, applications, and network activity to safeguard the server environment from
misuse and intrusions.
Management
Manage Proventia Server IPS agents with SiteProtector Version 2.0, Service Pack 7 or later.
Layered protection
The Proventia Server IPS agent provides the following components to protect your system:
Component Description
Firewall (FW) The firewall is the first line of defense against a
network-based attack. The firewall can block incoming
and outgoing packets from particular IP addresses, port
numbers, or protocols. It blocks many network attacks
before they can affect the system.
Intrusion Prevention System (IPS) As IP traffic enters or leaves your system, the IPS
analyzes it for malicious content. The IPS drops
offending packets, and allows the remaining traffic to
continue unhindered.
Operating System Events (OS Events) Operating system events detect threats to system
integrity and policy compliance through entries in
system log files. By monitoring changes to log files, the
agent can warn you of suspicious system activity and
allow you to mitigate damage to your system as a result
of malicious activity.
Buffer Overview Exploit Prevention (BOEP) The BOEP component is the last line of defense against
attacks. It comes into play only after the agent has
employed and exhausted all other protection methods.
This component blocks worms and other malicious code
that attempt to exploit buffer overflow vulnerabilities to
propagate or gain access to a system.
Note: BOEP is not currently supported on 64-bit
systems.
2 Proventia Server IPS for Linux: Installation Guide for Proventia Server IPS for Linux
Architecture
The following figure illustrates the architecture of a Proventia Server IPS for Linux agent/SiteProtector
deployment.
Chapter 1. Introduction to the Proventia Server IPS Agent 3

Chapter 2. Upgrading a Proventia Server IPS for Linux agent
If you have an earlier version of a Proventia Server IPS for Linux agent installed, you can upgrade that
agent to version 1.5. This chapter explains how to upgrade agents from version 1.0 to version 1.5.
Note: You cannot upgrade earlier versions of the agent to run on System z servers. The version 1.5 agent
is the first Proventia Server IPS for Linux agent designed to run on System z servers. To install an agent
on a System z server, see Chapter 4, “Installing Proventia Server IPS,” on page 17.
Topics
“Upgrade options” on page 6
“Upgrading an agent remotely” on page 6
“Upgrading an agent manually” on page 9
“After you upgrade” on page 10

Upgrade options
A product upgrade is a software release that includes enhancements to the previously released product.
You have two options when you upgrade a version 1.0 Proventia Server IPS for Linux agent to a version
1.5 agent.
Available options
Option Mechanism Description
Remote upgrade X-Press Update Applies the upgrade from the
management console
Manual upgrade Standalone upgrade Applies the upgrade locally at the
server
Settings that are migrated
Regardless of the mechanism you use to upgrade your agent, the following settings are migrated to your
new agent:
v The installation path
v The agent name
v The network monitoring component choice made when you installed your version 1.0 agent
v The SiteProtector settings specified when you installed your version 1.0 agent
v The Apache Web Server protection component options selected when you installed your version 1.0
agent (including the path to the httpd program file and the path to the httpd.conf file)
v The policy settings configured for your version 1.0 agent
Note: See the following section for specific policy settings that are not migrated.
Other considerations before you upgrade
Regardless of the mechanism you use to upgrade your agent or the configuration specified as part of
your version 1.0 deployment, consider the following points before you upgrade your agent:
v The Buffer Overflow Exploit Prevention component is installed as part of the upgrade process,
regardless of whether it was installed previously. While the component is installed as part of the
upgrade, it is not enabled. If you want the agent to provide protection, you must enable BOEP after the
upgrade is applied.
v The 1.0 Buffer Overflow Exploit Prevention policy configuration is not migrated to the 1.5 Buffer
Overflow Exploit Prevention policy. You must configure and deploy the 1.5 Buffer Overflow Exploit
Protection policy to any groups that contain version 1.5 agents.
v The 1.0 Update Settings policy configuration is not migrated to the 1.5 Update Settings policy. You
must configure and deploy the 1.5 Update Settings policy to any groups that contain version 1.5
agents.
Upgrading an agent remotely

A remote upgrade provides a mechanism to upgrade older versions of the agent to the current version
from the SiteProtector Console.
Before you begin
Consider testing the upgrade thoroughly in a non-production environment before you apply the upgrade
to a production system.
About this task
The package for remote upgrades is provided through the manual download center. To upgrade agents
from the management console using this package, you must perform the following actions on a local
X-Press Update Server:
1. Place the X-Press Update package and its associated XML Catalog on a local X-Press Update Server.
2. Configure the local X-Press Update Server to not download any files from the primary IBM ISS
Update Servers
3. Configure your Proventia Server for Linux agents from the Update Settings policy to connect to the
local X-Press Update Server.
Important: Every SiteProtector deployment has at least one local X-Press Update Server (the primary
Update Server) and optionally, one or more secondary Update Servers. Because installing this remote
upgrade requires that you configure your Update Server to not download files from the primary IBM ISS
Update Servers, in order to keep other components in your SiteProtector deployment up to date, you
should copy and configure the files on a secondary local X-Press Update Server. If you decide to disable
your primary Update Server from contacting the IBM ISS Update Servers, you can safely re-enable this
option once each agent you intend to upgrade to version 1.5 reports into SiteProtector as version 1.5.
Installing a local X-Press Update Server

About this task
This procedure details the steps for installing an X-Press Update Server. If you already have a secondary
X-Press Update Server or you want to use your primary X-Press Update Server for this upgrade, see
“Configuring the local x-press update server to preserve update files” on page 8.
Procedure
1. Connect to the Deployment Manager on the computer where you want to install the XPU Server.
Note: Do not install the XPU Server on the same computer where the Agent Manager is installed. If
you do, then the Agent Manager might experience performance issues.
2. Select Install SiteProtector.
3. On the SiteProtector Installation page, select Additional X-Press Update Server Installation.
4. On the Prerequisites page, review the prerequisites, and then click Next.
5. On the Prepare to Install page, click Install.
6. On the File Download page, click Open.
7. On the InstallShield Wizard Welcome page, click Next.
8. On the License Agreement page, review the terms of the license agreement, click I Accept, and then
click Next.
9. On the Choose Destination Location page, select a destination folder, and then click Next.
10. On the X-Press Update Server Configuration (Specify Agent Manager location) page, complete the
following fields, and then click Next:
Field Description
Name The name of the Agent Manager that the XPU Server
will connect to. Example: AgentManager_100
Address (IP or DNS) Either the IP address or DNS where the Agent Manager
is located.
Port The port the XPU Server should use to communicate
with the Agent Manager. (3995 is the default port.)
Chapter 2. Upgrading a Proventia Server IPS for Linux agent 7

Field Description
Account Name The user name the XPU Server should use to initiate
communication with the Agent Manager.
Password The password the XPU Server must use to initiate
communication with the Agent Manager. The X-Press
Update Server Configuration (Specify SiteProtector
Group Name) window appears.
11. Complete the following fields, and then click Next:
Field Description
SiteProtector Group Name The name of the group where you to put the XPU Server.
If you leave this field blank, then your SiteProtector
system puts the XPU Server in Ungrouped Assets.
X-Press Update Server security mode One of the following:
v Trust all, which allows other servers to connect to the
XPU Server every time it attempts a connection; no
certificates are used for authentication.
v First time trust, which allows other servers to connect
to this XPU Server one time only. After the first
connection, the XPU Server uses the connecting
server’s certificate to authenticate all future
connections.
v Explicit trust, which requires this XPU Server to use a
local certificate to authenticate the server it is
connecting to.
Primary IP If the local computer has more than one network
interface, select the IP address that will be used for XPU
Server communication.
Address (IP or DNS) If the XPU Server will require access through a firewall
or proxy server, then enter the IP address or DNS of the
firewall or proxy server.
Port The port through which the XPU Server will access the
firewall or proxy server.
12. In the Folder box, type the location where you want to archive private keys, and then click Next.
Tip: IBM ISS recommends that you archive keys on a removable medium.
13. Click Install.
14. Click Finish.
Configuring the local x-press update server to preserve update files

Procedure
1. In the left pane, select the group that contains the X-Press Update Server.
2. In the Go to list, select Agent.
3. In the right pane, right-click the X-Press Update Server, and then select Manage Policy. The Policy
tab appears.
4. In the right-pane, right-click Server Settings, and then select Open Policy. The Policy tab appears.
5. Verify that the Download from other XPU Servers option is disabled.
6. Save this change and deploy this change to your local X-Press Update Server.
Making updates available from the local X-Press Update Server
Procedure
1. Download the 1.5 update package from http://www.iss.net/download/
2. Place the .XPU and .xml files in the folder C:\Program Files\ISS\SiteProtector\X-Press Update
Server\webserver\Apache2\htdocs\XPU\SiteProtector
3. Open the Update Settings policy for the group or agents you want to update.
4. Click the Servers tab.
5. Click the Add icon.
6. Set the following options:
Option Description
Name Specifies the name of the local update server
Host or IP Specifies the IP address of the local update server
Port Specifies the port number the agent uses to communicate
with the update server
Note: Port 3994 is the default port the agent uses to
check for updates. Do not change this port number
unless a representative from IBM ISS Technical Support
tells you to.
7. Select Update Settings.

8. Ensure that the Download updates automatically check box is selected.
9. Ensure that the Install updates automatically check box is selected.
10. In the Check for new updates every field, type 1 for 1 hour intervals.
11. Select Advanced Parameters.
12. Ensure that the update.MaxProductUpdate parameter is not set to a value that will prevent the
update from being installed.
Note: This value must be left blank or set to 5 or greater.

13. Save the policy. Within 30 minutes to as much as two hours after the agent receives the updated
policy, the agent will begin its update process.
Upgrading an agent manually

A manual upgrade provides a mechanism to upgrade older versions of the agent to the current version.
Before you begin
Consider testing the upgrade thoroughly in a non-production environment before you apply the upgrade
to a production system.
Procedure
1. Download the installation package for Intel® systems from http://www.iss.net/download/.
2. Using a superuser account, such as root, log on to the system where the 1.0 agent is installed.
3. Copy the installation package to your local drive.
4. Type sh full path to the program file. The system starts the installation program.
5. When the installation package locates the previously installed agent, type y, and then press ENTER to
upgrade the agent.
6. Type y, and then press ENTER to migrate the settings from the previously installed agent. The system
starts the upgrade process. After the upgrade process is complete, the agent is automatically restarted.
Chapter 2. Upgrading a Proventia Server IPS for Linux agent 9

After you upgrade
Depending on the configuration you had before you upgraded your agent, you will need to complete
certain additional tasks after you complete the upgrade process.
Required task
After you complete the upgrade of your version 1.0 agent to version 1.5, you must configure the Update
Settings policy. The 1.5 version of the agent has a new Update Settings policy, so settings from your 1.0
agent cannot be migrated during the upgrade.
Optional tasks
After you complete the upgrade of your version 1.0 agent to version 1.5, and you chose to upgrade with
the Buffer Overflow Exploit Prevention (BOEP) component installed, you will need to configure the BOEP
policy.
If the BOEP component was installed prior to the upgrade, policy settings are not migrated and the
policy is disabled by default. If the BOEP component was not installed prior to the upgrade, you can
choose to install it as part of the upgrade and the policy is disabled by default. If you want the protection
offered by the BOEP component, open the BOEP policy and enable BOEP.
Chapter 3. Before You Install
This chapter outlines the decisions you need to make, and the information you need to know, before you
install a Proventia Server IPS for Linux agent.
Topics
“Preinstallation checklist” on page 12
“Running a pilot program” on page 12
“Agent naming conventions” on page 14
“Unhardening the operating system” on page 14
“Ensuring the agent host name can resolve” on page 14
“Configuring hosts file entries for each Network Interface Card (NIC)” on page 14
“Determining the SiteProtector management group name” on page 15
“Use of iptables in the Proventia Server IPS for Linux agent” on page 15
“Protecting an Apache Web Server” on page 15

Preinstallation checklist
This topic provides a checklist of preinstallation issues that you should consider before you begin to
install a Proventia Server IPS for Linux agent.
Preinstallation checklist
Use the following checklist to confirm your system is ready for your installation:
U Task Reference
h Review the System Requirements document. http://www.iss.net/support/documentation/
h Consider running a pilot program. “Running a pilot program”
h Ensure you have a OneTrust token and OneTrust “OneTrust tokens and entitlements” on page 13
entitlements for your agent.
h Download the installation program file. http://www.iss.net/download/
h Create a naming convention for agents. “Agent naming conventions” on page 14
h Unharden the operating system. “Unhardening the operating system” on page 14
h Uninstall any previously installed instances of a “Uninstalling a Proventia Server IPS Agent” on
RealSecure® Server Sensor, a Proventia Server IPS page 26 or the RealSecure Server Sensor Installation
for Linux agent, or the ISSDaemon. Guide
h Confirm that the hostname and IP address of the “Ensuring the agent host name can resolve” on
agent can be resolved at the agent system. page 14
h Confirm that the /etc/hosts file has an entry for “Configuring hosts file entries for each Network
each NIC on the server. Interface Card (NIC)” on page 14
h Determine the IP address or host name of the
Agent Manager this agent connects to.
h Determine the name of the SiteProtector “Determining the SiteProtector management group
management group that the agent will belong to. name” on page 15
h Review the prerequisites for monitoring Apache “Protecting an Apache Web Server” on page 15
Web Servers.
h Plan to stop using your iptables. “Use of iptables in the Proventia Server IPS for
Linux agent” on page 15
h Plan your installation for a time when it is
convenient to restart your Web Server.
Running a pilot program

Consider running a pilot program before you deploy agents in your production environment. A pilot
program is a small scale deployment of agents (usually on non-production systems) that allows you to
test different policy settings in relative safety.
Benefits
A pilot program provides the following benefits:

v Allows you to collect valuable information that can help you customize policies and refine your
deployment strategies
v Allows you to identify potential problems specific to your environment before you roll out a large-scale
deployment of Proventia Server IPS for Linux agents
OneTrust tokens and entitlements
The Proventia Server IPS for Linux agent uses the OneTrust Licensing System to simplify the
management of product licenses. The OneTrust Licensing System uses customer data and product
entitlements to provide or to restrict access to product updates.
Benefits of the OneTrust Licensing System
The OneTrust Licensing System offers the following benefits:

v provides a single, simplified licensing process for IBM ISS products
v decreases the time required to deploy IBM ISS products
v decreases the time spent managing license keys
v shifts the management of product maintenance to IBM ISS
What is a One Trust token and an entitlement?
The OneTrust Credential Token (token) is an alphanumeric ID that is associated with your IBM ISS
customer ID. If you have purchased any product from IBM ISS, then you have a OneTrust token. When
you purchase a product from IBM ISS, you also receive an entitlement, which identifies the maintenance
expiration dates for each product you have purchased. When you purchase additional products, your
entitlements are updated to reflect the maintenance expiration dates for the new products; when you
renew the maintenance on previously owned products, your entitlements are updated to reflect the new
maintenance expiration dates for those products.
Why you need a One Trust token and entitlements
When you attempt to update your IBM ISS products, your OneTrust token identifies who you are and
your entitlements identify the product updates you are eligible to receive. Without a valid token and
entitlement, you will not be able to install updates to your products.
What to do with a One Trust token when you get it
You must register your token in SiteProtector before you can see the entitlements you have for your
products. In SiteProtector, click Tools → Licensing → OneTrust to work with your OneTrust tokens.
Note: If you already have OneTrust enabled products, your OneTrust token is probably already registered
with SiteProtector.
Registering a token automatically in SiteProtector
If your SiteProtector X-Press Update Server has Internet access, you can configure SiteProtector to
automatically download your token from the Download Center. To use this feature, you must provide
SiteProtector with a valid MyISS username or Order Confirmation Number (OCN) and password.
For detailed steps on adding and managing your OneTrust token, see the SiteProtector Online Help.
Registering a token manually in SiteProtector
You can use the SiteProtector Manual Upgrader tool to download your token and license file from a
system with Internet access, and then import it into SiteProtector.
For detailed steps on adding and managing your OneTrust token manually, see the SiteProtector Online
Help.
Chapter 3. Before You Install 13

Agent naming conventions
An agent naming convention helps you identify agents in the Console.
For example, you may want your agent name to indicate whether an agent is inside or outside the
firewall or in a specific department.
Naming an agent
During the installation process you can assign a custom name to the agent or accept the default name.
You cannot rename an agent after you have installed it.
Important: Agent names can contain only alphanumeric characters with underscores or dashes.
Changing the name of an agent
You can only change the name of an agent by uninstalling and then reinstalling the agent; therefore, it is
important that you establish a logical naming convention before you deploy your agents.
Example
The following naming convention categorizes agents by physical and geographical location and also
identifies their host name:
v nyc_dmz_hostname1
v nyc_int_hostname2
v atl_dmz_hostname3
v atl_int_hostname4
Unhardening the operating system

The installation program must write critical files to be successful; however, the installation program
cannot write critical files to a hardened or locked-down operating system.
Action
You must unharden the operating system before you begin the installation process. You can reharden the
system after the installation is complete.
Ensuring the agent host name can resolve

If the host name and IP address of the system on which you plan to install a Proventia Server IPS for
Linux agent cannot resolve, the installation process will not complete.
You can ensure that the host name can resolve by configuring at least one of the following:
v The /etc/hosts file
v The Domain Name System (DNS) server entries
v The Network Information System (NIS) server entries
Configuring hosts file entries for each Network Interface Card (NIC)
If you have multiple NICs on the server where you plan to install a Proventia Server IPS for Linux agent,
you must add the IP address and host name of each interface you want the agent to protect to the
/etc/hosts file.
Determining the SiteProtector management group name
You manage Proventia Server IPS for Linux agents using SiteProtector groups. By default, the installation
program uses the group name “Proventia Servers for Linux”; you can, however, specify a custom group
name if the default name does not meet your needs.
Customizing the group name
During the installation process you can accept the default group name or you can specify a custom group
name by typing the name of the SiteProtector group this agent will belong to.
Use of iptables in the Proventia Server IPS for Linux agent

When you install a Proventia Server IPS for Linux agent, the installation process flushes the iptables and
any entries you had are lost. In addition, adding entries to the iptables after you have installed an agent
may adversely affect the operation of the agent.
Recommendation
Before you install a Proventia Server IPS for Linux agent, copy your iptables entries. After you complete
the installation process, use the agent to accomplish the same protection.
Protecting an Apache Web Server

Before you install a Proventia Server IPS for Linux agent on a 32-bit system that is running an Apache
Web Server, you must do the following things:
v Gather information about the Apache Web Server
v Confirm that the Apache Web Server supports Dynamic Shared Object (DSO)
Note: The Apache Web Server Protection component is not currently supported on 64-bit systems.
Apache Web Server information
During the installation process you must have the following information about your Apache Web Server
installation:
v the name and location of the httpd program file and the httpd.conf file you want to protect if you do
not want to protect the Apache files detected by the installation package
v whether the modssl module is enabled
Determining support for Dynamic Shared Object
To determine whether the Apache Web Server supports Dynamic Shared Object (DSO):
v Do one of the following:
If you are using... Then type this...

Apache httpd -1
Apache+modssl httpd -1
Apache+OpenSSL httpsd -1
If the Web Server supports DSO, the result contains mod_so.c.
Chapter 3. Before You Install 15

If the Web Server does not support DSO
If the Web Server does not support DSO, go to http://www.apache.org to obtain the Apache source, and
then compile the source with mod_so enabled.
Chapter 4. Installing Proventia Server IPS
You should install a Proventia Server IPS for Linux agent on any server that contains information that
you want to protect. This chapter describes the installation options and the installation procedures you
can use to install an agent.
Topics
“Installation options” on page 18
“Typical installation” on page 18
“Custom installation” on page 20
“About automated installations” on page 23
“Automated installation with the IBM ISS response file” on page 23
“Automated installation with a custom response file” on page 24
“Automated installation with an existing response file” on page 25
“Confirming the success of an automated installation” on page 26
“Uninstalling a Proventia Server IPS Agent” on page 26

Installation options
there are several installation options available when you want to install a Proventia Server IPS for Linux
agent.
v Typical
v Custom
v Automated
Typical installation
If the following default settings meet your needs, use the typical installation option:
Item Default setting

Installation path /opt/ISS
Agent name proventia_server_1
Network monitoring component Enabled
Installs and enables the network monitoring component.

Apache monitoring component Enabled
The installation program checks for an Apache Web

Server, confirms that you want to protect the Web Server,
and installs the Apache monitoring component.
Note: The Apache Web Server Protection component is
not currently supported on 64-bit systems.
Custom installation
If the Typical installation settings do not meet your needs, use the custom installation option.
Automated installation
If you plan to install several agents with the same settings, you should use the automated installation
option. The automated installation option records the responses to installation questions, and then uses
those responses to install agents on other identical systems.
Typical installation
The typical installation option uses default settings to quickly install a Proventia Server IPS for Linux
agent on your server.
About this task
Installing a Proventia Server IPS for Linux agent will flush the iptables on your system. See “Use of
iptables in the Proventia Server IPS for Linux agent” on page 15.
Procedure
1. Log on using a superuser account, such as “root”.
3. Type sh full path to the program file.
4. If the installation package locates a previously installed agent, type y, and then press ENTER to
upgrade the agent.
5. Type n, and then press ENTER to upgrade to the 1.5 agent without migrating the settings from the
previously installed agent.
Important: If you choose to upgrade without migrating settings, the currently installed agent is
uninstalled and the installation for the new 1.5 agent begins. To migrate the settings from the
previously installed agent, see “Upgrading an agent manually” on page 9 for more information.
6. Type 1, and then press ENTER to accept the license agreement.
Note: If you do not accept the license agreement, the installation program stops without installing
an agent.
7. Type y, and then press ENTER to use the settings listed in the table in “Installation options” on page
18.
8. Continue through the installation questions, accepting the default settings where applicable. Use the
following table as a guide:
Setting Option
SiteProtector connection parameters Type the hostname or IP address of the Agent Manager
this agent connects to.
Note: The agent cannot communicate with SiteProtector
without this information. If you do not set this option
now, you must manually set it after the installation
process is completed. See “Defining SiteProtector
connection settings” on page 28.
Group name Type the name of the SiteProtector group this agent
belongs to.
Note: The default group name is Proventia Servers for
Linux. If you want a custom name, such as Mailservers
or Atlanta Webservers, type that name here.
Buffer Overflow Exploit Prevention (BOEP) Install BOEP now, or do not install BOEP. If you choose
to install BOEP at a later time, you must reinstall the
agent.
When you use the BOEP component, other buffer

overflow technologies, such as ExecShield and the
NX/XD bit, will be disabled. When you enable or disable
BOEP within the policy, you must restart the system to
make any changes to other buffer overflow technologies
effective.
Attention: BOEP is not currently supported on 64-bit

systems.
Install the Apache Web Server Protection component Note: This component is not currently supported on
64-bit systems.
Do one of the following things:

v If you want to protect the Apache Web Server located
by the installation program, type y, and then press
ENTER. Go to Step 8.
v If you want to protect a different Apache Web Server
than the one located by the installation program, type
n, and then press ENTER.
v If you do not want to protect an Apache Web Server,
type n, and then press ENTER. Go to Step 8.
Chapter 4. Installing Proventia Server IPS 19

Setting Option
Full path to the Apache httpd program file If you want to protect an Apache program file other than
the one detected by the installation program, type the
full path to the file you do want to protect, and then
press ENTER.
Full path to the Apache httpd.conf file If you want to protect an Apache configuration file other
that the one detected by the installation program, type
the full path of the file you do want to protect, and then
press ENTER.
9. Do you want to start the agent when the installation is finished?

v If yes, type y, and then press ENTER.
v If no, type n, and then press ENTER.
Reference: See “Manually starting the Proventia Server IPS agent” on page 28.
The installation begins. See “Results” next in this topic.
10. Restart your Web Server.
Results
After the installation process is completed, you may see the following warning messages:
v A message that the kernel may be tainted and messages about unresolved symbols. These messages are
a result of the installation of the BOEP module. You can safely ignore these warnings.
v A message that Apache may crash. This message appears if you installed the agent on a system
running Apache 1.x. You can safely ignore these warnings.
Custom installation
The custom installation option allows you to specify settings as you install the agent.
Before you begin
If you are reinstalling the agent to a directory where it was previously installed, you must first manually
remove the installation directory. The reinstallation cannot complete successfully until the files that
remain from the previous installation are removed.
About this task
Procedure
1. Log on using a superuser account, such as “root”.
3. Type sh full path to the program file.
4. If the installation package locates a previously installed agent, type y, and then press ENTER to
upgrade the agent.
5. Type n, and then press ENTER to upgrade to the 1.5 agent without migrating the settings from the
previously installed agent.
Important: If you choose to upgrade without migrating settings, the currently installed agent is
uninstalled and the installation for the new 1.5 agent begins. To migrate the settings from the
previously installed agent, see “Upgrading an agent manually” on page 9 for more information.
6. Type 1, and then press ENTER to accept the license agreement.
Note: If you do not accept the license agreement, the installation program stops without installing
an agent.
7. Type n, and then press ENTER to define custom installation settings.
8. Continue through the installation using the following table as a guide:
Setting Option
Installation path Do one of the following things:
v Press ENTER to accept the default path.
[/opt/ISS]
v Type the full path to where you want the agent
installed, and then press ENTER.
Important: The custom path cannot be /opt or a
sub-directory of opt/ISS.
Note: The agent creates a symlink from /opt/ISS to
the custom directory you specify.
Proventia Server Name Do one of the following things:
v Press ENTER to accept the default name.
[proventia_server_1]
v Type the custom name, and then press ENTER.
Note: Agent names should be alphanumeric and
should not contain any spaces.
Note: The agent creates a symlink from the custom
agent name you specify to /path/ISS/
issSensors/proventia_server_1.
Network monitoring Do one of the following things:
v Type y, and then press ENTER.
v Type n, and then press ENTER.
Note: If you disable this component, the Refresh Agent
feature in SiteProtector will not function.
SiteProtector connection parameters Type the hostname or IP address of the Agent Manager
this agent connects to.
Note: The agent cannot communicate with SiteProtector
without this information. If you do not set this option
now, you must manually set it after the installation
process is completed. See “Defining SiteProtector
connection settings” on page 28.
Group name Type the name of the SiteProtector group this agent
belongs to.
Note: The default group name is Provemtia Servers for
Linux. If you want a custom name, type that name here.

Setting Option
Buffer Overflow Exploit Prevention (BOEP) Install BOEP now, or do not install BOEP. If you choose
to install BOEP at a later time, you must reinstall the
agent.

effective.

systems.
Install the Apache Web Server Protection component Note: This component is not currently supported on
64-bit systems.
Do one of the following things:

v If you want to protect the Apache Web Server located
by the installation program, type y, and then press
ENTER. Go to Step 8.
v If you want to protect a different Apache Web Server
than the one located by the installation program, type
n, and then press ENTER.
v If you do not want to protect an Apache Web Server,
type n, and then press ENTER. Go to Step 8.
Full path to the Apache httpd program file If you want to protect an Apache program file other than
the one detected by the installation program, type the
full path to the file you do want to protect, and then
press ENTER.
Full path to the Apache httpd.conf file If you want to protect an Apache configuration file other
that the one detected by the installation program, type
the full path to the file you do want to protect, and then
press ENTER.
9. To start the agent when the installation is finished, type y, and then press ENTER.
Note: See “Manually starting the Proventia Server IPS agent” on page 28 to start the agent at a later
time.
The installation begins. See “Results” next in this topic.
10. Restart your Web Server.
Results
v A message that Apache may crash. This message appears if you installed a Proventia Server IPS agent
on a system running Apache 1.x. You can safely ignore these warnings.
About automated installations
If you plan to install several agents using the same installation options, you can use the automated
installation option to simplify your task. The automated installation option uses a response file that
contains the responses to installation questions to install agents on other systems using one simple
command.
Automated installation options
When you use the automated installation option, you have the following choices:
v install an agent using a response file provided by IBM ISS
Reference: See ″Settings in IBM ISS response file″ in “Automated installation with the IBM ISS
response file” for the response file settings.
v Generate a custom response file without installing an agent
v Generate a custom response file and install an agent at the same time
v Install an agent using a preexisting response file
Important
You can only use the automated installation option to install agents on systems that are identical.
Automated installation with the IBM ISS response file

The installation package for Proventia Server IPS for Linux agents comes with a response file that you
can use to install agents.
Important
Settings in the IBM ISS response file
The following table lists the settings in the response file provided by IBM ISS:

Installation path /opt/ISS
Agent name proventia_server_1
Network monitoring component Enabled
Installs the network monitoring component of the

Proventia Server IPS agent.
SiteProtector Connection Parameter Not set.
The agent cannot communicate with SiteProtector

without this information. You must manually enter this
information after the installation process is complete. See
“Defining SiteProtector connection settings” on page 28.

Buffer Overflow Exploit Prevention (BOEP) Enabled
Installs the Buffer Overflow Exploit Prevention

component.

effective.

systems.
Apache monitoring component Enabled
Installs the Apache monitoring component of the agent.

Note: The Apache Web Server Protection component is
not currently supported on 64-bit systems.
Path of httpd program file Path to the httpd file, as determined from the host RPM
database.
Path of httpd.conf Path to the http.conf file, as determined from the host
RPM database.
Start Proventia Server IPS Enabled
Installing an agent using the IBM ISS response file

If the settings in the IBM ISS response file meet your needs, use this procedure to install an agent with
the response file.
Procedure
Type the following:

sh full path to the program file -s -sp SiteProtector IP address
where full path to the program file is the location and file name of the installation program file and
SiteProtector IP address is the address of the SiteProtector system.
Results
v A message that Apache may crash. This message appears if you installed the agent on a system
Automated installation with a custom response file

If the installation options of the IBM ISS response file do not meet your needs, you can create a custom
response file.
About this task
When you create a custom response file, you can do one of the following:
v Create a response file, but not install an agent
v Create a response file and install an agent at the same time
Creating a response file without installing an agent

Procedure
1. Type the following:
sh full path to the program file -r full path of the response file
where full path to the program file is the location and file name of the installation program file and full
path of the response file is the location where the response file will be saved.
2. Follow the agent installation process using the procedure in “Typical installation” on page 18 or the
procedure in “Custom installation” on page 20. The installation program creates and saves a response
file that contains your installation options.
Installing an agent and creating a response file

About this task
Procedure
1. Type the following:
sh full path to the program file -ri full path of the response file
where full path to the program file is the location and file name of the installation program file and full
path of the response file is the location where the response file will be saved.
2. Install the agent using the procedure in “Custom installation” on page 20.
Results
v A message that Apache may crash. This message appears if you installed an agent on a system that is
Automated installation with an existing response file

If you already have a response file, you can install agents with that file.
About this task
Procedure
Type the following:

sh full path to the program file -s full path of the response file
where full path to the program file is the location and file name of the installation program file and full path
of the response file is the location where the response file is located.
A Proventia Server IPS agent is installed with the settings defined in the response file.

Results
v A message that Apache may crash. This message appears if you installed a Proventia Server IPS agent
on a system running Apache 1.x. You can safely ignore these warnings.
Confirming the success of an automated installation

After you have run an automated installation, you may want to confirm that the installation was
successful.
Confirming the installation
When you use the automated installation option to install an agent, the installation process generates a
log file called install.log. You can find this file in the installation directory. If the automated installation
was successful, the log file looks as follows:
Silent Installation Log File

Date = <Installation time and date>
Product = Proventia Server for Linux
Version= 1.0
[Response Result]
Result : SUCCESS
Uninstalling a Proventia Server IPS Agent

Uninstalling a Proventia Server IPS for Linux agent is a simple process.
About this task
If you enabled BOEP as part of the agent configuration on a Red Hat system, then ExecShield is disabled
and is re-enabled when you uninstall the agent. If you enabled BOEP as part of the agent configuration
on any system, then No Execute (NX bit) is disabled and is re-enabled when BOEP is disabled.
Note: After uninstalling the agent, ExecShield will return to the pre-installation state and NX bit will be
enabled.
Time needed to uninstall: The uninstallation process may take several minutes, because the agent must
unregister from SiteProtector before the process can begin.
Procedure
1. On the system where you installed the agent, change the directory to /opt/ISS.
2. Type the following command:
sh uninstall.sh
Chapter 5. After You Install
This chapter describes procedures you may have to complete after you have installed a Proventia Server
IPS for Linux agent. You should review the topics in this chapter to ensure that the agent is ready to
begin protecting your server.
Topics
“Defining SiteProtector connection settings” on page 28
“Manually starting the Proventia Server IPS agent” on page 28
“Monitoring SSL traffic on additional ports” on page 29

Defining SiteProtector connection settings
If you did not specify the IP address for the Agent Manager that the agent connects to when you
installed the agent, you must define that information before your agent can communicate with
SiteProtector.
Do I need to do this?
If you specified the IP address for the Agent Manager that the agent connects to during the typical
installation procedure, the custom installation procedure, or during the automated installation procedure
using a custom response file, you do not need to perform the procedures in this topic.
Background
Before your agent can communicate with SiteProtector, you must specify the IP address or host name of
the Agent Manager the agent connects to. You can do this as part of the installation process for all
installation options except the automated installation using the IBM ISS response file. If you used the IBM
ISS response file with the automated installation procedure, or if you chose not to specify the IP address
or host name as part of another installation option, you must perform the procedures in this topic.
Process overview
The following table outlines the process for manually configuring the agent-SiteProtector connection:
Task Description
1 You must define the Agent Manager host name on the
server where you installed the Proventia Server IPS
agent.
2 You must start the SPA (SiteProtector Adapter) service.
Defining the Agent Manager host name

Procedure
1. On the system where you installed the agent, locate the following file:
/opt/ISS/issSensors/sensor_name/rs PostGroupsSettings.xml
2. On the host-name line, type the host name/IP Address of the Agent Manager.
Starting the SPA service

Procedure
On the system where you installed the agent, type the following command:
/etc/init.d/iss-spa start
Manually starting the Proventia Server IPS agent

Do I need to do this?
During the installation process you could choose whether to start the agent after the installation process
completed or at a later time. If you chose to start the agent at a later time, you need to perform the
procedures in this topic.
Process overview
The following table outlines the process for manually starting a Proventia Server IPS agent:
Task Description
1 You must start Proventia Server IPS.
2 You must start the SPA (SiteProtector Adapter) service.
Starting the agent

Procedure
/etc/init.d/proventiaserver start
Starting the SPA service

Procedure
/etc/init.d/iss-spa start
Monitoring SSL traffic on additional ports

By default, the installation configures monitoring of SSL traffic on port 443. You can, however, configure
your system to monitor SSL traffic on additional ports.
About this task
The Apache Web Server Protection component is not currently supported on 64-bit systems.
Procedure
1. In the httpd.conf file, locate the following entry:
SetIssPamPorts 443
2. Type the port number for the port you want to add.
Note: Separate the port numbers with a comma.
Example
In addition to monitoring port 443, you want to monitor port 3994. Change the line SetIssPamPorts 443 to
SetIssPamPorts 443,3994.
Chapter 5. After You Install 29

Index
A F P
agent firewall component 2 pilot program 12
manually starting 28, 29 preinstallation checklist 12
naming convention 14
upgrading 5 G
Agent Manager host name
manually defining 28
group name R
custom 15 reinstalling
agent names
default 15 prerequisite 20
alphanumeric characters 14
response file
dashes 14
creating while installing 25
underscores 14
alphanumeric characters H creating without installing 25
IBM ISS provided 23
in agent names 14 heartbeat channel 3
installing with existing 25
Apache crash host name resolution 14
warning message 20, 22, 24, 25, 26 httpd program file 15
architecture 3 httpd.conf file 15
automated installation S
confirming installation 26 Secure Sockets Layer (SSL)
IBM ISS response file 24
identical systems 23
I default port 29
monitoring additional ports 29
IBM Internet Security Systems
install.log location 26 monitoring traffic using 18
technical support viii
installation options 23 SiteProtector Adapter 28
Web site viii
with existing response file 25 SSL (Secure Sockets Layer)
IBM ISS response file settings 23
default port 29
installation
monitoring additional ports 29
custom path 21
B hardened system 14
monitoring traffic using 18
starting agent 29
BOEP module prerequisites 11
symlink
warning message 20, 22, 24, 25, 26 installing
custom agent name 21
buffer overflow exploit prevention 2 reinstalling prerequisite 20
custom installation path 21
intrusion prevention system 2
iptables
C flushing 15
T
checklist 12
technical support viii
custom agent name
symlink 21 L typical installation
default settings 18
custom group name 15 layered protection 2
custom installation
symlink 21
custom installation path 21 M U
underscores
management component 2
in agent names 14
modssl module 15
D multiple NICs 15
uninstalling
time needed 26
dashes
update channel 3
in agent names 14
upgrading
default group name 15
Dynamic Shared Object
N agents 5
naming agents
determining support for 15
custom name 14
example 14
renaming 14
W
E No Execute 19, 22, 24, 26
warning messages
installation 20, 22, 24, 25, 26
event channel 3 NX bit 19, 22, 24, 26
ExecShield 19, 22, 24, 26
Execute Disable 19, 22, 24, 26
O X
XD bit 19, 22, 24, 26
OneTrust 13
operating system events 2

PSVR Linux IG

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PSVR Linux IG

Uploaded by

Copyright:

Available Formats

IBM Proventia Server Intrusion Prevention System 򔻐򗗠򙳰

Installation Guide for Proventia Server IPS

© Copyright IBM Corporation 2006, 2009.

Publication Date: 30 September 2009

© Copyright IBM Corp. 2006, 2009 iii

© Copyright IBM Corp. 2006, 2009 v

How to use Proventia Server IPS documentation

© Copyright IBM Corp. 2006, 2009 vii

The Customer Support Web page (http://www.ibm.com/services/us/iss/support/) provides direct

“About the Proventia Server IPS for Linux agent” on page 2

© Copyright IBM Corp. 2006, 2009 1

About the Proventia Server IPS for Linux agent

Chapter 1. Introduction to the Proventia Server IPS Agent 3

“Upgrade options” on page 6

“Upgrading an agent remotely” on page 6

“Upgrading an agent manually” on page 9

“After you upgrade” on page 10

© Copyright IBM Corp. 2006, 2009 5

Settings that are migrated

Other considerations before you upgrade

Upgrading an agent remotely

Before you begin

Installing a local X-Press Update Server

Chapter 2. Upgrading a Proventia Server IPS for Linux agent 7

11. Complete the following fields, and then click Next:

Configuring the local x-press update server to preserve update files

7. Select Update Settings.

Note: This value must be left blank or set to 5 or greater.

Upgrading an agent manually

Before you begin

Chapter 2. Upgrading a Proventia Server IPS for Linux agent 9

“Preinstallation checklist” on page 12

“Running a pilot program” on page 12

“Agent naming conventions” on page 14

“Unhardening the operating system” on page 14

“Ensuring the agent host name can resolve” on page 14

“Determining the SiteProtector management group name” on page 15

“Protecting an Apache Web Server” on page 15

© Copyright IBM Corp. 2006, 2009 11

Running a pilot program

A pilot program provides the following benefits:

Benefits of the OneTrust Licensing System

The OneTrust Licensing System offers the following benefits:

What is a One Trust token and an entitlement?

Why you need a One Trust token and entitlements

What to do with a One Trust token when you get it

Registering a token automatically in SiteProtector

Registering a token manually in SiteProtector

Chapter 3. Before You Install 13

Changing the name of an agent

Unhardening the operating system

Ensuring the agent host name can resolve

Customizing the group name

Use of iptables in the Proventia Server IPS for Linux agent

Protecting an Apache Web Server

Apache Web Server information

Determining support for Dynamic Shared Object

If you are using... Then type this...

If the Web Server supports DSO, the result contains mod_so.c.

Chapter 3. Before You Install 15

“Installation options” on page 18

“Typical installation” on page 18