Professional Documents
Culture Documents
Prepared By:
Version:
Publication Date:
Version:
1.00
Date Created:
Created By:
Date Published:
Security Classification:
Creation Software:
Contributors:
MSOC Team
CHANGE HISTORY
Ver.
Date
Change Description
0.01
1.00
Approval
DOCUMENT REVIEW
Name/Title
Signature
Date
DISTRIBUTION LIST
IBM Managed Services
Champion Managed Services
All IBM/CSG Managed Services Clients
DOCUMENT LOCATION
This document is available via the Champion Portal at https://www.championpulse.com.
Clients Final:
The Clients copy is stored on the CMS portal under their specific document area.
Template
To obtain the internal template from the CMS portal, please click here, or enter
the following URL in your Web browser:
https://portal.championpulse.com/C15/MSOCPoliciesNProcedures/Templates/Op
erations_Run_Book.doc.
Page - i
Table of Contents
Introduction........................................................................................................................................... 1
Contact Information............................................................................................................................... 1
Champion.......................................................................................................................................... 1
IBM.................................................................................................................................................... 1
Escalation Process............................................................................................................................ 2
Champion Group / IBM Web Portal................................................................................................... 3
Champion Group / IBM Phone Support............................................................................................. 3
Infrastructure......................................................................................................................................... 3
Facility Overview............................................................................................................................... 3
Site restrictions.............................................................................................................................. 3
Fire and emergencies.................................................................................................................... 3
Shipping To The Facility................................................................................................................ 3
Directions...................................................................................................................................... 4
Shared Common Areas................................................................................................................. 4
Hardware Configuration.................................................................................................................... 5
Operating Procedures Overview........................................................................................................... 6
Introduction....................................................................................................................................... 6
Remote Accessibility......................................................................................................................... 7
Purpose......................................................................................................................................... 7
Scope............................................................................................................................................ 7
General Policy............................................................................................................................... 7
Requirements................................................................................................................................ 7
Enforcement.................................................................................................................................. 8
Operating System Permissions......................................................................................................... 8
Purpose......................................................................................................................................... 8
Scope............................................................................................................................................ 8
Policy............................................................................................................................................. 8
Ownership and Responsibilities.................................................................................................... 8
General Configuration Guidelines................................................................................................. 9
Compliance................................................................................................................................... 9
Server Setup..................................................................................................................................... 9
Overview....................................................................................................................................... 9
Scope............................................................................................................................................ 9
Copyright 2005 Champion Solutions Group
Operations Run Book
Page - ii
Policy............................................................................................................................................. 9
Ownership and Responsibilities.................................................................................................. 10
Backup Configuration...................................................................................................................... 11
Software...................................................................................................................................... 11
Policies........................................................................................................................................ 11
Data Restoration Process............................................................................................................... 12
Purpose....................................................................................................................................... 12
Overview..................................................................................................................................... 12
Incremental Backups................................................................................................................... 12
Incremental Restores.................................................................................................................. 12
Database Restores...................................................................................................................... 12
Tape Handling and Retention.......................................................................................................... 12
Retention Policies........................................................................................................................ 12
DRM Tape Handling.................................................................................................................... 12
Responding To Alerts...................................................................................................................... 14
Change Management.......................................................................................................................... 14
Types Of Change Requests............................................................................................................ 14
Severity And Priority........................................................................................................................ 15
Change Request Classifications...................................................................................................... 15
Scheduled Client Change............................................................................................................ 16
Scheduled CSG Change............................................................................................................. 16
Emergency Changes................................................................................................................... 16
Who Is Authorized To Request A Change?......................................................................................16
How Is A Request Submitted?......................................................................................................... 16
Scheduled Maintenance Windows.................................................................................................. 17
Change Control Board..................................................................................................................... 17
Decision Categories........................................................................................................................ 17
Turnaround Time............................................................................................................................. 17
Problem Management......................................................................................................................... 17
Business Hours............................................................................................................................... 17
Contacting Support (Champion MSOC).......................................................................................... 17
Telephone.................................................................................................................................... 17
Champion Portal.......................................................................................................................... 17
Severity And Priority Levels............................................................................................................. 17
Trouble Ticket Workflow.................................................................................................................. 20
Monitoring Standards.......................................................................................................................... 21
Copyright 2005 Champion Solutions Group
Operations Run Book
Page - iii
PURPOSE....................................................................................................................................... 21
REFERENCE.................................................................................................................................. 21
OVERVIEW..................................................................................................................................... 21
Thresholds...................................................................................................................................... 21
IIS Services................................................................................................................................. 21
SQL Server 2000......................................................................................................................... 23
Storage Area Network Switches.................................................................................................. 24
Webservers................................................................................................................................. 24
Network Intrusion Detection System............................................................................................... 24
Purpose....................................................................................................................................... 24
Scope.......................................................................................................................................... 24
General / Policy........................................................................................................................... 24
Enforcement................................................................................................................................ 24
Operating System Patches / Service Packs....................................................................................25
Microsoft...................................................................................................................................... 25
AIX.............................................................................................................................................. 25
Linux............................................................................................................................................ 25
Disaster Recovery........................................................................................................................... 25
Appendix A Windows Server Security Checklist.............................................................................26
Linux Security Checklist.............................................................................................................. 33
Page - iv
Introduction
Welcome to the Managed Services Operations Center (MSOC) for Champion Solutions Group (CSG).
As an IBM business partner, the MSOC has been established for the purpose of providing managed
services for customers. This document serves as a centralized repository for all policies, procedures,
and supporting documents that are associated with the day-to-day operations of the MSOC. The
administrators and engineers are provided the ability to quickly and easily navigate to documentation
needed to perform assigned duties accordingly.
Contact Information
Champion
Managed Services Operations Center (To submit a request for service)
Telephone:
(888) 997-7789
Web Portal
https://www.championpulse.com
IBM
Name & Title
Telephone
Page - 1
Escalation Process
The escalation process describes the information flow in case of non-compliance with minimum
service levels. This escalation process applies to severity 1 calls only.
The following escalation sequence is to be utilized if a service is not delivered in a specific timeframe.
ESCALATION
15 Minutes
AND
AND
MSOC Manager,
Jay Kobert
954-646-2784
Page - 2
Infrastructure
Facility Overview
The customers environment is maintained in the Champion Managed Services facility located in the
IBM Atlanta BellSouth eBHC (eBusiness). For the purpose of this document, we will refer to eBHC as
the facility.
The facility maintains several security features for your protection. Security technology may include
biometric readers, cyberlocks, and interior and exterior motion-activated video surveillance cameras
in selected areas.
SITE RESTRICTIONS
Smoking is not allowed in the facility. Unauthorized recording devices, including cameras and
video recorders, are not permitted.
Page - 3
Be prepared to provide the following shipping information to the MSOC when scheduling the
delivery:
Name of carrier
Way bill number
If the shipment is going to be delayed, contact the MSOC to modify the shipping information.
All carriers must be instructed that all deliveries must indicate Inside Delivery.
Shipments must be addressed to:
BellSouth c/o IBM Site Manager
Customer name/identifier
BellSouth Trouble Ticket Number
675 W. Peachtree Street NW
Atlanta, GA 30308-1989
DIRECTIONS
The address is:
675 W. Peachtree Street NW
Atlanta, GA 30308-1989
From Atlanta Hartsfield Airport
1. Follow the airport exit signs to Camp Creek Parkway.
2. Merge onto I-85 N toward I-75 N/ATLANTA.
3. Take the US-19/SPRING STREET exit (exit number 249D) toward US-29/W.
PEACHTREE STREET.
4. Take the ramp toward US-19/US-29 N/US-78/W. PEACHTREE STREET.
5. Turn SLIGHTLY RIGHT onto LINDEN AVENUE NW.
6. Turn LEFT onto W. PEACHTREE STREET NW.
Eating area
Vending machines
Coffee machine
Restrooms
Conference room
Page - 4
Hardware Configuration
Part No.
Qty.
Description
Server Name
Operating
System
Page - 5
Part No.
Qty.
Description
Server Name
Operating
System
Page - 6
Remote Accessibility
PURPOSE
The purpose of this policy is to define standards for connecting to Champion Managed Service's
network and any hosted network environment that Champion manages from any host. These
standards are designed to minimize the potential exposure to Champion Managed Services, and
managed network infrastructures, from damages which may result from unauthorized use, outdated / insecure encryption methods, and unsupported methods of connection to Champion
Managed Services resources. Damages include the loss of sensitive or company confidential
data, intellectual property, damage to public image, infrastructure device and/or Operating system
configurations, and damage to critical Champion Managed Services internal systems.
SCOPE
This policy applies to all Champion Managed Service customers, customer clients, employees,
contractors, vendors and agents that require connection to the Champion Managed Service
network and customer-hosted network environments. Remote access implementations that are
covered by this policy include, but are not limited to, dedicated internet circuits, dial-in modems,
frame relay, ISDN, DSL, VPN, SSH, and cable modems, etc.
GENERAL POLICY
1. The following policies outline details about different methods of accessing network resources
via remote access methods, and acceptable use of Champion Managed Service's managed
networks:
A. Acceptable Encryption Policy
1) ESP-3DES
2) Hash / ESP Authentication = MD5, SHA, AES-128, AES-192, AES-256
3) D-H group = 2
B. Virtual Private Network (VPN) Policy
1) Site to Site = IPSEC
2) Remote Access VPN = PPTP (Microsoft Client)
2. Based on business and application requirements for administration; the following additional
methods are acceptable once a secure tunnel has been established or requests from the
customer with acknowledgement of their insecurities can be established.
A. Microsoft Terminal Services / Remote Desktop Protocol
B. Secure Shell
D. PCAnywhere
C. Telnet
E. RealVNC / VNC
REQUIREMENTS
1. Secure remote access must be strictly controlled. Control will be enforced via submission of a
change request through the MSOC from authorized personnel from the customer.
2. At no time should anyone provide their login or email password to anyone.
3. Customers must submit all encryption details with the specific source to destination for the
customer network. Details include the following: Peer IP Address, Pre-Shared Key, Specific
host / network to Specific host / network destination.
4. Firewall change requests must be submitted to the MSOC. No firewall change requests will
be completed without the completion of a Firewall Rule Request Form.
5. Frame Relay must meet minimum authentication requirements of DLCI standards.
Page - 7
ENFORCEMENT
If any of the above requirements are not met, Champion Managed Services will be responsible
for damages that may be caused from the misuse of remote access policies. Service requests
that do not comply with the policies in this run book may be subject to rejection by Champion
Managed Services.
SCOPE
This policy applies to server equipment supported by Champion Managed Services.
POLICY
Champion Managed Services recommends the utilization of the server security best practices
(see Appendix A). All server security considerations for application management will be defined
by the customer, which may or may not affect SLA availability credits.
All servers supported by Champion Managed Services are owned by the MSOC. The MSOC
is divided into technical verticals to ensure efficient problem resolution. The technical
verticals include:
o Network
o Server
o Storage
o
Data Management
Servers are registered within Champions enterprise management system. At a minimum, the
following information is required to positively identify a given system:
o Server contact(s) and location, and a backup contact
o
Configuration changes for production servers follow the appropriate change management
procedures.
Page - 8
The latest security patches must be installed on all systems as business permits.
Always use standard security principles of least required access to perform a function.
COMPLIANCE
Server Setup
OVERVIEW
The purpose of this policy is to establish standards for the base configuration on server
equipment that is managed by Champion Managed Services. Effective implementation of this
policy will minimize server setup time and ensure stability across environments.
SCOPE
This policy applies to server equipment owned by the customer and managed by Champion
Managed Services. It defines the process of how the logical operating systems are setup in the
Champion managed environment and prepped to accommodate customer SOR.
POLICY
1. Champion Managed Services support and install the following operating system
manufacturers and versions.
A. Microsoft
i. Windows 2000 Server
ii. Windows 2000 Advanced Server
iii. Windows 2003 Standard Server
iv. Windows 2003 Web Server
v. Windows 2003 Enterprise Server
B. Linux
i. Red Had, versions 9.0, AS 2.1, and AS 3.0
C. AIX, version 5.1 or later
D. VMWare
i. ESX
2. Once a server has been integrated with all work order hardware resource allocations (such as
Processors, RAM, NICs, HBAs, etc.), the following processes are followed:
A. Surveys of hardware chassis light indicators are conducted prior to operating system
logical configuration.
B. Successful posts with bios confirmation of installed hardware on servers are conducted
prior to operating system logical configuration.
Page - 9
C. Bios, RAID, etc. firmware versions are verified and updated with the latest available and
recommended versions from the manufacturer.
D. Disk partitions are built as necessary based on customer requirements regarding OS
partitions, and/or data partitions. (Pre-determined by customer and IBM/Champion
Managed Services technical teams prior to build dates.)
E. Servers are installed with the requested Operating System. (Pre-determined by
customer and IBM/Champion Managed Services technical teams prior to build dates.)
F. At the time of server operating systems installation, all available security and critical
updates from the OS manufacturer are applied unless otherwise agreed to in writing by
the customer and IBM/Champion.
G. IPs will be bound to the network interface cards with verified server name instances per
the customer requirements.
H. Operating system resources will be kept on the servers for necessary OS-based
applications. For example: i386 directory will be kept on a windows 2003 server root
drive unless customer requests otherwise.
I. For monitoring reasons, a local or domain account is created for perfmon statistics to be
used for proactive monitoring. The use of a local account or domain account is
determined by the projected network environment that the server will be participating in.
Page - 10
Backup Configuration
The policies discussed in this section are established standards, but can vary per a mutual
agreement to a customer request.
SOFTWARE
Server
TIVOLI Storage Manager (TSM) 5.x
Client
AIX, TSM Client Module 5.x
AIX, Agent Client 3.x
AIX, ORACLE (TDP Agent)
Linux, TSM Client Module 5.x
Linux, Agent Client x.x
Linux, ORACLE (TDP Agent)
Windows, TSM Client Module 5.x
Windows, Agent Client x.x
Windows, ORACLE ***
Licensed, installable physical media is required for ALL of the above.
POLICIES
Policies are rules that are set at the IBM Tivoli Storage Manager server to manage client data.
Policies control how and when client data is stored, for example:
How and when files are backed up and archived to server storage
How space-managed files are migrated to server storage
The number of copies of a file and the length of time copies are kept in server storage
The standard policy consists of a standard policy domain, policy set, management class, backup
copy group, and archive copy group. The attributes of the default policy are as follows:
Backup Policy
Daily incremental backups will be taken
An incremental backup is performed only if the file has changed since the last backup.
Up to two backup versions of a file on the clients system are retained in server storage.
The most recent backup version is retained for as long as the original file is on the client
file system. All other versions are retained for up to 30 days after they become inactive.
One backup version of a file that has been deleted from the clients system is retained in
server storage for 60 days.
DRM Policy
The offsite storage retention policy follows the backup retention policy above.
Page - 11
OVERVIEW
In order to understand how to restore data from backups it is necessary to understand how
backups are organized. Backups are organized on all systems with the following concept. All
non-database related files are backed up under a nightly incremental backup. Database related
files are backed up either using the associated TDP (Tivoli Data Protection) agent, or cold
database backup.
INCREMENTAL BACKUPS
Tivoli Storage Manager (TSM) uses a incremental strategy where the first time an incremental
backup is run against the server everything is backed up. Everything means all data except
database related files, which are excluded in a list maintained in the TSM client path. After the
first backup of all files that have had their timestamp updated are incrementally backed up. This
incremental method ensures that a full restore is available at all times.
INCREMENTAL RESTORES
Incremental restores may be requested, and must include:
Source
Destination
Path
Date of source and path
Node
DATABASE RESTORES
Oracle Database restores in an UNIX environment is handled by the customer via custom RMAN
scripts. The RMAN script connects to the TSM Server and opens channel(s) to tape drives and
passes the restore request between Oracle and TSM.
How and when files are backed up and archived to server storage
The number of copies of a file and the length of time copies are kept in server storage
The standard policy consists of a standard policy domain, policy set, management class, backup
copy group, and archive copy group.
Page - 12
Page - 13
Responding To Alerts
1. Verify that the alert is a legitimate alert and not just a monitoring time out.
2. If it is during working hours, check with team to see if anyone is working on the system.
3. Create a ticket in heat.
4. Contact customer to make them aware of the alert and to see if they are doing anything to the
system. In the event that the alert is acknowledged by the customer as a valid alert, proceed
with the following:
Request from the customer if alert constitutes a priority one ticket.
In the event that it is a priority one ticket, contact the corresponding IBM Project
Manager and inform them of the status of the ticket.
5. Notify the customer that you are creating a ticket and assigning it to an engineer.
6. Contact the appropriate engineering team to notify them of the alert.
NOTE: In the event of the alert being a Priority 1, maintain an hourly contact with the engineer
and customer with status updates until the alert is cleared.
Change Management
The Managed Services division of Champion Solutions Group (CSG) performs monitoring services for
Clients. These services are performed through CSG's Managed Services Operations Center (MSOC),
which is staffed around the clock by MSOC engineers whose primary responsibilities are to:
Monitor, maintain, and protect the MSOC, the Client's environment and networks, and the
equipment
Fulfill the Client's requests for modifications and problem reports as needed
Perform backups and other procedures in compliance with appropriate service level agreements
There are occasions or circumstances that require changes to the Client's environment or monitoring
needs. In order to maintain Client satisfaction by providing timely responses to such requests, this
section provides an overview of the change management policy, and offers guidelines for preparing a
change request.
For the steps required for accepting and monitoring a Client's change request, please refer to the
appropriate procedures that reside on the Champion Managed Services portal
(https://www.championpulse.com):
Operating System
Storage
Network
Database
Page - 14
The Champion Managed Service Operations Center (MSOC) provides the Client with the ability to
submit service requests by utilizing the Helpdesk System on the MSOC Portal via the Internet, or by
telephone. It is recommended that priorities 1 and 2 should be submitted via telephone.
Either process provides the capability of submitting any of the three types of requests:
Administrative Request (AR) Does not affect production systems, and implementation times are
minimal. For example: new user, change password, change rights . . .
Problem/Fix Request (PF) This is similar to an administrative request except that you may be
experiencing some outage problems. For instance, a server or a network may be down.
Change Request (CR) A change request is associated with a change in the environment. For
example, if you have a piece of hardware that is already installed, but is not currently configured in
the infrastructure, a change request is required in order to make the hardware part of the steady state
infrastructure.
Priority 2:
Emergency Request The Client needs a quick response for reasons defined by
the requester. It is Champion's objective to respond to all priority 2 requests
within four (4) hours.
Reminder: Per contract, the Client is permitted only three priority 2 requests per
month, which are included in the Client's monthly fees. There will be
additional charges if the Client exceeds the limit of three priority 2
requests per month.
Priority 3:
Priority 4:
Page - 15
EMERGENCY CHANGES
If a Client needs to make modifications that require a reboot of the system, the Client should call
the MSOC manager to implement this change immediately. The MSOC manager verifies that the
person calling is authorized to request changes of this nature. Once the Client is verified, a ticket
is opened to track the change. The Client receives a ticket number for tracking purposes. The
MSOC manager is the primary contact for the Client and continues to monitor and track the
change request until it is completed.
NOTE: With proper authorization, an MSOC Administrator may perform this function for the
MSOC manager.
The Client's authorized requesters are pre-defined during phase IV of the post-sales
process. During this phase, the MSOC manager is responsible for contacting the Client
to obtain the name(s) of the individual(s) who are authorized to request changes.
IBM
For a current list of the IBM authorized requesters, see Contact Information on the
Champion portal (https://www.championpulse.com).
CSG
NOTE: Emergency requests will be handled on an ad hoc basis and must be thoroughly
documented to include the requester's complete identification, which may
require a specific authorization code.
Page - 16
Any request received after end-of-day Wednesday will not be addressed until Thursday of the
following week.
When a request is received, use the Requests For Service procedure to accommodate the Client's
needs.
IBM
Operations Manager
MSOC Supervisor
Technical Leads
Senior Application/Web Development Lead
NOTE:
AUTHORS NOTE:
The NOTE above refers to a specific timetable for holding the CCB meeting(s). We
need to clarify the frequency at which they are actually held. Is it on Monday,
Wednesday, and Friday? Is it everyday?
There are other sections that may require modifications too. AND, the policies and
procedures will require the same modifications.
Decision Categories
There are three categories into which a decision may be defined:
Approved
Declined
The Client is informed why the change was declined. This may require that a
completely new sales process begin in order to define specific requirements.
Postponed
The Client and the CCB mutually agree to postpone the change until a later date.
This may require that a completely new sales process begin in order to define
Page - 17
specific requirements.
Turnaround Time
While an immediate acknowledgement is provided to the Client, the turnaround time for scheduled
changes is within one week from the day the request is reviewed by the CCB. Many tasks associated
with a request may be performed the same day, or within one week from the day the request is
approved.
A request for change must be received no later than end-of-day on Wednesday to be reviewed and
considered by the Change Control Board on the Thursday of the same week. Any request received
after end-of-day Wednesday will not be addressed until Thursday of the following week.
NOTE: Emergency requests will be handled on an ad hoc basis and must be thoroughly documented
to include the requester's complete identification, which may require a specific authorization
code.
Problem Management
Business Hours
Champions Managed Service Operations Center (MSOC) is available 24 hours per day, 7 days per
week. The MSOC provides you with the ability to submit service requests by utilizing the Helpdesk
System on the MSOC Portal via the Internet, or by telephone.
TELEPHONE
Toll Free: 888-997-7789
Local / Direct: 561-997-7789
CHAMPION PORTAL
1. Log on to the Champion Portal. https://www.championpulse.com
2. Click the Support button at the top of the page.
3. From the left navigation panel, click Helpdesk System. In an effort to protect your records, you
may need to enter your user name and password to enter this restricted area.
Instructions for Creating A Request For Service is provided on the portal under Documents.
Page - 18
Priority 2:
Emergency Request The Client needs a quick response for reasons defined by
the requester. It is Champion's objective to respond to all priority 2 requests
within four (4) hours.
Reminder: Per contract, the Client is permitted only three priority 2 requests per
month, which are included in the Client's monthly fees. There will be
additional charges if the Client exceeds the limit of three priority 2
requests per month.
Priority 3:
Priority 4:
Page - 19
Page - 20
Monitoring Standards
PURPOSE
The purpose of this document is to outline the thresholds that are associated with standard
monitoring activities performed for the Champion Managed Services Clients.
REFERENCE
Please contact the Manager of Managed Services Operations Center to clarify any process within this
procedure and for any concern beyond its scope.
OVERVIEW
Champion Managed Services provides monitoring services based on the standard, global thresholds
for the activities listed below under Thresholds. While the standard monitoring thresholds are listed
below, Champion Managed Services is able to provide modified settings for the alerts associated with
particular monitoring activities. Any alternative parameters will be assessed on an as-needed basis.
Thresholds
The following are the current standard monitoring functions and their respective thresholds.
IIS SERVICES
The following are the thresholds for IIS Services:
PerfMon Status
Warning Level Alert
Page File
Warning Level Alert
>= 90% 5 Minutes
Memory Usage
Warning Level Alert
>= 80% 5 Minutes
Disk Usage
Copyright 2005 Champion Solutions Group
Operations Run Book
Page - 21
CPU Usage
Warning Level Alert
>= 90% 3 Minutes
Ping
Warning Level Alert
PerfMon Status
Warning Level Alert
Processor Time
Warning Level Alert
>= 80% 5 Minutes
Log Used
Warning Level Alert
>= 80% 5 Minutes
Page - 23
Port Status
Warning Level Alert
WEBSERVERS
The following are the thresholds for Webservers:
Available Memory
Warning Level Alert
<=128M
SCOPE
Champion Managed Services provides network intrusion detection on all incoming and outgoing
internet traffic. Signature updates are applied as supplied by the manufacturer within one week
of posting for validation and testing purposes. IDS logs are available to the customer via service
request ticket submission.
GENERAL / POLICY
Champion Managed Services monitors activity logs and responds as alert thresholds are met
pertaining to manufacturer supplied signatures. The IDS system is designed to accurately
identify and classify known and unknown threats targeting your network, including worms, denialof-service (DoS), and application attacks. Multiple detection methods are employed, thus
ensuring comprehensive coverage. The methods include stateful pattern recognition, protocol
analysis, traffic anomaly detection, and protocol anomaly detection. The IDS technology
implemented by Champion Managed Services uses multilayer protection options to prevent an
attack from successfully reaching targets. After the attack is accurately identified and classified,
the system can stop the attack before damage occurs.
ENFORCEMENT
In the event of any type of threat that is deemed to require attention and actions, Champion
Managed Services Network team will assess the activity, deem necessary actions, and contact
the customer. The customer contact will be constructed with information of attack, necessary
Page - 24
AIX
Patches and service packs are to be installed during the specified scheduled maintenance
windows. Please reference the section heading Scheduled Maintenance Windows for more
information.
NOTE: Some patches may require a reboot at this time.
LINUX
Patches and service packs are to be installed during the specified scheduled maintenance
windows. Please reference the section heading Scheduled Maintenance Windows for more
information.
NOTE: Some patches may require a reboot at this time.
Disaster Recovery
All "hardened" IBM facilities are of an enterprise class nature, complete with redundant power
including generator back-ups. All data management is maintained using multiple copies of critical
data to be stored onsite in the hardened facility, as well as an alternative location offsite.
Disaster Recovery is not included in the basic managed service offering and will not be addressed
unless otherwise agreed to in writing between the customer and IBM.
Page - 25
Page - 26
try to hack other accounts as they go to improve their access. If you rename the account,
try not to use the word 'Admin" in its name. Pick something that won't sound like it has
rights to anything.
Consider creating a dummy Administrator account
Another strategy is to create a local account named "Administrator", then giving that
account no privileges and impossible to guess +10 digit complex password. This should
keep the script kiddies busy for a while. If you create a dummy Administrative account,
enabled auditing so you'll know when it is being tampered with.
Replace the "Everyone" Group with "Authenticated Users" on file shares
"Everyone" in the context of Windows 2000 security, means anyone who gains access to
your network can access the data. Never assign the "Everyone" Group to have access to a
file share on your network, use "Authenticated Users" instead. This is especially important
for printers, who have the "Everyone" Group assigned by default.
Password Security
A good password policy is essential to your network security, but is often overlooked. In
large organizations there is a huge temptation for lazy administrators to create all local
Administrator accounts (or worse, a common domain level administrator account) that uses
a variation of the company name, computer name, or advertising tag line. i.e.
%companyname%#1, win2k%companyname%, etc. Even worse are new user accounts
with simple passwords such as "welcome", "letmein", "new2you", that aren't required to
changed the password after the first logon. Use complex passwords that are changed at
least every 60 -90 days. Passwords should contain at least eight characters, and preferably
nine (recent security information reports that many cracking programs are using the eight
character standard as a starting point). Also, each password must follow the standards set
for strong passwords .
Password protect the screensaver
Once again this is a basic security step that is often circumvented by users. Make sure all
of your workstations and servers have this feature enabled to prevent an internal threat
from taking advantage of an unlocked console. For best results, choose the blank
screensaver or logon screensaver. Avoid the OpenGL and graphic intensive program that
eat CPU cycles and memory. Make sure the wait setting is appropriate for your business. If
you can get your users in the habit of manually locking their workstations when they walk
away from their desks, you can probably get away with an idle time of 15 minutes or more.
You can keep users from changing this setting via Group Policy.
Use NTFS on all partitions
FAT and FAT32 File systems don't support file level security and give hackers a big wide
open door to your system. Make sure all of your system partitions are formatted using
NTFS.
Always run Anti-Virus software
Again, this is something that is considered a basic tenet of security, but you would be
surprised at how many companies don't run Anti-Virus software, or run it but don't update it.
Today's AV software does more than just check for known viruses, many scan for other
types of malicious code as well.
Page - 27
TCP/IP
NetBIOS
Helper
Spooler
Server
WINS
Workstation
Page - 28
RPC Service
Event Log
Windows 2000 has not been submitted for C2 certification by Microsoft, so an updated list
of services is not available. What services are deemed unnecessary may vary based on
the function of your server and/or workstations. Please test your specific configuration in a
lab environment before enabling it in your production network. A list of services available in
Windows 2000 Server (as well as their default settings) can be found here
Shut down unnecessary ports
This is a judgment call based on your needs and risks. Workstations aren't normally at risk
behind a firewall, but never assume your servers are safe! A hackers first attempt at
rattling the doors and windows usually involves using a port scanner. You can find out a list
of open ports on your local system by opening the file located at %systemroot
%\drivers\etc\services. You can configure your ports via the TCP/IP Security console
located in the TCP/IP properties (Control Panel > Network and Dial Up Connections >
Local Area Connection > Internet Protocol (TCP/IP) > Properties > Advanced >
Options > TCP/IP Filtering) To allow only TCP and ICMP connections, configure the UDP
and IP Protocol check boxes to "Permit Only" and leave the fields blank. A list of default
ports for Windows 2000 Domain Controllers can be found here
Enable Auditing
The most basic form of Intrusion Detection for Windows 2000 is to enable auditing. This will
alert you to changes in account policies, attempted password hacks, unauthorized file
access, etc., Most users are unaware of the types of doors they have unknowingly left
open on their local workstation, and these risks are often discovered only after a serious
security breach has occurred. At the very minimum, consider auditing the following events:
Event
Level of Auditing
Success, failure
Account management
Success, failure
Logon events
Success, failure
Object access
Success
Policy change
Success, failure
Privilege use
Success, failure
System events
Success, failure
Page - 29
storing all of a users data (documents, spreadsheets, project files, etc.,) on a secured
server, where the data is backed up regularly. Modify the parameters for the "My
Documents" folder to always point to the users network share on a secured server. For
laptop users, enable the "Make available offline" capabilities to synchronize the folder's
content.
Prevent the last logged-in user name from being displayed
When you press Ctrl-Alt-Del, a login dialog box appears which displays the name of the
last user who logged in to the computer, and makes it easier to discover a user name that
can later be used in a password-guessing attack. This can be disabled using the security
templates provided on the installation CD, or via Group Policy snap in. For more
information, see Microsoft KB Article Q310125
Check Microsoft's web site for the latest hotfixes
Nobody writes 30 million lines of code and is going to have it perfect the first time, so
updating service packs and hotfixes can go a long way to plug security holes. The problem
is that hotfixes and service packs aren't regression-tested as thoroughly as service packs
and can come with bugs of their own. You should always test them on a comparable, non
production system before deploying them. Check Microsoft's TechNet Security Page
frequently for the latest hotfixes and decide which ones you need to roll out. Tip: Our home
page at LabMice.net always features Microsoft's latest hotfix to save you time.
Page - 30
HKeyLocal
Machine\SYSTEM\CurrentControlSet\Services\LanManServer\Parameters. For
Servers edit AutoShareServer with a REG_DWORD Value of 0. For Workstations, the edit
AutoShareWks. Keep in mind that disabling these shares provide an extra measure of
security, but may cause problems with applications. Test your changes in a lab before
disabling these in a production environment. The default hidden shares are:
Share
FAX$
IPC$
NetLogon This share is used by the Net Logon service of a Windows 2000
Server computer while processing domain logon requests.
PRINT$
Page - 31
HKEY_LOCAL_MACHINE\SOFTWARE
Page - 32
Key:
HKEY_LOCAL_MACHINE\SYSTEM
Os2LibPath
Value:
delete entry
Key:
HKEY_LOCAL_MACHINE\SYSTEM
Optional
Key:
HKEY_LOCAL_MACHINE\SYSTEM
The changes take effect the next time the computer is started. You might want to update
the emergency repair disk to reflect these changes.
Consider using SmartCard or Biometric devices instead of passwords.
The more stringent your password policy is, the more likely your users will begin keeping
paper password lists in their desk drawers, or taped to the bottom of their keyboard.
Windows 2000 supports these devices, so consider the costs vs. risks of your most
sensitive data.
Consider implementing IPSec
Basically, IPSec provides encryption for network sessions using the Internet Protocol (IP)
and promises to offer transparent and automatic encryption of network connections. For
more information, click here
Page - 33
The following is a recommended security checklist for Linux servers. This document should be
used as a guide to the installation and configuration of Linux Servers in conjunction with an
agreed security plan for the identified systems. The document is designed for use by experienced
system administrators. Some of the settings may be dependant on the patch levels of the
components in use, and therefore differencies may exist between this document and the actual
file paths and access control settings on your machine. Most of the points below can be
addressed by running security scripts made specifically for every system (fx. Harden_suse), but
due to the general nature of these scripts or applications it is not advised to use them without
proper testing.
Initial Installation
Install the Latest Patches
In most cases distribution vendors will provide an update facility for the distribution of patches.
The latest system patches should be installed prior to operational deployment. Particular
attention should be paid to those network services that the operating system makes available to
remote clients (eg: Web (Apache), Mail (sendmail/postfix/imapd), and so on.
It is also recommended that the system be updated with newly realeased patches as soon as
operational circumstances allow.
Bypassing the vendor, and installing patches directly from the application provider (eg: from
apache.org) may also be appropriate in some circumstances, where the problem in question is
significant, or the distribution vendor response to security issues is poor.
Latest Patches can be found at
Debian ftp://ftp.debian.org/debian/dists/stable-proposed-updates/
RedHat ftp://ftp.redhat.com/pub/redhat/linux/updates
SuSe ftp://ftp.suse.com/pub/suse/i386/update/
In order to stay updated with latest vulnerabilities on Solaris systems and patches required for it
Sun issues a security bulletin. To receive security bulletins directly from:
Debian http://www.debian.org/MailingLists/subscribe#debian-security-announce
SuSe http://www.suse.com/us/private/support/mailinglists/index.html
File Systems
Per default Linux mounts remote or local filesystems are mounted with read-write privileges with
possibility to have suid or sgid files.
In order to prevent that filesystems that don't require extra privileges should be limited.
In /etc/fstab there should be an entry nosuid or noexec for external devices like cdrom or
filesystems in that specific row in the fourth field .
Time Settings
Copyright 2005 Champion Solutions Group
Operations Run Book
Page - 34
All the servers should have the same time settings in order to be able to evaluate logs properly.
1. There should be a time-zone entry in /etc/sysconfig/clock containing
ZONE=Europe/Berlin. Or in Debian /etc/timezone should contain Europe/Berlin
2. There should be a NTP system installed with timeservers configured for synchronisation
(fx. /etc/ntp.conf should contain server a.b.c.d prefer)
Timeservers in OssBss are 10.130.200.70 or 10.130.200.80
o In Management network 10.10.8.70 or 10.10.8.80
Page - 35
NOTE:
The users may see both the /etc/motd and the /etc/issue messages when they login.
SSH daemon should be configured to display the message by putting this line into sshd_config:
Printmotd yes
Privileged Account Login Source
In order to ensure security of the root account there should be limitations placed on the source of
login.
Root should be able to log into the system only locally (via console or with su command).
This can be ensured by :
1. In /etc/nologin there should be all the administrative accounts
2. In /etc/security/access.conf there should be a line
-:ALL EXCEPT wheel shutdown sync:console
-:ALL EXCEPT root:ALL EXCEPT console
3. In sshd_conf put line PermitRootLogin no
Network driver configuration
Make the following adjustments to the /etc/sysctl.conf to protect the machine from some types of
network attacks.
1.
2.
3.
4.
5.
6.
7.
net.ipv4.ip_forward = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
Disable multicasting:
ifconfig [interface] -allmulti -multicast
System Network Services
Network Services Summary
All the unnecessary network services should be switched off.
1. 1./etc/inetd.conf should not contain any entries unless specifically required by
applications.
Here is a quick rundown of the risks associated with services started in /etc/inetd.conf :
ftp: enables an FTP server that introduces a variety of insecurities and is the cause of many
intrusions. Disable this and use SSH instead to transfer files between systems.
telnet, shell, login, exec: allows users from other systems to log into and run commands on your
machine. This is useful, but the more useful something is, the more likely it is that someone will
find a way to exploit it. Disable these services and, if you do need to allow remote logins, use
SSH instead.
Copyright 2005 Champion Solutions Group
Operations Run Book
Page - 36
comsat: a daemon which is used to notify users of newly arrived email. There are alternate
means of doing the same thing, and there are occasional rumors of security problems with
comsat. Unless you have some overwhelming need for this, turn it off.
talk: allows users to communicate by typing at each others terminals.
uucp: Nobody uses uucp anymore - disable this. While you are at it, you may as well turn off
execute permission on the uucp-related shell commands.
tftp: FTP without any security. This should be needed only if your system will be used for booting
workstations. If this is the case, you must invoke the daemon with the -s flag, as in:
tftp dgram udp wait root in.tftpd -s /tftpboot
If you don't, tftp can be used to retrieve any file from your system, anonymously. Also make all the
files in the bootfile directory read-only. Finally, restrict access to the service using TCPwrappers
and
IPFilter/IPChains.
finger: this gives out information on who is loggedin, or people's phone numbers and offices.
Unfortunately this information can be used by a potential intruder to find accounts to attack. You
may wish to disable this, run a custom finger daemon, or restrict access to it using TCPwrappers
and
IPFilter/IPChains.
systat, netstat: these services give out information about your system. The comments for finger
apply to these.
time : Gives out the system time to any remote host that asks for it. Probably safe but can be
disabled without impacting the system.
echo, discard, daytime, chargen: these are used for testing, and are generally safe, though
there have been reports of TCP packets with forged IP source addresses being used to trick a
system into sending echo packets to itself, causing a packet storm on the local ethernet segment.
Disable them and only turn them on while testing.
rexd - this is the Remote Procedure Call mechanism. It has minimal authentication, so disable it
and use SSH instead.
walld: allows people to send messages to all logged in users. Useful, but easily abused.
ttdbserverd (tooltalk): used by some convenient desktop elements but not important from a
system operation standpoint. Some versions of this service contain serious remote exploits and
should be disabled (dsabling this service causes virtually no operational degradation).
rpc.cmsd (calendar manager) : used to share calendar information over the network but not
important from a system operation standpoint. Some versions of this service contain serious
remote exploits and should be disabled.
others : Other services such as sadmind (once found to be vulnerabale to remote root exploit)
and kerberos can be disabled without impacting the system.
There should not be any services listening on the network unless required by applications.
Fx. XFree86 listens on port 6000+n, where n is the display number. This
connection type can be disabled with the -nolisten option (see the Xserver(1) man
page for details).
File Transfer
Page - 37
Ftp service should be disabled and secure ftp or secure copy should be used. /etc/ftpusers
should contain all the account names except those that should be allowed to access the system
via FTP.
Electronic Mail
There should be no email service running on the system for local use (email servers have email
agents installed as application and should follow the application security part of this document).
Domain Name Service
There should be no DNS servers running on the system (DNS servers should be treated as an
application and should follow application security part of this document).
Remote shell / copy services
All the systems should have the latest SSH installed in order to allow remote administration of the
server with encrypted interconnection. Sshd_config should contain also these features:
Protocol 2
UsePrivilegeSeparation yes
ServerKeyBits 768
SyslogFacility AUTH
LogLevel INFO
LoginGraceTime 600
PermitRootLogin no
StrictModes yes
RSAAuthentication yes
PubkeyAuthentication yes
RhostsAuthentication no
IgnoreRhosts yes
RhostsRSAAuthentication no
HostbasedAuthentication no
PermitEmptyPasswords no
PasswordAuthentication yes
PrintMotd yes
PrintLastLog no
MaxStartups 10:30:60
ReverseMappingCheck yes
Page - 38
/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
Page - 39
# Account management
login account required
su account required
other account required
/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
# Session management
su session required
other session required
/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
# Password management
other password required
/usr/lib/security/pam_unix.so.1
If http authentication is supposed to be used then there should be extra entries specifying http
authentication facility.
File and Object Access
Umask settings
Set user file creation mask
In each of the files /etc/csh.cshrc and /etc/profile, there should be an invocation of the umask
command. This invocation should be positioned immediately after the initial comments. The
value passed to umask is an octal mask of the mode bits that are not set when a file is
created. Acceptable values are 022, 026 (suggested) and 027. Each of these has advantages
and disadvantages. Please read the umask manual page prior to selecting the value to be
set.
Set FTP file creation mask
Add the following line at the end of the /etc/proftpd.conf file. This line contains the default umask
value that will be used by FTP when a file is created.
UMASK=022
Set daemon umask umask 022
In /etc/init.d/functions add a line umask 022 (redhat)
In /etc/rc.status add a line umask 022 (others)
Permissions tightening
1. Minimize file or object access to only groups or users that will access them (fx. Oracle
daemon should be executable by user oracle only)
2. Crontab access restrictions should be put into /etc/cron.allow (debian, redhat) or
/var/spool/cron/allow (SuSe).
3. At access retrictions should be put into /etc/at.allow.
Page - 40
Page - 41
Page - 42