1

Legend
Pattern A: 5 Dedicated Environments / Hot DR
Location / Site A
In this pattern, a complete application stack --- necessary for the successful running of a single software system --- is replicated into
Location / Site B five separate, dedicated environments for the purpose of initial development, testing and quality assurance, staging software to
production, production, and disaster recovery. This pattern is recommended for all software systems at Capital Group.

NOTE: If server virtualization is employed, it is recommended that environmental integrity and tier levels be maintained on server
Development Test UAT / ITE Production DR hardware. That is, Development and QA/ITE environments should not co-exist with Production on the same box.
ADVANTAGES DISADVANTAGES
Primary Secondary
The DR environment can be utilized immediately in case there Cost
is a threat to our production business environment.

Dev BSA ARS User Dev BSA ARS User Dev BSA ARS User Dev BSA APS User Dev BSA APS User

This is a well established pattern with numerous examples both inside and outside of GIG.

Legend
Pattern B: 4 Shared Environments / Cold DR
Location / Site A In this pattern, the DR environment is shared with the integrated testing / quality assurance environment in a separate location from
Location / Site B
the remaining infrastructure.

Because the Test environment will need to be cleaned up in order for DR to occur, this pattern is not recommended unless
clustering technologies can help automate the staging and de-staging of both environments.
Development Test Production UAT / ITE DR ADVANTAGES DISADVANTAGES
Cluster-Automated Environment
Primary Secondary Staging / Destaging Better server utilization in the DR environment DR environment cannot be utilized until QA/ITE configurations
are removed.
There may be a performance impact to the QA/ITE
environment if inter-operability with external systems in a
different site is required.

Dev BSA ARS User Dev BSA ARS User Dev BSA APS User Dev BSA APS User

This is a well established pattern with numerous examples both inside and outside of GIG.

1-Environment Pattern D: iSCSI or FCoE / IP SAN Pattern D: Virtual Private Networking

INDEX OF Pattern A: 5 Dedicated Environments / Hot DR Pattern E: Hardware RAID Pattern E: Virtual IP Address (VIP)
INFRASTRUCTURE Pattern
2-Server & OS
B: 4 Shared Environments / Cold DR Pattern
Pattern
F:
G:
Software RAID / Volume Management
File System
Pattern F:
Pattern G:
Quality
Network
of Service (QoS)
Deceleration
ARCHITECTURE Pattern A: Mainframe Computing Pattern H: Registry / Directory Services Pattern H: Network Acceleration
Pattern B: Dedicated Infrastructure / Stand Alone Srvr. Pattern I: Databases (RDBMS / ODBMS) Pattern I: Content Load Balancers
PATTERNS Pattern C: Blade / Thin Servers Pattern J: Data Grid / Distributed Database Pattern J: Network Management Services
Pattern D: Hardware Partitioning Pattern K: Storage MultiPathing 5-Availability
Pattern E: Bare Metal Virtualization Pattern L: File System SnapShots Pattern A: Application-Governed HA/DR
Pattern F: Para-Virtualization Pattern M: Software Replication Pattern B: Infra. Hardware-Governed HA/DR
Pattern G: Guest OS Virtualization Pattern N: Local Hardware Replication Pattern C: Infra. Software-Governed HA/DR
Pattern H: OS Partitioning Pattern O: Remote Hardware Replication Pattern D: Manual HA/DR
Pattern I: Virtual Container Migration (Ex) Pattern P: Storage Virtualization (pic) 6-Recoverability
Pattern J: Cluster Computing / Server Farm Pattern Q: Data DeDuplication Pattern A: No HA or DR (txt)
Pattern K: Grid Computing Pattern R: Tiered Storage / ILM Pattern B: DR without HA (txt)
Pattern L: Cloud Computing / Utility Computing Pattern S: Content-Addressed Storage (CAS) (pic, txt) Pattern C: HA and DR (txt)
Pattern M: Virtual Desktop / Remote Display Pattern T: RDBMS Archiving (pic, txt) Pattern D: HA and DR-with-HA (txt)
GIG 3-Storage 4-Networking 7-Notification
Infrastructure
Architecture Pattern A: Direct Attached Storage (DAS) Pattern A: TCP/IP over Ethernet Pattern A: Network Monitoring Tools (pic, txt)
Services Pattern B: Network Attached Storage (NAS) Pattern B: Network Devices Pattern B: Server Monitoring Tools
Pattern C: Fibre Channel Storage Area Networking Pattern C: Virtual LAN (VLAN) Pattern C: Application Monitoring Tools (pic, txt)

DRAFT Version 0.97 09/01/09 SGM

 2

Pattern A: Mainframe Computing

In this pattern, data processing is performed in powerful, centralized computers, whereas information is presented to users via
Serial Connection dumb terminals with no ability to cache or otherwise process data, or via Unix application servers to a personal computer.
Mainframe computing has largely been leveraged by large organizations for their critical applications and bulk data processing.

Dumb Terminal / ADVANTAGES DISADVANTAGES

Thin Client High acquisition costs for both hardware and operating
Redundant internal engineering
High reliability and security systems
Ethernet Little market competition
Designed to handle very high volume input and output (I/O)
and emphasize throughput computing Not ideal for decentralized computing environments
Ability to run or host multiple operating systems
UNIX Server simultaneously (i.e., as virtual machines)
Mainframe Can replace hundreds of smaller servers
Personal Computer
This is a well established pattern with numerous examples in GIG, including MARS, PA, TPS, and NAV.

Pattern B: Dedicated Infrastructure / Stand Alone Server

This pattern is generally recommended for infrastructure software, which other internal servers and applications require in order to
function properly (i.e., DNS, LDAP, messaging components, etc).

NOTE: Application stacking --- consolidating unrelated applications within a single Operating System environment --- is not
recommended due to the potential for security vulnerabilities as well as resource contention.

ADVANTAGES DISADVANTAGES
No virtualization latency May contribute to server under-utilization
May contribute to server proliferation within the enterprise

Server

This is a well established pattern with numerous examples both inside and outside of GIG.

Pattern C: Blade / Thin Servers

Blade servers are discrete, all-inclusive compute servers residing on a single silicon board, designed primarily for enclosure in a high
density, rack-mounted chassis. Blade servers only offer computer CPU and memory, and are lightweight because hard drives,
power supplies, and I/O devices are connected to them externally.

ADVANTAGES DISADVANTAGES
hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 hp bc1500 Very high densities (i.e.., hundreds of discrete servers per Blades can exacerbate a datacenters heat distribution
rack space) are achievable problem (have high power consumption)
Blade Server Enclosure
Blades are cheaper to manufacture because they are Moving parts are harder to replace due to density
HP BladeSystem PC Blade Enclosure
(Fully Populated Blades)
with 20 RDP Licenses

simplified computers
They offer higher compute-power per square foot of
datacenter space, from traditional standalone servers
1Gb rj45 or SFP 1Gb rj45 or SFP BladeSystem PC Blade Admin 100t and Serial 1Gb rj45 or SFP 1Gb rj45 or SFP
43 44 42 45 46

Blade Server Enclosure

Switch

(One Blade Removed)

Not a CG standard.
 3

Pattern D: Hardware Partitioning

This pattern is characterized by the electrical segregation of server components --- where specialized, vendor-supported hardware is
available --- to increase server utilization, while maintaining proper isolation of unrelated software components. It is recommended
whenever and wherever hardware specifications vastly exceed application requirements.

OS OS
1 2
ADVANTAGES DISADVANTAGES
Cost (increased hardware utilization) May contribute to OS proliferation within the enterprise
No virtualization latency May not be able to add / remove components on the fly
True separation of duties for each subsystem (CPU, Memory, (hardware vendor and OS dependent)
CPU CPU CPU CPU CPU CPU and Disk I/O)
I/O I/O Operating systems are independent of one another, and may
be at different version levels
Hardware Partitioning
i.e., HP nPARs, Sun multi-domain servers, IBM HMC Both HPUX nPARs and Suns multi-domain servers are used extensively both inside and outside of GIG.

Pattern E: Bare Metal Virtualization

In this pattern a lightweight Host OS utilizes a software Hypervisor that the Guest OS is unaware of. This pattern is highly
recommended for Test/Dev and QA environments, and also acceptable for DR environments. Where Production environments are
involved, however, it is important to carefully review performance requirements, recoverability tier levels, and general Host platform
availability.
OS OS OS
1 2 3
ADVANTAGES DISADVANTAGES
Cost (increased hardware utilization) OS environments are subject to Host platform component
Allows for virtualization on non-specialized / commodity failures
Virtualization Software High virtualization latency: hypervisor software can consumes
hardware
CPU CPU CPU CPU CPU CPU Operating systems are independent of one another, and may up to 20% of system resources (vendor dependent)
be at different version levels May contribute to OS proliferation within the enterprise
I/O

Bare Metal Virtualization

i.e., VMware ESX Server, HP vPARs, HP Integrity VMs, Egenera PAN Manager,
Sun xVM Server/OpCenter, IBM LPARs VMwares ESX Server is utilized by the following application teams: XXX.

Pattern F: Para-Virtualization
In this pattern the lightweight Host OS utilizes a software Hypervisor with special APIs. Significantly, the Guest OS realizes it is
being virtualized, and cooperates in the virtualization process. This pattern is highly recommended for Test/Dev and QA
environments, and acceptable for DR environments. Where Production environments are involved, however, it is important to
carefully revisit performance requirements, recoverability tier levels, and general Host platform availability.
OS OS OS
1 2 3

ADVANTAGES DISADVANTAGES
Virtual Hardware API Cost (increased hardware utilization) Not all Oses (namely Windows) come with drivers to take
Virtualization Software Higher performance than in Bare Metal Virtualization advantage of the Hypervisor APIs
Allows for virtualization on non-specialized / commodity However, with some hardware assistance (i.e., Intel V and
CPU CPU CPU CPU CPU CPU
hardware AMD-V), there is less emulation and interception by the
I/O hypervisor, and even non-cooperative OSes function properly
Operating systems are independent of one another, and may
be at different version levels May contribute to OS proliferation within the enterprise
Para-Virtualization
i.e., Xen, VirtualIron Not a CG Standard. Xen is losing industry acceptance due to a proliferation of other technologies.
 4

Pattern G: Guest OS Virtualization

In contrast to Bare Metal Virtualization, virtualization is achieved in this pattern through a permanent, full-featured Host OS and
specialized third party software (no Hypervisor is involved). This pattern is recommended where short-term virtualization is
sufficient, i.e., to streamline the development lifecycle for testing purposes, and is often utilized on developers laptops and
workstations. However, this pattern should not be used to circumvent migration integrity associated with dedicated server
OS OS OS environments.
2 3 4
ADVANTAGES DISADVANTAGES
Cost (increased hardware utilization) Guest OS performance is far from optimal
Virtualization Software
Allows for virtualization on non-specialized / commodity
OS 1 hardware
CPU CPU CPU CPU CPU CPU Operating systems are independent of one another, and may
be at different version levels
I/O
Useful to streamline the development lifecycle
Guest OS Virtualization
i.e., VMware Server, VMware Workstation, VMware Fusion, Sun xVM VirtualBox,
VMware Workstation is used extensively in GIGs Technology Engineering group.
MS Virtual PC / Server, Mac Parallels Desktop, SWSoft Virtuozzo

Pattern H: OS Partitioning
In this pattern the Operating System itself is responsible for erecting barriers between applications and the OS, and a single kernel
runs and manages all virtual application partitions. (No Hypervisor or third party virtualization software is involved.) This pattern is
highly recommended for Test/Dev and QA environments. Where Production environments are involved, it is important to carefully
revisit performance requirements, recoverability tier levels, and general Host platform availability.
App App App
1 2 3
ADVANTAGES DISADVANTAGES
Cost (increased hardware utilization) An OS-level update or patch affects all application containers
Very high performance / far more efficient than other
OS
virtualization types
CPU CPU CPU CPU CPU CPU

I/O

OS Partitioning
i.e., Solaris Containers, BSD Jails, Linux Vserver, Microsoft Softricity Solaris Containers are being used for Production in AFISG. GIG is developing a standard for its deployment.

Pattern I: Virtual Container Migration

In this pattern, a virtual container --- consisting of an operating system instance, one or more applications, the associated storage,
App App App App and cached memory buffers --- is moved from one physical server to another for performance, high availability, or disaster recovery.

OS OS OS OS
Some vendors support live migration of running virtual machines with zero downtime and continuous service availability.

ADVANTAGES DISADVANTAGES
Automatic allocation of virtual machines within resource pools Cost
Enables hardware maintenance without disruption to business
Migration Software operations / scheduled downtime
Virtualization Software Virtualization Software

Hardware Hardware

Virtual Container Migration

i.e., VMware VMotion VMware's VMotion technology is used within GIG by the ABC and XYZ application teams.
 5

Interconnect Pattern J: Cluster Computing / Server Farm

This pattern is a form of distributed computing whereby a single, specialized application service is delivered by a cluster of two or
more networked, dedicated, tightly-coupled computers. This pattern is recommended when improved performance and/or availability
are required, and where cost-effectiveness is more important than procuring larger computers of comparable speed or availability.

Basic clusters can be extended into larger networks to host web farms, database farms, VMware farms, etc.
ADVANTAGES DISADVANTAGES
Improved availability Increased complexity of administration
Improved performance (depending upon implementation) Generally not an option for servers across different locations
i.e., HPs MC Service Guard, Veritas Cluster Server,
Storage
Switches Microsoft Cluster Server Improved scalability
Cost effective

Storage Storage Storage Tangosols RDBMS cache clustering technology is currently utilized by the ETP and FPAC teams.

Pattern K: Grid Computing

This pattern is a form of parallel computing whereby a virtual cluster is composed of two or more networked, loosely-coupled
systems, acting in concert to perform large tasks. Grids tend to be more loosely coupled, heterogeneous, and geographically
Task dispersed than other forms of cluster computing. Therefore, nodes may drop from the network intermittently.

Grid can be achieved by third party middleware applications and software libraries, and are typically used for computationally-
Client intensive scientific, mathematical, and academic problems. Corporations often deploy grids on commodity hardware.
Control Server
ADVANTAGES DISADVANTAGES
All of the above-mentioned advantages Lack of a high speed interconnect
Best suited for problems which can easily be parallellized: Not suited for serial tasks or tasks which cannot be broken up
broken into smaller tasks which require little communication Grid nodes may become unavailable unpredictably
between worker threads
Grid nodes must trust the Control Server not to compromise
the integrity of other running applications, transmit private
data, etc.

i.e., SunOne Grid Engine, Googles data center, Human

Genome Project Not a CG Standard.
Grid Node

Pattern L: Cloud Computing / Utility Computing

Cloud Computing is a form of distributed, utility computing whereby information is stored permanently on servers connected to the
Internet or on an intranet (cloud), and IT capabilities are cached temporarily and provided to clients as a service. Cloud Computing
Service is a general concept that incorporates software as a service (SaaS), Data as a service (DaaS), and dynamic web technologies (Web
Software
2.0). If your client needs specific software or hardware --- beyond a web browser --- youre not cloud computing.

Web Client
ADVANTAGES
Hardware
Allows users to utilize services without knowledge of,
expertise with, or control over technology infrastructure.
Allows enterprises to procure IT resources as a utility, instead
Storage of investing in a large scale computing infrastructure.
Centralized software are easier to upgrade than distributed
thick client applications
DISADVANTAGES
Application providers must ensure their services are highly
available and scale to demand
True utility computing may involve internationalization, while
cross-border privacy and legal processes are still immature
Traditional computers are not optimized for cloud computing,
though new low-power and low-cost computing appliances
are emerging

The Internet i.e., Google Apps, Zoho, SalesForce.com, Blist, SlideRocket,

Zimbra Not a CG Standard.
 6

Pattern M: Virtual Desktop / Remote Display Delivery

Virtual Desktops are actually a user interface (UI) pattern --- usually implemented by an OS, but sometimes via third party software -
-- whereby a computers desktop environment is expanded beyond what is provided by the systems physical hardware display. The
additional virtual space allows a user to place application windows, icons, and other GUI elements in a manner that reduces clutter.
Some windows managers include remote delivery mechanisms which can be useful for computing across a WAN, or in securing
desktop applications. Other remote desktop tools are not integrated into window managers or the OS at all.
ADVANTAGES DISADVANTAGES
Affords the user increased workspace Connection Brokers may be required for remote delivery of a
Hotkey combinations allow a user to quickly switch between Windows display.
HP BladeSystem PC Blade Enclosure with 20 RDP Licenses different screens

1Gb rj45 or SFP 1Gb rj45 or SFP BladeSystem PC Blade Admin 100t and Serial 1Gb rj45 or SFP 1Gb rj45 or SFP
43 44 42 45 46
Switch

Unix systems utilize the X Window windows manager to implement popular desktop environments including CDE, KDE, Sun Java
Currently a CG standard, but may newer technologies be relevant for computer systems on the Trading floor.
Desktop, and Gnome. VNC, Citrix, and VMware Virtual Desktop manager are remote delivery mechanisms for Windows.
 7

Legend
TCP/IP
Pattern A: Direct Attached Storage (DAS)
SCSI Cable
This pattern is pervasive in IT: the vast majority of client computers have internal hard drives, and USB drives are now prevalent.
Where servers are concerned, however, DAS refers to the addition of storage media through SCSI cabling. However, this pattern is
generally recommended for server platforms only where static web content is involved.
TCP/IP Network
Router
ADVANTAGES DISADVANTAGES
Easy Low reliability
Fast (40-160 Mbps) No re-allocate or re-deploy / Poor storage utilization
Client 1 Client 2
Known No central management
Internal Internal Internal
Secure Limited scalability
Disk 1 Disk 2 Disk 3
No additional drivers necessary Traditionally difficult to implement RAID on DAS
Maintained by System Administrators, not Storage Team
NOTE: Internal Disk 2" cannot be re-deployed easily from Client 1 to Client 2 within a secure Data Center environment.
This is a well established pattern with numerous examples both inside and outside of GIG.

Legend Pattern B: Network Attached Storage (NAS)

TCP/IP
SCSI Cable
Network Attached Storage devices are computer subsystems dedicated to providing both data storage and file systems services to
other servers and applications. Significantly, NAS is characterized by file level - not block level --- storage. In this pattern entire
files are accessed over an Ethernet network using either the SMB/CIFS (for Windows) or NFS (for Unix) protocol.

Client
ADVANTAGES DISADVANTAGES
All of the above-mentioned advantages Slow (10 MBps-1 Gbps)
TCP/IP Network
Fast re-allocate and re-deploy of storage If only a portion of a file is changed, the whole file must be
Router
Good storage utilization NAS I/O slows down the network / high overhead on CSDMA
Centralized storage management (TCP/IP) networks
Name space cannot traverse NAS devices
Better reliability than DAS
Point-in-time / snapshot copies of data
NAS Head
Clients
This is a well established pattern with numerous examples both inside and outside of GIG. In the GIG LOB, see the
Capital Group uses NetApp products from the low-end FAS-2000 to higher end FAS-3000 filers. PRS application (GRIDS team) for references. In the AF LOB, see the AWD and DAVAST applications.

Legend Pattern C: Fibre Channel Storage Area Networking (FC SAN)

TCP/IP
Fibre Channel SAN Storage Area Networking is an infrastructure pattern whereby servers are connected to remote storage devices in a way that
appears local to the computer operating system. SAN therefore leaves file system concerns to the client. Most storage networks
utilize SCSI, which is the communication protocol used by applications to talk to block storage devices, which is what a disk really is.

This pattern is highly recommended for storage of RDBMS data

Server Client
ADVANTAGES DISADVANTAGES
All of the above-mentioned advantages Cost
TCP/IP Network
Fibre Channel
Fastest (200 MBps+), Highest security Significant complexity / administration
Router
Switch Centralized backup Backups and mirrors require a block by block copy, even if
Growth of storage does not burden the server (as DAS does) blocks are empty
Does not slow down the TCP/IP network (as NAS does)
SAN Storage Array
Better reliability than DAS and NAS
The SAN
Server CG uses mid-range Hitachi storage where manual storage recovery is acceptable; and high-end EMC products for
Includes Host Bus Adaptors installed in Servers, Fibre Channel cabling and switches, Clients
SAN Storage Arrays, and SAN Disk. business critical applications, and where synchronous replication and active-active load balancing are required.
 8

Pattern D: iSCSI or FCoE / IP Storage Area Networking (IP SAN)

iSCSI Disk Array Gigabit Ethernet
Switch
Application Server
IP SAN
IP Storage Area Networking (IP SANs) allows computer clients to access data storage services over Ethernet. It is alternatively
referred to by its Internet SCSI (iSCSI) protocol or as fibre channel over Ethernet (FCoE). Stated differently, IP SANs are SANs that
are accessed like NAS, in that they offer block level storage I/O (SCSI-3 or Fibre Channel protocol) over standard TCP/IP.
Fibre Channel
Switch This pattern are recommended for remote offices to access a centralized corporate storage infrastructure.
ADVANTAGES DISADVANTAGES
Tape Library Fibre Channel Disk Array
Less complex than traditional FC SANs As reliable or unreliable as your network
Suitable for small to medium sized companies to connect to a Cost: A dedicated iSOE (iSCSI Over Ethernet) card is
centralized storage infrastructure. recommended for access to remote storage
Because iSCSI storage devices appear as local resources to
the computer operating system, its implementation may entail
the use of 3rd party Volume Managers and File Systems,
Backup Server Application Server
(though these are not technically required).
FC SAN Not a CG Standard.

Pattern E: Hardware RAID

A Redundant Array of Independent or Inexpensive Disks (RAID) is a hardware technology whereby physical disk characteristics are abstracted from users and
Logical applications, and manipulated in order to achieve higher levels of reliability, performance, and larger storage capacity. In effect RAID arrays distribute data across
Disks several disks, while the user, application, and operating system see the array as a single disk.
200 GB 100 GB
Various RAID levels exist and each configuration has its own requirements and advantages. RAID 0 stripes data across volumes for performance, RAID 1 mirrors
data for protection, and RAID 5 stripes data with parity in order to achieve both. RAID 0+1 and 10 are also popular, while other RAID levels are proprietary.

ADVANTAGES DISADVANTAGES
Data protection can be seamless to applications, users, and Not all RAID levels achieve data protection
A1 A2 A1 A1
operating systems Some RAID levels compromise total storage for reliability
Physical A3 A4 A2 A2
Disks A5 A6 A3
RAID 0: fast reads and writes (without redundancy) Many RAID levels are designed to hedge against a specific
A3
RAID 1: fast reads type of risk or failure, and may not mitigate actual risks
100 GB 100 GB 100 GB 100 GB encountered
RAID 5: slow writes; fast reads (good for small I/O transfers)
RAID 10: fast reads; fast writes
RAID 0: Striping RAID 1: Mirroring
CG uses a mixture of RAID 1, RAID 5, and RAID 10 on EMC and Hitachi storage arrays. Highly transactional
i.e., RAID 0, 1, 5, 0+1, 10, NetApps RAID-DP
applications at CG --- such as those within GIG and CRMC investment systems --- take advantage of RAID 1.

File /
Pattern F: Software RAID / Volume Management
Systems
/home
Volume Management is a logical storage manipulation technology used to more flexibly allocate space on mass storage devices,
than conventional disk partitioning schemes allow. Volume managers are especially used to concatenate storage across physical
Free Space Logical Volume / Logical Volume /home hard disks, write stripes across them for speed, and help manipulate the location of information while it is being used by or within
Volumes
8 GB 2 GB 20 GB computer systems. Like file systems, they are distinguished through a host of specialized features.

Volume Logical Volume Group ADVANTAGES DISADVANTAGES

Groups (10 GB x 3) There are very many, but most are specific to each volume There may be a small performance setbacks for using them
manager. Software RAID consumes host CPU cycles and is therefore
May help to implement software RAID not as efficient as hardware RAID
Sub Disk Partition 0
Physical May help to implement data snapshots Shifts the burden of storage management to the server
Disks Sub Disk Partition 1
Whole Disk May help to move hard disk devices between servers running administration team
Sub Disk Partition 7 different Operating Systems

10 GB 10 GB 10 GB
i.e., Veritas VxVM and CVM; Solaris Volume Manager and Disk Suite; Windows LDM; HPs LVM This is a well established pattern with numerous examples both inside and outside of GIG.
 9

Pattern G: File System

/ File Systems provide a method for storing computer files and data on hard drives. They organize data into an array of fixed-size file
system blocks, which correlate to sectors on the underlying storage device. Effectively, therefore, file systems are indexes which
/bin /etc /usr /var FS Blocks /opt /oracle maintain the physical location of files. However, file systems are distinguished through a host of specialized features including:
maximum number of files, maximum addressing capability, journaling capabilities, shared file I/O, security, etc. Distributed file
UNIX Server systems also exist, and provide access to a file name space which spans multiple physical storage arrays.
/app1 /u001 /u002 ADVANTAGES DISADVANTAGES
Disk Sector
There are very many, but most are file-system specific. There may be a small performance setback for some (highly
Distributed file systems are a type of Storage Virtualization transactional) databases that write to file systems instead of
raw disk devices directly. However, file systems offer a layer
technology
of abstraction useful in backup and recovery.

Disk 1 Disk 1 Disk 2

i.e., CDFS; Veritas VxFS and CFS; Solaris NFS, UFS, ZFS, and PROCFS; Windows FAT, FAT32, and NTFS;
Apples HFS2; Red Hat RiserFS, GFS, EXT3; Oracles BTRFS and OCFS2. This is a well established pattern with numerous examples both inside and outside of GIG.

Pattern H: Registry / Directory Services

LDAP
Client Local A Directory Service is a software system that stores, organizes, and provides lookup access services to its clients. Unlike a DBMS,
Directory
Data however, Directory Services is read-intensive, not write-intensive. Traditional directory schemas do not have many-to-many
relationships.

ADVANTAGES DISADVANTAGES
Client LDAP Remote
Gateway Centralized access to information Performance may be limited to network speeds
Data

LDAP-Enabled
Directory Service Remote
Client Data

i.e., DNS, NIS+, LDAP, Microsoft Active Directory and NTDS, Red Hat Directory Server, Apples Open Directory,
Apache Directory Server, Oracle Internet Directory, OpenDS, OpenLDAP, Novells eDirectory This is a well established pattern with numerous examples both inside and outside of GIG.

Pattern I: Databases (RDBMS / ODBMS)

Database Management Systems (DBMS) are computer software which is used to store, update, and retrieve information organized
as a structured collection of records. The smallest logical construct of this information is the database block, which may in turn be
Relation
Relation
stored directly on physical disk, or within other logical constructs such as volumes and file systems. DBMSes may use any of a
variety of database models, including but not limited to the relational model (RDBMS) and object-oriented model (ODBMS).
Table A Table B Table C Table A Table B
ADVANTAGES DISADVANTAGES
Provides the ability for many different users to share data Information is only valuable if it is maintained
Some products (i.e., MySQL) may not require dedicated IT
support staff for administration
Table A Data Table B Data Table C Data

RDBMS ODBMS
i.e., Versant Object Database, FastObjects, GemStone
i.e., Sybase, Microsoft SQL Server, Oracle, MySQL This is a well established pattern with numerous examples both inside and outside of GIG.
Smalltalk, db4o object, Objectivity/DB, ObjectStore
 10

Pattern J: Data Grid / Distributed Database

A Data Grid is a loosely coupled, parallel computing subsystem, dedicated to the controlled sharing and management of large
Data amounts of distributed information.
Request
A Distributed Database (DDBMS) is a database that is under the control of a central DBMS, although its storage resides on multiple
computers located in the same physical location, or dispersed over a network of connected computers.
Client
Control Server ADVANTAGES DISADVANTAGES
Improved availability Increased complexity of administration
Improved scalability Generally not an option for servers across different locations
Lack of a high speed interconnect
Grid nodes may become unavailable unpredictably

i.e., Oracle Tangosol This is a well established pattern used by the PRS, ETP, FPAC, and COPS applications.
Grid Node

Pattern K: Storage MultiPathing

In this pattern redundant access to a fibre channel (FC) storage area network (SAN) is implemented using multiple host bus adaptor
(HBA) cards. Specialized software is then used to abstract storage connectivity, enable failover, and increase storage access
performance.
HBA 1
FC Switch
This pattern is recommend for high throughput applications (such as RDBMS) and for high availability storage access.
ADVANTAGES DISADVANTAGES
Software
Increased throughput and performance Cost
Seamless failover upon failures with HBAs, fibre channel
Multi-Pathed Server cabling, fibre channel switches, down to the storage array
EMC PowerPath also load balances for availability and
HBA 2
FC Switch
performance within EMCs storage products, past the storage
SAN Storage Array controller, down to a LUN or disk.

Veritass DMP is a standard across both CRMC and GIG for SAN connectivity from Solaris clients. Likewise,
i.e., Solaris MPxIO, Veritas Dynamic Multi-Pathing (DMP), EMC PowerPath EMC PowerPath ensures that storage networking failures do not cause downtime to Sybase databases on HPUX.

Pattern L: File System SnapShots

In this pattern a read-only copy of files and directories is created by manipulating the underlying inode table of a file system. As
Active File System Active File System Active such, it is a function of an operating system or a NAS filer.
File System SnapShot File System SnapShot File System
This pattern is recommended so that dynamic data can safely be written to tape, for the purpose of temporary archival, or as a point-
in-time reference before major application upgrades or operating system security patches.
Change ADVANTAGES DISADVANTAGES
2 Prolonged usage of file system snapshots or multiple
Freezes dynamic data so that corrupt data is not backed up to
tape snapshots can result in disk space consumption
A B C D1 A B C D1 A B C D1 D2 Host-based snapshots can consume system resources and
Enables live backup of 24x7 uptime systems
1 lead to severe performance degradation
Useful for risk mitigation for major application upgrades and
Copy
OS patches
Before Snapshot After the Snapshot Copy on Write

i.e., Solaris UFS and ZFS SnapShots, NetApp Snapshots, Mac OS X TimeMachine
This is a well established pattern with numerous examples both inside and outside of GIG.
 11

Development
Pattern M: Software Replication
In this pattern, data is replicated between servers --- regardless of storage interconnects, server location, and storage vendors --- by
Logical Volume XYZ virtue of the underlying volume manager and network I/O. Because server CPU cycles, not storage array resources, are required for
Production (Server B) this pattern, it is not recommended where high performance applications are found.
Synchronous or
Asynchronous
Logical Volume XYZ
Some applications also perform software replication (i.e., Oracle DataGuard, Sybase Replication Server, Lotus Notes).
(Server A) Local Disk
ADVANTAGES DISADVANTAGES
Server B Consumes a servers CPU cycles
Synchronous or Asynchronous replication
EMC Disk
DR Runs on all storage hardware platforms to eliminate vendor-
Server A Logical Volume XYZ specific replication limitations
(Server C) Enables hardware maintenance without disruption to business
Synchronous or
Asynchronous operations / scheduled downtime

Hitachi Disk

Host-based storage replication (VVR) is not a CG Standard, but some application-based software replication is
i.e., Veritas Volume Replicator (VVR) Server C
used. For example, the PA and NAV teams use Sybase Replication Server.

Pattern N: Local Hardware Replication

Hardware replication is a feature of many storage arrays, which aims to achieve a block for block copy of information. Within an
Third Mirror / array, a detachable third mirror can be used to create a point-in-time copy of data.
Mirrored Disks
Point in Time Copy
This pattern is recommended within an array to achieve storage replication for high availability.

ADVANTAGES DISADVANTAGES
Synchronous or Asynchronous replication Cost
Synchronous replication guarantees zero data loss
Does not consume server CPU cycles

SAN Storage

In-System Replication
EMC BCVs, Hitachi Shadow Image GIGs CID application uses BCVs.

Pattern O: Remote Hardware Replication

Production DR
Hardware replication is a feature of many storage arrays, which aims to achieve a block for block copy of information. When
implemented across storage systems, only block level changes are sent across the network.

This pattern is recommended between multiple storage arrays to achieve remote replication for disaster recovery purposes.

ADVANTAGES DISADVANTAGES
Synchronous or Asynchronous replication Cost
Synchronous or Some asynchronous replication technologies (i.e., EMC)
Asynchronous
guarantee database consistency and restart-ability even
though data loss may occur.
Does not consume server CPU cycles

SAN Array 1 SAN Array 2

Remote Replication All investment systems at CG, including GIG and CRMC, currently use S-SRDF EMC replication. In the AFS
EMC SRDF, Hitachi TrueCopy, NetApp SnapMirror Replication line of business, the SMART application uses EMCs A-SRDF technology, as will VMware and SAP.
 12

Host-Based Storage Virtualization Storage-Device Based Virtualization

Pattern P: Storage Virtualization
With storage virtualization, multiple independent storage devices --- that may be scattered over a network --- appear to be a single
monolithic storage device, which is managed centrally. While RAID and Volume Management are certainly types of virtualization,
i.e., Hitachi Data Systems Universal Storage Platform, 3PAR the term storage virtualization is reserved for technologies which enable abstraction of the physical location of data altogether.
Whether it uses address space remapping or I/O redirection; and whether it is host-, switch-, appliance-, or storage device-based;
storage virtualization enables the movement of data from one storage enclosure to another, while it is being utilized by a client.
ADVANTAGES DISADVANTAGES
Network-Based Storage Virtualization
Fewer points of management Traditional storage controller management --- the creation and
i.e., EMC Invista, StarWind, FalconStor, Coraid VS21, DataCore Improved storage utilization maintenance of RAID arrays, error, and fault management ---
Logical SANsymphony & SANmelody, IBM SAN Volume Controller is still required.
File File File File Disk Abstraction of physical location of data
Container Container Container Container (SVC), Incipient Network Storage Platform (iNSP),
NetApps V-Series, Openfiler Non-disruptive data migration
Storage Grid
Appliance-Based Storage Virtualization

i.e., Hadoop i.e., a Not yet a CG Standard.

Pattern Q: Data DeDuplication

Before Deduplication After Deduplication
Data deduplication is a method of reducing storage needs by eliminating redundant data. Only one instance of data (i.e., an email
attachment) is retained on storage media, and redundant data (i.e., other recipients of the email attachment) are replaced with a
Attachment
1 MB 1 MB
Attachment pointer to the unique data copy. Data deduplication can operate at the file, block, and the bit level, and hash algorithms are used to
identify duplication. This pattern is both emerging and paradigm shifting, and is being adopted by both storage and backup vendors.
1 MB
Attachment Link ADVANTAGES DISADVANTAGES
Cost effective use of storage and backup infrastructure In rare cases, the hash algorithm may produce the same hash
1 MB number for two different chunks of data. A hash collision can
Attachment Link be a problem, because it results in data loss.
Block and bit deduplication consume significant host
1 MB
Attachment Link
processing power.

1 MB
Attachment Link

i.e., DataDomain and EMC Avamar backup products; NetApp storage replication on ??? products Not a CG Standard.

i.e., EMC Invista (storage device-), NetApp Multistore and FlexVol (appliance-), and Veritas Storage Foundation (host-based)
Pattern R: Tiered Storage / Information Lifecycle Management (ILM)
File Information Lifecycle Management refers to a wide range of strategies --- i.e., data backup, protection, disaster recovery, restoration,
and replication technologies --- for administering storage subsystems. Primarily, it is the practice of applying different policies to
effectively manage data throughout its useful life. Thus, a file may exist on an expensive storage medium when it is first created
(i.e., fibre channel disk) but reside on a very different storage tier (i.e., tape) when its usefulness has transpired, and data is ready
for archival. This pattern is especially useful for document management, archive, and retrieval systems.
pe C1
Ta C A B1 ADVANTAGES DISADVANTAGES
B A1 A2
Tape Unavailability of the ILM system may result in an impact to
Applications and data can be managed separately
Unmirrored ATA Mirrored FC Disk
Disk Efficient storage utilization multiple high criticality applications

Cost effective
Off-Site Tape Vault Tape Library

Offline Copy Copy1T2 Copy1T1 Copy1T0

Not currently a CG Standard. The DAVAST application, however, has business requirements which may be met by
i.e., Quantum StorNext data management software (SNMS) the implementation of an information lifecycle management system.
 13

Pattern S: Content-Addressed Storage (CAS)

Is a mechanism
(1) the long-term retention of content for compliance and/or regulatory purposes, and the (2) archiving of massive amounts of
records, images or other information that rarely change
Uses hash algorithm to create a digital footprint to each stored object
Optimal as a secondary storage solution for archiving structured and unstructured data.
ADVANTAGES DISADVANTAGES
Cost effective use of storage for rarely-accessed data Performance
Single-instance storage (similar to data de-duplication) May require file system gateway for namespace translation
Online data archive eXtensible Access Method (XAM) is a new standard for
Self-managing and self-healing data connecting applications to object-based storage systems

Ensures data is unchanged for compliance

Often deployed as a redundant array of independent nodes
i.e., EMC Centera, Caringo CAStor, HDS's Content Archive Platform (HCAP), HPs Integrated Archive Platform (i.e., RISS), IBMs
(RAIN), whereby data is distributed in a cluster
DR550, NetApps NearStore, NEC Americas HydraStor, ProStors InfiniVault, Sun Microsystems StorageTek 5800, and
Permabits Enterprise Archive. Not yet a CG Standard.

Pattern T: RDBMS Archiving

Removing data from the production database and archiving it to a different set of media:
1. They evaluate data-retention policies and constraints to ensure that a business transaction can be archived/relocated.
2. They copy the business transaction to the archive.
3. They delete the business transaction from the production database.

ADVANTAGES DISADVANTAGES
Cost effective use of storage for rarely-accessed data a
Circumvents the need for more hardware
Performance gains for RDBMS and their applications
Decreased backup/restore windows
Reduced risk and time for database/application upgrades
Lower licensing where costs are tied to CPU/storage
i.e., HP's Reference Information Manager (RIM, including OuterBay LiveArchive), Princeton Softech's Active Archive (renamed Helps start the ILM process
Optim), Informatica Applimation Informia Archive Not yet a CG Standard.
Helps meet compliance requirements
 14

Pattern A: TCP/IP over Ethernet

7 Data Application Network Process to Application
Ethernet (IEEE 802.3) is a cabling and signaling specification for local area networks (LANs) and, as such, corresponds to the
Physical and the Data Link layers (Layer 1 and 2) of the 7 layer Open Systems Interconnect (OSI) architecture. It is often deployed
Host Layers

6 Data Presentation Data Representation and Encryption in a bus or star topology to connect computing nodes together. Ethernet is supported on both copper and fibre channel cabling.
TCP Communication on Ethernet networks is said to be based upon collision detection (CSMA/CD). That is, if two devices transmit at the
5 Data Session Interhost Communication same time in an Ethernet network, the signals from both devices collide, frames are lost, and re-transmission becomes necessary.
ADVANTAGES DISADVANTAGES
4 Segment Transport End-to-End Connections and Reliability A pervasive industry standard Ethernet collisions necessitate re-transmission
Inexpensive The more nodes there are in an Ethernet LAN, the more
3 Packet Network Logical Addressing: Path Determination IP TCP/IP and Ethernet are independent technologies overhead consumes all networking transactions
Media Layers

Supports multiple cabling standards (copper and fiber

2 Frame Data Link Physical Addressing: MAC and LLC channel) as well as
Ethernet
1 Bit Physical Media, Signal, and Binary Transmission Version 4.0 of the IP protocol (IPv4) is a well-established pattern with numerous examples both inside and outside
of CG. IPv6 is an emerging standard.

Sending Traffic from Server F to Server C Pattern B: Network Devices (Hub, Bridge, Switch, Router)
A A
A Hub is a repeater, operating on Layer 1 of the OSI model. Every node connected to the hub sees everything elses traffic.
H B H B
A Bridge works at Layer 2 of the OSI model and simply forwards data packets based upon MAC addressing. Bridges are used to
connect two segments of the same LAN, that use the same protocol. Bridges direct traffic away from unintended recipients.
A Switch is a Bridge with more than two interfaces (ports). Like all bridges, Switches provide bandwidth control.
A Router works at Layer 3 and forwards packets based on IP addressing; they isolate LANs into separate subnets. (See VLAN).
ADVANTAGES DISADVANTAGES
Bridge / These devices work to both segment and connect traffic Hubs are inefficient
G Hub C G C
Switch
Bridges, Switches, and Routers provide bandwidth control, Network devices require IT support staff
traffic control, and traffic priority control Bridges are no longer used industry-wide

F D F D
E E

A Hub forwards traffic to all nodes it connects. A Switch forwards traffic only to the intended data recipient. This is a well-established pattern with numerous examples both inside and outside of CG.

1 Hardware Router,
Pattern C: Virtual LAN (VLAN) and Traffic Fencing
3 Partitioned VLANs Virtual LANs are an infrastructure pattern closely related to virtual private networking, whereby physically networked devices are
VLAN 3 : Test / QA logically separated from one another.

Traffic fencing --- a slightly different technology based upon a routers Access Control Lists (ACLs) --- can likewise be used to keep
VLAN 1 : Development certain nodes from communicating with one another.
ADVANTAGES DISADVANTAGES
Traffic segregation These technologies are often found in commercial grade
routers. SOHO routers may not support these features.
TCP/IP Network
Router
Virtual LANs and Traffic Fencing --- when used together ---
can be difficult to maintain and troubleshoot, especially in
emergency situations.
Traffic Fencing is not as scalable as firewall-based application
VLAN 2 : Production
segmentation.

Traffic fencing is heavily utilized by the GIGs ETS and ETP Trading teams to separate Development Rendezvous
traffic from Production consumption. VLAN separation is used to segregate VOIP and data within CG.
 15

Physically on Network Y,
Pattern D: Virtual Private Networking (VPN)
Logically on Network X, also

In this pattern, a computer system is logically networked to systems which are physically or logically separate from them, through the
System Running use of secure, virtual connections. These open circuits can be administered through hardware, software, or a combination thereof.
VPN Software

One common use of VPNs is to secure communication through the public Internet, though security features such as content
The Internet
encryption and authentication need not be leveraged wherever VPNs are employed.
ADVANTAGES DISADVANTAGES
VPN Tunnel VPNs enable mobility Not all applications work well over a VPN connection
Router
Firewall with VPN Option Network Y
Computer inter-networking Corporate networks can be inadvertently exposed to security
Network X (VPN Gateway)
Enables private encryption of data vulnerabilities such as viruses through the use of this
technology
Network Address Translation (NAT) problems can be difficult
to troubleshoot without specialized help

The laptop has an IP Address on both Network Y, where it physically resides, and Network X, to which it is logically connected. This is a well established pattern with numerous examples both inside and outside of CG.

Server Example Networking Example

Pattern E: Virtual IP Address (VIP)
EXIT A Virtual IP Address is one that is not connected to a specific computer, NIC, or network device. It is generally used for fault-
tolerance or for load balancing. For example, a VIP may be assigned to a hardware load balancing device, and forward traffic to one
Stand-By of many web servers. Alternatively, a VIP on a multi-homed server can be dynamically assigned to any number of active network
IP 1 Router VIP connections, each with its own IP address. In this way TCP/IP packets bound for a VIP must travel through network interfaces,
TCP/IP Network
Router IP 1
though VIPs are not bound to them.

HSRP ADVANTAGES DISADVANTAGES

VIP
Low cost Complexity
IP 2 Enables load balancing Troubleshooting can be difficult
Multi-Homed Server Switch Active Enables fault tolerance / the use of redundant hardware at
Router the NIC or at the network hardware level.
TCP/IP Network IP 2
Router
Client
i.e., Solaris IP Network Multi-Pathing (IPMP) Ciscos HSRP protocol ensures that client traffic bound for a Gateways
VIP, travels through the IP of the active router along its path. This is a well established pattern with numerous examples both inside and outside of GIG.

Pattern F: Quality of Service (QoS)

5

RCVR XMTR RCLK FERF RL

FE 0/0 FE 0/1
ACT ACT

LNK LNK
0 1
SLOT 0 SLOT 1

CONS AUX
PWR

SYS
RDY
0 This pattern refers to reservation and control mechanisms for specialized network hardware, in which differing priorities are assigned
to applications, users, and data flows. With QoS enabled, bit rates, delay, jitter, and packet dropping probability can be minimized,
EN
2

AIS OOF LL 7120 - T3

and error rates can be guaranteed.

Priority 10
This pattern is recommended for latency sensitive applications such as streaming multimedia applications on a resource-constrained
LAN.
ADVANTAGES DISADVANTAGES
Priority 2 A guaranteed level of service to business critical applications Guaranteed service at the expense of defined data loss for
Provides consistent delivery and performance for critical unprioritized traffic
applications QoS doesn't go into effect until thresholds are reached, or at
times of bad traffic congestion.

Priority 5

LAN Switch
This pattern is used at CG to ensure that voice, video, and data can coexist on the same TCP/IP backbone. It is
also leveraged in the application space by both the ETS application and for our FTP servers.
 16

Pattern G: Network Impairment

In this pattern, a network deceleration device or software application holds TCP/IP packets from a client for a specified time, before
releasing it to the intended recipient. Alternatively, network traffic is generated to inject latency for testing purposes.
Time +100 ms
This pattern is recommended for network simulation and testing, to gauge application latency across the WAN.

ADVANTAGES DISADVANTAGES
Network deceleration and generation products can help Network generation products may not accurately simulate
Time +250 ms predict an application latency across the WAN. traffic patterns for different parts of a work day or work week.
Network
Decelerator
Client

Time +500 ms

i.e., Network Nightmare, Itheon, and Shunra products This is a well established pattern. The IA Lab has both hardware and software network deceleration products.

Pattern H: Network Acceleration

In this pattern, WAN optimization technologies --- whether hardware or software --- help overcome network latency associated with
bandwidth limitation, TCP performance, and lost IP packets through a mixture of data caching, compression, data streamlining, and
System Running
No Specialized Software transport streamlining solutions. On a LAN, standard proxy caches, transparent caches, and reverse proxy caches may be used.

The Internet This pattern is recommended for web applications, but not necessarily as a remedy for chatty applications.
ADVANTAGES DISADVANTAGES
SLOT 0 SLOT 1 SLOT 0 SLOT 1

AC T ACT PWR AC T ACT PWR

5 0 5 0

Usually implemented as point-to-point technologies

LNK L NK SYS LNK L NK SYS
FE 0/0 FE 0/1 0 1 CONS AUX FE 0/0 FE 0/1 0 1 CONS AUX
RDY RDY
RCVR XMTR RC LK FERF RL RCVR XMTR RC LK FERF RL
EN EN

Helps applications perform over long distance connections

2 2

AIS OOF LL 7120 - T3 AIS OOF LL 7120 - T3

Virtual Tunnel
Generally requires coordination of networking expertise on
Network X Network Y both the local and remote subnet
(i.e., SNO) (i.e., TKO)
If applied broadly and without adequate testing, optimization
techniques may adversely affect some business applications
while helping others
Can introduce single points for failure in the network
Cache Engines are a CG Standard for quickly delivering external internet traffic, while point-to-point WAN
i.e., Riverbed, Juniper, Blue Coat, and OrbitalData hardware; and Aspera software products optimization technologies are still emerging.

Pattern I: Content Load Balancers

i.e., Cisco or Big IP server load balancers This pattern uses a VIP associated with a specialized Link Load Balancer or WAN Load Balancer appliance. Traffic bound for the
(SLBs) or Application Delivery Controllers VIP is forwarded to one or more content servers. Content load balancers allow use of all ports at the same time to increase the total
Load Balancer (ADCs) available bandwidth, and they detect link saturation and failures in real time to redirect traffic. Moreover, they utilize algorithms to
perform traffic management, and real time DNS resolution for in-bound balancing. This pattern is recommended for n-tier web
applications.
ADVANTAGES DISADVANTAGES
Low-budget approach to high availability Trouble shooting can be complex especially if SSL termination
App App
1 2 Performance is increased due to caching of static content is required at the application
ADCs are most costly than traditional SLBs
Real time traffic load balancing
OS OS Modern ADCs can perform bidirectional application layer
CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU proxies, content transformation, XML validation and
I/O I/O transformation, and implement web application firewalls
Server 1 Server 2
Active/Active SLBs are used by web servers in the AFS line of business.
 17

Pattern J: Network Management Services

In this pattern, networking protocols --- such as TCP/IP, Windows NetBIOS, and Netwares IPX/SPX are managed through
management software dedicated to each protocol or service. DHCP is an Internet protocol used to automatically assign IP
addresses to computers. WINS and DNS are both name resolution services for TCP/IP networks: WINS resolves names in the
NetBIOS namespace, while DNS resolves names for TCP/IP. Other network management protocols are specific to networking
hardware vendors such as Cisco.
3 ? 1 2 ? "
ss .11 DHCP Server C3 .1.13 WINS Server ADVANTAGES DISADVANTAGES
dre 8.1 e r12 8.1. DNS Server P
s 68
d
A .1 6 rv 6 w
P Se 2 .1 ind
o 2 .1
a n I 1 9 2 i s 1 9 W s 19 DHCP simplifies the lease and expiry of TCP/IP addresses Backup DHCP, DNS, and WINS servers are recommended
d e ho s i s a t
e e Ta k W That h o Th DNS and WINS provide authoritative name resolution Some protocols and services do not natively span LANs, and
IN W
services therefore require servers on each LAN, or require that routers
DHCP, DNS, and WINS servers help to centralize be configured to forward traffic appropriately.
management within a domain and can be combined on a
single server where appropriate
DHCP Client DNS Client WINS Client

i.e., Dynamic Host Configuration Protocol (DHCP), Domain Name Service (DNS), and Windows Internet Name Service (WINS) This is a well established pattern both inside and outside of CG.
 18

Pattern A: Application-Governed HA
In this pattern, applications are made highly available by virtue of virtual IP addressing and application state information that is
managed internally, not by the underlying infrastructure.

App App This pattern is recommended wherever possible. When implemented on commodity hardware, this pattern can result in significant
1 2 cost savings.
ADVANTAGES DISADVANTAGES
Cost Software vendors must support Application HA: it cannot be
OS OS HA is managed by application administrators, not OS imposed on stateless applications
CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU administrators

I/O I/O

Server 1 Server 2
i.e., BEA WebLogic, Oracle Tangosol RDBMS Clustering BEA WebLogic manages application availability states internally, and is used by the OPAC, PRS, and CRD teams.

Pattern B: Infrastructure Hardware-Governed HA

App App App App
In this pattern, high availability is achieved through hardware. If the hardware is a networking device, content load balancers may be
OS OS OS OS
employed; if storage is leveraged, localized hardware replication technologies (EMC BCVs, Hitachi ShadowImage, or NAS
SnapShots) may be leveraged; if server and OS hardware are used, VMware's VMotion technologies may help achieve high
Migration Software
availability for applications.
Virtualization Software
Hardware
Virtualization Software
Hardware
ADVANTAGES DISADVANTAGES
Load Balancer STD BCV
Virtual Container Migration HA is achieved independently of applications Costly
Server Hardware

App App
1 2

C C COSC C C C C COSC C C
P P P P P P P P P P P P
U U U U U U U U U U U U
I/O I/O

Server 1 Server 2
SAN Storage
Active/Active
Networking Hardware Storage Hardware This is a well established pattern with numerous examples both inside and outside of GIG.

Pattern C: Infrastructure Software-Governed HA

Stateless applications are made available across two or more servers by virtue of third party software installed atop the OS.
App App Generally, application state information is communicated over a cross-over networking cable, a cluster or heart-beat VLAN, or via
1 2
shared disk. Recommended for applications which do not support clustering natively, and which are not accessed via the web.
Clustering Software Clustering Software If infrastructure software is networking-oriented, IPMP or DNS aliases may be used to ensure applications remain available.
OS OS If infrastructure software is storage-oriented, volume replication may be used to ensure applications remain highly available.
CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU CPU
I/O Shared I/O I/O ADVANTAGES DISADVANTAGES
Server 1 Server 2
Logical Volume Group
(Server 1) Any stateless application can be made highly available May impede application performance
TCP/IP Network Server Software
Router
A single administrative console can be leveraged for Complexity
availability of an n-tier application
When process, storage, or hardware faults are detected,
Logical Volume Group customized alerts can be sent to application and operations
Multi-Homed
Server (Server 2) teams
TCP/IP Network
Router

Networking Software Storage Software VCS is used by the ROC I&O application.
 19

Legend Pattern D: Manual HA / Server Recovery

Location / Site A Production Production
Location / Site B In this pattern, hardware storage replication and/or tape is leveraged to transport user and application data, while servers are
Primary Secondary Primary Secondary renamed to simulate recovery of a failed server.

This pattern is not recommend, and is being phased out at CG in general and within GIG in particular.

ADVANTAGES DISADVANTAGES
Tape Inexpensive Lack of automation
Lack of vendor support
Prone to human error
Server A Server B Server A Renamed as
Server A

Before After
i.e., CGs SSR methodology for HPUX servers This is a well established pattern with numerous examples both inside and outside of GIG.
 20

Legend Production Disaster Recovery Pattern A: No HA or DR

Location / Site A
Primary Secondary Primary Secondary
Location / Site B

Application
Tier X
ADVANTAGES DISADVANTAGES

Application
Tier X+1

This is a well established pattern with numerous examples both inside and outside of GIG.

Legend Production Disaster Recovery Pattern B: DR without HA

Location / Site A Primary Secondary Primary Secondary
Location / Site B

Application
Tier X
ADVANTAGES DISADVANTAGES

Application
Tier X+1

This is a well established pattern with numerous examples both inside and outside of GIG.

Legend Production Disaster Recovery Pattern C: HA and DR

Location / Site A
Primary Secondary Primary Secondary
Location / Site B

Application
Tier X
i.e., Hot - Warm
ADVANTAGES DISADVANTAGES

Application
Tier X+1
i.e., Active - Passive

This is a well established pattern with numerous examples both inside and outside of GIG.
 21

Legend Production Disaster Recovery Pattern D: HA and DR-with-HA

Location / Site A
Primary Secondary Primary Secondary
Location / Site B

Application
Tier X
i.e., Hot / Hot i.e., Hot / Hot
ADVANTAGES DISADVANTAGES

Application
Tier X+1
i.e., Active / Active i.e., Active / Active

This is a well established pattern with numerous examples both inside and outside of GIG.
 22

Pattern A: Network Monitoring

In this pattern, network inspection technologies are leveraged to ensure proper utilization of corporate networking resources
including --- but not limited to --- flow monitoring, sniffers, and content filtering mechanisms.
NetScout (owns network general), WireShark (also used by ), WireTaps is a physical device - to make it easier ... LEAVE IT OUT
Flow monitoring: NetQoS Reporter Analyzer (shows ???),
Cool Alert (monitor of monitor), no auto ticket creation, does not integrate into HPOV, routers, switches, load balancers, links,
circuits, voice, no overlap, uses SNMP
ADVANTAGES DISADVANTAGES
See application traffic at the frame level Expensive (no load introduced)
Security (anything in clear text)

A.

Pattern B: Server Monitoring

HPOV In this pattern, HPs SNMP implementation, HPOV templates, and HP Smart Plug-Ins (SPIs) tightly integrate with our HP OpenView
Monitor, (HPOV) monitor of monitors; OR third party monitoring tools send messages to HPOV via OPC Messages.
HPOV Remedy
Template,
(Monitor of (Auto-Generated
or Smart Monitoring Tools) HPOV either creates Remedy incidents automatically or via the manual effort of a DataCenter Operations agent. Operations utilizes
Event)
Plug-In the on-call database wherever escalation is required.
(SPI)
ADVANTAGES DISADVANTAGES
Alerts
Tight integration with HPOV Minor customizations may be necessary for implementing third
Third Party party monitoring tools
Remedy Flexibility of third party monitoring tools
Monitoring (Manually Created
Tool
OPC HP SPIs can be chatty, and therefore make Operations lose
Message Event)
focus from relevant event notification
Support may be more difficult if multiple vendors are involved
DataCenter Operations On-Call DB Escalation

The AutoSys job scheduler, Tibcos Hawk monitor, and CA Wilys Introscope Java monitor currently utilize OPC Messaging. HPs SNMP used for system baseline monitoring. HPOV templates available for log file encapsulation. HP Smart
Clustering products such as VCS+GCO, MS, and HP MCSG may soon implement this, as well. Plug-Ins (SPIs) not used due to chattiness.

Pattern C: Application Monitoring

In this pattern,

ADVANTAGES DISADVANTAGES

i.e., Hyperic, CA Wily A.

Change Log

Version 0.97
- Standardized on two environments (dedicated or shared)
- Re-Did all Recoverability patterns
- Re-Did all Notification patterns
- Added ADCs to Network Pattern I: Content Load Balancers
- Miscellaneous updates to
Server & OS Pattern I: Virtual Container Migration
Storage Pattern P: Storage Virtualization
Pattern R: Tiered Storage / ILM
Pattern S: Content-Addressed Storage (CAS)

Version 0.96
Incorporated feedback from the Networking team (PXED, SFM,
CONSJS, KEVC)

Version 0.95
Incorporated feedback from the Storage team

Version 0.94
Initial distribution beyond GIG Technology Engineering