BCO 90 Architecture Scalability

BMC Capacity Optimization Best Practice
INTERNATIONAL TOLL FREE

Int'l Toll: 913-312-0828
US/CAN Toll free: 866-816-1982
Argentina: 0800 444 6440
Australia: 1 800 612 415
Austria: 0800 295 780
Bahamas: 1 800 389 0491
Belgium: 0 800 75 636
Brazil: 0800 891 0266
Bulgaria: 00 800 115 1141
Chile: 123 0020 6707
China, Northern Region: 10 800 714 1509
China, Southern Region: 10 800 140 1376
Colombia: 01 800 518 1171
Czech Republic: 800 700 715
Denmark: 80 883 277
Dominican Republic: 1 888 752 0002
France: 0 800 914 176
Germany: 0 800 183 0299
Greece: 00 800 161 2205 6440
HongKong:
800 968 066
Copyright 10/31/2012 BMC Software, Inc
Hungary: 06 800 112 82

India: 000 800 1007 613
Indonesia: 001 803 017 6440
Ireland: 1 800 947 415
Israel: 1 80 925 6440
Italy: 800 789 377
Japan: 00348 0040 1009
Latvia: 8000 3523
Lithuania: 8 800 3 09 64
Luxembourg: 800 2 3214
Malaysia: 1 800 814 723
Mexico: 001 800 514 6440
Monaco: 800 39 593
Netherlands: 0 800 022 1465
New Zealand: 0 800 451 520
Norway: 800 138 41
Panama: 00 800 226 6440
BMC Proprietary & Confidential
Peru: 0800 54 129

Philippines: 1 800 111 010 55
Poland: 00 800 112 41 42
Portugal: 800 827 538
Russia: 810 800 2915 1012
Singapore: 800 101 2320
Slovenia: 0 800 80439
South Africa: 0 800 982 304
South Korea: 003 0813 2344
Spain: 900 937 665
Sweden: 02 079 3266
Switzerland: 0 800 894 821
Taiwan: 00 801 127 186
Thailand: 001 800 156 205 2068
Trinidad and Tobago: 1 800 205 6440
United Kingdom: 0 808 101 7156
Uruguay: 0004 019 0348
Venezuela: 0 800 100 8540
BMC Capacity Optimization Best Practices

BCO Architecture and Scalability
With 9.0 updates
Legal Notice
The information contained in this presentation is the confidential information of BMC,

Inc. and is being provided to you with the express understanding that without the prior
written consent of BMC, customers and partners may not discuss or otherwise disclose
this information to any third party or otherwise make use of this information for any
purpose other than for which BMC intended.
All of the recommendations and information described herein are at the sole discretion
of BMC and are subject to change and/or cancellation, and in no way should this content
be viewed as guarantees or warrantees on BMCs part.
Overview
Best Practice vs. How To
Does not address every scenario
Prior knowledge of BCO components and terms
Updates with 9.0 SP1
BCO 9.0 best practice update

Topic
Session
Speaker
BCO Architecture and Scalability
This session
Sudheer
BCO Data Integration
Dec 2012
Giuseppe
Managing BPA to BCO data integration
Jan 2013
Mike
Deploying and configuring BCO-CLM integration
Feb 2013
Sudheer
Storage Capacity Management
Mar 2013
Sudheer
Architecture and Scalability Review

Scalability for ETL engines local and remote
HA deployments of BCO
LDAP deployments and upgrades

BCO main components
BCO Application Server

provides communication bus for all other components
all analytical, modeling and reporting tasks
supports BCO presentation layers
supported on:
RedHat Enterprise Linux
SuSE Linux Enterprise
CentOS
Solaris (x86 architectures)
supported on:
Same as Application Server
BCO ETL Engine Server

supports all data integration tasks
provides secondary scheduling services
BCO Database Server

Data management and warehousing
Data export for third-party reporting tools
supported on:
any OS supported by Oracle
and their major sizing drivers
Component responsibilities
ETLEngine
AS Datahub
DatabaseServer
Extract/
parse/poll
Transferandload
forremote EE
Summarize
Transform
Runbackground
tasks
Load
Runservices
Calculate
customstats
AS web
Buildandnavigate
usercontext
Runanalyses
andmodels
Materialize
datamarts
Serve data
Runreports
If you separate these two, be sure to share the repository between them!
AS installation more flexible OOTB
10
Throughput Load = samples per day

Entities
- Monitored devices or servers that performance metrics are collected from
- Examples
Server OS
Database
Application
Network Device
- Typically 90% of the data collected
Business KPIs
- Non technical performance metrics
- Examples
Financial Data
Business Transaction Volumes
Facility Data
- Typically 10% of the data collected
11
Throughput load examples

Example Data Processing Volumes
Samples/day
5 million
10 million
20 million
Number of entities
Number of metrics
(average by entity)
Time
granularity
1250
40
15 minutes
2500
40
30 minutes
5000
40
1 hour
2500
40
15 minutes
5000
40
30 minutes
10000
40
1 hour
5000
40
15 minutes
10000
40
30 minutes
20000
40
1 hour
12
Minimum Hardware
Role
CPU
RAM
Disk
Database Server
2 cores @2GHz up to 5Mln rows/day;

1 additional core @2GHz every
additional 5Mln rows/day
2GB
(SCSI or external storage recommended)
Application Server
2 cores @2GHz
8GB
8GB free + 20GB for the repository (SCSI

recommended)
ETL Engine
2 cores @2GHz
2GB
8GB free
13
Database Server
Major Considerations
- Horizontal and Vertical scalability
- Number of entities
- Number of metrics
- Sample rate
- Retention periods
Minimum Hardware
- CPU 2 cores at 2 GHz,
- Memory 2 GB RAM
- SCSI Disk minimum
- At least two sets of disks
- Disk Space 50 GB for every 1 million samples per day
based on BCO standard summarization policies
14
Database Server
Tablespace Sizing
- Should be increased for large environments
- Default Sizing
BCO -- 150 MB
BCO_IDX -- 150 MB
BCO_DATA -- 8 GB
BCO_DATA_IDX -- 8 GB
BCO_DASH -- 100 MB
-
Typical Increased Sizing for large environments

BCO -- 512 MB (with 128 KB extents)
BCO_IDX -- 512 MB (with 128 KB extents)
BCO_DASH -- 384 MB (with 128 KB extents)
BCO_DATA -- 60% of remaining space (with 8 MB or 16 MB extents)
BCO_DATA_IDX -- 40% of remaining space (with 8 MB or 16 MB extents)
Temporary Tablespace should be greater than 3GB
UNDO Tablespace should be greater than 3GB
15
Database Server
Oracle Express Edition (XE)
- Database size is limited by the Oracle licensing
- Different Oracle license agreements have different quotas
- Use only for very small (or pilot) installations
Additional Recommendations
- Use a dedicated instance
- Physical server improves I/O compared to virtual server
- Use Oracle Partitioning Option (Enterprise Edition)
- Use Oracle Automatic Storage Management (ASM)
-
Use database JVM option installed and enabled.

Must be enabled on the database instance.
16
Database Server
Minimum Hardware Scalability
- Up to 5 million data samples a day
- Default retention policies
Scaling
- Increase hardware resources
- For each additional 5 million samples per day
1 CPU cores at 2 GHz
1 Gig RAM
- Disk
50 GB of storage is required for every 1 million of samples a day
Grows linearly
Default summarization policies
17
Database Server
IOPS
- 100 IOPS are required for every 1 million of samples a day
- Grows linearly
- Default retention policies
Out-sync samples
- Add additional processing & Storage
- Manage as corrective factors for the estimates
- Minimum estimate 0.1% of overall samples for entity metrics
- 1.0% of overall samples for business metrics
18
Database Server
Example
- Conditions
5000 systems
100 metrics per system
Hourly sample rate
Detailed aging policy 95 days
-
Analysis
12 million daily samples (initial estimate)
600 GB of storage (initial estimate)
Assume 1.0% out-of synch (pessimistic)
Corrected samples = 12 * 1.01 = 12.12
Corrected storage = 12.12 * 50 GB = 606 GB 610 GB
Account for aging policy 670 GB
19
Application Server
- Number of users
- Number reports
- Required data volume processing throughput
Minimum Hardware
- CPU 2 cores at 2GHz,
- Memory 8GB RAM
- SCSI Disk
- Disk Space - 8GB
- Repository Disk Space - 20GB
20
Application Server
Minimum Hardware Scalability
- 100 users
- 100 reports
- up to 10 million data samples a day
Vertical Scaling
- Increase hardware resources
- 20 million samples per day
- 4 CPU cores at 2 GHz
- 16 Gig RAM
- A single application server should be limited to 20 million samples per day
21
Application Server
Horizontal Scaling
- Add additional Application Servers
- Requires load balancing on the front-end
See product documentation for configuration
Additional Considerations
- One primary scheduler for the environment
- One Data Hub for the environment
- Remote Engine ETL Servers
Requires additional resources for the Data Hub
See more on this later.
22
Application Server Deployment

Supported Operating Systems newer OS support added, also dropped EL4
- SUSE 11 added, etc.
See next slide
Additional Software
- JRE automatically installed
- Apache web server automatically installed
-
Oracle Client 11.2.x recommended, both for 32 bit and 64 bit.

See product documentation for required options and libraries
OS Account Requirements
- The system user configured to run BCO and ETL components must have rights to
crontab
- Cpit is the default OS ID who owns the BCO files, etc.
- The install is run as non-privileged account
- If you have a cron.deny policy, make sure that the cpit user is not included
- If you have a cron.allow policy, include cpit in it
23
Operating system versions
Operatingsystem
Version
Architecture
x86
Architecture
x86_64
RedHatEnterprise
Linux
5.x
Yes
Yes
6.x
Yes
OracleLinux
6.x
Yes
LinuxCentOS
5.x
SUSELinux
Enterprise
10.x
Yes
11.x
Yes
10.x
Yes
11.x
yes
OracleSolaris
Yes
Yes
24
Application Server Deployment

Recommendations
- BCO base installation directory
$CPITBASE
suggested value /opt/bmc/BCO
A frontend web server is configured OOTB
Uses Apache
Allows caching
Supports compression
- Multiple Application Servers
Repository directory needs to be shared
NFS share (i.e. by configuring on all servers a mount point that maps to external storage)
SAN LUN.
25
ETL Engine Server

- Number of connectors
- Number entities
- Number of metrics
- Sample rates
- Required data volume processing throughput
More on this in the next topic
26
General Installation Notes

BCO Installation Versions
- Please make sure that you downloaded the correct installation package for your
(supported) platform and architecture.
- Install only on supported platforms
- Installation may successfully complete on a non-supported platform, but BCO services
may not start and the product may not work as expected.
Installation Order
- Database
- Application Server
- ETL Servers
- Remote ETL Servers
- Perform Initial Configuration
- Install and configure additional solution components
CLM integration, Chargeback, Consolidation, Virtual Farm
27

28
ETL engine sizing guidelines

Vertical scaling:
ETL Engine configuration
Disk space
No. of connectors
Samples per day
2 CPU cores @ 2GHz, 4 GB RAM
8 GB free
100 instances
10 million
4 CPU cores@ 2 GHz, 8 GB RAM
16 GB free
100 instances
20 million
Limit each connector task scheduled to 2 million samples.

Avoid scheduling more than 1 connector task per CPU at the same time.
Allow twice as much memory for Java ETLs as for perl ETLs.
Horizontal scaling:
The limit is the size of the BCO database.
Special considerations for remote ETL engines (see next slide).
29
Local versus remote ETL Engine
Local ETL Engine
extract
Oracle DB server
load
Remote ETL Engine
Data hub
Remote ETL Engine increases CPU, memory, disk I/O on Data hub machine
Local ETL Engine
Remote ETL Engine
Oracle DB server
Data hub
extract
transfer
JMS service
load
save
JMS queue
30
Why use a remote ETL engine?

Only when:
- A limited bandwidth or unstable connection is present between data
source and the BCO database.
Remote ETL Engine separates extraction phase from load phase.

Data extraction occurs reliably close to the datasource, and the load phase
occurs at the Data hub.
Store-and-forward messaging infrastructure makes transfer reliable.
It is not possible to expose the needed TCP ports on the BCO

database.
Remote ETL Engine communicates with the BCO Data hub via HTTP or
HTTPS ports.
Dont use a remote ETL Engine just because the data source is remote.
The name "local" in a local ETL Engine does not imply that it must be on the
same LAN as the database. As long as the ETL Engine can reach the
database, a local ETL Engine is to be preferred.
31
Resource impact of remote EE on Data hub

Remote ETL engines use a two-step process:
A store-and-forward messaging infrastructure transfers data to the Data hub

machine (usually the same as the Application Server machine).
Data is loaded into the data warehouse by connecting to the Oracle database.
This two-step process involves use of the following resources:
Reads and writes to the disk on the Data hub machine.

Disk space on the EE machine to accumulate pre-transfer data for many days.
(Note: today there is no check!) Accumulate for ever!
Enough disk space on the Data hub machine to accumulate transferred data
for 3 days.
Allows for space in case of down time for a weekend.
CPU and memory on the Data hub machine to parse the transferred data, do
lookups, and format the samples for loading into the database.
Database connection pools for the Data hub component.
Even if there are sufficient resources for all of the above, the overall
process also takes longer to complete than for local ETL Engines.
32
Setting the Data hub JVM heap size

Warning
Be careful when modifying the heap size of the JVM! Wrong settings may
cause unpredictable and hard to diagnose failures.
Edit the file customenvpre.sh.
Uncomment the lines:
- #DATAHUB_HEAP_SIZE="1024m"
- #export DATAHUB_HEAP_SIZE
and replace the value with "8192m" before restarting the Data hub
service.
33
Increasing the database connection pool size

Warning
When modifying the database connection pool size, it is important to
verify that the Oracle database also has corresponding sizes for the
number of concurrent sessions.
Edit the file caplan-ds.xml.
Look for the element
- <jndi-name>CaplanDHDS</jndi-name>
Replace the value for the element
- <max-pool-size>40</max-pool-size>.
Warning
Make sure to edit the setting for the CaplanDHDS datasource only!
34

35
Component single points of failure
Component
Singlepointoffailure?
Approachforhighavailability
Datawarehouse
Yes
UsemultiplemachinesandOracleRACoption
Webapplication
Yes,ifonlyoneinstance
UsecommercialclustermanagerforA/P
Installonmultiplemachines,anduseloadbalancer
forA/A
DatahubandPrimary
scheduler
Yes
ETLEngineandscheduler
Yes,foritsportionofdata
connectors
BCOinstallationdirectory
Yes
Useeitherstoragereplicationorhostbased
mirroring
BCORepository
Yes
Useeitherstoragereplicationorhostbased
mirroring
36
BCO machine configurations
Configuration
BCOcomponentsinstalled
ASALL
Webapplication,Datahub,Primaryscheduler
ASWEB
Webapplication
ASHUB
Datahub,Primaryscheduler
EE
ETLEngine,scheduler
37
HA for AS-ALL configuration (1)

Configure two machines M1 and M2 in A/P failover using OS clustering
software.
Configure the following resources to be shared and enabled among M1
and M2:
A floating IP address associated with the BMC Capacity Optimization URL via
DNS
Front-end web server (Apache), when configured
BMC Capacity Optimization Console JSP engine (Apache Tomcat)
Primary scheduler
Data hub (JBoss)
38
HA for AS-ALL configuration (2)

You need:
Primary machine M1, Secondary machine M2

Third-party clustering software
Shared or synchronized file system between M1 and M2 for installation files
Shared or synchronized directory for BCO repository
Steps:
Configure clustering software (typically scripts) for failover and giveback.

Temporarily change the hostname of M1 to the cluster name
using hostname command as root.
Install BCO on M1. Wait for the post-installation to complete.
Change the hostname of M1 back to the individual machine name.
Shut down BCO services.
Move installation directory into shared or synchronized mount point, and leave
a symbolic link in the original installation directory.
Create a similar symbolic link on machine M2.
Restart BCO services on M1 and start using BCO.
Now M1 will be used as primary, and M2 as secondary.
39
HA for AS-WEB configuration

Option 1: Configure two machines M1 and M2 in A/P failover:
- Configure the following resources to be shared and enabled among
M1 and M2:
A floating IP address associated with the BMC Capacity Optimization URL via
DNS
Front-end web server (Apache), when configured
BMC Capacity Optimization Console JSP engine (Apache Tomcat)
Then, proceed exactly as in AS-ALL configuration.
Option 2: Configure as many machines as needed in A/A using a

load balancer.
40
HA for AS-HUB configuration

software.
Configure the following resources to be shared and enabled among M1
and M2:
- A floating IP address
- Primary scheduler
- Data hub (JBoss)
Then, proceed exactly as in AS-ALL configuration above.
41
HA for EE configuration
software.
Configure the following resources to be shared and enabled between M1
and M2:
- Scheduler
- Data accumulator
Then, proceed exactly as in AS-ALL configuration above.
42

43
LDAP native mode

Use LDAP only for authentication
1
BCO admin creates user
and authorizes user
4 LDAP bind, search

sequence
LDAP
server
3
User logs in with name and password
4a
BCO records user

and authorization
5
BCO authorizes user
User authentication as part of

bind in sequence
Authentication
server
44
LDAP managed mode

Create user account automatically
2 LDAP bind, search

sequence
LDAP
server
3 Return user entry
User logs in with name and password
2a
4
BCO creates user and authorization on first
login
User authentication as part of

bind in sequence
Authentication
server
45
BCO LDAP sequence

bind directly with BCO user account
user
LDAP
server
BCO
LDAP bind using username and password

Name and password
Site-specific
authentication
Return success for bind
LDAP search using user bind context (search for groups)

Return user entry information
Use user entry info for

authorization
46
BCO LDAP sequence (in BCO 4.0-9.0)

bind using separate account
user
LDAP
server
BCO
LDAP bind using admin account

Name and password
LDAP search using admin context (retrieve user DN from attributes)

Return user DN for next bind
Second bind using DN and password
Site-specific
authentication

LDAP search using user bind context

authorization
47
BCO LDAP sequence 9.0 SP1

search LDAP through separate account
user
LDAP
server
BCO
LDAP bind using admin account

Name and password
LDAP search using admin context (retrieve user DN from attributes)
Return user DN for bind
LDAP search using admin bind context (search for groups)
LDAP bind using DN and password
Site-specific
authentication

authorization
48
LDAP group mapping
BCOversion
LDAPgroupmappedtoBCO
role
LDAP groupmappedtoBCOaccess
group
4.5
External nameonrole
(notavailable)
9.0SP1
Externalnameonrole
Externalnameonaccessgroup
49
LDAP integration in 9.0 SP1

4.5
- LDAP groups can be mapped as external names for a role.
Semicolon-separated list of names specified in configuration screen.
When logged-in BCO user is found in LDAP, and the user is part of
any of these groups, then the user is automatically assigned that role.
9.0
- Roles AND access groups can be assigned external names.
- When the user is part of any of these groups, then the user is
automatically added to the ac group.
For migration from 4.5 to 9.0
- You want to set up LDAP once.
Sign up once for the right groups in LDAP.

When users log into BCO, automatically get their access rights.
50
Migrating from 4.5

Today:
- Identify which acgroups a user should be member of.
- Assign explicitly the user to the acgroups in BCO.
For preparation for 9.0:
- Create LDAP groups corresponding to the acgroups
- Add these users to the appropriate LDAP groups.
After migrating to 9.0:
- Specify external names for the acgroups.
- These users will automatically be added to the acgroups on login.
If a user was not added in LDAP, then he will be removed from the acgroup.
An acgroup can either be manually populated with users, or automatically
using LDAP external names. Not both.
Workaround if LDAP modifications are delayed:

- Create a separate acgroup with mnaual membership.
51
Additional Resources & Information

Product Documentation
-
https://docs.bmc.com/docs/display/public/bcmco90/Home
BMC Communities (public forum)

-
https://communities.bmc.com/communities/community/bmcdn/service_assurance
BMC Support Knowledge Articles

-
https://communities.bmc.com/communities/docs/DOC-18600
Chatter BCO Group All about BCO

-
https://na13.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F9300000004FI
N
52
53

BCO 90 Architecture Scalability

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BCO 90 Architecture Scalability

Uploaded by

Copyright:

Available Formats

BMC Capacity Optimization Best Practice

INTERNATIONAL TOLL FREE

Hungary: 06 800 112 82

BMC Proprietary & Confidential

Peru: 0800 54 129

BMC Capacity Optimization Best Practices

The information contained in this presentation is the confidential information of BMC,

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

BCO 9.0 best practice update

BCO Architecture and Scalability

BCO Data Integration

Managing BPA to BCO data integration

Deploying and configuring BCO-CLM integration

Storage Capacity Management

Architecture and Scalability Review

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

BCO 9.0 best practice update

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

BCO main components

BCO Application Server

BCO ETL Engine Server

BCO Database Server

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

and their major sizing drivers

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

AS installation more flexible OOTB

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Throughput Load = samples per day

BMC Proprietary & Confidential

Throughput load examples

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

2 cores @2GHz up to 5Mln rows/day;

(SCSI or external storage recommended)

8GB free + 20GB for the repository (SCSI

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Typical Increased Sizing for large environments

Temporary Tablespace should be greater than 3GB

UNDO Tablespace should be greater than 3GB

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Use database JVM option installed and enabled.

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Copyright 10/31/2012 BMC Software, Inc

BMC Proprietary & Confidential

Copyright 10/31/2012 BMC Software, Inc