Professional Documents
Culture Documents
29 April 2010
Cubing Services
Business Analytics
© 2010 IBM Corporation
Business Analytics
Disclaimer
The information on any new products in this presentation is
intended to outline our general product direction and it should
not be relied on in making a purchasing decision. The
information on new products is for informational purposes only
and may not be incorporated into any contract. The
information on new products is not a commitment, promise, or
legal obligation to deliver any material, code or functionality.
The development, release, and timing of any features or
functionality described for our products remains at our sole
discretion.
Agenda
IBM Cognos
USER
BI / FPM
Off-line portable/ partitioned storage Dynamic changing dimensions and hierarchy, Optimized ROLAP
what-if scenarios, data contribution Cubing Services
Rapid startup for advanced business Zero Latency
user self service modeling Common dimensions shared across multi- High volume concurrency
Optimized forseries
Automatic time broad, general
analysis & Optimized
cube modelsfor write back and high Optimized for very large datasets
Largest Data Volumes
purpose
trending BI usage volatility
Budgetingapplications
with very large dimensions
and planning
Centralized IT management of information
Point in time data Personal and corporate data sources
Personal and corporate data sources
Ideal for:
• Very large data sets with very
large dimensions – SKU level
data
• Enterprise rollouts requiring near
real time data
Cubing Services
Because of its unique ability to:
Warehouse Modeling • Accelerate OLAP queries
optimized on InfoSphere
Warehouse
• Integrated IT tooling
InfoSphere
Warehouse Primary owner - IT departments
Excel
64-bit in-memory caching
XMLA
OLAP Metadata MDX Optimization
Eclipse Design Studio Calculation Advisor
Builder
• Pervasive BI tool
• Real-time, live connection to
warehouse data via Excel
• Can save point-in-time data, and
refresh at any time
• Full use of pivot table services for
analysis
Design
Cubing Services
Cube Server Analyze Advisor
Run
DB2 Design Advisor
Optimize
Deploy
Administration Console
Creating a Cube
Reverse engineer
schema Export Import
DB2 LUW
OLAP Metadata
DB2
Metadata files
MQT
Optimization Advisor
BI Designer DBA
DB2
OLAP Metadata
MQTs
Integrate refresh script in SQW process flows that update the data warehouse
tables.
DBA
Cube Server
Admin Console
Import
OLAP Metadata
DB2
Metadata files
MQTs
OLAP metadata
A cube is accessible by OLAP clients if it is running on a cube server. Use the Cubing Services
administration features in Administration Console to:
– Import OLAP metadata (that was previously exported via Design Studio) into the
metadata database for test/production deployments
– Map metadata to relational data source
– Create and configure cube server
– Deploy cubes to cube server
– Configure cube runtime tuning parameters
– Define cube security
– Control cube server and cube runtime states (start, stop, …)
Other
Alphablox Excel BI tools
MDX
Cube server and
Cube Server
Admin Console cube administration
DB2
OLAP Metadata
Data cache
DB2
Dim Member
Result set MDX calculation engine Cache
MDX query
Data cache
SQL
• Simple, multi-pass SQL
DB2
Performance Factors
Tune configuration
Cube Server
Log files
DBA
DB2
Design
Advisor Create MQTs, MDC,
DB2
Indexes
Best Practices
Cube server creates six log files (activity, exception, trace, MDX, performance, SQL)
Enable MDX log, SQL log and performance log to analyze query performance
High SQL elapsed times could indicate that appropriate indexes and/or MQTs have not been created or are not being used effectively by the DB2
optimizer.
Use SQL queries as an input workload to the DB2 Design Advisor. The tool will recommend additional MQTs,
indexes, MDC, and other optimization objects to further optimize DB2 for this Cubing Services workload.
Faster performance
Virtual cubes
– Enables fast performance in a low latency, real-time warehouse
– Facilitates better data management
Dimensional security
– Fine grain access control down to the dimension members
Results:
Benefits:
• Faster cold cache response times makes real-
time OLAP easier since cache refreshes are
less impactful to user experience 9.5.1 9.5.2 9.7 9.7.1
Sales
Inventory Combines cubes with nearly
identical dimensions
Store Sales
Web Sales
Definition
Time
Current Month
41 Footer Field © 2010 IBM Corporation
Business Analytics
Sales
Inventory
Combines cubes with nearly
identical dimensions
Store Sales
Web Sales
Users can merge dimensions with different names by changing the names to match.
Details follow.
Member Merging
CubeA CubeB
Merge operators
• Examples:
• Merge using add operation "+“ CubeA CubeB
Intersection of security
settings from CubeA
Virtual and CubeB
Cube
Set dimensional
security here Set dimensional
security here
CubeA CubeB
Dimensional security
• Fine-grained access control to OLAP metadata
• Allows you to limit the members of a dimension that a role can
access
• By default, allows access to all members
USA USA
WA CA WA CA
Security enforcement
Single role/single policy
A user in a single role that is assigned a single policy on a dimension is
allowed to access the set of allowed members except the set of denied
members
Example:
Allowed Members = {Ascendants([Store].[CA]), Descendants([Store].
[CA])}
Denied Members = [Store].[City].Members
USA
CA
San Jose
Store1
Security enforcement
Multiple roles/multiple policies
A user in multiple roles or a single role with multiple policies is allowed to access union
(allowed members) except union(denied members)
AM = Allowed
DM Member
DM DM = Denied Member
AM DM
DM
DM
Security enforcement
Hierarchy navigation
Restricted members are skipped in operations like .parent
and .children.
Parent Member
Member Children
Server 2
Server 1
Cube Server 2
WebSphere
Cube Server 1
Admin Console
Server 3
Cube Server 3
Repository
database
Proof points
Recent success stories
Telecom customer
Retail customer
Industry: Telecom
InfoSphere Balanced Warehouse like system
– Database module - 42 data partitions
– Application module - Model 9133-55A, 4 PowerPC_POWER5 processors, 32GB memory
Software
– DB2 9.5
– InfoSphere Warehouse 9.7.1
– Cognos 8.4
Warehouse
– Four star schemas
– Largest fact table has 1.5 billion rows. Other fact tables have 50 million, 20 million and 8 million rows.
– 8-12 dimensions per fact table (most shared)
Cubes
– Cube per star schema
– Virtual cube merges the four cubes
• 12 dimensions
• 25 measures – varying complexity
Cognos reports took 5-7 seconds
Virtual Virtual
Cube 1 Cube 2
Toss
Activation Reactivation ARPU
Adjustment
Tuning
– New database with no optimization
– Defined primary keys on dimension tables
– Defined constraints (not enforced) on star schema joins
– Ran Cubing Services Advisor (no data sampling)
• Recommended indexes on fact table foreign keys and dimension level keys
• Recommended MQTs
– Built recommended indexes and MQTs
Additional tuning that could be done
– Define MDC for fact table
– Run Cubing Services Advisor with data sampling to get better recommendations
– Run DB2 Design Advisor for additional index/MQT recommendations
Industry: Retail
IBM Smart Analytics System 7600 v9.7 (pSeries and AIX)
– Database module - 1 admin node + 2 data nodes (8+1 DPF partitions), Power6, 4-core, 32 GB
– Application module - 4-core, 32 GB (actually 8-core 64 GB but only used half)
Software
– DB2 9.7.1
– InfoSphere Warehouse 9.7.1
– Cognos 8.4
Warehouse
– One star schema
– Fact table has 2 billion rows
– Four dimensions
Cubes
– One cube
• 4 dimensions
• 12 measures – simple aggregations
Cognos reports took 5-7 seconds
Month Department Area • Which stores generated the highest total sales
Software:
– DB2 9.5.2
– InfoSphere Warehouse 9.5.2
– Alphablox 9.5.2
– AIX 5.3
Warehouse:
– 1 large table with dimensions embedded within fact table – over 70 million rows and growing
– Offline for maintenance only 1x week for 4 hours
– Refreshing cubes takes minutes in most case, up to 20 minutes for largest cube
300 users worldwide
Saved $6 million in first year, and expect to save more than double that in the second year
Technical resources
Information Center for 9.7
http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/index.jsp
Backup
Operational reporting
Sales trend & ad hoc analysis
Optimized for:
PowerCube • Highest consistent query
performance
Business modeling • Enterprise rollouts such as
internet delivery
Operational Planning
Financial analytics &
reporting
TM1 Architect
Ideal for:
• write-back planning
applications in moderate sized
communities;
• complex models demanding
read/write interactivity
Quickly load & merge Easily create and access Advanced rules language
large data volumes (intra- reports or input templates for complex financial
day or nightly) from ODBC deployed via Excel or applications
/ ODBO & flat files web
Client perspective
Cubes
Virtual Measures
In Admin Console, virtual cubes belong to a cube model called VIRTUAL. This is where they
can be deleted.
Starting, deploying to a cube server, stopping, and undeploying virtual cubes operations are similar to real
cubes except for a few new dependencies:
– A virtual cube can be started/deployed only after both its direct cubes are started/deployed
– A cube or virtual cube can only be stopped/undeployed after all the virtual cubes that directly use it
are stopped/undeployed
Virtual Cube
1. Select the database that contains the tables that are referenced by the cubes in
the XML file
2. Specify the XML file that contains the security metadata
3. Map roles
4. Select collision resolution option - Merge Replace, Merge Ignore, Replace
Dimension: Measures
Hit “Reload
Security” button to
update the
security metadata
in a running cube
server
Import wizard
Imports security model from XML file to database in project or database explorer
Deploy wizard
Exports security model from database in project explorer to database in database
explorer
Create database from reverse engineering
Exports security model from database in database explorer to database in project
explorer
Database explorer
Displays security in the repository database including default cube security
Thank you!