Professional Documents
Culture Documents
Concepts and
Techniques
Slides for Textbook
Chapter 2
Jiawei Han and Micheline Kamber
Revised by Zhongfei (Mark) Zhang
Computer Science Department
Binghamton University
zhongfei@cs.binghamton.edu
January 18, 2
Data Mining: Concept
January 18, 2
January 18, 2
Data WarehouseSubjectOriented
January 18, 2
Data WarehouseIntegrated
January 18, 2
January 18, 2
Data WarehouseNon-Volatile
January 18, 2
January 18, 2
January 18, 2
OLAP
users
clerk, IT professional
knowledge worker
function
decision support
DB design
application-oriented
subject-oriented
data
current, up-to-date
detailed, flat relational
isolated
repetitive
historical,
summarized, multidimensional
integrated, consolidated
ad-hoc
lots of scans
unit of work
read/write
index/hash on prim. key
short, simple transaction
# records accessed
tens
millions
#users
thousands
hundreds
DB size
100MB-GB
100GB-TB
metric
transaction throughput
usage
access
January 18, 2
complex query
10
January 18, 2
11
January 18, 2
12
January 18, 2
13
Cube: A Lattice of
Cuboids
all
time
time,item
0-D(apex) cuboid
item
time,location
location
item,location
time,supplier
time,item,location
supplier
location,supplier
item,supplier
time,location,supplier
time,item,supplier
1-D cuboids
2-D cuboids
3-D cuboids
item,location,supplier
4-D(base) cuboid
January 18, 2
14
Conceptual Modeling
of Data Warehouses
January 18, 2
15
time
time_key
day
day_of_the_week
month
quarter
year
branch
branch_key
branch_name
branch_type
location_key
units_sold
dollars_sold
avg_sales
item_key
item_name
brand
type
supplier_type
location
location_key
street
city
province_or_street
country
Measures
January 18, 2
16
time
time_key
day
day_of_the_week
month
quarter
year
item
Sales Fact Table
time_key
item_key
branch_key
branch
location_key
branch_key
branch_name
branch_type
item_key
item_name
brand
type
supplier_key
units_sold
dollars_sold
supplier_key
supplier_type
location
location_key
street
city_key
avg_sales
Measures
January 18, 2
supplier
city
city_key
city
province_or_street
country
17
Example of Fact
Constellation
time
time_key
day
day_of_the_week
month
quarter
year
item
Sales Fact Table
time_key
item_key
item_key
item_name
brand
type
supplier_type
branch_key
location_key
branch
branch_key
branch_name
branch_type
units_sold
dollars_sold
avg_sales
location
to_location
location_key
street
city
province_or_street
country
dollars_cost
Measures
January 18, 2
units_shipped
shipper
shipper_key
shipper_name
location_key
shipper_type18
January 18, 2
19
January 18, 2
20
January 18, 2
21
January 18, 2
22
Measures: Three
Categories
January 18, 2
23
all
Europe
region
country
city
office
January 18, 2
Germany
Frankfurt
...
...
...
Spain
North_America
Canada
Vancouver ...
L. Chan
...
...
Mexico
Toronto
M. Wind
24
Specification of hierarchies
Schema hierarchy
day < {month < quarter;
week} < year
Set_grouping hierarchy
{1..10} < inexpensive
January 18, 2
25
Multidimensional Data
Sales volume as a function of product,
month, and region
Re
gi
on
Industry Region
Year
Product
City
Office
Month Week
Day
Month
January 18, 2
26
Pr
od
u
TV
PC
VCR
sum
1Qtr
2Qtr
Date
3Qtr
4Qtr
sum
Country
ct
sum
January 18, 2
27
date
country
product,country
1-D cuboids
date, country
2-D cuboids
January 18, 2
3-D(base) cuboid
28
Visualization
OLAP capabilities
Interactive manipulation
Data Mining: Concept
29
January 18, 2
Pivot (rotate):
Other operations
January 18, 2
30
Shipping Method
Customer
CONTRACTS
AIR-EXPRESS
TRUCK
Time
ORDER
PRODUCT LINE
ANNUALY QTRLY
DAILY
PRODUCT ITEM
CITY
Product
PRODUCT GROUP
SALES PERSON
COUNTRY
DISTRICT
REGION
Location
January
Each circle is
Promotion
called a
18, footprint
2
Data Mining:
DIVISION
Concept
Organization
31
January 18, 2
32
Top-down view
January 18, 2
33
January 18, 2
34
Multi-Tiered Architecture
other
Metadata
source
s
Operational
DBs
Extract
Transform
Load
Refresh
Monitor
&
Integrator
Data
Warehouse
OLAP Server
Serve
Analysis
Query
Reports
Data mining
Data Marts
Data Sources
January 18, 2
Data Storage
OLAP Engine Front-End Tools
Data Mining: Concept
35
Enterprise warehouse
collects all of the information about subjects spanning
the entire organization
Data Mart
a subset of corporate-wide data that is of value to a
specific groups of users. Its scope is confined to
specific, selected groups, such as marketing data mart
Virtual warehouse
A set of views over operational databases
Only some of the possible summary views may be
materialized
January 18, 2
36
Data Warehouse
Development: A
Recommended Approach
Multi-Tier Data
Warehouse
Distributed
Data Marts
Data
Mart
Data
Mart
Model refinement
Enterprise
Data
Warehouse
Model refinement
January 18, 2
37
January 18, 2
38
January 18, 2
39
January 18, 2
40
Cube Operation
(city)
()
(item)
(year)
January 18, 2
41
January 18, 2
42
c3 61
62
63
64
c2 45
46
47
48
c1 29
30
31
32
c0
b3
B13
b2
b1
b0
14
15
16
a0
a1
a2
a3
January 18, 2 A
60
44
28 56
40
24 52
36
20
43
c3 61
62
63
64
c2 45
46
47
48
c1 29
30
31
32
c0
b3
b2
B13
14
15
16
28
24
b1
b0
a0
a1
a2
a3
20
44
40
36
60
56
52
January 18, 2
44
c3 61
62
63
64
c2 45
46
47
48
c1 29
30
31
32
c0
b3
b2
B13
14
15
16
28
24
b1
b0
a0
a1
a2
a3
20
44
40
36
60
56
52
January 18, 2
45
January 18, 2
46
Base table
Cust
C1
C2
C3
C4
C5
Region
Asia
Europe
Asia
America
Europe
January 18, 2
Index on Region
Index on Type
47
January 18, 2
48
January 18, 2
49
Metadata Repository
Operational meta-data
Business data
January 18, 2
50
Data extraction:
Data cleaning:
Load:
Data transformation:
Refresh
January 18, 2
51
January 18, 2
52
Discovery-Driven Exploration of
Data Cubes
January 18, 2
53
January 18, 2
54
Continuing the last example, among the max price tuples, find
the min and max shelf life, and find the fraction of the total sales
due to tuples that have min shelf life within the set of all max
price tuples
January 18, 2
55
January 18, 2
56
Information processing
Analytical processing
Data mining
January 18, 2
57
Architecture of OLAM
January 18, 2
Data Mining: Concept
58
An OLAM Architecture
Mining query
Mining result
Layer4
User Interface
OLAM
Engine
OLAP
Engine
Layer3
OLAP/OLAM
MDDB
Filtering&Integration
Database API
MDDB
Meta
Data
Filtering
Layer1
Databases
January 18, 2
Data cleaning
Data
Data integration Warehouse
Data
Repository59
Summary
Data warehouse
January 18, 2
60
References (I)
K. Beyer and R. Ramakrishnan. Bottom-Up Computation of Sparse and Iceberg CUBEs. In Proc.
1999 ACM-SIGMOD Int. Conf. Management of Data (SIGMOD'99), 359-370, Philadelphia, PA,
June 1999.
S. Chaudhuri and U. Dayal. An overview of data warehousing and OLAP technology. ACM
SIGMOD Record, 26:65-74, 1997.
January 18, 2
61
References (II)
K. Ross and D. Srivastava. Fast computation of sparse datacubes. In Proc. 1997 Int. Conf.
Very Large Data Bases, 116-125, Athens, Greece, Aug. 1997.
E. Thomsen. OLAP Solutions: Building Multidimensional Information Systems. John Wiley &
Sons, 1997.
January 18, 2
62