Professional Documents
Culture Documents
IT509
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Introduction
• Motivation: Why data mining?
• What is data mining?
• Data Mining: On what kind of data?
• Data mining functionality
• Are all the patterns interesting?
• Classification of data mining systems
• Major issues in data mining
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Motivation: “Necessity is the Mother of
Invention”
• Data explosion problem
– Automated data collection tools and mature database technology lead
to tremendous amounts of data stored in databases, data warehouses
and other information repositories
• We are drowning in data, but starving for knowledge!
• Solution: Data warehousing and data mining
– Data warehousing and on-line analytical processing
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Evolution of Database Technology
• 1960s:
– Data collection, database creation, IMS and network DBMS
• 1970s:
– Relational data model, relational DBMS implementation
• 1980s:
– RDBMS, advanced data models (extended-relational, OO,
deductive, etc.) and application-oriented DBMS (spatial,
scientific, engineering, etc.)
• 1990s—2000s:
– Data mining and data warehousing, multimedia databases, and
Web databases
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Why Mine Data? Commercial Viewpoint
• Lots of data is being collected
and warehoused
– Web data, e-commerce
– purchases at department/
grocery stores
– Bank/Credit Card
transactions
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Why Mine Data? Scientific Viewpoint
• Data collected and stored at
enormous speeds (GB/hour)
– remote sensors on a satellite
– telescopes scanning the skies
– microarrays generating gene
expression data
– scientific simulations
generating terabytes of data
• Traditional techniques infeasible for raw data
• Data mining may help scientists
– in classifying and segmenting data
– in Hypothesis Formation
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Mining Large Data Sets - Motivation
4,000,000
3,500,000
3,000,000
The Data Gap
2,500,000
2,000,000
500,000 Number of
0
analysts
1995 1996 1997 1998 1999
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, and
From: R. Grossman, C. Kamath, V. Kumar, “Data Mining for Scientific Steinbach & Kumar
Engineering Applications”
What is Data Mining?
• Many Definitions
– Non-trivial extraction of implicit, previously unknown and
potentially useful information from data
– Exploration & analysis, by automatic or
semi-automatic means, of
large quantities of data
in order to discover
meaningful patterns
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Data Mining: A KDD Process
Pattern Evaluation
– Data mining: the core of
knowledge discovery process.
Data Mining
Task-relevant Data
Data Selection
Warehouse
Data Cleaning
Data Integration
Databases
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Steps of a KDD Process
• Learning the application domain:
– relevant prior knowledge and goals of application
• Creating a target data set: data selection
• Data cleaning and preprocessing: (may take 60% of effort!)
• Data reduction and transformation:
– Find useful features, dimensionality/variable reduction, invariant
representation.
• Choosing functions of data mining
– summarization, classification, regression, association, clustering.
• Choosing the mining algorithm(s)
• Data mining: search for patterns of interest
• Pattern evaluation and knowledge presentation
– visualization, transformation, removing redundant patterns, etc.
• Use of discovered knowledge
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Data Mining and Business Intelligence
Increasing potential
to support
business decisions End User
Making
Decisions
Data Exploration
Statistical Analysis, Querying and Reporting
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Architecture of a Typical Data Mining System
Pattern evaluation
Data
Databases Warehouse
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
What is (not) Data Mining?
What is not Data What is Data Mining?
Mining?
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Data Mining Tasks
• Prediction Methods
– Use some variables to predict unknown or
future values of other variables.
• Description Methods
– Find human-interpretable patterns that
describe the data.
From [Fayyad, et.al.] Advances in Knowledge Discovery and Data Mining, 1996
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Data Mining Tasks...
• Classification [Predictive]
• Clustering [Descriptive]
• Association Rule Discovery [Descriptive]
• Sequential Pattern Discovery [Descriptive]
• Regression [Predictive]
• Deviation Detection [Predictive]
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
What Is Data Mining?
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Why Data Mining? — Potential Applications
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Market Analysis and Management (1)
• Where are the data sources for analysis?
– Credit card transactions, loyalty cards, discount coupons, customer
complaint calls, plus (public) lifestyle studies
• Target marketing
– Find clusters of “model” customers who share the same
characteristics: interest, income level, spending habits, etc.
• Determine customer purchasing patterns over time
– Conversion of single to a joint bank account: marriage, etc.
• Cross-market analysis
– Associations/co-relations between product sales
– Prediction based on the association information
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Market Analysis and Management (2)
• Customer profiling
– data mining can tell you what types of customers buy what
products (clustering or classification)
• Identifying customer requirements
– identifying the best products for different customers
– use prediction to find what factors will attract new customers
• Provides summary information
– various multidimensional summary reports
– statistical summary information (data central tendency and
variation)
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Corporate Analysis and Risk Management
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Fraud Detection and Management (1)
• Applications
– widely used in health care, retail, credit card services,
telecommunications (phone card fraud), etc.
• Approach
– use historical data to build models of fraudulent behavior and
use data mining to help identify similar instances
• Examples
– auto insurance: detect a group of people who stage accidents to
collect on insurance
– money laundering: detect suspicious money transactions (US
Treasury's Financial Crimes Enforcement Network)
– medical insurance: detect professional patients and ring of
doctors and ring of references
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Fraud Detection and Management (2)
• Detecting inappropriate medical treatment
– Australian Health Insurance Commission identifies that in many
cases blanket screening tests were requested (save Australian
$1m/yr).
• Detecting telephone fraud
– Telephone call model: destination of the call, duration, time of
day or week. Analyze patterns that deviate from an expected
norm.
– British Telecom identified discrete groups of callers with frequent
intra-group calls, especially mobile phones, and broke a
multimillion dollar fraud.
• Retail
– Analysts estimate that 38% of retail shrink is due to dishonest
employees.
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Other Applications
• Sports
– IBM Advanced Scout analyzed NBA game statistics (shots
blocked, assists, and fouls) to gain competitive advantage for
New York Knicks and Miami Heat
• Astronomy
– JPL and the Palomar Observatory discovered 22 quasars with
the help of data mining
• Internet Web Surf-Aid
– IBM Surf-Aid applies data mining algorithms to Web access
logs for market-related pages to discover customer preference
and behavior pages, analyzing effectiveness of Web marketing,
improving Web site organization, etc.
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Data Mining: On What Kind of Data?
• Relational databases
• Data warehouses
• Transactional databases
• Advanced DB and information repositories
– Object-oriented and object-relational databases
– Spatial databases
– Time-series data and temporal data
– Text databases and multimedia databases
– Heterogeneous and legacy databases
– WWW
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Classification: Definition
• Given a collection of records (training set )
– Each record contains a set of attributes, one of the attributes is the class.
• Find a model for class attribute as a function of the values of other attributes.
– A test set is used to determine the accuracy of the model. Usually, the
given data set is divided into training and test sets, with training set used to
build the model and test set used to validate it.
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Classification Example
cal cal us
ri or
i uo
go g tin
t e t e n ss
ca ca c o
cl
a
Tid Refund Marital Taxable Refund Marital Taxable
Status Income Cheat Status Income Cheat
Set Classifier
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Classification: Application 1
• Direct Marketing
– Goal: Reduce cost of mailing by targeting a set of consumers likely to
buy a new cell-phone product.
– Approach:
• Use the data for a similar product introduced before.
• We know which customers decided to buy and which decided
otherwise. This {buy, don’t buy} decision forms the class attribute.
• Collect various demographic, lifestyle, and company-interaction
related information about all such customers.
– Type of business, where they stay, how much they earn, etc.
• Use this information as input attributes to learn a classifier model.
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Classification: Application 3
• Customer Attrition/Churn:
– Approach:
• Use detailed record of transactions with each of the past and
present customers, to find attributes.
– How often the customer calls, where he calls, what time-
of-the day he calls most, his financial status, marital
status, etc.
• Label the customers as loyal or disloyal.
• Find a model for loyalty.
– Approach:
• Segment the image.
• Measure image attributes (features) - 40 of them per object.
• Model the class based on these features.
• Success Story: Could find 16 new high red-shift quasars,
some of the farthest objects that are difficult to find!
From [Fayyad, et.al.] Advances in Knowledge Discovery and Data Mining, 1996
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Classifying Galaxies
Courtesy: http://aps.umn.edu
Late
Data Size:
• 72 million stars, 20 million galaxies
• Object Catalog: 9 GB
• Image Database: 150 GB
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Clustering Definition
• Given a set of data points, each having a set of attributes, and a
similarity measure among them, find clusters such that
• Similarity Measures:
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Illustrating Clustering
Intracluster
Intraclusterdistances
distances Intercluster
Interclusterdistances
distances
are
areminimized
minimized are
aremaximized
maximized
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Clustering: Application 1
• Market Segmentation:
– Approach:
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Clustering: Application 2
• Document Clustering:
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Illustrating Document Clustering
• Clustering Points: 3204 Articles of Los Angeles Times.
• Similarity Measure: How many words are common in these
documents (after some word filtering).
National 273 36
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Clustering of S&P 500 Stock Data
Observe Stock Movements every day.
Clustering points: Stock-{UP/DOWN}
Similarity Measure: Two points are more similar if the events described by
them frequently happen together on the same day.
We used association rules to quantify a similarity measure.
1
Applied-Matl-DOW N,Bay-Net work-Down,3-COM-DOWN,
Cabletron-Sys-DOWN,CISCO-DOWN,HP-DOWN,
DSC-Co mm-DOW N,INTEL-DOWN,LSI-Logic-DOWN,
Micron-Tech-DOWN,Texas-Inst-Down,Tellabs-Inc-Down,
Technology1-DOWN
Natl-Semiconduct-DOWN,Oracl-DOWN,SGI-DOW N,
Sun-DOW N
2
Apple-Co mp-DOW N,Autodesk-DOWN,DEC-DOWN,
ADV-M icro-Device-DOWN,Andrew-Corp-DOWN,
Co mputer-Assoc-DOWN,Circuit-City-DOWN,
Technology2-DOWN
Co mpaq-DOWN, EM C-Corp-DOWN, Gen-Inst-DOWN,
Motorola-DOW N,Microsoft-DOWN,Scientific-Atl-DOWN
3
Fannie-Mae-DOWN,Fed-Ho me-Loan-DOW N,
MBNA-Corp -DOWN,Morgan-Stanley-DOWN Financial-DOWN
4
Baker-Hughes-UP,Dresser-Inds-UP,Halliburton-HLD-UP,
Louisiana-Land-UP,Phillips-Petro-UP,Unocal-UP, Oil-UP
Schlu mberger-UP
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Association Rule Discovery: Definition
TID Items
1 Bread, Coke, Milk
Rules
RulesDiscovered:
Discovered:
2 Beer, Bread
{Milk}
{Milk}-->
-->{Coke}
{Coke}
3 Beer, Coke, Diaper, Milk {Diaper,
{Diaper,Milk}
Milk}-->
-->{Beer}
{Beer}
4 Beer, Bread, Diaper, Milk
5 Coke, Diaper, Milk
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Association Rule Discovery: Application 1
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Association Rule Discovery: Application 2
• Supermarket shelf management.
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Association Rule Discovery: Application 3
• Inventory Management:
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Sequential Pattern Discovery: Definition
• Given is a set of objects, with each object associated with its own timeline of events, find rules that predict strong sequential dependencies among different events.
• Rules are formed by first discovering patterns. Event occurrences in the patterns are governed by timing constraints.
(A B) (C) (D E)
(A B) (C) (D E)
<= xg >ng <= ws
<= ms
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Sequential Pattern Discovery: Examples
• In telecommunications alarm logs,
– (Inverter_Problem Excessive_Line_Current)
(Rectifier_Alarm) --> (Fire_Alarm)
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Regression
• Predict a value of a given continuous valued variable based on the
values of other variables, assuming a linear or nonlinear model of
dependency.
• Examples:
– Predicting sales amounts of new product based on advertising
expenditure.
– Predicting wind velocities as a function of temperature, humidity,
air pressure, etc.
– Time series prediction of stock market indices.
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
Deviation/Anomaly Detection
• Detect significant deviations from normal behavior
• Applications:
– Network Intrusion
Detection
Typical network traffic at University level may reach over 100 million connections per
12/08/21 daythe slides of Jiawei Han & Micheline Kamber and that of
A lot of help was taken from
Tan, Steinbach & Kumar
Challenges of Data Mining
• Scalability
• Dimensionality
• Complex and Heterogeneous Data
• Data Quality
• Data Ownership and Distribution
• Privacy Preservation
• Streaming Data
• Ability to deal with different types of attributes
• Discovery of clusters with arbitrary shape
• Minimal requirements for domain knowledge to determine input parameters
• Able to deal with noise and outliers
• Insensitive to order of input records
• Incorporation of user-specified constraints
• Interpretability and usability
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar
References
12/08/21 A lot of help was taken from the slides of Jiawei Han & Micheline Kamber and that of
Tan, Steinbach & Kumar