BITS Pilani presentation

BITS Pilani
Pilani Campus

N.MEHALA FACULTY,CS/IS GROUP

DATA MINING
IS ZC415 Second Semester 2013-14

2/15/2014

IS ZC415

2
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Text & Reference Books

Text Book
Tan P. N. , Steinbach M & Kumar V. Introduction to Data Mining Pearson Education, 2006.

Reference Books
Han J & Kamber M, Data Mining: Concepts and Techniques, Morgan Kaufmann Publishers, Second Edition, 2006 Dunhum M.H. & Sridhar S. Data MiningIntroductory and Advanced Topics, Pearson Education, 2006.
2/15/2014 IS ZC415 3

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Agenda
Motivation: Why data mining? What is data mining? Data Mining: KDD Process? Data mining tasks Major issues in data mining Applications

2/15/2014

IS ZC415

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Why Data Mining?

The Explosive Growth of Data: from terabytes to petabytes [Every two days now we create as much information as we did from the dawn of civilization up until 2003, Eric Schmidt ] Data collection and data availability Automated data collection tools, database systems, Web, computerized society Major sources of abundant data Business: Web, e-commerce, transactions, stocks, Science: Remote sensing, bioinformatics, scientific simulation, Society and everyone: news, digital cameras, YouTube We are drowning in data, but starving for knowledge! Necessity is the mother of invention - Plato :Data mining: Automated analysis of massive data sets
IS ZC415 5 2/15/2014
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

The Evolution of Data Analysis

Evolutionary Step Business Question Enabling Technologies Product Providers Characteristics

Data Collection (1960s)

"What was my total Computers, tapes, revenue in the last disks five years?" "What were unit sales in New England last March?" Relational databases (RDBMS), Structured Query Language (SQL), ODBC On-line analytic processing (OLAP), multidimensional databases, data warehouses Advanced algorithms, multiprocessor computers, massive databases
IS ZC415

IBM, CDC

Retrospective, static data delivery

Data Access (1980s)

Oracle, Sybase, Informix, IBM, Microsoft

Retrospective, dynamic data delivery at record level

Data Warehousing & Decision Support (1990s)

"What were unit sales in New England last March? Drill down to Boston." "Whats likely to happen to Boston unit sales next month? Why?"

SPSS, Comshare, Retrospective, Arbor, Cognos, dynamic data Microstrategy,NCR delivery at multiple levels

Data Mining (Emerging Today)

SPSS/Clementine, Lockheed, IBM, SGI, SAS, NCR, Oracle, numerous startups

Prospective, proactive information delivery

6

2/15/2014

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Why Mine Data? Commercial Viewpoint

Lots of data is being collected and warehoused Web data, e-commerce purchases at department/grocery stores Bank/Credit Card transactions Computers have become cheaper and more powerful Competitive Pressure is Strong Provide better, customized services for an edge (e.g. in Customer Relationship Management)

2/15/2014

IS ZC415

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Why Mine Data? Scientific Viewpoint

Data collected and stored at enormous speeds (GB/hour) remote sensors on a satellite telescopes scanning the skies microarrays generating gene expression data scientific simulations generating terabytes of data Traditional techniques infeasible for raw data Data mining may help scientists in classifying and segmenting data

2/15/2014

IS ZC415

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

What is NOT Data Mining?

Searching a phone number in a phone book Searching a keyword on Google Generating histograms of salaries for different age groups Issuing SQL query to a database and reading the reply

2/15/2014

IS ZC415

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining is NOT

Data Warehousing (Deductive) query processing
SQL/ Reporting

Software Agents Expert Systems Online Analytical Processing (OLAP) Statistical Analysis Tool Data visualization
IS ZC415 10
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

2/15/2014

What is Data Mining?

Discovery of useful summaries of data - Ullman Extracting or Mining knowledge from large amounts of data The efficient discovery of previously unknown patterns in large databases Technology which predict future trends based on historical data It helps businesses make proactive and knowledge-driven decisions
IS ZC415

2/15/2014

11

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

What is Data Mining?

Extraction of interesting (non-trivial, implicit, previously unknown and potentially useful) information or patterns from data in large databases

2/15/2014

IS ZC415

12

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

What is Data Mining?

Data mining is an integral part of knowledge discovery in databases (KDD), which is the overall process of converting raw data into useful information. This process consists of series of transformation steps from preprocessing to postprocessing of data mining results
2/15/2014 IS ZC415 13

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Process of Knowledge Discovery in Database(KDD)

Normalization. Data subsetting

Filtering Patterns,Visualizatio n,Pattern Interpretation

Input Data

Data Preprocessing

Data Mining

PostProcessing

Information

2/15/2014

IS ZC415

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Input data 14

Data Mining: A KDD Process

Pattern Evaluation

Data mining: the core of knowledge discovery process.

Data Mining

Task-relevant Data Selection

Data Warehouse

Data Cleaning
Data Integration

2/15/2014

Databases

IS ZC415

15

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Steps of a KDD Process

Learning the application domain:
relevant prior knowledge and goals of application

Creating a target data set: data selection Data cleaning and preprocessing: (may take 60% of effort!) Data reduction and transformation:
Find useful features, dimensionality/variable reduction, invariant representation.
2/15/2014 IS ZC415 16
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Steps of a KDD Process

Choosing functions of data mining summarization, classification, regression, association, clustering. Choosing the mining algorithm(s) Data mining: search for patterns of interest Pattern evaluation and knowledge presentation visualization, transformation, removing redundant patterns, etc. Use of discovered knowledge
2/15/2014 IS ZC415 17
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining and Business Intelligence

Increasing potential to support business decisions

Making Decisions
Data Presentation Visualization Techniques Data Mining Information Discovery Data Exploration Statistical Analysis, Querying and Reporting

End User

Business Analyst Data Analyst

Data Warehouses / Data Marts OLAP, MDA Data Sources Paper, Files, Information Providers, Database Systems, OLTP
2/15/2014 IS ZC415

DBA

18

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining: Confluence of Multiple Disciplines

Database Technology Statistics

Machine Learning

Data Mining

Visualization

Information Science
2/15/2014 IS ZC415

Other Disciplines
19
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining vs. Statistical Analysis

Statistical Analysis:
Ill-suited for Nominal and Structured Data Types Completely data driven - incorporation of domain knowledge not possible Interpretation of results is difficult and daunting Requires expert user guidance

Data Mining:

2/15/2014

Large Data sets Efficiency of Algorithms is important Scalability of Algorithms is important Real World Data Lots of Missing Values Pre-existing data - not user generated Data not static - prone to updates Efficient methods for data retrieval available for use
IS ZC415 20
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining vs. DBMS

Example DBMS Reports
Last months sales for each service type Sales per service grouped by customer sex or age bracket List of customers who lapsed their policy

Questions answered using Data Mining

What characteristics do customers that lapse their policy have in common and how do they differ from customers who renew their policy? Which motor insurance policy holders would be potential customers for my House Content Insurance policy?

2/15/2014

IS ZC415

21

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining and Data Warehousing

Data Warehouse: a centralized data repository which can be queried for business benefit. Data Warehousing makes it possible to extract archived operational data overcome inconsistencies between different legacy data formats integrate data throughout an enterprise, regardless of location, format, or communication requirements incorporate additional or expert information OLAP: On-line Analytical Processing Multi-Dimensional Data Model (Data Cube) Operations: Roll-up Drill-down Slice and dice Rotate
IS ZC415 22
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

2/15/2014

DBMS, OLAP, and Data Mining

DBMS
Task

OLAP
Summaries, trends and forecasts
Analysis Multidimensional data modeling, Aggregation, Statistics What is the average income of mutual fund buyers by region by year?

Data Mining
Knowledge discovery of hidden patterns and insights
Insight and Prediction Induction (Build the model, apply it to new data, get the result) Who will buy a mutual fund in the next 6 months and why?

Extraction of detailed and summary data

Information Deduction (Ask the question, verify with data)

Type of result

Method

Example question

Who purchased mutual funds in the last 3 years?

2/15/2014

IS ZC415

23

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Example of DBMS, OLAP and Data Mining: Weather Data

DBMS:
Day 1 2 3 outlook sunny sunny overcast temperature 85 80 83 humidity 85 90 86 windy false true false play no no yes

4
5 6 7 8 9

rainy
rainy rainy overcast sunny sunny

70
68 65 64 72 69

96
80 70 65 95 70

false
false true true false false

yes
yes no yes no yes

10
11 12 13 14 2/15/2014

rainy
sunny overcast overcast rainy

75
75 72 81 71

80
70 90 75 91 IS ZC415

false
true true false true

yes
yes yes yes no 24

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Example of DBMS, OLAP and Data Mining: Weather Data

By querying a DBMS containing the above table we may answer questions like:
What was the temperature in the sunny days? {85, 80, 72, 69, 75} Which days the humidity was less than 75? {6, 7, 9, 11} Which days the temperature was greater than 70? {1, 2, 3, 8, 10, 11, 12, 13, 14} Which days the temperature was greater than 70 and the humidity was less than 75? The intersection of the above two: {11}

2/15/2014

IS ZC415

25

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Example of DBMS, OLAP and Data Mining: Weather Data

OLAP: Using OLAP we can create a Multidimensional 9/5 Model of our data Week 1 (Data Cube). Week 2 For example using the dimensions: time, outlook and play we can create the following model. 2/15/2014 IS ZC415

sunny 0/2 2/1

rainy 2/1 1/1

overcast 2/0 2/0

26

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Example of DBMS, OLAP and Data Mining: Weather Data

Data Mining: Using the ID3 algorithm we can produce the following decision tree:
outlook = sunny
humidity = high: no humidity = normal: yes

outlook = overcast: yes outlook = rainy

windy = true: no windy = false: yes
2/15/2014 IS ZC415 27

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Major Issues in Data Mining

Mining methodology and user interaction Mining different kinds of knowledge in databases Interactive mining of knowledge at multiple levels of abstraction Incorporation of background knowledge Data mining query languages and ad-hoc data mining Expression and visualization of data mining results Handling noise and incomplete data Pattern evaluation: the interestingness problem Performance and scalability Efficiency and scalability of data mining algorithms Parallel, distributed and incremental mining methods

2/15/2014

IS ZC415

28

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Major Issues in Data Mining

Issues relating to the diversity of data types Handling relational and complex types of data Mining information from heterogeneous databases and global information systems (WWW) Issues related to applications and social impacts Application of discovered knowledge Domain-specific data mining tools Intelligent query answering Process control and decision making Integration of the discovered knowledge with existing knowledge: A knowledge fusion problem Protection of data security, integrity, and privacy
IS ZC415

2/15/2014

29

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining: Classification Schemes

Decisions in data mining
Kinds of databases to be mined Kinds of knowledge to be discovered Kinds of techniques utilized Kinds of applications adapted

2/15/2014

IS ZC415

30

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Decisions in Data Mining

Databases to be mined Relational, transactional, object-oriented, object-relational, active, spatial, time-series, text, multi-media, heterogeneous, legacy, WWW, etc. Knowledge to be mined Characterization, discrimination, association, classification, clustering, trend, deviation and outlier analysis, etc. Multiple/integrated functions and mining at multiple levels Techniques utilized Database-oriented, data warehouse (OLAP), machine learning, statistics, visualization, neural network, etc. Applications adapted Retail, telecommunication, banking, fraud analysis, DNA mining, stock market analysis, Web mining, Weblog analysis, etc.
2/15/2014 IS ZC415 31
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining Tasks

Data Mining is generally divided into two tasks. 1. Predictive tasks 2. Descriptive tasks

2/15/2014

IS ZC415

32

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Predictive Tasks
Objective: Predict the value of a specific attribute (target/dependent variable)based on the value of other attributes (explanatory). Example1: Judge if a patient has specific disease based on his/her medical tests results.

2/15/2014

IS ZC415

33

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Predictive Model
Example2: Credit Card Company Every purchase is placed in 1 of 4 classes Authorize Ask for further identification before authorizing Do not authorize Do not authorize but contact police Two functions of Data Mining Examine historical data to determine how the data fit into 4 classes Apply the model to each new purchase
IS ZC415

2/15/2014

34

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Descriptive Tasks
Objective: To derive patterns (correlation,trends,trajectories) that summarizes the underlying relationship between data. Example: Identifying web pages that are accessed together.

2/15/2014

IS ZC415

35

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Data Mining tasks

Classification

Regression

Predictive
DATA MINING

Time Series
Analysis Prediction

Clustering
Summarizatio n

Descriptive

Association

Rules Sequence
Discovery Figure taken from M H Dunham book on Data Mining
2/15/2014 IS ZC415 36
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Summary

2/15/2014

IS ZC415

37

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Conferences and Journals on Data Mining

KDD Conferences ACM SIGKDD Int. Conf. on Knowledge Discovery in Databases and Data Mining (KDD) SIAM Data Mining Conf. (SDM) (IEEE) Int. Conf. on Data Mining (ICDM) Conf. on Principles and practices of Knowledge Discovery and Data Mining (PKDD) Pacific-Asia Conf. on Knowledge Discovery and Data Mining (PAKDD)

Other related conferences

ACM SIGMOD
VLDB (IEEE) ICDE WWW, SIGIR

ICML, CVPR, NIPS

Data Mining and Knowledge Discovery (DAMI or DMKD) IEEE Trans. On Knowledge and Data Eng. (TKDE) KDD Explorations ACM Trans. on KDD
38

Journals

2/15/2014

IS ZC415

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Where to Find References? DBLP, CiteSeer, Google

Data mining and KDD Conferences: ACM-SIGKDD, IEEE-ICDM, SIAM-DM, PKDD, PAKDD, etc. Journal: Data Mining and Knowledge Discovery, KDD Explorations, ACM TKDD Database systems Conferences: ACM-SIGMOD, ACM-PODS, VLDB, IEEE-ICDE, EDBT, ICDT, DASFAA Journals: IEEE-TKDE, ACM-TODS/TOIS, JIIS, J. ACM, VLDB J., Info. Sys., etc. AI & Machine Learning Conferences: Machine learning (ML), AAAI, IJCAI, COLT (Learning Theory), CVPR, NIPS, etc. Journals: Machine Learning, Artificial Intelligence, Knowledge and Information Systems, IEEE-PAMI, etc. Web and IR Conferences: SIGIR, WWW, CIKM, etc. Journals: WWW: Internet and Web Information Systems, Statistics Conferences: Joint Stat. Meeting, etc. Journals: Annals of statistics, etc. Visualization Conference proceedings: CHI, ACM-SIGGraph, etc. Journals: IEEE Trans. visualization and graphics, etc. IS computer ZC415

2/15/2014

39

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Recommended Reference Books

S. Chakrabarti. Mining the Web: Statistical Analysis of Hypertex and Semi-Structured Data. Morgan Kaufmann, 2002 R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2ed., Wiley-Interscience, 2000 T. Dasu and T. Johnson. Exploratory Data Mining and Data Cleaning. John Wiley & Sons, 2003 U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy. Advances in Knowledge Discovery and Data Mining. AAAI/MIT Press, 1996 U. Fayyad, G. Grinstein, and A. Wierse, Information Visualization in Data Mining and Knowledge Discovery, Morgan Kaufmann, 2001 J. Han and M. Kamber. Data Mining: Concepts and Techniques. Morgan Kaufmann, 2nd ed., 2006 D. J. Hand, H. Mannila, and P. Smyth, Principles of Data Mining, MIT Press, 2001 T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag, 2001 B. Liu, Web Data Mining, Springer 2006. T. M. Mitchell, Machine Learning, McGraw Hill, 1997 G. Piatetsky-Shapiro and W. J. Frawley. Knowledge Discovery in Databases. AAAI/MIT Press, 1991 S. M. Weiss and N. Indurkhya, Predictive Data Mining, Morgan Kaufmann, 1998 I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations, Morgan Kaufmann, 2nd ed. 2005
IS ZC415 40

2/15/2014

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Web Resources
Text book's website at:
http://wwwusers.cs.umn.edu/~kumar/dmbook/.

PowerPoints for the text book are available via anonymous ftp at:
ftp://ftp.aw.com/cseng/authors/tan

Other Resources:
http://web.ccsu.edu/datamining/resources.html

2/15/2014

IS ZC415

41

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956