You are on page 1of 17

Data Mining and

Neural Networks

Danny Leung
CS157B, Spring 2006
Professor Sin-Min Lee
Artificial Intelligence
for Data Mining
Neural networks are useful for data mining and
decision-support applications.

People are good at generalizing from experience.

Computers excel at following explicit instructions


over and over.

Neural networks bridge this gap by modeling, on


a computer, the neural behavior of human
brains.

2
Neural Network
Characteristics

Neural networks are useful for pattern


recognition or data classification,
through a learning process.

Neural networks simulate biological


systems, where learning involves
adjustments to the synaptic
connections between neurons
3
Anatomy of a Neural
Network
Neural Networks map a
set of input-nodes to a
set of output-nodes

Number of
inputs/outputs is variable

The Network itself is


composed of an arbitrary
number of nodes with an
arbitrary topology

4
Biological Background
A neuron: many-inputs / one-output unit

Output can be excited or not excited

Incoming signals from other neurons


determine if the neuron shall excite ("fire")

Output subject to attenuation in the


synapses, which are junction parts of the
neuron

5
Basics of a Node
A node is an
element which
performs a
function

y = fH((wixi) +
Connection
Wb)
Node

6
A Simple Preceptron
Binary logic
application

fH(x) [linear
threshold]

Wi = random(-1,1)

Y = u(W0X0 + W1X1
+ Wb)
7
Preceptron Training
Its a single-unit network

Adjust weights based on a how well the current


weights match an objective

Perceptron Learning Rule

Wi = * (D-Y).Ii

= Learning Rate
D = Desired Output

8
Neural Network
Learning
From experience: examples / training
data

Strength of connection between the


neurons is stored as a weight-value for
the specific connection

Learning the solution to a problem =


changing the connection weights
9
Neural Network
Learning
Continuous Learning Process

Evaluate output

Adapt weights

Take new inputs

Learning causes stable state of the


weights
10
Learning Performance
Supervised
Need to be trained ahead of time with lots of data

Unsupervised networks adapt to the input


Applications in Clustering and reducing dimensionality
Learning may be very slow
No help from the outside
No training data, no information available on the
desired output
Learning by doing
Used to pick out structure in the input:
Clustering
Compression

11
Topologies Back-
Propogated Networks
Inputs are put
through a
Hidden Layer
before the output
layer

All nodes
connected
between layers
12
BP Network
Supervised Training
Desired output of the training examples

Error = difference between actual & desired output

Change weight relative to error size

Calculate output layer error , then propagate back to


previous layer

Hidden weights updated

Improved performance
13
Neural Network
Topology
Characteristics
Set of inputs

Set of hidden nodes

Set of outputs

Increasing nodes makes network more


difficult to train
14
Applications of Neural
Networks
Prediction weather, stocks, disease

Classification financial risk assessment, image


processing

Data association Text Recognition (OCR)

Data conceptualization Customer purchasing


habits

Filtering Normalizing telephone signals (static)

15
Overview
Advantages
Adapt to unknown situations
Robustness: fault tolerance due to network
redundancy
Autonomous learning and generalization

Disadvantages
Not exact
Large complexity of the network structure
16
Referenced Work
Intro to Neural Networks - Computer Vision Applications and Training
Techniques. Doug Gray. www.soe.ucsc.edu/~taoswap/
GroupMeeting/NN_Doug_2004_12_1.ppt

Introduction to Artificial Neural Networks. Nicolas Galoppo von


Borries. www.cs.unc.edu/~nico/courses/ comp290-58/nn-
presentation/ann-intro.ppt

17