Professional Documents
Culture Documents
Schedule
Oct 26 Oct 28 Nov 2 Nov 4 Nov 9 Machine Learning, Version Space Method Decision Trees Supervised and Unsupervised Learning Perceptron Neural Network
Office hour: exam. (logic)
Outline
Introduction to machine learning
What is machine learning? Applications of machine learning
http://www.jaist.ac.jp/~tsuruoka/lectures/
Hastie, Tibshirani and Friedman (2008). The Elements of Statistical Learning (2nd edition). Springer-Verlag.
Unsupervised learning
No output is given Analyses relations between instances
Reinforcement learning
Supervision is given via rewards
Too many rules Hard to keep consistency Each rule may not be completely correct
Concept Learning Training examples Representing hypotheses Find-S algorithm Version space Candidate-Elimination algorithm
attributes
Humidity Normal High High High Wind Strong Strong Strong Strong Water Warm Warm Warm Cool Forecast Same Same Change Change EnjoySport Yes Yes No Yes
Hypotheses
Representing hypotheses
h1 = <Sunny, ?, ?, Strong, ?, ?>
Weather = Sunny, Wind = Strong (the other attributes can be any values)
h2 = <Sunny, ?, ?, ?, ?, ?>
Weather = Sunny
Find-S Algorithm
1. Initialize h to the most specific hypothesis in H 2. For each positive training instance x
For each attribute constraint ai in h
If the constraint ai is satisfied by x Then do nothing Else replace ai in h by the next more general constraint that is satisfied by x
3. Output hypothesis h
Example
h0 = <0, 0, 0, 0, 0, 0> x1 = <Sunny, Warm, Normal, Strong, Warm, Same>, yes h1 = <Sunny, Warm, Normal, Strong, Warm, Same> x2 = <Sunny, Warm, High, Strong, Warm, Same>, yes h2 = <Sunny, Warm, ?, Strong, Warm, Same> x3 = <Rainy, Cold, High, Strong, Warm, Change>, no h3 = <Sunny, Warm, ?, Strong, Warm, Same> x4 = <Sunny, Warm, High, Strong, Cool, Change>, yes h4 = <Sunny, Warm, ?, Strong, ?, ?>
Version Space
Definition
Hypothesis space H Training examples D Version space:
VSH , D h H Consistenth, D
The subset of hypotheses from H consistent with the training examples in D
LIST-THEN-ELIMINATE algorithm
1. VersionSpace a list containing every hypothesis in H 2. For each training example, <x, c(x)>
Remove from VersionSpace any hypothesis h for which h(x) c(x)
Version Space
Specific boundary and General boundary
S: { <Sunny, Warm, ?, Strong, ?, ?> }
The version space can be represented with S and G. You dont have to list all the hypotheses.
Candidate-Elimination algorithm
Initialization
G: the set of maximally general hypotheses in H S: the set of maximally specific hypotheses in H
Remove from S any hypothesis that is more general than another hypothesis in S
If d is a negative example
Example
1st training example
<Sunny, Warm, Normal, Strong, Warm, Same>, yes S0: { <0, 0, 0, 0, 0, 0> } S1: { <Sunny, Warm, Normal, Strong, Warm, Same> }
Example
2nd training example
<Sunny, Warm, High, Strong, Warm, Same>, yes S1: { <Sunny, Warm, Normal, Strong, Warm, Same> }
Example
3rd training example <Rainy, Cold, High, Strong, Warm, Change>, no S2,S3 :{ <Sunny, Warm, ?, Strong, Warm, Same> }
G2 : { <?, ?, ?, ?, ?, ?> }
Example
4th training example <Sunny, Warm, High, Strong, Cool, Change>, yes S3 :{ <Sunny, Warm, ?, Strong, Warm, Same> } S4 :{ <Sunny, Warm, ?, Strong, ?, ?> }
<Sunny, ?, ?, Strong, ?, ?> <Sunny, Warm, ?, ?, ?, ?> <?, Warm, ?, Strong, ?, ?>