Artificial Intelligence

Fundamentals

Learning: Identification Trees,

Disorder
 Egg fight problem
Champion ? Multiple Big ? Origin ? Top?
layers of
painting ?
No ? No Chicken Round
No Yes Yes Duck Pointed
Yes ? No Guinea hen Round
Yes Yes No Guinea hen Pointed
Yes No No Chicken Pointed
No No Yes Duck Flat
No ? No Chicken Flat
No ? Yes Chicken Flat

Why we cannot use a nearest neighbor algorithm?

 Data set
• Non numeric
• Some characteristics don’t matter
• Some characteristics do matter, but only part
of the time
• Cost – some of the tests can be more
expensive that others
We need a method that enables computers to learn by assembling tests
into an identification tree.

How we build that tree ? What will be a good characteristic of that tree?
 Occam’s razor
• The word is inherently simple. Therefore the
smallest identification tree that is consistent
with the samples is the one that is most likely
to identify unknown objects correctly.
• How we can construct the smallest
identification tree?
 Tests
Painting layers Big
0 3
? Yes No Yes No

--- + -+ -+ --- -- +++

Origin Top
4 3
C G R P F
D

---+ -- ++ -+ ++- ---

 Big 2
Painting layers 4
Yes No
? Yes No
+- --
--- +

Origin Top 4
R F
C D G P
- + --
 Final identification tree

Origin

C D G

Painting layers No Yes

? No

No Yes
Measuring the disorder - entropy
P P N N
D( set ) =
− log 2 − log 2
T T T T
1.0
0.9 entropy – convenient mechanism
0.8
linear
gaussian

0 1/3 1/2 1 P/T

 Measuring the quality of the test
# of samples in set
Q(Test ) = ∑
SETS
D( set ) *
# of samples handled by test
PRODUCED

Painting layers
1/2 No 1/4
? Yes 1/4

--- + -+ -+
0.8 1 1

1 1 1
Q(Test Painting)= 0.8* + 1* + 1* = 0.9
2 4 4
 Numeric data
Chance of
precipitation

Threshold3

Threshold2

Threshold1 Temperature
 Related resources

Readings
• Artificial Intelligence (3rd Edition), Patrick Winston, Chapter 21