What is a decision tree ?

•Decision tree is a classifier in the form of a tree structure , where each

node is either:
• a leaf node - indicates the value of the target attribute (class) of examples
• a decision node - specifies some test to be carried out on a single attribute-value,
with one branch and sub-tree for each possible outcome of the test.
• Decision Trees are useful tools for helping you to choose between
several courses of action

B K=X

Decision Node Leaf Node

 What is a decision tree ? (contd..)

• Particularly useful for choosing between different strategies, projects

or investment opportunities, particularly when your resources are
limited.
• Provide a highly effective structure within which you can explore
options, and investigate the possible outcomes of choosing those
options
An example of a simple decision tree
 What is a decision tree ? (contd..)

• A decision tree can be used to classify an example by starting at the

root of the tree and moving through it until a leaf node, which
provides the classification of the instance.
• Decision Tree Representation:
– Each internal node test an attribute
– Each branch corresponds to attribute value
– Each leaf node assigns a classification
 When to consider Decision Tree ?

• Instances describable by attributes – value pairs

• Target Function is discrete valued
• Possibly noisy training data

Examples
• Equipment or medical diagnosis
• Credit risk analysis
• Modeling calendar scheduling preferences
 Converting Decision to Rules
IF ((A=‘Red’) ^( B< 4.5))
THEN K=Y
IF((A=‘Red’ ) ^ (B>=4.5))
THEN K=X
Etc..
 The strengths of decision tree
• Decision trees :
• are Simple to understand and interpret.
– People are able to understand decision tree models after a brief
explanation.
• are able to generate understandable rules.
• requires little data preparation
– Other techniques often require data normalization, dummy variables
need to be created and blank values to be removed.
• perform classification without requiring much computation.
• possible to validate a model using statistical tests.
– makes it possible to account for the reliability of the model.
• are able to handle both continuous and categorical variables.
– Ex: relation rules can be used only with nominal variables while neural
networks can be used only with numerical variables.
• provide a clear indication of which fields are most important
for prediction or classification.
 Weaknesses of decision tree

• Decision trees are less appropriate for estimation tasks where

the goal is to predict the value of a continuous attribute.
• Decision-tree learners create over-complex trees that do not
generalise the data well
• Decision tree can be computationally expensive.
-The process of growing a decision tree is computationally expensive. At each
node, each candidate splitting field must be sorted before its best split can
be found. In some algorithms, combinations of fields are used and a search
must be made for optimal combining weights.
– There are concepts that are hard to learn because decision
trees do not express them easily
Practical Example 1:
• In the example in Figure 2, the value for 'new product,
thorough development' is:

• 0.4 (probability good outcome) x $1,000,000 (value) = $400,000

• 0.4 (probability moderate outcome) x £50,000 (value) = $20,000
• 0.2 (probability poor outcome) x £2,000 (value) = $400
• + $420,400
Practical Example 2:
• Assume XYZ Corporation wishes to
introduce one of two products to the
market this year. The probabilities and
present values (PV) of projected cash
inflows follow:
Practical Example(contd..)
Based on the expected net present value, the company should choose
product A over product B.
 Queries,
Comments
&
Suggestions
???
 References

• http://dms.irb.hr/tutorial/tut_dtrees.php
• http://www.mindtools.com/dectree.html
• http://en.wikipedia.org/wiki/Decision_tree_l
earning
• http://www.answers.com/topic/decision-
tree