Tracking Signal

A Measure of Forecast Accuracy

Prepared by:

Tyler Hedin

Agenda
Tracking Signal Defined
Tracking Signal and Forecasting

Application, Advantages & Disadvantages How it works

Step by step formula

Company XYZ Example Exercise Summary Readings list Useful websites Appendix A

What is Tracking Signal?

A measure that indicates whether the forecast average is keeping pace with any genuine upward or downward changes in demand*

Tracking Signal and Forecasting

Continuous control indicator Monitor effectiveness of forecasting method Provide control limits

Application
Evaluates forecasting method Indicator of change in demand patterns Used in conjunction with anything dependent on future demand
Sales Inventory

Advantages
Unbiased Versatile
Can be used with any type of forecasting method (time series, regression line, etc.)

Disadvantages
Could wrongfully flag perfect forecasts
Unlikely

Small differences in the same direction could cause signal to go outside of control limits

How it Works Forecast Error

Difference between actual demand and forecast

Week

Actual Demand 21 25 22

Forecasted Demand 19 22 24

Forecast Error 2 3 -2

1 2 3

How it Works Absolute Values

Express the forecast errors as absolute values
Week 1 2 3 Actual Demand 21 25 22 Forecasted Demand 19 22 24 Forecast Error 2 3 -2 Absolute Value 2 3 2

How it Works Running Sum

Keep a continuous running sum of the forecast errors Do not add absolute values
Week 1 2 3 Actual Demand 21 25 22 Forecasted Demand 19 22 24 Forecast Error 2 3 -2 Absolute Value 2 3 2 Running Sum 2 5 3

How it Works - MAD

Divide the summed absolute values by the number of periods to calculate MAD.
Week 1 2 3 Actual Demand 21 25 22 Forecasted Demand 19 22 24 Forecast Error 2 3 -2 Absolute Value 2 3 2 Running Sum 2 5 3 MAD

2.00 2.50 2.33

The Equation
Tracking signal is mathematically defined as the sum of the forecast errors divided by the mean absolute deviation (Dt Ft) Tracking signal = MAD

How it Works Tracking Signal

Divide the running sum of forecast errors by the corresponding MAD value

Week 1 2 3

Actual Demand 21 25 22

Forecasted Demand 19 22 24

Forecast Error 2 3 -2

Absolute Value 2 3 -2

Running Sum 2 5 3

MAD

Tracking Signal 1.00 2.00 1.29

2.00 2.50 2.33

What Do These Values Mean?

Ratio of cumulative error to average deviation 0.8 ~ 1.25 MAD Limits are usually between 2 to 5 standard deviations

Example 1
Company XYZ has implemented a linear regression method to forecast sales. Actual sales for the months of January 2005 through January 2006 are given in Table 1 along with their corresponding forecasts.

Table 1
SALES (in thousands) MONTH January-05 February-05 March-05 April-05 May-05 June-05 July-05 August-05 September-05 October-05 PERIOD 1 2 3 4 5 6 7 8 9 10 DEMAND $37 $40 $41 $37 $45 $50 $43 $47 $56 $52 FORECAST 37.35 38.97 40.60 42.23 43.85 45.48 47.10 48.73 50.36 51.98

November-05
December-05 January-06

11
12 13

$55
$54 $55

53.61
55.23 56.86

Example 1
Company XYZ would like to employ a tracking signal to measure the performance of its forecasting method.

Table 2
TRACKING TRACKING SIGNAL
MONTH January-05 February-05 March-05 April-05 May-05 June-05 July-05 August-05 September-05 October-05 November-05 PERIOD 1 2 3 4 5 6 7 8 9 10 11 DEMAND 37 40 41 37 45 50 43 47 56 52 55 FORECAST 37.35 38.97 40.60 42.23 43.85 45.48 47.10 48.73 50.36 51.98 53.61 ERROR -0.35 1.03 0.40 -5.23 1.15 4.52 -4.10 -1.73 5.64 0.02 1.39 ABS DVN 0.35 1.03 0.40 5.23 1.15 4.52 4.10 1.73 5.64 0.02 1.39 RUNNING SUM -0.35 0.68 1.08 -4.15 -3.00 1.52 -2.58 -4.31 1.33 1.35 2.74 MAD 0.35 0.69 0.59 1.75 1.63 2.11 2.40 2.31 2.68 2.42 2.32

-1.00 0.99 1.82

SIGNAL -1.00 0.99 1.82

-2.37-2.37 -1.84-1.84

0.72

0.72

-1.08-1.08 -1.86-1.86 0.50

0.50 0.56 1.18

0.56
1.18 0.68 -0.16

December-05 January-06

12 13

54 55

55.23 56.86

-1.23 -1.86

1.23 1.86

1.51 -0.35

2.23 2.20

0.68 -0.16

Exercise
Your employer, Jones & Associates, has been using a linear regression method to forecast sales for 2006. After nine months have passed and actual sales data have been collected, your boss asks you to develop a tracking signal to measure the accuracy of the forecasts. The data for actual sales and forecasted sales is in Table 3.

Table 3
SALES MONTH January-06 February-06 March-06 April-06 May-06 June-06 PERIOD 1 2 3 4 5 6 DEMAND $3,769 $3,912 $4,212 $4,861 $4,672 $4,937 FORECAST 3664.18 3953.92 4243.65 4533.39 4823.13 5112.87

July-06
August-06 September-06

7
8 9

$5,346
$5,783 $6,021

5402.61
5692.35 5982.08

Summary
A tracking signal statistically determines if a forecasting method is out-of-control.
As long as tracking signal stays within 3 standard deviations, probability of forecast error caused by random variation is high

Used by companies to track changes in demand patterns Calculated by dividing the most recent sum of forecast errors by the most recent estimate of MAD A tracking signal outside of established limits indicates that a forecasting method should be modified. Compatible with any forecasting method

Readings List
Chase, R. B. et al. (2004). Operations Management for Competitive Advantage 10th edition. McGraw-Hill Higher Education. Duncan, Robert M. (1992). Quality Forecasting Drives Quality Inventory at GE. Industrial Engineer, January edition. Hanke, J.E. & Wichern, D. W. (2004). Business Forecasting. Prentice Hall. Lawrence, F. B. (1999). Closing the logistics loop: A tutorial. Production & Inventory Management Journal, 40(1).

Useful Websites
http://www.bestforecastingsoftware.com http://www.IdeaWins.com http://www.lehigh.edu/~rhs2/IBE098/forecating.ppt http://is.ba.ttu.edu/faculty/ch11.ppt http://www.microsoft.com/dynamics/intro/default.mspx

Appendix A Solution to Exercise

MONTH January-05 February-05 March-05 April-05 May-05 June-05 July-05 August-05 September-05 October-05 PERIOD 1 2 3 4 5 6 7 8 9 10 DEMAND 37 40 41 37 45 50 43 47 56 52 FORECAST 37.35 38.97 40.60 42.23 43.85 45.48 47.10 48.73 50.36 51.98 ERROR -0.35 1.03 0.40 -5.23 1.15 4.52 -4.10 -1.73 5.64 0.02 ABS DVN 0.35 1.03 0.40 5.23 1.15 4.52 4.10 1.73 5.64 0.02 RUNNING SUM -0.35 0.68 1.08 -4.15 -3.00 1.52 -2.58 -4.31 1.33 1.35 MAD 0.35 0.69 0.59 1.75 1.63 2.11 2.40 2.31 2.68 2.42 TRACKING SIGNAL -1.00 0.99 1.82 -2.37 -1.84 0.72 -1.08 -1.86 0.50 0.56

November-05
December-05 January-06

11
12 13

55
54 55

53.61
55.23 56.86

1.39
-1.23 -1.86

1.39
1.23 1.86

2.74
1.51 -0.35

2.32
2.23 2.20

1.18
0.68 -0.16