DYNAMIC PROGRAMMING

Introduction
• Dynamic programming is a useful mathematical technique for making a sequence of
interrelated decisions.
• It provides a systematic procedure for determining the optimal combination of decisions.
• In contrast to linear programming, there does not exist a standard mathematical
formulation of “the” dynamic programming problem.
• Rather, dynamic programming is a general type of approach to problem solving, and the
particular equations used must be developed to fit each situation.
• Therefore, a certain degree of ingenuity and insight into the general structure of dynamic
programming problems is required to recognize when and how a problem can be solved
by dynamic programming procedures.
• These abilities can best be developed by an exposure to a wide variety of dynamic
programming applications and a study of the characteristics that are common to all these
situations.
• The technique is useful in a large number of multi-period business problems, such as
(a) smoothing production employment,
(b) allocating capital funds,
(c) Allocating salespeople to marketing areas, and (d) evaluating investment
opportunities.

Definition
the mathematical technique of optimizing a sequence of inter-related decisions over a
period of time is called dynamic programming.

Differences between Linear Programming (LP) and Dynamic Programming (DP)

(1) First, there is no algorithm (like the simplex method) that can be programmed to solve all
problems. Instead, dynamic programming is a technique that allows a difficult problem to be
broken down into a sequence of easier sub-problems, which are then evaluated by stages. (2)
Second, linear programming is a method that gives single-stage (i.e., one-time period) solutions.
Dynamic programming has the power to determine the optimal solution over a one year time
horizon by breaking the problem into 12 smaller one-month horizon problems and to solve each
of these optimally. Hence, it uses a multistage approach.

Dynamic Programming uses the backward recursive method for solving the problems
Bellman’s Principle of Optimality Bellman’s principle of optimality is the principle which
guides the solution of a problem using dynamic programming.
It states:

1
 Regardless of the decisions taken to enter a particular state in a particular stage,
the remaining decisions made for leaving that stage must constitute an optimal
policy”

Steps in Dynamic Programming

1. Divide the original problem into sub-problems called stages.

2. Solve the last stage of the problem for all possible conditions or states.

3. Working backward from that last stage, solve each intermediate stage.

4. Obtain the optimal solution for the original problem by solving all stages sequentially.

Terminology of Dynamic Programming

1. Stage: a period or a logical sub-problem.

2. State variables: possible beginning situations or conditions of a stage. These have also
been called the input variables.

3. Decision variables: alternatives or possible decisions that exist at each stage.

4. Decision criterion: a statement concerning the objective of the problem.

5. Optimal policy: a set of decision rules, developed as a result of the decision criteria, that
gives optimal decisions for any entering condition at any stage.

6. Transformation: normally, an algebraic statement that reveals the relationship between

stages.

In most applications, dynamic programming obtains solutions by working backward

from the end of the problem toward the beginning, thus breaking up a large, unwieldy
problem into a series of smaller, more tractable problems

Example: Shortest Route Problem

Find the shortest path from City 1 to City 10 in the diagram shown below using
the recursive principle of Dynamic Programming.

2
Solution

Stage 1
• There is only one path from node (8) to node (10). The distance of (8) – (10) = 6.
• Therefore, label node (8) with 6. Indicate the segment (8) –(10) with a *
• There is only one path from node (9) to node (10). The distance of (9) – (10) = 7.
therefore, label node (9) with 7. Indicate the segment (9) –(10) with a *

Stage 2 from node (5)

• There are two paths from (5) to (10) viz., (5)-(8)-(10) and (5)-(9)-(10)
• Distance of path (5)-(8)-(10) = Length of (5)-(8) + Label of (8) = 5 + 6 = 11
• Distance of path (5)-(9)-(10) = Length of (5)-(9) + Label of (9) = 3 + 7 = 10
• Therefore label of (5) = Min {11, 10} = 10. Indicate (5) –(9) with a *

From node (6)

• There are two paths from (6) to (10) viz., (6)-(8)-(10) and (6)-(9)-(10)
• Distance of path (6)-(8)-(10) = Length of (6)-(8) + Label of (8) = 7 + 6 = 13
• Distance of path (6)-(9)-(10) = Length of (6)-(9) + Label of (9) = 8 + 7 = 15
• Therefore label of (6) = Min {13, 15} = 13. Indicate (6) –(8) with a *

From node (7)

• There are two paths from (7) to (10) viz., (7)-(8)-(10) and (7)-(9)-(10)
• Distance of path (7)-(8)-(10) = Length of (7)-(8) + Label of (8) = 2 + 6 = 8
• Distance of path (7)-(9)-(10) = Length of (7)-(9) + Label of (9) = 4 + 7 = 11
• Therefore label of (7) = Min {8, 11} = 8. Indicate (7) –(8) with a *

3
Stage 3 from node (2)
• There are three paths from (2) to (10) viz., (2)-(5), (2)-(6) and (2)-(7)
• Distance of path (2)-(5) = Length of (2)-(5) + Label of (5) = 4 +10 = 14
• Distance of path (2)-(6) = Length of (2)-(6) + Label of (6) = 2 + 13 = 15
• Distance of path (2)-(7) = Length of (2)-(7) + Label of (7) = 5 + 8 = 13
• Therefore label of (2) = Min {14, 15, 13} = 13. Indicate (2) –(7) with a *

From node (3)

• There are three paths from (3) to (10) viz., (3)-(5), (3)-(6) and (3)-(7)
• Distance of path (3)-(5) = Length of (3)-(5) + Label of (5) = 4 +10 = 14
• Distance of path (3)-(6) = Length of (3)-(6) + Label of (6) = 3 + 13 = 16
• Distance of path (3)-(7) = Length of (3)-(7) + Label of (7) = 7 + 8 = 15
• Therefore label of (3) = Min {14, 16, 15} = 14. Indicate (3) –(5) with a *

From node (4)

• There are three paths from (4) to (10) viz., (4)-(5), (4)-(6) and (4)-(7)
• Distance of path (4)-(5) = Length of (4)-(5) + Label of (5) = 2 +10 = 12
• Distance of path (4)-(6) = Length of (4)-(6) + Label of (6) = 1 + 13 = 14
• Distance of path (4)-(7) = Length of (4)-(7) + Label of (7) = 5 + 8 = 13
• Therefore label of (4) = Min {12, 14, 13} = 12. Indicate (4) –(5) with a *

Stage 4 from node (1)

• There are three paths from (1) to (10) viz., (1)-(2), (1)-(3) and (1)-(4)
• Distance of path (1)-(2) = Length of (1)-(2) + Label of (2) = 5 +13 = 18
• Distance of path (1)-(3) = Length of (1)-(3) + Label of (3) = 7 + 14 = 21
• Distance of path (1)-(4) = Length of (1)-(4) + Label of (4) = 3 + 12 = 15
• Therefore label of (1) = Min {18, 21, 15} = 15. Indicate (1) –(4) with a *
• Since the label of (1) is 15, the distance of the shortest path from (1) to (10) = 15.

Starting from (1) and following the paths indicated by *s,

the shortest path is (1)-(4)-(5)-(9)-(10). Its distance is 15.

4
 SIMULATION
INTRUDCTION

 A simulation of a system is the operation of a model. The model can be reconfigured and
experimented with; The operation of the model can be studied, and hence, properties
concerning the behaviour of the actual system or its subsystem.

 In its broadest sense, simulation is a tool to evaluate the performance of a system, existing or
proposed, under different configurations of interest and over long periods of real time.

 Simulation is used before an existing system is altered or a new system built, to reduce the
chances of failure to meet specifications, to eliminate unforeseen bottlenecks, to prevent
under or overutilization of resources, and to optimize system performance.

Advantages of simulation
 New policies, operating procedures, information flows and son on can be explored without
disrupting ongoing operation of the real system.

 New hardware designs, physical layouts, transportation systems and … can be tested
without committing resources for their acquisition.

 Time can be compressed or expanded to allow for a speedup or slow-down of the

phenomenon (clock is self-control).

 Insight can be obtained about interaction of variables and important variables to the
performance.

 Bottleneck analysis can be performed to discover where work in process, the system is
delayed.

 A simulation study can help in understanding how the system operates.

 “What if” questions can be answered?

5
Disadvantages of simulation
Model building requires special training.
- Vendors of simulation software have been actively developing packages that contain
models that only need input (templates).

 Simulation results can be difficult to interpret.

 Simulation modelling and analysis can be time consuming and expensive.

- Many simulation software have output-analysis.