BACKTRACKING

GENERAL METHOD
• Problems searching for a set of solutions or which require
an optimal solution can be solved using the backtracking
method .

• To apply the backtrack method, the desired solution must

be expressible as an n-tuple(x1,…,xn), where the xi are
chosen from some finite set si

• The solution vector must minimize or maximize or

satisfy the criterion function P(x1 , ….. , xn).
1
 Example
 Sorting an array of integers in a[1:n] is a
problem whose solution is expressible by an n-
tuple, where xi is the index of the ith smallest
element.

The criterion function P is the inequality

a[xi]≤a[xi+1] for 1≤i≤n. The set si is finite and
includes the integers 1 through n.
 BACKTRACKING (Contd..)
• Suppose there are m n-tuples which are
possible candidates for satisfying the
function P.

• Then m= m1, m2…..mn where miis size of

set si 1<=i<=n.

• The brute force approach would be to form

all of these n-tuples and evaluate each one
with P, saving those which yield the
optimum.
2
 BACKTRACKING (Contd..)

• The backtracking algorithm has the ability to yield

the same answer with far fewer than m-trials.
• In backtracking, the solution is built one
component at a time.
• Modified criterion functions Pi (x1...xn) called
bounding functions are used to test whether the
partial vector (x1,x2,......,xi) being formed has any
chance of success.
• The major advantage of this method is realized that
the partial vector (x1,...xi) can in no way lead to an
optimal solution, then mi+1,....,mn possible test
vector can be ignored.
3
 BACKTRACKING (Contd..)
• The constraints may be of two categories.
• EXPLICIT CONSTRAINTS are rules which restrict each xi to take on
values only from a given set.
Examples xi 0 or si={ all non-negative real numbers}
x1= 0 or 1 or si={ 0,1}
li  xi  ui or si={a: li  a  ui. }
• The explicit constraints depend on the perticular instance I of the problem
being solved. All tuples that satisfy the explicit constraints define a
possible solution space for I.

• IMPLICIT CONSTRAINTSdescribe the way in which the xi must

relate to each other .
• Implicit constraints are rules that determine which of the tuples in
4
the solution space that satisfy the criterion function.
Create PDF files without this message by purchasing novaPDF printer (http://www.novapdf.com)
 Example : 8 queens problem
• The problem is to place eight queens on an 8 x 8 chess
board so that no two queens attack i.e. no two of them are
on the same row, column or diagonal.
• Strategy : The rows and columns are numbered
through 1 to 8.

• The queens are also numbered through 1 to 8.

• Since each queen is to be on a different row without loss of

generality, we assume queen i is to be placed on row i .
5
 8 queens problem (Contd..)

• The solution is an 8 tuple (x1,x2,.....,x8) where xi is the

column on which queen i is placed.

• The explicit constraints are :

Si = {1,2,3,4,5,6,7,8} 1  i n or 1  xi  8,

6
• The solution space consists of 88 8- tuples.
Create PDF files without this message by purchasing novaPDF printer (http://www.novapdf.com)
 8 queens problem (Contd..)

The implicit constraints are :

(i) no two xis can be the same that is, all queens must
be on different columns.

(ii) no two queens can be on the same diagonal.

(i) reduces the size of solution space from 88 to 8! 8 –
tuples.

Two solutions are (4,6,8,2,7,1,3,5) and

(3,8,4,7,1,6,2,5)
7

Create PDF files without this message by purchasing novaPDF printer (http://www.novapdf.com)
 8 queens problem (Contd..)

1 2 3 4 5 6 7 8
1 Q
2 Q
3 Q
4 Q
5 Q
6 Q
7 Q
8 Q
8
 State space tree representation
Solution Space :
• Tuples that satisfy the explicit constraints define a solution space.

• The solution space can be organized into a tree.

• Each node in the tree defines a problem state.

• All paths from the root to other nodes define the state- space of the
problem.

• Solution States: are those states leading to a tuple in the solution

space.

• Answer Nodes: are those solution states leading to an answer-tuple( i.e.

tuples which satisfy implicit constraints).
9
 State space tree representation contd..

• The problem may be solved by systematically generating

the problem states determining which are solution states,
and determining the answer states.

Let us see the following terminology

• LIVE NODE A node which has been generated and all of
whose children are not yet been generated .

• E-NODE (Node being expanded) - The live node whose

children are currently being generated . 10
 State space tree representation contd..

• DEAD NODE - A node that is either not to be expanded

further, or for which all of its children have been
generated.

• DEPTH FIRST NODE GENERATION- In this, as soon

as a new child C of the current E-node R is generated, C
will become the new E-node.

• R will become E-node again when C has been fully

explored.
11
 State space tree representation contd..

• BOUNDING FUNCTION - will be used to kill live

nodes without generating all their children.

• BACTRACKING-is depth – first node generation

with bounding functions.

• BRANCH-and-BOUND is a method in which E-

node remains E-node until it is dead.
 State space tree representation contd..

• BREADTH-FIRST-SEARCH : Branch-and Bound with

each new node placed in a queue . The front of the
queen becomes the new E- node.

• DEPTH-SEARCH (D-Search) : New nodes are placed

into a stack.

The last node added is the first to be explored.

13
 Example : 4 Queens problem

1 1 1 1
. . 2 2 2
3
. . . .

1 1
2
3
. . 4
14
 State space tree: 4 Queens problem

1
x1 = 1 x1=2
2 18
x2=2 3 4 x2=1 x2=3 x2 = 4
B 3 8 13 19 24 29
x3=3 x3=4 2 3 B B
4 6 14 16 x3 = 1
x4=4 3 B 30
5 7 15 x4 = 3
B 31
15
State space tree: 4 Queens problem contd..

• If (x1….xi) is the path to the current E-node

, a bounding function has the criterion that
(x1..xi+1) represents a chessboard
configuration, in which no queens are
attacking.
• A node that gets killed as a result of the
bounding function has a B under it.

16
State space tree: 4 Queens problem contd..

• We start with root node as the only live node. The

path is ( ); we generate a child node 2.
• The path is (1).This corresponds to placing queen
1 on column 1 .
• Node 2 becomes the E node. Node 3 is generated
and immediately killed. (because x1=1,x2=2).
• As node 3 is killed, nodes 4,5,6,7 need not be
generated.

17
State space tree: 4 Queens problem contd..
• Node 8 is generated, and the path is (1,3).
• Node 8 gets killed as all its children
represent board configurations that cannot
lead to answer. We backtrack to node 2 and
generate another child node 13.
• But the path (1,4) cannot lead to answer
nodes.
• So , we backtrack to 1 and generate the
path (2) with node 18. We observe that the
path to answer node is (2 4 1 3 )
• Other solution is (3, 1, 4, 2)
18
 GENERAL BACKTRACKING METHOD

• All answer nodes are to be found

• If (x1…..xi) is a path from root to a node then T (x1,….,xi)
be the set of all possible values for Xi+1,such that
(x1,x2,…….,xi,xi+1) is also a path from root to a problem
state.

• B(x1…xi+1) or Bxi+1is false for the path (x1,..,xi+1) if the path

cannot reach an answer node.

• The solution vectors X (1: n) are those values which are

generated by T and satisfy Bi+1.
19
 The 8-Queen’s Problem

Consider an n×n chessboard and try to find all ways to place n

nonattacking queens. We observed from the 4-queens problem that
we can let{x1,…..,xn} represent solution in which xi is the column
of the ith queen is placed. The xi will be distinct since no two
queens can be placed in the same column. Now how do we test
whether two queens are on the same diagonal?

If we imagine the chessboard squares being numbered as the

indices of the two-dimensional array a[1:n,1:n] then we observe
that every element on the same diagonal that runs from upper left
to the lower right has the same row-column value.
 8 queens problem (Contd..)

1 2 3 4 5 6 7 8
1 Q
2 Q
3 Q
4 Q
5 Q
6 Q
7 Q
8 Q
8
 8 queens problem (Contd..)

Consider the queen at a[4,2]. The squares that are diagonal

that runs from the upper left to lower right are a[3,1], a[5,3],
a[6,4], a[7,5] and a[8,6].

All these squares have a row-column value of 2. Also, every

element on the same diagonal that goes from the upper right to
lower left has the same row + column value. Suppose two
queens are placed at positions (i,j) and (k,l). Then by the above
they are on the same diagonal only if.
 8 queens problem (Contd..)

 i-j=k-l or i+j=k+l

he first equation implies

j-l = i-k

The second implies

j-l=k-i

Therefore two queens lie on the same diagonal if and only if

| j-l | = | i-k |.
Place(k,i) algorithm returns a Boolean value that is true if the kth
queen can be placed in column i. It tests both whether i is distinct
from all previous values x[1],…..,x[k-1] and whether there is no
other queen on the same diagonal. Its computing time is O(k-1).
Using Place, we can refine the general backtracking method as
given by algorithm and give a precise solution to the n-queen
problem. The array x[] is global. The algorithm is invoked by
Nqueens(1,n);
 SUM OF SUBSETS
 Suppose we are given n distinct positive numbers (usually
called weights) and we desire to find all combinations of these
numbers whose sums are m. This is called the sum of subsets
problem.

Example
Given positive numbers wi, 1<=i<=n, and m, if n=4,
(w1,w2,w3,w4)=(11,13,24,7), and m=31, then the desired
subsets are (11,13,7) and (24,7). Rather than representing
solution vector by the wi, which sum is m, we could represent
the solution vector by giving the indices of these wi. Now the
two solutions are described by the vectors(1,2,4) and (3,4).
 SUM OF SUBSETS
In general, all solutions are k-tuples (x1,x2,…..,xk), 1≤k ≤ n,
and different solutions may have different sized tuples.

Explicit constrain: xiϵ {j| j is an integer and 1 ≤ j ≤ n}.

Implicit constraint: No two be the same and that the sum of
the corresponding wi’s be m.

Since, we wish to avoid generating multiple instances of the

same subset (e.g, (1,2,4) and (1,4,2) represent the same
subset). Another implicit constraint that is imposed is that
xi<xi+1, 1 ≤ i ≤ k.
 SUM OF SUBSETS
 We could formulate this problem using either fixed or variable
sized tuples.

VARIABLE SIZED TUPLES

 VARIABLE TUPLE SIZE
The tree figure corresponds to the variable tuple size
formulation. The edges are labeled such that an edge from level
i node to a level i+1 node represents a value xi.

At each node, the solution space is defined by all paths from
the root node to any node in the tree, since any such path
corresponds to subset satisfying the explicit constrains.

The possible paths are (1), (1,2), (1,2,3), (1,2,3,4), (1,2,4),

(1,3,4), (2), (2,3), (2,3,4), (2,4), (3,4), (4). Thus the leftmost
subtree defines all subsets containing w1, the next subtree
defines all subsets containing w2 but not w1, an so on.
FIXED TUPLE SIZE
 FIXED TUPLE SIZE
The tree figure corresponds to fixed tuple sized formulation.
Edges from level i nodes to level i+1 nodes are labelled with the
value of xi, which is either zero or one.

All paths from root to a leaf node define the solution space. The
left subtree of the root defines all subsets containing w1, the
right subtree of the root defines all subsets not containing w1,
and so on.

Now there are 24 leaf nodes which represent 16 possible tuples.

A Simple choice for the bounding function is
Bk(x1,x2,….,xk) = true iff

Clearly x1,….xk cannot lead to an answer node if

this condition is not satisfied.
The bounding functions can be strengthened if we assume the
wi’s are initially in non-decreasing order. In this case
x1,….,xk cannot lead to an answer node if

The bounding functions we use are therefore

GRAPH COLORING
Hamiltonian Cycle
 BRANCH AND BOUND

Branch and bound is a general algorithmic method for finding

optimal solutions of various optimization problems.

Branch and bounding method is a general technique that applies

where the greedy method and dynamic programming fail.

However, it is much slower. Indeed, if applied carefully, it can

lead to algorithms that run resonably fast on average.
 GENERAL METHOD
 In branch and bound method a state space tree is built and all
the children of E-nodes ( a live node whose children are currently
being generated) are generated before any other node can become
a live node.
For exploring new nodes either a BFS or D-search technique
can be used.
In branch and Bound technique, BFS like state space search will
be called FIFO(First In First Out)search. This is because the list
of live node is first in first out list(queue). On the other hand the
D-search like state space search will be called LIFO search
because the list of live node is Last in First Out list(stack).
 Ways to select E-Node
1

2 3 4 5

2 3 4 5
8 9 6 7

LIFO Branch & Bound (D-Search)

Live Node: 2, 3, 4, and 5 Children of E-node are inserted in
1 a stack.

2 3 4 5

6 7 8 9

FIFO Branch & Bound (BFS)

Children of E-node are inserted in a queue.
 GENERAL METHOD

In this method a space tree of possible solutions are generated.

Then partitioning(called as branching) is done at each node of the
tree.

We compute lower bound and upper bound at each node. This
computation leads to selection of answer node.

Bounding functions are used to avoid the generation of subtrees

that do not contain an answer node.
 LEAST COST SEARCH

 In branch and bound the basic idea is selection of E-node. The

selection of E-node should so perfect that we will reach to answer
node quickly.

Using FIFO and LIFO branch and bound method the selection of
E-node is very complicates and somewhat blind.

For speeding up the search process we need to intelligent ranking

function for live nodes. Each time, the next E-node is selected on
the basis of this ranking function.

The Least Cost (LC) search is a kind of search in which least cost
is involved for reaching to answer node. At each E-node the
probability of being an answer node is checked.
 LEAST COST SEARCH
 BFS and D-search are special cases of LC search.

Each time the next E-node is selected on the basis of the ranking
function(smallest c^(x)). Let g^(x) be an estimate of the additional
effort needed to reach an answer node x. Let h(x) to be the cost of
reaching x from the root and f(x) to be any non-decreasing function
such that
C^(x)=f(h(x))+g^(x).

If we set g^(x)=0 and f(h(x)) to be level of node x then we have

BFS.

If we ser f(h(x))=0 and g^(x) ≤ g^(y) whenever y is a child of x

then the search is a D-search.

An LC search, the cost function c(.) can be defined as

 LEAST COST SEARCH

 i) If x is an answer node then c(x) is the cost computed by

the path from x to root in the state space tree.

ii) If x is not an answer node such that subtree of x node is

also not containing the answer node then c(x)=∞.

iii) Otherwise c(x) is equal to the cost of minimum cost

answer node in subtree x.
C^(.) with f(h(x))=h(x) can be approximation of c(.)
 BOUNDING

•As we know that the bounding functions are used to avoid the
generation of subtrees that do not contain the answer nodes. In
bounding lower bound and upper bounds are generated at each
node.

•A cost function c^(x) is such that c^(x) ≤ c(x) is used to provide

the lower bounds on solution obtained from any node x.

•Let upper is an upper bound on cost of minimum-cost solution.

In that all the live nodes with c^(x)>upper can be killed.

•At the start the upper is usually set to ∞. After generating the
children current E-node, upper can be updated by minimum cost
answer node. Each time a new answer node can be obtained.