Backtracking: Presented By: Smt. Smitha M.L. Research Scholar

Backtracking
Presented by: Smt. Smitha M.L. Research Scholar
Coloring a map
You wish to color a map with not more than four colors red, yellow, green, blue Adjacent countries must be in different colors
Solving a maze

Given a maze, find a path from start to finish At each intersection, you have to decide between three or fewer choices:

Go straight Go left Go right
Outline
Introduction to Backtracking Method The General method The 8-queens Problem Sum of Subsets
Backtracking
Suppose you have to make a series of decisions, among various choices, where
You dont have enough information to know what to choose Each decision leads to a new set of choices Some sequence of choices (possibly more than one) may be a solution to your problem
Backtracking is a methodical way of trying out various sequences of decisions, until you find one that works
5
The Idea
Problem solving: search in a space of possible states.
Target state1
Initial state
Target state2
Generate and Test Method

Take a state Generate next states by applying relevant rules (not all rules are applicable for a given state!) Test to see if the goal state is reached. If yes - stop If no - add the generated states into a stack/queue (do not add if the state has already been generated) and repeat the procedure. To get the solution: Record the paths
7
Backtracking
dead end ? dead end start ? ? ? success! ? dead end dead end dead end
The backtracking method
A given problem has a set of constraints and possibly an objective function The solution optimizes an objective function, and/or is feasible. We can represent the solution space for the problem using a state space tree

The root of the tree represents 0 choices, Nodes at depth 1 represent first choice Nodes at depth 2 represent the second choice, etc. In this tree a path from a root to a leaf represents a candidate solution
Terminology I
A tree is composed of nodes
There are three kinds of nodes: The (one) root node Internal nodes Leaf nodes Backtracking can be thought of as searching a tree for a particular goal leaf node
10
Terminology II
Each non-leaf node in a tree is a parent of one or more other nodes (its children) Each node in the tree, other than the root, has exactly one parent
parent Usually, however, we draw our trees downward, with the root at the top children children
11
parent
The backtracking algorithm
Backtracking is really quite simple--we explore each node, as follows: To explore node N:
1. If N is a goal node, return success 2. If N is a leaf node, return failure 3. For each child C of N, 3.1. Explore C 3.1.1. If C was successful, return success 4. Return failure
12
General Method

Backtracking
Requires less than m trials to determine the solution. Form a solution (partial vector) and check at every step if this has any chance of success. If the solution at any point seems non-promising, ignore it. If the partial vector (x1, x2, . . . , xi) does not yield an optimal solution, ignore mi+1 mn possible test vectors even without looking at them.
All the solutions require a set of constraints divided into two categories: explicit and implicit constraints
13
Definition 1
Explicit constraints are rules that restrict each xi to take on values only from a given set. Explicit constraints depend on the particular instance I of problem being solved. All tuples that satisfy the explicit constraints define a possible solution space for I.
14
Definition 2: Implicit constraints are rules that determine which of the tuples in the solution space of I satisfy the criterion function.
Implicit constraints describe the way in which the xi s must relate to each other. Determine problem solution by systematically searching the solution space for the given problem instance. Use a tree organization for solution space.
15
Backtracking Concept
Backtracking is a systematic way to go through all the possible configurations of a search space. In the general case, we assume our solution is a vector: a = (a[1], a[2], , a[n]) where each element a[i] is selected from a finite ordered set S[i].
16
We build a partial solution of length k a = (a[1], a[2], , a[k]) and try to extend it by adding another element. After extending it, we will test whether what we have so far is still possible as a partial solution.
17
If it is still a candidate solution, great!
If not, we delete a[k] and try the next element from S[k].
18
Backtracking
Construct the state space tree: Root represents an initial state Nodes reflect specific choices made for a solutions components. Promising and nonpromising nodes Leaves Explore the state space tree using depth-first search Prune non-promising nodes DFS stops exploring subtree rooted at nodes leading to no solutions and... backtracks to its parent node
19
State Space Tree of the Four-queens Problem
20
The N-Queens Problem

Suppose you have 8 chess queens... ...and a chess board
21

Can the queens be placed on the board so that no two queens are attacking each other ?
22

Two queens are not allowed in the same row...
23

Two queens are not allowed in the same row, or in the same column...
24

Two queens are not allowed in the same row, or in the same column, or along the same diagonal.
25

The number of queens, and the size of the board can vary. N Queens
N columns
ro ws N
26

We will write a program which tries to find a way to place N queens on an N x N chess board.
27
How the program works ?

The program uses a stack to keep track of where each queen is placed.
28

Each time the program decides to place a queen on the board, the position of the new queen is stored in a record which is placed in the stack.
ROW 1, COL 1
29

We also have an integer variable to keep track of how many rows have been filled so far.
ROW 1, COL 1
filled
30

Each time we try to place a new queen in the next row, we start by placing the queen in the first column...
ROW 2, COL 1 ROW 1, COL 1
filled
31

...if there is a conflict with another queen, then we shift the new queen to the next column.
filled
32

If another conflict occurs, the queen is shifted rightward again.
filled
33

When there are no conflicts, we stop and add one to the value of filled.
filled
34

Let's look at the third row. The first position we try has a conflict...
ROW 3, COL 1 ROW 2, COL 3 ROW 1, COL 1
filled
35

...so we shift to column 2. But another conflict arises...
filled
36

...and we shift to the third column. Yet another conflict arises...
filled
37

...and we shift to column 4. There's still a conflict in column 4, so we try to shift rightward again...
filled
38

...but there's nowhere else to go.
filled
39

When we run out of room in a row: pop the stack, reduce filled by 1 and continue working on the previous row.
filled
40

Now we continue working on row 2, shifting the queen to the right.
filled
41

This position has no conflicts, so we can increase filled by 1, and move to row 3.
filled
42

In row 3, we start again at the first column. Conflict occurs.
filled
43

Then, shift to 2nd column. Increment filled by one.( 3 rows filled)
filled
44

In row 4, we start again at the first column.
ROW 4, COL 1
Conflict occurs
filled
45

Shift to 2nd column. Conflict occurs .
ROW 4, COL 2 ROW 3, COL 2 ROW 2, COL 4 ROW 1, COL 1
filled
46

Shift again to 3rd column Conflict occurs again
filled
47

In row 4, shift queen again to 4th column. Conflict occurs again.
filled
48

In row 4, shift queen again. No room to go..
filled
49

Pop the stack. Shift queen in row 3 towards right. Again conflict occurs
filled
50

Shift queen in row 3 towards right. Again conflict occurs
ROW 3, COL4 ROW 2, COL 4 ROW 1, COL 1
filled
51

. Shift queen in row 3 towards right. No room to go Pop off the stack
filled
52

In row 2, shift queen again. No room to go.. Pop the stack.
ROW 1, COL 1
filled
53

Pop the stack. In row 1, shift queen towards right by 1.
filled
54

Repeat the process again starting with (ROW 1 ,COL 2 )
ROW 1, COL2
filled
55

Add queen to Row 2 Col 1 Conflict occurs again
ROW 2, COL1 ROW 1, COL 2
filled
56

Shift queen towards right by 1 in row 2. Conflict occurs again
filled
57

Shift queen towards right by 1 in row 2. Conflict occurs again
filled
58

Shift queen towards right by 1 in row 2. No conflict Hence add the queen in that position.
filled
59

Add third queen to row3 column 1. No conflict Hence add the queen in that position. Increment filled by 1.
ROW3, COL 1 ROW 2, COL 4 ROW 1, COL 2
filled
60

Add 4th queen to row4 column 1. Conflict occurs Hence ,shift one position towards right.
ROW4, COL 1 ROW3, COL 1 ROW 2, COL 4 ROW 1, COL 2
filled
61

Shift queen to row4 column 2. Conflict occurs Hence ,shift one position towards right.
filled
62

Shift queen to row4 column 3. No conflict. Hence , place the queen in that position.
filled
63
Solution for 4-queens problem
filled
64
Pseudocode for N-Queens

Initialize a stack where we can keep track of our decisions. Place the first queen, pushing its position onto the stack and setting filled to 0. repeat these steps if there are no conflicts with the queens... else if there is a conflict and there is room to shift the current queen rightward... else if there is a conflict and there is no room to shift the current queen rightward...
65
repeat these steps if there are no conflicts with the queens...
Increase filled by 1. If filled is now N, then the algorithm is done. Otherwise, move to the next row and place a queen in the first column.
66
repeat these steps if there are no conflicts with the queens... else if there is a conflict and there is room to shift the current queen rightward...
Move the current queen rightward, adjusting the record on top of the stack to indicate the new position.
67
repeat these steps if there are no conflicts with the queens... else if there is a conflict and there is room to shift the current queen rightward... else if there is a conflict and there is no room to shift the current queen rightward...
Backtrack! Keep popping the stack, and reducing filled by 1, until you reach a row where the queen can be shifted rightward. Shift this queen right.
68
Summary: N-Queens Problem
Stacks have many applications. The application shown above is called backtracking. The key to backtracking: Each choice is recorded in a stack. When you run out of choices for the current decision, you pop the stack, and continue trying different choices for the previous decision.
69
Eight Queen Problem

Place 8 queens in a chessboard so that no two queens are in the same row, column, or diagonal.
A solution
Not a solution
70
648 states with 8 queens
One strategy: guess at a solution
There are 4,426,165,368 ways to arrange 8 queens on a chessboard of 64 squares
An observation that eliminates many arrangements from consideration
No queen can reside in a row or a column that contains another queen Now: only 40,320 (8!) arrangements of queens to be checked for attacks along diagonals
71
Eight Queen Problem

Some solutions from 92 Solutions
72
The Eight Queens Problem
A recursive algorithm that places a queen in a column
Base case If there are no more columns to consider You are finished Recursive step If you successfully place a queen in the current column Consider the next column If you cannot place a queen in the current column You need to backtrack
73
void placeQueen() // Recursive method which tries to place queen k in row k. { if (k == n) // All queens have been placed. { h++; if (h <= hmax) printSolution(); } else for (int i = 0; i < n; i++) if (coln[i] && main[i - k + n - 1] && anti[i + k]) // Place queen k at position i of row k. { pstn[k] = i; coln[i] = main[i - k + n - 1] = anti[i + k] = false; k++; placeQueen(); k--; coln[i] = main[i - k + n - 1] = anti[i + k] = true; }
74
Analysis
Here coln[], main[] and anti[] are boolean arrays which have been initialized with all positions equal to true. They are used to mark occupied columns, diagonals and antidiagonals, respectively.
The time consumption of the program is O(n * c(n)) where c(n) gives the number of calls to the method placeQueen().
75
Eight Queen Problem

Idea of solution: Each recursive call attempts to place a queen in a specific column.
A loop is used, since there are 8 squares in the column.
For a given call, the state of the board from previous placements is known (i.e. where are the other queens?)
76
Analysis of n-queens problem

Its much harder to determine the complexity of this algorithm. It tests at each point along the paths, not just at the end of the paths. So thats more work..! On the other hand, many paths are abandoned before all n Queens have been placed. Time complexity is exponential.
77
Total solutions for N queens
78
Comments
Backtracking provides the hope to solve some problem instances of nontrivial sizes by pruning non-promising branches of the state-space tree. The success of backtracking varies from problem to problem and from instance to instance. Backtracking possibly generates all possible candidates in an exponentially growing state-space tree. 79
Sum of subsets
Problem: Given n positive integers w1, ... wn and a positive integer S. Find all subsets of w1, ...
wn that sum to S.
Example: Given n=3, S=6, and w1=2, w2=4, w3=6 Solutions: {2,4} and {6}
80
Sum of subsets
We will assume a binary state space tree. The nodes at depth 1 are for including item 1, the nodes at depth 2 are for item 2, etc. The left branch includes wi, and the right branch excludes wi. The nodes contain the sum of the weights included so far.
81
Sum of subset Problem: State SpaceTree for 3 items
w1 = 2, w2 = 4, w3 = 6 and S = 6
with 2
without 2
i1
with 4
2
without 4
with 4
0
without 4
i2
with 6
6
without 6 with 6
2
without 6
with 6
4
without 6
with 6
0
without 6
i3
12
10
Solution: {2,4} and {6} Variable sized tuple

82
Sum of subset Problem: State SpaceTree for 3 items
w1 = 2, w2 = 4, w3 = 6 and S = 6
1
i1
1
2
0
1
0
0
i2
1
6
0
1
2
0
1
4
0
1
0
0
i3
12
10
Solution: {1,1,0} and {0,0,1} Fixed sized tuple

83
A Depth First Search solution
Problems can be solved using depth first search of the (implicit) state space tree. Each node will save its depth and its (possibly partial) current solution DFS can check whether node v is a leaf.
If it is a leaf then check if the current solution satisfies the constraints Code can be added to find the optimal solution
84
A DFS solution
Such a DFS algorithm will be very slow. It does not check for every solution state (node) whether a solution has been reached, or whether a partial solution can lead to a feasible solution Is there a more efficient solution?
85
Backtracking
Definition: We call a node nonpromising if it cannot lead to a feasible (or optimal) solution, otherwise it is promising. Main idea: Backtracking consists of doing a DFS of the state space tree, checking whether each node is promising and if the node is nonpromising, backtrack to the nodes parent.
86
Backtracking
The state space tree consisting of expanded nodes only is called the pruned state space tree. The following slide shows the pruned state space tree for the sum of subsets example.
87
A Pruned State Space Tree (find all solutions) w1 = 3, w2 = 4, w3 = 5, w4 = 6; S = 13

0 3 3 4 7 5 12 6
w/o 5
w/o 3 0
w/o 4 3 5 8
w/o 6 w/o 5
4 4 5 9
w/o 5
w/o 4
13 Solution
Solution: {3,4,6}
There are only 15 nodes in the pruned state 88 space tree. The full state space tree has 31 nodes.
Backtracking
Live node: A node which has been generated and all whose children have not yet been generated. E-node: The live node whose children are currently being generated. Dead node: A generated node which is not to be expanded further or all of whose children have been generated. Bounding functions are used to kill live nodes without generating all their children. Depth first node generation with bounding functions is called backtracking.
89
Backtracking algorithm
void checknode (node v) { node u if (promising ( v )) if (aSolutionAt( v )) write the solution else //expand the node for ( each child u of v ) checknode ( u ) }
90
Checknode
Checknode uses the functions:
promising(v) which checks that the partial solution represented by v can lead to the required solution. aSolutionAt(v) which checks whether the partial solution represented by node v solves the problem.
91
When is a node non promising?

Consider a node at depth i. weightSoFar = weight of node, i.e., sum of numbers included in partial solution node.
totalPossibleLeft = weight of the remaining items i+1 to n (for a node at depth i) A node at depth i is non-promising if (weightSoFar + totalPossibleLeft < S ) or (weightSoFar + w[i+1] > S ) If (weightSoFar == S) then it is a promising node at depth i.
92
A Pruned State Space Tree w1 = 3, w2 = 4, w3 = 5, w4 = 6; S = 13

1
0 3 0
2
4
3 0
11
4 3 0
0 0
3
5
7 0
8
5
12 13
9 5
4 0
15
4
12
5
6
9
0 7
10
14
6 13 solution
- backtrack
Nodes are numbered in call order.
93
Algorithm SumOfSubsets ( i, weightSoFar, totalPossibleLeft )

if (promising ( i )) then //may lead to solution if ( weightSoFar == S ) then print include[ 1 ] to include[ i ] //found solution else //expand the node when weightSoFar < S if ( weightSoFar S ) include [ i + 1 ] = "yes //try including SumOfSubsets ( i + 1, weightSoFar + w[i + 1], totalPossibleLeft - w[i +
1] ); else if ( weightSoFar S )
include [ i + 1 ] = "no
//try excluding SumOfSubsets ( i + 1, weightSoFar , totalPossibleLeft - w[i + 1] );
94
boolean promising (i ) {
return ( weightSoFar + totalPossibleLeft S) && ( weightSoFar == S || weightSoFar + w[i + 1] S )
Prints all solutions! }
95
General Iterative Backtracking method

Algorithm IterBacktrack(n)
// This schema describes the backtracking process // using iterative process. All solutions are generated in x[1..n] // and printed as soon as they are determined.
{ K:=1; while (k 0) do
if ( there remains an untried x[k] T(x[1], , x[k-1]) and Bk (x[1], x[2], , x[k]) is true ) then
{
if (x[1], x[2], , x[k] is a path to an answer node) then write (x[1:k]); k := k + 1; // Consider the next set
}
else k := k 1;
}
}
96
Recursive Backtracking algorithm

Algorithm Backtrack(k)
// This schema describes the backtracking process // using recursion. On entering, the first k-1 values // x[1], x[2], ., x[k-1] of the solution vector // x[1:n] have been assigned. x[] and n are global.
for (each x[k] T(x[1], , x[k-1]) do { if (Bk (x[1], x[2], , x[k]) 0) then { if (x[1], x[2], , x[k] is a path to an answer node) then write (x[1:k]); if (k < n) then Backtrack(k+1); } }
97
Analysis of Backtracking algorithms:
Generate all possible solutions DFS technique is used to generate the solutions. Backtracking algorithm time complexity is exponential.
98
What are the requirements that are needed for performing Backtracking?
To solve any problem using backtracking, it requires that all the solutions satisfy a complex set of constraints. They are: Explicit constraints Implicit constraints
99
What are the factors that influence the efficiency of the backtracking algorithm?
The efficiency of the backtracking algorithm depends on the following four factors. The time needed to generate the next xk The number of xk satisfying the explicit constraints. The time for the bounding functions Bk The number of xk satisfying the Bk.
100
Analysis
Once
the state space tree organization is selected, The first three factors are relatively independent of the problem instance being solved. Only the fourth factor varies from one problem instance to another.
101
Analysis
A backtracking algorithm on one problem instance might generate only O(n) nodes whereas on a different instance it might generate almost all the nodes in the state space tree.
102
Analysis
If the number of nodes in the solution space is 2n or n!
Worst case time will be O( p(n) 2n ) or O( q(n) n! ) respectively.
Note: p(n) and q(n) are polynomials in n.
103
Importance of backtracking
Ability to solve some instances with large n in a very small amount of time.
Difficult task.! Predicting the behavior of a backtracking algorithm for the problem instance we wish to solve.
104
Monte Carlo Method
Estimate the number of nodes that will be generated by a backtracking algorithm working on a certain instance.
105
Estimating the efficiency of Backtracking

Algorithm EstimateBacktrack()
// This algorithm follows a random path in a state space tree // and produces an estimate of the number of nodes in the tree.
{ K:=1; m := 1; r := 1;
repeat
Tk := { x[k] | x[k] T(x[1], , x[k-1]) and Bk (x[1], x[2], , x[k]) is true ) }; if ( Size( Tk ) = 0) then return m; r := r * Size( Tk ) ; m := m + r; x[k] := Choose ( Tk ); k := k + 1;
} until (false)
}
106
Run time of randomized algorithm

107
Run time of recursive algorithm
As can be seen from both the analysis and performance measurements the randomized algorithm is far more efficient than the algorithm that uses recursive backtracking.
108
Summar y : Concept
Backtracking
Recursion can be used for elegant and easy implementation of backtracking. Backtracking can easily be used to iterate through all subsets or permutations of a set. Backtracking ensures correctness by enumerating all possibilities. For backtracking to be efficient, we must prune the search space.
109

Backtracking: Presented By: Smt. Smitha M.L. Research Scholar

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Backtracking: Presented By: Smt. Smitha M.L. Research Scholar

Uploaded by

Copyright:

Available Formats

Backtracking

Presented by: Smt. Smitha M.L. Research Scholar

Go straight Go left Go right

Generate and Test Method

The backtracking method

The backtracking algorithm

State Space Tree of the Four-queens Problem

The N-Queens Problem

Suppose you have 8 chess queens... ...and a chess board

The N-Queens Problem

The N-Queens Problem

The N-Queens Problem

The N-Queens Problem

The N-Queens Problem

The N-Queens Problem

How the program works ?

How the program works ?

How the program works ?

How the program works ?

ROW 2, COL 1 ROW 1, COL 1

How the program works ?

ROW 2, COL 2 ROW 1, COL 1

How the program works ?

ROW 2, COL 3 ROW 1, COL 1

How the program works ?

ROW 2, COL 3 ROW 1, COL 1

How the program works ?

How the program works ?

ROW 3, COL 2 ROW 2, COL 3 ROW 1, COL 1

How the program works ?

How the program works ?

How the program works ?

ROW 3, COL 4 ROW 2, COL 3 ROW 1, COL 1

How the program works ?

ROW 2, COL 3 ROW 1, COL 1

How the program works ?

ROW 2, COL 4 ROW 1, COL 1

How the program works ?

ROW 2, COL 4 ROW 1, COL 1

How the program works ?

How the program works ?

How the program works ?

ROW 3, COL 2 ROW 2, COL 4 ROW 1, COL 1

How the program works ?

How the program works ?

How the program works ?

How the program works ?

How the program works ?

ROW 3, COL 3 ROW 2, COL 4 ROW 1, COL 1

How the program works ?

How the program works ?

How the program works ?

How the program works ?

How the program works ?

How the program works ?

ROW 2, COL1 ROW 1, COL 2

How the program works ?

How the program works ?

How the program works ?

How the program works ?

How the program works ?

How the program works ?