Contents

Problem Introduction.................................................................................................................................... 2 Assumption ................................................................................................................................................... 2 Brute Force Algorithm................................................................................................................................... 2 Pseudocode ............................................................................................................................................... 2 Brute Force Time Complexity.................................................................................................................... 3 Problem 1 (Brute force) ............................................................................................................................ 3 Problem 2 (Brute force) ............................................................................................................................ 5 Prims Algorithm ........................................................................................................................................... 7 Pseudocode ............................................................................................................................................... 7 Prims Time Complexity ............................................................................................................................ 7 Problem 1 (Prims) .................................................................................................................................... 8 Problem 2 (Prims) .................................................................................................................................. 12 Kruskals Algorithm ..................................................................................................................................... 14 Pseudocode ............................................................................................................................................. 14 Kruskals Time Complexity ...................................................................................................................... 14 Problem 1 (Kruskals) .............................................................................................................................. 15 Problem 2 (Kruskals) .............................................................................................................................. 18 Dijkstra Algorithm ....................................................................................................................................... 20 Pseudocode ............................................................................................................................................. 20 Time complexity Dijkstra......................................................................................................................... 21 Problem 1 (Dijkstra) ................................................................................................................................ 21 Problem 2 (Dijkstra) ................................................................................................................................ 22 Comparison of different algorithms for these problems ............................................................................ 23 Complexity Class ......................................................................................................................................... 25 Complexity classes .................................................................................................................................. 25 Justification ............................................................................................................................................. 27 Reference .................................................................................................................................................... 28

Problem Introduction
There are 8 islands which should be connected to each other with the minimum cost by 7 bridges. The cost of construction is related to the length of the bridges to be built. The problem can be seen as a graph optimization problem and there are different solutions available for it. Few algorithms will be applied to these problems and the efficiency of them will be compared later.

Assumption
Since demonstrating the brute force solution is very difficult since the answer will be very long and performing the algorithm completely will be very time consuming, Here only 2 examples of routes that brute force will find is demonstrated.

Brute Force Algorithm

Brute force is a very simple but may not be very efficient solution for this type of problems. It is not very efficient because in large number of nodes, brute force may take years to complete the answer. But it can be used for small graphs. Brute force lists all of the possible routes, evaluates their weight and then compares them to find the smallest.

Pseudocode
Generate first candidate While candidate null C1 C2 1 n! C3 C4 n! 1

If candidate is the solution Output the candidate

Generate next candidate

C5

n!

Brute Force Time Complexity

Running time for brute force is calculated as shows in below: T(n) = C1 + n!(C2+C3) + C4 + n!(C5) T(n) = C1 + C4 + n!(C2+C3+C5) For example for the existing problem, the running time will be 8! = 40320.

Problem 1 (Brute force)

Since the number of possible answers is very large, here 2 possible routes are drawn. Route No 1:

Route No 2:

As it is shown, these are two random answers for the problem. When one connection is changed, the total number of weight differs. steps 1-2 2-3 3-4 4-5 5-6 6-7 7-8 1-2 2-3 3-4 4-5 5-6 6-7 7-8 Total Weight 1765

Route 1

Route 2

1580

Looking at the tables shows that in these 2 routes with only moving one connection, there is a drastic change in total weight. So in order to find the best route all of the possible routes must be compared.

Problem 2 (Brute force)

For the second problem, 2 bridges are already built. So using brute force, we create random graphs which include bridge between island 5&7 and 3&6. First route:

And the second route:

steps Route 1 1-2 2-3 3-4 4-5 5-7 3-6 8-1 1-2 2-3 3-4 4-5 5-7 7-8 3-6

Total Weight 1795

Route 2

1980

Prims Algorithm
Prims is one of the greedy algorithms to find the minimum spanning tree in graphs. Prims algorithm finds a subset of the graph which includes all of the nodes and ensures the total weigth of edges is kept minimal.

Pseudocode
(V = No of vertex) Tree = NULL C1 1 C2 1

E = sequence of all edges by weight For i = 1 to V For j = 1 to E C3 C4 V+1 E+1

If ej NO cycle with edges & incident to a vertex Add ej to tree C6 Break V-1

C5

E-1

Prims Time Complexity

T(V) = c1 + C2 + (V+1)C3 + V(E+1)C4 + V(E)(E-1)C5 T(v) = c1 + c2 + vC3 + C3 + VEC4 + VC4 + VE^2C5 VEC5 The complexity for the algorithm is: O(|V||E|) Minimum edge weight data structure Adjacency matrix, searching Binary heap and adjacency list Fibonacci heap and adjacency list Time complexity O(V*V) O((V+E)log(v)) = O(E+log(V)) O(E+Vlog(V))

Problem 1 (Prims)
Island 1 Island 2 Island 3 Island 4 Island 5 Island 6 Island 7 Island 8 Island 1 Island 2 Island 3 Island 4 Island 5 Island 6 Island 7 Island 8 240 210 265 340 175 260 280 215 115 160 200 180 350 330 360 345 185 435 295 400 175 120 155 195 230 170 205 305 -

To find the minimum spanning tree using prims algorithm for this question, first we have to choose a node to start. It is better to choose a node which is related to the smallest weight in edges but since all the nodes will be incorporated in answer, really there is no different about which node to start with. Thus, we will start by the island 1 as known as Node 1. Step 1 We need to find the smallest edge connected to node 1. From the first row we can see that the edge between node 1 and 8 is the smallest. So: Nodes 1-8 Nodes 1-8 Weight 120 Weight 120
1 120 8

Step 2 From node 8 or 1, we have to find an edge with smallest weight:

120

155

Nodes 1-8 8-2

Weight 120 155

Step 3 Nodes 1-8 8-2 8-5 Weight 120 155 170

Step 4 Nodes 1-8 8-2 8-5 5-3 Weight 120 155 170 115

Step 5 Nodes 1-8 8-2 8-5 5-3 5-4 Weight 120 155 170 115 160

Step 6 Nodes 1-8 8-2 8-5 5-3 5-4 2-6 Weight 120 155 170 115 160 180

10

Step 7 Nodes 1-8 8-2 8-5 5-3 5-4 2-6 6-7 Weight 120 155 170 115 160 180 175

Now, because all of the 8 islands are connected, there is no need for more edges and the solution is complete. The total weight is: 1075

11

Problem 2 (Prims)
Step 1 Nodes 5-7 3-6 5-3 Weight 400 350 115

Step 2 Nodes 5-7 3-6 5-3 5-4 Weight 400 350 115 160

Step 3 Nodes 5-7 3-6 5-3 5-4 5-8 Weight 400 350 115 160 170

Step 4

12

Nodes 5-7 3-6 5-3 5-4 5-8 8-1

Weight 400 350 115 160 170 120

Step 5 Nodes 5-7 3-6 5-3 5-4 5-8 8-1 8-2 Weight 400 350 115 160 170 120 155

Now all of the 8 islands are connected to each other by 7 bridges and no more bridge is needed. Total weight is: 1470

13

Kruskals Algorithm
Kruskals algorithm is another greedy algorithm based on generic minimum spanning tree algorithm. It finds a safe edge to add the forest. At each step it adds the possible edge with least weight.

Pseudocode
Define an empty tree T For each vertex v Define a cluster C(v) C1 C2 C3 1 V+1 V C4 E

Initialize a priority queue Q using weights as keys to conclude all edges While edges in T<V-1 (V=No of vertices) (u,v) <-Q.removeMin() If C(v) C(u) Add edge (u,v) to T C6 C7 C8 C5 E logE E-1 V-1 C9 E

Merge C(v) and C(u) into one cluster //union Return T

V-1

Kruskals Time Complexity

T(v) = (C1+C10) + v(c3) + (v+1)(C2) + E(C4+c5) + (E log E)(C6) + (E-1)(C7) + (v-1)(C8+C9) T(v) = (C1+C2+C10-C7-C8-C9) + v(C2+C3+C8+C9) + E(C4+C5+C7) + (E log E)(C6)

The complexity for the algorithm is O(|E|log|V|)

14

Problem 1 (Kruskals)
Island 1 Island 2 Island 3 Island 4 Island 5 Island 6 Island 7 Island 8 Island 1 Island 2 Island 3 Island 4 Island 5 Island 6 Island 7 Island 8 240 210 265 340 175 260 280 215 115 160 200 180 350 330 360 345 185 435 295 400 175 120 155 195 230 170 205 305 -

Step 1 Nodes 3-5 Weight 115

The edge between the nodes 3 and 5 hast the smallest weight. Step 2 The next smallest weight in edges is: Nodes 3-5 8-1 Weight 115 120

15

Step 3 Nodes 3-5 8-1 8-2 Weight 115 120 155

Step 4 Nodes 3-5 8-1 8-2 5-4 Weight 115 120 155 160

Step 5 Nodes 3-5 8-1 8-2 5-4 8-5 Weight 115 120 155 160 170

16

Step 6 Nodes 3-5 8-1 8-2 5-4 8-5 6-7 Weight 115 120 155 160 170 175

Step 7 Nodes 3-5 8-1 8-2 5-4 8-5 6-7 6-2 Weight 115 120 155 160 170 175 180

We can see that all of the islands are connected to each other and no more edge is required, so the solution is done. The total weight is: 1075 The answer is similar to the answer when using prims algorithm which is obvious because we are trying to find 1 thing using these 2 algorithms and the algorithms seek similar things and the only difference between them is the steps.

17

Problem 2 (Kruskals)
Step 1 Nodes Weight 5-7 3-6 3-5 400 350 115

Step 2 Nodes Weight 5-7 3-6 3-5 8-1 400 350 115 120

Step 3 Nodes Weight 5-7 3-6 3-5 8-1 8-2 400 350 115 120 155
18

Step 4 Nodes Weight 5-7 3-6 3-5 8-1 8-2 5-4 400 350 115 120 155 160

Step 5 Nodes Weight 5-7 3-6 3-5 8-1 8-2 5-4 5-8 400 350 115 120 155 160 170

Total weight is 1470.

19

Dijkstra Algorithm
It solves the single source shortest path problem for graphs with weight and direction.

Pseudocode
Assume Node A C1 1 m C3 m

For all nodes N adjacent to node A (m) C2

Store the edge between node N and node A Link N to node A For all nodes V C5 n n C4 m

currDist(v) = C6 currDist(A) = 0 C7 1

ToBeChecked = all nodes

C8

n C9 n+1

While ToBeChecked is not empty {

V = a node in ToBeChecked with minimal currDist(V) C10

v
v 1

Remove V from ToBeChecked C11

For nodes U adjacent to V & in ToBeChecked{ C12

t
v 1

If currDist(U)>currDist(V) + weight(VU){

C13

(t
v 1

1)

currDist(U)=currDist(V) + weight(UV) C14 predecessor(U) = V }end if }end for }end while 20 C15

Time complexity Dijkstra

T(n) = (C1 + C7) + m(C2 + C3 + C4) + n (C5 + C6 + C7 + C8 + C9 + C11 C13) +

v (C10)
v 1

t
v 1

(C12+C13) + x(C14+C15)

We know that V = a node in ToBeChecked with minimum currDist(V) will be executed

v
v 1

times. It will run n times at the beginning then repeats n-1 times and n-2 and..1 which is sum from 1 to n meaning n^2. So its time complexity will be O(n^2).When there is one path between each pair of nodes, tv will be equal to C so t v =
v 1 n

v
v 1

and also time complexity = O(n^2). So

it can be said that when all nodes are connected to each other time complexity of dijkstra algorithm will be O(n^2).

Problem 1 (Dijkstra)
Dijsktra algorithm helps to find the shortest path in a graph. For this problem, I want to connect island 1 to all of the other islands. 2 240 155+120 240 180+200 240 265+210 240 3 210 145+120 210 350+200 210 4 340 230+120 340 330+200 340 260+210 340 175+240 340 160+280 340 5 280 230+120 280 360+200 280 160+210 280 215+240 280 6 200 205+120 200 7 345 305+120 345 175+200 345 435+210 345 185+240 345 400+280 345 245+340 345 8 120 -

1 8 6 3 2 5 4

21

The table shows that the shortest path between island number 1 and all of the other islands is the direct path between 1 and every other island.

120

240

345 1 210

200 280

340

4 6

Total weight for problem 1 using dijkstra algorithm is: 1735

Problem 2 (Dijkstra)
Here we have to consider that 2 bridges are already built. Island 5 is selected for the first node.
1 280 210+115 280 340+160 280 120+170 280 240+215 280 2 215 265+115 215 175+160 215 155+170 215 3 115 4 160 260+115 160 6 360 350 7 400 8 170 195+115 170 230+160 170 -

5 3 4 8 2

22

8 280 170 215

5 400 7 160 115 3 350 6

Total weight for problem 2 using dijkstra algorithm is:

Comparison of different algorithms for these problems

As said before, the main problem was to connect the 8 islands with 7 bridges and keep the cost as low as possible. There were 4 different algorithms applied to solve this problem. But the efficiency of the algorithms may differ according to the suitability of the algorithm for that particular problem.

23

Here we can see the details of each algorithm: Brute Force Aim Any Problem Prims Minimum Spanning Tree Running Time Complexity Solution Total Weight P1 P2 Yes 1765/1580 1795/1980 Yes 1075 1470 Yes 1075 1470 Yes 1735 1690 O(n!) O(|E||V|) Kruskals Minimum Spanning Tree O(ElogE) O(n^2) Dijkstra Shortest Path

We know that between every node of this graph there is an edge. To find the total number of edge we can use this formula: n(n-1)/2 so there are totally 28 edges in this graph and 8 nodes. Running time for each algorithm will be: Brute Force Running Time Complexity N= 8 = 40320 |E| = 28 |V| = 8 = 224 E = 28 Log28 = 1.4471580 = 40.520424 64 O(n!) Prims O(|E||V|) Kruskals O(ElogE) Dijkstra O(n^2)

This table shows for this number of nodes and edges, dijkstra executes commands 64 times and it is the most efficient algorithm.

24

Complexity Class

Each complexity class consists of problems of related resource based complexity. It usually uses running time as complexity measure. Problems of same class can be solved by an abstract machine M using O(f(n)) of resource R, where n is the number of input. Notes: Decision Problems are the kind of problems that the answer to them will be Yes or No or 0 or 1. The algorithm is applies on the input to find the answer. Turing machine is a device that uses logics to find the answer. It executes a series of discrete transitions as determined by its transition table and by the initial characters on the tape.

Complexity classes
P complexity class is class of decision problems that can be solved with a deterministic Turing machine in polynomial time. Class P contains problems that can be solved efficiently such as finding shortest paths in networks. They can be solved by deterministic programs and solving their worst case takes reasonable time. Class P consists of problems that can be solved in time O(nk) (n is the size of input and k is constant). But p also includes problems that the best algorithms to solve them have time complexity n10^500 although they are not computationally feasible. NP complexity class is the class for decision problems that can be solved by non-deterministic Turing machine in polynomial time. This class is also defined by means other than nondeterministic Turing machines. NP is the class of problems that the best solutions for them can be verified fast using deterministic machines in polynomial time. It should be mentioned that any problem in class P can be classified as NP also because when a problem is P, it is possible to be solved without even needing verification.

25

Example: A decision problem where instances of the problem for which the answer is yes have proofs that can be verified in polynomial time. This means that if someone gives us an instance of the problem and a certificate (sometimes called a witness) to the answer being yes, we can check that it is correct in polynomial time. NP hard complexity class includes the NP problems that can be efficiently reduced to it. It means if there is a polynomial time algorithm that translates any instance of NP problem into an instance of the problem in question. When a decision version of a combinational optimization problem is proved to belong to NP-complete problems, the optimization problem is NP hard. Example: this problem: is there a Hamiltonian cycle with length less than K belong to NP complete class. But this question what is the shortest tour? is NP hard since there is no easy way to determine if a certificate is the shortest. NP complete complexity class is the class for which answers can be checked for correctness given a certificate by an algorithm that its run time is polynomial in size of the input. The answers of a NP complete problem can be verified quickly and a quick algorithm to solve this problem can be used to all other NP problems quickly. Example: This is the problem wherein we are given a conjunction of 3-clause disjunctions (i.e., statements of the form
(x_v11 or x_v21 or x_v31) and (x_v12 or x_v22 or x_v32) and ... and

(x_v1n or x_v2n or x_v3n)

where each x_vij is a boolean variable or the negation of a variable from a finite predefined list(x_1, x_2, ... x_n). It can be shown that every NP problem can be reduced to 3-SAT. The proof of this is technical and requires use of the technical definition of NP (based on nondeterministic Turing machines and the like). This is known as Cook's theorem.

26

What makes NP-complete problems important is that if a deterministic polynomial time algorithm can be found to solve one of them, every NP problem is solvable in polynomial time (one problem to rule them all).

Justification
In the definition of P class it was told that P contains familiar problems that can be solved efficiently. Finding the shortest path in networks and graphs is also one of the P problems. The problem in this assignment also can be solved with finding the shortest path and there is at least one algorithm that can solve the problem in polynomial time. This problem in 8 nodes (limited size of input) can be solved in polynomial time but in larger sizes of input cannot be solved in polynomial time. Also, since it is possible to say that because the problem is P, it also can be NP because there is no evidence that P problems do not belong to NP class. Greedy algorithms such as Prims and Kruskals are P but Brute force is not greedy and in larger instances it can be NP.

27

Reference
Cormen Thomas H., Introduction to Algorithms, 2nd edn, McGraw-Hill, US Eric Allender , Complexity Classes, (Online) Reached from: http://ftp.cs.rutgers.edu/pub/allender/ALRch33.pdf [Visited on 14th 5 2012] Unknown, NP vs NP-complete vs NP HARD, (Online) Reached from: http://stackoverflow.com/questions/1857244/np-vs-np-complete-vs-np-hard-what-doesit-all-mean [Visited on 19th 5 2012] Scott Aaronson, Computational Complexity and the Anthropic Principle, 2006, (Online) Reached from: http://www.scottaaronson.com/talks/anthropic.html [Visited on 20th 5 2012]

28