Heuristic and Metaheuristic Algorithms

Sam Allen sda@cs.nott.ac.uk ASAP Research Group

Heuristic algorithms What are they?

Wikipedia says: In computer science, a heuristic is a technique designed to solve a problem that ignores whether the solution can be proven to be correct, but which usually produces a good solution or solves a simpler problem that contains or intersects with the solution of the more complex problem.

Heuristics are intended to gain computational performance or conceptual simplicity, potentially at the cost of accuracy or precision.

Huh?

Basically heuristic algorithms solve really difficult problems reasonably well (which is pretty subjective) in a reasonable amount of time (also pretty subjective). Examples of problems that are tackled with heuristic algorithms (more on these in Matts lecture on Friday):

Cutting/Packing Graph problems, i.e. Travelling Salesman Problem Scheduling/Timetabling

2D packing an example

2D packing an example
1 2 3 4 5

Sequence = 1,2,3,4,5

2D packing an example
1 4 3 2 5

Sequence = 1, 4, 3, 2, 5

That was easy!

For 5 pieces there are 5 x 4 x 3 x 2 x 1 different orderings for the placement = 120 combinations Piece of cake for a computer to try each combination in (nearly) no time at all

How do we pack all of these?

(50 pieces)

Thats not so easy

50 pieces means 50 x 49 x..2 x 1 different orderings = 3041409320171337804361260816606476884 4377641568960512000000000000 Thats quite a lot really. If a computer could evaluate 1000 orderings per second itd still take approximately: 9644245688011598821541288738605012951 66718720476931506 years! (and thats without being allowed to rotate the pieces)

Heuristic algorithms
Cast your mind all the way back to the second slide. Maybe a heuristic algorithm would be good for tackling this type of problem? (commonly known as a combinatorial optimi[s,z]ation problem) I hope so! (my PhD depends on it)

My current research
The 3-D strip packing problem Given a container (a big box, a lorry, an aeroplane cargo hold etc) and a number of boxes to load inside the container what is the best way to place them all so that they take up the shortest (height) space as possible?

Applications of strip packing

This problem can be seen to have applications directly in industry:

Loading pallets Cutting blocks of polystyrene or wood from a larger block minimising wastage

And more theoretical applications:

multi-dimensional resource scheduling

How it works (Preprocessing stage)

First the algorithm takes a list of boxes, and the width and length of the container The boxes are then rotated so that each of their width length height They are then sorted decreasingly by width

How it works (Packing stage)

1 2

Gap

How it works (Packing stage)

2

1 Gap

How it works (Packing stage)

2

How it works (Packing stage)

2

How it works (Packing stage)

2

Gap 1 X

How it works (Packing stage)

3 4

2 1

Gap
X

How it works (Packing stage)

4

2 1

3
X

Gap

How it works (Packing stage)

4

Gap 2 1

X X

How it works (Packing stage)

4 2 1

3 3

X X X

How it works (postprocessing)

Due to the nature of the algorithm, towers may form. Towers are boxes with large height dimensions and lower width and length dimensions that are placed late on in the packing process, jutting out over the top of the profile. The tallest tower is removed from the packing, rotated so that it is effectively knocked down and placed back into the packing at the lowest point available. This is repeated until the solution is not improved any further.

Gap 9 12 10 6 11 4 3 7 5

8 32

1 9 12 10 6 11 4 3 7 5

8 32

9 12 10 6 11 4 3 7

8 32
5

1 9 12 10 6 11 4 3 7 5

8 32

old height
Improvement new height 1 9 12 10 6 11 4 3 7 5

8 32

Have you been paying attention?

1 2 3 4

Q) How many different orders can these boxes be placed in?

A) 4 x 3 x 2 x 1 = 24 different orders

Q) Which of the following problems could be solved with heuristic techniques?

Cracking a combination lock

Planning a route from Nottingham to London

Metaheuristics what are they?

Wikipedia says A metaheuristic is a heuristic method for solving a very general class of computational problems by combining user given black-box procedures usually heuristics themselves in a hopefully efficient way.

Examples of metaheuristics
Hill climbing / Greedy search Tabu search Simulated Annealing Genetic algorithms Ant colony optimisation many more

Explanation part 1

A metaheuristic is a higher level algorithm

This means it doesnt actually know what the problem is its solving So it can be applied to many different types of problems

Explanation part 2

So all a programmer needs to do to adapt a metaheuristic to their new problem is provide:

A function to generate new valid solutions based on the current one g() An evaluation function f()

Explanation part 3

Advantage of metaheuristics:

Generally produce higher quality results (given enough time) than simple heuristics
They take a lot longer as they have to generate and evaluate many solutions rather than just one

Disadvantage:

Greedy search for 2d packing

Lets define our function g then.

g will take a solution, in this case the order of boxes to pack It will return a new solution by randomly swapping the order of 2 of the boxes E.g. 1,2,3,4 -> 3,2,1,4 or 4,2,3,1 etc So g([1,2,3,4]) = [3,2,1,4] perhaps

Greedy search for 2d packing

So now we have g (the harder function to define in this case) lets define f
f() simply returns the highest location in the packing when placed in a bottom right way E.g. f([1,2,3,4]) = 6

4 2 1 3 6 units high

Greedy search for 2d packing

Were going to run 3 iterations of a greedy search to find the best packing we can 1) Start with our initial solution [1,2,3,4,5] 2) f([1,2,3,4,5]) = 14, our best solution so far 3) Iteration 1: g([1,2,3,4,5]) = [1,2,5,4,3] 4) f([1,2,5,4,3]) = 12, better than our best so far so well take this new order as our current one

Greedy search for 2d packing

5)

6)
7) 8)

9)

Iteration 2: g([1,2,5,4,3]) = [2,1,5,4,3] f([2,1,5,4,3]) = 13, worse than our current best so ignore this order Iteration 3: g([1,2,5,4,3]) = [1,4,5,2,3] f([1,4,5,2,3]) = 10, our best so far so we keep this Return [1,4,5,2,3] as our best solution

Demonstration/Questions?

http://www.cs.nott.ac.uk/~sda/viewer.zip