Heaps (Min-heap)

Heaps Complete binary tree that stores a collection of keys

(or key-element pairs) at its internal nodes and that satisfies
• Heaps the additional property:
• Properties REMEMBER: key(parent) ≤ key(child)
• Deletion, Insertion, Construction complete binary tree 4
all levels are full, except the
• Implementation of the Heap last one, which is left-filled 5 6

• Implementation of Priority Queue

15 9 7 20
using a Heap
• An application: HeapSort 16 25 14 12 11 8

1 2

Max-heap
We store the keys in the internal nodes only
key(parent) ≥ key(child)
5

40
15 9

35 26
16

15 19 17 20

1 13 14 12 11 8

After adding the leaves the resulting tree is full

3 4

1
 Height of a Heap Notice that ….

• We could use a heap to implement a priority queue

• Theorem: A heap storing n keys has height O (log n) • We store a (key, element) item at each internal
Proof: node
– Let h be the height of a heap storing n keys
removeMin():
– Since there are 2i keys at depth i = 0, … , h - 2 and at least one  Remove the root
key at depth h - 1, we have n ≥ 1 + 2 + 4 + … + 2h-2 + 1
– Thus, n ≥ 2h-1 , i.e., h ≤ log n + 1  Re-arrange the heap!
depth keys
0 1 (2, Sue)

1 2 (5, Pat) (6, Mark)

h-2 2h-2
(9, Jeff) (7, Anna)
h-1 at least 1
h
5 6

Downheap
Removal From a Heap

• The removal of the

top key leaves a hole

• We need to fix the

heap

• First, replace the hole

with the last key in
the heap

• Then, begin Downheap

7
• Downheap compares the parent with the smallest 8
… child. If the child is smaller, it switches the two.

2
 Downheap Continues Downheap Continues

9 10

End of Downheap Heap Insertion

The key to insert is 6

• Downheap terminates when the key is greater than the

keys of both its children or the bottom of the heap is
reached. 11 12

3
 Heap Insertion
Upheap
• Swap parent-child keys out of order

13 14

Upheap Continues End of Upheap

• Upheap terminates when new key is greater than the key

15 of its parent or the top of the heap is reached 16

• (total #swaps) ≤ (h - 1), which is O(log n)

4
 Heap Construction Bottom-up Heap
Construction
We could insert the Items one at the time with
a sequence of Heap Insertions:
n • We can construct a heap
Σ log k = O(n log n) storing n given keys using
k=1
a bottom-up construction

But we can do better ….

17 18

Construction of a Heap Example 1 (Max-Heap)

2

--- keys already in the tree ---

Idea: Recursively re-arrange each sub- 4 5

tree in the heap starting with the leaves 7 3 10 8 6

1 9 6 3

I am now drawing the

6th 5th 3↔6 leaves anymore here
begin here
4th 3rd 2nd 1st
begin here

2 2 2

 
4 5 4 5 4 10

 
7 6 10 8 9  6 10 8 9  6 5 8

HEAP 1 9 3 1 7 3 1 7 3
HEAP
7↔9 5 ↔ 10
HEAP

19 20

5
 Example 1
2 Example 1
 2
4 10
! !

9 10
9  6 5 8
10
7 6 5 8
1 7 3 2
Finally: 2 ↔ 10
9 1 4 3
9 8
5
4↔9
4
7
This is not a heap ! 4↔7 10
1 7
1 4
2↔8
2

5 8

10
2
! ! 9 8
9 10

7 6 5 2
7 6 5 8
21 1 4 3
22
1 4 3

--- keys given one at a time --- Example 2

Example 2 (min-heap) 20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]
[20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]

25 5 11 27

16 15 4 12 6 7 23 20 16 15 4 12 6 9 23 20

25 5 11 27 15 4 6 23

16 15 4 12 6 7 23 20 16 25 5 12 11 9 27 20

23 24

6
 Example 2 Example 2
20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]
20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]

10

7 8 4 6

15 4 6 23 15 5 8 23

16 25 5 12 11 9 27 20 16 25 7 12 11 9 27 20

4 6 5 6

15 5 8 23 15 7 8 23

16 25 7 12 11 9 27 20 16 25 10 12 11 9 27 20

25 26

Analysis of Heap Construction

Analysis of Heap Construction
(let us not consider the dummy leaves)
level 0
Number of swaps Number of swaps 1
h=4

3 swaps 2
level 0

at most
2 swaps level 1 h
At level i the number of swaps is
4 5

1 swap 7 3 10 8
level 2
≤ h–i for each node
0 swaps level 3
1 9 6

At level i there are ≤ 2i nodes

h
h is the max level
Total: ≤ Σ(h – i)·2i
level i -------- h-i swaps i=0

27 28

7
 h

Let j = h-i, then i = h-j and 2h Σj/2 j

≤ 2h+1
j=1
h h h

Σ(h – i)·2i = Σj 2h-j = 2h Σ j 2-j

j=0 Where h is O(log n)
i=0 j=0

Consider Σ j 2-j :
Σ j 2-j = 1/2 + 2 1/4 + 3 1/8 + 4 1/16 + … So, the number of swaps is ≤ O(n)
= 1/2 + 1/4 + 1/8 + 1/16 + … <= 1
+ 1/4 + 1/8 + 1/16 + … <= 1/2
+ + 1/8 + 1/16 + … <= 1/4

Σ j 2-j <= 2
29 30
So 2h Σ j 2-j <= 2. 2h = 2 n O(n)

Implementing a Heap
with an Array Example
A heap can be nicely represented by an array list 1
H

(array-based), 1
2
where the node at rank i has 2 3 D I
3

4 5 6 4
7 B
5
E
6
L
7
O

- left child at rank 2i 8 8 9 10 11 12 13

A C F G H N

and i

- right child at rank 2i + 1

2i 2i+1

1 2 3 4 5 6 7 8 9 10 11 12 13
1 2 3 4 5 6 7 8 H D I B E L O A C F G H N

The leaves do no need to be explicitly stored 31 32

8
Reminder ….. Implementation of a Priority
Left child of T[i] T[2i] if 2i ≤ n Queue with a Heap
Right child of
T[2i+1] if 2i + 1 ≤ n
T[i]
Parent of T[i] T[i div 2] if i>1
The Root T[1] if T≠0
Leaf? T[i] TRUE if 2i > n

1 n = 11

2 3
I

4 5 6 7

8 9 10 11

33 34

Application: Sorting Heap Sort

(upheap) PriorityQueueSort where the PQ is implemented with a HEAP

Algorithm PriorityQueueSort(S, P):
Input: A sequence S storing n elements, on which a
O(log n) total order relation is defined, and a Priority
O(log 1) Queue P that compares keys with the same relation
Output: The Sequence S sorted by the total
O(log n) order relation
while ¬ S.isEmpty() do
e ← S.removeFirst() Build Heap
P.insertItem(e, e)
while ¬ P.isEmpty() do
(remove root + downheap) e ← P.removeMin() Remove from heap
S.insertLast(e)
35 36

9
 Application: Sorting
Heap Sort When there are i nodes left in the PQ: log i
n

Construct initial heap O(n)

TOT = Σ log i
i=1

= O(n log n)
remove root O(1)
n re-arrange O(log n)
times
remove root O(1)

re-arrange O(log (n-1))

L M
37 38
L

HeapSort in Place

How about implementing a double PQ ??

removeMin() AND removeMax()

Example on the board ….

With a Min-heap: With a Max-Heap:

removeMin() is O(log n) removeMin() is ?

removeMax() is ? removeMax() is O(log n)

39 40

10
 Another type of Heap … removeMin()

Values stored at nodes on even (odd) levels are

smaller (greater) then or equal to the values of their 5

descendants 65 80

25 37 8 15

5 57 36 45 59 20 14 32 18

65 80 28 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17

25 37 8 15

57 36 45 59 20 14 32 18

28 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17

41 42

removeMin() removeMin()

5 Substitute with last leaf

smallest 17 8

65 80 65 80

25 37 8 15 25 37 17 15

57 36 45 59 20 14 32 18 57 36 45 59 20 14 32 18

28 30 34 27 39 38 45 50 15 12 13 10 30 31 16 28 30 34 27 39 38 45 50 15 12 13 10 30 31 16

smallest

43 44

11
 5
removeMin() Insert(x)
65 80

25 37 8 15

57 36 45 59 20 14 32 18

28 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17
8
x
65 80 If in Max-level:
25 37 10 15 If parent is greater, swap and BubbleUpMin
If parent is smaller, BubbleUpMax
57 36 45 59 20 14 32 18
BubbleUpMin
28 30 34 27 39 38 45 50 15 12 13 17 30 31 16
If grandparent is greater, swap
BubbleUpMin(grandparent)

BubbleUpMax
If grandparent is smaller, swap
Check parent BubbleUpMax(grandparent)
45 46

5 5
Insert(13) Insert(13)
65 80 65 80

25 37 8 15 25 37 8 15

57 36 45 59 20 14 32 18 57 36 45 59 20 14 32 18

28 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17 13 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17

13 28
If in Max-level:
If parent is greater, swap and BubbleUpMin
If parent is smaller, BubbleUpMax

BubbleUpMin
If grandparent is greater, swap
BubbleUpMin(grandparent)

47 48

12
 5 5
Insert(13) Insert(61)
65 80 65 80

13 37 8 15 25 37 8 15

57 36 45 59 20 14 32 18 57 36 45 59 20 14 32 18

25 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17 28 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17

28 61
If in Max-level:
If parent is greater, swap and BubbleUpMin
If parent is smaller, BubbleUpMax

BubbleUpMax
If grandparent is smaller, swap
BubbleUpMax(grandparent)
49 50

5
Insert(61)
65 80

25 37 8 15

61 36 45 59 20 14 32 18

28 30 34 27 39 38 45 50 15 12 13 10 30 31 16 17
What is the cost of
57
If in Max-level: removeMin() AND removeMax()
If parent is greater, swap and BubbleUpMin
If parent is smaller, BubbleUpMax With this heap ?

BubbleUpMax
If grandparent is smaller, swap
BubbleUpMax(grandparent)
51 52

13