Professional Documents
Culture Documents
Fibonacci heaps,
and applications
1
Binomial trees
B0
B1
Bi
B(i-1)
B(i-1)
2
Binomial trees
Bi
......
B0
B(i-1) B(i-2) B1
3
Properties of binomial trees
1) | Bk | = 2k
2) degree(root(Bk)) =
k
3) depth(Bk) =
k
4
Binomial heaps (def)
A collection of binomial trees at most one of every rank.
Items at the nodes, heap ordered.
1 5 5
2 5 6 6
5 9 8
Possible rep: Doubly link roots and
10 children of every nodes. Parent
pointers needed for delete.
5
Binomial heaps (operations)
Operations are defined via a basic operation, called linking, of
binomial trees:
Produce a Bk from two Bk-1, keep heap order.
1
4 2 5 6
5 6 11 5 9 8
6 9 9 10
10
6
Binomial heaps (ops cont.)
Basic operation is meld(h1,h2):
Like addition of binary numbers.
B5 B4 B2 B1
h1: B4 B3 B1 B0
h2: B4 B3 B0 +
B5 B4 B2
7
Binomial heaps (ops cont.)
Findmin(h): obvious
Insert(x,h) : meld a new heap with a single B0 containing
x, with h
deletemin(h) : Chop off the minimal root. Meld the
subtrees with h. Update minimum pointer if needed.
delete(x,h) : Bubble up and continue like delete-min
decrease-key(x,h,) : Bubble up, update min ptr if needed
8
Amortized analysis
9
Amortized analysis (Cont.)
(c) = ?
Amortized(increment) = actual(increment) +
10
Amortized analysis (Cont.)
Amortized(increment1) = actual(increment1) + 1- 0
Amortized(increment2) = actual(increment2) + 2- 1
+
iAmortized(incrementi) = iactual(incrementi) + n- 0
iAmortized(incrementi) iactual(incrementi)
if n- 0 0 11
Amortized analysis (Cont.)
Define a potential of the counter:
(c) = #(ones)
Amortized(increment) = actual(increment) +
12
Binomial heaps - amortized ana.
13
Binomial heaps + lazy meld
Meld (h1,h2) :
Concatenate the lists of binomial trees.
Update the minimum pointer to be the smaller of the
minimums
14
Binomial heaps + lazy meld
As long as we do not do a delete-min our heaps are just
doubly linked lists:
9 5 9 11 4 6
15
Binomial heaps + lazy meld
Possible implementation of delete-min is using an array
indexed by rank to keep at most one binomial tree of each
rank that we already traversed.
Once we encounter a second tree of some rank we link them
and keep linking until we do not have two trees of the same
rank. We record the resulting tree in the array
Amortized(delete-min) =
= (#(trees at the end) + #links + max-rank) - #links
(2log(n) + #links) - #links = O(log(n))
16
Binomial heaps + lazy delete
How do we do findmin ?
17
Binomial heaps + lazy delete
18
Binomial heaps + lazy delete
19
Binomial heaps + lazy delete
20
Binomial heaps + lazy delete (ana.)
21
Binomial heaps + lazy delete (ana.)
What is the amortized cost of find-min ?
amortized(find-min) = amortized(purging) +
amortized(successive linking +
scan of undeleted
nodes)
We saw that: amortized(successive linking) = O(log(n))
Proof.
24
Application: The round robin algorithm
of Cheriton and Tarjan (76) for MST
We shall use a Union-Find data structure.
The union find problem is where we want to maintain a
collection of disjoint sets under the operations
1) S=Union(S1,S2)
2) S=find(x)
Can do it in O(1) amortized time for union and O((k,n))
amortized time for find, where k is the # of finds, and n is the
number of items (assuming k n).
25
A greedy algorithm for MST
Start with a forest where each tree is a singleton.
Repeat the following step until there is only one tree in the
forest:
Pick T F, pick a minimum cost edge e connecting a vertex
of T to a vertex in T, add e to the forest (merge T and T to
one tree)
26
Cheriton & Tarjans ideas
Keep the trees in a queue, pick the first tree, T, in the queue,
pick the lightest edge e connecting it to another tree T.
Remove T from the queue, connect T and T with e. Add the
resulting tree to the end of the queue.
27
Cheriton & Tarjan (cont.)
T
T e
T
T e 28
Cheriton & Tarjan (implementation)
The vertices of each tree T will be a set in a Union-Find data
structure. Denote it also by T
Edges with one endpoint in T are stored in a heap data structure.
Denoted by h(T). We use binomial queues with lazy meld and
deletion.
Note: The meld implicitly delete edges. Every edge in h(T) with
both endpoints in T is considered as marked deleted.
We never explicitly delete edges!
We can determine whether an edge is deleted or not by two find
operations.
30
Cheriton & Tarjan (analysis)
Assume for the moment that find costs O(1) time.
Then we can determine whether a node is marked deleted in O(1)
time, and our analysis is still valid.
So, we have
at most 2m implicit delete operations that cost O(m).
at most n find operations that cost O(n).
at most n meld and union operations that cost O(n).
at most n find-min operations. The complexity of these find-min
operations dominates the complexity of the algorithm.
31
Cheriton & Tarjan (analysis)
Let mi be the number of edges in the heap at the i-th iteration.
Let pi be the number of deleted edges purged from the heap at the
find-min performed by the i-th iteration.
So, we proved that the i-th find-min costs O(pi *(log(mi / pi)+ 1) ).
We want to bound the sum of these expressions.
32
Cheriton & Tarjan (analysis)
Divide the iterations into passes as follows.
Pass 1 is when we remove the original singleton trees from the
queue.
Pass i is when we remove trees added to the queue at pass i-1.
33
Cheriton & Tarjan (analysis)
An edge can occur in at most two heaps of trees in one pass.
So i mi 2m log(n)
34
Cheriton & Tarjan (analysis)
We obtained a time bound of O(m loglog(n)) under the
assumption that find takes O(1) time.
But if you amortize the cost of the finds on O(m loglog(n))
operations then the cost per find is really
(m loglog(n),n) = O(1)
35
Fibonacci heaps (Fredman & Tarjan 84)
Why ?
36
Dijkstras shortest path algorithm
3
3
s 2 1
3
2
37
Dijkstras shortest path algorithm
Dijkstra: Maintain an upper bound d(v) on d(s,v).
Every vertex is either scanned, labeled, or unlabeled.
Initially: d(s) = 0 and d(v) = for every v s.
s is labeled and all others are unlabeled.
Pick a labeled vertex with d(v) minimum. Make v scanned.
For every edge (v,w) if d(v) + w(v,w) < d(w) then
1) d(w) := d(v) + w(v,w)
2) label w if it is not labeled already
38
Dijkstras shortest path algorithm (implementation)
Maintain the labeled vertices in a heap, using d(v) as the key of v.
39
Back to Fibonacci heaps
Suggested implementation for decrease-key(x,h,):
If x with its new key is smaller than its parent, cut the subtree
rooted at x and add it to the forest. Update the minimum
pointer if necessary.
40
2 5 5
5 3 6 6
5 9 8
10
2 1 5 5
5 6 8 6
5 9
10
41
Decrease-key (cont.)
Does it work ?
42
Decrease-key (cont.)
9 5 3 6 5 6
43
Fibonacci heaps (cont.)
We shall allow non-binomial trees, but will keep the degrees
logarithmic in the number of nodes.
Rank of a tree = degree of the root.
Delete-min: do successive linking of trees of the same rank and
update the minimum pointer as before.
Insert and meld also work as before.
44
Fibonacci heaps (cont.)
Decrease-key (x,h,): indeed cuts the subtree rooted by x if
necessary as we showed.
in addition we maintain a mark bit for every node. When we
cut the subtree rooted by x we check the mark bit of p(x). If it
is set then we cut p(x) too. We continue this way until either
we reach an unmarked node in which case we mark it, or we
reach the root.
This mechanism is called cascading cuts.
45
2
4 20
5 8 11
9 6 14
10 16
12 15
7 9 5 4 2
16 6 14 8 11 20
12 15
46
Fibonacci heaps (delete)
47
Fibonacci heaps (analysis)
Want everything to be O(1) time except for delete and
delete-min.
==> cascading cuts should pay for themselves
49
Fibonacci heaps (analysis)
Lemma 1 : Let x be any node in an F-heap. Arrange the children
of x in the order they were linked to x, from earliest to latest.
Then the i-th child of x has rank at least i-2.
2 1
Proof:
When the i-th node was linked it must have had at least i-1
children.
Since then it could have lost at most one.
50
Fibonacci heaps (analysis)
Corollary1 : A node x of rank k in a F-heap has at least k
descendants, where = (1 + 5)/2 is the golden ratio.
Proof:
Let sk be the minimum number of descendants of a node of rank
k in a F-heap.
k-2
By Lemma 1 sk s + 2
i=0 i x
s0=1, s1= 2
51
Fibonacci heaps (analysis)
Proof (cont):
Fibonnaci numbers satisfy
Fk+2 = i=2
k
Fi + 2, for k 2, and F2=1
so by induction sk Fk+2
It is well known that Fk+2 k
52
Application #2 : Prims algorithm for
MST
Start with T a singleton vertex.
Grow a tree by repeating the following step:
Add the minimum cost edge connecting a vertex in T to a
vertex out of T.
53
Application #2 : Prims algorithm for
MST
Maintain the vertices out of T but adjacent to T in a heap.
The key of a vertex v is the weight of the lightest edge (v,w)
where w is in the tree.
Iteration: Do a delete-min. Let v be the minimum vertex and
(v,w) the lightest edge as above. Add (v,w) to T. For each edge
(w,u) where uT,
if key(u) = insert u into the heap with key(u) = w(w,u)
if w(w,u) < key(u) decrease the key of u to be w(w,u).
55
Thin binomial trees
A thin binomial tree is a binomial tree where each nonroot
and nonleaf node may have lost its leftmost child
Bi
...... B0
.. B(i-2) B1
B(i-3)
B(i-1)
56
Thin binomial trees (cont)
So either rank(x) = degree(x) or rank(x) = degree(x) + 1
In the latter case we say that the node is marked
Bi
...... B0
.. B(i-2) B1
B(i-3)
B(i-1)
57
Thin heaps
Thin heaps maintain the tree to be thin binomial tree by
changing the way we do cascading cuts.
58
Cascading cuts
Bi
...... B0
.. B(i-2) B1
B(i-3)
B(i-1)
We may get an illegal hole
59
Cascading cuts
Bi
...... B0
.. B(i-2) B1
B(i-3)
B(i-1)
Or a rank violation
60
How do we fix a hole ?
...... B0
.. B(i-2) B1
B(i-3)
B(i-1)
61
How do we fix a hole ?
...... B0
.. B(i-2) B1
B(i-3)
If it is unmarked
Bi
...... B0
.. B(i-2) B1
B(i-3)
B(i-2)
63
How do we fix a rank violation ?
...... B0
.. B(i-2) B1
B(i-3)
B(i-1)
You may create a violation at the
parent
64
Application #3 : Improving the
O(mloglog(n)) bound for MST on sparse
graphs
Iteration i: We grow a forest, tree by tree, as follows.
Start with a singleton vertex and continue as in Prims algorithm
until either
1) The size of the heap is larger than ki
2) Next edge picked is connected to an already grown tree
3) Heap is empty (if the graph is connected this will happen
only at the very end)
65
Application #3 : Improving the
O(mloglog(n)) bound for MST on sparse
graphs (cont)
Contract each tree into a single vertex and start iteration i+1.
How do we contract ?
Do a DFS on the tree, marking for each vertex the # of the tree
which contains it. Each edge e gets two numbers l(e), h(e) of the
trees at its endpoints.
If h(e) = l(e) remove e (self loop).
(stable) Bucket sort by h(e) and by l(e), parallel edge then
become consecutive so we can easily remove them.
O(m) time overall. 66
Application #3 : Improving the
O(mloglog(n)) bound for MST on sparse
graphs (cont)
Complexity of iteration i:
67
Application #3 : Improving the
O(mloglog(n)) ...
Once ki n we stop.
2m/n
2
2
i n
22
69
Application #3 : Improving the
O(mloglog(n)) (Summary)
70
Further research
71