ECE750 Lecture 3:

Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ECE750 Lecture 3: Divide and conquer,
Abstract Data Types, Lists, Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Electrical & Computer Engineering
University of Waterloo
Canada
Sept. 28, 2007
1 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Part I
Divide and conquer
2 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Divide And Conquer
BinarySearch is an (arguably degenerate) instance of a
basic algorithm design pattern:
Divide And Conquer
1. If the problem is of trivial size, solve it and return.
2. Otherwise:
2.1 Divide the problem into several problems of smaller size.
2.2 Conquer these smaller problems by recursively applying
the divide and conquer pattern.
2.3 Combine the answers to the smaller problems into an
answer to the whole problem.
3 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Divide and Conquer
Binary Search as Divide-and-Conquer:
1. If the problem is of size n = 1, answer is obvious
return.
2. Otherwise:

Split the array into two halves. Principle:

(x in A[l ..h]) (x in A[l ..i ]) (x in A[i + 1..j ])

Search the two halves: for one half, call self recursively;
for other half, answer is false.

Combine the two answers: since one answer is always

false, and x or false is just x, simply return the
answer from the half we searched.
4 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Divide-and-Conquer Recurrences
If
1. The base cases (trivially small problems) require time
O(1), and
2. a problem of size n is split into k subproblems of size
s(n), and
3. splitting the problem into subproblems and combining
the answers takes time f (n)
then the general form of the time recurrence is
T(n) = c + kT(s(n)) + f (n)
e.g. for binary search we had k = 1 (we only had to search
one half), s(n) = n/2, and f (n) = 0.
5 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Strassen Matrix Multiplication

Recall that our MatrixMultiply routine took (n

3
)
time.

Can we do better? No one thought so until...

A landmark paper:
Volker Strassen, Gaussian Elimination is not optimal
(1969). [1]

An o(n
3
) divide-and-conquer approach to matrix
multiplication.
6 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Strassens method
Strassen: If A, B are matrices of order m2
k+1
to be
multiplied, write
A =

A
11
A
12
A
21
A
22

B =

B
11
B
12
B
21
B
22

C =

C
11
C
12
C
21
C
22

where the A
ik
, B
ik
, C
ik
matrices are of order m2
k
...
7 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Strassens method
Then compute
(7 subproblems)

I = (A
11
+ A
22
)(B
11
+ B
22
)
II = (A
21
+ A
22
)B
11
III = A
11
(B
12
B
22
)
IV = A
22
(B
11
+ B
21
)
V = (A
11
+ A
12
)B
22
VI = (A
11
+ A
21
)(B
11
+ B
12
)
VII = (A
12
A
22
)(B
21
+ B
22
)
(and combine)

C
11
= I + IV V + VII
C
21
= II + IV
C
12
= III + V
C
22
= I + III II + VI
8 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Strassens method
1. Subproblems: compute 7 matrix multiplications of size
n/2.
2. Constructing the subproblems and combining the
answers is done with matrix additions/subtractions,
taking (n
2
) time.
3. Apply general divide-and-conquer recurrence with
k = 7, s(n) = n/2, f (n) = (n
2
):
T(n) = c + 7T(n/2) + (n
2
)
9 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Strassens method
Recall that (n
2
) means some function f (n) for which
f (n)
n
2
is eventually restricted to some nite positive interval
[c
1
, c
2
].
Pick a value c > c
2
; then eventually f (n) cn
2
.
Solve recurrence:
T(n) = c + 7T(n/2) + cn
2
This will give an asymptotically correct bound, but possibly
T(n) is less than the actual time required for small n.
10 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Strassens method

Let r () = T(2

), and = log
2
n. Recurrence becomes
r () = c + 7r ( 1) + c(2

)
2
= c + 7r ( 1) + c(4

Z-transform:
Z [c] =
c
1 z
1
Z [7r ( 1)] = 7z
1
R(z)
Z [c(4

)] =
c
1 4z
1

Z-transform version of recurrence is:

R(z) =
c
1 z
1
+ 7z
1
R(z) +
c
1 4z
1

Solve:
R(z) =
c2(1
5
2
z
1
)
(1 z
1
)(1 4z
1
)(1 7z
1
)
11 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Strassens method

Look at singularities: zero at z =

5
2
, poles at z = 1, 4, 7.

Pole at z = 7 is asymptotically dominant:

r () c
1
7

Change variables back: = log

2
n:
T(n) c
1
7
log
2
n
= c
1
n
log
2
7
= c
1
n
2.807

Strassen matrix multiplication takes (n

2.807
) time.
12 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Divide-and-conquer recurrences I

Our analysis of Strassens algorithm is easily generalized.

Consider a divide-and-conquer recurrence of the form

T(n) = kT(n/s) + (n
d
) (1)

To obtain an asymptotic upper bound we can solve the

related recurrence
T

(n) = kT

(n/s) + cn
d
(2)

It can be shown that T(n) T

(n).

Analyzing for the case when n = s

with N, we can
change variables to turn Eqn. (2) into:
r () = kr ( 1) + c(s
d
)

13 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Divide-and-conquer recurrences II

Z-transform:
R(z) = kz
1
R(z) +
c
1 s
d
z
1

Solve:
R(z) =
c
(1 kz
1
)(1 s
d
z
1
)

Which is the dominant pole? Depends on the values of

k, s, d.

Three cases: s
d
< k, s
d
= k, s
d
> k.
1. s
d
< k. Then the dominant pole is z = k. Get
r () k

Since n = s

, use = log
s
n:
T

(n) k
log
s
n
= (2
log k
)
log n
log s
= (2
log n
)
log k
log s
= n
log k
log s
14 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Divide-and-conquer recurrences III
2. s
d
= k. Then get a double pole at z = k. Recall that
Z
1

kz
1
(1 kz
1
)
2

= k

End up with
r () k

(n) (log
s
n)k
log
s
n
(log n)2
log klog n
log s
= n
log k
log s
log n
= n
log s
d
log s
log n = n
d
log n
3. s
d
> k. Then dominant singularity is z = s
d
. Get
r () (s
d
)

(n) (s
d
)
log
s
n
= (2
d log s
)
log n
log s
= n
d
15 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Master Theorem
Theorem (Master)
The solution to a divide-and-conquer recurrence of the form
T(n) = kT(n/s|) + (n
d
)
where s > 1, is
T(n) =

n
log k
log s

if s
d
< k

n
d
log n

if s
d
= k
(n
d
) if s
d
> k

Examples:

Binary search: k=1, s=2, d=0: second case,

log k
log s
= 0,
so T(n) = (n
0
log n) = (log n).

Strassen: k = 7, s = 2, d = 2: rst case,

log k
log s
2.807,
so T(n) = (n
2.807
).
16 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Part II
Abstract Data Types (ADTs)
17 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Abstract Data Types

A basic software engineering principle:

Separate the interface (what you can do) from
the implementation (how it is done.)

An abstract data type is a an interface to a collection of

data.

There may be numerous ways to implement an ADT,

each with dierent performance characteristics.

An ADT consists of
1. Some types that may be required to provide operations,
relations, and satisfy properties. e.g. a totally ordered
set.
2. An interface: the capabilities provided by the ADT.
18 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Abstract Data Types

Notation for types: T, f

1
, f
2
, , R
1
, R
2
, ) indicates
a set T together with

Some operators or functions f

1
, f
2
, . . .

Some relations R
1
, R
2
, .
This is the standard notation for a structure in logic:
could be

an algebra (functions but no relations) e.g. a eld

F, +, , ,
1
, 0, 1);

a relational structure (relations but no functions) e.g. a

total order T, );

some structure with both functions and relations e.g. an

ordered eld F, +, , ,
1
, 0, 1, )

Often a type is required to satisfy certain axioms, or

belong to a specied class of structures, e.g. (a eld, a
total order, a poset).
19 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ADT: Dictionary[K, V]
Dictionary[K,V]

Stores a set of pairs (key, value), and nds values by

key. At most one value per key is permitted.

Types:

K): key type (e.g. a word).

V, 0): value type (e.g., a denition of a word). The

value 0 is a special value used to indicate absence of a
dictionary entry.

Operations:

insert(k, v): insert a key-value pair into the dictionary.

If an entry for the key is already present, it is replaced.

V nd(k): if (k, v) is in the dictionary, returns v;

otherwise returns 0.

remove(k): deletes the entry for key k, if present.

20 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Abstract Data Types
ADT
Implementation
Data structure
Linked list
Dictionary
55
k
k
k
k
k
k
k
k
k
//
))
S
S
S
S
S
S
S
S
S
Sorted array
Search tree
Implementation insert time nd time remove time
Linked list O(1) O(n) O(n)
Sorted array O(n) O(log n) O(n)
Search tree O(log n) O(log n) O(log n)
21 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
The role of ADTs in choosing data structures
Questions to consider when choosing a data structure:
1. What type of data do you need to store?

Does it have a natural total order? e.g. integers, strings

under lexicographic ordering, database keys, etc.

Does it bear some natural partial order?

Do elements represent points or regions in some metric

space, e.g., R
n
? (e.g., screen regions, boxes in a
three-dimensional space, etc.)
The order relation(s) or geometric organization of your
data may allow the use of ADTs that exploit those
properties to allow ecient access.
22 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
The role of ADTs in choosing data structures
2. What operations are required? Do you need to

determine whether a value is in the data set (search)?

insert, modify, delete elements?

iterate through the elements (order doesnt matter)?

iterate through the elements in some sorted order?

nd the biggest or smallest element?

nd elements that are close to some value?

These requirements can be compared with the
interfaces provided by ADTs, to decide what ADTs
might be suitable.
23 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
The role of ADTs in choosing data structures
3. What is the typical mixture of operations you will
perform? Will you

insert frequently?

delete frequently?

search frequently?
Dierent ADT implementations may oer dierent
performance characteristics. By understanding the
typical mixture of operations you can choose an
implementation with the most suitable performance
characteristics.
24 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ADT: Array[V]
Array[V]

A nite sequence of cells, each containing a value.

Types :

V, 0): a value type, with a default value 0 used to

initialize cells.

Operations :

Array(n): create an array of length n, where n N is a

positive integer. Cells are initialized to 0.

integer length(): returns the length of the array

get(i ): returns the value of cell i . It is required that

0 i length() 1.

set(i , v): sets the value of cell i to v. It is required that

0 i length() 1.
25 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ADT: Set[V]
Set[V]

Stores a set of values. Permits inserting, removing, and

testing whether a value is in the set. At most one
instance of a value can be in the set.

Types:

V): a value type

Operations:

insert(v): adds v to the set, if absent. Inserting an

element already in the set causes no change.

remove(v): removes v from the set, if present;

boolean contains(v): returns true if and only if v is in

the set.
26 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ADT: Multiset[V]
Multiset[V]

Stores a multiset of values (i.e., with duplicate elements

permitted.) Permits inserting, removing, and testing
whether a value is in the set. Sometimes called a bag.

Types:

V): a value type

Operations:

insert(v): adds v to the multiset.

remove(v): removes an instance of v from the multiset,

if present;

boolean contains(v): returns true if and only if v is in

the set.
27 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Stacks, Queues, and Priority Queues

Informally, a queue is a collection of objects awaiting

their turn. e.g., customers queueing at a grocery store.
The queueing policy governs who goes next.

First In, First Out (FIFO): like a line at the grocery

store: the element that was added least recently goes
next.

Last In, First Out (LIFO): the item added most recently
goes next. A stack: like an in-box of work where new
items are placed on the top, and what ever is on the top
of the stack gets processed next.

Priority Queueing: items are associated with priorities;

the item with the highest priority goes next.
28 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ADT: Queue[V]
Queue[V]

A FIFO queue (rst in, rst out) of objects.

Types :

V): a value type

Operations :

insert(v): adds the object v to the end of the queue

boolean isEmpty(): returns true just when the queue is

empty

V next(): returns and removes the value at the front of

the queue. It is an error to perform this operation when
the queue is empty.
29 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ADT: Stack[V]
Stack[V]

A LIFO (last in, rst out) stack of objects.

Types :

V): a value type

Operations :

push(v): adds a value to the top of the stack.

boolean isEmpty(): returns true just when the stack

contains no elements.

V pop(): returns the value at the top of the stack. It is

an error to perform this operation when the stack is
empty.
30 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
ADT: PriorityQueue[P,V]
PriorityQueue[P,V]

A queue of objects, each with an associated priority, in

which an object with maximal priority is always chosen
next.

Types :

P, ): a priority type, with a total order .

V): a value type

Operations :

insert(p, v): insert a pair (p, v) where p P is a

priority, and v V is a value;

boolean isEmpty(): returns true just when the queue is

empty;

(P, V) next(): returns and removes the object at the

front of the queue, which is guaranteed to have
maximal priority. It is an error to perform this operation
on an empty queue.
31 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Part III
Linked Lists
32 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Data Structure: Linked List

Realizes a list of items, e.g., (2, 3, 5, 7, 11)

Inserting and removing elements at the front of the list

requires O(1) time.

Searching requires iterating through the list: O(n) time

List can be iterated through from front to back.

Basic building block is a node that contains:

data: A piece of data;

next: A pointer to the next node, or a null pointer if

the node is the end of the list.
33 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Data Structure: Linked List
public class LinkedList <T>
Node<T> head; // Pointer to the rst node in the list
...

class Node<T>
T data; // Data item
Node<T> next; // Next node in the list , if any
Node(T data, Node<T> next)

data = data;
next = next;

34 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Data Structure: Linked List I

Insert a new data element:

public void insert (T data)

head = new Node<T>(data, head);

Remove the front element, if any:

public T removeFirst()

if (head == null)
throw new RuntimeException(List is empty.);
else
T data = head.data;
head = head.next;
return data;

35 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Data Structure: Linked List II

Check if the list is empty:

public boolean isEmpty()

return (head == null);

36 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Implementing Stack[V] with a Linked List

The ADT Stack[V] is naturally implemented by a

linked list.
public class Stack<V>
LinkedList <V> list;
public void push(V v) list . insert (v);
public boolean isEmpty() return list . isEmpty();
public V pop() return list . removeFirst ();

push(v), isEmpty(), and pop() require O(1) time.

37 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Iterators
Iterator[V]

An iterator is an ADT that provides traversal through

some container. It abstracts away from the details of
how the data are stored, and presents a simple interface
for retrieving the elements of the container one at a
time.

Types:

V: the value type stored in the container

Operations:

Iterator(C): initialize the iterator to point to the rst

element in the container C;

boolean hasNext(): returns true just when there is

another element in the container;

V next(): returns the next element in the container,

and advances the iterator.
38 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
An Iterator for a linked list
public class ListIterator <T>
Node<T> node;
ListIterator ( LinkedList <T> list)
node = list . head;
boolean hasNext()
return (node == null);
T next()

if (node == null)
throw new RuntimeException(Tried to iterate past end of list );
T data = node.data;
node = node.next;
return data;

39 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Iterators
Example usage of an iterator:
ListIterator <Integer> iter = new ListIterator <Integer>( list );
while ( iter . hasNext())

System.out. println (The next element is + iter . next ());

40 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Implementing Queue[V] with a Linked List

Recall that a Queue[V] ADT requires rst-in rst-out

queueing. But, the list as we have shown it only
supports inserting and removing at one end.

Simple variant of a linked list: each list item contains a

pointer to the next and previous items.
41 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List
class BiNode<T>
T data; // Data item
BiNode<T> next; // Next node in the list , if any
BiNode<T> prev; // Previous node in the list , if any
BiNode(T data, BiNode<T> next)

data = data;
next = next;
next. prev = this;

42 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List
43 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List
public class BiLinkedList <T>
BiNode<T> head; // Pointer to the rst node in the list
BiNode<T> tail; // Pointer to the last node in the list
public void insert (T data)

head = new BiNode<T>(data, head);

if ( tail == null)
tail = head;

44 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List
public T removeLast()

if ( tail == null)
throw new RuntimeException(List is empty.);
else
T data = tail . data;
if ( tail . prev == null)

head = null;
tail = null;

else
tail = tail . prev;
tail . next = null;

return data;

45 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Implementing a Queue < V > with a
Bidirectional Linked List
public class Queue<V>
BiLinkedList <V> list;
public void insert (V v) list . insert (v);
public boolean isEmpty() return list . isEmpty();
public V next() return list . removeLast();

insert(v), isEmpty() and next() all take O(1) time.

46 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List

Bidirectional Linked List

Each node has a link to both the next and previous

items in the list.

We maintain a pointer to both the front and the back.

We can insert and remove items at both the front and

back of the list.

We will use Bidirectional Linked Lists to illustrate two

basic, but extremely useful principles:
1. Maintaining invariants of data structures;
2. Symmetry.
47 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List

We have encountered invariants already, in the

correctness sketch of binary search. That was an
invariant for a recursive algorithm, and was required to
be true of each recursive invokation of the function.
Here we discuss data structure invariants.
An invariant of a data structure is a property
that is required to be always true, except
when we are performing some transient
update operation.

Invariants help us to implement data structures

correctly: many basic operations can be viewed as
disruptions of the invariant (e.g., inserting an element)
after which we need to repair or maintain the invariant.
48 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List

As in a linked list, the basic building block of a

bidirectional linked list is a node.
class BiNode<T>
T data; / Data item /
BiNode<T> next; / Next node in the list , if any /
BiNode<T> prev; / Previous node in the list , if any /
BiNode(T data)

data = data;
next = null;
prev = null;

49 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Invariants

Lets look at a few examples to see what invariants

suggest themselves.
50 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Invariants

Here are the invariants we will use:

1. (front ,= null) implies (front.prev = null). (If the list is
nonempty, there is no element before the front element.)
2. (back ,= null) implies (back.next = null). (If the list is
nonempty, there is no element after the last element.)
3. (front = null) if and only if (back = null).
4. For any node x,
4.1 (x.next = null) implies x.next.prev = x;
4.2 (x.prev = null) implies x.prev.next = x.
51 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Symmetry

Bidirectional linked lists have a natural symmetry: if we

reverse the list by swapping the front back
pointers and each of the next prev pointers, we get
another bidirectional linked list.

Well call this the dual list.

The symmetry extends to operations on the list:

insertFront() is dual to insertBack(), and
removeFront() is dual to removeBack().

This kind of duality has the following nice property:

Carrying out a sequence of operations op

1
, . . . , op
k
gives the same result as

Taking the dual list, carrying out the dual sequence of

operations op
1
, . . . , op
2
, and taking the dual list.
52 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Symmetry

Example: starting from the list [1, 2], we can

insertBack(3) to give [1, 2, 3];

removeFront() to give [2, 3].

The dual version:

take the dual list [2, 1];

insertFront(3) to give [3, 2, 1];

removeBack() to give [3, 2];

take the dual list [2, 3].

Get the same answer both ways.
53 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Symmetry

Why we care about symmetry: if our implementation is

correct,

We can obtain the routine insertBack() by taking the

code for insertFront() and swapping front/back and
prev/next;

Ditto for removeBack() and removeFront();

The set of invariants should not change under swapping

front/back and prev/next. Example: For any node x,
1. (x.next = null) implies x.next.prev = x;
2. (x.prev = null) implies x.prev.next = x.
If we swap next/prev, we get
1. (x.prev = null) implies x.prev.next = x;
2. (x.next = null) implies x.next.prev = x.
54 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Implementation

For operations at the front of the list, there are three

cases to consider:
1. front=null (empty list)
2. front ,= null and front.next = null (one-element list)
3. front ,= null and front.next ,= null (multi-element list)

We need to consider each of these cases when we

implement, and ensure that in each case, the invariants
are maintained.
55 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Implementation
public void insertFront (T data)

BiNode<T> node = new BiNode<T>(data);

if ( front == null) / Case 1 /

front = node; / Both made nonnull for Inv. 3 /

back = node;
else / Case 2,3 /
front . prev = node; / Inv 4.1 /
node.next = front; / Inv 4.2 /
front = node;

56 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Implementation
public void insertBack(T data)

BiNode<T> node = new BiNode<T>(data);

if (back == null) / Case 1 /

back = node; / Both made nonnull for Inv. 3 /

front = node;
else / Case 2,3 /
back.next = node; / Inv 4.1 /
node.prev = back; / Inv 4.2 /
back = node;

57 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Implementation
public T removeFront()

if ( front == null) / Case 1 /

throw new RuntimeException(Empty list.);
else
T data = front.data;
front = front. next;
if ( front == null) / Case 2 /
back = null;
else
front . prev = null; / Case 3 /

return data;

58 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bidirectional Linked List: Implementation
public T removeBack()

if (back == null) / Case 1 /

throw new RuntimeException(Empty list.);
else
T data = back.data;
back = back.prev;
if (back == null) / Case 2 /
front = null;
else
back.next = null; / Case 3 /

return data;

59 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Part IV
Heaps
60 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Heaps

A heap dynamically maintains the maximum element in

a collection (or, dually, the minimum element). A
binary heap can:

Obtain the maximum element in O(1) time;

Remove the maximum element in O(log n) time;

Insert new element in O(log n) time.

Heaps are a natural implementation of the
PriorityQueue ADT.

There are several avours of heaps: binary heaps,

binomial heaps, bonacci heaps, pairing heaps. The
more sophisticated of these support merging (melding)
two heaps.

We will look at binary heaps.

61 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Binary Heap Invariants
1. A binary heap is a complete binary tree of height h 1,
plus a possibly incomplete level of height h lled from
left to right.
2. The key stored at each node is the key(s) stored in its
children (if used, you get a min heap.)
62 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Binary Heap

A binary heap may be stored as a (1-based) array,

where

Parent(j ) = j /2|

LeftChild(i ) = 2 i

RightChild(i ) = 2 i + 1

e.g., [17, 11, 13, 9, 6, 2, 12, 4, 3, 1] is an array

representation of the heap:
63 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Heap operations

To insert a key k into the heap:

Place k at the next available position.

Swap k with its parent(s) until the heap invariant is

satised. (Takes O(log n) time.)

The maximum element is just the key stored at the

root, which can be read o in O(1) time.

To delete the maximum element:

Place the key at the last heap position at the root

(overwriting the current maximum), and decrease the
size of the heap by one.

Choose the largest of the root and its two children, and
make this the root; perform this procedure recursively
until the heap invariant is satised.
64 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Heap: insert example

Example: insert 23 into the heap and restore the heap

invariant.
65 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Heap: delete-max example

To delete the max element, move the element from the

last position (2) to the root;

To restore heap invariant, swap root with the largest

child greater than it, if any, and repeat down the heap.
66 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Heapsort

Heaps can be used to implement a simple sorting

algorithm that runs in (n log n) worst case: given an
unordered array A[0..n 1],
1. Create an empty min heap.
2. Insert elements A[0], . . . , A[n 1] into the heap.
3. For i = 0 . . . n 1, remove the min element from the
heap and store it at A[i ].
67 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
tveldhui@acm.org
Bibliography
Bibliography I
[1] V. Strassen.
Gaussian elimination is not optimal.
Numerische Mathematik, 13:354356, 1969.
68 / 68