Data Structures and Algorithms

 Instructors:
Data Structures and • Marius Joldoş (lectures + lab. sessions),
• Room D3, 71-73 DorobanŃilor St.
Algorithms • Tel. 0264-401276
• Marius.Joldos@cs.utcluj.ro,
• Paulina Mitrea (lab. sessions)
• Paulina.Mitrea@cs.utcluj.ro

About the course (objectives, outline,  Course web site:

recommended reading) • http://users.utcluj.ro/~jim/DSA

Problem solving • http://172.27.207.2:81/DSA local in the laboratory (but fully

Notions of Algorithmics (growth of featured)
functions, efficiency, programming model,  Schedule: as set in the timetable
example analysis)

Lists, stacks, queues.
• Lectures: Friday 12-14, room P03, 26-28 G.BariŃiu St.
• Labs: room D3, 71-73 DorobanŃilor St. every week : 30411-2:
Wednesday, 30413-4: Thursday, as scheduled

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 1 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 2

Objectives Organization & interaction

 To get familiar with various ways to characterize
properties of algorithms and data structures.
 To learn to compare and algorithms and data
structures.
 To get familiar with standard algorithms and data
structures so that you can find appropriate
algorithms and data structures when they are
available.
 To learn to design your own algorithms and data
structures when no appropriate can be found in
the literature and libraries.
 During lectures there will be 15 min, on assigned
topics (one week before) presentations made by
you
• Followed by discussions on the topic
• Are part of the requirements for DSA
 Laboratory sessions (already mentioned)
• Work in teams of two
• Split the task between team members
• Present your work in class
• Criticize (constructively) the work – learn how to evaluate
work
• Deadlines MUST be met

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 3 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 4

 Evaluation
 Activity during the semester – 40%
• Assigned programming tasks
• Presentations, etc.
 Written exams
• Midterm, in class – 30%
• After lecture 6?
• Final, during summer session – 70% of written
(theory + problem)
 There may be bonuses ( <= 1 point)
Recommended books
 Aho, Hopcroft, Ullman. Data Structures and
Algorithms, Addison-Wesley, 427 pages, 1987.
• The classic book on the topic, still most valuable introductory
book, though it misses a few topics. Uses “extended Pascal”.
 Cormen, Leiserson, Rivest, (Stein): Introduction
of written
to Algorithms. MIT Press / McGraw Hill, (2nd
edition),1028 pages, 1990 (2002).
• Written for all levels, with introductory chapters in discrete
math and offering specialized topics in later chapters.
• Uses pseudocode, which makes the presentation compact,
easy to follow and “timeless”. Also useful as a reference book.

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 5 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 6

Recommended books Outline (preliminary)

 Preiss. Data Structures and Algorithms with object-  Introduction: algorithmics  Undirected graphs:
Oriented Design Patterns in C++, John Wiley and Sons, and asymptotics. Lists, minimum cost spanning
660 pages, 1999. stacks and queues trees (MST), MST
• An unusual algorithms and data structures book in that it  Trees, binary trees, binary algorithms, traversals,
stresses from the beginning the role of design patterns for the search trees graph matching
implementation of data structures as classes. An excellent  Basic ADTs for sets:  Algorithm analysis and
source for good object-oriented design principles, with an design techniques: divide
dictionary, hash tables,
accessible introduction to complexity analysis. and conquer, dynamic
mappings, priority queues
• Available online (see course web site)
 Advanced ADTs for sets: 2- programming, greedy
 Knuth. The Art of Computer Programing, Addison Wesley, 3 trees, AVL trees, union- algorithms, backtracking,
three volumes, reprinted several times. fiind sets local search algorithms
• A classical book, extremely well written and documented  Directed graph ADTs:  Sorting
by a Master of the field. Unfortunately uses the language single source shortest path,  Data Structures and
of the hypothetical machine MIX for describing the all pairs shortest path, algorithms for external
algorithms which makes understanding more difficult. traversals storage. B-trees.

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 7 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 8

 Problem Solving
 First construct an exact model in terms of
which we can express allowed solutions.
• Finding such a model is already half the solution.
Any branch of mathematics or science can be
called into service to help model the problem
domain.
 Once we have a suitable mathematical
model, we can specify a solution in terms of
that model.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos
Basic Notions
 Model of Computation: An abstract sequential
computer, called a Random Access Machine or
RAM. Uniform cost model.
 Computational Problem: A specification in
general terms of inputs and outputs and the desired
input/output relationship.
 Problem Instance: A particular collection of inputs
for a given problem.
 Algorithm: A method of solving a problem which
can be implemented on a computer. Usually there
are many algorithms for a given problem.
 Program: Particular implementation of some
algorithm.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos
Representation of real world objects
 Issues when dealing with the representation of real
world objects in a computer program
• how real world objects are modeled as
mathematical entities,
• the set of operations that we define over these
mathematical entities,
• how these entities are stored in a computer's
memory (e.g. how they are aggregated as fields in
records and how these records are arranged in
memory, perhaps as arrays or as linked
structures), and
• the algorithms that are used to perform these
operations.
9 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 10
Example of computational problem: sorting
 Statement of problem:
• Input: A sequence of n numbers <a1, a2, …, an>
• Output: A reordering of the input sequence <a´1,
a´2, …, a´n> so that a´i ≤ a´j whenever i < j
 Instance: The sequence <7, 5, 4, 10, 5>
 Algorithms:
• Selection sort
• Insertion sort
• Merge sort
• (many others)
11 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 12
 Algorithm Design Algorithm Analysis of algorithms
 Determine the amount of some resource required
by an algorithm (usually depending on the size of
the input).
 The resource might be:
• running time
• memory usage (space)
• number of accesses to secondary storage
• number of basic arithmetic operations
• network traffic
 Formally, we define the running time of an
algorithm on a particular input instance to be the
number of computation steps performed by the
algorithm on this instance.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 13 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 14

Growth of functions A General Portable Performance Metric

 Informally, time to solve a problem of size, n,
 Typically, problems become computationally T(n) is O(log n) if T(n) = c log2n
intensive as the input size grows.  Formally:
 We look at input sizes large enough to make • O( g(n) ) is the set of functions, f, such that
only the order of the growth of the running f(n) < c g(n)
time relevant for the analysis and for some constant, c > 0, and n > N (i.e.
comparison of algorithms. sufficiently large N)
f(n)
 Hence we are studying the asymptotic • Alternatively, we may write lim ≤ c
n→∞ g(n)
efficiency of algorithms.
and say g is an upper bound for f

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 15 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 16

 Big-oh Big-omega
 O(g) is the set of functions that grow no faster  Ω(g) is the set of
functions, f, such that
than g.
f(n) > c g(n)
 g(n) describes the
worst case behavior of for some constant, c,
and n > N
an algorithm that is
 Ω(g(n)): class of
O(g) functions f(n) that grow
 Examples: at least as fast as g(n)
 g(n) describes the best  Example:
n lg n + n = O(n2)
case behavior of an a n2 + b n + c = Ω(n)
lgk n = O(n) for all k ∈ N
algorithm that is Ω(g) provided a > 0

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 17 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 18

Big-theta Basic Asymptotic Efficiency classes

 Θ (g(n)): class of functions f(n) that grow at same rate as
g(n) 1 constant
 Example:
log n logarithmic
n2 / 2 - 3 n = Θ(n2)
We have to determine c1 > 0, n linear
c2 > 0, n0 ∈ N such that: n log n n log n
c2 n2 ≤ n2 / 2 - 3 n ≤ c1 n2 for
any n > n0
n2 quadratic
 Example: n3 cubic
Dividing by n2 yields:
c2 ≤ 1 / 2 – 3 / n ≤ c1 2n exponential
This is satisfied for
n! factorial
c2 = 1 / 14, c1 = 1 / 2, n0 = 7.

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 19 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 20

 A Simple Programming Model Theoretical analysis of time efficiency
 Prog = statement; Prog': Time(statement) + Time  Time efficiency is analyzed by determining
(Prog') the number of repetitions of the basic
 Prog = “var = expr”: 1 + Time(expr) operation as a function of input size
 Prog = “expr1 op expr2”: 1 + Time(expr1) + • Basic operation: the operation that contributes
Time(expr2) for primitive operations most towards the running time of the algorithm.
 Prog = “array[expr]'': 1 + Time(expr)
input size
 Prog = “for i = 1 to n, do Prog”: n times Time(Prog')
 Prog = function call: Time(function) T(n) ≈ copC(n)
 Prog = “while(expr), do Prog' ”: Time(expr) +
Time(Prog') times running time execution time Number of times
for basic operation basic operation is
executed

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 21 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 22

Time efficiency of nonrecursive algorithms Example (nonrecursive)

 Steps in mathematical analysis of

nonrecursive algorithms:
• Decide on parameter n indicating input size
• Identify algorithm’s basic operation
• Determine worst, average, and best case for input
of size n
• Set up summation for C(n) reflecting algorithm’s
loop structure
• Simplify summation using standard formulas
 Other issues regarding this algorithm implementation

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 23 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 24

 Time efficiency of recursive algorithms Example (recursive)

 Steps in mathematical analysis of recursive

algorithms:
• Decide on parameter n indicating input size
• Identify algorithm’s basic operation
• Determine worst, average, and best case for input
of size n
• Set up a recurrence relation and initial condition(s)
for C(n) − the number of times the basic operation
will be executed for an input of size n (alternatively
count recursive calls).
• Solve the recurrence to obtain a closed form or
estimate the order of magnitude of the solution

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 25 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 26

Example (recursive) Analysis Example (recursive) Analysis

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 27 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 28

 Stacks, queues, linked lists
 Stacks
• A stack is a container of StackElements that are
inserted and removed according to the last-in-first-
out (LIFO) principle.
• StackElements can be inserted at any time, but
only the last (the most-recently inserted)
StackElement can be removed.
• Inserting an item is known as “pushing” onto the
stack. “Popping” off the stack is synonymous with
removing an item.
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos
The Stack Abstract Data Type
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos
The Stack Abstract Data Type
 A stack is an abstract data type (ADT) that supports two main
operations:
• push(x): Inserts StackElement x onto top of stack
• Input: StackElement; Output: none
• pop(): Removes the top StackElement of stack and returns it; if
stack is empty an error occurs
• Input: none; Output: StackElement
 The following support operations should also be defined:
• size(): Returns the number of StackElements in stack
• Input: none; Output: integer
• isEmpty(): Return a boolean indicating if stack is empty.
• Input: none; Output: boolean
• top(): return the top StackElement of the stack, without removing
it; if the stack is empty an error occurs.
• Input: none; Output: StackElement
29 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 30
Stacks - Relevance
 Stacks appear in computer programs
• Key to call / return in functions & procedures
• Stack frame allows recursive calls
• Call: push stack frame
• Return: pop stack frame
 Stack frame
• Function arguments
• Return address
• Local variables
31 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 32
 Queues
 A queue differs from a stack in that its
insertion and removal routines follows the
first-in-first-out (FIFO) principle.
 Elements may be inserted at any time, but
only the element which has been in the
queue the longest may be removed.
 Elements are inserted at the rear
(enqueued) and removed from the front
(dequeued)
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos
The Queue Abstract Data Type
 Operations given for a
circular queue
DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos
The Queue Abstract Data Type
 The queue supports two fundamental operations:
• enqueue(o): Insert QueueElement o at the rear of the
queue
• Input: QueueElement; Output: none
• dequeue(): Remove the QueueElement from the front of the
queue and return it; an error occurs if the queue is empty
• Input: none; Output: QueueElement
 These support methods should also be defined:
• size(): Return the number of QueueElements in the queue
• Input: none; Output: integer
• isEmpty(): Return a boolean value that indicates whether
the queue is empty
• Input: none; Output: boolean
• front(): Return, but do not remove, the front QueueElement
in the queue; an error occurs if the queue is empty
• Input: none; Output: QueueElement
33 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 34
List example (doubly linked)

An empty list contains no elements

Insertions occur at the front of the list

Deletions done by key (e.g. deletion of element of key 18)
35 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 36
 The List Abstract Data Type The List ADT. Operations
 The list supports three fundamental operations:
• insert(x): Insert ListElement x at the front of the list
• Input: ListElement; Output: none
• delete(x): Remove the ListElement from the front of the list; an error occurs if
the list is empty
• Input: pointer to element to delete; Output: none
• search(k): search for ListElement of key k on list
• Input: key ot search; Output: pointer to ListElement or nil if not found
 These support methods should also be defined:
• size(): Return the number of ListElements in the lits
• Input: none; Output: integer
• isEmpty(): Return a boolean value that indicates whether the list is empty
• Input: none; Output: boolean
• first(): Return, but do not remove, the first ListElement in the list; an error
occurs if the queue is empty
• Input: none; Output: ListElement
• last(): Return, but do not remove, the last ListElement in the list; an error
occurs if the queue is empty
• Input: none; Output: ListElement
• prev(x), next(x): ListElement preceeding/succeeding ListElement x on list

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 37 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 38

The List ADT. Operations List examples with sentinels

An empty list always contains the sentinel (grey
cell)

Insertion is done at the front of the list

List with elements of keys 6,18, and 11 inserted

After insertion of 27

After deletion of 18

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 39 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 40

List ADT (w/ sentinels) Reading

 AHU, chapters 1 & 2

 CLR, chapters 2, 11.1-11.3
 Preiss, chapters: Algorithm Analysis.
Asymptotic Notation. Foundational Data
Structures. Data Types and Abstraction.
Stacks, Queues and Dequeues-4ups.
Ordered Lists and Sorted Lists.
 Knuth, vol. 1, 2.1, 2.2
 Notes

DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 41 DSA - Lecture 1- T.U.Cluj-Napoca - M. Joldos 42