CSE 2320 Lab Assignment 2 - ABET Outcome B (Experimentation)

Assessment Project
Due: Tuesday May 3, 2011.
Goal:
Demonstration of the ability to design and conduct experiments, analyze and interpret
data (ABE outcome b! based on the follo"ing narrati#e$
he code and the report should be turned in both electronically (using the mechanism
described abo#e! and in hardcopy. he hardcopy report should also include a copy of the
code. he hardcopy code and report should be turned in at the beginning of class.
he ob%ecti#e of this assignment is to study ho" the theoretical analysis of a #ariety of
sorting algorithms compares "ith their actual experimentally measured performance. he
sorting algorithms you "ill study are$
(&! 'nsertion sort
((! )eap sort
(*! +erge sort
o (*a! top,do"n
o (*b! bottom,up
(-! .uic/ sort "ith different pi#ot choices$
o (-a! he last element in the se0uence
o (-b! he median of the first, middle, and last elements in the se0uence
o (-c! A random element in the se0uence
he ma%or emphasis of this assignment "ill be to analyze the performance of the
algorithms (not on coding the algorithms!.
Coding Portion (50 points).
1or each of the algorithms, determine the best, a#erage, and "orst,case inputs.
2rite a program to time these algorithms gi#en their best, a#erage and "orst,case
inputs. 3ou should also determine the input sizes and the number of times you
"ill repeat each experiment.
45E6$
he code of the sorting algorithms is gi#en in http$77algs-.cs.princeton.edu7code7 .
3ou can re,use the classes "hich implement the abo#e algorithms. 3ou "ill need
to ma/e a fe" modifications to test out different strategies.
+ost of your time should be spent on designing careful test cases and analyzing
your results in order to dra" conclusions regarding the performance of the #arious
algorithms and to study ho" the experimental performance is correlated, or not,
"ith the theoretical performance.
3ou should be able to generate random numbers, "hich "ill be the inputs to the
algorithms.
3ou ha#e to thin/ ho" to measure time in a "ay that could be as ob%ecti#e as
possible and gi#e the necessary information to the reader, so that the reproduction
of the experiment could be possible.
3ou ha#e to determine the input size in your experiment by obser#ing the
expected cost for each algorithm and considering that the experimentally
measured cost differences ha#e to become ob#ious.
he code should follo" all con#entions mentioned in the syllabus and "ill be
graded accordingly.
he code has to be demonstrated to the 8A, other"ise the assignment "ill not
be graded.
Report (50 points). 3ou "ill "rite a brief report that includes theoretical analysis, a
description of your experiments, and discussion of your results. +inimum re0uirements
for your report include$
'ntroduction. Describe the ob%ecti#e of this assignment. [3
heoretical analysis. 8i#e the best, a#erage and the "orst,case inputs of the
sorting algorithms and briefly analyze their complexity. [!
Description of your experiment setup, "hich includes but is not limited to [10
o +achine specification.
o 2hat is your timing mechanism9
o )o" did you generate the best, a#erage and the "orst,case inputs for the
algorithms9
o 2hat input sizes did you test9 2hy9
o )o" many times did you repeat each experiment9
Experimental results. [15
o 8raph the a#erage case, best case and "orst case running time #ersus the
input size for all the sorting algorithms.
Discussion of your results, "hich includes but is not limited to [15$
o :omparison of different #ersions of 0uic/sort
o :omparison of different #ersions of mergesort
o 2hich of all sorts seems to perform the best (consider the best #ersion of
0uic/ sort!9
o o "hat extent do the best, a#erage and "orst case analyses (from
class7textboo/! of each sort agree "ith the experimental results9 'f not,
"hat might be the explanation9
o :onsideration of errors and discussion of possible additional "or/.
45E$ 5ne "ay to estimate asymptotic complexity by your experimental results could
be to graph the running time "ith a doubly logarithmic (log,log! plot; that is, to plot log
(n! #ersus log n. hen use linear regression (the method of least s0uares! to fit the
doubly logarithmic cur#e "ith a line and find the slope / and the intercept c of the line.
'nput sizes that are doubled each time could be useful, for example, &<<, (<<, -<<, =<<,
&><< and etc. hey gi#e e0ual inter#als to e#ery t"o ad%acent points of the plot.
"rading Ru#ri$:
:&$ .uestions and related bac/ground sho" that student clearly understands the issues to
examine.
?$ 6ummary indicates that student understands the issues and "hat should be explored.
*$ 6ummary indicates that student should learn something from their experiments.
&$ 4ot clear that student "ill be performing an organized experiment.
:($ 6tudent has decomposed the problem into one or more experimentally testable
hypotheses.
?$ )ypotheses indicate that student performed initial "or/ leading to testable hypotheses
for the gi#en problem.
*$ )ypotheses indicate that initial "or/ "as hastily performed.
&$ )ypotheses are fla"ed and may not be testable.
:*$ he components (e.g. code for (*(<! ha#e been implemented, tested, and could be
used by others.
?$ :omponents are designed to be useful for anyone performing related experiments.
*$ :omponents are insufficient for performing necessary experiments.
&$ :omponents are incorrect.
:-$ he nature of the collected data is accurately described.
?$ .uantity of data is appropriate, but not o#er"helming, for dra"ing conclusions.
*$ .uantity of data is barely sufficient for dra"ing conclusions.
&$ Experimental set,up precludes obtaining data.
:?$ Data is presented appropriately in tables or graphical forms.
?$ rends in the data are clearly identifiable.
*$ rends appear, but are not ob#ious.
&$ Data is #ery incomplete.
:>$ :onclusion(s! regarding hypotheses are presented "ith support from the data.
@ossible errors are noted. Aemaining issues are discussed, along "ith additional
experimental "or/ that could be performed.
?$ :onclusions for all hypotheses, "ith support from data.
*$ :onclusions are related to hypotheses, but 0uestionable support from data.
&$ :onclusions related to hypotheses are lac/ing.