13/11/2013&

::&

Performance*metrics*

What*are*we*trying*to*measure?*
Any&opera0on&on&any&data&structure&will&have&a&cost;&
e.g.&construc0ng,&adding&an&item,&delete,&copy,&nd,&
destructor&etc.&etc.&
Performance&is&oCen&described&as&fast&or&ecient&&
these&are&generally&useless!&
They&only&make&sense&if&they&are&part&of&a&comparison&/&
contrast,&so&faster&&&more&ecient&might&be&
helpful&

Which*cost?*

Would&we&want&to&measure&

The&cost&is&important&in&2&ways&
1.
2.

Some*(other)*variability*

0me&cost&(how&long?)&

Best&case&performance?&

Worst&case?&
Average&(whatever&average&might&mean)?&

space&cost&(how&much&space?)&

The&actual&cost&(in&0me&units&(Earth&seconds)&or&space&units)&
can&be&calculated&by&stepQcoun0ng,&but&never&is;&
what&we&want&is&a&more&general&descrip0on&of&the&
performance.&

The&main&(only?)&one&used&is&big&Oh&which&is&wriWen&&O(&)&
This&describes&the&upper&limit&on&cost.&

O(#)#
O(&)&says&nothing&about&the&actual&cost&of&using&a&given&
data&structure&/&algorithm&in&a&par0cular&situa0on&
(in&ms&or&bytes).&What&it&does&describe&is&the&way&
that&performance&is&related&to&the&size&of&the&data&
set&(what&data&can&vary?).!

Actual*O(#)*values*could*be*

Linear&&e.g.&the&BST&is&space&linear&in&the&number&
of&nodes;&&
double&the&number&of&elements&doubles&the&space&
cost&
halve&the&number&of&elements&halves&the&space&cost&
If&the&number&of&elements&is&n,&the&space&performance&is&
O(n)&

1&

13/11/2013&

or*

or*

constant&&e.g.&array&access&(read&&&write)&is&0me&
constant;&&

accessing&an&element&in&an&array&of&100&takes&the&
same&0me&as&accessing&one&in&an&array&of&100000&
(or&10000000000)&

&
If&the&number&of&elements&is&n,&we&say&the&0me&
performance&is&O(c)&or&O(1)&

What*about*
std::vector::push_back()&
std::vector::insert()&
std::list::insert()&
std::map::insert()&
::erase()&

logarithmic&&e.g.&the&number&of&comparisons&(&&so&the&0me&
cost)&needed&to&insert&an&element&into&a&tree&is&limited&by&the&
height/depth&of&the&tree;&for&a&wellQbalanced,&binary&tree&of&n&
nodes,&the&depth&is&log2(n)&

doubling&the&number&of&nodes&in&the&tree&adds&only&a&
constant&amount&to&the&search/inser0on&0me&(instead&of&
doubling&it&

OR&if&I&can&search&a&tree&of&n&nodes&in&0me&t&s,&in&0me&2t&
s,&I&can&search&a&tree&of&n2&nodes&
&
Binary&tree&inser0on&is&&O(log2&n)/

Consider*a*heapsort*
A&heapsort&will&have&&O(nlog2&n)&performance&&each&
element&inser0on&will&be&a&O(log2&n)&opera0on&and&
there&will&be&n&inser0ons,&so&the&total&cost&will&be&
O(nlog2&n)&&

&

The&output&phase&of&the&sort&will&be&also&be&O(nlog2&n),&
but&the&2x&in&2O(nlog2&n)&is&a&constant&factor&so&the&
overall&cost&remains&O(nlog2&n)&&

std::list(const&std::list&&)&//copy&constructor&

A&quicksort&is&O(nlog2&n)&&Q&is&it&beWer&than&a&heapsort?&

std::container::operator=&

&*getting*more*expensive**
Bubble&sort&&&exchange&sort&are&both&O(n2)&&this&is&
called&quadra3c./
This&means&doubling&the&size&of&the&data&set&will&
quadruple&the&execu0on&0me.&
quadra3c/algorithms&should&be&used&with&care&as&
they&get&v&expensive&quickly!&
Are&these&beWer&than&a&heapsort&/&quicksort?&
Surprising,&some0mes&yes&(n&<&&&&&&&&)&and&hybrid&
algorithms&will&use&quicksort&un0l&the&recursion&
reduces&the&data&set&to&~30&elements/

Other*polynomials*and*worse*
More&expensive&than&quadra0c&are&cubic&func0ons&
O(n3)&or&higher&powers&of&n&&these&higher&powers&
might&be&wriWen&O(nk)&&
Much&worse&than&polynomial&is&exponen3al/where&the&
order&is&2n&
Generally&polynomial&algorithms&can&be&used&in&
par0cular&situa0ons,&but&exponen0al&algorithms&are&
not&usually&computable,&except&for&very&small&n&

2&

13/11/2013&

Presenting*results**a*warning*

These&are&sta0s0cs,&as&in&lies,&damned&lies&.&

Excel&is&v.&much&doubleQedged&

v&slick&(professional)&

wizards&not&always&helpful&

Things*to*watch*

linear&can&be&good&(for&a&sort&if&you&could&manage&
it)&or&bad&(for&a&tree&lookup)&

simple&(?)&graphs&can&easily&mislead&

Excel&will&stretch&the&axes&to&ll&the&page,&rather&
than&using&xed&values&(by&default&at&least)&&this&
means&you&should&always&compare&on&the&same&
chart&

Log&axes&are&ne&if&you&are&sure&the&data&is&
logarithmic,&but&are&you&sure?&

(Great)&care&needed&when&interpre0ng&the&results&

O(log n)

n*log2 n

16
700000

14
600000

12
500000

400000

Series1

n log n

log n

10

n*log2 n

300000

200000

2
100000

0
0

5000

10000

15000

20000

25000

30000

35000

40000

5000

10000

15000

20000

25000

30000

35000

40000

700000

n squared

1600000000
600000
1400000000
500000

1200000000

400000

1000000000
n
n

n*log2 n

300000

800000000

n*n

600000000
200000
400000000
100000

200000000

0
0

5000

10000

15000

20000
n

25000

30000

35000

40000

5000

10000

15000

20000

25000

30000

35000

40000

3&

13/11/2013&

n cubed

n log n & n squared

6E+13

1600000000

1400000000
5E+13

1200000000
4E+13

n*n*n

1000000000
n*log2 n

800000000

n*n

3E+13

n*n*n

600000000
2E+13

400000000
1E+13

200000000

5000

10000

15000

20000

25000

30000

35000

40000

5000

10000

15000

20000

5E+13

4E+13

n*n

3E+13

n*n*n

2E+13

1E+13

0
5000

10000

15000

20000

30000

35000

40000

What*about*these?*

n sq & n cubed
6E+13

25000

25000

30000

35000

40000

t1

t2

t3

t4

1000

99.66

1210.00

100.00

1.49

2000

109.66

2420.00

400.00

3.29

3000

115.51

3630.00

900.00

5.20

4000

119.66

4840.00

1600.00

7.18

5000

122.88

6050.00

2500.00

9.22

6000

125.51

7260.00

3600.00

11.30

7000

127.73

8470.00

4900.00

13.41

8000

129.66

9680.00

6400.00

15.56

9000

131.36

10890.00

8100.00

17.73

10000

132.88

12100.00

10000.00

19.93

4&