3 views

Uploaded by vliviu

- 15_chapter6
- Daniel Lemire, A Better Alternative to Piecewise Linear Time Series Segmentation, SIAM Data Mining 2007, 2007.
- Aflv 01 Introduction
- m.tech r13 First Mid q
- Intro
- EEC 70 Quiz Solutions
- Computational Issues
- Pricing
- Lecture8 Dan Simon
- 08 Algorithmic Analysis
- Analysis
- There is No Software
- A Theory for Memory-Based Learning*
- OR_Theory With Examples
- cmput101.ch5.1.ppt
- Design of Algo
- Lecture04 Machine Programming 1 Basics
- ARM7100vA_2
- ADOH-TSK3000AInstructionSet-250513-0640-10392
- 256 Report

You are on page 1of 4

August 3, 2007

An algorithm is a sequence of computational steps which solves a well-dened computational problem. An algorithm receives input of the problem and produces as output the solution of the problem. For example, a sorting algorithm receives a set of numbers and outputs them in sorted order. An algorithm can be described in various ways. For example, an algorithm can be described in any of the programming languages. However, there is no reason to prefer one language over the other in describing the algorithm. Moreover, describing the algorithm in a specic programming language we might have to worry about strict syntactic rules of the language which might distract the reader from the main understanding of the algorithm. Due to all these reasons, it is better to describe an algorithm in English in a way which is very close to the any of these programming languages. This representation of the algorithm will be called its Pseudocode. In this course, we shall always describe an algorithm by writing its pseudocode. (In this chapter we shall describe two algorithms in pseudocode). A step in this pseudocode will usually correspond to an instruction in a high level programming languages. Designing efcient algorithms for various computational problems is one of the fundamental aim of compute science. The term efcient needs to be understood very clearly. We want to design the algorithms which can be executed in least amount of time when implemented on the computers. However, the real time is not a right measure of efciency due to the following reason. For a xed input, an algorithm which took days of running time on computer twenty years back would take a few minutes today and may be few seconds after few years. It therefore makes sense that the number of operations is more suitable measure of efciency of the algorithm. The aim is therefore to design an algorithm which takes least number of operations. Note that an operation in an algorithm would correspond to an instruction in high level language, and maps to a few instructions at assembly language. The number of operations performed by an algorithm to solve a given problem will depend upon the size of the input of the given instance of the problem. There is a difference between size and value of an input. Here one must understand that the size/length of the input is the number of bits required to store the input. We always assume that the input is given in binary format. For example an integer 1013 will need 10 bits in binary representation. In order to calculate time complexity of an algorithm, it is important to specify the underlying model of computation. There are many models of computation and they are equivalent in the sense that a problem which can be solved on one can also be solved by another. However, it is the cost for the basic computational steps where these models differ. In some model, a basic computational step (for example addition/multiplication) can be accomplished in one step whereas in other model it may take many steps depending upon the length of the operands for example. We shall rst describe two models of computation. 1. Logarithmic cost model : the cost of performing arithmetic/logical operation depends upon the size of the operand. For example, adding two numbers of n bits will take cn number of steps and multiplying two numbers using the simple add-and-shift method will take c n2 steps from some constant c > 0. 2. Uniform cost model : the cost of performing arithmetic/logical operation does not depend upon the size of the operand. So in this model, the time complexity of an algorithm is the count of the number of operations performed.

Now we shall describe the model of computation which is the most common model of computation for analyzing time complexity. This model is also very close to the real computers. RAM model : There are the following two features of this model. We shall soon provide justication for these features and show that RAM model is indeed close to the real computers. 1. The entire input resides in the RAM (random access memory). Each element of the input can be accessed in constant time from the RAM. 2. Let the input consists of n items, where each item could either be a basic data types or a combination of a constant number of these basic data types. If operands of an operation are of length c log n bits for a constant c, then the number of steps required for an operation is just constant. For example, sorting n numbers of log n bits each by selection sort will have time complexity bn2 for some constant b. Note : Throughout this course, whenever we have to analyze the running time of an algorithm in RAM model on an input of n items, we shall assume that each data item is of size c log n for some constant c. How realistic is the RAM model ? For most of the computational problems in real life, the input is stored in the internal memory (RAM), and of course the time required to access any data item from RAM is same irrespective of its location in the internal memory. Hence the rst feature of the RAM model is realistic. Let us now discuss the second feature. Before that, let us rst see how does a computer executes a program. A program in high level language is translated by compiler into machine language. In this translation, one instruction in high level language is translated to a constant number of machine language instructions. The CPU reads program which is a sequence of machine language instructions. A typical instruction involves performing some arithmetic/logical operation on its operands. The address of these operands is a binary number and is mentioned as a part of the instruction. The CPU executes this instruction by fetching the operands from the memory and then executing the corresponding operation. The result can further be written into some location specied in the instruction. In order to assign unique address to each location in the memory, it follows easily that if M is the size of the memory in bytes/words, then log2 M bits are necessary for specifying address of a data item stored in memory. In practice, it has always been the case that the length of the operands for arithmetic/logical operation supported by the architecture is at least as much as the number of bits to address a data element from memory. Consider for example, the most recent computer architecture. It has 2-4 GB of RAM, and it supports operation on 64 bits operands. For example, it can add two 64-bit numbers in one step. If input of an algorithm is n then based on the rst feature, it follows that memory size is large enough to accommodate the entire input, that is, n < 4 109 . Now log n = 32, and the architecture even supports operation with operands having length 64 (which is twice the value of log n). Thus the second feature of the RAM model, that an operation on operands with log n bits length takes single step is well supported by real computer as well. In short, the following guidelines should be followed when analyzing time complexity of an algorithm in RAM model. 1. In RAM model, when we analyze an algorithm on input of n basic data types, size of each item is assumed to be log n bits unless mentioned otherwise. And so, each operation can be viewed as an atomic operation (executed in single step) and time complexity analysis requires counting the number of operations. So in this sense, RAM model is uniform cost model. 2. To analyze the problems where the operand size is much more than log n bits, it is better to switch to logarithmic cost model for its time complexity. 3. Numerical computational problem are problems which involve arithmetic operations on numbers. In such applications, it is useful to get a bound on the maximum length of the operands which get generated during the algorithm. Sometimes operands generated during the algorithm may be of much 2

larger size than the input items(numbers). Gaussian elimination is such a numerical problem and its trivial implementation may generate numbers which may require n bits even when the initial matrix has entries with size log n at most. In any such situation, it is not suitable to use RAM model for the time complexity analysis of the algorithm. For algorithms for numerical problems, it is always desirable that to get an estimate of the maximum size of the numbers which may be generated during the algorithm, and if the maximum size does not exceed log n, then it is safe to just count the number of operation for time complexity of the algorithm in the RAM model.

Examples

Let us consider a few examples in order to better understand the two models we described above. We shall design three algorithms for one problem, and the problem is computing nth Fibonacci number. Fibonacci numbers are dened as follows. F (0) = 0, F (1) = 1, and F (i) = F (i 1) + f (i 2)for i > 1

Exercise 0.1 Show that F (i) > abi for some positive constants a, b with b > 1. It follows from the above exercise that the number of bits required for representing nth Fibonacci number is at least cn for some constant c. Algorithm 1 int recursiveb(i) { if i == 0 return 0 else if i == 1 return 1 else return (recursiveb(i-1) + recursiveb(i-2)) } Let T (n) be the number of steps taken to compute F (n) according to above algorithm. It follows that computing F (n) requires computing F (n 1) and F (n 2) recursively and separately, followed by constant number of steps. Hence T (n) > T (n 1) + T (n 2) + c. Comparing this inequality with the formula of Fibonacci number and using the exercise given above, it follows that T (n) is exponential in n. More specically it can be shown that the time complexity is exponential in terms of value of the number n in both the models. Algorithm 2 (iterative algorithm) int iterativeb(i) { if i==0 return 0 else if i ==1 return 1 else { b00, b11; 0; While( i) do { temp b0 + b1; b0 b1; b1 temp; +1 } return b1; } } 3

The above algorithm performs cn steps to compute nth Fibonacci number. So the time complexity in uniform cost model is linear in n and the time complexity in logarithmic cost model is quadratic in n (since addition of two numbers of k bits length takes k steps). Algorithm 3 is based on the following equation. F (i) F (i 1) Hence it follows that F (n) F (n 1) = 1 1 1 0 = 1 1 1 0 F (i 1) F (i 2)

n1

1 0

Although the above equation does not give us any improvement in running time, we can use the idea of repeated squaring to get better running time as follows. Let x be a number and we wish to compute xk . k1 Observe that xk = (xk/2 )2 if k is even and xk = x (x 2 )2 if k is odd. In this manner, it follows that we can compute xk using 2 log n multiplication operations. This approach can be used for computing the (n 1)th power of the 2 2 matrix. So in uniform cost model, the number of steps required will be c log n for some constant c. However, in the logarithmic cost model, the number of steps required to compute nth Fibonacci number will still be c n2 form some constant c . Do it as an exercise. You might have to use Exercise 0.1 and the fact that the number of steps required to multiply two k -bit numbers in logarithmic cost model by add-shift method is quadratic in k . Note : In some future class, we shall present an algorithm which can multiply two k -bit numbers in k log2 3 steps. This algorithm will lead to algorithm in logarithmic cost model which takes nlog2 3 steps to compute F (n).

- 15_chapter6Uploaded byKampa Lavanya
- Daniel Lemire, A Better Alternative to Piecewise Linear Time Series Segmentation, SIAM Data Mining 2007, 2007.Uploaded byDaniel Lemire
- Aflv 01 IntroductionUploaded byAbdelrahman Moharib
- m.tech r13 First Mid qUploaded byK Kumar
- IntroUploaded byHemesh Singh
- EEC 70 Quiz SolutionsUploaded byTiffany Lee
- Computational IssuesUploaded bypaminan
- PricingUploaded bysudhikushwah
- Lecture8 Dan SimonUploaded by11111
- 08 Algorithmic AnalysisUploaded bySaiful Islam
- AnalysisUploaded bySree Ps
- There is No SoftwareUploaded byDanae Kleida
- A Theory for Memory-Based Learning*Uploaded byDevika Kishnan
- OR_Theory With ExamplesUploaded bymaximin
- cmput101.ch5.1.pptUploaded byAnonymous 5ljloO
- Design of AlgoUploaded byAnkit Agarwal
- Lecture04 Machine Programming 1 BasicsUploaded bydoublefelix921
- ARM7100vA_2Uploaded byVijay Preetham
- ADOH-TSK3000AInstructionSet-250513-0640-10392Uploaded bynevdull
- 256 ReportUploaded byDaniela Alexandra Lungeanu
- cpm1Uploaded byVaggelis Petratos
- Gem 0001Uploaded bySilenthand
- Lec15Uploaded byKevin Rojas Bohorquez
- Mips Architecture OverviewUploaded bykumarguptav
- 70043FUploaded byJhoann Concha Canto
- dsp2Uploaded byMukul Aggarwal
- chapter 7 tocUploaded byYubraj Chaudhary
- ca_2 MarksUploaded bydevasenabalasubramanian92
- Algorithmics Performance Descriptors Unit3 Outcome3Uploaded byAnonymous LGB1O2fA
- a2 HandoutUploaded byethan werther

- BioinfNetSOM.pdfUploaded byvliviu
- Versiune Finala in Proceedings Id941Uploaded byvliviu
- Versiune Finala in Proceedings Id941Uploaded byvliviu
- Versiune Finala in Proceedings Id941Uploaded byvliviu
- Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in PathologyUploaded byvliviu
- Neural_Networks_approaches_for_children's_emotion_recognition_in_inteligent_learning_applications_887.pdfUploaded byvliviu
- Enabling Security Essentials for CIOsUploaded byvliviu
- Goda Morphological Determination of Pathological PCG Signals by Time and Frequency Domain AnalysisUploaded byvliviu
- Tannenbaum Topological Analysis of PAM50 and 21-Gene Breast Cancer AssaysUploaded byvliviu
- Curs 14 Programare ParalelaUploaded byvliviu
- Presentation Edulearn2015Uploaded byvliviu
- Presentation Edulearn2015Uploaded byvliviu
- wseas_TransOnSystemsUploaded byvliviu
- Regulament AppUploaded byvliviu
- wseas09_VladutuUploaded byvliviu
- Online Neural Network Training for Automatic Ischemia Episode DetectionUploaded byvliviu
- MBEC Final SubmissionUploaded byvliviu
- Java1.5 Con Currency IBM TutorialUploaded byakshayapandey
- BioinfNetSOMUploaded byvliviu
- nhess-10-2527-2010Uploaded byvliviu
- nhess-10-2527-2010Uploaded byvliviu
- 02-06reportUploaded byvliviu
- A Replacement for Voronoi Diagrams of Near Linear SizeUploaded byvliviu
- Submission EMBC1Uploaded byvliviu
- The Supervised Network Self-Organizing Map for Classification of Large Data SetsUploaded byvliviu
- ieee_tfsUploaded byvliviu
- Parallel vs DistributedUploaded byvliviu
- Adaptive Stepsize Alg 4online TrainingUploaded byvliviu
- Adaptive Stepsize Alg 4online TrainingUploaded byvliviu

- monitorUploaded byIndira Devi Pedaprolu
- Caspiche Discovery History PaperUploaded byXavier Toledo
- Lecture 2- Participants in the Construction ProjectUploaded byjbjuanzon
- 2013_Series_100_RMST_Car_10-19-12Uploaded byAnonymous Jk6qaXw
- Atmosferic CorrosionUploaded by241295mekan
- MIS of PTCL PakistanUploaded byArslan Rajput
- ISaGRAF(1)Uploaded byalibob78
- Highway Material Testing Laboratory Manual by Khanna and Justo PDFUploaded bySrinivas P
- CCNA Cyber Ops SECOPS 210-255 Official Cert GuideUploaded byDeusty Nkanga
- Minelab 4500 User ManualUploaded byslicktricker
- Pond ConstructionUploaded bymike_pilar
- iAXE624-BK_P0826_I_ENUploaded byMichael Karako
- Excel TipsUploaded bynvc_vishwanathan
- zf5hp24-BMW-AudiUploaded byPalomino Erick
- RM1xE1Uploaded bycirbo
- Turing MachinesUploaded bySahil Gupta
- Sap System ProfilesUploaded bypreethamsap82
- Formal Report- EliminationUploaded byLevy Medina Traya
- Asme Section Viii Div. 1 General Hugo Eng ModifieUploaded byshahmkamal
- ISE3004Uploaded bybd87gl
- 3D complex shape characterization by statistical analysisUploaded byStellina Zola
- GSM Multi-sector Solution Guide(GSM BSS Draft a)Uploaded byDavid Istur
- Cables - Guidelines for the Interpretation of DGA for Paper-Insulated Underground Transmission Cable SystemsUploaded bycadtil
- Packaging ReportUploaded byPuspita Sari
- ISecMg T1 to T8 Answers.pdfUploaded byGordon
- Solutions Ch16Uploaded byMihaela Gheorghe
- 7250SAS Alcatel SwitchUploaded byCarlos Galdamez
- 1_5024240515088908344Uploaded byNavvab Salehi
- IIT STS6 Questions SolutionsUploaded byPiyush Mishra
- AC FLIR E Series Infrared Camera ManualUploaded byaracelimccarthybod