Solving Recurrence Relations A recurrence relation is a function that tells how to find the n th element of a sequence in terms of one

or more of its predecessor. By solving a recurrence relation we mean that we will find a Closed form solution--a function that can be used to compute an without knowledge of an-1, an-2, etc. Homogeneous Linear Recurrences The easiest type of recurrence to solve is the linear homogeneous recurrence with constant coefficients. In other words, the recurrence does not have any of the following forms: an = an-1 an-2, an = an-1 + 5 or an = n an-1. Example 1: an = 2 an-1 + 3 an-2 and a0 = a1 = 1 Solution by use of the characteristic equation: 1. Substitute rn for an (rn-1 for an-1, etc.) and simplify the result. For this example the characteristic equation is rn = 2rn-1 + 3rn-2 which simplifies to: r2 = 2r + 3 2. Find the roots of the characteristic equation: r2 - 2r - 3 = 0 factors as (r - 3)(r + 1) giving roots r1 = 3 and r2 = -1 When there are two distinct roots, the general form of the solution is: a n = 1r1n + 2r2n where 1 and 2 are constants. In this case, we have: an = 13n + 2(-1)n 3. Using the initial conditions we can find the constants 1 and 2 a0 = 1 = 1 + 2 a1 = 1 = 31 - 2 so 1 = 1/2 and 2 = 1/2 and the final solution is an = (1/2)3n + (1/2)(-1)n.

If a characteristic equation has equal roots, (i.e. r 1 =r2), then the general solution has the form: an = 1rn + 2nrn Example 2: an = 2an-2 an-4 with a0 = 0, a1 = -2, a2 = 4 and a3 = -4. In this case, we have a degree four linear homogeneous recurrence whose characteristic equation is r4 = 2 r2 - 1 or r4 -2 r2 + 1 = 0. This factors as: (r2 - 1)( r2 - 1) = 0 so the roots are r1 = r2 = 1 and r3 = r4 = -1. (The order of the roots is arbitrary.) Note there are two pairs of equal roots so there will be two terms each with n as a factor. The form of the general solution is: an = 1r1n + 2nr2n + 3r3n + 4nr4n Setting up the equations we have: a0 = 0 = 1 + 3 a1 = -2 = 1 + 2 - 3 - 4 a2 = 4 = 1 + 22 + 3 + 24

a3 = -4 = 1 +32 - 3 - 34 so 1 = -1/2, 2 = 1/2, 3 = 1/2 and 4 = 3/2 and the final solution is: an = (-1/2)1n + (1/2)n1n + (1/2) (-1)n + (3/2)n(-1)n = -1/2 + n/2 + (1/2)(-1)n + (3/2)n(-1)n

Other Solution Methods Example 3: Draw n straight lines on a paper so that every pair of lines intersect, but no three lines intersect at a common point. Determine how many regions the plane is divided into if n lines are used. (Draw yourself some pictures.) If n = 0 then there is 1 region. If n = 1 there are 2 regions. If n = 2 there are 4 regions. If n = 3 there are 7 regions. In general, the nth line intersects n -1 others and each intersection subdivides a region, so the number of regions that are subdivided by the n th line is: 1 before the first line, 1 after the last line, and n - 2 regions between the n -1 lines. This gives us the following recurrence relation: an = an-1 + n This can be solved iteratively by backward substitution. Specifically, = an-2 + (n - 1) + n = an-3 + (n - 2) + (n - 1) + n n = a0 + 1 + 2 + ... + (n - 2) + (n - 1) + n = a0 + i = 1 + n(n + 1)/2
i=1

Divide and Conquer Recurrence Relations Example 4: Find the largest and smallest numbers in a list. The following approach is used: divide the list in half, find the largest and smallest numbers in each half, and compare the large values to each other and the small values to each other. The measure of complexity is the number of comparisons of list elements. The base case is when a list has length 2. In this case, it takes one comparison to determine which is the larger number. In general, for this approach, the amount of work required is the work needed to solve each of the sublists plus the amount of work needed to put the solutions from the sublists together. Here, each sublist is half as large as the original, and it requires two comparisons to determine the largest and smallest values in the combined list. This leads to the recurrence relation below. Each time, the underlined term is replaced by the quantity in brackets in the line below. T(n) = 2T(n/2) + 2 with the base case T(2) = 1. Assuming n = 2k, we have 2 2 2 T(n) = 2[2T(n/4) + 2] + 2 = 2 T(n/2 ) + 2 + 2 = 22[2T(n/8) + 2] + 22 + 2 = 23T(n/23) + 23 + 22 + 2 = 2k-1T(n/2k-1) + 2k-1 + 2k-2 + 23 + 22 + 2 = 2k-1T(2) + 2k-1 + 2k-2 + 23 + 22 + 2
k-1

=2

k-1

+ 2i = 2k-1 + 2k - 2
i=1

= 2k + 2k-1 - 2 = 2k + 2k/2 - 2

= (3/2) 2k - 2 = (3/2) n - 2 since n = 2k. One of the most common recurrences is a geometric progression in which each term is obtained from its predecessor by multiplying by a constant. For example, 5, 15, 45, ... is described by the recurrence relation an = 3 an-1. Various methods (including backward substitution) can be used to obtain the closed form solution which is an = 53n. Example 5: Consider the relation an2 = 5(an-1) 2 where an > 0 and a0 = 2. Make the following change of variable: let b n = an2. Then, b0 = 4 and bn = 5 bn-1. n Since this is a geometric series, its solution is bn = 45 . Now substituting back, an = bn = 25n for n 0. Inhomogeneous Recurrence Relations We now turn to inhomogeneous recurrence relations and look at two distinct methods of solving them. These recurrence relations in general have the form a n = cn-1an-1 + g(n) where g(n) is a function of n. (g(n) could be a constant) One of the first examples of these recurrences usually encountered is the tower of Hanoi recurrence relation: H(n) = 2H(n-1) + 1 which has as its solution H(n) = 2 n 1. One way to solve this is by use of backward substitution as above. Well see another method shortly. Example 6: Consider the sequence: 0, 2, 6, 12, 20, 30, 42, ... . Notice how the difference between successive terms increases: a1 a 0 = 2 a2 a 1 = 4 a3 a 2 = 6 an an-1 = 2n Now, add all the equations above to get: an a0 = 2 + 4 + 6 + ... + 2n = 2[1 + 2 + + n] = 2[n(n+1)/2].

Since a0 = 0, we have an = n(n+1).

Generalizing, this is a first-order inhomogeneous recurrence of the form a n = cn-1an-1 + g(n) for n 1.
n

When c = 1, its general solution is an = a0 + g(i). To see this, notice the right hand side of each
i=1

equation in example 6 above can be replaced by the appropriate function value. That is, a1 a0 = g(1) a2 a1 = g(2) a3 a2 = g(3)

an an-1 = g(n) Then, summing these equations gives the formula above. We could also use backward substitution to obtain the same result. While there is no one method of solution that is always guaranteed to work, here's a basic approach that frequently does. We consider inhomogeneous recurrences of degree at most two: an = c1 an-1 + c2 an-2 + g(n) where g(n) is a function of n. The techniques below apply to recurrences in which g(n) is a polynomial or a constant. Two steps are needed to obtain a solution: 1) first find a solution for the homogeneous part (i.e. , a n = c1 an-1 + c2 an-2) and then 2) find one particular solution of the recurrence and combine the two solutions. As above, solutions of the homogeneous part will be of the form f(n) = a n = 1r1n + 2r2n. The general forms of the particular solutions are described below. Once a form is found, it is substituted into the recurrence in order to solve for the constants. Call this particular solution p(n). Then, the solution for the original recurrence is a n = f(n) + p(n). All that remains is to use one or more of the initial conditions to solve for the remaining constants. Looking again at the first-order inhomogeneous recurrence, a n = can-1 + g(n), the solution to the homogeneous part (an = can-1) is an = cn. In finding a particular solution p(n), there are several cases to consider. These are summarized in the table below. Observe that if g(n) is a polynomial, then p(n) is a polynomial of the same degree.

g(n) d, a constant dn dn d
n 2

p(n) 1n + 0 2n + 1n + 0 d
n 2

(See Tucker, Alan, Applied Combinatorics, 2nd ed., p. 286) Example 7: Consider the Tower of Hanoi recurrence again: H(n) = 2H(n-1) + 1 with H(1) = 1. The solution to the homogeneous part is 2n. Since g(n) = 1 is a constant, the particular solution has the form p(n) = . Substituting into the original recurrence to solve for we get: = 2 +1 so that = -1. Thus, p(n) = -1. This means that H(n) = 2n 1. Using the fact that H(1) = 1, gives 1 = 2 1 so = 1. Thus, the solution to the original recurrence is H(n) = 2 n 1. (Note that alternative ways to get this solution are backward substitution and the method shown below.) Example 8: an = 2an-1 + n2 with a1 = 3. The homogeneous part an = 2an-1 has a root of 2, so f(n) = 2n. The particular solution should have the form an = 2n2 + 1n + 0. Substituting this into the original recurrence gives 2n2 + 1n + 0 = 2[2(n-1)2 + 1(n-1) + 0] + n2. Expanding and combining like terms gives us

0 = n2(2 + 1) + n(-42 + 1) + (22 - 21 + 0). Notice that each parenthesized expression must evaluate to 0 since the left hand side of the equation is 0. This gives us the following values for the constants: 2 = -1, 1 = -4 and 0 = -6. Thus, p(n) = -n2 -4n - 6. Then we have an = 2n -n2 -4n - 6. This gives us = 7 and the final solution is an = 72n -n2 - 4n - 6. If g(n) consists of two or more terms, then each term is handled separately using the table above. For example, solving an = 3 an-1 + 6n 2n, requires finding particular solutions for two recurrences an = 3 an-1 + 6n and an = 3 an-1 2n. There is one case in which the particular solutions above will not work. This happens when the particular solution is a solution of the homogeneous recurrence. Then, the particular solution that should be tried is a higher degree polynomial. For example. if g(n) = d n and d is a solution of the homogeneous part then try ndn as the particular solution. Example 9: an = 3an-1 + 2n with a1 = 5. The homogeneous part a n = 3an-1 has a root of 3, so f(n) = 3n. The particular solution should have the form an = 2n. Substituting this into the original recurrence gives 2n = 3 2n-1 + 2n. Solving this equation for gives us the particular solution p(n) = -2 n+1. Thus the solution has the form an = 3n - 2n+1. Using the initial condition that a1 = 5 we get 5 = 3 - 4 which gives = 3. Thus, the final solution is an = 3n+1 - 2n+1. Another method of solving inhomogeneous recurrence relations It is sometimes possible to convert an inhomogeneous recurrence relation into a homogeneous one. Although this method does not widely appear in the literature references to it can be found at the following URLs: http://www.answers.com/topic/recurrence-relation and http://mathcircle.berkeley.edu/BMC3/Bjorn1/node10.html. Lets consider the Tower of Hanoi recurrence again: H(n) = 2H(n-1) + 1 or H(n) 2H(n-1) = 1. Substitute n 1 for n in this equation to get H(n-1) 2 H(n-2) = 1. Notice what happens if we subtract the second equation from the first: H(n) 2H(n-1) H(n-1) +2H(n-2) = 1 1, or H(n) 3H(n-1) + 2H(n-2) = 0, a homogeneous linear recurrence relation. The characteristic equation x2 3x + 2 = 0 factors as (x-2)(x-1) = 0 so the general form of the solution is H(n) = 1 2n + 21n = 1 2n + 2. Using the initial conditions to obtain the constants we find that 1 = 1 and 2 = -1 and thus we have H(n) = 2 n 1 as we found before. Example 10: an an-1 = 2n with a0 = 0 (Example 6) Using the same method as above we find that a n-1 - an-2 = 2(n-1). Now, subtracting we get an an-1 = 2n -an-1 + an-2 = -2(n-1) an -2an-1 + an-2 = 2 This is not yet a homogeneous recurrence but one more application of the method will give us a linear homogeneous recurrence relation. an -2an-1 + an-2 =2 -an-1 + 2an-2 an-3 = -2 an -3an-1 +3an-2 an-3 = 0 has characteristic equation x3 3x2 + 3x - 1 = 0 that factors as (x 1)3.

The general form of the answer must be a n = 11n +2n1n + 3n21n or 1 + 2n + 3n2. Since a0=0 we immediately get that 1 = 0. a1 = 2 and a2 = 6 giving us 2 = 2 + 3 and 6 = 22 + 4 3. Solving for 2 and 3 we get 2 = 1 and 3 = 1 so the solution to the recurrence is an = n + n2 or n(n + 1). The same method can be used to solve recurrence relations in which g(n) is a higher order polynomial. The substitute n 1 for n and subtract may have to be used several times, but eventually you will obtain a linear homogeneous recurrence relation. (However, factoring the characteristic equation could turn into a challenge.) A similar method can be used if the function g(n) is an exponential. Example 11: an = 3an-1 + 2n with a1 = 5. (Example 9) Rewriting the recurrence relation gives us an - 3an-1 = 2n. First we multiply by 2 to get 2an - 6an-1 = 2n+1. Now substitute n 1 for n in this new equation to get 2an-1 - 6an-2 = 2n. Now, subtracting this new recurrence from the original one we get: an - 3an-1 = 2n -2an-1 + 6an-2 = -2n an 5an-1 + 6an-2 = 0. The characteristic equation is x 2 5x + 6 = 0 which factors as (x - 3)(x 2) = 0, so the answer has the form an = 13n + 22n. Using the facts that a1 = 5 and a2 = 19 we get 5 = 31 + 2 2 and 19 = 91 + 42. Solving for the constants we get 1 = 3 and 2 = -2 giving us the solution an = 33n + (-2)2n = 3n+1 2n+1.