Question: Let Ui , i = 1, 2, . . . be i.i.d random variables uniformly distributed between 0 and 1.

Let N (x) be the minimum number of such Ui that need to be added together for the sum to exceed x for some x > 0. What is the expected value f (x) Solution: Formally, we dene
n

E(N (x))?

N (x) = min{n :
i=1

Ui > x}

(1)

The more famous version of this problem merely asks you to nd the expectation of N (1). However, a slight extension of the approach can give the answer for a general x. We shall tackle this problem using recursive expectation which is a fairly natural line of attack for these kinds of problems. Before proceeding further, let me note that it is also possible to nd this expectation by actually computing the pmf of N (x) (at least when 0 < x < 1. I dont know it if the pmf is easy to compute for x > 1). For reasons that will become obvious shortly, let us for now restrict x to lie between 0 and 1. A recursion can be derived for f (x) by conditioning on the value of the rst random variable U1 .
1

f (x) =
0

E(N (x) | U1 = a)da

(2)

This integral naturally splits into two as

x 1

f (x) =
0

E(N (x)|U1 = a)da +

x

E(N (x)|U1 = a)da

(3)

If the value of U1 is greater than x (as is the case in the second integral), the sum has already exceeded x which implies that E(N (x)|U1 = a) = 1 in this case. If U1 < x, we get E(N (x)|U1 = a) = 1 + f (x a). This is the crucial step where recursion steps in. Substituting these back into the expression for f (x) above, we get f (x) = 1 +
o x

f (x a)da
x

(4)

which is the same as f (x) = 1 +

o

f (a)da

(5)

This integral equation can be solved by converting it into a dierential equation. Dierentiating yields, d f (x) = f (x) f or 0 x < 1 dx (6)

The solution in the familiar exponential function f (x) = Cex . It is easy to see that as x 0, f (x) 1 which implies that the constant of integration must be 1. Therefore, we get that f (x) = ex f or0 x < 1. Therefore, the expected number of uniform random variables one needs to add to cross a threshold of unity is simply e!!! Take a moment to reect on how cool this result is. Uniform random variables are everywhere around us and by simply adding them up till they cross a threshold, we can estimate one of the fundamental constants of all mathematics. This is similar to the Buons needle problem which one can use to estimate by simply throwing a needle onto a ruled sheet. It comes with a caveat though: The variance of N (x) is quite high

( 0.8) which means that, just as in the Buons needle problem, this is a highly inecient way of estimating the underlying constant. Let us forge on and try to compute f (x) for x > 1. A similar recursive equation can be derived by conditioning on U1 as f (x) =
0 1

E(N (x)|U1 = a)da

(7)

However, the dierence is that when x > 1, the integral doesnt split into two parts like it did in the case when x 1. Instead the only possible simplication is by replacing E(N (x)|U1 = a) by 1 + f (x a) as before and changing the variable of integration. This gives
x

f (x) = 1 +
x1

f (a)da

(8)

Dierentiating with respect to x gives the dierential equation d f (x) = f (x) f (x 1) dx (9)

I dont know if there is a easy way to solve this equation but I tackled it in the following manner. We already know the form of f (x) when 0 x < 1. So, when 1 x < 2, the above equation can be converted to a familiar linear rst order dierential equation by substituting f (x 1) = e(x1) . This results in the dierential equation d f (x) = f (x) ex1 1x<2 dx whose solution is (with the boundary condition that f (1) = e) f (x) = ex (x 1)ex1 1 x < 2. (10)

(11)

Armed with this, we can solve the dierential equation for 2 x < 3 and so on. For example, we have d f (x) = f (x) ex1 + (x 2)ex2 dx whose solution is f (x) = ex (x 1)ex1 + 2x<3 (12)

(x 2)2 x2 e 2 x < 3. (13) 2 The form of the solution for a general x can be easily guessed by working out the rst couple of terms in

the above manner. Once the form is guessed, it is easy to show that it indeed satises the above dierential equation and that the solution is unique. Cutting to the chase, the nal expression is
[x]

f (x) =
k=0

(1)k

(x k)k xk e k!

(14)

where [x] is the integer part of x. There is more to be said about f (x). As x , the round-o eects of the random variables Ui being bound between 0 and 1 disappear and we expect f (x) to grow linearly with x. More precisely, we expect every random variable Ui to be roughly half and so we need around 2x random variables to cross the threshold of x. Plotting f (x) in equation (14), we see that for large x, 2
f (x) 2x

1. A more interesting phenomenon is that

as x , we have f (x) 2x 2 . In other words, for large x, the expected number of uniform random 3
2 variables we need to cross the threshold of x is roughly 2x + 3 . This form is suciently simple that we can

ask ourself the following question(s): Is there a rationalization for this asymptotic form? Can it be guessed without much derivation? What if we replace the uniform distribution by some other distribution on R+ ? Can we still guess the asymptotic form of f (x)? I asked this question on MathOverow a while back and got two very nice explanations - the rst one is by Leonid and the second one is by Douglas Zare. Leonids explanation goes as follows: For very large x, if we look at the sum of Ui when we condition the sum to lie in [x, x + 1), we would expect it to be x + V where V is uniform in [0, 1). Lets now look at the average value of the sum the rst time it exceeds x. To get an idea of what this sum looks like on an average, we use the following observation: the rst time the sum exceeds x (say the sum equals x + V ), the last Ui that we added to the sum would have to be greater than the overshoot V . Thus, the expected value of the overshoot is the expected value of the minimum of two uniform random variables (V and Ui ) which is 1 . If the rst time the sum exceeds x, it equals x + 1 , then the number of variables it takes to reach that 3 3 sum is
x+ 1 3
1 2

= 2x + 2 . Before proceeding further, it is worth keeping in mind that this reasoning works only 3

for the case when Ui is uniformly distributed. In such a case, the sum x + V is also uniformly distributed in [x, x + 1) and we get the above reasoning. In the general case, x + V can have a complicated distribution and this reasoning will fail to provide any insights. Douglas Zares explanation is more generalizable to other distributions and involves a little more math. Let the random variables Ui be distributed according to the distribution () with support on R+ . Then the integral equation for the expectation f (x) becomes
x

f (x) = 1 +
0

f (x t)(t)dt

(15)

Let the mean of Ui be . Since f (x) satises a similar integral equation as above, we expect f (x) to be roughly equal to
x

plus a constant term. To derive this more formally, consider the Laplace transform of

the integral equation for f (x). Dene the Laplace transform as F (s) =
0

f (x)esx dx

(16)

With this, the integral equation becomes F (s) = or equivalently, 1 + F (s)A(s) s (17)

1 (18) s(1 A(s)) From this point, deriving the asymptotic behaviour of f (x) involves some complex analysis tools that I F (s) = am not familiar with. Instead, we shall cheat a bit and assume that the nal form of f (x) is linear. This assumption isnt completely rigorous mathematically but is justied by our knowledge of the problem. Then, we expect the function to look like f (x) ax + b and the corresponding Laplace transform as F (s) = Since A(s) is the Laplace transform of a probability distribution, it can be written as A(s) = 1 s1 + 3 s2 2 ... 2! (19)
as+b s2 .

where i is the ith moment of the distribution (). Substituting this into the Laplace transform of F (s), we get F (s) = 1 s1 s =
1 1 s2 1 s2 2 2! s2 21

... + ...
s2 21 2 s 22 1

(20) (21) (22) (23)

1 1 s2 1 s2

1+
1 1

Comparing this to the form that we guessed for F (s), we see that the linear approximation for f (x) that we are after is f (x) = x 2 + 2 + o(1) 1 21
1 2

(24) and 2 =
1 3

For the uniform random variable case that we started with, we have 1 = our earlier result.

and we recover

We next take the example of Ui being exponential random variables with parameter . In this case, we can guess the exact form of f (x) by purely logical reasoning. Since Ui are memoryless, the overshoot when the sum crosses x for the rst time is just the mean of Ui , viz.,
1 x+ 1

1 .

Therefore, f (x) must in this case equal

= x + 1. This is not an approximation but an exact solution to equation (15). For this distribution,
1

we have 1 =

and 2 =

2 2

and once again, we recover the correct result.

Other questions: 1. How do I derive the asymptotic form of f (x) from the exact solution in equation (14)? 2. What about the variance of N (x)? Recursive equations for variance estimation are more cumbersome but the reason I am interested is this: Computer simulations suggest that the variance of N (x) isnt quite monotone increasing in [0, 1). Is this a simulation artifact or is there a deeper reason? 3. Is there a way to guess that the constant term in the asymptotic form of f (x) is through the math?
2 22 1

without going