Basic Probability Concepts

I. Basics

Random Variable
A random variable refers to any quantity with uncertain expected future values. For
example, time is not a random variable since we know that tomorrow will have 24 hours,
the month of January will have 31 days and so on. However, the expected rate of return
on a mutual fund and the expected standard deviation of those returns are random
variables. We attempt to forecast these random variables based on past history and on
our forecast for the economy and interest rates, but we cannot say for certain what the
variables will be in the future - all we have are forecasts or expectations.

Outcome
Outcome refers to any possible value that a random variable can take. For expected
rate of return, the range of outcomes naturally depends on the particular investment or
proposition. Lottery players have a near-certain probability of losing all of their
investment (-100% return), with a very small chance of becoming a multimillionaire
(+1,000,000% return - or higher!). Thus for a lottery ticket, there are usually just two
extreme outcomes. Mutual funds that invest primarily in blue chip stocks will involve a
much narrower series of outcomes and a distribution of possibilities around a specific
mean expectation. When a particular outcome or a series of outcomes are defined, it is
referred to as an event. If our goal for the blue chip mutual fund is to produce a
minimum 8% return every year on average, and we want to assess the chances that our
goal will not be met, our event is defined as average annual returns below 8%. We use
probability concepts to ask what the chances are that our event will take place.

Event
If a list of events ismutually exclusive, it means that only one of them can possibly take
place. Exhaustive events refer to the need to incorporate all potential outcomes in the
defined events. For return expectations, if we define our two events as annual returns
equal to or greater than 8% and annual returns equal to or less than 8%, these two
events would not meet the definition of mutually exclusive since a return of exactly 8%
falls into both categories. If our defined two events were annual returns less than 8%
and annual returns greater than 8%, we've covered all outcomes except for the
possibility of an 8% return; thus our events are not exhaustive.

The Defining Properties of Probability

Probability has two defining properties:

1. The probability of any event is a number between 0 and 1, or 0 < P(E) < 1. A P
followed by parentheses is the probability of (event E) occurring. Probabilities fall
on a scale between 0, or 0%, (impossible) and 1, or 100%, (certain). There is no
 such thing as a negative probability (less than impossible?) or a probability
greater than 1 (more certain than certain?).

2. The sum of all probabilities of all events equals 1, provided the events are both
mutually exclusive and exhaustive. If events are not mutually exclusive, the
probabilities would add up to a number greater than 1, and if they were not
exhaustive, the sum of probabilities would be less than 1. Thus, there is a need
to qualify this second property to ensure the events are properly defined
(mutually exclusive, exhaustive). On an exam question, if the probabilities in a
research study are added to a number besides 1, you might question whether
this principle has been met.

These terms refer to the particular approach an analyst has used to define the events
and make predictions on probabilities (i.e. the likelihood of each event occurring). How
exactly does the analyst arrive at these probabilities? What exactly are the numbers
based upon? The approach is empirical, subjective or a priori.

Empirical Probabilities
Empirical probabilities are objectively drawn from historical data. If we assembled a
return distribution based on the past 20 years of data, and then used that same
distribution to make forecasts, we have used an empirical approach. Of course, we
know that past performance does not guarantee future results, so a purely empirical
approach has its drawbacks.

Subjective Probabilities
Relationships must be stable for empirical probabilities to be accurate and for
investments and the economy, relationships change. Thus, subjective probabilities are
calculated; these draw upon experience and judgment to make forecasts or modify the
probabilities indicated from a purely empirical approach. Of course, subjective
probabilities are unique to the person making them and depend on his or her talents -
the investment world is filled with people making incorrect subjective judgments.

A Priori Probabilities
A priori probabilities represent probabilities that are objective and based on deduction
and reasoning about a particular case. For example, if we forecast that a company is
70% likely to win a bid on a contract (based on an either empirical or subjective
approach), and we know this firm has just one business competitor, then we can also
make an a priori forecast that there is a 30% probability that the bid will go to the
competitor.

e empirical, subjective and a priori probabilities listed above.

Stating the Probability of an Event as Odds "For" or "Against"

Given a probability P(E),

Odds "FOR" E = P(E)/[1 - P(E)] A probability of 20% would be "1 to 4".

Odds "AGAINST" E = [1 - P(E)]/P(E) A probability of 20% would be "4 to 1".

Example:
Take an example of two financial-services companies, whose publicly traded share
prices reflect a certain probability that interest rates will fall. Both firms will receive an
equal benefit from lower interest rates. However, an analyst's research reveals that Firm
A's shares, at current prices, reflect a 75% likelihood that rates will fall, and Firm B's
shares only suggest a 40% chance of lower rates. The analyst has discovered (in
probability terms) mutually inconsistent probabilities. In other words, rates can't be
(simultaneously) both 75% and 40% likely to fall. If the true probability of lower rates is
75% (i.e. the market has fairly priced this probability into Firm A's shares), then as
investors we could profit by buying Firm B's undervalued shares. If the true probability is
40%, we could profit by short selling Firm A's overpriced shares. By taking both actions
(in a classic pairs arbitrage trade), we would theoretically profit no matter the actual
probability since one stock or the other eventually has to move. Many investment
decisions are made based on an analyst's perception of mutually inconsistent
probabilities.
Unconditional Probability
Unconditional probability is the straightforward answer to this question: what is the
probability of this one event occurring? In probability notation, the unconditional
probability of event A is P(A), which asks, what is the probability of event A? If we
believe that a stock is 70% likely to return 15% in the next year, then P(A) = 0.7, which
is that event's unconditional probability.

Conditional Probability
Conditional probability answers this question: what is the probability of this one event
occurring, given that another event has already taken place? A conditional probability
has the notation P(A | B), which represents the probability of event A, given B. If we
believe that a stock is 70% likely to return 15% in the next year, as long as GDP growth
is at least 3%, then we have made our prediction conditional on a second event (GDP
growth). In other words, event A is the stock will rise 15% in the next year; event B is
GDP growth is at least 3%; and our conditional probability is P(A | B) = 0.9.
Exponential distribution
How much time will elapse before an earthquake occurs in a given region? How long do we need to wait

until a customer enters our shop? How long will it take before a call center receives the next phone call?

How long will a piece of machinery work without breaking down?

Questions such as these are frequently answered in probabilistic terms by using the exponential

distribution.

All these questions concern the time we need to wait before a given event occurs. If this waiting time is

unknown, it is often appropriate to think of it as a random variable having an exponential distribution.

Roughly speaking, the time we need to wait before an event occurs has an exponential distribution if the

probability that the event occurs during a certain time interval is proportional to the length of that time

interval.

More precisely, has an exponential distribution if the conditional probability is approximately

proportional to the length of the time interval comprised between the times and , for any time instant .

In many practical situations this property is very realistic. This is the reason why the exponential

distribution is so widely used to model waiting times.

The exponential distribution is strictly related to the Poisson distribution. If 1) an event can occur more

than once and 2) the time elapsed between two successive occurrences is exponentially distributed and

independent of previous occurrences, then the number of occurrences of the event within a given unit of

time has a Poisson distribution. We invite the reader to see the lecture on the Poisson distribution for a

more detailed explanation and an intuitive graphical representation of this fact.

Definition
The exponential distribution is characterized as follows.
Definition Let be a continuous random variable. Let its support be the set of positive real numbers:

Let . We say that has an exponential distribution with parameter if and only if itsprobability

density function isThe parameter is called rate parameter.

A random variable having an exponential distribution is also called an exponential random variable.

The following is a proof that is a legitimate probability density function.

Proof

To better understand the exponential distribution, you can have a look at its density plots.

The rate parameter and its interpretation

We have mentioned that the probability that the event occurs between two dates and is proportional

to (conditional on the information that it has not occurred before ). The rate parameter is the constant of

proportionality:where is an infinitesimal of higher order than (i.e. a function of that goes to zero more

quickly than does).

The above proportionality condition is also sufficient to completely characterize the exponential

distribution.

Proposition The proportionality conditionis satisfied only if has an exponential distribution.

Proof

Expected value
The expected value of an exponential random variable is
Proof

Variance
The variance of an exponential random variable is
Proof

Moment generating function

The moment generating function of an exponential random variable is defined for any :
Proof

Characteristic function
The characteristic function of an exponential random variable is
Proof
Distribution function
The distribution function of an exponential random variable is
Proof

More details
In the following subsections you can find more details about the exponential distribution.

Memoryless property
One of the most important properties of the exponential distribution is the memoryless property:for

any .
Proof

is the time we need to wait before a certain event occurs. The above property says that the probability

that the event happens during a time interval of length is independent of how much time has already

elapsed () without the event happening.

The sum of exponential random variables is a Gamma random

variable
Suppose , , ..., are mutually independent random variables having exponential distribution with

parameter .

Define

Then, the sum is a Gamma random variable with parameters and .

Proof

The random variable is also sometimes said to have an Erlang distribution. The Erlang distribution is

just a special case of the Gamma distribution: a Gamma random variable is also an Erlang random

variable when it can be written as a sum of exponential random variables.

Density plot
The next plot shows how the density of the exponential distribution changes by changing the rate

parameter:

 the first graph (red line) is the probability density function of an exponential random variable with

rate parameter ;
  the second graph (blue line) is the probability density function of an exponential random variable

with rate parameter .

The thin vertical lines indicate the means of the two distributions. Note that, by increasing the rate

parameter, we decrease the mean of the distribution from to .