STOCHASTIC SIGNAL PROCESSING

Although modulation and demodulation are deterministic, the information to be

transmitted over a communication system, as well as the noise encountered in the phy-
sical transmission media, is random or stochastic. Although these phenomena cannot
be predicted in advance, they have certain predictable characteristics which can be
summarized in the random process model. The design of a digital communication
system heavily exploits these characteristics.
In this chapter we review the notation that will be used for random variables and
processes, and cover several topics in more detail that may be new to some readers
and are particularly important in the sequel. These include the Chernoff bounding
techniques, Bayes rule, and mixtures of discrete-time and continuous-time random
processes. Markov chains are discussed in section 3.3, and are used in a diverse set of
applications in chapters 7, 8, 10-12, and 17. Section 3.4, on Poisson processes, uses
the Markov chain results to explain Poisson processes and shot noise, which are
important to the understanding of optical fiber systems in chapters 5, 6, and 8.

3.1. RANDOM VARIABLES

Before we review the theory of the stochastic process, let us review some theory and
notation associated with random variables. In digital communication it is common to
encounter combinations of discrete and continuous-valued random variables, so this
will be emphasized.
E. A. Lee et al., Digital Communication
© Kluwer Academic Publishers, Boston 1988
SEC. 3.1 RANDOM VARIABLES 35

We denote a random variable by a capital letter, such as X , and we denote an outcome

of the random variable by a lower-case letter, such as x. The random variable is a real
or complex-valued function defined on the sample space n of all possible outcomes,
but we suppress this dependence in our notation. An event E is a set of possible out-
comes and is assigned a probability that we denote by Pr[E], where o:s; Pr[E]:S; 1.
Since an event is a set, we can define the union of two events, E 1UE 2 or the intersec-
tion of events E l(JE 2. The basic formula
Pr[E 1U E V = Pr[E tl + Pr[E V - Pr[E l(JE V (3.1)
leads to the very useful union bound,
Pr[E lUE V:s; Pr[E tl + Pr[E 2] . (3.2)
The cumulative distribution/unction (c.df.) is the probability of the event X :s; x,
Fx(x) = Pr[X :s;x] . (3.3)
Where there is no confusion, we often omit the subscript, writing the c.d.f. as F (x).
For a complex-valued random variable Y,
Fr(y) = Pr[Re{ Y } :s; Re{ y }, Im{ Y } :s; Im{ y }] . (3.4)
For a continuous-valued random variable, the probability density function (p.d!)
/ x (x) is defined such that for any interval I c R
Pr[X EI] =f /x(x)dx. (3.5)
I
For a complex-valued random variable, I is a region in the complex plane. For a
real-valued random variable X,

/x(x) = dxd Fx(x), (3.6)

where the derivative exists. We will often use the generalized derivative, so that when
the c.d.f. includes a step function the corresponding p.d.f. has a delta function.

Example 3-1.
For the c.d.f. shown below,

F
_1..j.8_
_(X_J
o 1
-+======~-->X
the p.d.f. consists exclusively of delta functions,
/ (x) = O.s-o(x) + O.s-O(x-l) , (3.7)
which is characteristic of a discrete random variable. 0
For a discrete-valued random variable X, we will denote the probability of an out-
come x Enas
Px(x) = Pr[X =x], (3.8)