Chapter 7

Generating and Processing

Random Signals

第一組
電機四 B93902016 蔡馭理
資工四 B93902076 林宜鴻

1
Outline

Stationary and Ergodic Process

Uniform Random Number Generator
Mapping Uniform RVs to an Arbitrary pdf
Generating Uncorrelated Gaussian RV
Generating correlated Gaussian RV
PN Sequence Generators
Signal processing

2
Random Number Generator

Noise, interference
Random Number Generator-
computational or physical device designed
to generate a sequence of numbers or
symbols that lack any pattern, i.e. appear
random, pseudo-random sequence
MATLAB - rand(m,n) , randn(m,n)

3
Stationary and Ergodic Process
 strict-sense stationary (SSS)
 wide-sense stationary (WSS)
Gaussian

 SSS =>WSS ; WSS=>SSS

 Time average v.s ensemble average
 The ergodicity requirement is that the ensemble
average coincide with the time average
 Sample function generated to represent signals,
noise, interference should be ergodic
4
Time average v.s ensemble average

 Time average  ensemble average

5
Example 7.1 (N=100)
x(t , ξ i )  A cos(2πft  φi )

y nsemble-avarag(t) x nsemble-avarage(t)
1 0.5
x(t)

0 0

-1 -0.5

e
0 0.5 1 1.5 2 0 0.5 1 1.5 2

1 1
y(t)

0 0

-1 -1

e
0 0.5 1 1.5 2 0 0.5 1 1.5 2
z nsemble-avarag(t)
2 2
z(t)

0 0

-2 -2
e

0 0.5 1 1.5 2 0 0.5 1 1.5 2

x(t , ξ i )  A(1  μi ) cos(2πft  φi ) 6
Uniform Random Number Genrator

Generate a random variable that is

uniformly distributed on the interval (0,1)
Generate a sequence of numbers (integer)
between 0 and M and the divide each
element of the sequence by M
The most common technique is linear
congruence genrator (LCG)

7
Linear Congruence

LCG is defined by the operation:

xi+1=[axi+c]mod(m)
x0 is seed number of the generator
a, c, m, x0 are integer
Desirable property- full period

8
Technique A: The Mixed Congruence
Algorithm
The mixed linear algorithm takes the form:
xi+1=[axi+c]mod(m)
- c≠0 and relative prime to m
- a-1 is a multiple of p, where p is the
prime factors of m
- a-1 is a multiple of 4 if m is a
multiple of 4

9
Example 7.4
m=5000=(23)(54)
c=(33)(72)=1323
a-1=k1‧2 or k2‧5 or 4‧k3
so, a-1=4‧2‧5‧k =40k
With k=6, we have a=241
xi+1=[241xi+ 1323]mod(5000)
We can verify the period is 5000, so it’s full
period
10
Technique B: The Multiplication Algorithm
With Prime Modulus
The multiplicative generator defined as :
xi+1=[axi]mod(m)
- m is prime (usaually large)
- a is a primitive element mod(m)
am-1/m = k =interger
ai-1/m ≠ k, i=1, 2, 3,…, m-2

11
Technique C: The Multiplication Algorithm
With Nonprime Modulus
The most important case of this generator
having m equal to a power of two :
xi+1=[axi]mod(2n)
The maximum period is 2n/4= 2n-2
the period is achieved if
- The multiplier a is 3 or 5
- The seed x0 is odd

12
Example of Multiplication Algorithm With
Nonprime Modulus
11

10

9 a=3
8

7
c=0
6

5
m=16
4
x0=1
3

1
0 5 10 15 20 25 30 35

13
Testing Random Number Generator

Chi-square test, spectral test……

Testing the randomness of a given
sequence
Scatterplots
- a plot of xi+1 as a function of xi
Durbin-Watson Test
(1/N)n2 (X[n]  X[n  1]) 2
N
-
D
(1/N)n2 X 2[n]
N

14
 Scatterplots
Example 7.5
1 1 1

0.9 0.9 0.9

(i) rand(1,2048)
0.8 0.8 0.8

0.7 0.7 0.7

0.6 0.6 0.6 (ii)xi+1=[65xi+1]

0.5 0.5 0.5 mod(2048)
0.4 0.4 0.4

0.3 0.3 0.3

0.2 0.2 0.2

(iii)xi+1=[1229xi+
0.1 0.1 0.1 1]mod(2048)
0 0 0
0 0.5 1 0 0.5 1 0 0.5 1

15
Durbin-Watson Test (1)
(1/N)n2 (X[n]  X[n  1]) 2
N

D
(1/N)n2 X 2[n]
N

Let X = X[n] & Y = X[n-1]

Assume X[n] and X[n-1] are correlated and X[n]
is an ergodic process
E{( X - Y)2 } 1
D   E{( X - Y) 2
}
E{X } 2
x
2

Let Y  ρX  1  ρ 2 Z 1  ρ  1
16
 Durbin-Watson Test (2)

X and Z are uncorrelated and zero mean

1

D  2 E (1  ρ) 2 X 2  2(1  ρ) (1  ρ) 2 XZ  (1  ρ) 2 Z 2
σ

(1  ρ 2 )σ 2  (1  ρ 2 )σ 2
D  2(1  ρ)
σ 2

D>2 – negative correlation

D=2 –- uncorrelation (most desired)
D<2 – positive correlation
17
Example 7.6

rand(1,2048) - The value of D is 2.0081 and

ρ is 0.0041.

xi+1=[65xi+1]mod(2048) - The value of D is

1.9925 and ρ is 0.0037273.

xi+1=[1229xi+1]mod(2048) - The value of D

is 1.6037 and ρ is 0.19814.
18
Minimum Standards

Full period
Passes all applicable statistical tests for
randomness.
Easily transportable from one computer to
another
Lewis, Goodman, and Miller Minimum
Standard (prior to MATLAB 5)
xi+1=[16807xi]mod(231-1)
19
Mapping Uniform RVs to an Arbitrary pdf

The cumulative distribution for the target

random variable is known in closed form –
Inverse Transform Method
The pdf of target random variable is
known in closed form but the CDF is not
known in closed form – Rejection Method
Neither the pdf nor CDF are known in
closed form – Histogram Method
20
Inverse Transform Method
FX(x)
 CDF FX(X) are known in 1
closed form
U

 U = FX (X) = Pr { X≦ x }
X = FX-1 (U)
FX-1(U) x
 FX (X) = Pr { FX-1 (U) ≦ x }
= Pr {U ≦ FX (x) }
= FX (x)
21
Example 7.8 (1)

Rayleigh random variable with pdf –

r  r2 
f R (r)  2 exp   2 u(r)
σ  2σ 
r y  y2   r2 
∴ FR (r)   2 exp  2 dy  1  exp   2 
0 σ
 2σ   2σ 

Setting FR(R) = U
 r2 
1  exp   2   U
 2σ  22
Example 7.8 (2)
∵ RV 1-U is equivalent to U (have same pdf)
∴  r2 
exp   2   U
 2σ 
Solving for R gives
R   2σ ln( U )
2

 [n,xout] = hist(Y,nbins) -
bar(xout,n) - plot the histogram
23
Example 7.8 (3)
1500
Number of Samples

1000

500

0
0 1 2 3 4 5 6 7 8 9
Independent Variable - x

0.4
true pdf
Probability Density

0.3 samples from histogram

0.2

0.1

0
0 1 2 3 4 5 6 7 8
Independent Variable - x 24
The Histogram Method

CDF and pdf are unknown

Pi = Pr{xi-1 < x < xi} = ci(xi-xi-1)
FX(x) = Fi-1 + ci(xi-xi-1)
i 1
Fi 1  Pr{ X  X i 1}   Pi 1
j 1
FX(X) = U = Fi-1 + ci(X-xi) more samples
1 more accuracy!
X  xi 1  (U  Fi 1)
ci
25
Rejection Methods (1)
 Having a target pdf
 MgX(x) ≧ fX(x), all x

b  M / a 0 xa
Mg X (x)  
 0, otherwise MgX(x)

M M/a=b

b  max{ f X (x)}
fX(x)

a
gX(x)

1/a

0
0 x a 26
x+dx
Rejection Methods (2)

Generate U1 and U2 uniform in (0,1)

Generate V1 uniform in (0,a), where a is
the maximum value of X
Generate V2 uniform in (0,b), where b is at
least the maximum value of fX(x)
If V2≦ fX(V1), set X= V1. If the inequality is
not satisfied, V1 and V2 are discarded and
the process is repeated from step 1
27
Example 7.9 (1)

 4
 2 R2  x2 0xR
f X ( x)   πR
 0, otherwise

M 4 MgX(x)

R πR

fX(x)

gX(x)
1
R

0
0 R
28
Example 7.9 (2)
150
Number of Samples

100

50

0
0 1 2 3 4 5 6 7
Independent Variable - x

0.2
Probability Density

0.15

0.1

0.05 true pdf

samples from histogram
0
0 1 2 3 4 5 6 7
Independent Variable - x 29
Generating Uncorrelated Gaussian RV

Its CDF can’t be written in closed form，so

Inverse method can’t be used and
rejection method are not efficient
Other techniques
1.The sum of uniform method
2.Mapping a Rayleigh to Gaussian RV
3.The polar method

30
The Sum of Uniforms Method(1)
 1.Central limit theorem
 2.See next
N
1
Y  B  (U i  )
. i 0 2

Ui i  1, 2.., N represent independent uniform R.V

B is a constant that decides the var of Y

 3. N   Y converges to a Gaussian R.V.

31
The Sum of Uniforms Method(2)
 Expectation and Variance


N
1 1
E{U i }  E{Y }  B  ( E{U i }  )  0 
2 i 0 2


1 1/ 2 1 N
1 NB 2
var{U i  }   x 2 dx    B  var{Ui  } 
2 2
2 1/ 2 12 y
i 1 2 12
12
 We can set to any desired value B y
N
 Nonzero at  y N 12   y 3N
2 N

32
The Sum of Uniforms Method(3)

Approximate Gaussian
Maybe not a realistic situation.

33
Mapping a Rayleigh to Gaussian RV(1)

Rayleigh can be generated by

R  2 2 ln U U is the uniform RV in [0,1]

Assume X and Y are indep. Gaussian RV

and their joint pdf
 f XY ( x, y) 
2 2
1 x 1 x
exp( 2 ) exp( 2 )
2 2 2 2
1 x2  y 2
 exp( )
2 2
2 2

34
Mapping a Rayleigh to Gaussian RV(2)

Transform
 let x  r cos and y  r sin 
1 y
 x 2
 y 2
 r 2
and   tan ( )
x
 f R (r , )dAR  f XY ( x, y)dAXY
dAXY ( x, y) dx / dr dx / d
  r
dAR (r , ) dy / dr dy / d
r r2
 f R (r , )  exp( 2 )
2 2
2
35
Mapping a Rayleigh to Gaussian RV(3)

Examine the marginal pdf

2 r r2 r r2
f R (r )   exp( 2 )d  2 exp( 2 ) 0  r  
0 2 2 2  2
 r r2 1
f ( )   exp(  ) dr  0    2
0 2 2 2 2
2
R is Rayleigh RV and  is uniform RV
X  R cos  X  2 2 ln(U1 ) cos 2U 2

Y  R sin   Y  2 2 ln(U1 ) sin 2U 2

36
The Polar Method

From previous
X  2 2 ln(U1 ) cos 2U 2 Y  2 2 ln(U1 ) sin 2U 2

We may transform

s  R2  u 2  v2 (R  s )
u u
cos 2 U1  cos   
R s
v v
sin 2 U 2  sin   
R s
u 2 2 ln( s)
X  2 ln(U1 ) cos 2 U 2  2 ln( s)( )  u
2 2

s s
v 2 2 ln( s)
Y  2 ln(U1 ) sin 2 U 2  2 ln( s)( )  v
2 2

s s
37
The Polar Method Alothgrithm

1.Generate two uniform RV，U1 and U 2

and they are all on the interval (0,1)
2.Let V1  2U1  1 and V2  2U 2  1，so they are
independent and uniform on (-1,1)
3.Let S  V12  V22 if S  1 continue，
else back to step2
4.Form A(S )  (2 ln S ) / S
2

5.Set X  A(S )V1 and Y  A(S )V2

38
Establishing a Given Correlation Coefficient(1)

Assume two Gaussian RV X and Y ，

they are zero mean and uncorrelated
Define a new RV Z   X  1   2 Y |  | 1
We also can see Z is Gaussian RV
Show  is correlation coefficient relating
X and Z

39
Establishing a Given Correlation Coefficient(2)

Mean，Variance，Correlation coefficient
E{Z }  E{ X }  E{Y }  0
E{ XY }  E{ X }E{Y }  0
 X2   Y2  E{ X 2 }  ( E{ X }) 2  E{ X 2 }  E{Y 2 }   2
  E{[  X  1   Y ] }
2
Z
2 2

  2 E{ X 2 }  2  1   2 E{ XY }  (1   2 ) E{Y 2 }
  2 2  (1   2 ) 2   2
40
Establishing a Given Correlation Coefficient(3)

Covariance between X and Z

E{ XZ }  E{ X [  X  (1   )Y ]}

  E{ X }  (1   ) E{ XY }
2

  E{X 2 }   2

E{ XZ }  2
  XZ 
 X Z
 2 
 as desired

41
Pseudonoise(PN) Sequence Genarators

PN generator produces periodic sequence

that appears to be random
Generated by algorithm using initial seed
Although not random，but can pass many
tests of randomness
Unless algorithm and seed are known，
the sequence is impractical to predict

42
PN Generator implementation

43
Property of Linear Feedback Shift Register(LFSR)

Nearly random with long period

May have max period
If output satisfy period ，is called
max-length sequence or m-sequence
We define generator polynomial as

The coefficient to generate m-sequence

can always be found
44
Example of PN generator

45
Different seed for the PN generator

46
Family of M-sequences

47
Property of m-sequence

Has ones， zeros

The periodic autocorrelation of a 1 m-
sequence is

If PN has a large period，autocorrelation

function approaches an impulse，and
PSD is approximately white as desired
48
PN Autocorrelation Function

49
Signal Processing

Relationship
1.mean of input and output
2.variance of input and output
3.input-output cross-correlation
4.autocorrelation and PSD

50
Input/Output Means

Assume system is linearconvolution

k 
y[n]   h[k ]x[n  k ]
k 
 
E{ y[n]}  E{  h[k ]x[n  k ]}   h[k ]E{x[n  k ]}
k  k 

Assume stationarity assumption

 E{x[n  k ]}  E{x[n]}

We can get E{ y}  E{x}  h[k ]
 k 

and  h[k ]  H (0)  E{ y}  H (0) E{x}

k 

51
Input/Output Cross-Correlation

The Cross-Correlation is defined by


E{x[n] y[n  m]}  Rxy [m]  E{x[n]  h[ j ]x[n  j  m]}

 j 
Rxy [m]   h[ j]E{x[n  j  m]}
j 

  h[ j]R
j 
xx [m  j ]

This use is used in the development of a

number of performance estimators，which
will be developed in chapter 8
52
Output Autocorrelation Function(1)
Autocorrelation of the output
E{ y[n] y[n  m]}  Ryy [m]
 
 E{  h[ j ]x[n  j ]  h[k ]x[n  k  m]}
j  k 
 
Ryy [m]    h[ j]h[k ]E{x[n  j]x[n  m  k ]}
j  k 
 
   h[ j]h[k ]R
j  k 
xx (m  k  j )

Can’t be simplified without knowledge of

the Statistics of x[ n ]

53
Output Autocorrelation Function(2)

If input is delta-correlated(i.e. white noise)

 x2 m  0 
Rxx [m]  E{x[n]x[n  m]}      x  [ m]
2

0 m  0 
substitute previous equation R yy [ m]
 
Ryy [m]   x2   h[ j]h[k ] (m  k  j)
j  k 

 2
x  h[ j]h[ j  m]
j 

54
Input/Output Variances

By definition  Ryy [0]  E{ y 2[n]}

Let m=0 substitute into R yy [ m]

 
 y2  Ryy [0]    h[ j]h[k ]R
j  k 
xx [ j  k]

But if x[ n ] is white noise sequence


 y2  Ryy [0]   x2  h2 [ j ]
j 

55
The End
Thanks for listening

56