NyquistShannon sampling theorem

From Wikipedia, the free encyclopedia

Example of magnitude of the Fourier transform of a bandlimited function

In the field of digital signal processing, the sampling theorem is a fundamental bridge
between continuous-time signals (often called "analog signals") and discrete-time signals (often
called "digital signals"). It establishes a sufficient condition for a sample rate that permits a discrete
sequence of samples to capture all the information from a continuous-time signal of finite bandwidth.
Strictly speaking, the theorem only applies to a class of mathematical functions having a Fourier
transform that is zero outside of a finite region of frequencies. Intuitively we expect that when one
reduces a continuous function to a discrete sequence and interpolates back to a continuous function,
the fidelity of the result depends on the density (or sample rate) of the original samples. The
sampling theorem introduces the concept of a sample rate that is sufficient for perfect fidelity for the
class of functions that are bandlimited to a given bandwidth, such that no actual information is lost in
the sampling process. It expresses the sufficient sample rate in terms of the bandwidth for the class
of functions. The theorem also leads to a formula for perfectly reconstructing the original continuous-
time function from the samples.
Perfect reconstruction may still be possible when the sample-rate criterion is not satisfied, provided
other constraints on the signal are known. (See Sampling of non-baseband signalsbelow,
and compressed sensing.) In some cases (when the sample-rate criterion is not satisfied), utilizing
additional constraints allows for approximate reconstructions. The fidelity of these reconstructions
can be verified and quantified utilizing Bochner's theorem.[1]
The name NyquistShannon sampling theorem honors Harry Nyquist and Claude Shannon. The
theorem was also discovered independently by E. T. Whittaker, by Vladimir Kotelnikov, and by
others. It is thus also known by the names NyquistShannonKotelnikov, WhittakerShannon
Kotelnikov, WhittakerNyquistKotelnikovShannon, and cardinal theorem of interpolation.

Contents
[hide]

1Introduction
2Aliasing
3Derivation as a special case of Poisson summation
4Shannon's original proof
o 4.1Notes
5Application to multivariable signals and images
6Critical frequency
7Sampling of non-baseband signals
8Nonuniform sampling
9Sampling below the Nyquist rate under additional restrictions
10Historical background
 o 10.1Other discoverers
o 10.2Why Nyquist?
11See also
12Notes
13References
14Further reading
15External links

Introduction[edit]
Sampling is a process of converting a signal (for example, a function of continuous time and/or
space) into a numeric sequence (a function of discrete time and/or space). Shannon's version of the
theorem states:[2]
If a function x(t) contains no frequencies higher than B hertz, it is completely determined by giving its
ordinates at a series of points spaced 1/(2B) seconds apart.

A sufficient sample-rate is therefore 2B samples/second, or anything larger. Equivalently, for a given

sample rate fs, perfect reconstruction is guaranteed possible for a bandlimit B < fs/2.
When the bandlimit is too high (or there is no bandlimit), the reconstruction exhibits imperfections
known as aliasing. Modern statements of the theorem are sometimes careful to explicitly state
that x(t) must contain no sinusoidal component at exactly frequency B, or that B must be strictly less
than the sample rate. The two thresholds, 2B and fs/2 are respectively called the Nyquist
rateand Nyquist frequency. And respectively, they are attributes of x(t) and of the sampling
equipment. The condition described by these inequalities is called the Nyquist criterion, or
sometimes the Raabe condition. The theorem is also applicable to functions of other domains, such
as space, in the case of a digitized image. The only change, in the case of other domains, is the
units of measure applied to t, fs, and B.

The normalized sinc function: sin(x) / (x) ... showing the central peak at x= 0, and zero-crossings at the other
integer values of x.

The symbol T = 1/fs is customarily used to represent the interval between samples and is called
the sample period or sampling interval. And the samples of function x(t) are commonly denoted
by x[n] = x(nT) (alternatively "xn" in older signal processing literature), for all integer values of n. A
mathematically ideal way to interpolate the sequence involves the use of sinc functions. Each
sample in the sequence is replaced by a sinc function, centered on the time axis at the original
location of the sample, nT, with the amplitude of the sinc function scaled to the sample value, x[n].
Subsequently, the sinc functions are summed into a continuous function. A mathematically
equivalent method is to convolve one sinc function with a series of Dirac deltapulses, weighted by
the sample values. Neither method is numerically practical. Instead, some type of approximation of
the sinc functions, finite in length, is used. The imperfections attributable to the approximation are
known as interpolation error.
Practical digital-to-analog converters produce neither scaled and delayed sinc functions, nor ideal
Dirac pulses. Instead they produce a piecewise-constant sequence of scaled and
delayed rectangular pulses (the zero-order hold), usually followed by an "anti-imaging filter" to clean
up spurious high-frequency content.

Aliasing[edit]
Main article: Aliasing

The samples of two sine waves can be identical when at least one of them is at a frequency above half the
sample rate.

When x(t) is a function with a Fourier transform, X(f):

the Poisson summation formula indicates that the samples, x(nT),

of x(t) are sufficient to create a periodic summation of X(f). The result
is:

(Eq.1)

X(f) (top blue) and XA(f) (bottom blue) are continuous Fourier
transforms of two different functions, x(t) and xA(t) (not shown). When
the functions are sampled at rate fs, the images (green) are added to
the original transforms (blue) when one examines the discrete-time
Fourier transforms (DTFT) of the sequences. In this hypothetical
 example, the DTFTs are identical, which means the sampled
sequences are identical, even though the original continuous pre-
sampled functions are not. If these were audio signals, x(t) and xA(t)
might not sound the same. But their samples (taken at rate fs) are
identical and would lead to identical reproduced sounds; thus xA(t) is
an alias of x(t) at this sample rate.

which is a periodic function and its equivalent representation as

a Fourier series, whose coefficients are Tx(nT). This function is
also known as the discrete-time Fourier transform(DTFT) of the
sequence Tx(nT), for integers n.
As depicted, copies of X(f) are shifted by multiples of fsand
combined by addition. For a band-limited function (X(f) = 0 for all
|f| B), and sufficiently large fs, it is possible for the copies to
remain distinct from each other. But if the Nyquist criterion is not
satisfied, adjacent copies overlap, and it is not possible in general
to discern an unambiguous X(f). Any frequency component
above fs/2 is indistinguishable from a lower-frequency component,
called an alias, associated with one of the copies. In such cases,
the customary interpolation techniques produce the alias, rather
than the original component. When the sample-rate is pre-
determined by other considerations (such as an industry
standard), x(t) is usually filtered to reduce its high frequencies to
acceptable levels before it is sampled. The type of filter required is
a lowpass filter, and in this application it is called an anti-aliasing
filter.

Spectrum, Xs(f), of a properly sampled bandlimited signal (blue) and

the adjacent DTFT images (green) that do not overlap. A brick-
wall low-pass filter, H(f), removes the images, leaves the original
spectrum, X(f), and recovers the original signal from its samples.

Derivation as a special case of Poisson

summation[edit]
When there is no overlap of the copies (aka "images") of X(f),
the k = 0 term of Xs(f) can be recovered by the product:

where:

At this point, the sampling theorem is proved, since X(f)

uniquely determines x(t).
All that remains is to derive the formula for
reconstruction. H(f) need not be precisely defined in the
region [B, fs B] because Xs(f) is zero in that region.
However, the worst case is when B = fs/2, the Nyquist
frequency. A function that is sufficient for that and all less
severe cases is:

where rect() is the rectangular function. Therefore:

(from Eq.1, above).

[note 1]

The inverse transform of both sides produces

the WhittakerShannon interpolation formula:

which shows how the samples, x(nT), can be

combined to reconstruct x(t).

Larger-than-necessary values
of fs (smaller values of T),
called oversampling, have no effect on
the outcome of the reconstruction and
have the benefit of leaving room for
a transition bandin which H(f) is free to
take intermediate values. Undersampling,
which causes aliasing, is not in general a
reversible operation.
Theoretically, the interpolation formula
can be implemented as a low pass filter,
whose impulse response is sinc(t/T) and

whose input is which is a Dirac

combfunction modulated by the signal
samples. Practical digital-to-analog
converters (DAC) implement an
 approximation like the zero-order hold. In
that case, oversampling can reduce the
approximation error.

Shannon's original proof[edit]

Poisson shows that the Fourier series
in Eq.1 produces the periodic summation
of X(f), regardless of fs and B. Shannon,
however, only derives the series coefficients
for the case fs = 2B. Virtually quoting
Shannon's original paper:

Let be the spectrum of Then

since is assumed to be zero outside the band . If we let

where n is any positive or negative integer, we obtain

On the left are values of at the sampling points. The integral

on the right will be recognized as essentially[n 1] the nth coefficient in

a Fourier-series expansion of the function taking the interval

B to B as a fundamental period. This means that the values of the

samples determine the Fourier coefficients in the series

expansion of Thus they determine since is zero

for frequencies greater than B, and for lower frequencies is

determined if its Fourier coefficients are determined.

But determines the original function completely, since a

function is determined if its spectrum is known. Therefore the

original samples determine the function completely.

Shannon's proof
of the theorem is
complete at that
point, but he
goes on to
discuss
 reconstruction
via sinc
functions, what
we now call
the Whittaker
Shannon
interpolation
formula as
discussed
above. He does
not derive or
prove the
properties of the
sinc function, but
these would
have
been[weasel words] fa
miliar to
engineers
reading his
works at the
time, since the
Fourier pair
relationship
between rect (th
e rectangular
function) and
sinc was well
known.

Let be the nth sample. Then the function is represented

by:

As in
the
other
proof,
the
existenc
e of the
Fourier
transfor
m of the
original
signal is
assume
d, so the
proof
does not
say
whether
the
samplin
g
theorem
extends
to
bandlimi
ted
stationar
y
random
process
es.
Notes[
edit]

1. J
u
m
p

u
p
^

T
h
e

a
c
t
u
a
l
c
o
e
f
f
i
c
i
e
n
t

f
o
r
m
u
l
a

c
o
n
t
a
i
n
s

a
n

a
d
d
i
t
i
o
n
a
l
f
a
c
t
o
r

o
f

S
o

S
h
a
n
n
o
n
'
s

c
o
e
f
f
i
c
i
e
n
t
s
 a
r
e

w
h
i
c
h

a
g
r
e
e
s

w
i
t
h

E
q
.
1
.

Appli
catio
n to
multi
varia
ble
signa
ls
and
imag
es[edit]
Main
article:
Multidim
ensional
samplin
g
Subs
ampl
ed
imag
e
show
ing
a Moi
r
patte
rn

Prop
erly
samp
led
 imag
e

The
samplin
g
theorem
is
usually
formulat
ed for
function
s of a
single
variable.
Conseq
uently,
the
theorem
is
directly
applicab
le to
time-
depend
ent
signals
and is
normally
formulat
ed in
that
context.
Howeve
r, the
samplin
g
theorem
can be
extende
d in a
straightf
orward
way to
function
s of
arbitraril
y many
variable
s.
Graysca
le
images,
for
example
, are
often
represe
nted as
two-
dimensi
onal
arrays
(or
matrices
) of real
number
s
represe
nting the
relative
intensiti
es
of pixels
(picture
element
s)
located
at the
intersect
ions of
row and
column
sample
location
s. As a
result,
images
require
two
indepen
dent
variable
s, or
indices,
to
specify
each
pixel
uniquely
one
for the
row, and
one for
the
column.
Color
images
typically
consist
of a
composi
te of
three
separat
e
grayscal
e
images,
one to
represe
nt each
of the
three
primary
colors
red,
green,
and
blue,
or RGB
for
short.
Other
colorspa
ces
using 3-
vectors
for
colors
include
HSV,
CIELAB,
XYZ,
etc.
Some
colorspa
ces
such as
cyan,
magent
a,
yellow,
and
black
(CMYK)
may
represe
nt color
by four
dimensi
ons. All
of these
are
treated
as vecto
r-valued
function
s over a
two-
dimensi
onal
sampled
domain.
Similar
to one-
dimensi
onal
discrete-
time
signals,
images
can also
suffer
from
aliasing
if the
samplin
g
resolutio
n, or
pixel
density,
is
inadequ
ate. For
example
,a
digital
photogr
aph of a
striped
shirt
with
high
frequen
cies (in
other
words,
the
distance
between
the
stripes
is
small),
can
cause
aliasing
of the
shirt
when it
is
sampled
by the
camera'
s image
sensor.
The
aliasing
appears
as
a moir
pattern.
The
"solution
" to
higher
samplin
g in the
spatial
domain
for this
case
would
be to
move
closer to
the shirt,
use a
higher
resolutio
n
sensor,
or to
optically
blur the
image
before
acquirin
g it with
the
sensor.
Another
example
is
shown
to the
right in
the brick
patterns
. The
top
image
shows
the
effects
when
the
samplin
g
theorem'
s
conditio
n is not
satisfied
. When
software
rescales
an
image
(the
same
process
that
creates
the
thumbn
ail
shown
in the
lower
image)
it, in
effect,
runs the
image
through
a low-
pass
filter first
and
then do
wnsamp
les the
image to
result in
a
smaller
image
that
does not
exhibit
the moir

pattern.
The top
image is
what
happens
when
the
image is
downsa
mpled
without
low-
pass
filtering:
aliasing
results.
The
applicati
on of
the
samplin
g
theorem
to
images
should
be
made
with
care.
For
example
, the
samplin
g
process
in any
standar
d image
sensor
(CCD or
CMOS
camera)
is
relativel
y far
from the
ideal
samplin
g which
would
measur
e the
image
intensity
at a
single
point.
Instead
these
devices
have a
relativel
y large
sensor
area at
each
sample
point in
order to
obtain
sufficien
t
amount
of light.
In other
words,
any
detector
has a
finite-
width po
int
spread
function.
The
analog
optical
image
intensity
function
which is
sampled
by the
sensor
device
is not in
general
bandlimi
ted, and
the non-
ideal
samplin
g is
itself a
useful
type of
low-
pass
filter,
though
not
always
sufficien
t to
remove
enough
high
frequen
cies to
sufficien
tly
reduce
aliasing.
When
the area
of the
samplin
g spot
(the size
of the
pixel
sensor)
is not
large
enough
to
provide
sufficien
t spatial
anti-
aliasing,
a
separat
e anti-
aliasing
filter
(optical
low-
pass
filter) is
typically
included
in a
camera
system
to
further
blur the
optical
image.
Despite
images
having
these
problem
s in
relation
to the
samplin
g
theorem
, the
theorem
can be
used to
describe
the
basics
of down
and up
samplin
g of
images.

Critic
al
frequ
ency[
edit]
To
illustrate
the
necessit
y of fs >
2B,
consider
the
family of
sinusoid
s
generat
ed by
different
values
of in
this
formula:

f
a
m
il
y
o
f
s
i
n
u
s
o
i
d
s
a
t
t
h
e
c
r
it
i
c
a
l
f
r
e
q
u
e
n
c
y
,
a
ll
h
a
v
i
n
g

t
h
e

s
a
m
e

s
a
m
p
l
e

s
e
q
u
e
n
c
e
s
o
f
a
lt
e
r
n
a
ti
n
g

+
1

a
n
d

1
.
T
h
a
t
i
s
,
t
h
e
y
a
ll
a
r
e

a
li
a
s
e
s
o
f
e
a
c
h

o
t
h
e
r
,
e
v
e
n
t
h
o
u
g
h

t
h
e
i
r
f
r
e
q
u
e
n
c
y
i
s
n
o
t
a
b
o
v
e

h
a
lf
t
h
e
 s
a
m
p
l
e

r
a
t
e
.

With
fs =
2B o
r
equi
vale
ntly
T=
1/(2
B),
the
sam
ples
are
give
n
by:

r
e
g
a
r
d
l
e
s
s

o
f
t
h
e

v
a
l
u
e

o
f

T
h
a
t

s
o
r
t

o
f

a
m
b
i
g
u
i
t
y

i
s

t
h
e

r
e
a
s
o
n
f
o
r

t
h
e

s
t
r
i
c
t

i
n
e
q
u
a
l
i
t
y

o
f

t
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m
'
s

c
o
n
d
i
t
i
o
n
.

S
a
m
p
l
i
n
g

o
f

n
o
n
-
b
a
s
e
b
a
n
d

s
i
g
 n
a
l
s
[
e
d
i
t
]
A
s

d
i
s
c
u
s
s
e
d

b
y

S
h
a
n
n
o
n
:
[
2
]

A similar result is true if the band does not start at zero frequency
but at some higher value, and can be proved by a linear translation
(corresponding physically to single-sideband modulation) of the
zero-frequency case. In this case the elementary pulse is obtained
from sin(x)/x by single-side-band modulation.

T
h
a
t

i
s
,

s
u
f
f
i
c
i
e
n
t

n
o
-
l
o
s
s

c
o
n
d
i
t
i
o
n

f
o
r

s
a
m
p
l
i
n
g

s
i
g
n
a
l
s
t
h
a
t

d
o

n
o
t

h
a
v
e

b
a
s
e
b
a
n
d

c
o
m
p
o
n
e
n
t
s

e
x
i
s
t
s

t
h
a
t

i
n
v
o
l
v
e
s

t
h
e

w
i
d
t
h

o
f

t
h
e

n
o
n
-
z
e
r
o

f
r
e
q
u
e
n
c
y

i
n
t
e
r
v
a
l

a
s
o
p
p
o
s
e
d

t
o

i
t
s

h
i
g
h
e
s
t

f
r
e
q
u
e
n
c
y

c
o
m
p
o
n
e
n
t
.

S
e
e

S
a
m
p
l
i
n
g

(
s
i
g
n
a
l

p
r
o
c
e
s
s
i
n
g
)

f
o
r

m
o
r
e

d
e
t
a
i
l
s

a
n
d

e
x
a
m
p
l
e
s
.
F
o
r

e
x
a
m
p
l
e
,

i
n

o
r
d
e
r

t
o

s
a
m
p
l
e

t
h
e

F
M

r
a
d
i
o

s
i
g
n
a
l
s

i
n

t
h
e

f
r
e
q
u
e
n
c
y

r
a
n
g
e

o
f

1
0
0

1
0
2

M
H
z
,

i
t

i
s

n
o
t

n
e
c
e
s
s
a
r
y

t
o

s
a
m
p
l
e

a
t

2
0
4

M
H
z

(
t
w
i
c
e

t
h
e

u
p
p
e
r

f
r
e
q
u
e
n
c
y
)
,

b
u
t

r
a
t
h
e
r

i
t

i
s

s
u
f
f
i
c
i
e
n
t

t
o

s
a
m
p
l
e

a
t

M
H
z

(
t
w
i
c
e

t
h
e

w
i
d
t
h

o
f

t
h
e

f
r
e
q
u
e
n
c
y

i
n
t
e
r
v
a
l
)
.
A

b
a
n
d
p
a
s
s
c
o
n
d
i
t
i
o
n

i
s

t
h
a
t

X
(
f
)

0
,

f
o
r

a
l
l

n
o
n
n
e
g
a
t
i
v
e

o
u
t
s
i
d
e

t
h
e

o
p
e
n

b
a
n
d

o
f

f
r
e
q
u
e
n
c
i
e
s
:

f
o
r

s
o
m
e

n
o
n
n
e
g
a
t
i
v
e

i
n
t
e
g
e
r

N
.

T
h
i
s

f
o
r
m
u
l
a
t
i
o
n

i
n
c
l
u
d
e
s

t
h
e

n
o
r
m
a
l
b
a
s
e
b
a
n
d

c
o
n
d
i
t
i
o
n

a
s

t
h
e

c
a
s
e

N
=
0
.
T
h
e

c
o
r
r
e
s
p
o
n
d
i
n
g
i
n
t
e
r
p
o
l
a
t
i
o
n

f
u
n
c
t
i
o
n

i
s

t
h
e

i
m
p
u
l
s
e

r
e
s
p
o
n
s
e

o
f

a
n
i
d
e
a
l

b
r
i
c
k
-
w
a
l
l

b
a
n
d
p
a
s
s

f
i
l
t
e
r

(
a
s

o
p
p
o
s
e
d

t
o

t
h
e
i
d
e
a
l

b
r
i
c
k
-
w
a
l
l

l
o
w
p
a
s
s

f
i
l
t
e
r

u
s
e
d

a
b
o
v
e
)

w
i
t
h

c
u
t
o
f
f
s

a
t

t
h
e

u
p
p
e
r

a
n
d

l
o
w
e
r

e
d
g
e
s

o
f

t
h
e

s
p
e
c
i
f
i
e
d

b
a
n
d
,

w
h
i
c
h

i
s

t
h
e

d
i
f
f
e
r
e
n
c
e

b
e
t
w
e
e
n

p
a
i
r

o
f

l
o
w
p
a
s
s
i
m
p
u
l
s
e

r
e
s
p
o
n
s
e
s
:

O
t
h
e
r

g
e
n
e
r
a
l
i
z
a
t
i
o
n
s
,

f
o
r

e
x
a
m
p
l
e

t
o

s
i
g
n
a
l
s

o
c
c
u
p
y
i
n
g

m
u
l
t
i
p
l
e

n
o
n
-
c
o
n
t
i
g
u
o
u
s

b
a
n
d
s
,

a
r
e

p
o
s
s
i
b
l
e

a
s

w
e
l
l
.

E
v
e
n

t
h
e

m
o
s
t

g
e
n
e
r
a
l
i
z
e
d

f
o
r
m

o
f

t
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m

d
o
e
s

n
o
t

h
a
v
e

p
r
o
v
a
b
l
y

t
r
u
e

c
o
n
v
e
r
s
e
.

T
h
a
t

i
s
,

o
n
e

c
a
n
n
o
t

c
o
n
c
l
u
d
e

t
h
a
t

i
n
f
o
r
m
a
t
i
o
n

i
s

n
e
c
e
s
s
a
r
i
l
y

l
o
s
t

j
u
s
t

b
e
c
a
u
s
e

t
h
e

c
o
n
d
i
t
i
o
n
s
o
f

t
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m

a
r
e

n
o
t

s
a
t
i
s
f
i
e
d
;

f
r
o
m

a
n

e
n
g
i
n
e
e
r
i
n
g

p
e
r
s
p
e
c
t
i
v
e
,

h
o
w
e
v
e
r
,

i
t

i
s

g
e
n
e
r
a
l
l
y

s
a
f
e
t
o

a
s
s
u
m
e

t
h
a
t

i
f

t
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m

i
s

n
o
t

s
a
t
i
s
f
i
e
d

t
h
e
n

i
n
f
o
r
m
a
t
i
o
n

w
i
l
l

m
o
s
t

l
i
k
e
l
y

b
e

l
o
s
t
.

N
o
n
u
n
i
f
o
r
m

s
a
m
p
l
i
n
g
[
e
d
i
t
]
T
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
y

o
f
S
h
a
n
n
o
n

c
a
n

b
e

g
e
n
e
r
a
l
i
z
e
d

f
o
r

t
h
e

c
a
s
e

o
f

n
o
n
u
n
i
f
o
r
m
s
a
m
p
l
i
n
g
,

t
h
a
t

i
s
,

s
a
m
p
l
e
s

n
o
t

t
a
k
e
n

e
q
u
a
l
l
y

s
p
a
c
e
d
i
n

t
i
m
e
.

T
h
e

S
h
a
n
n
o
n

s
a
m
p
l
i
n
g

t
h
e
o
r
y

f
o
r

n
o
n
-
u
n
i
f
o
r
m
s
a
m
p
l
i
n
g

s
t
a
t
e
s

t
h
a
t

b
a
n
d
-
l
i
m
i
t
e
d

s
i
g
n
a
l

c
a
n

b
e

p
e
r
f
e
c
t
l
y

r
e
c
o
n
s
t
r
u
c
t
e
d

f
r
o
m

i
t
s

s
a
m
p
l
e
s

i
f

t
h
e

a
v
e
r
a
g
e
s
a
m
p
l
i
n
g

r
a
t
e

s
a
t
i
s
f
i
e
s

t
h
e

N
y
q
u
i
s
t

c
o
n
d
i
t
i
o
n
.
[
3
]

T
h
e
r
e
f
o
r
e
,

a
l
t
h
o
u
g
h

u
n
i
f
o
r
m
l
y

s
p
a
c
e
d

s
a
m
p
l
e
s

m
a
y

r
e
s
u
l
t

i
n

e
a
s
i
e
r

r
e
c
o
n
s
t
r
u
c
t
i
o
n

a
l
g
o
r
i
t
h
m
s
,

i
t

i
s

n
o
t

n
e
c
e
s
s
a
r
y

c
o
n
d
i
t
i
o
n

f
o
r

p
e
r
f
e
c
t

r
e
c
o
n
s
t
r
u
c
t
i
o
n
.
T
h
e

g
e
n
e
r
a
l
t
h
e
o
r
y

f
o
r

n
o
n
-
b
a
s
e
b
a
n
d

a
n
d

n
o
n
u
n
i
f
o
r
m

s
a
m
p
l
e
s

w
a
s

d
e
v
e
l
o
p
e
d

i
n

1
9
6
7

b
y

H
e
n
r
y

L
a
n
d
a
u
.
[
4
]

H
e

p
r
o
v
e
d

t
h
a
t

t
h
e

a
v
e
r
a
g
e

s
a
m
p
l
i
n
g

r
a
t
e

(
u
n
i
f
o
r
m

o
r

o
t
h
e
r
w
i
s
e
)

m
u
s
t

b
e

t
w
i
c
e

t
h
e

o
c
c
u
p
i
e
d

b
a
n
d
w
i
d
t
h

o
f

t
h
e

s
i
g
n
a
l
,

a
s
s
u
m
i
n
g

i
t

i
s

p
r
i
o
r
i

k
n
o
w
n

w
h
a
t

p
o
r
t
i
o
n

o
f

t
h
e

s
p
e
c
t
r
u
m

w
a
s

o
c
c
u
p
i
e
d
.

I
n

t
h
e

l
a
t
e

1
9
9
0
s
,

t
h
i
s

w
o
r
k

w
a
s

p
a
r
t
i
a
l
l
y

e
x
t
e
n
d
e
d

t
o

c
o
v
e
r

s
i
g
n
a
l
s

o
f

w
h
e
n

t
h
e

a
m
o
u
n
t

o
f

o
c
c
u
p
i
e
d

b
a
n
d
w
i
d
t
h

w
a
s

k
n
o
w
n
,

b
u
t

t
h
e

a
c
t
u
a
l

o
c
c
u
p
i
e
d

p
o
r
t
i
o
n

o
f

t
h
e

s
p
e
c
t
r
u
m

w
a
s

u
n
k
n
o
w
n
.
[
5
]

I
n

t
h
e

2
0
0
0
s
,

a
c
o
m
p
l
e
t
e

t
h
e
o
r
y

w
a
s

d
e
v
e
l
o
p
e
d

(
s
e
e

t
h
e

s
e
c
t
i
o
n

B
e
y
o
n
d

N
y
q
u
i
s
t
b
e
l
o
w
)

u
s
i
n
g

c
o
m
p
r
e
s
s
e
d

s
e
n
s
i
n
g
.

I
n

p
a
r
t
i
c
u
l
a
r
,

t
h
e

t
h
e
o
r
y
,

u
s
i
n
g

s
i
g
n
a
l

p
r
o
c
e
s
s
i
n
g

l
a
n
g
u
a
g
e
,

i
s
d
e
s
c
r
i
b
e
d

i
n

t
h
i
s

2
0
0
9

p
a
p
e
r
.
[
6
]

T
h
e
y

s
h
o
w
,

a
m
o
n
g

o
t
h
e
r

t
h
i
n
g
s
,

t
h
a
t

i
f

t
h
e

f
r
e
q
u
e
n
c
y

l
o
c
a
t
i
o
n
s

a
r
e

u
n
k
n
o
w
n
,

t
h
e
n

i
t

i
s

n
e
c
e
s
s
a
r
y

t
o

s
a
m
p
l
e

a
t

l
e
a
s
t

a
t

t
w
i
c
e

t
h
e

N
y
q
u
i
s
t

c
r
i
t
e
r
i
a
;

i
n

o
t
h
e
r

w
o
r
d
s
,

y
o
u

m
u
s
t

p
a
y

a
t

l
e
a
s
t

f
a
c
t
o
r

o
f

f
o
r

n
o
t

k
n
o
w
i
n
g

t
h
e

l
o
c
a
t
i
o
n

o
f

t
h
e

s
p
e
c
t
r
u
m
.

N
o
t
e

t
h
a
t

m
i
n
i
m
u
m

s
a
m
p
l
i
n
g

r
e
q
u
i
r
e
m
e
n
t
s

d
o

n
o
t

n
e
c
e
s
s
a
r
i
l
y

g
u
a
r
a
n
t
e
e

s
t
a
b
i
l
i
t
y
.

S
a
m
p
l
i
n
g
b
e
l
o
w

t
h
e

N
y
q
u
i
s
t

r
a
t
e

u
n
d
e
r

a
d
d
i
t
i
o
n
a
l

r
e
s
t
r
i
c
t
i
o
n
s
[
e
d
i
t
]
M
a
i
n

a
r
t
i
c
l
e
:

U
n
d
e
r
s
a
m
p
l
i
n
g

T
h
e

N
y
q
u
i
s
t

S
h
a
n
n
o
n

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m

p
r
o
v
i
d
e
s
a

s
u
f
f
i
c
i
e
n
t

c
o
n
d
i
t
i
o
n

f
o
r

t
h
e

s
a
m
p
l
i
n
g

a
n
d

r
e
c
o
n
s
t
r
u
c
t
i
o
n

o
f

b
a
n
d
-
l
i
m
i
t
e
d

s
i
g
n
a
l
.

W
h
e
n

r
e
c
o
n
s
t
r
u
c
t
i
o
n
i
s

d
o
n
e

v
i
a

t
h
e

W
h
i
t
t
a
k
e
r

S
h
a
n
n
o
n

i
n
t
e
r
p
o
l
a
t
i
o
n

f
o
r
m
u
l
a
,

t
h
e

N
y
q
u
i
s
t

c
r
i
t
e
r
i
o
n

i
s

a
l
s
o

n
e
c
e
s
s
a
r
y

c
o
n
d
i
t
i
o
n

t
o

a
v
o
i
d

a
l
i
a
s
i
n
g
,

i
n

t
h
e

s
e
n
s
e

t
h
a
t

i
f

s
a
m
p
l
e
s

a
r
e

t
a
k
e
n

a
t

s
l
o
w
e
r

r
a
t
e

t
h
a
n

t
w
i
c
e

t
h
e

b
a
n
d

l
i
m
i
t
,

t
h
e
n

t
h
e
r
e

a
r
e

s
o
m
e

s
i
g
n
a
l
s

t
h
a
t

w
i
l
l

n
o
t

b
e

c
o
r
r
e
c
t
l
y
r
e
c
o
n
s
t
r
u
c
t
e
d
.

H
o
w
e
v
e
r
,

i
f

f
u
r
t
h
e
r

r
e
s
t
r
i
c
t
i
o
n
s

a
r
e
i
m
p
o
s
e
d

o
n

t
h
e

s
i
g
n
a
l
,

t
h
e
n

t
h
e

N
y
q
u
i
s
t

c
r
i
t
e
r
i
o
n

m
a
y
n
o

l
o
n
g
e
r

b
e

n
e
c
e
s
s
a
r
y

c
o
n
d
i
t
i
o
n
.
A

n
o
n
-
t
r
i
v
i
a
l

e
x
a
m
p
l
e

o
f

e
x
p
l
o
i
t
i
n
g

e
x
t
r
a

a
s
s
u
m
p
t
i
o
n
s

a
b
o
u
t

t
h
e

s
i
g
n
a
l
i
s

g
i
v
e
n

b
y

t
h
e

r
e
c
e
n
t

f
i
e
l
d

o
f

c
o
m
p
r
e
s
s
e
d

s
e
n
s
i
n
g
,
w
h
i
c
h

a
l
l
o
w
s

f
o
r

f
u
l
l

r
e
c
o
n
s
t
r
u
c
t
i
o
n

w
i
t
h

s
u
b
-
N
y
q
u
i
s
t

s
a
m
p
l
i
n
g

r
a
t
e
.

S
p
e
c
i
f
i
c
a
l
l
y
,

t
h
i
s

a
p
p
l
i
e
s

t
o

s
i
g
n
a
l
s

t
h
a
t

a
r
e

s
p
a
r
s
e

(
o
r

c
o
m
p
r
e
s
s
i
b
l
e
)

i
n

s
o
m
e

d
o
m
a
i
n
.
A
s

a
n

e
x
a
m
p
l
e
,

c
o
m
p
r
e
s
s
e
d

s
e
n
s
i
n
g

d
e
a
l
s

w
i
t
h

s
i
g
n
a
l
s
t
h
a
t

m
a
y

h
a
v
e

l
o
w

o
v
e
r
-
a
l
l

b
a
n
d
w
i
d
t
h

(
s
a
y
,

t
h
e

e
f
f
e
c
t
i
v
e

b
a
n
d
w
i
d
t
h

E
B
)
,

b
u
t

t
h
e

f
r
e
q
u
e
n
c
y

l
o
c
a
t
i
o
n
s

a
r
e
u
n
k
n
o
w
n
,

r
a
t
h
e
r

t
h
a
n

a
l
l

t
o
g
e
t
h
e
r

i
n

s
i
n
g
l
e

b
a
n
d
,

s
o

t
h
a
t

t
h
e

p
a
s
s
b
a
n
d

t
e
c
h
n
i
q
u
e

d
o
e
s
n
'
t

a
p
p
l
y
.

I
n

o
t
h
e
r
w
o
r
d
s
,

t
h
e

f
r
e
q
u
e
n
c
y

s
p
e
c
t
r
u
m

i
s

s
p
a
r
s
e
.

T
r
a
d
i
t
i
o
n
a
l
l
y
,

t
h
e

n
e
c
e
s
s
a
r
y

s
a
m
p
l
i
n
g

r
a
t
e

i
s

t
h
u
s

2
B
.

U
s
i
n
g

c
o
m
p
r
e
s
s
e
d

s
e
n
s
i
n
g

t
e
c
h
n
i
q
u
e
s
,

t
h
e

s
i
g
n
a
l

c
o
u
l
d

b
e

p
e
r
f
e
c
t
l
y

r
e
c
o
n
s
t
r
u
c
t
e
d

i
f

i
t

i
s

s
a
m
p
l
e
d

a
t

r
a
t
e

s
l
i
g
h
t
l
y

l
o
w
e
r

t
h
a
n

2
E
B
.

T
h
e

d
o
w
n
s
i
d
e

o
f

t
h
i
s

a
p
p
r
o
a
c
h

i
s

t
h
a
t

r
e
c
o
n
s
t
r
u
c
t
i
o
n

i
s

n
o

l
o
n
g
e
r

g
i
v
e
n

b
y

f
o
r
m
u
l
a
,

b
u
t

i
n
s
t
e
a
d

b
y

t
h
e

s
o
l
u
t
i
o
n

t
o

c
o
n
v
e
x

o
p
t
i
m
i
z
a
t
i
o
n

p
r
o
g
r
a
m

w
h
i
c
h

r
e
q
u
i
r
e
s

w
e
l
l
-
s
t
u
d
i
e
d

b
u
t

n
o
n
l
i
n
e
a
r

m
e
t
h
o
d
s
.
A
n
o
t
h
e
r

e
x
a
m
p
l
e

w
h
e
r
e

s
u
b
-
N
y
q
u
i
s
t

s
a
m
p
l
i
n
g

i
s

o
p
t
i
m
a
l

a
r
i
s
e
s

u
n
d
e
r

t
h
e

a
d
d
i
t
i
o
n
a
l

c
o
n
s
t
r
a
i
n
t

t
h
a
t

t
h
e

s
a
m
p
l
e
s

a
r
e

q
u
a
n
t
i
z
e
d

i
n

a
n

o
p
t
i
m
a
l

m
a
n
n
e
r
,

a
s

i
n

c
o
m
b
i
n
e
d

s
y
s
t
e
m

o
f

s
a
m
p
l
i
n
g

a
n
d

o
p
t
i
m
a
l

l
o
s
s
y

c
o
m
p
r
e
s
s
i
o
n
.
[
7
]

T
h
i
s

s
e
t
t
i
n
g

i
s

r
e
l
e
v
a
n
t

i
n

c
a
s
e
s

w
h
e
r
e

t
h
e

j
o
i
n
t

e
f
f
e
c
t

o
f

s
a
m
p
l
i
n
g

a
n
d

q
u
a
n
t
i
z
a
t
i
o
n

i
s

t
o

b
e

c
o
n
s
i
d
e
r
e
d
,

a
n
d

c
a
n

p
r
o
v
i
d
e

l
o
w
e
r

b
o
u
n
d

f
o
r

t
h
e

m
i
n
i
m
a
l
r
e
c
o
n
s
t
r
u
c
t
i
o
n

e
r
r
o
r

t
h
a
t

c
a
n

b
e

a
t
t
a
i
n
e
d

i
n

s
a
m
p
l
i
n
g
a
n
d

q
u
a
n
t
i
z
i
n
g

r
a
n
d
o
m

s
i
g
n
a
l
.

F
o
r

s
t
a
t
i
o
n
a
r
y

G
a
u
s
s
i
a
n

r
a
n
d
o
m

s
i
g
n
a
l
s
,

t
h
i
s

l
o
w
e
r

b
o
u
n
d

i
s

u
s
u
a
l
l
y

a
t
t
a
i
n
e
d

a
t

s
u
b
-
N
y
q
u
i
s
t

s
a
m
p
l
i
n
g

r
a
t
e
,

i
n
d
i
c
a
t
i
n
g

t
h
a
t

s
u
b
-
N
y
q
u
i
s
t

s
a
m
p
l
i
n
g

i
s

o
p
t
i
m
a
l

f
o
r

t
h
i
s

s
i
g
n
a
l

m
o
d
e
l
u
n
d
e
r

o
p
t
i
m
a
l

q
u
a
n
t
i
z
a
t
i
o
n
.
[
8
]

H
i
s
t
o
r
i
c
a
l

b
a
c
k
g
r
o
u
n
d
[
e
d
i
t
]
T
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m

w
a
s

i
m
p
l
i
e
d

b
y
t
h
e

w
o
r
k

o
f

H
a
r
r
y

N
y
q
u
i
s
t

i
n

1
9
2
8
,
[
9
]

i
n

w
h
i
c
h

h
e

s
h
o
w
e
d

t
h
a
t

u
p

t
o

2
B

i
n
d
e
p
e
n
d
e
n
t

p
u
l
s
e

s
a
m
p
l
e
s

c
o
u
l
d

b
e
s
e
n
t

t
h
r
o
u
g
h

s
y
s
t
e
m

o
f

b
a
n
d
w
i
d
t
h

B
;

b
u
t

h
e

d
i
d

n
o
t
e
x
p
l
i
c
i
t
l
y

c
o
n
s
i
d
e
r

t
h
e

p
r
o
b
l
e
m

o
f

s
a
m
p
l
i
n
g

a
n
d

r
e
c
o
n
s
t
r
u
c
t
i
o
n

o
f

c
o
n
t
i
n
u
o
u
s

s
i
g
n
a
l
s
.

A
b
o
u
t

t
h
e

s
a
m
e

t
i
m
e
,

K
a
r
l

p
f
m

l
l
e
r

s
h
o
w
e
d

s
i
m
i
l
a
r

r
e
s
u
l
t
,
[
1
0
]

a
n
d

d
i
s
c
u
s
s
e
d

t
h
e

s
i
n
c
-
f
u
n
c
t
i
o
n

i
m
p
u
l
s
e

r
e
s
p
o
n
s
e

o
f

b
a
n
d
-
l
i
m
i
t
i
n
g

f
i
l
t
e
r
,

v
i
a

i
t
s

i
n
t
e
g
r
a
l
,

t
h
e

s
t
e
p

r
e
s
p
o
n
s
e

I
n
t
e
g
r
a
l
s
i
n
u
s
;

t
h
i
s

b
a
n
d
l
i
m
i
t
i
n
g

a
n
d

r
e
c
o
n
s
t
r
u
c
t
i
o
n

f
i
l
t
e
r

t
h
a
t

i
s

s
o

c
e
n
t
r
a
l

t
o

t
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m

i
s

s
o
m
e
t
i
m
e
s

r
e
f
e
r
r
e
d

t
o

a
s

p
f
m

l
l
e
r

f
i
l
t
e
r

(
b
u
t

s
e
l
d
o
m

s
o

i
n

E
n
g
l
i
s
h
)
.
T
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m
,

e
s
s
e
n
t
i
a
l
l
y

a
d
u
a
l

o
f

N
y
q
u
i
s
t
'
s

r
e
s
u
l
t
,

w
a
s

p
r
o
v
e
d

b
y

C
l
a
u
d
e

E
.

S
h
a
n
n
o
n
.
[
2
]

V
.

A
.

K
o
t
e
l
n
i
k
o
v

p
u
b
l
i
s
h
e
d

s
i
m
i
l
a
r

r
e
s
u
l
t
s

i
n

1
9
3
3
,
[
1
1
]

a
s

d
i
d

t
h
e

m
a
t
h
e
m
a
t
i
c
i
a
n

E
.

T
.

W
h
i
t
t
a
k
e
r
i
n

1
9
1
5
,
[
1
2
]

J
.

M
.

W
h
i
t
t
a
k
e
r

i
n

1
9
3
5
,
[
1
3
]

a
n
d

G
a
b
o
r

i
n

1
9
4
6

(
"
T
h
e
o
r
y

o
f

c
o
m
m
u
n
i
c
a
t
i
o
n
"
)
.

I
n

1
9
9
9
,

t
h
e

E
d
u
a
r
d

R
h
e
i
n

F
o
u
n
d
a
t
i
o
n

a
w
a
r
d
e
d

K
o
t
e
l
n
i
k
o
v

t
h
e
i
r

B
a
s
i
c

R
e
s
e
a
r
c
h

A
w
a
r
d

"
f
o
r

t
h
e

f
i
r
s
t

t
h
e
o
r
e
t
i
c
a
l
l
y

e
x
a
c
t

f
o
r
m
u
l
a
t
i
o
n

o
f

t
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m
.
"
I
n

1
9
4
8

a
n
d

1
9
4
9
,

C
l
a
u
d
e

E
.

S
h
a
n
n
o
n

p
u
b
l
i
s
h
e
d

t
h
e

t
w
o

r
e
v
o
l
u
t
i
o
n
a
r
y

p
a
p
e
r
s

i
n

w
h
i
c
h

h
e

f
o
u
n
d
e
d

t
h
e

i
n
f
o
r
m
a
t
i
o
n

t
h
e
o
r
y
.
[
1
4
]
[
1
5
]
[
2
]

I
n

S
h
a
n
n
o
n

1
9
4
8

t
h
e

s
a
m
p
l
i
n
g

t
h
e
o
r
e
m

i
s

f
o
r
m
u
l
a
t
e
d
a
s

T
h
e
o
r
e
m

1
3

L
e
t

f
(
t
)

c
o
n
t
a
i
n

n
o

f
r
e
q
u
e
n
c
i
e
s

o
v
e
r

W
.

T
h
e
n
Sampling theorem (of information theory)
Nyquist's result that equi-spaced data, with two or more points per
cycle of highest frequency, allows reconstruction of band-limited
functions. (See Cardinal theorem.)
Cardinal theorem (of interpolation theory)
A precise statement of the conditions under which values given at
a doubly infinite set of equally spaced points can be interpolated to
yield a continuous band-limited function with the aid of the function
"If the essential frequency range is limited to B cycles per second,
2B was given by Nyquist as the maximum number of code
elements per second that could be unambiguously resolved,
assuming the peak interference is less half a quantum step. This
rate is generally referred to as signaling at the Nyquist rate and
1/(2B) has been termed a Nyquist interval." (bold added for
emphasis; italics as in the original)
 [show]

e
Digital signal processing

[show]

t
e
Data compression methods
 N
a
v
i
g
a
t
i
o
n

m
e
n
u

 R
e
a
d

E
d
i
t

V
i
e
w

h
i
s
t
o
r
y