Chapter 1.

The Signal
Theory Class

This class code is 01LSHJA

The teachers for this class are:
Prof. Pierluigi Poggiolini



Prof. Monica Visintin


You can write to us at:
pierluigi.poggiolini@polito.it
monica.visintin@polito.it
You can also call us at 011 0904240 or 011 0904153.


Prof. Poggiolinis groups
website is www.optcom.polito.it,
where you can find out about his
research and all the people that do
research with him. His primary
field of interest is: ultra-high
speed long-distance
transmission systems over optical fibers. If you are interested
in this exciting topic, please talk to Prof. Poggiolini after class.

Prof. Visintin mainly works on her own under ESA contracts
on topics related to digital mo-demodulators affected by
impairments such as synchronisation errors, nonlinearities,
approximations etc, with performance evaluation through
simulation.
1.1 Signal Theory Syllabus
This class is somewhat work in progress. So the following
syllabus is subject to changes. It wont be very different,
though. We might actually take out a few things.
1.1.1 General goals of the class
This course aims at providing the student with those
theoretical and methodological tools that are essential for the
description and the processing of signals, both of the
deterministic type and of the stochastic type. This knowledge
and such skills will then be applied to the context of data
transmission, whose fundamental principles and elementary
techniques will be dealt with.
1.1.2 Prerequisites
Students are expected to master the mathematical tools and
skills provided to them by the Math courses of the first year.
Also, they have to be able to manipulate complex numbers
with ease. Random variables theory and Fourier/Laplace
transforms are also important pre-requisites.
1.1.3 What you will learn here
Students will be capable of studying, analyzing and
processing time-continuous signals, both deterministic and
stochastic, both in time domain and in frequency domain. They
will also be capable of characterizing single-input/single-output
linear time-invariant (LTI) systems. They will be able to
compute the deterministic system output signal of an LTI
system given a deterministic input signal. They will also be
able to statistically characterize the stochastic output signals of
an LTI system given stochastic input signals. Students will
learn the basics of data transmission theory and the simplest
data transmission systems. They will be able to study and
analyze such systems and compute their performance.
Students will also learn how to deal with discrete-time signals
and systems.

1.1.4 Syllabus details
- Time-continuous deterministic signals.
- Geometric representation of signals as elements of a
Hilbert space.
- Time-frequency analysis:
finite-energy signals: Fourier transform,
autocorrelation, cross-correlation, power
spectra;
periodic signals: Fourier transform; power
spectra;
signals with finite average power: power
spectra and autocorrelation functions.
- Linear time-invariant systems: impulse response;
transfer function; input/output laws in time and
frequency domain; physically realizable systems
and stability constraints.
- Digital signals: the sampling theorem, anti-aliasing
filter, reconstruction filters, examples.
- Analysis of discrete-time signals and systems using
the Z transform and the FFT
- An introduction to stochastic processes (random
signals): joint random variables; first and higher-
order statistical characterization of random
variables; stochastic processes as collections of
random variables; stochastic processes as random
collections of signal instances; autocorrelation and
auto-covariance functions; higher order moments;
stationary processes; ergodic processes; power
spectra of stochastic processes.
- Stochastic processes (random signals) as inputs to
LTI systems; properties of Gaussian processes;
white noise and filtered white noise.
- Basics of data transmission systems: 2PAM, OOK,
M-PAM, M-PSK. Transmitter and receiver
structures, error probability, bandwidth.
- [to be confirmed] PCM: quantization noise,
performance of a uniform quantizer.
- [to be confirmed] Analog transmission: DSB, AM,
PM, FM
1.1.5 Problem-solving sessions
Problems involving the theoretical notions imparted during
lectures will be solved by the teachers in class.
This course typically has no formal homework assignments.
We will propose a few problems for you to solve on your own
but there will be no formal grading of such informal
homework.
Problems will try and address real-world applications of the
theory learned in class.
1.1.6 Material
The course theory is fully covered by the class notes, which
will be made available on the course. As supplementary
material, the following books can be consulted by the students.

A.V. Oppenheim, R.W. Schafer, "Discrete-Time Signal
Processing", Prentice-Hall, 1989.

A. Papoulis, "Probability, Random Variables and Stochastic
Processes", Mc Graw Hill, 1984.

W. Gardner,"Introduction to Random Processes with
Applications to Signals and Systems", Mc. Graw Hill, 1990.
1.1.7 Exams
No midterms are foreseen. Final exams take place as follows.
The students are given a short problem to solve by themselves,
with no books available. The allotted time is between half an
hour and one hour. Following, the test is oral and includes a
discussion of the solution to the problem, as worked out by the
student. Further questions will then be asked orally on any of
the topics dealt with in the course.
1.1.8 Extra points
Extra points will be awarded to students providing well-
documented, well written and useful Matlab or Mathematica
programs exemplifying any topic related to the course syllabus
or documenting the solution of a problem. Visual output in
the form of plots is requested (wherever applicable). Check
with the teachers to discuss your ideas before undertaking
them.
1.2 Class Schedule
The course has regular hours in room 1B on:
- Monday, 2:00 PM 6:00 PM
- Tuesday, 9:00 AM 1:00 PM
- Thursday, 2:00 PM 6:00 PM

Certain lectures may be canceled or re-scheduled. If you do
not attend lectures regularly, check on the website for possible
rescheduling.
1.2.1 Consulting
You can come and ask us questions on Thursdays, 12:00
(noon) to 1 PM. However, consulting is by appointment, that is
you have to let us know in advance, for instance by e-mail or
during a previous lecture.
Chapter 2. Signals
2.1 What is a signal?
A signal ( ) s t is, from a mathematical viewpoint, a real or a
complex function of the independent variable t e . In other
words, we typically have:

( ) : s t
( ) : s t

The domain of the independent variable t can be sometimes
restricted to a specific interval, i.e.,
0 1
[ , ] t t t e , whereas the co-
domain is typically all of or . A signal ( ) s t can be
discontinuous, continuous, or differentiable any number of
times. In this course, we will use signals that have different
properties in this respect.
2.2 Sampled and quantized signals
With the advent of computers and of digital signal
processing, new types of signals have emerged. Specifically,
the so-called sampled or discrete-time signals and the
quantized or discrete-value signals.
All of these special signals can be given a mathematical
representation in terms of a conventional function of time.
However, for practical reasons, more compact notations may
be used, which will be introduced later on.
2.3 What are signals used for?
A signal ( ) s t is typically used in engineering applications to
represent the change over time of some physical quantity, such
as a current, voltage, electric field, sound pressure,
temperature, a spatial coordinate, a force, and so on. As a
result, signals are used in all engineering branches and the
theory of their properties and how to use or analyze them is
fundamental to many if not all engineering sectors.

2.4 Frequently-used signals
In this section, several signals that will be used very
frequently in this course are introduced. The independent
variable t spans all of , unless otherwise pointed out.
2.4.1 The rectangular signal
The rectangular signal is defined as follows:
( )
1, / 2
R 0, / 2
1
, / 2
2
T
t T
t t T
t T

<

= >

.
Note that this signal is discontinuous for two values of t , and
is continuous everywhere else. The subscript to the symbol R
indicates the length of the time interval over which the signal is
different from zero, which is obviously T .
t
R ( )
T
t
2
T

2
T
1
2
1
2
1
t
R ( )
T
t
2
T

2
T
1
2
1
2
1

Figura 2.1: The rectangular signal
( )
R
T
t
2.4.2 The triangular signal
The triangular signal is defined as follows:
( )
2 / 1 0
2
Tri 1 2 / 0
2
0
2
T
T
t T t
T
t t T t
T
t

+ < s

= < <

>

.
The shape of the signal is that of an isosceles triangle, of base
T and height 1. The vertex is located at 0 t = . The triangular
signal is continuous everywhere, but is not differentiable, as its
first derivative is discontinuous at 2, 0, 2 t T T = .
t
Tri ( )
T
t
2
T

2
T
1
t
Tri ( )
T
t
2
T

2
T
1

Figura 2.2: The triangular signal
( )
Tri
T
t
2.4.3 The unilateral step signal
t
u( ) t
0
1
2
1
t
u( ) t
0
1
2
1


( )
0 0
1
u 0
2
1 0
t
t t
t
<

= =

>

2.4.4 The sign signal
t
sign( ) t
0
1
1
t
sign( ) t
0
1
1


( )
1 0
sign 0 0
1 0
t
t t
t
<

= =

>

2.4.5 The sine and cosine signals
-5 -4 -3 -2 -1 0 1 2 3 4 5
[s] t
sin( ) t 1
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
[s] t
sin( ) t 1
1

-5 -4 -3 -2 -1 0 1 2 3 4 5
[s] t
cos( ) t
1
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
[s] t
cos( ) t
1
1


The sine and cosine signals are perhaps the most important
signals that we use in this course. Their plots as a function of
time are shown above. Typically, however, their argument is
not simply t . Time is almost always multiplied by a constant,
and in fact a very special one, called frequency.
2.4.5.1 The concept of frequency
In Signal Theory the sin and cos signals are typically
expressed as:
( ) ( )
0 0
sin 2 , cos 2 f t f t t t , where the costant
0
f
is called frequency. Note that the subscript 0 is used only
to remind the reader that, within
( )
0
sin 2 f t t and
( )
0
cos 2 f t t ,
0
f is a constant, whereas t is the independent variable.
The meaning of frequency can be explained in various
equivalent ways, but it has to do fundamentally with the fact
that both sin and cos are periodic signals, that is, they
repeat themselves at regular intervals. Specifically, they
repeat themselves every 2t :
( )
( ) sin sin 2 x x k t = +
( )
( ) cos cos 2 x x k t = +
where k can be any integer number.
Given this, it is easy to see that in
( ) ( )
0 0
sin 2 , cos 2 f t f t t t :
1.
0 0
1/ T f = is how long it takes (in time) to accumulate
2t in the argument, that is how long it takes before
the sin or cos signals start repeating themselves
(prove it as an exercise);
2.
0
f tells us how many repetitions of the sin or cos
signals occur in one second (prove it as an exercise);
note that
0
f need not be an integer number because,
for instance, we can have 5 and repetitions of the
sin signal in one second.

Note that
0
f has dimensions of [s
-1
], or [Hz]. Also, the
overall argument of the sin and cos signals should always be
dimensionless: writing ( ) sin t assumes that in fact we are
looking at
( )
0
sin 2 f t t with
0
1/ 2 f t = .
Note finally that instead of
0
f you can sometimes find
0
e ,
with the following meaning:
( ) ( )
0 0
sin 2 sin f t t t e = . The
relation between the two is therefore:
0 0
2 f e t = .
The use of
0
e allows to avoid explicitly writing the 2t
factor. However, even though it may be cumbersome to always
write the 2t factor, in Signal Theory the use of
0
e brings
about a number of notational problems which largely
overwhelm the advantage of dropping the 2t . Therefore in
this class we will always use
0
f .
An incredible array of physical phenomena show a periodic
or oscillatory behavior. In fact, the sin and cos functions or
signals are truly fundamental to all sciences, and frequency is
an extremely important related concept.
2.4.6 The complex exponential signal
As important as sin and cos is the complex exponential
signal. As is well-known, the following rule holds, called
Eulers rule:
( ) ( )
exp( ) cos sin ,
jx
e jx x j x x = = + e
It is also well-known that
jx
e spans a full circle in the
complex plane, of radius one, for x spanning the interval
0, 2t . In addition,
jx
e is periodic in x , that is it repeats itself
every 2t . This property is obviously inherited from the cos
and sin functions that make it up.
As a result, we can introduce a complex signal, incorporating
the concept of frequency, as for the cos and sin, as follows:
( ) ( )
0
2
0 0 0
exp( 2 ) cos 2 sin 2
j f t
e j f t f t j f t
t
t t t = = +
In the context of the complex exponential signal, the meaning
of frequency can be viewed in yet another way:
3.
0
f tells us how many turns the complex point
0
2 j f t
z e
t
= executes in one second over a circle of
radius 1 in the complex plane (prove it as an
exercise); note that
0
f need not be an integer number
because, for instance, we can have 5 and turns in
one second.

The complex exponential is no less important than its
components sin and cos. In fact, it is even more important, as it
allows to handle many calculations more effectively than using
sin and cos.
2.4.7 The Sinc signal
The sinc signal is defined as follows:
( )
sin( )
Sinc .
t
t
t
=
Both the numerator and the denominator are continuous
functions. However, for 0 t = an undetermined form develops
of the 0 0 type. This problem can be solved by conventionally
assigning to Sinc(0) the limit for 0 t of the Sinc itself. It is
easy to see that:

0 0
sin( )
limSinc( ) lim 1
t t
t
t
t

= =

therefore the complete definition of the Sinc function is:

( )
sin( )
, 0
Sinc .
1, 0
t
t
t
t
t

With this definition, the Sinc function becomes both
continuous and differentiable. In particular, it can be
differentiated any number of times.
In typical signal theory applications, the Sinc function
argument is written as
t
T
t
| |
|
\ .
:
sin
Sinc
t
t
T
t
T
T
t
t
t
| |
|
| |
\ .
=
|
\ .
.
The resulting Sinc has nulls at all multiples of . T In other
words:
Sinc 0, , 2 , 3 ,...
t
t T T T
T
t
| |
= =
|
\ .

An important property of the Sinc signal is the following
(prove it as an exercise):
1
Sinc
t T
T t
t
t
| |
s
|
\ .

This means that the Sinc signal does tend to go to zero as
t , but it does so very slowly, only as fast as T t t . This
can also be seen in Figura 2.4
-11T -10T -9T -8T -7T -6T -5T -4T -3T
-2T
-T
0
T
2T
3T 4T 5T 6T 7T 8T 9T 10 11T
[s] t
Sinc
t
T
t
| |
|
\ .
1
-11T -10T -9T -8T -7T -6T -5T -4T -3T
-2T
-T
0
T
2T
3T 4T 5T 6T 7T 8T 9T 10 11T
[s] t
Sinc
t
T
t
| |
|
\ .
1


Figura 2.3: Plot of the Sinc
t
T
t
| |
|
\ .
signal
-15 -14 -13 -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-0.5
0
0.5
1
1.5

Figura 2.4: Plot of the Sinc
t
T
t
| |
|
\ .
signal together with T t t for
1 T =
2.4.8 The single-sided decreasing exponential
signal
Another important signal is the so called single-sided or
unilateral decreasing exponential signal. Its analytical
expression is:
u( ) , 0
at
t e a a

e >
Note that per se the exponential functions of a real negative
argument would be non-zero over all , but the presence of
the unilateral step signal makes this signal non-zero only for
0 t > .
Note that this signal is discontinuous at 0 t = .
What is its actual value for 0 t = ? Find out. Also, plot this
signal using Matlab or similar software.
2.4.9 Gaussian signal
Another important signal is the Gaussian signal. Its
expression is:
2
2
2
G( ) , 0
t
T
t e T T

= e >
Of course the Gaussian function is very important in statistics
as well. In that context it is generally normalized differently.
Note the following:
0 G( ) 1
G( ) 0.6
2 G( ) 0.14
3 G( ) 0.011
10 G( ) 0.000000000000000000002
t t
t T t
t T t
t T t
t T t
= =
=
=
=
=

The Gaussian signal, though mathematically non-zero over
the whole of , in fact vanishes very fast.
2.4.10 The constant unit signal and the constant
signal
A constant unit signal is a signal whose value is 1 at all times.
We will write it as 1( ) t . From this elementary signal, arbitrary
valued constant signals can be derived by simply multiplying
1( ) t by a number o :
const( ) 1( ) t t o =

2.4.11 The constant zero signal
A constant zero signal is a signal whose value is zero at all
times. We will write it as 0( ) t . Note that even though it is
often written in textbooks as just 0, the signal 0( ) t is quite a
different object than the number 0.
2.4.12 On your own
On your own, look at how the above signals behave for
t . What rough classes of signals can you identify? Also,
try and classify them in terms of continuity and
differentiability.
All of these signals can be re-scaled and translated left and
right. How do you do it? Draw a few examples on your own.
2.5 Diracs delta
Diracs delta formal writing ( ) t o may lead to mistaking
it for a conventional signal, that is, a function of time.
However, Diracs delta is not a proper function, because its
value for 0 t = is undefined, even though informally such value
is said to be infinity.
In fact, Diracs delta is an object that acquires actual meaning
only within an integral operator. The actual definition of
Diracs delta can be written resorting to its essential integral
property: the ( ) t o is a mathematical object such that:

( ) ( ) (0) t s t dt s o
+

Note that this is true only if ( ) s t is continuous at 0 t = . We
assume so, for the moment.
Note also that as a corollary of this property, by choosing
( ) 1( ), , s t t t = e we have:

( ) 1 t dt o
+

These integral definitions, however, do not provide much
insight or intuition regarding the concept of the delta and
therefore it is useful to resort to one of the many possible ways
of arriving at the delta as a sort of limit (in a special way) of
a sequence of ordinary signals.
To show how this is done, we start out with a claim and then
try and justify this claim.
Claim:
the following equality holds:

( )
0
R
lim ( ) (0)
T
T
t
s t dt s
T
+

=

Eq. 2.1

Note that Eq. 2.1 seems to directly suggest that:
( )
0
R
lim ( )
T
T
t
t
T
o

=
but we forewarn the reader that this is not correct in a strict
mathematical sense. We will come back to this issue, after
proving Eq. 2.1.

Proof of claim:
We start out by writing down a well-known equality:

/ 2
/ 2
( ) min{ ( )}, max{ ( )}
T
T
s t dt T s t s t
+

I I


where the integration interval is
| | 2, 2 T T = I and the min
and max of ( ) s t are taken over the same integration interval I .
We also assume that ( ) s t is continuous over I . This equality is
quite easy to prove graphically, which we leave up to you to do
on your own.
We then remark that we can also write:
( )
/ 2
/ 2
( ) R ( )
T
T
T
s t dt t s t dt
+

=


so that we immediately have:
( )
R ( ) min{ ( )}, max{ ( )}
T
t s t dt T s t s t

I I

and also:
( )
R
( ) min{ ( )}, max{ ( )}
T
t
s t dt s t s t
T

I I

We now take the limit for 0 T of both sides of the
equation:
( )
0 0
R
lim ( ) lim min{ ( )}, max{ ( )}
T
T T
t
s t dt s t s t
T

I I

If we look at the right-hand side it is obvious that:
0
lim min{ ( )}, max{ ( )} (0)
T
s t s t s

=

I I

Therefore, it is proved that:
( )
0
R
lim ( ) (0)
T
T
t
s t dt s
T
+

=

Eq. 2.2
If we now remember that:
( ) ( ) (0) t s t dt s o
+

Eq. 2.3
then by comparison with Eq. 2.2 we can write:

( )
0
R
lim ( ) ( ) ( )
T
T
t
s t dt t s t dt
T
o
+ +

=


This equality is very important and shows that ( ) t o is perhaps
not such a mysterious object. Rather, it appears to be the
limit of a simple rectangular function of the form:
( )
R
T
t
T
.
for 0 T . One could in fact be tempted to write:
( )
0
R
lim ( )
T
T
t
t
T
o

=
Unfortunately, this limit is meaningless, in the sense of
conventional functions, since, as pointed out, ( ) t o is not a
properly defined function. Nonetheless, the integral identity:
( )
0
R
lim ( ) ( ) ( )
T
T
t
s t dt t s t dt
T
o
+ +

=


does have a mathematical meaning in the framework of the so-
called theory of generalized functions or distributions. Diracs
delta is not a function but, within this theory, it is a
generalized function or distribution. Then, in the
framework of such generalized functions, one can write:
( )
0
R
lim ( )
T
T
t
t
T
o

=
where this new limit operator limis said to be in the sense of
distributions. Apart from the mathematical details, what this
limit operator really means is that:

- the
( )
R
T
t
T
family of functions tends to acquire the same
integral properties as the Diracs delta, in the limit of
0 T .

Note that there are many families of signals that have the
same properties, such as the Gaussian signal or the triangular
signal. In fact, it can be shown that:
( )
2 2 2
0
lim exp / 2 2 ( ) t t
o
o to o

= .
( )
0
2 Tri
lim ( )
T
T
t
t
T
o

=
Less obvious, but very important in practice, also the Sinc
signal can be made to generate ( ) t o :
0 0
sin
1
lim Sinc lim ( )
T T
t
t
T
t
T T t
t
t
o
t

| |
|
| |
\ .
= =
|
\ .

2.5.1.1 computational rules
There are some useful immediate extensions of the basic
integral property of Diracs delta. It can be easily seen that:

0 1 0 1
( ) ( ) ( ) t t s t t dt s t t o
+

More in general, to be able to solve an integral involving a
delta with a complex argument, it is enough to:
1) find out what the integration variable is (it is the one
appearing in the differential, in this case t)
2) check whether the integration variable appears in
the deltas argument without any multiplicative
constants (if it does, the rule below does not work
and you need to first apply a change of integration
variable to remove the multiplicative constant)
3) find the value of the integration variable for which
the argument of the delta is zero; in the example
above (t t
0
= 0 t = t
0
);
4) the result of the integral is then the integrand
function (excluding delta) evaluated at t = t
0
; in the
example above it clearly yields
0 1
( ) s t t .

Notice the time-sampling property of the delta function:
0 0
( ) ( ) ( ) t t s t dt s t o
+

In other words, by integrating a signal together with a delta
function centered at a specific time instant
0
t , one can
extract from the signal the value that the signal itself takes on
at
0
t . That is, the signal ( ) s t is sampled at the instant
0
t .
From the property above, we can also conclude that:
0 0 0
( ) ( ) ( ) ( ) t t s t t t s t o o =
Why? Can you prove this? (Remember that the delta function
is defined in terms of its integral behavior so perhaps the above
identity should be checked under integration.)
The delta function has other important properties in
connection with the convolution integral. We will look at
convolutions in depth, later on. However, it is important to start
memorizing the following property, also called the time-
shifting property of Diracs delta:
0 0
0 0
( ) ( ) ( ) ( )
( ) ( ) ( )
s t t t t s t d
t t s d s t t
o o t t t
o t t t
+

- =
= =

By convolving any signal ( ) s t with
0
( ) t t o , the signal ( ) s t
gets time-shifted (that is, delayed) by an amount
0
t . Note also
the degenerate but important case:
( ) ( ) ( ) ( ) ( ) ( ) ( ) s t t s t d t s d s t o o t t t o t t t
+ +

- = = =


whereby nothing happens to ( ) s t . This is why ( ) t o is also
called the unit signal of the convolution product:
multiplying (under convolution product) any signal by ( ) t o ,
nothing happens, similarly to what happens for a conventional
product when multiplying any number times 1 (the unit for
the conventional product): the number does not change.
Chapter 3. Energy and
Power of Signals
Signals often represent physical quantities to which a
physical power, or energy, can be associated. These concepts
can also be given a strictly mathematical definition, not
necessarily related to any physical phenomenon. However, the
mathematical definitions are chosen so that they are consistent
with the physical definitions, so that if signals actually
represented physical quantities, then the physical and
mathematical definitions would coincide.
In this chapter we first introduce the important concept of
time-average and then provide the basic mathematical
definitions of energy and power for signals. We also show
a few examples, some of which are related to physical
phenomena.
Finally, we introduce a special signal classification based on
energy and average power.
3.1 Time Averages
Let us define the time average of a signal over a time
interval
| |
0 1
, t t = I as follows:
| |
1
0 1
0
,
1 0
1
( ) ( ) ( )
t
t t
t
s t s t s t dt
t t
= =

I
.
The result of the average operator is clearly not a signal but
just a number.
The average can be given an intuitive meaning. If we
consider a constant signal 1( ) t o over the interval
| |
0 1
, t t = I ,
whose constant value is set equal to the average of ( ) s t , that is
| | ,
0 1
( )
t t
s t o = , then we have that such constant signal has the
same time-average of ( ) s t :
| | | |
0 1 0 1
, ,
( ) 1( )
t t t t
s t t o o = =
This may appear as obvious but in many practical cases this
is of interest. For instance, when a cause and an effect are
related through an integral operator over time, the time-average
tells us what the value is of a constant cause that would have
the same effect as the actual time-varying cause.
This will be clarified through this example.
Example
There is a water tank which is filled by a faucet whose water
output is irregular (not constant over time) and is represented
by a function of time ( ) s t [l/s] (liters per second). Assuming
the tank was empty at time
0
t , we want to compute the total
water accumulated in the tank over the interval:
| |
0 1
, t t = I .
Simple physical arguments suggest that the water
accumulated in the tank at time
1
t will be:
1
0
( )
t
t
W s t dt =

where W is liters [l].
We now would like to know what constant water output
would have the same effect. That is, assuming the faucet had a
constant output o [l/s], what value should o have to
accumulate the same amount of water W over the same interval
| |
0 1
, t t = I ?
The answer is quite simply that the value of o should be
equal to the average water output of ( ) s t over
| |
0 1
, t t = I , that
is:
| | ,
0 1
( )
t t
s t o = . In other words, a faucet with constant output
1( ) t o would accumulate the same number of liters W in the
tank as ( ) s t , over
| |
0 1
, t t = I .

Averages can be extended at will, over larger and larger
intervals. However, if one wants to actually extend an average
over the whole of , then a problem is incurred: the factor
( )
1 0
1 t t goes to zero as
0
t and
1
t +. So, it is
necessary to introduce a limit operator:
1
0 0
1
1 0
1
( ) lim ( )
t
t t
t
s t s t dt
t t

+
=

When averaging over the whole of it is handy to define a
single time parameter T , which can replace
0 1
, t t since there is
no need to keep
0 1
, t t distinct as they go to . Doing so we
get:
2
2
1
( ) lim ( )
T
T
T
s t s t dt
T

=


Note that there is no guarantee that such an average
converges to a finite value.
3.2 Instantaneous and Average
Power
Let us define the instantaneous power of a signal ( ) s t as:
2
( ) ( )
s
P t s t =
Although we introduce it as a mathematical definition, it is
consistent with many physical situations. For instance,
assuming ( ) s t to be a current in Ampere [A], then the
instantaneous power delivered by such a current into a resistor
R would in fact be:
2
( ) ( )
s
P t s t R =
The constant resistance R is there to adjust dimensions and
make the result appear as Watts [W], but the key fact is that
power is physically proportional to the square of the current
signal, and this is consistent with the mathematical definition
that we have given.
Note that ( )
s
P t is a signal itself, so we can for instance take
its time-average. By combining the definition of instantaneous
power of a signal and that of time-average, we obtain the
average power of a signal ( ) s t over a certain interval
| |
0 1
, t t = I as:
| |
1 1
0 1
0 0
2
,
1 0 1 0
1 1
( ) ( ) ( )
t t
s s
t t
t t
P t P t dt s t dt
t t t t
= =


.
As shown in Sect. 3.1 above, time-averages can be extended
to the whole of . This allows us to introduce the very
important concept of average power of a signal ( ) s t over all
time:
2
2 2
2 2
1 1
( ) lim ( ) lim ( )
T T
s s
T T
T T
P t P t dt s t dt
T T

= =


Again, there is no guarantee that this limit converges to a
finite value.
3.3 Energy
In Physics, energy is the integral of instantaneous power over
a certain time interval. In signal theory we adopt the same
definition, so:
| |
{ }
1 1
0 1
0 0
2
,
( ) ( ) ( )
t t
s t t
t t
s t P t dt s t dt = =

E
Eq. 3.1
where
| |
{ }
0 1
,
( )
t t
s t E is the energy operator extracting the
energy of ( ) s t over the interval
| |
0 1
, t t = I .
Energy too can be computed over the whole of . In this
case, differently from average power, we do not need the
limit operator because we do not have the factor 1/ T before
the integral. So, we simply extend the integration range to the
whole of :
{ }
2
( ) ( ) ( )
s
s t P t dt s t dt


= =

E
Again, depending on the signal, this integral may or may not
converge.

On your own, show that the following relationship holds:
| |
{ }
| |
( )
0 1
0 1
1 0 ,
,
( ) ( )
s t t
t t
s t P t t t = E
In other words, the energy delivered over an interval
| |
0 1
, t t = I is equal to the average power over the same interval,
times the length (in time) of the interval.
Also, try and think whether the relation:
| |
{ }
| |
( )
0 1
0 1
1 0 ,
,
( ) ( )
s t t
t t
s t P t t t = E
is still valid for
| |
0 1
, t t = I . What happens?

Example
Compute the energy of the unilateral decreasing exponential
signal over the whole of :
u( ) , 0
at
t e a a

e > R
We directly apply the definition:
{ }
2
2
2
2 2
0 0
0
( ) ( ) ( ) u( )
1 1
2 2
at
s
at at at
s t P t dt s t dt t e dt
e dt e dt e
a a

= = =

= = = =


E


On your own: what can you say about the average power of a
signal whose energy, computed over the whole of , is finite
and non-zero? After thinking about it, read on and see if your
thoughts agree with what is written next.

3.4 Classification of Signals Based
on Average Power and Energy
Signals are typically divided into various classes depending
on their energy and power properties. These properties will
have a big impact on certain key aspects of signal analysis and
representation that will be introduced in the next chapter.
3.4.1 Finite energy
3.4.1.1 finite energy over a finite interval
Finite energy signals over a finite interval
| |
0 1
, t t = I are all
those signals for which:
| |
{ }
1
0 1
0
2
,
( ) ( )
t
t t
t
s t s t dt = <

E
All physical signals must satisfy this condition.
As a sufficient condition for a signal to be finite-energy, it is
enough that:
| |
2
0 1
( ) , , s t t t t < e = I .
Note that this condition must be satisfied at the extremes of
the interval, too. The same sufficient condition could also be
written in terms of the instantaneous power of the same signal:
| |
0 1
( ) , ,
s
P t t t t < e = I .
This means that every signal whose instantaneous power is
finite at all times over
| |
0 1
, t t = I also has finite energy over the
same interval.

On your own: try and find the mathematical expression of at
least one signal for which
| |
{ }
0 1
,
( )
t t
s t E is not < . Then verify
that such signal does not satisfy the sufficient condition for
finite-energy given above.
3.4.1.2 finite energy over the whole of
Finite energy signals over the whole of are all those
signals for which:
{ }
2
( ) ( ) s t s t dt
+

= <

E
3.4.2 Finite average power
3.4.2.1 finite average power over a finite interval
Finite-average-power signals over a finite interval
| |
0 1
, t t = I
are all those signal for which:
| |
1 1
0 1
0 0
2
,
1 0 1 0
1 1
( ) ( ) ( )
t t
s s
t t
t t
P t P t dt s t dt
t t t t
= = <



Note however that average power and energy are related
through this obvious formula:
| |
| |
{ }
( )
0 1
0 1
,
,
1 0
( )
( )
t t
s
t t
s t
P t
t t
=

E

Since, given a finite interval
| |
0 1
, t t = I , clearly:
( )
1 0
0 t t < < , then it turns out that a signal ( ) s t that has
finite energy over an interval
| |
0 1
, t t = I also has finite average
power over the same interval, and vice-versa. In other words,
all signals that have finite energy over
| |
0 1
, t t = I also have
finite average power over
| |
0 1
, t t = I (and vice-versa).
Put another way, the set of all signals that have finite energy
over
| |
0 1
, t t = I coincides with the set of all signals that have
finite average power over the same interval. This situation is
depicted in Fig. 3.1

Set of finite-energy signals over
Set of finite-average-power signals over
| |
0 1
, t t = I
| |
0 1
, t t = I

Fig. 3.1
3.4.2.2 finite average power over the whole of
A finite average power signal over the whole of is such
that:
2
2 2
2 2
1 1
( ) lim ( ) lim ( )
T T
s s
T T
T T
P t P t dt s t dt
T T

= = <



In the previous section (Section 3.4.2.1), we have seen that
the set of finite energy signals and the set of finite average
power signals coincide, as long as the interval
| |
0 1
, t t = I is
finite. However, when the time-interval extends to all of ,
the set of finite energy and finite average power signals do not
coincide anymore.
We have the following results:
1. a finite-energy signal over the whole of has zero
average power over the whole of
2. a finite-power signal over the whole of has infinite
energy over the whole of
This situation is depicted in Fig. 3.2. Note that the set P E,
meaning the set that includes of elements of P except those
that also belong to E , consists of all signals with finite-
average-power over , whose average power is non-zero.

The two sets DO NOT coincide
E: set of finite-energy signals over
P: set of finite-average-power signals over
E
P

Fig. 3.2

On your own: try the above results on a few signals.
Compute the energy and average power (both over ) for
the following signals:
10
R ( ) t ,
10
Tri ( ) t ,
( )
0
cos 2 f t t , sign( ) t ,
u( )
t
t e

. In your opinion, why do the results (1) and (2)

above occur? What is the difference between finite-energy
and finite-power signals (over ), in simple words?

3.4.3 Energy and power of periodic signals
For a periodic signal, the energy of the signal is the same
over any interval whose length is exactly one period,
independently of where the interval starts. Specifically, given a
periodic signal ( ) s t of period T then :
| |
{ }
0 0
0 0
0 0
2
,
( ) ( ) ( )
t T t T
s t t T
t t
s t P t dt s t dt
+ +
+
= =

E
this expression is a constant,
0
t .
On your own: try and prove it.

As a result, the average power over a period is a constant as
well, whose value is:
| |
| |
{ }
0 0
0 0
,
,
( )
( )
t t T
s
t t T
s t
P t
T
+
+
E
=
On your own: try and prove it. Also on your own: what is the
energy of a periodic signal over the whole of ? What is the
average power of a periodic signal over the whole of ?



1
Power and Energy Power and Energy



Chapter 4.
Signal Spaces
In this chapter we will show that signals can be structured
in a way quite similar to vectors in
n
or
n
. By this we mean
that similar operators to those that can be applied to vectors
in
n
or
n
can also be applied to signals. Among such
operators: the sum, the product times a real or complex
number, and the so-called inner-product, or scalar product.
Thanks to this, it will be possible to treat suitable sets of
signals as linear (or vector) spaces and scalar product spaces.
We will find out that under certain conditions, certain sets of
signals may even form a Hilbert space.
We will then see that it is possible to project a signal onto a
set of orthonormal signals, like in a
n
or
n
a vector can
be projected onto a set of orthonormal vectors (called unit
vectors). Also, like in
n
or
n
, a basis for the set of signals
can be found.
These specific aspects pave the way for a large number of
applications in the real world, especially in the realm of digital
transmission and digital signal processing, including modern
compression techniques.
4.1 Linear Spaces
A linear space is a set of elements such that a sum
operator and a multiplication by a number operator can be
defined in such a way that they satisfy certain properties.
The theory of linear spaces is well described in countless
textbooks and will not be repeated here. It is however
important to recall at least a few fundamental notions regarding
such operators.
Given any two elements , x y e , then the result of their
sum:

z y x = +

must still be an element of , i.e., z e .
The multiplication by a number operator requires that,
given any x e and o e then:

z x o =

must still be an element of , i.e., z e .
Note that these may seem quite obvious properties because
we are accustomed to such linear spaces as
2
or
3
, which
we know well and are simple to deal with. However, for
generic spaces whose elements may be quite unusual
objects, this fundamental property may not be so
straightforward.
One simple example of a linear space is indeed
2
, the
Euclidean space of two-dimensional vectors lying on a plane.
In this space, it is straightforward to define the sum and
product operator in such a way that their results is still an
element of
2
. In particular, the sum of two vectors can be
geometrically defined through the parallelogram rule, which
clearly always yields another vector lying on the same plane.
As for the product operator, it can be defined as a new vector
whose direction is the same as the old one and whose length (or
magnitude) is the old one multiplied by o . If o is negative,
the direction is reversed. In any case, the result is clearly still a
vector on the same plane.
Note that if o is allowed to be complex, i.e.,o e , then the
simple parallelogram rule cannot be used anymore. Same is
true for the simple multiplication rule. Extensions to handle a
complex o can be defined, but this shows that even obvious
properties are in fact not so obvious as soon as one departs
from simple objects such as vectors in
2
.
The sum and multiplication operators are not independent
in the sense that they must be able to interact according to
certain properties, the most important of which is the
distributive property of the multiplication over the sum. It
states that given o e , , x y e , then the following must be
true:
( ) y x y x o o o + = +
Using the parallelogram rule and the rule of multiplication, it is
quite easy to see that this property is satisfied if
2
= . Once
again, just by allowing o e things are not quite so simple
and alternative rules of sum and multiplication must be found.
4.2 Signals and Linear Spaces
Let us now look at signals to see whether a properly defined
set of such signals can be viewed as a linear space. First of all,
let us define as the set of all signals, i.e., of all proper
functions of time, whose domain is
| |
1 2
, t t = I . This set can be
easily shown to be a linear space because a sum and
multiplication operator satisfying the necessary properties can
be defined in a very simple way. Given the signals
( ), ( ) x t y t e , then their sum is written as:

( ) ( ) ( ) z t x t y t = + ,

where the + operator operates as the sum of real or complex
numbers. It is then quite obvious that ( ) z t e as well. Note
however that, though intuitive, this definition of the sum
operator is not so trivial. The formula means that for each time
t , we must assign to the signal ( ) z t the sum of the values of
the signals ( ) x t and ( ) y t at the same time t . So this is more
than a simple sum, it is in fact a rule for executing an infinite
number of sums, one for each
| |
1 2
, t t t e .
Also, the rule ( ) ( ) z t x t o = clearly yields ( ) z t e . Finally,
there is no doubt that:
( ) ( ) ( ) ( ) ( ) x t y t x t y t o o o + = + .

Since we have discovered that the signal set is a linear
space, we can use the vector notation in as well. Given a
signal ( ) x t e we can also write it as x , keeping in mind the
equivalence ( ) x x t (the symbol means is equivalent
to). We can therefore write:
z x y = +
z x o =

keeping in mind that the former means ( ) ( ) ( ) z t x t y t = + and
the latter means ( ) ( ) z t x t o = .
These definitions turn into a linear space, though some
caution is in order. For instance, when a signal has a singularity
for some
s
t t = , such singularity must be discussed and
eliminated. One example is the signal 1/ t at 0 t = . Note
however that 1/ t is perfectly defined for any t other than
0 t = . So we only need to assign a conventional value to 1/ t
for 0 t = to solve the problem. For instance, the value 0 could
be selected. The new signal:

{
1/ 0
( )
0 0
t t
s t
t
=
=
=


is then defined for all t e and can be summed with any other
signal and multiplied by any number.
4.3 Scalar Product Spaces
If a Linear Space is such that for all pairs , x y e it is
possible to compute a so-called scalar product, then is
also a scalar product space, which we will call H.
Typical examples of scalar product spaces are the customary
Euclidean spaces
n
or the complex space
n
.
These spaces are introduced in other courses and will not be
dealt with in detail here. However, we recall here certain key
properties of the scalar product and of scalar product spaces.
The scalar product of any two elements
, x y eH
can be
computed according to the native rules of the space H,
which are typically different depending on the nature of the
space. For instance, in
2
the native rule for the scalar
product is geometric and states that
( ) , x y
is equal to the
product of the length of the vector x (which you could
measure using a ruler) times the length of the vector y ,
times the cosine of the angle between the two vectors, as you
could measure it using a goniometer.
4.3.1 Properties of the scalar product
The scalar product takes two elements of a scalar product
space and returns a number:
( )
, x y o =
where o e or o e depending on the space being real or
complex. Such product may be defined in different ways,
depending on the nature of the elements of the space. In all
cases it must have the following key properties.

Property 1 Conjugate commutativity
Given
( )
, x y o = , then
( )
, y x o
-
= , for any , x y eH.

Property 2 Product of an element by itself
( )
, 0, , x x u u = > e
for all x eH.
In fact, there is only one element of the space for which:
( )
, 0 x x =
Such element is called the zero element of the space, or 0 .
Note that, for all x eH, it must be:
( ) ( )
, 0 0, 0 x x = =

Property 3 Distributivity
The scalar product is distributive:
( ) ( ) ( )
, , , x y z x z y z o | o | + = +
Note however that due to Property (1) there is a difference
between the left and right argument distribution property:
( ) ( ) ( )
, , , z x y z x z y o | o |
- -
+ = +

Based on the scalar product, we can define certain important
quantities. Two of them are the the norm and the energy.
4.3.1.1 norm of an element in H
The norm of an element of a scalar product space is defined
as:
( )
, x x x =
Due to Property 2 of the scalar product,
( )
, , 0 x x e > and
therefore the norm
( )
, , 0 x x x = e > too.
Elements in H whose norm is 1 are called unit elements, or
normal elements. Normal elements are often represented with a
hat on top, rather than the usual bar. In other words writing
x implies that
( )
, 1 x x x = =
4.3.1.2 energy of an element in H
The definition of energy of an element in H is:
{ }
( )
, x x x = E
Eq. 4.1
Clearly, norm and energy are strictly related:
{ }
2
x x = E
Due again to Property 1 and 2 of the scalar product, energy is
always real and positive, as one would expect it to be in
accordance with physical rules.

4.3.1.3 distance between two elements in H
A very important quantity derived from the norm is the
distance of two elements of a scalar product space. Given
, x y eH, then the distance between these two elements is
defined as:
{ } , x y x y A =

4.3.1.4 orthogonality between two elements in H
Given , x y eH, , 0 x y = , they are said to be orthogonal, if:
( )
, 0 x y =
which can be written in short as: x y

4.3.2 Orthonormal sets in H
A set of normal elements in H that are all mutually
orthogonal is said to form an orthonormal set. In other words,
an orthonormal set of N orthonormal elements of H is as
follows:

{ }
( )
1 2
, ,...,
,
N
n k nk
u u u
u u o

=


where we have used the symbol
nk
o , which is called
Kroneckers delta, to mean:

{
0 if
1 if
nk
n k
n k
o
=
=
=


It is then easy to see that the condition
( )
,
n k nk
u u o = ensures
that all elements in are orthonormal.
4.3.2.1 projecting an element over an orthonormal
set
Given any x eH and an orthonormal set eH it is always
possible to project x over . By this, we mean computing
N scalar products, one for each element in , as follows:
( )
, , 1, 2,...
n n
x x u n N = =

The number
n
x is called the projection or component of x
with respect to the unit element
n
u .
The N components of x with respect to the N unit elements
in are often written as an N-tuple, as follows:
| |
1 2
, ,...,
N
x x x
4.3.2.2 reconstructing an element from its
components
Using the components
n
x of x with respect to , we
construct an element of H as follows:
app 1 1 2 2
1
...
N
N N n n
n
x x u x u x u x u
=
= + + + =

It is clear that
app
x must have some resemblance to x and
this is why the subscript is app which stands for
approximated. For instance, it is clear that the projection of
both x and
app
x over produce the exact same N-tuple
| |
1 2
, ,...,
N
x x x (prove it on your own).
However, the element
app
x is, in general, different from x .
Specifically, in general:
app err
0 x x x = =
Only in certain special cases it happens that, given an
orthonormal set , then:

app
0 x x x eH =

In words, for this special orthonormal set , given any
elements x in H, the reconstructed element
app
x always
perfectly coincides with x . Not all orthonormal sets have this
property. If a certain set does, it is said that is a
complete basis for H, or simply a basis for H.
4.3.2.3 reconstructing x using a complete basis
If a complete basis is available for H, then one can
unambiguously describe any x eH by supplying the complete
basis and just the components of x , i.e.,
| |
1 2
, ,...,
N
x x x ,
with respect to .
The reason for this is that one can immediately reconstruct
x by summation:
1

N
n n
n
x x u
=
=

4.3.2.4 scalar products when a complete basis is
available
The scalar product of any two elements
, x y eH
can be
computed according to the native rules of the space H,
which are typically different depending on the nature of the
space. For instance, in
2
the native rule for the scalar
product is geometric and states that
( ) , x y
is equal to the
product of the length of the vector x (as you could measure
it using a ruler) times the length of the vector y , times the
cosine of the angle between the two vectors, as you could
measure it using a goniometer.
However, if the space has a complete basis , we know that
all elements in H can be represented as:
1

N
n n
n
x x u
=
=

As a result, the scalar product
( ) , x y
can be written as:
( )
1 1
, ,
N N
n n m m
n m
x y x u y u
= =
| |
=
|
\ .


Now using the distribution property of the scalar product, we
have:
( ) ( ) ( )
1 1 1 1
1 1 1
, , ,
N N N N
n n m m n m n m
n m n m
N N N
n m mn n m
n m n
x y x u y u x y u u
x y x y o
-
= = = =
- -
= = =
= =
= =



This result is one of the most important results of the theory
of scalar product spaces. It says that:
provided that a scalar product space H has a complete
basis, then one can carry out the scalar product using a
standardized rule, based on the sum of the products of same-
index components, independently of the nature of the objects in
H:
( )
1
,
N
n n
n
x y x y
-
=
=

As a corollary to this, in such spaces with a complete basis,
then the norm becomes standardized too:

( )
2
1 1
,
N N
n n n
n n
x x x x x x
-
= =
= =

=

This rule is extremely famous and goes under the name of
Parsevals rule. Note that for N=2 such rule has been known for
millennia because in
2
it is equivalent to the celebrated
Pythagorean theorem! (Prove it on your own!)

Finally, the distance between two elements of H enjoys a
standardized form too:

{ }
( )
1 1
2
1 1
,

N N
n n n n
n m
N N
n n n n n
n n
x y x y x u y u
x y u x y
= =
= =
A = = =
=




On your own: try and verify as many of the above properties
as possible based on what you know about
2
and
2
.

4.3.2.5 generating a scalar product space from an
orthonormal set
Let be an orthonormal set in H, and let
'
H the set of all
elements x created by all possible linear combinations of the
unit elements of :

1 1 2 2
1
...
N
N N n n
n
x x u x u x u x u
=
= + + + =

Eq. 4.2

It is easy to show that '
H is a scalar product space too. In
addition is a complete basis for '
H , because by definition it
generates all possible elements of '
H through Eq. 4.2.
However, in general,
'
H _H. Specifically, if is a
complete basis for H as well, then clearly '
H H. Otherwise,
in the more general case in which is not a complete basis for
H, then '
H cH. Note that the case '
H H is impossible.

On your own: you can easily verify that '
H is a scalar
product space, by checking that all the required properties of
linear and scalar product spaces apply to '
H as well. Also, why
is it impossible that '
H H?
4.4 Signals Spaces as Scalar
Product Spaces
4.4.1 The scalar product for signals
We already know that the set of signals over an interval I is
a linear (or vector) space . We shall now see that it also has a
scalar product and therefore it is a scalar product space H.
The scalar product between two signals x(t) e y(t) is defined
as:
( )
*
( ), ( ) ( ) ( ) x t y t x t y t dt =

I

Eq. 4.3

where I is the range of t over which the signals are defined
for a specific scalar product space H.
4.4.1.1 element or native notation
In Eq. 4.3 We could also have written the left-hand side as:
( ) , x y , using the element notation, which is standard for all
linear and scalar product spaces, independently of the actual
nature of the element and of the scalar product space. In the
right hand side, we have introduced the native scalar product
rule for signals and therefore we had to explicitly write the
elements , x y as signals ( ), ( ) x t y t .
In the following, by default, we will use the element
notation as in x , y etc. We will resort to the native notation
( ), ( ) x t y t only when strictly needed.
4.4.1.2 properties of the signal scalar product
We should now try and see whether the required properties of
the scalar product are satisfied by the definition Eq. 4.3. This is
easily proved, so we will omit it. We will only show that the
distribution property holds:
( ) | |
( ) ( )
*
* *
, ( ) ( ) ( )
( ) ( ) ( ) ( ) , ,
I
I I
x y z x t y t z t dt
x t z t dt y t z t dt x z y z
o | o |
o | o |
+ = + =
+ = +

.

Based on the scalar product, we can for instance compute the
energy of an element x . By following the definition Eq. 4.1
the energy of x is:

{ }
( )
2
, ( ) x x x x t dt = =

I
E
Eq. 4.4
Note that this value of the energy is completely consistent
with that of Eq. 3.1 which was given in a totally different
context. In other words:
{ }
{ } ( ) x x t =
I
E E . Since they coincide,
{ }
x E too returns results that are consistent with physical
arguments regarding energy and power.
The norm of an element x is the square-root of the energy,
so:
( )
2
, ( ) x x x x t dt = =

I

4.5 Simple Signal Spaces
4.5.1 Generating signal scalar product spaces by
means of simple orthonormal sets
In Sect. 4.3.2.5 we showed that an orthonormal set can be
used to generate a scalar product space '
H , of which is, by
construction of
'
H , a complete basis. In the following, we will
drop the prime and call the scalar product space generated by
simply as H.
Since is a complete basis for H, we can then use both the
native rules of signal spaces to compute norms, distances, sclar
products, etc., or we can use the standardized general rules,
operating on the components of the signals in H with respect
to , as shown in Section 4.3.2.4.
4.5.1.1 example of a simple signal basis
A simple example of an orthonormal signal set is the
following:

Fig. 4.1

{ } ( )
4
1
1
, R 1 2
n n
n
u u t n
=
= +

with t which belongs to the interval [0, 4] = I .
Let us check the orthogonality on the unit elements
1 2
, u u :

( )
4
*
1 2 1 1
0
1 3
, R R 0
2 2
u u t t dt
| | | |
= =
| |
\ . \ .

They are clearly orthogonal because the integrand function is
zero everywhere.
On your own, check the orthogonality of all other pairs.

Normality is also straightforward. For element
1
u :

( )
4 1
*
1 1 1 1
0 0
1 1
, R R 1(t) 1
2 2
u u t t dt dt
| | | |
= = =
| |
\ . \ .



and likewise for the others elements.


Fig. 4.2

4.5.1.2 example of generating signals using a
simple orthonormal set
We now use the very simple orthonormal set:

{ } ( )
2
1
1
, R 1 2
n n
n
u u t n
=
= + with [0, 2] = I

which is depicted in Fig. 4.2. We use this basis to generate
two simple signals. Specifically, we generate ( ) s t as:

1 2
( ) 1 ( ) 1 ( ) s t u t u t = +
and ( ) w t as:
1 2
( ) 1 ( ) 1 ( ) w t u t u t =

These signals are shown in Fig. 4.3 and Fig. 4.4.


Fig. 4.3

Fig. 4.4

Using the n-tuple (or array) format, we can also write them as:

| |
1,1 s = ,
| |
1, 1 w=

4.5.1.3 taking the scalar product of two signals
using the standardized rule
Let us now compute the scalar product of these two signals.
Using the native rule we get:

( ) ( )
( ) ( )
2 2
2 1 1
0 0
1 2
2 1 2 1
0 1
1 2
0 1
1 3
, ( ) ( ) R 1 R R
2 2
1 1
R 1 R R 1 R
2 2
1( ) 1( ) 1 1 0
s w s t w t dt t t t dt
t t dt t t dt
t dt t dt

| | | |
= = =
| |

\ . \ .

| | | |
= =
| |
\ . \ .
= = =




that is, the two signals are orthogonal.
We now verify that the same result is obtained when using
the standardize rule based on the n-tuple representation:

( )
( )
1
1 1 2 2
1
, 1 1 1 1 0
n n
n
s w s w s w s w
=
= = + = + =

So, in fact, both the native rule and the standardized rule
produce the same result.

4.5.1.4 remarks on orthogonality of signals
To discuss signal orthogonality, we first introduce the
definition of support of a signal. In mathematics, the support of
a function is the set of points where the function is not zero. In
signal theory, the support of a signal is the set of all time-
instants where the signal is not zero.

An interesting result about supports is found when two
signals are multiplied.
Given the signal: ( ) ( ) ( ) z t x t y t = , and given the support of
( ) x t , called X and the support of ( ) y t , called Y, then the
support of ( ) z t is the intersection of the supports of ( ) x t and
( ) y t : = Z X Y.
As a corollary, if two signals ( ) x t and ( ) y t have completely
disjoint supports, that is, if ( ) 0 x t = then ( ) 0 y t = and vice-
versa, then = =C Z X Y . That is, the support of ( ) z t is an
empty set; in other words, ( ) 0 z t = at all times.
This allows us to find a sufficient condition for two signals to
be orthogonal. Given ( ) x t and ( ) y t , they are orthogonal if
their supports are disjoint, i.e., if = C X Y .
On your own, prove the simple results just mentioned above.

For instance, this is why the four signals:

{ } ( )
4
1
1
, R 1 2
n n
n
u u t n
=
= +

are orthogonal to one another: their supports are all mutually
disjoint.
This condition for orthogonality is sufficient but not
necessary. In other words, there are signals whose supports are
not disjoint, that are orthogonal nonetheless. We saw an
example in Sect. 4.5.1.2. The supports of the signals ( ) s t and
( ) w t were not disjoint, but such two signals were orthogonal
nonetheless.
On your own: consider the two signals ( )
( ) cos 2
o
x t f t t =
and ( )
( ) sin 2
o
y t f t t = over the interval
| |
0,T with
0
1/ T f = .
The two signals have supports that are fully superimposed.
Carry out the scalar product and check that they are
nonetheless orthogonal.
4.6 Incomplete Bases,
Approximations and Errors
As mentioned, an orthonormal set such as:
{ }
1
( )
N
n
n
u t
=
=

constitutes a complete basis when it can exactly represent all
the elements of a linear space, that is:

( ) ( )

=
=
N
n
n n
t u v t v
1
for any
( )
v t eH
4.6.1 Incomplete basis and definition of error
If the above condition is not satisfied, one can say that is
an incomplete basis for the scalar product space H.
If we take a signal
'
) ( S t w e , we can define a signal ) (t w
app

which is the approximated representation of ) (t w by means of
basis .

( ) ( ) ( )
1 2 3
, , ,...,
app N
w t w t w w w w ~

Eq. 4.5

Since the representation of ) (t w is inexact, it will be possible
to define an error as follows:

( ) ( ) ( ) t w t w t w
app err
=
Eq. 4.6

Note that the signal ) (t w can be described exactly as:

( ) ( ) ( ) t w t w t w
err app
+ =
Eq. 4.7

4.6.2 Properties of the error signal
The error signal presents important features which are
summarized in the three theorems that follow.
Theorem 1
Given:
( ) ( ) ( ) t w t w t w
err app
+ =
Eq. 4.8
,

the error signal ) (t w
err
is orthogonal to each unit element
(or unit signal) of the incomplete basis used for the
representation of the signal ) (t w
app
.
If we consider the representation:

( ) ( )

=
=
N
n
n n app
t u w t w
1

Eq. 4.9
together with Error! Reference source not found. we have:

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( )
( ) ( ) ( )
0
,
,
, , ,
1
,
1
1
= = =
= =
= |
.
|

\
|
=
= =

=
=
=
n n
N
m
m n n n
N
m
n m m n
n
N
m
m m n
n app n n err
w w w w
t u t u w w
t u t u w w
t u t w t u t w t u t w
o

Eq. 4.10
In short:
( )
err
, 0,
n
w u n =
so the theorem is proved.
Theorem 2
The approximate signal ) (t w
app
and the error signal ) (t w
err

are orthogonal to each other.
We directly calculate the scalar product between ) (t w
app
and
) (t w
err
:
( ) ( )
( )
( ) ( ) ( ) ( ) ( )
1 1
, , ,
N N
app err m m err m m err
m m
w t w t w u t w t w u t w t
= =
| |
= =
|
\ .


Eq. 4.11.
But from Theorem 1 we know that:
( ) ( ) ( )
, 0
m err
u t w t m =
So all the terms in the last summation are zero and therefore:
( ) ( )
( )
, 0
app err
w t w t =
Theorem 3
Given ( ) ( ) ( ) t w t w t w
err app
+ = , the energy of the signal
( )
w t
is equal to the sum of the energy of the approximated signal
with the energy of the error signal:

{ } { } { } ) ( ) ( ) ( t w E t w E t w E
err app
+ =
Eq 1

The proof is again a direct calculation:
{ } { }
( ) ( ) ( )
( ( ) ( ), ( ) ( ))
( ( ), ( )) ( ( ), ( ))
( ( ), ( )) ( ( ), ( ))
( ( ), ( )) ( ( ), ( )) 0 0
( ( ), ( )) ( ( ),
app err
app err app err
app app err err
app err err app
app app err err
app app err
E w t E w t w t
w t w t w t w t
w t w t w t w t
w t w t w t w t
w t w t w t w t
w t w t w t
= + =
= + + =
= +
+ + =
= + + + =
= +
{ } { }
( ))
( ) ( )
err
app err
w t
E w t E w t
=
= +

Eq. 4.12

The cross scalar products ( ( ), ( )) ( ( ), ( ))
app err err app
w t w t w t w t +
are both zero owing to what we demonstrated in Theorem 2.
We remark that, given a certain signal, the higher is the
energy of the approximated signal, the lower is the one of the
error.
As a corollary, note the following result:
{ }
{ }
app
w w s E E
On your own: prove this corollary.

4.6.2.2 optimality of the canonical projection
Given ( ) S t s e with | |
1 0
, t t t e ;
given:
( ) ( )

=
=
N
n
n n app
t u s t s
1

Eq. 4.13
where the

n
s
are the components on the basis ( ) t u
n
used, let
us try to reduce the error energy by choosing
n n
s s =
' . In other
words, we try to find, if possible, a new approximation:
( ) ( )

=
' = '
N
n
n n app
t u s t s
1

Eq. 4.14
such that we manage to reduce the Energy of the error:
( ) { } ( ) { } t s t s
err err
c c <
'

Eq. 4.15
First of all, if we compute a new
( )
app
s t '
, then we will also
have a new error signal
( )
err
s t '
:
( ) ( ) ( ) t s t s t s
err app
+ =
( ) ( ) ( ) t s t s t s
err app
' + ' =
Subtracting by columns the previous two equations we get:
( ) ( ) ( ) ( ) t s t s t s t s
err err app app
'
+
'
= 0
We then define a new signal
( )
s t
o
, which represents the
change in the error signal between using the canonical
approximant
( )
app
s t and the new approximant
( )
app
s t '
:
( ) ( ) ( )
err err
s t s t s t
o
'
Eq. 4.16
By consequence we also get:
( ) ( ) ( )
app app
s t s t s t
o
' =
Now we remark that by changing
( )
app
s t into
( )
app
s t ' only
three things can happen:
( ) { } ( ) { }
0
0
0
err err
s t s t c c
=

'
>

<

In the first case we find an error equal to the error of the
canonical approximant. In the second case we find a bigger
error and therefore it is not useful. In the third case we find a
smaller error and this would mean that the use of
n
s
' is
preferable with respect to the canonical projections
n
s .
To find out which of the three possible cases occurs, we
directly compute the error energy difference:

( ) { } ( ) { }
err err
s t s t c c '
Now, using Eq. 4.16 we get:

( ) { } ( ) { } ( ) ( ) { } ( ) { }
( ) ( ) ( ) ( ) ( ) ( ) { }
( ) { } ( ) { } ( ) ( ) ( ) ( ) ( ) ( ) ( ) { }
( ) { } ( ) ( ) ( ) ( ) ( ) ( )
,
, ,
, ,
err err err err
err err err
err err err err
err err
s t s t s t s t s t
s t s t s t s t s t
s t s t s t s t s t s t s t
s t s t s t s t s t
o
o o
o o o
o o o
c c c c
c
c c c
c
' = + =
+ + =
+ + + =
+ +

Eq. 4.17
We concentrate on the term
( ) ( ) ( )
,
err
s t s t
o
.
First of all we know that:
( ) ( ) ( ) ( ) ( )
1 1
N N
app app n n n n
n n
s t s t s t s u t s u t
o
= =
' ' = = =


( ) ( ) ( )
,
1 1
N N
n n n n n
n n
s s u t s u t
o
= =
' =


Eq. 4.18
Therefore ( ) s t
o
is a linear combination of the basis too. So if
we evaluate
( ) ( ) ( )
,
err
s t s t
o
, we get:
( ) ( ) ( ) ( ) ( )
( )
( ) ( ) ( )
,
1
,
1
, ,
,
N
err err n n
n
N
n err n
n
s t s t s t s u t
s s t u t
o o
o
=
=
= =
=

However, from Theorem 1 we know that
( ) ( ) ( )
, 0
err n
s t u t =
for all n , i.e., the error signal is orthogonal to all elements of
the incomplete basis. Therefore all the terms in the last
summation are zero and hence:
( ) ( ) ( )
, 0
err
s t s t
o
=
As a result, Eq. 4.17simply becomes:
( ) { } ( ) { } ( ) { } t s t s t s
err err o
c c c =
'

Eq. 4.19
Since energy is always greater than 0, then the right-hand
side is greater than 0 and therefore:
( ) { } ( ) { }
0
err err
s t s t c c ' >
This equation shows that the energy of the error, given a
different choice of components than the canonical ones, is
always greater than the energy of the error when the canonical
components are used.
In fact, if the right-hand side of Eq. 4.19 is explicitly
evaluated, one finds the following:
( ) { } ( )
2
2
2
1 1
N N
n n n
n n
I
s t s t dt s s s
o o o
c
= =
' = = =

For the energy of
( )
s t
o
to be equal to 0, every contribution to
the last summation must be equal to 0, which implies:
n n
s s n ' = .
which confirms that the best choice of the approximant is in
fact the canonical one.

Example in R
2


2
9 e v

I decide to use an incomplete basis in
2
9 , I use only a unit
vector instead of two. I try to obtain the best approximation of
my : v

u
v

Fig. 4.5

u v u u v v v
u app
) ( = = ~




err app
v v v

+ =
Eq. 1


Now by carrying out
the same process as
previously done, we try to
find a
u
v different from the one that we found with the
canonical projection and we try to see if the approximant
improves or it becomes worse:

u v
app

p =

where p is a number that changes the length of the
approximant.







app
v

err
v

Fig. 4.6
app
v

err
v
'

Fig. 4.7






By taking a longer approximant we can see that the error
increases. We now try with an approximant shorter than the
projection we have used before:














Also in this case we can visually see that the error is
increased.
app
v

err
v
' '

Fig. 4.8
4.6.3 Example problems
4.6.3.1 problem 1
Given the set of signals:

{ }
10
1
( )
n
n
u t
=
=
Eq. 4.20
with
| |
0, t T e = I

where:

2
( ) cos
n n
u t A nt
T
t
o
| |
= +
|
\ .

Eq. 4.21

we want to calculate the values of o and
n
A so that is an
orthonormal set, that is:
,
( ( ), ( ))
n m n m
u t u t o =
Eq. 4.22
We first calculate the scalar product between any two ( )
n
u t
and ( )
m
u t :

| | | |
*
0
*
0
2 2
( ( ), ( )) cos cos
1 2 2
cos 2 cos
2
T
n m n m
T
n m
u t u t A nt A mt dt
T T
A A n m t n m t dt
T T
t t
o o
t t
o
| | | |
= + + =
| |
\ . \ .
| | | |
+ + +
| |
\ . \ .

Eq. 4.23

Each term of the sum that we get in the integral can be
expanded as follows:

| |
| |
| |
| |
| | ( ) ( ) | |
| |
( ) ( ) | | 0 2 2
2
1
2 2 2
2
1
0
2
2
2
1
2
2
cos
0
=
+
=
= + +
+
=
=
|
.
|

\
|
+ +
+
=
=
|
.
|

\
|
+ +

o o
t
o o t
t
o
t
t
o
t
sen sen
m n
T
sen m n sen
m n
T
T
t m n
T
sen
m n
T
dt t m n
T
T


The result could be foreseen because the integral of a circular
function (sine, cosine, complex exponential) calculated over a
period, or over an exact multiple of the period of the circular
function itself, is always null.
The same result can be obtained for | |
|
.
|

\
|
t m n
T
t 2
cos , with
m n = .
So, if m n = we get:

( ) ( ) ( )
*
1
, (0 0) 0
2
n m n m
u t u t A A = + =

Vice versa if m n = we get :

( ) ( ) ( )
2
2
0
2 2
0 0
2
2
, cos
1 4
cos
2
2
T
n n n
T T
n n
n
u t u t A nt dt
T
A dt A nt dt
T
A T
t
o
t
o
| |
= + =
|
\ .
| |
= + + =
|
\ .
=

Therefore, to ensure normality, it must be:
2
1
2
n
A T
=
that is:
2
n
A
T
= .
In summary, becomes an orthonormal set for any value of
o and for
2
n
A
T
= .
4.6.3.2 problem 2
We use the basis shown in Fig. 4.1.
Given the signal


Fig. 4.9

) (t v is approximated by the signal ) (t v
app
defined as:
( )

=
=
4
1
) ( ) (
n
n n app
t u t v t v con
| |
0, 4 t e = I

we want to find the array representation of ) (t v
app


| |
1 2 3 4
( ) , , ,
app
v t v v v v

that is, we want to calculate the coefficients
n
v which
represent it.
Each
n
v is defined as the scalar product between ) (t v and
) (t u
n
, so the various
n
v can be calculated as:

( ) ( ) ( ) ( )
8
7
8
1
1
0
1
8 4
1
4
1 ,
2
1
0
1
4
0
1 1
= = =
|
.
|

\
|
=
|
.
|

\
|
= =

t
t dt
t
dt t u
t
t u t v v

( ) ( ) ( ) ( )
4 2
2 2 2
0 1
2
, 1 1
4 4
4 1 5
2
2 1
1
8 8 8 8
t t
v v t u t u t dt dt
t
t
| | | |
= = =
| |
\ . \ .
= = + =



( ) ( ) ( ) ( )
4 3
3 3 3
0 2
2
, 1 1
4 4
9 4 3
3
3 2
2
8 8 8 8
t t
v v t u t u t dt dt
t
t
| | | |
= = =
| |
\ . \ .
= = + =



( ) ( ) ( ) ( )
4 4
4 4 4
0 3
2
, 1 1
4 4
16 9 1
4
4 3
3
8 8 8 8
t t
v v t u t u t dt dt
t
t
| | | |
= = =
| |
\ . \ .
= = + =



as a result ) (t v
app
in array form will be:

7 5 3 1
( ) , , ,
8 8 8 8
app
v t

=




If we want to calculate the energy of the signal ) (t v we
proceed as follows:

2
4 4
2
0 0
4
3 2
0
1 1 ,1 1 1
4 4 4 4 16 2
64 16 64 4
4
48 4 48 4 48 3
t t t t t t
dt dt
t t
t
| |
| | | |
= = = + =
`
| | |
) \ . \ .
\ .
| |
= + = + = =
|
\ .

E


From Theorem 3 we learn that this energy is equal to the sum
of the energy of the approximated signal ) (t v
app
and the error
) (t v
err
.
If we calculate the energy of the approximated signal ) (t v
app

we can therefore obtain the energy of the error.
So:
{ }
4
2
1
49 25 9 1 84 21
( )
64 64 64 64 64 16
app n
n
v t v
=
= = + + + = =

E

We calculate the difference between the energy of the signal
and the one of its approximation, to find the energy of the
error:
{ }
4 21 64 63 1
( )
3 16 48 48
err
v t

= = = E

Note that the ratio between the energy of the error and the
energy of the signal is:

{ ( )} 1 48 1
0, 016 1.6%
{ ( )} 64 48 64
err
v t
v t
= = ~ =
E
E


We plot the signals ) (t v
app
and ) (t v
err
:

Fig. 4.10

Fig. 4.11
As the chart shows, despite the fact that the incomplete basis
used for the calculus is rather elementary, the energy of the
error is low compared to the energy of the signal. In fact the
energy of the error corresponds to only 1.6% of the energy of
the signal.

Fig. 4.12
On your own: redo the same problem as above, assuming as
orthogonal set:
{ }
( )
8
1/2
1
, R / 2 1/ 4
n n
n
u u t n
=
= +
Note that the orthogonal functions need to be normalized.
What is the new error energy? Can you guess what would
happen to the error energy if you used the orthogonal set:
{ }
( )
16
1/4
1
, R / 4 1/ 8
n n
n
u u t n
=
= + ?
4.7 Infinite Dimension Spaces
Given an inner product space H, such space is said to be N -
dimensional if there is an orthonormal set
{ }
1

N
n
u
=
= which is
a complete basis for H, i.e., such that it can generate all
x eH. Any element x can then be fully represented using N
real or complex numbers:
| |
1 2 3
, , ,...,
N
x x x x x =
Signal spaces contain very complex elements (the signals)
and thus it is obvious that very large bases may be needed to
represent such elements. In fact, for certain signal spaces, a
basis can only be found if N is allowed to become infinite,
that is, the basis is made up of an infinite number of
orthonormal elements:
{ }
1

n
u

=
= . In this case, the scalar
product space is said to be of infinite dimension.
One example of such infinite-dimension scalar product space
is the set of all finite-energy signals over a finite interval. It is
called
0 1
2
[ , ] t t
L . Therefore, for ( ) s t to be in
0 1
2
[ , ] t t
L it must be:
| |
{ }
1
0 1
0
2
,
( ) ( )
t
t t
t
s t s t = <

E
Theorems assure us that such space does have a complete
basis, but such basis has an infinite number of elements. In the
next chapter we will specifically deal with one such basis,
called the Fourier Basis. Note however that in general there are
many different possible bases, in fact an infinite amount of
different bases.
4.7.1 Hilbert spaces
Hilbert spaces and scalar product spaces are almost
synonyms. All Hilbert spaces are scalar product spaces.
However, by saying that an inner product space is a Hilbert
space too, we are stressing a property that Hilbert spaces must
have by definition: completeness.
To properly deal with this concept we should introduce the
so-called Cauchy sequences and other concepts, which well
omit here.
However, in essence, a complete inner product space is
such that it has no holes or no elements missing. In other
words, given an element, there is always another one that is
infinitesimally near, in the sense of the distance defined as
{ } , x y x y A = .
For instance,
3
is a Hilbert space. However, taking out even
a single element such as [3, 5, 2] , the remaining space
3
[3, 5, 2] is no longer a Hilbert space.
We state without proof the following very important result.
Result:

0 1
2
[ , ] t t
L is a complete inner product space, i.e., a Hilbert space.
4.7.2 Completeness of a basis
We assume to be dealing with a Hilbert space of signals
2
L
I
,
with
| |
1 2
, t t = I ,
1 2
, t t < We also assume to have found an
infinite orthonormal set
{ }
1

n
u

=
= in
2
L
I
. We want to find out
whether such orthonormal set is a complete basis for
2
L
I
.
To do so, we will try to find some conditions that, if verified,
tell us that for sure is complete basis.
We first recall that if was complete, then it must be, for
any
2
( ) s t L e
I
:
( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
2 2
1 1
1 1
1 1
( ),
( ) ( )
n n n n
n n
t t
n n n n
n n
t t
s t s u t s t u t u t
u t s u d s u t u d t t t t t t

= =

- -
= =
= =
= =




We then recall the well-known result:
( )
( ) ( ) s t d s t t o t t

This result is also valid if the integration range is just
| |
1 2
, t t ,
provided that
| |
1 2
, t t t e
1
. Then:

( )
| |
2
1
1 2
( ) ( ), ,
t
t
s t d s t t t t t o t t = e

We then compare the previous two results:
( ) ( ) ( )
2
1
1
( ) ( )
t
n n
n
t
s t s u t u d t t t

-
=
=

Eq. 4.24
( )
2
1
( ) ( ) ( )
t
t
s t s t d s t t o t t = =

Eq. 4.25
The comparison tells us that, for Eq. 4.24 to be verified, it
must be:
( ) ( )
( )
| |
1 2
1
, , ,
n n
n
u t u t t t t t o t t

-
=
= e

If so, Eq. 4.24 is correct and therefore the basis is complete.
We can therefore state the very important result:
given a Hilbert space
2
L
I
with
| |
1 2
, t t = I ,
1 2
, t t < , and given
an orthonormal set
{ }
1

n
u

=
= in
2
L
I
, such orthonormal set is a
complete basis for
2
L
I
if and only if:

1
The range should actually be
| |
1 2
, t t t e . It can be extended to the
extremes using certain cautions and assumptions. We skip these details
which are unimportant in this context and simply refer to the overall
interval, assuming that proper cautions have been exercised.

( ) ( )
( )
| |
1 2
1
, , ,
n n
n
u t u t t t t t o t t

-
=
= e

This result is sometimes re-phrased as follows: an
orthonormal signal set
{ }
1

n
u

=
= in
2
L
I
is a complete basis of
2
L
I
if it can represent the delta (or it can resolve the
delta).
Chapter 5. The Fourier Basis
The Fourier basis is by far the most famous complete basis
for the signal space
0 1
2
[ , ] t t
L . Other bases are also well-known,
such as the Legendre polynomial basis, but the Fourier basis
has unique properties which make it very successful in many
practical applications.
In the following, such basis is introduced and its main
properties are described. Examples are provided. At the end of
the chapter, a few hints will be given as to the proof of the
completeness of such basis.
5.1 Definitions
The Fourier basis is defined as follows. Given the Hilbert
space
0 1
2
[ , ] t t
L of finite-energy signals over the finite interval
0 1
[ , ] t t e I , then a complete basis for such signal space is the
following:
2
0 1 0 1
1
[ , ],
t
j n
T
n
F e t t t T t t
T
t
+

=

= e
`
)


Lets first check the orthonormality of this set. We compute
the scalar product of two elements with different index m n = :

( )
1
0
1
1
0
0
1 0
0 1 0
2 2 2 2
2 2
[ ] [ ]
2 2
[ ] [ ]
2 2
[ ] [ ]
1 1 1
,
1 1 1
2
[ ]
1
2 [ ]
1
1
2 [ ]
t
t t t t
j n j m j n j m
T T T T
t
t
t
t t
j n m j n m
T T
t
t
j n m t j n m t
T T
j n m t j n m t t
T T
e e e e dt
T
T T
e dt e
T T
j n m
T
e e
j n m
e e
j n m
t t t t
t t
t t
t t
t
t
t

| |
=
|
\ .

= = =

= =

0
2
[ ]
2 [ ]
1
1 0,
2 [ ]
j n m t
j n m
T
e e n m
j n m
t
t
t


= = =

As a result, this is certainly an orthogonal set. As for
normality, we look at the case m n = :
1
0
1 1
0 0
2 2 2 2
2
[ ]
1 0
1 1 1
,
1 1
1( ) 1
t
t t t t
j n j n j n j n
T T T T
t
t t
t
j n n
T
t t
e e e e dt
T
T T
t t T
e dt t dt
T T T T
t t t t
t

| |
= =
|
\ .

= = = = =

From this we have:
2 2
1 1
,
t t
j n j m
T T
mn
e e
T T
t t
o

| |
=
|
\ .


which means that F is indeed an orthonormal set.
5.2 The Fourier Series
Given any signal
0 1
2
[ , ]
( )
t t
s t L e , then it is possible to exactly
express it is as a linear combination of the elements of the
Fourier basis, in the canonical way:

2
( ) ( )
t
j n
T
n n n
n
n
e
s t s u t s
T
t
+
=
= =


Eq. 5.1

where the coefficients
n
s are the projections of the signal
( ) s t over each orthonormal element ( )
n
u t of the basis F :

( )
1
0
2
2
1
( ), ( ) ( ), ( )
t
j n
t
t
T
j n
T
n n
t
e
s s t u t s t s t e dt
T T
t
t

| |
|
= = =
|
|
\ .



Eq. 5.1 is often called Fourier series.
Since the Fourier basis is complete, we can also fully
represent any signal
0 1
2
[ , ]
( )
t t
s t L e through an infinite array of
components:
0 1 1 2 2
( ) [ , , , , ,..., , ,...]
n n
s t s s s s s s s s

=
As for any element of a Hilbert space, the energy and the
norm of a signal ( ) s t can be computed as:

{ }
2
( )
n
n
s t s
+
=
=

E
2
( )
n
n
s t s
+
=
=



Also, the scalar product of any two signals ( ), ( ), s t w t with
Fourier components ,
n n
s w , can be computed as:

( )
*
( ), ( )
n n
n
s t w t s w
+
=
=

Finally, the distance between any two such signals can be
calculated as:

{ } ( )
2
( ), ( ) ( ) ( )
n n
n
s t w t s t w t s w
+
=
A = =

5.2.1.1 example of Fourier components calculation
Given ( ) R ( ) s t t
t
= , with T t < , then
2
[ / 2, / 2]
( )
T T
s t L

e . As a
result, it is certainly possible to compute its Fourier
components and it is also possible to write the signal as a
Fourier Series.
We first evaluate the Fourier components:

2
2 2
2
2 2 2
2
2
2
2 2
2 2
1
R ( ), R ( )
1 1 1
2
1 1
2
1 1 2
2 sin
2
2
1
sin
T
t
j n
t
T
j n
T
n
T
t t
j n j n
T T
j n j n
T T
e
s t t e dt
T T
e dt e
T T
j n
T
e e
T
j n
T
j n
T
T
j n
T
T n
n
T
t
t
t t
t
t
t t
t
t
t t
t t
t
t
t t
t
t
t

+
+

| |
|
= =
|
|
\ .

= = =

| |
= =
|
\ .

sin 0
T n
n
T n T
t
t t
t

| | | |
= =
| |

\ . \ .


Note that the final result of the above calculation is fine for
all n except 0 n = because then we end up with a form
sin(0)
0

which we do not know how to handle. Instead, for 0 n = we
restart the calculation from scratch and easily find:

2
0
0
2 2
2 2
1( )
R ( ), R ( ),
1 1
R ( ) 1( ) 1( )
t
j n
T
n
T
T
e t
s t t
T T
t t dt t dt
T T T
t
t t
t
t
t
t

=
+ +

| |
| |
|
= =
|
|
\ .
|
\ .
= = =



Note that
n
s can be written in a compact form valid for all n
as:
Sinc
n
n
s
T
T
t t
t

| |
=
|
\ .


because by definition
( )
Sinc 0 1 = .
We can then reconstruct R ( ) t
t
as a linear combination of
the Fourier basis orthonormal signals:

2
2
Sinc
R ( ) ( )
Sinc
t
j n
T
n n
n n
t
j n
T
n
n e
t s u t
T
T T
n
e
T T
t
t
t
t t
t
t t
t

+ +
= =

+
=

| |

= =
|
\ .

| |
=
|
\ .

The energy of R ( ) t
t
is:

{ }
2
2
2
2
2 2
R ( ) Sinc
sin
n
n n
n
n
t s
T T
T n
n T
t
t t
t
t
t t
t
+ +
= =
+
=

| |
= = =
|
\ .

| |
= =
|

\ .

E


On your own: verify that the resulting value is indeed t by
numerically adding up a sufficient number of series
coefficients and by direct calculation using the native
definition of the scalar product.
On your own: write a computer program that sums an
increasing number of contributions in the formula:
2
,app
R ( ) Sinc
t
N
j n
T
n N
n
t e
T T
t
t
t t
t

+
=

| |
=
|
\ .

,
that is, N is increased gradually. Plot the result and compare
,app
R ( ) t
t
with R ( ) t
t
. Verify you get the following figure:

5.2.2 The signal spectrum
The spectrum of a signal ( ) s t is a representation of the
Fourier components
n
s of the signal itself.
-30 -20 -10 0 10 20 30
-0.1
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35

Fig. 5.1 Spectrum of the signal R ( ) t
t
for / 1/ 3 T t = . The
abscissa is n , corresponding to frequency
0
n f

If instead of
n
s we plot
2
n
s , then we have the energy
spectrum of the signal. The name derives from the fact that
each
2
n
s contributes to the total signal energy:
{ }
2
( )
n
n
s t s
+
=
=

E
-30 -20 -10 0 10 20 30
0
0.02
0.04
0.06
0.08
0.1
0.12

Fig. 5.2 Energy spectrum of the signal R ( ) t
t
for / 1/ 3 T t = . The
abscissa is n , corresponding to frequency
0
n f
5.2.3 Some properties of the Fourier components
5.2.3.1 property 1: conjugate symmetry
Given
2
( ) s t L e
I
with ( ) s t e too, then:
n n
s s
-

= ,
n n
s s

= ,
n n
s s

Z = Z
Proof:
2
( )
j n t
T
n
s s t e dt
t

=

I
,

2 2
( )
( ) ( )
j n t j n t
T T
n
s s t e dt s t e dt
t t

= =

I I

and therefore
2
*
( )
j n t
T
n n
s s t e dt s
t

= =

I


because ) ( ) (
*
t s t s = since ( ) s t e .
Note also that obviously
*
n n n n
s s s s
-

= = too. Finally, it is
easy to see that
0
s e .
5.2.3.2 property 2: delay
Given a signal
2
( ) s t L e
I
, then the following property holds:
2
( )
( )
d
n
j n t
T
d n
s t s
s t t s e
t

However, the additional assumption that the support of
( )
d
s t t remains comprised within the interval I is necessary.
Otherwise, the signal would get truncated and the property
would not be valid.
5.2.3.3 property 3: time inversion
Given a signal
2
( ) s t L e
I
, then the following property holds:
( )
( )
n
n
s t s
s t s

However, the additional assumption that the support of ( ) s t
remains comprised within the interval I is necessary.
Otherwise, the signal would get truncated and the property
would not be valid.
5.2.3.4 property 4: scalar product simplified
Given two signals,
( ) ( )
2
, v t w t L e
I
,
( ) ( )
, v t w t e , then
their scalar product can be written as:

( ) ( ) ( ) { }
*
0 0
1
, 2
n n
n
v t w t v w v w

=
= 9 +

Eq. 5.2
Proof:
( ) ( ) ( )
( )
{ }
*
1
* * * * * *
0 0 0 0
1 1 1
* * * * * *
0 0 0 0
1 1 1
* *
0 0
1
,
2
n n
n
n n n n n n n n
n n n n
n n n n n n n n
n n n
n n
n
v t w t v w
v w v w v w v w v w v w
v w v w v w v w v w v w
v w v w

=


= = = =

= = =

=
=
= + + = + +
= + + = + +
= 9 +

Note that the term for 0 n = apparently is different than in
Eq. 5.2:
*
0 0
v w rather than
0 0
v w . However, we know that:

( ) ( ) ( )
2
0
0
1 1
, ,
nt
j
T
n I
e
w w t w t w t dt
T T T
t
=
| |
| |
|
= = =
|
|
\ .
|
\ .



Since by assumption
( )
w t e and since the integral of a real
function is real, then
0
w e and therefore
0 0
w w
-
= . As a result
Eq. 5.2 is verified.
5.2.3.5 Problem
Given the signals:
( )
x t and ( ) y t , both in
| |
2
2;2
L

:
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5
-1
-0.5
0
0.5
1
1.5
t
y
(
t
)

Fig. 5.3: signal
( )
x t
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5
-1.5
-1
-0.5
0
0.5
1
1.5
t
y
(
t
)

Fig. 5.4: signal
( )
y t

we want to compute their scalar product:
( ) ( ) ( )
, x t y t .
Solution
We first look at the native rule for calculating the scalar
product:
( ) ( ) ( ) ( ) ( )
2
2
, , x t y t x t y t dt
-

The integrand function is:
*
( ) ( ) ( ) z t x t y t = . If we plot it over
| | 2, 2 we get:
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5
-1.5
-1
-0.5
0
0.5
1
1.5
t
y
(
t
)

Fig. 5.5

The integral result is zero because the area of the surface
subtended by ( ) z t when ( ) 0 z t > is equal to the area of the
surface subtended by ( ) z t when ( ) 0 z t < . The same result can
also be found by simply remarking that ( ) z t is an odd function,
that is, ( ) ( ) z t z t = . Any odd function integrated between
symmetric limits has zero integral.
Therefore, we can conclude that
( ) ( ) ( ) ( ) ( )
2
2
, , 0 x t y t x t y t dt
-

= =

and so
( )
x t and ( ) y t are orthogonal.
We now redo the same calculation is the generalized scalar
product rule based on the signal components with respect to a
complete basis:
( ) ( ) ( )
,
n n
n
x t y t x y
+
-
=
=


We first point out that
( )
x t and ( ) y t are both real signals.
Therefore, as proved in Sect. 5.2.3.4, the scalar product rule
can be simplified to:

( ) ( ) ( ) { }
0 0
, 2 Re
n n n n
n n
x t y t x y x y x y
+ +
- - -
= =
= = +



As complete basis, we choose the Fourier basis. The signal
( )
x t is in fact a rectangular signal
( )
2
R t . From Sect. 5.2.1.1
we immediately find:
Sinc Sinc
2
n
n n
x
T
T
t t t
t

| | | |
= =
| |
\ . \ .

Eq. 5.3
As for ( ) y t , we could perform the direct calculation.
( )
2
,
j nt
T
n
e
y y t
T
t
| |
|
=
|
|
\ .

However, we notice that
1 1
1 1
( ) R R
2 2
y t t t
| | | |
= +
| |
\ . \ .
. The
scalar product is distributive so:
( )
2 2
1 1
2 2
1 1
1 1
, R R ,
2 2
1 1
R , R ,
2 2
j nt j nt
T T
n
j nt j nt
T T
e e
y y t t t
T T
e e
t t
T T
t t
t t
| | | |
| | | |
| |
= = + =
| |
| |
\ . \ .
| |
\ . \ .
| | | |
| | | |
| |
= +
| |
| |
\ . \ .
| |
\ . \ .


From Sect. 5.2.3.2 we know that for the Fourier components
of a signal the following rule holds:

( )
( )
2
d
n
j nt
T
d n
v t v
v t t v e
t

Eq. 5.4
In this case:
( )
2
1
1
2
2 1
2
1
2
1
1
R , Sinc Sinc
2 4
1 1 1
R , Sinc Sinc
2 2 4 2 4
1 1
R , Sinc
2 2 4
j nt
T
j nt
T
j n
j n
T
T
j nt
T
j
e n
t n
T
T T
e
t n e n e
T
e
t n e
T
t
t
t
t
t
t
t
t t t
t
t t
t
=
| |

|
\ .

| |

| | | |
|
= =
| |
|
\ . \ .
|
\ .
| |
| | | | | |
|
+ = =
| | |
|
\ . \ . \ .
|
\ .
| |
| | | |
|
=
| |
|
\ . \ .
|
\ .
n
T

Eq. 5.5
As a result:
4 4
1
Sinc
2 4
Sinc sin
4 4
j n j n
n
y n e e
j n n
t t
t
t t

| |
= =
|
\ .

| | | |
=
| |
\ . \ .

Eq. 5.6

We can now compute the scalar product using:
( ) ( ) ( ) { }
0 0
, 2 Re
n n
n
x t y t x y x y
+
- -
=
= +

Eq. 5.7
The last term is zero because
0
0 y = . All the other terms are
of the form:
{ } Re Re Sinc Sinc sin
2 4 4
n n
n
x y j n n
t t t
-

| | | | | |
=
`
| | |
\ . \ . \ .
)

Eq. 5.8

However, each one of the
n n
x y
-
is purely imaginary, so
{ } Re 0,
n n
x y n
-
= . As a result, every term of the scalar product
(Eq. 5.7) is zero and therefore:
( ) ( ) ( )
, 0 x t y t =
Eq. 5.9

5.2.4 Problem
Find the energy of the following signal.

( ) ( ) ( ) ( )
0 0 0
cos 2 2sin 4 2 cos 8 x t f t f t f t t t t = + +
| | T I , 0 =
T
f
1
0
=
Solution:
This problem can be solved either directly, using the native
rule:
| |
{ } ( )
2
0,
0
( ) ( ), ( ) ( )
T
T
x t x t x t x t = =

E

or it can be solved using Parsevals formula:
( ) { }
2
n
n
x t x
+
=
=

E

We choose to use Parsevals formula but, to be able to use it,
we need to project ( )
x t over a suitable complete basis. We
then choose to use the Fourier basis. As a result we would have
to evaluate the Fourier components:

( )
2 2
0
1 1
( ),
T j nt j nt
T T
n
x x t e x t e dt
T T
t t
| |
= =
|
\ .



However, by inspecting ( )
x t , it is evident that such signal
can be easily re-written in the form of a Fourier series:
( )
2
j nt
T
n
n
e
x t x
T
t
+
=
=


by simple manipulations. Specifically, one can use the
inverse Eulers formulas:
( )
0 0
2 2
0
cos 2
2
j nf t j nf t
e e
nf t
t t
t

+
=

( )
0 0
2 2
0
sin 2
2
j nf t j nf t
e e
nf t
j
t t
t

=

If we use such formulas on ( )
x t , we get:

( ) ( ) ( ) ( )
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0
0 0 0
2 2 4 4 8 8
2 2 4 4 8 8
2 2 4 4
cos 2 2sin 4 2 cos 8
2 2
2 2 2
1 1 1 1 2 2
2 2 2 2
2 2
j f t j f t j f t j f t j f t j f t
j f t j f t j f t j f t j f t j f t
j f t j f t j f t j f t
x t f t f t f t
e e e e e e
j
e e e e e e
j j
T e T e T e T e
j j
T T T T
t t t t t t
t t t t t t
t t t t
t t t



= + +
+ +
= + + =
= + + + + =
= + + +
0 0
8 8
2 2
2 2
j f t j f t
T e T e
T T
t t
+

0
1
1 2
1
2
3
2
4 2
0
0
0, 4
j
n
x
x T
x T
x
x T
x n
=
=
=
=
=
= >

1
1 2
2
3
2
4 2
0
0, 4
n
x T
x j T
x
x T
x n

=
=
=
=
= >


Note that
( )
x t e so it must be:
*
n n
x x

= . In fact a quick
check of the above results show that such condition is satisfied.
Finally, we apply Parsevals formula and get:

( ) { } ( )
2
7 1 1 1 1
4 4 2 2 2
1 1
n
n
x t x T T
+
=
= = + + + + + =

E

5.2.5 Problem
Find the scalar product ( ) ( ) ( ) t y t x , of the following signals:

( ) ( ) ( ) ( ) ( ) t f t f t f t f t x
0 0 0 0 2
1
8 sin 2 4 cos 2 sin 2 2 cos t t t t + + + =

( ) ( ) ( ) ( ) t f t f t f t y
0 3
1
0 0 2
1
4 cos 2 sin 2 cos t t t =
| | T I , 0 =
T
f
1
0
=
Solution:
This is arrived at in a similar way to the previous exercise.
First of all we decompose
( )
x t using Eulers inverse
formulas:
( ) ( ) ( ) ( ) ( )
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0
1
0 0 0 0 2
2 2 2 2 4 4 8 8
2 2 2 2 4 4 8
cos 2 2sin 2 cos 4 2sin 8
1
2 2
2
2 2 2 2
1 1 1 1 1 1 1 1
4 4 2 2
j f t j f t j f t j f t j f t j f t j f t j f t
j f t j f t j f t j f t j f t j f t j f t j
x t f t f t f t f t
e e e e e e e e
j j
e e e e e e e e
j j j j
t t t t t t t t
t t t t t t t
t t t t


= + + + =
+ +
= + + + =
= + + + + +
( ) ( )
( ) ( )
0
0 0 0 0 0 0
0 0 0 0 0 0
8
2 2 4 4 8 8
2 2 4 4 8 8
1 1 1 1 1 1 1 1
4 4 2 2
1 1 1 1
4 4 2 2
f t
j f t j f t j f t j f t j f t j f t
j f t j f t j f t j f t j f t j f t
e e e e e e
j j j j
e e e e e e
T T T T
T T
j j j j
T T T T T T
t
t t t t t t
t t t t t t


=
= + + + + + =
= + + + + +

We then do the same with
( )
y t :

( ) ( ) ( ) ( )
( ) ( )
0 0 0 0 0 0
0 0 0 0 0 0
0 0
1 1
0 0 0 2 3
2 2 2 2 4 4
2 2 2 2 4 4
2 2 4
cos 2 sin 2 cos 4
1 1
2 3
2 2 2
1 1 1 1 1 1
4 4 2 2 6 6
1 1 1 1 1
4 2 4 2 6
j f t j f t j f t j f t j f t j f t
j f t j f t j f t j f t j f t j f t
j f t j f t j f
y t f t f t f t
e e e e e e
j
e e e e e e
j j
e e e
j j
t t t t t t
t t t t t t
t t t
t t t

=
+ +
= =
= + + =
= + +
( ) ( )
0 0
0 0 0 0
4
2 2 4 4
1
6
1 1 1 1
4 2 4 2 6 6
t j f t
j f t j f t j f t j f t
e
e e e e
T T
T T
j j
T T T T
t
t t t t

=
= + +


Collecting all components, we get:

( )
0
1 1
1 4
1
2 2
3
1
4
0
0
j
j
x
x T
x T
x
x T
=
= +
=
=
=

( )
1 1
1 4
1
2 2
3
1
4
0
j
j
x T
x T
x
x T

=
=
=
=

( )
0
1 1
1 4 2
1
2 6
0
j
y
y T
y T
=
=
=

( )
1 1
1 4 2
1
2 6
j
y T
y T

= +
=


We must now carry out the scalar product. We remark that
since
( ) ( )
, x t y t e , then:

( ) ( ) ( ) { }
0 0
, 2 Re
n n n n
n n
x t y t x y x y x y
+ +
- - -
= =
= = +



The easy calculations are as follows:

( ) ( ) ( ) { } { }
( )( ) { }
( )
( ) { }
( ) ( ) ( ) ( )
0 0 1 1 2 2
18 1 1 1 1 1 1
4 4 2 16 2 16
1 1 1
2 6 6
18 1
16 6
, 2Re 2Re 0 0 0...
2Re 2
2Re
54 8
31
,
24
48
j j
x t y t x y x y x y
T T T
T T
x t y t T T T
- -
= + + + + +
+ + = =
=

= = =

5.3 Recording a Signal
Given an audio track of duration 200 seconds,
( )

+
=
=
n
t nf j
n
T
e
c t s
0
2t
| | 200 , 0 e t ( ) | | 200 , 0
2
L t s e
let us evaluate the disk storage space occupied by the signal in
MBytes.
First, psycho-acoustic tells us that the human ear is able to
hear frequencies only up to 20 kHz. Therefore we are going to
store an approximated signal which does not keep all
frequencies but only enough to ensure satisfactory listening.

( )

=
=
N
N n
t nf j
n app
T
e
c t s
0
2t

Hz Nf 000 . 20
0
=
200
1 1
0
= =
T
f
000 . 000 . 4 200 000 . 20 = - = N

Since the representation base is known beforehand, we need
only store the Fourier components
n
c of the signal.
Also, the signal is an acoustic wave and therefore it is a real
signal. So
*
n n
c c =

. Therefore we only need to store

components for 0 n > . So we do not have to store 8,000,000
values of
n
c but only 4,000,000.
However, each
n
c is a complex number and so we have to
store two real numbers for each one of them (8 million real
numbers). Moreover the signal should be stereo (two channels)
and so we have to store twice the information (16 million real
numbers).
Given that each value is represented using 2 Bytes (16 bits)
then: 16.000.000*2 32 = MBytes.
This result is consistent with the typical size of any audio
track of the same time-length that can be found on an audio CD
(approximately 30-40 Mbytes). Note however that the format
in which the signal is stored on audio CDs is different than the
one we discussed here. A comparison with the actual audio CD
format will be proposed later on, after the sampling theorem
has been introduced.
Note also that typical MP3 tracks are only about 2-3 Mbytes,
that is about 1/10 the value found above. This is because MP3
compresses the signal using various techniques. MP3
compression is however lossy, that is, there is a difference
between the original signal and the one recorded and
reproduced after MP3. Some of the fidelity is therefore lost.
5.4 Completeness of the Fourier
basis
We have shown that given a Hilbert space
2
L
I
with
| |
1 2
, t t = I ,
1 2
, t t < , and given an orthonormal set
{ }
1
( )
n
n
u t

=
= in
2
L
I
,
such orthonormal set is a complete basis for
2
L
I
if:

( ) ( )
( )
| |
1 2
1
, , ,
n n
n
u t u t t t t t o t t

-
=
= e

Eq. 5.10
We would like to check whether this is the case for the
Fourier basis. By substituting the Fourier basis into Eq. 5.10,
we find:
( ) ( )
( )
2 2
2
1 1 1
1
j nt j n
T T
j n t
T
n n
n n n
e e
u t u e
T
T T
t t
t
t
t
t

-
= = =
= =


So we have to check whether:
( )
( ) | |
2
1 2
1
1
, , ,
j n t
T
n
e t t t t
T
t
t
o t t

=
= e

This is obviously equivalent to checking:
( )
2
1
1
, ,
2 2
j nt
T
n
T T
e t t
T
t
o

=

= e

Eq. 5.11
This could also be written (in the sense of distributions) as:
( )
2
1
1
lim , ,
2 2
j nt
N
T
n
N
T T
e t t
T
t
o
=

= e

Eq. 5.12
This is in fact true, although we give no formal proof of it in
this course. Eq. 5.12 is actually a very important result which
we will use repeatedly in the remainder of the class.
Since Eq. 5.11 holds, then we can conclude that the Fourier
basis is indeed a complete basis for
2
L
I
.
On your own: write a computer program that sums increasing
numbers of series terms of Eq. 5.12 and plot the result. Does it
appear to converge to a ( )
t o ?
5.5 Periodicity of the Fourier
Series
On your own: prove that the signal reconstructed using a
Fourier series is periodic, of period T.
5.6 The train of deltas
On your own: show that Eq. 5.12, due to the periodicity of
the Fourier series, actually generates a train of deltas, with
period T.