1.

Continuous vs Discrete functions

Table of Contents
1.1 The continuous function .......................................................................................... 1
1.2 Creating an equidistant grid and sampling the continuous function ................................... 2
1.3 The discrete function -- a vector ................................................................................ 3
1.4 Finite difference approximation of derivatives ............................................................. 4
1.5 Higher order approximations .................................................................................... 6
1.6 Representing numerical derivatives by matrices ........................................................... 8
References ................................................................................................................. 9

Lectures on Numerical Methods for Differential Equations FMNN10/NUMN12

Copyright G Sderlind 2010 - 2011

1.1 The continuous function

Most functions of interest in connection with differential equations are continuous and differentiable.
Such a function takes any x in an interval and maps it to y(x). Let's plot a function y on the interval
[0,1]. In Matlab, we first create the interval [0,1], then evaluate y as a function of the independent vari-
able x, and finally plot the graph.

x = linspace(0,1)';
y = 1 + x/2 - x.^2 + 0.3*exp(-x) + 0.5*sin(3*x).^3;
plot(x,y);
axis([0 1 0 2])
title('A continuous function')
hold on

1
 1. Continuous vs Discrete functions

1.2 Creating an equidistant grid and sampling

the continuous function
A discrete function is defined only at discrete points, called the grid or the mesh. Let's choose an
equidistant grid of 19 internal points plus the two boundary points, 0 and 1. This is done using lin-
space. The grid points are marked on the x axis. They have an equal spacing of Delta x = 1/20, called
the mesh width.

If the function y above is sampled at the grid points, we get a discrete function, or a grid function. It is
a vector. Here it has 21 components, as the grid has 21 points (boundary points included). The grid is
marked with blue dots in the graph below, and the samples are marked by red.

% Generate equidistant grid on [0,1] and plot grid points

xgrid = linspace(0,1,21)';
null = zeros(size(xgrid));
plot(xgrid,null,'.','MarkerSize',15)
% Sample function at the grid points and plot samples
ysample = 1+xgrid/2-xgrid.^2+0.3*exp(-xgrid)+0.5*sin(3*xgrid).^3;
plot(xgrid,ysample,'r.','MarkerSize',15);
title('Sampling a continuous function on the grid')
hold off

2
 1. Continuous vs Discrete functions

1.3 The discrete function -- a vector

When we remove the continuous graph, the discrete function remains. The whole process above can be
viewed as an analog-to-digital conversion that takes us from the continuous ("analog") world to the
discrete ("digital") world.

plot(xgrid,ysample,'r.','MarkerSize',15);
axis([0 1 0 2])
title('A discrete function')

3
 1. Continuous vs Discrete functions

In the continuous world we say that y is a function of x -- it maps any x in [0,1] to the corresponding
function value y(x).

The discrete function, on the other hand, is a vector. It is not a function of x. Instead, for each index i
there is a corresponding vector component y_i. It is equal to, or possibly an approximation to, the func-
tion value y(x_i), where x_i is the i'th grid point.

If the mesh width is small enough, the shape of the original continuous function is apparent in the graph
of the discrete function, see above.

1.4 Finite difference approximation of derivat-

ives
A standard approach in the numerical solution of differential equations is to compute an approximate
solution on a grid. The numerical solution is a discrete function.

For this to work we need to replace differential operators by discrete operators. This technique is gen-
erally called discretization.

Let D denote d/dx. This is a linear operator, meaning that

We need to create a discrete operator that has the same property, but rather than acting on continuous
funtions, it must act on discrete functions. Because discrete functions are vectors, the discrete operator
must be a matrix -- all linear operators mapping vectors to vectors are matrices. Clearly, property (1)
holds also if D is a matrix and f and g are vectors.

4
 1. Continuous vs Discrete functions

The simplest way to do this is to use finite differences. Recall the definition if the derivative,

A finite difference is an approximation to the derivative, ignoring the limit. Thus, one uses finite
(however small) values of h, but not infinitesimal values.

The trick is to use the grid function and take h equal to the mesh width. This means we approximate

Suppose we want to "differentiate" the function y above. Let us first have a look at a simple function,
say exp(x), and try to differentiate it numerically at x=1. The derivative is also exp(x), and the true value
of the derivative is exp(1). Lets see how good the numerical approximation is.

h = 2.^(-(3:25)); % Try different mesh widths

uno = ones(size(h));
numderiv = (exp(uno+h)-exp(uno))./h;
exactderiv = exp(uno);
error = abs(numderiv-exactderiv);
loglog(h,error) % Plot in log-log diagram
title('Approximation error vs mesh width')
xlabel('log h')
ylabel('log error')
grid

5
 1. Continuous vs Discrete functions

Here we see that the error becomes smaller and smaller when we decrease the mesh width h. Moreover,
we see that if we reduce h by a factor of 100 (two orders of magnitude), the error is also reduced by the
same amount. We say that the approximation is first order, as the error is proportional to h, i.e., r ~
C*h.

1.5 Higher order approximations

A higher order approximation means that the error decreases faster than in direct proportion to h. This
increases the accuracy. For example, if r ~ C*h^2 we say that the approximation is second order. By
taking logarithms, we get log r ~ log C + 2*log h. By plotting the error in a log-log diagram, we there-
fore get a straight line of slope 2. The order of the approximation is equal to the slope of the straight
line. Slopes can easily be read out, so the easiest way to determine the order experimentally is to plot the
error in a log-log diagram. This technique will be used time and again in the course. Look at the diagram
above. The straight line has slope 1.

A second order approximation to the derivative can be obtained by using a symmetric difference quo-
tient.

h = 2.^(-(3:17)); % Try different mesh widths

uno = ones(size(h));
numderiv = (exp(uno+h)-exp(uno-h))./h/2;
exactderiv = exp(uno);
error = abs(numderiv-exactderiv);
loglog(h,error) % Plot in log-log diagram
title('Approximation error vs mesh width')
grid

Now we see a clean slope of 2 -- when the mesh width is reduced by one order of magnitude (a factor of

6
 1. Continuous vs Discrete functions

ten), the error is reduced by two orders of magnitude (a factor of one hundred). Check out the program
to see how the 2nd order approximation is constructed.

If we run very small mesh widths, we will encounter some problems, however. This is caused by round-
off errors. Let's run the same program as above, but now for very small values of h.

h = 2.^(-(15:200)/5); % Try different mesh widths

uno = ones(size(h));
numderiv = (exp(uno+h)-exp(uno-h))./h/2;
exactderiv = exp(uno);
error = abs(numderiv-exactderiv);
loglog(h,error) % Plot in log-log diagram
title('Approximation error vs mesh width')
xlabel('log h')
ylabel('log error')
grid

Here we see a rough, but straight, trend of slope -1 when the mesh width takes very small values. There
is a "best" mesh width around h=1e-5 in this case. If we take it smaller we do not gain any more accur-
acy but instead lose it. The round-off error contaminates the results, and it is inversely proportional to h.

The round-off is caused by the IEEE computer number system. It can be viewed as a "quantization ef-
fect" in the language of digital signal processing. Because not every real number exists in the computer
number system, the function values f(x) can only take certain values. Thus, at a "microscopic level" the
computed function is not smooth, implying that an attempt to numerically approximate a derivative
eventually will produce a result contaminated by noise. As we shall see later this is rarely if ever of any
concern when we solve differential equations numerically.

7
 1. Continuous vs Discrete functions

1.6 Representing numerical derivatives by

matrices
As we noted above, differentiation is a linear operation. Upon discretizing a function, we obtain a vec-
tor, and we can compute numerical derivatives over the entire mesh. Let our function be y, as above, and
sample it exactly as before. Now use the second order approximation to the derivative, writing all dif-
ference quotients simultaneously to get the matrix--vector representation of a numerical derivative as

If we take a single equation from this matrix--vector relation, we find

which is the standard, symmetric, 2nd order approximation.

The matrix--vector equation means that the derivative, which in the continuous case is a function, is now
immediately represented as a vector. It has two components fewer than the original vector function
(why?). Let's try it out in Matlab.

% Generate equidistant grid on [0,1]

N = 39; % Number of internal grid points
h = 1/(N+1); % Mesh width
xgrid = linspace(0,1,N+2)';
% Sample function at the grid points
ysample = 1+xgrid/2-xgrid.^2+0.3*exp(-xgrid)+0.5*sin(3*xgrid).^3;
% Create matrix
null = zeros(N,1);
col = null;
col(2) = -1;
row = -col';
D = toeplitz(col,row);
left = null;
right = null;
left(1) = -1;
right(N) = 1;
D = [left D right];
D = D/h/2; % Finite difference matrix
yprimesample = D*ysample; % Numerical differentiation
xprimegrid = xgrid(2:N+1);
% Plot function (blue) and derivative (red)
plot(xgrid,ysample,'b.','MarkerSize',10);
hold on
plot(xprimegrid,yprimesample,'r.','Markersize',10);
title('A discrete function (blue) and its numerical derivative (red)')
hold off

8
 1. Continuous vs Discrete functions

In this example we took a discrete function on the grid -- a vector -- and found a "discrete derivative" by
multiplying that vector by the matrix that corresponds to a symmetric 2nd order finite difference approx-
imation to the derivative. Note that because the grid was given, one cannot choose the mesh width h ar-
bitrarily -- it is already defined by the original vector.

References
Lennart Edsberg: Introduction to computation and modelling for differential equations, John Wiley
2008

Published with MATLAB 7.10