Meshfree Chapter 12

MATH 590: Meshfree Methods
Chapter 12: Interpolation with Compactly Supported RBFs in

M ATLAB
Greg Fasshauer
Department of Applied Mathematics

Illinois Institute of Technology
Fall 2010
fasshauer@iit.edu MATH 590 Chapter 12 1

Outline
1 Introduction
2 kd-Trees
3 Assembly of the Sparse Interpolation Matrix
4 Numerical Experiments with CSRBFs

Introduction
We now have an alternative way to construct an RBF interpolant

to scattered data in Rs .
If we use the compactly supported radial functions of the previous
chapter then the main difference to our previous interpolants is
that now the interpolation matrix can be made sparse by scaling
the support of the basic function appropriately.
To achieve this we use as we did earlier the basic functions
(r ) = (r ).
Thus, a large value of corresponds to a small support.
In other words, if the support of is the interval [0, 1], then the
support radius of is given by = 1/ so that (r ) = 0 for
r > = 1/.

Introduction
Since we know that the interpolation matrix will be a sparse matrix,

we want to write M ATLAB code to efficiently assemble the matrix.
Once we have defined a sparse matrix, M ATLAB will automatically
use state-of-the-art sparse matrix techniques to solve the linear
system.
Obviously, we dont want to compute the matrix entries for all pairs
of points since we know all of the entries for far away points will be
zero.
Therefore, an efficient data structure is needed.
We use kd-trees implemented in an external M ATLAB package
from [MCFE] (more later).
Some information on kd-trees is provided in the next section.
Data structures for the use with meshfree approximation methods
are also discussed in [Wendland (2005a)].

kd-Trees
In order to deal with large sets of data efficiently we frequently use

compactly supported basic functions.
For their successful implementation certain geometric information
is required.
We need to know which data sites lie in the support of a given basis
function. Such a query is known as a range search.
Were also interested in finding all centers whose support contains
a given (evaluation) point x. Such a query is known as a
containment query.
We might also be interested in finding the (n) nearest neighbors of
a given point (for instance if we need to find the separation distance
qX of a set of points X ).
One way to accomplish these tasks is via kd-trees.

kd-Trees
A kd-tree (short for k -dimensional tree) is a space-partitioning

data structure for organizing points in k -dimensional space.
Purpose of kd-trees: to hierarchically decompose a set of N data
points in Rk into a relatively small number of subsets such that
each subset contains roughly the same number of data sites.
Each node in the tree is defined by a splitting plane that is
perpendicular to one of the coordinate axes and passes through
one of the data points.
Therefore the splitting planes partition the set of points at the
median into left and right (or top and bottom) subsets, each
with roughly half the points of the parent node.

kd-Trees
These children are again partitioned into equal halves, using

planes through a different dimension (usually one keeps on
cycling through the dimensions when determining the next
splitting plane).
This partitioning process stops after log N levels.
In the end, every node of the kd-tree, from the root to the leaves,
stores a point.

kd-Trees
0.9
5
1
0.8
7
0.7
0.6
8 2 3
9
0.5
2
0.4
1
4 5 6 7
0.3
4
0.2
0.1 3 8 9
6
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Figure: Standard median-based partitioning of nine Halton points in [0, 1]2

(left) and associated kd-tree (right).

kd-Trees
The computational complexity for building a kd-tree from N points

in Rk is O(kN log N).
Once the tree is built, a range query can be performed in O(log N)
time.
This compares favorably with the O(N) time it would take to
search the raw data set.

kd-Trees
We use an external kd-tree package. There are several of these

available at [MCFE]:
In the book, I used a set of M ATLAB MEX-files called k-D tree
written by Guy Shechter.
However, since MEX files are not a real part of M ATLAB this might
cause complications when moving between OSs.
I now suggest using a new all-M ATLAB package called Kdtree
implementation in matlab written by Pramod Vemulapalli.
The major disadvantage of this package is that it is quite slow.
One of the most popular C++ implementations of nearest neighbor
searching is called ANN and provided in [ANN].
The Statistical Learning Toolbox from [MCFE] written by Dahua Lin
provides a M ATLAB wrapper for the ANN C++ library as well as a
number of other tools. This toolbox requires the M ATLAB
Optimization Toolbox.

kd-Trees
In our M ATLAB examples we use the functions

kdtree or kd_buildtree
kdrangequery or kd_rangequery
from the kd-tree libraries k-D tree or Kdtree, respectively.
However, the original code for kd_rangequery had to be re-written to

deal more efficiently with isotropic L2 -radial functions. It is now
available on the class download page as kd_rangequery_ball.

Assembly of the Sparse Interpolation Matrix
We have structured the scattered data interpolation program in the

compactly supported case analogous to the code for the global
interpolants, i.e.,
first construct a distance matrix,
and then apply the anonymous function rbf to obtain the
interpolation/evaluation matrix (as on lines 1314 and 1516 of
RBFInterpolation2D.m).
However, it turns out that it is easier to deal with the compact support if
we compute the distance matrix corresponding to the (1 r )+ term
since otherwise those entries of the distance matrix that are zero
(since the mutual distance between two identical points is zero) would
be lost in the sparse representation of the matrix.

Program (DistanceMatrixCSRBF.m)
1 function DM = DistanceMatrixCSRBF(dsites,ctrs,ep)
2 N = size(dsites,1); M = size(ctrs,1);
% For each ctr/dsite, find the dsites/ctrs
% in its support along with u-distance u=1-ep*r
3 supp = 1/ep; nzmax = 25*N; DM = spalloc(N,M,nzmax);
4 if M > N % faster if more centers than data sites
5 T = kd_buildtree(ctrs,0);
6 for i = 1:N
7 [idx,dist,pts]=kd_rangequery_ball(T,dsites(i,:),supp);
8 DM(i,idx) = 1-ep*dist;
9 end
10 else
11 tTree = kd_buildtree(dsites,0);
12 for j = 1:M
13 [idx,dist,pts]=kd_rangequery_ball(T,ctrs(j,:),supp);
14 DM(idx,j) = 1-ep*dist;
15 end
16 end
17 clear T

Remark
The M ATLAB code DistanceMatrixCSRBF.m contains two similar
blocks that will be used depending on whether we have more centers
than data sites or vice versa.
Example
If there are more data sites than centers (cf. lines 59), then we
build a kd-tree for the data sites and
find for each center x j those data sites within the support of
the basis function centered at x j ,
i.e., we construct the (sparse) matrix column by column.
In the other case (cf. lines 1115) we
start with a tree for the centers and
build the matrix row by row.
This is accomplished by determining for each data site x i all
centers whose associated basis function covers data site x i .
Remark
As mentioned above, kd_buildtree is provided by the kd-tree
library and kd_rangequery_ball is a modified version of the
library code kd_rangequery (which finds points inside a
rectangular box surrounding the query point instead of inside a
hyper-sphere as needed for RBFs).
The call in line 5 (respectively 11) of DistanceMatrixCSRBF.m
generates the kd-tree of all the centers (data sites), and with the
call to kd_rangequery_ball in line 7 (respectively 13) we find
all centers (data sites) that lie within an isotropic L2 -distance supp
of the jth center point (data site).
The actual distances are returned in the vector dist and the
indices into the list of all data sites are provided in idx.
The distances for these points only are stored in the matrix DM.

Remark
For maximum efficiency (in order to avoid dynamic memory
allocation) it is important to have a good estimate of the number of
nonzero entries in the matrix for the allocation statement in line 3.
The reason for coding DistanceMatrixCSRBF.m in two different
ways is so that we will be able to speed up the program when
dealing with non-square (evaluation) matrices (for example in the
context of MLS approximation (c.f. Chapter 24).

The following implementation of DistanceMatrixCSRBF might be

more efficient, but more difficult to follow.

Program (DistanceMatrixCSRBFA.m)
1 function DM = DistanceMatrixCSRBF(dsites,ctrs,ep)
2 N = size(dsites,1); M = size(ctrs,1);
3 supp = 1/ep; nzmax = 25*N;
4 rowidx = zeros(1,nzmax); colidx = zeros(1,nzmax);
5 validx = zeros(1,nzmax); istart = 1; iend = 0;
6 if M > N % faster if more centers than data sites
7 T = kd_buildtree(ctrs,0);
8 for i = 1:N
9 [idx,dist,pts]=kd_rangequery_ball(T,dsites(i,:),supp);
10 newentries = length(idx);
11 iend = iend + newentries;
12 rowidx(istart:iend) = repmat(i,1,newentries);
13 colidx(istart:iend) = idx;
14 validx(istart:iend) = 1-ep*dist;
15 istart = istart + newentries;
16 end
17 else [ similar code ] end
29 idx = find(rowidx);
30 DM = sparse(rowidx(idx),colidx(idx),validx(idx),N,M);
31 clear T

The interpolation program is virtually identical to

RBFInterpolation2D. The only changes are to replace lines 13 and
15 by the corresponding lines
13 DM_data = DistanceMatrixCSRBF(dsites,ctrs,ep);
15 DM_eval = DistanceMatrixCSRBF(epoints,ctrs,ep);
and to define the RBF in shifted form, i.e., instead of representing, e.g.,
the C 2 Wendland function 3,1 on line 1 by
1 rbf = @(e,r) max(1-e*r,0).^4.*(4*e*r+1); ep=0.7;
we now write
1 rbf = @(e,r) r.^4.*(5*spones(r)-4*r); ep=0.7;
Remark
Note the use of the sparse matrix of ones spones. Had we used
5-4*r instead, then a full matrix would have been generated (with
many additional and unwanted ones).
Remark
In order to speed up the solution of the (symmetric positive
definite) sparse linear system we could use the preconditioned
conjugate gradient algorithm (pcg in M ATLAB) instead of the basic
backslash \ (or matrix left division mldivide) operation, i.e., we
could replace line 17 of RBFInterpolation2D by
17 c = pcg(IM,rhs); Pf = EM * c;
Note, however, that the backslash \ operator also employs
state-of-the-art direct sparse solvers by first applying a minimum
degree preordering.

Numerical Experiments with CSRBFs The Stationary Scheme
We now present two sets of interpolation experiments with

compactly supported RBFs.
The results presented below were obtained with the older
MEX-file library k-D tree by Guy Shechter (the new code provides
identical errors, but takes much longer).
First we use the stationary approach,
i.e., we scale the support size of the basis functions proportionally
to the fill distance hX , (defined in Chapter 2).
Here the bandwidth of the interpolation matrix A is constant.
This theoretically results in O(N) computational complexity, i.e., a
very efficient interpolation method.
The stationary interpolation method is also numerically stable, but
there will be essentially no convergence.

Example
We use Wendlands compactly supported function
3,1 (r ) = (1 r )4+ (4r + 1)
to interpolate Frankes function on grids of equally spaced points

in the unit square [0, 1]2 .
In the stationary case the support of the basis function starts out
with an initial scale parameter = 0.7 which is subsequently
multiplied by a factor of two whenever the fill distance is halved,
i.e., when we repeat the experiment on the next finer grid.
This corresponds to keeping a constant number of roughly 25 data
sites within the support of any basis function.
Therefore, the bandwidth of the interpolation matrix A is kept
constant (at 25), so that A is very sparse for finer grids.

N RMS-error rate % nonzero time
9 1.562729e-001 100 0.23

25 2.690350e-002 2.5382 57.8 0.31
81 1.027881e-002 1.3881 23.2 0.33
289 6.589552e-003 0.6414 7.47 0.41
1089 3.891263e-003 0.7599 2.13 0.63
4225 3.726913e-003 0.0623 0.57 1.23
16641 2.638296e-003 0.4984 0.15 3.75
66049 2.467867e-003 0.0963 0.04 15.48
Table: Stationary interpolation at N equally spaced points in [0, 1]2 (constant

25 points in support) with Wendlands function (r ) = (1 r )4+ (4r + 1).

Remark
The rate listed in the table is the exponent of the observed
RMS-convergence rate O(hrate ).
It is computed using the formula
ln(ek 1 /ek )
ratek = , k = 2, 3, . . . , (1)
ln(hk 1 /hk )
where ek is the error for experiment number k , and hk is the fill

distance of the k th computational mesh.
For uniformly spaced points the ratio of fill distances of two
consecutive meshes will always be two,
while for random points (such as Halton points) we estimate the fill
distance via
hX = max(min(DM_eval)).

Remark
The % nonzero column indicates the sparsity of the interpolation
matrices,
and the time is measured in seconds.
Errors are computed on an evaluation grid of 40 40 equally
spaced points in [0, 1]2 .

Remark
We can observe nice convergence for the first few iterations, but once
an RMS-error of approximately 5 103 is reached, there is not much
further improvement.
This behavior is not yet fully understood.
However, it is similar to what happens in the approximate
approximation method of Mazya (see, e.g.,
[Mazya and Schmidt (2001), Mazya and Schmidt (2007)] and our
discussion in Chapter 26).

Numerical Experiments with CSRBFs The Non-Stationary Scheme
Example
Now we use the non-stationary approach to interpolation,
i.e., we use basis functions without adjusting their support size, i.e.,
= 0.7 is kept fixed for all experiments.
We have convergence although it is not obvious what the rate
might be.
However, the matrices become increasingly denser, computation
requires lots of system memory, and therefore the non-stationary
approach is very inefficient.

N RMS-error rate time
9 1.562729e-001 0.03
25 2.807706e-002 2.4766 0.04
81 4.853006e-003 2.5324 0.12
289 2.006041e-004 4.5965 0.45
1089 1.288000e-005 3.9611 2.75
4225 1.382497e-006 3.2198 47.92
Table: Non-stationary interpolation at N equally spaced points in [0, 1]2

( = 0.7 fixed) with Wendlands function (r ) = (1 r )4+ (4r + 1).

Remark
The time comparison between the entries in the two tables above is
not a straightforward one since we used the (dense) code
RBFInterpolation2D to do the experiments for the non-stationary
experiment since there is no sparseness to be exploited and the
kd-trees actually introduce additional overhead.

The interplay between computational efficiency and

non-convergence in the stationary case and convergence and
computational inefficiency in the non-stationary case is again a
trade-off principle similar to the interplay between accuracy and
ill-conditioning for globally supported RBFs (c.f. Chapter 2).
These trade-off principles were explained theoretically as well as
illustrated with numerical experiments in [Schaback (1997b)], and
we will consider them in Chapter 16.

Numerical Experiments with CSRBFs Oscillatory CSRBFs
Example
For comparison purposes we repeat the experiments with the
oscillatory basic function

(r ) = 2 (r ) = (1 r )6+ 3 + 18r + 3r 2 192r 3 ,
which is also C 4 smooth and strictly positive definite and radial on Rs

for s 3 (see Chapter 11).
Note that the function is implemented as

rbf = @(e,r) -r.^6.*(168*spones(r)-552*r...
+573*r.^2-192*r.^3);
in the sparse setting and as
rbf = @(e,r) max(1-e*r,0).^6.*(3+18*e*r...
+3*(e*r).^2-192*(e*r).^3);
for the dense code.
N RMS-error rate % nonzero time
9 1.655969e-001 100 0.28

25 3.941226e-002 2.0710 57.8 0.34
81 2.978973e-002 0.4038 23.2 0.36
289 2.914215e-002 0.0317 7.47 0.42
1089 3.063424e-002 0.0720 2.13 0.64
4225 3.094308e-002 0.0145 0.57 1.31
16641 3.089882e-002 0.0021 0.15 4.13
66049 3.086639e-002 0.0015 0.04 16.81
Table: Stationary interpolation at N equally spaced points in [0, 1]2 (constant

25 points in support) with the oscillatory function
(r ) = (1 r )6+ 3 + 18r + 3r 2 192r 3 .

N RMS-error rate time
9 1.655969e-001 0.03
25 3.097850e-002 2.4183 0.06
81 4.612941e-003 2.7475 0.20
289 1.305297e-004 5.1432 0.72
1089 4.780575e-006 4.7711 4.06
4225 2.687479e-007 4.1529 55.09
Table: Non-stationary interpolation at N equally spaced points in [0, 1]2

( = 0.7 fixed) with the oscillatory function
(r ) = (1 r )6+ 3 + 18r + 3r 2 192r 3 .


Remark
While the performance of the oscillatory functions for the
stationary experiment is even more disappointing than that of
Wendlands functions, the situation is reversed in the
non-stationary case.
In fact, the errors obtained with the oscillatory basis functions are
almost as good as those achieved with optimally scaled
Gaussians (c.f. Chapter 2).
In order to overcome the problems due to the trade-off principle
that are apparent in both the stationary and non-stationary
approach to interpolation with compactly supported radial
functions we will later consider using a multilevel stationary
scheme (see Chapter 32).

Appendix References
References I
Buhmann, M. D. (2003).
Radial Basis Functions: Theory and Implementations.
Cambridge University Press.
Fasshauer, G. E. (2007).
Meshfree Approximation Methods with M ATLAB.
World Scientific Publishers.
Higham, D. J. and Higham, N. J. (2005).
M ATLAB Guide.
SIAM (2nd ed.), Philadelphia.
Iske, A. (2004).
Multiresolution Methods in Scattered Data Modelling.
Lecture Notes in Computational Science and Engineering 37, Springer Verlag
(Berlin).

Appendix References
References II
Mazya, V. and Schmidt, G. (2007).

Approximate Approximations.
Mathematical Surveys and Monographs, vol. 141, Americal Mathematical Society
(Providence, RI).
Wendland, H. (2005a).
Scattered Data Approximation.
Cambridge University Press (Cambridge).
Mazya, V. and Schmidt, G. (2001).
On quasi-interpolation with non-uniformly distributed centers on domains and
manifolds.
J. Approx. Theory 110, pp. 125145.
Schaback, R. (1997b).
On the efficiency of interpolation by radial basis functions.
in Surface Fitting and Multiresolution Methods, A. Le Mhaut, C. Rabut, and L.
L. Schumaker (eds.), Vanderbilt University Press (Nashville, TN), pp. 309318.

Appendix References
References III
Mount, D. M. and Arya, S. (2010)

ANN: A Library for Approximate Nearest Neighbor Searching.
Available online at http://www.cs.umd.edu/ mount/ANN/.
M ATLAB Central File Exchange.

Available online at
http://www.mathworks.com/matlabcentral/fileexchange/.

Meshfree Chapter 12

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Meshfree Chapter 12

Uploaded by

Copyright:

Available Formats

MATH 590: Meshfree Methods

Chapter 12: Interpolation with Compactly Supported RBFs in

Department of Applied Mathematics

fasshauer@iit.edu MATH 590 Chapter 12 1

3 Assembly of the Sparse Interpolation Matrix

4 Numerical Experiments with CSRBFs

fasshauer@iit.edu MATH 590 Chapter 12 2

We now have an alternative way to construct an RBF interpolant

fasshauer@iit.edu MATH 590 Chapter 12 4

Since we know that the interpolation matrix will be a sparse matrix,

fasshauer@iit.edu MATH 590 Chapter 12 5

In order to deal with large sets of data efficiently we frequently use

fasshauer@iit.edu MATH 590 Chapter 12 7

A kd-tree (short for k -dimensional tree) is a space-partitioning

fasshauer@iit.edu MATH 590 Chapter 12 8

These children are again partitioned into equal halves, using

fasshauer@iit.edu MATH 590 Chapter 12 9

Figure: Standard median-based partitioning of nine Halton points in [0, 1]2

fasshauer@iit.edu MATH 590 Chapter 12 10

The computational complexity for building a kd-tree from N points

fasshauer@iit.edu MATH 590 Chapter 12 11

We use an external kd-tree package. There are several of these

fasshauer@iit.edu MATH 590 Chapter 12 12

In our M ATLAB examples we use the functions

However, the original code for kd_rangequery had to be re-written to

fasshauer@iit.edu MATH 590 Chapter 12 13

We have structured the scattered data interpolation program in the

fasshauer@iit.edu MATH 590 Chapter 12 15

fasshauer@iit.edu MATH 590 Chapter 12 16

fasshauer@iit.edu MATH 590 Chapter 12 18

fasshauer@iit.edu MATH 590 Chapter 12 19

The following implementation of DistanceMatrixCSRBF might be

fasshauer@iit.edu MATH 590 Chapter 12 20

fasshauer@iit.edu MATH 590 Chapter 12 21

The interpolation program is virtually identical to

fasshauer@iit.edu MATH 590 Chapter 12 23

We now present two sets of interpolation experiments with

fasshauer@iit.edu MATH 590 Chapter 12 25

3,1 (r ) = (1 r )4+ (4r + 1)

to interpolate Frankes function on grids of equally spaced points

fasshauer@iit.edu MATH 590 Chapter 12 26

N RMS-error rate % nonzero time

9 1.562729e-001 100 0.23

Table: Stationary interpolation at N equally spaced points in [0, 1]2 (constant

fasshauer@iit.edu MATH 590 Chapter 12 27

where ek is the error for experiment number k , and hk is the fill

fasshauer@iit.edu MATH 590 Chapter 12 28

fasshauer@iit.edu MATH 590 Chapter 12 29

fasshauer@iit.edu MATH 590 Chapter 12 30

fasshauer@iit.edu MATH 590 Chapter 12 31

N RMS-error rate time

Table: Non-stationary interpolation at N equally spaced points in [0, 1]2

fasshauer@iit.edu MATH 590 Chapter 12 32

fasshauer@iit.edu MATH 590 Chapter 12 33

The interplay between computational efficiency and

fasshauer@iit.edu MATH 590 Chapter 12 34

which is also C 4 smooth and strictly positive definite and radial on Rs

Note that the function is implemented as

N RMS-error rate % nonzero time

9 1.655969e-001 100 0.28

Table: Stationary interpolation at N equally spaced points in [0, 1]2 (constant

fasshauer@iit.edu MATH 590 Chapter 12 36

N RMS-error rate time

Table: Non-stationary interpolation at N equally spaced points in [0, 1]2