You are on page 1of 7

EPIPOLAR RESAMPLING OF HIGH RESOLUTION SATELLITE IMAGERY

Tetsu Ono, Graduate School of Engineering, Kyoto University,


Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, JAPAN,
ono@jf.gee.kyoto-u.ac.jp

ABSTRACT
This paper presents a practical method of epipolar resampling of high-resolution satellite imagery. Satellite
imagery imaged with a linear array CCD sensor has quite different geometric characteristics from aerial
photographs, therefore the conventional method of epipolar resampling is not applicable to it. On the other hand,
epipolar resampling method based on rigorous orientation model with high geometric fidelity becomes too
complicated and then is not suitable practical use. In order to overcome this problem, the author proposes to
apply the well-established 2D affine orientation model to the epipolar resampling of satellite imagery. Firstly,
this paper roughly mentions the characteristics of this model. Then it is shown how the model is suitably
applicable to epipolar resampling. Secondly, the paper proposes the improved method that does not require any
DTM or rigorous geometric parameters for reduction of vertical parallaxes. Finally an experiment validates the
proposed method with SPOT imagery, where the RMS value of vertical parallaxes between a pair of stereo
epipolar images was less than half a pixel.
KEYWORDS: Epipolar Resampling, High-Resolution Satellite Imagery, CCD line scanner imagery, 2D Affine
Orientation Model, Affine Transformation
1. INTRODUCTION
High-resolution satellite imagery is expected to be a
major source of 3D measurement of ground in the
near future. Especially automatic stereo plotting to
generate DTMs by stereo matching technology is
highly required. However, the projection of satellite
imagery, which is imaged with a CCD line sensor, is
quite different from that of conventional aerial
photographs. This leads to failure of application of
well-known epipolar geometry. CCD line scanner
imagery is not characterized by rigorous three
dimentional perspective projection in a solid frame,
but two-dimensional sequential perspective projection
in a line. It is already reported that strict epipolar
images cannot be generated from SPOT imagery
without DTMs (Otto, 1988). The same applies to
high-resolution satellite imagery with CCD line
scanner.
For this reason, several procedures which generate
pseudo epipolar image by using DTMs have been
presented.
1. In order to generate coarse pseudo epipolar image,
each pixel of satellite image is projected on a
horizontal plane located at an average terrain

height. A pair of the pseudo epipolar images has


still large vertical parallaxes (Haala, 1998). If
DTMs of the corresponding area exist, the
satellite images are projected onto the DTMs and
reprojected onto a new image plane along epipolar
line (ONeill et al, 1988). The idea of this method
is very simple and applicable to every orientation
model. The procedures, however, are complicated
for practical purposes.
2. The epipolar images are resampled under the
assumption that all time-variant factors linearly
affect to satellite image (Otto, 1988). The
relationship between height differences and value
of vertical parallaxes can be described by an
equation. This procedure is not quite complicated
as far as non-linear effects are not considered.
Both of the procedures cannot achieve practical
accuracy without highly precise orientation
parameters. In this study, the author proposes an
alternate method to generate epipolar imagery, which
is very simple, highly accurate and does not require
the rigorous orientation parameters nor DTMs.

2. GEOMETRIC CHARACTERISTICS OF
HIGH-RESOLUTION SATELLITE IMAGERY
In comparison with mid resolution (5m-10m on the
ground) satellite imagery such as SPOT, 1m highresolution satellite imagery has a much narrower field
angle. This means that the projection of images is
nearly approximated by parallel rather than central
one. If conventional orientation parameters are used,
very high correlation between them occurs.
A high-resolution satellite image covers a very small
ground area in single scene, which is imaged in a
short time span on orbit. The movement of the
satellite can be approximately expressed by linear
function and its attitude parameters are expected to be
almost constant during the short period.
3. 2D AFFINE ORIENTATION MODEL

0 = a11(X-Xoi) + a12(Y-Yoi) + a13(Z-Zoi)


y = a22(X-Xoi) + a23(Y-Yoi) + a23(Z-Zoi)

Okamoto (1999) proposed the orientation theory of


CCD line sensor imagery based on 2D affine
projection. The model named 2D affine orientation
model can be derived from conventional collinearity
equation by consideration of situation mentioned in
the previous section.
Each line of an image is imaged by one-dimensional
central perspective, and each has different exterior
orientation. Let exterior orientation parameters for
line number i be expressed by coordinates of the
projection center Xoi, Yoi, Zoi and angles i, i, i.
These parameters are time variant. Many studies have
indicated that the satellite sensor geometry can be
modeled by elliptical orbit and in this case its attitude
parameters can be expressed with polynomials. The
effect of earth rotation and earth curvature must be
considered. The collinearity equation is described as:

(2)
(3)

where aij (i=1,2; j=1,2,3) are elements of the matrix


(RiRiRi) T . We assume further that the sensor
moves linearly in space and the attitude does not
change. The projection center in each line is
described as follows:
Xoi = Xo + X i
With Xo and X being constant value. The similar
expressions are defined likewise for Yoi and Zoi. Line
number i is expressed by Equation 2 and these ones.

i=

3.1 The Basic Equations

0
X X oi

T
y = (Ri Ri Ri ) Y Yoi
c
Z Z
oi

equation in Equation 1 loses the meaning and the


equation can be described as follows:

a11 ( X X o ) + a12 (Y Yo ) + a13 ( Z Z o )


(4)
a11 X + a12 Y + a13 Z

Now, line number i can be replaced by image


coordinate x. Assuming that the attitude does not
change, aij are regarded as constant parameters.
Equation 4 arranged for the constant coefficients is
described by algebraic expression.

x = A1 X + A2Y + A3 Z + A4

(5)

Equation 3 is also expressed by similar arrangement.

y = A5 X + A6Y + A7 Z + A8

(6)

where Ai (i=1,,8 ) are independent coefficients.


Equation 5 and 6 describe the collinearity relationship
between the coordinates (x,y) of 2D affine image and
ground coordinates (X,Y,Z).
3.2 Image Transformation

(1)

where (X,Y,Z) is the ground coordinates of an object


point, is scale parameter, c is principal distance, y is
coordinate of image point and Ri, Ri, Ri are rotation
matrixes.
Now that the scene is projected to the image by
parallel projection, c can be set to infinity. The third

In reality, satellite images are taken centralperspectively in scanning direction. For rigorous
analysis, the affine image coordinate y must be
transformed to the corresponding original image
coordinate yp. The relationship between yp and y at
plain field is given in the form (Okamoto et al, 1992)

y = y p /(1 (tan ) y p / c)

(7)

Fig. 1. Transformation of a central-perspective line


image into an affine one
However, at hilly area or mountainous area, we
cannot ignore the image transformation errors due to
height differences in the terrain. Let Z indicate
height difference of a ground point from the average
height and denote the half of the field angle of the
scanner. The image transformation error y due to
neglecting the height difference Z is described as
follows (Okamoto et al, 1992):

y = Z (tan( + ) tan ) cos

(8)

The model based on affine projection can express


any liner movement and distortion relating to the
images. Although the model is derived under the
assumption that the attitude of sensor does not change
during the acquisition of one scene image, small
changes of attitude parameters can be also estimated
by the model as far as the effects are regarded as
approximately linear.
Under the geometrically
peculiar condition of high-resolution satellite images,
the attitude parameters highly correlate to the
movement parameters. For example, small changes of
are very similar to changes of Zo. Small changes of
are almost same as changes of Xo and Yo. Because
the attitude of satellite is stable, the effects can be
embodied as linear movement or distortion in the
affine images. The effects of earth curvature and
earth rotation are also estimated with them in small
area.
Moreover, this model is capable of geometrically
preprocessed images (e.g., SPOT Level-1B), because
the model does not treat the geometric orientation
parameters directly and then the model allows
deformed images as far as the deformations are linear.
The model is applicable for even the rotated images or
flipped images. This characteristic is very important
for the epipolar resampling method in this study.
4. THE PRIMARY THEORY OF EPIPOLAR
RESAMPLING FOR AFFINE IMAGES
4.1 Epipolar Geometry of Affine Imagery

Fig. 2. Image Transformation Error Due to


Neglecting Height Difference in the Terrain

3.3 Characteristics of 2D Affine Orientation Model


2D affine orientation model has only 8 algebraic
parameters and the basic equations are linear with
respect to the object space coordinates. Therefore, it
is very simple, stable and fast for mapping processing.

The parallel projection can be expressed as the


central perspective projection with infinity focal
length. Accordingly, we can say that the affine
projection is a special case of the central perspective
projection. For this reason, epipolar geometry of the
affine images is considered by same approach as that
of the central perspective projection images. The
well-known epipolar geometry is illustrated by Fig. 3.
The epipolar plane is defined as a plane in which the
projection center of left image, that of right images
and an object point lie. The projection center of each
image is only one, thus the epipolar plane is
determined by each object point location.
On the contrary, the affine image has no projection
center. In the affine image the direction of projected
ray is same for any point on the image. Now
considering a ray projected from an object point to an
affine image, the epipolar plane can be defined as a
plane in which the rays of the two images lie. The

epipolar line is an intersection line between the affine


image and the epipolar plane. In order to generate the
epipolar images, each affine image can be projected to
a same virtual plane by parallel projection (Fig. 4, Fig.
5).
Let the virtual plane be a plane parallel to the right
image. The image projected on the plane from right
image is identical to the right image. On the
condition, projecting the left image to the virtual plane
is just same as processing affine transformation from
the left image to the right image. The relationship
between the left image point coordinates (xl, yl) and
the corresponding right image point coordinates (xr,
yr) is simply described in the following form:

xl = K 1 x r + K 2 y r + K 3
yl = K 4 x r + K 5 y r + K 6

Fig. 3. Well-Known Epipolar Geometry


(9)

where Ki (i = 1,,6) are independent coefficients.


Number of unknown parameters is 6 and number of
independent equations is 2. Thus, if more than 3
known points are given, the equations can be solved.
Another question is how to determine the direction
of the epipolar line. For this purpose, we shall
consider algebraic solutions of epipolar line of affine
images. As the author has mentioned before, the basic
equations of affine images are described by equation 5
and 6. These equations are written down for a stereo
pair of affine images in the form:

Fig. 4. Epipolar Geometry of Affine Imagery

xl = Al1 X + A2l Y + A3l Z + A4l


yl = A5l X + A6l Y + A7 l Z + A8l
x r = Alr X + A2 r Y + A3r Z + A4 r

(10)

y r = A5 r X + A6 r Y + A7 r Z + A8 r
By eliminating X and Y from these equations and
rearranging them, the relationship of the left image
point and the right image point with change an object
height Z is described as follows:

x r = B1 xl + B2 y l + B3 Z + B4
y r = B5 xl + B6 y l + B7 Z + B8

(11)

By eliminating Z from these equations, the epipolar


line of affine images is expressed in the form:

Fig. 5. Epipolar Resampling from


Affine Imagery

y r = C1 x r + C 2 y l + C 3 xl + C 4

(12)

Since number of unknown coefficients is 4 in


Equation 12, more than 4 known identical points
coordinates between left image and right image are
required for the solution of this equation. The
epipolar resampled images can be generated by
rotating the virtual plane images by the angle
corresponded to C1. Finally, in order to generate
epipolar images from affine images, all we need to
know is the coordinates of more than 4 identical
points of the left and right affine images. The
information such as the attitude of the images is not
required at all.
4.2 Application to satellite imagery
Since the actual satellite images are not affine ones,
the images should be approximately transformed into
affine ones by Equation 7 and 8. Equation 8 indicates
that DTMs are required for rigorous transformation.
As we shall see later, however, we do not have to
necessarily use DTMs for the purpose of epipolar
resampling. The transformation error y in scanning
direction causes vertical parallax x (Fig. 6). In order
to avoid this problem, the transformation should be
carried out along the epipolar lines instead of the
scanning lines. Although it is very hard to find the
true epipolar lines on the original image, the direction
of the epipolar line corresponding to the affine images
can be determined easily. The procedures of epipolar
resampling of satellite images are as follows.
1. Approximately, transform the original images to
the affine ones along scanning line
2. Determine the direction of the approximate
epipolar lines on the affine image
3. Transform the original images to the affine
images along the approximate epipolar lines
4. Determine the affine transformation coefficients
(Equation 9)
5. Carry out the affine transformation
6. Determine the direction of the accurate epipolar
line
7. Rotate the affine transformed images by angle of
the epipolar line.

Fig. 6. Vertical Parallax due to Transformation


Error into Affine Images

The epipolar images are generated from the affine


images by using affine transformation. The epipolar
image, that is, also can be treated as another affine
image.
As mentioned in the previous section,
therefore, 2D affine orientation model can be directly
applicable to the epipolar images. The relationship
between ground coordinates (X, Y, Z) of an object
point and image coordinates (xe, ye) of the
corresponding point on epipolar image are described
by same expression as Equation 5 and 6.

xe = D1 X + D2Y + D3 Z + D4
y e = D5 X + D6Y + D7 Z + D8

(13)

The coefficients Di ( i = 1,,8) are determined by the


least squares method with more than 4 ground control
points data. Since the basic equations are very simple,
this method is appropriate for real time mapping of
satellite imagery.

5. PRACTICAL EVALUATION
5.1 Test Field and Images

good. But, the vertical parallax by this approach was


comparatively large. It is likely that the geometric
parameters were not gotten accuracy enough.

Since unfortunately 1m high-resolution satellite


images were not available, the author used a stereo
pair of SPOT images in order to investigate the
characteristics of this method. Table-1 shows the
condition of the test images. The stereo scene covers
Hanshin area (Osaka, Kobe and the suburbs) in
JAPAN. The southern area of test field is city area
and almost flat. The northern area and the western
area are mountainous terrain. The maximum height
difference is about 1,000m. For the purpose of
verification, 141 check points were measured by
manual. The measurement accuracy of these points is
1/2 pixel to 1/4 pixel. 9 points among the check
points are used for determination of coefficients in
basic equations (Equation 13). Fig.7 shows the test
images and the distribution of the check points.
Table-1 Test Image Data
Left Image
Right Image
Image type
SPOT pan Level-1A
Date
1996.11
1995.2
Lat./Long.
N34.7/E135.5
N32.7/E135.2
Incident angle
L23.0
R17.9
B/H
0.75

Left Image

5.2 Results and Discussion


In this study, four different approaches for epipolar
resampling were evaluated.
1. Ottos approach which uses geometric orientation
parameters
2. Proposed approach, but direction of epipolar line
is not considered.
3. Proposed approach, but DTMs is used at
transformation into affine images.
4. Proposed approach
The results obtained by these approaches are shown
in Table-2. The accuracy of Ottos approach depends
on that of orientation parameters. In this study,
Equation 1 was used as the collinearity equations for
determination of the orientation parameters. The
changes of the parameters were assumed to be linear,
because non-linear model isnt appropriate for Ottos
approach. The RMSE in X,Y of the orientation was
5.1m and that in Z was 6.7m. This result was very

Right Image
Fig. 7. Test Images and Check Points
Black points: check points White points: control points

On the contrary, it seems that the proposed approach


worked efficiently.
The different between the
accuracy of 2nd approach and that of 4th approach
indicates that consideration of epipolar line direction

is effective for reduction of transformation error due


to neglecting height differences.
Besides, the
proposed approach without DTMs is no less accurate
than the case using DTMs. From these results, it can
be concluded that epipolar resampling of satellite
imagery in practical accuracy can achieve without
DTMs by using the proposed method.
For the reference of discussion, the results of the
orientation with 2D affine orientation model are
shown in Table-3. We can see that application of the
2D affine orientation model is quite adequate for the
epipolar images.
Table-2 Results of Each Approach (pixel)
1
2
3
4
RMS of
v-parallax

0.615

0.562

0.460

0.469

Table-3 Results of Orientation (m)


8

o (x 10 )
RMSE in X,Y
RMSE in Z

Original Images

Epipolar Images

5.92
5.814
7.951

4.92
5.996
7.671

6. CONCLUSIONS
The epipolar resampling approach presented in this
paper is based on affine projection imagery. This
method is, therefore, appropriate for small area
mapping with almost parallelly projected imagery
such as high-resolution satellite imagery.
The
proposed method does not require DTMs or rigorous
geometric orientation parameters. In the practical
experiments, it was shown that accuracy better than
half a pixel was achieved by the proposed approach.
Furthermore, since the resampling process is
independent from the orientation process and the basic
equations of the orientation are very simple, this
method is appropriate for real time mapping.
ACKNOWLEDGEMENTS
The author would like to dedicate this paper to the
late Dr. A. Okamoto, who had built the fundation of
this method and had given many advices. The author
also wishes to thank to Dr. S.Hattori and Mr. H.
Hasegawa for their assistance with the preparation of
this paper.

REFERENCES
1. G. P. Otto, 1988. Rectification of SPOT Data for
Stereo Image Matching, International Archives of
Photogrammetry and Remote Sensing, Vol.27,
B3, pp.635-645.
2. M. A. ONeill and I. J. Dowman, 1988. The
Generation of Epipolar Synthetic Stereo Mates for
SPOT Images Using a DEM, International
Archives of Photogrammetry and Remote
Sensing, Vol.27, B3, pp.587-598.
3. Norbert Haala, Dirk Stallmann and Christian
Sttter, 1998. On The Use of Multispectral and
Stereo Data from Airborne Scanning Systems for
DTM Generation and Landuse Classification,
Internationl Archives of Photogrammetry and
Remote Sensing, Vol. 32, B4, pp.204-210
4. T. Ono, A. Okamoto, S. Hattori, H. Hasegawa,
1996. Fundamental Analytic of Satellite CCD
Camera Imagery Using Affine Transformation,
International Archives of Photogrammetry and
Remote Sensing, Vol.31, Commision III, pp.611615.
5. A. Okamoto, C. Fraser, S. Hattori et al, 1998. An
Alternative Approach to the Triangulation of
SPOT Imagery, International Archives of
Photogrammetry and Remote Sensing, Vol.32,
B4, pp.457-462.
6. A. Okamoto, T. Ono, S. Akamatsu et al, 1999.
Geometric
Characteristics
of
Alternative
Triangulaion Models for Satellite Imagery,
Proceedings of ASPRS 1999 Annual Conference.
7. A. Okamoto, S. Akamatsu, H. Hasegawa, 1992.
Orientation Theory for Satellite CCD LineScanner Imagery of Hilly Terrains, International
Archives of Photogrammetry and Remote
Sensing, Vol.29, Commission II, pp.217-222.
8. O. Hofmann, 1986. Dynamische Photogrammetrie. BuL, Vol.54(5), 105-121.
9. V. Kratky, 1989. On-Line Aspects of Stereophotogrammetric Processing of SPOT Images,
Photogrammetric Engineering & Remote Sensing,
Vol,55(3), 311-316.