3D MODELLING OF TOPOGRAPHIC OBJECTS BY FUSING 2D MAPS AND LIDAR

DATA

Sander Oude Elberink and George Vosselman

International Institute for Geo-information Science and Earth Observation (ITC) –

(oudeelberink, vosselman)@itc.nl

Commission IV, WG IV/4

KEY WORDS: 3D modelling, topographic features, data fusion, laser scanner data, segmentation

ABSTRACT:

In the past few years the number of applications that use 3D information of topographical objects increased rapidly. With the
growing demand for 3D topographic data the need for automated 3D data acquisition also grows. Height information can be
extracted from airborne or terrestrial acquisition methods, but can also be modelled as implicit semantic information. Adding height
information to existing features is insufficient; additional features have to be acquired and existing features might get an extra
dimension (surfaces can be converted to volumes, etc). The challenge is to produce semantical, geometrical and topological correct
3D topography. In this paper we describe the steps to acquire 3D topographic information. Special attention lies on the user
requirements of 3D models. These requirements have been accomplished by information analysis at four major geo-information
organizations in The Netherlands. The four cases describe the wishes and requirements for 3D data and modelling. The most
important acquisition task is the modelling of 3D infrastructure and 3D building models. In this paper we will focus on the modelling
of 3D infrastructure in general, and specially the reconstruction of hidden infrastructural object parts. The developed method will be
demonstrated with a 3D reconstruction of the complex motorway interchange ‘Prins Claus Plein’ near The Hague, The Netherlands,
with multiple infrastructural objects crossing each other at different height levels. Although the focus in this paper will lie on the
modelling of 3D infrastructure, the presented 3D map includes the modelling of topographic features completely covering the terrain.

1. INTRODUCTION
In the past few years the number of applications that use the 3D
information of topographical objects increased rapidly.
Examples can be found in location based services, virtual
reality tasks, visualisation for city planning, etcetera. These
applications require 3D topographic input data. Acquiring 3D
topographic information is even more complicated than 2D
data.
With the growing demand for 3D topographic data the need for
automated 3D data acquisition also grows. 3D data acquisition
and object reconstruction is conventionally performed using
stereo image pairs. Photogrammetry is a classic, accurate and
operational approach for 3D data acquisition (Tao, 2005).
However, the automated reconstruction of buildings using only
aerial images as data source has been proven to be a very
difficult problem (Suveg and Vosselman, 2004).
The use of laser altimetry as data source for the (semi-)
automatic reconstruction has been described by several authors
(Brenner, 2000; Elaksher and Bethel, 2002; Maas, 2001;
Vosselman and Dijkman, 2001), and shows great potential to
reliable 3D surface modelling. Some of them use additional
information, like 2D GIS data, in one or more steps of their
methods.
Correctly combining height information with existing 2D maps
has a great potential for a fast, accurate and highly automated
acquisition of 3D maps. Several papers describe the advantage
of using both laser data and 2D maps (Brenner, 2000; Haala et
al., 1998; Hatger and Brenner, 2003; Hofmann, 2004; Koch,
2004; Rottensteiner and Briese, 2002; Vosselman and Dijkman,
2001). Topographic maps provide outlines, classified polygons
and topologic and 2D semantic information. The purposes for
integrating map data and laser data ranges from improving the
filtering process for DTM generation by explicitly modelling
3D breaklines (Briese, 2004) to rapid acquisition of 3D city
models for virtual reality applications (Haala et al., 1998).

In 3D maps it should be possible to acquire multiple

topographic features at one and the same 2D location (e.g.
tunnels, flyovers, etc). Height information can be extracted from
airborne or terrestrial acquisition methods, but can also be
modelled as implicit semantic information. Adding height
information to existing features is insufficient; additional
Figure 1 Example of a 3D modelling features have to be acquired and existing features might get an
application. extra dimension (surfaces can be converted to volumes, etc).
When aiming for fully 3D models, the volume below and on top
of the object surface has to be modelled. Not only does this
mean that acquisition of multiple cartographic object types at
one location is possible or necessary at interchanges, bridges
etc, but it also means that vertical object planes have to be
acquired. Existing 2D objects that indicate height information
have to be revised. In [Penninga, 2005] a summary is given of
some representations of height information: height contours,
shadowing, hatches to indicate height differences at banks and
dikes. If the user really can explore the third dimension some of
these objects will become superfluous.
This research is a part of a project to develop methods for
acquiring, storing, and querying 3D topographic data as a
feasibility study for a future national 3D topographic database.
Usage is therefore made of the current national 2D topographic
database TOP10vector and the national elevation model AHN.
The topic of this paper is twofold. Special attention lies on the
user cases of 3D models. These cases have been accomplished
by information analysis at four major geo-information
organizations in The Netherlands. The cases describe the wish
and requirements for 3D data and modelling, added with a list
of scientific activities to fill the gap between the wishes and
reality. These user cases have been highlighted in chapter 2.
Second issue is describing one of the first research activities:
the modelling of 3D infrastructure. In this paper, chapter 4, we
will focus on the modelling of 3D infrastructure in general, and
specially the reconstruction of hidden infrastructural object
parts.
2. USER CASES
In our research project we defined and analyzed four user cases.
Each user case represents the 3D model requirements of one
specific organization to perform their 3D modelling and
visualization tasks.
2.1 Survey department of Rijkswaterstaat
Rijkswaterstaat is responsible for the maintenance of national
highways and waterways, including bridges, dikes and the
navigability of canals. Geo-information of these infrastructural
objects has been acquired at several scales, using several
spaceborne, airborne, terrestrial and hydrographical
measurement techniques. Rijkswaterstaat focuses on improving
the acquisition, storage and distribution of their geo-
information data. To successfully offer their web based geo
information services and applications, Rijkswaterstaat is
looking for the optimal production of 2, 2.5 and 3D geo-
information. Their focus is on the large scale topography of
infrastructural objects and the medium scale landscape
modelling and visualization.
2.2 Municipality Den Bosch
Den Bosch aims for the production of a large scale 3D GEO
database to inform citizens and to support other departments for
real estate taxes, city planning, noise modelling etcetera. Their
main motive for acquiring 3D data is to model and visualise the
“as-is” situation. Den Bosch has many situations with multiple
land use, like shown in Figure 2. At the moment they have to
store these multiple classifications in multiple 2D layers, which
makes it hard to perform 3D modelling and visualisation tasks.
Figure 2 Multiple land use at one location: buildings
on top of a canal.
Their list with 3D model requirements starts with the modelling
of shapes of buildings, followed by the possibility to store and
analyse multiple objects on top of each other. Research
activities have been determined in the field of semi-automatic
reconstruction of buildings, using high point density airborne
and terrestrial laser scanner data together with a large scale base
map (scale 1:1000). These activities will be carried in 2007 and
2008.
2.3 Water boards
For the inspection and maintenance of regional dikes, bridges
and waterways, the water board needs up-to-date and reliable
geo-information. For several applications, e.g. when combining
topography with hydrological, geological and geotechnical
information, it is necessary to use full 3D topographic
information. At the moment most of the water boards use large
scale 2D base maps and high point density laser data separately
from each other. Integration has been done visually by the user:
information from one source can be used to better interpret the
other. Requirements for a 3D model is that breaklines, e.g. on
top and at the bottom of a dike, are modelled with high
precision. Breaklines are important features for the condition
(shape and strength) of dikes. The reconstruction of breaklines
from laser scanner data has previously been described in
(Briese, 2004). Our research activities are planned for 2007 and
will focus on the integration of existing 2D maps and high point
density laser scanner data, using a full 3D model data structure
(e.g. a TEN structure), as described in (Penninga, 2005).
2.4 Topographic Mapping Agency
The Topographic Mapping Agency of the Dutch Cadastre
produces national 2D topographic databases from scale
1:10.000 to 1:250:000. Users can be found in several public and
private sectors in the Netherlands. To come to meet the growing
3D desires of the users, the Mapping Agency would like to
implement the third dimension to their products. They
participate in this research project by providing data and by
helping to translate user requirements into research activities.
In the remaining part of the paper we will focus on the user case
of the Topographic Mapping Agency as described in 2.4.
3. DATA PROPERTIES
The Topographic Mapping Agency aims to integrate the third
dimension into their medium scale (1:10.000) topographic map,
called the TOP10vector. In this chapter a description is given of
the TOP10vector and the laser data set used in this project.
3.1 Topographic map
TOP10vector is a digital 2D topographic database for usage at a
scale around 1:10.000. It has been built up in a fully coded
object structure. The databases has been built from photographs
in a 1:18.000 scale and has an accuracy of 1-2 m. Small
buildings like houses, are stored in a different layer and are not
shown in Figure 3.
Figure 3 The study area in the TOP10vector database.
One property of the TOP10Vector is that a polygon can have
more than one classification, including the information whether
this class is visible from above or not.
3.2 Laser data
The national elevation model of the Netherlands (AHN) has an
average point density of 1 point per 16 m² or better and a height
precision of about 15 cm standard deviation per point. In the
standard production process the laser data has been filtered,
removing buildings, trees and outliers. This filtered dataset will
normally be interpolated to a regular grid, and delivered in grid
sizes of 5, 25 and 100 meter. However, in this project the
original, unfiltered irregular point cloud has been used in order
to use as much information from the point cloud as possible.
In the first step of data fusion it will be decided which parts of
the laser data will be used to provide height information to
which part of the 2D map. This is one of the most crucial steps
in the automation of the reconstruction process. The easiest way
to select the laser data is just to perform a points-in-polygon
function, where the outlines of the grouped 2D objects act as
polygons. In the ideal case this is enough to select the right
points. However, in many cases not all laser points represent
height information of the topographic object, but may indicate
height of details of this object, e.g. a laser point can lie on a
chimney instead of the roof, or on a car instead of the street. An
important step of the selection is segmenting the point cloud,
according to the rules and conditions of the object (group).
As described in more detail in (Oude Elberink and Vosselman,
2006), laser data has been filtered in a segment based approach
to eliminate laser points on small objects like cars, light poles,
traffic signs, and trees.
For the segmentation of the point cloud a surface growing
algorithm is used with some modifications that allow a fast
processing of large datasets (Vosselman et al., 2004). The
surface growing method consists of a seed surface detection
followed by the actual growing of the seed surface.
4. APPROACH
4.1 Surface modelling
Some aspects of the 3D reconstruction are independent of the
data source. Important examples are the type of surface
representations and object modelling. One way to represent the
terrain given by a set of surface points is to construct a
Delaunay Triangular Irregular Network (TIN). In (Verbree and
Oosterom, 2003) a surface reconstruction method has been
described, based on the Delaunay Tetrahedronised Irregular
Network (TEN), which tessellates the 3D-space with non-
overlapping, adjacent, tetrahedrons. In this part of the research
the surface is represented in a TIN structure; at a later stage a
TEN structure will be used to be able to model volume objects.
4.2 Data fusion
Laser data and 2D map data are integrated and processed in an
object-based approach. For groups of objects rules for 3D
reconstruction are being set-up. These rules have to ensure the
geometrical, topological and semantical correctness of the 3D
map. Adding height information to existing 2D features is not
sufficient; additional features have to be acquired and existing
features might get an extra dimension. Examples are given
showing the automated generation of additional polygons at real
3D objects like viaducts and flyovers. Road objects at those
locations will be converted automatically from surfaces to
volumes, in order to get realistic 3D data.
In this research we recognise and model height discontinuities
between objects that are adjacent in a 2D topographic database.
For modelling the surfaces of the 3D topographic object a point
cloud segmentation algorithm is used. This algorithm preserves
height discontinuities, but eliminates small objects like cars and
traffic signs that should not be included in the 3D topographic
database. Filtering algorithms are also used to select the correct
laser points for modelling the object surfaces.
(a) (b)
(c) (d)
Figure 4 Creating new objects: interchange in 2D (a); height
given to 2D features (b); connecting lower road
parts (c); filling gaps (d).
Figure 4 illustrates four stages in the reconstruction process,
starting with the 2D situation in (a) and ending with the 3D
situation in (d). In (b) heights have been calculated at
boundaries of visible objects, followed by the height
determination of ‘invisible’ objects in (c). In the next part of
this chapter we will describe the 3D modelling of existing
object boundaries in 0, creating new parts that did not exist in
2D, but are necessary in 3D to get a tight surface model (4.4
and 4.5).
4.3 Modelling 3D boundaries coincide within a certain threshold, the reconstructed polygon
will be accepted. With this method also slanted ‘invisible’ roads
As shown in Figure 4a-b, edges that are straight in the 2D map will be reconstructed correctly.
do not need to be straight in the 3D model. To correctly capture
the shape of the boundaries, the edges therefore need to be
described by more points. For this purpose, points were inserted
into the edges of the polygons at every 10 m. For all these
points and the original map points the height needs to be
determined from the laser data. Every map point belongs to two
or more polygons. In each of the neighbouring polygons laser
data is selected to calculate the height at the map point, see
Figure 5. Laser data has been filtered to remove small objects
like cars and traffic signs. By calculating multiple heights at
every map point, height discontinuities can be detected and
modelled. Several constraints have been introduced to get a
topological correct model, see (Oude Elberink and Vosselman,
2006). Figure 7 Polygon connecting two lower road parts, shown as
bold polygon.

4.5 Completing surface model

Figure 5 Calculation of map point height, from grass land (left)
and road object (right).
In figure 6 results are shown for the modelling of 3D
boundaries of a simple crossing. The 3D map points have been
visualised as small red dots. Note that the density of map points
is much higher in the 3D model than in the 2D map.
After the previous step many other gaps remain at both sides of
the ‘invisible’ polygons, as can be seen in Figure 4c. These can
be filled by creating new polygons, which have the 2D shape
(and topology) of the road polygons lying above them. The
heights of the new nodes are determined by searching for map
points at neighbouring polygons that lie on the ground surface.
Doing so, these new polygons are connected to lower
neighbouring polygons, like in Figure 4d.
Adding height to a 2D object not only means giving height to
the boundaries of this object, but also to the surface of the
object. Most of the terrain objects show some relief at its
surface. Laser points lying on the terrain (i.e. not on buildings,
roads, trees, water) are used as nodes in the surface TIN model.
To get a smooth surface at road objects, map points at road
boundaries have been used to generate a constrained TIN
model, without adding laser points lying on that road. Trees and
buildings have not been modelled in this part of the research
project.

Figure 6 Oblique view on boundaries of crossing roads in 2D

(left) and the 3D boundaries (right).

4.4 Connecting road parts

As we can see in Figure 4b gaps will occur when only

modelling visible map features. Additional features have to be
created under bridges and interchanges. The first step in filling
the road gaps is the reconstruction of polygons marked as
‘invisible’, like in Figure 4c. Although it is likely that no laser
points may be available, constraints in the model can fill the
gap and connect two parts of the model in 3D. The modelling of Figure 8 Constrained TIN model with roads and grass land.
the invisible polygons is accepted if the nodes successfully fit to
potential neighbouring polygons. Successfully means that the 5. RESULTS
reconstructed polygon is smooth and connect to neighbouring
road parts. Planes have been fitted through laser points and 3D 5.1 Project results
map points on those neighbouring road parts. At the centre
point of the missing polygon, the height of each plane has been In Figure 9 and in Figure 10 results are shown for the fusion of
determined. If the heights of the planes at the centre point a medium scale topographic map (TOP10Vector) with laser
data with a point density of one point per five m2 (original
dataset of part of the AHN). The developed method is
demonstrated with a 3D reconstruction of the complex
motorway interchange ‘Prins Claus Plein’, with multiple
infrastructural objects crossing each other at different height
levels.
Figure 9 Reconstructed model of Prins Claus Plein.
Figure 10 3D Modelling of existing polygons (left); with
additional features (right).
The major disadvantage of the proposed method is the strong
dependency on the quality of the 2D map. In Figure 11 five of
many missing polygon edges are highlighted. Due to the lack of
2D edges, it is impossible to automatically reconstruct the
accompanying 3D edges. In a later stage of the project a semi-
automated approach will be introduced to be able to intervene
in the reconstruction process.
Figure 11 Several polygon edges missing in 2D map at complex
interchange.
Another disadvantage is the possible time difference between
the acquisition of the map data and in the laser data, resulting in
two different recorded situations.
5.2 Applications
During the modelling of the scene the user can choose to derive
several supplementary products. One side product can be a
Digital Terrain Model (DTM), instead of a Digital Surface
Model (DSM). Objects located above or on top of the surface
can easily be left out when deriving a DTM from the laser point
cloud. Doing so, this DTM excludes 3D objects like buildings,
trees and interchanges but includes semantically correct break
lines at topographic features; Figure 13 shows the DTM in
which 3D objects have been filtered from the laser point cloud
(Figure 12).
Figure 12 TIN of laser points at Prins Claus Plein.
Figure 13 DTM of Prins Claus Plein, derived by filtering 3D
objects.
Several producers of 2D topographic maps struggle with the
implementation of the third dimension, as we have seen in user
case 4 (chapter 2.4). One barrier is the increasing amount of
data when adding laser points as nodes on the 3D surface, the
other is the change of the products’ topologic structure. The
first step to implementation could be the determination of the
height of map nodes, which can be derived from the 3D
boundaries. By doing so, the topology of the map does not have
to change, and the laser points have only been used to
determine the height, but are not part of the end product. This
3D boundary product gives the user height information at nodes
of the 2D map. However, when visualising 3D maps the 3D
surfaces have to be triangulated. This implies a change of the
topological structure of the product.
Several software packages can be used to further process 3D
topographic objects. To give an eye-catching example, recently
Google Sketchup became available for free, allowing basic
handling and editing of 3D data for a large group of users.
When exporting to Google Earth this 3D data can easily be
visualised and distributed, as shown in Figure 14.
Figure 14 Road objects imported in Google Earth.

6. CONCLUSION & OUTLOOK
We have presented a method that recognises and models height
discontinuities between objects that are adjacent in a 2D
topographic database. A segmentation algorithm has
successfully been used to connect laser points on smooth
surfaces and remove small segments. First, the 3D boundaries
have been determined by fitting planes to neighbouring
dominant laser segments. Several connection rules have been
applied to get a tight model at object boundaries. Several
conditions have been applied to get horizontal lakes and smooth
roads. At interchanges and flyovers additional boundaries have
automatically been reconstructed to allow the reconstruction of
3D objects. We’ve added missing polygons to hidden objects to
get a tight surface model.
Together with other research partners we are working on the
modelling of volume objects in a TEN data structure. This gives
the opportunity to reconstruct 3D models with 3D primitives
instead of with 2D surfaces. Next, focus will be on the detailed
reconstruction of buildings, by fusing higher point density laser
data with large scale topographic maps.
ACKNOWLEDGEMENT
This research is partially funded by the Dutch BSIK research
programme Space for Geo-Information, project 3D
Topography. The authors would also like to thank the
Topographic Service of the Dutch Cadastre as well as the
Steering Committee AHN for providing the data.
REFERENCES
Brenner, C., 2000. Towards Fully Automatic Generation of City
Models, XIX ISPRS Congress. IAPRS, Amsterdam,
The Netherlands, pp. 85-92.
Briese, C., 2004. Three-Dimensional Modelling of Breaklines
From Airborne Laser Scanner Data, XXth ISPRS
Congress: Geo-Imagery Bridging Continents. IAPRS,
Istanbul, Turkey.
Elaksher, A. and Bethel, J., 2002. Reconstructing 3D Buildings
From Lidar Data, Symposium 2002 Photogrammetric
Computer Vision. IAPRS, Graz, Austria.
Haala, N., C. Brenner and Anders, K.-H., 1998. 3D Urban GIS
From Laser Altimeter and 2D Map Data, ISPRS
Commission IV – GIS Between Visions and
Applications. IAPRS, Ohio, USA.
Hatger, C. and Brenner, C., 2003. Extraction of Road Geometry
Parameters From Laser Scanning and Existing
Databases, ISPRS working group III/3 workshop `3-
D reconstruction from airborne laserscanner and
InSAR data'. IAPRS, Dresden, Germany.
Hofmann, A., 2004. Analysis of TIN-Structure Parameter
Spaces in Airborne Laser Scanner Data for 3-D
Building Model Generation, XXth ISPRS Congress:
Geo-Imagery Bridging Continents. IAPRS, Istanbul,
Turkey.
Koch, A., 2004. An Approach for the Semantically Correct
Integration of a DTM and 2D GIS Vector Data, XXth
ISPRS Congress: Geo-Imagery Bridging Continents.
IAPRS, Istanbul, Turkey.
Maas, H.-G., 2001. The suitability of Airborne Laser Scanner
Data for Automatic 3D Object Reconstruction, Third
International Workshop on Automatic Extraction of
Man-Made Objects from Aerial and Space Images,
Ascona, Switzerland.
Oude Elberink, S. and Vosselman, G., 2006. Adding the Third
Dimension to a Topographic Database Using
Airborne Laser Scanner Data (to be published),
Photogrammetric Computer Vision 2006. IAPRS,
Bonn, Germany.
Penninga, F., 2005. 3D Topographic Data Modelling: Why
Rigidity Is Preferable to Pragmatism. In:
A.G.C.a.D.M. Mark (Editor), Spatial Information
Theory, COSIT, Ellicotville, USA, pp. 409-425.
Rottensteiner, F. and Briese, C., 2002. A New Method for
Building Extraction in Urban Areas from High-
Resolution Lidar Data, Symposium 2002
Photogrammetric Computer Vision. IAPRS, Graz,
Austria, pp. 295-301.
Suveg, I. and Vosselman, G., 2004. Reconstruction of 3D
building models from aerial images and maps. ISPRS
Journal of Photogrammetry and Remote Sensing,
58(3-4): 202-224.
Tao, V.C., 2005. 3D Data Acquisition and Object
Reconstruction for AEC and CAD. In: S. Zlatanova
and D. Prosperi (Editors), Large-scale 3D Data
Integration. CRCpress, Boca Raton, Florida, USA,
pp. 245.
Verbree, E. and Oosterom, P.V., 2003. The Stin Method: 3D-
Surface Reconstruction by Observation Lines and
Delaunay TENS, ISPRS working group III/3
workshop `3-D reconstruction from airborne
laserscanner and InSAR data', Dresden, Germany.
Vosselman, G., B. Gorte, G. Sithole and Rabbani, T., 2004.
Recognising Structure in Laser Scanner Point Clouds,
International Conference NATSCAN “Laser-Scanners
for Fores and Landscpae Assessment – Instruments,
Processing Methods and Applications, ISPRS
working group VIII/2, Freiburg im Breisgau,
Germany.
Vosselman, G. and Dijkman, S., 2001. 3D Building Model
Reconstruction from Point Clouds and Ground Plans,
ISPRS Workshop Land Surface Mapping and
Characterization Using Laser Altimetry. IAPRS,
Annapolis, USA.