#010

Automatic Structure Detection in a Point-Cloud of an Urban Landscape

George Sithole, George Vosselman
structures that are extensions to the Bare Earth (e.g., bridges, ramps, etc.,). The work presented is an extension to the work done on the filtering and segmentation of irregular ALS pointclouds [11]. The paper has been organized into six main sections. The second section looks at the definition of different elements of the landscape for the purposes of structure detection. Section three discusses some related work. In section four, five and six the algorithm is elaborated. In section seven an implementation is demonstrated and results are presented and discussed.

Abstract A method for detecting urban structures in an irregularly spaced point-cloud of an urban landscape is proposed. The method is especially designed for detecting structures that are extensions to the bare-earth (e.g., bridges, ramps, etc.,). The method involves a segmentation of a point-cloud followed by a classification. Both the segmentation and classification of the data are based on the analysis of a data structure in which the pointcloud is represented as an orthogonal set of profiles. Also proposed is a conceptual and logical model of the Landscape for the structure detection problem. Index Terms Classification, LIDAR, Segmentation, Surface reconstruction.

II. THE LANDSCAPE I. INTRODUCTION utomated detection of structures (buildings, vegetation, bare-earth, etc.,) from analyses of point-clouds of urban landscapes has become increasingly more important as urban planners and designers have become more demanding for ontime digitally classified urban data. This demand is driven by the necessity to adjust to the needs of rapidly changing urban environments and populations. Research in automated urban structure detection to match this demand has been extensive, as evidenced by the number of algorithms developed in this area. Furthermore, in recent times Airborne Laser Scanning (ALS) has become a popular data acquisition tool (producing irregular point-clouds). One of the most demanding tasks in ALS is the abstraction (filtering) of the Bare Earth from ALS point-clouds. This abstraction is made even more demanding by the fact that certain features such as bridges, and ramps are either part of the Bare Earth or Not Bare Earth depending on user interpretation. Therefore, an explicit detection of such features is necessary for a reliable abstraction of the Bare Earth based on user needs. Most of the current filtering algorithms are only designed to detect vegetation and freestanding buildings. Therefore, the work presented here is focused on the detection of urban structures in an irregularly spaced point-cloud of an urban landscape, especially those
George Sithole (M.Sc.) is with the Delft University of Technology, The Netherlands. Thijsseweg 11 NL-2629 JA Delft, The Netherlands. E-mail:g.sithole@citg.tudelft.nl. George Vosselman (Prof.) is with the Delft University of Technology, The Netherlands. Thijsseweg 11 NL-2629 JA Delft, The Netherlands. E-mail: M.G.Vosselman@geo.tudelft.nl.

Before any structures can be detected they need to be defined, both conceptually and mathematically. The Landscape, the topography abstracted in an ALS point-cloud, is here conceptually defined as being constituted of three classes (Fig. 1): Bare Earth topsoil or any thin layering (asphalt, pavement, etc.,) covering it. Detached Objects - objects that rise vertically (on all sides) above the Bare Earth or other neighboring objects, e.g., free standing buildings, vegetation, etc., Attached Objects - objects that rise vertically above the bare earth only on some sides but not all, e.g., bridges, gangways, ramps, etc., The objective of the algorithm is to classify all points in the point-cloud into one of these three general classes. III. RELATED WORK The first problem of structure detection in an ALS pointcloud is to separate points that belong to the Bare Earth, and those that belong to Objects and outliers. This is commonly known as filtering. A number of filtering algorithms have been developed for this purpose, [1], [2], [6], [7], [8], [10], [12], [13], [14], and are generally successful. However, this separation is made non-trivial because portions of the terrain share similar geometric characteristics with the objects on the landscape. For example, a break-line in the terrain and the side of a building are geometrically similar. Therefore, in these

#010

2 profiles are sampled off the landscape, and sampled in different directions. Because the profiles overlap and because the positions of discontinuities in the profiles are known, it is possible to aggregated the information from the profiles and determine where there are obstacles in the terrain. Even more importantly it is possible to determine routes for traversing from one point to another while avoiding discontinuities. Furthermore, by aggregating information about the nature of discontinuities at an object it is also possible to make assumptions and classify features on the landscape.
Attached - Bridge

Detached

Attached - Ramps

Fig. 1. Conceptual Objects: Detached from and Attached to the BareEarth

V. SEGMENTATION As mentioned above the segmentation approach used in the algorithm is partly based on surface reconstruction from scanlines. A point-cloud is partitioned into two sets of families of orthogonal profiles running along the x and y direction. In each profile points are connected if they conform to some rule, thus yielding disconnected line segments, Fig. 3.

scenarios an algorithm based on clustering or segmentation have a higher likelihood of succeeding, because they take into account the neighborhood of the break-line. Clustering based on feature spaces, [3], [9], was considered but was deemed inconvenient because the extracted clusters would need to be further processed to obtain adjacency and connectivity information that is necessary in the classification step. Moreover, classifying the feature space (which has to be built into the detection process) makes the prediction of failure unreliable. ALS point-clouds tend to be large (millions of points) and therefore processing time can be critical. A segmentation approach based on TIN was also considered, [1], but was sidelined because of the difficulties presented when handling points that lie vertically above each other (e.g., power lines, vegetation, etc.,) and the fact that TIN based segmentation is generally slow.

Profile

Line Segment

es d

eh

Multiple Line Segments over same position

C A B a

Fig. 3. Profiles, Line segments, Connecting-the-dots (generating line segments)

Fig. 2. Concept. (left) profiles (A, B, C) through the landscape, and (right) Traversing from any point to another in a surface segment.

IV. CONCEPT Because of the problems mentioned above a segmentation algorithm based on a modified scan-line surface reconstruction algorithm was opted for. The requirements placed on the algorithm were (i) should be computationally efficient (ii) should be able to handle surfaces lying vertically above each other (iii) should be adaptable for solving different detection tasks, and (iv) should in general perform better than other segmentation algorithms [4]. A profile (Fig. 2 left) is taken off a point-cloud of a landscape. Adjacent points in the profile are joined with lines to simulate travel through the landscape along the profile. An obstruction (discontinuity) is deemed to exist at a significant height or slope change between any two points. Now many

A. Connecting the points Information about the connectivity and continuity of the surface segments is derived from the line segments generated during the profiling. Connecting-the-points (dots) is a classical problem but most of the methods were considered too computationally intensive for the purposes of the algorithm. Linear regression or a spline interpolation was ruled out because in an ALS point-cloud points are not always off a single surface (e.g., in vegetated areas, power-lines, etc.,). Two different approaches for connecting the dots have been tested. Connection by height difference - The height difference between any two consecutive points in a line segment should not exceed a chosen threshold. This type of connection uses the definition of continuity given by the maximum allowable height difference, eh, between two points, where the continuity is tested in a planimetric

#010 neighborhood, d, Fig. 3. The size of eh is tuned to the planimetric spacing between the points. For example eh may not exceed the average point spacing, otherwise the line will start to climb up walls or vegetation. The neighborhood d is chosen based on the largest distance between points that line segments are expected to bridge. Too small a d will result in undesirably small surface segments, and too large a d will result in the merging of Object surface segments. Therefore, the effectiveness of the technique depends very much on the point density of the point-cloud. Connection by slope The slope between any two consecutive points does not exceed a chosen threshold. This type of connection uses the definition of continuity given by the maximum allowable slope, es, between two points, where the continuity is tested in a planimetric neighborhood, d, Fig. 3. The drawback with this approach is that a relatively small value has to be used for the threshold, es, and because of this segmentation in steep slopes will be weak. As already mentioned a notable characteristic of the line segment generation is that line segments may overlap. This is a useful property for segmenting ALS point-clouds where points (from vegetation, power lines, phone lines, etc.,) often occupy the same planimetric position as the Bare Earth or other Object surfaces.

3 represent Object surfaces (as opposed to Object facets that are obtained in other segmentation algorithms) and that because line segments in profiles can overlap so too can surface segments overlap each other. VI. CLASSIFICATION The classification is performed in two steps, Fig. 5. In the first stage, surfaces are classified as Terrain, Detached Object or Unclassified. In the second step further separation within the above three classes is done so that eventually all points are classified as Bare Earth, Attached Object, Building, Vegetation or Unclassified. These classes are in effect the mathematical definitions for the conceptual classes described in section II. Terrain: That largest surface that is constituted from connected line segments. Because of this definition, necessarily a point-cloud has to be large. This class contains both the Bare Earth and Attached Objects. Detached Objects: Connected line segments (surfaces) that are not Terrain and are predominantly composed of raised line segments (this is explained in the next section, Fig. 6). Vegetation: All Detached Objects that consist of less than a certain preset number of points. Building: All Detached Objects that consist of more than a certain preset number of points. Bare-Earth: That part of the Terrain surface that does not contain Attached Objects. Attached Objects: That Terrain surface segment that is disconnected from some neighboring Terrain surface segments, but connected to at least one neighboring Terrain surface segment. A special case of this type is a bridge, which is disconnected from neighboring segments on diametrically opposite sides of its boundary. Unclassified: Cannot be placed into any of the above classes. This class is used for failure prediction. A. Classifiying the configuration of line segments Before Terrain, Detached Object, Unclassified classification can start, the configuration of every line segment (in relation to other neighboring line segments) in a profile needs to be classified. Configuration here is defined by the height difference of the end points of line segments with respect to their neighboring line segments. Fig. 6. shows the nine different configurations possible. Each configuration gives insight to the feature from which it was abstracted. For example a line that is raised above its neighborhood at both ends (left and right) is potentially off a Detached Object. The classification of the configuration of a line segment is stored in the label of its points (i.e., every point carries a label to its

Fig. 4 Segmentation - (left) Shaded relief, (right) Segmentation = Overlay of segmented profiles.

B. Connecting the line segments - Surface generation In this step line segments running in the x and y directions are overlaid. As already mentioned line segments are connected if they share a point in common. Therefore, looked at differently a profile is a 2 degree disconnected graph. By overlaying all x and y profiles a further disconnected graph is obtained in which each connected sub-graph is a surface segment (Fig. 4). The connected sub-graphs arise from the fact that line segments from orthogonal profiles will pass through the same point if they are off the same surface. It should be noted that surface segments may contain break-lines (a useful property). Therefore, the result of the segmentation is a disconnected graph in which each connected sub-graph is a surface segment. Therefore, it is possible to travel from any one point to another in a surface segment. A notable characteristic of the segmentation is that segments

#010
R R R

Terrain T

Be

Bare Earth

Oa

Attached Objects
L Terraced Left

L Low Left L

L High Left L

Detached Objects Points Object Surface Segments Od

Bld

Building

Veg

Vegetation

U Unclassified Step 1 Step 2

Terraced Right

R R Low Right R R L L

R High Right

Fig. 5. Nine possible configurations of line segments based on height differences at their ends.

Raised

Lowered

Open

parent line segment). This is important when classifying Attached Objects. B. Classifying the configuration of surface segments The classification of the configuration of line segments can now be extended to the surface segments by an analysis of the configuration of the component line segments of a surface. If most component line segments of a surface are raised (Fig. 6) line segments then that surface is classified as Detached Object. The surface segment with the largest number of points is always classified as Terrain. If a surface is predominantly made up of NOT raised line segments it is classified as Unclassified. C. Classifying vegetation All surface segments classified as Detached Object are reclassified as vegetation if the number of points in those segments is less than or equal to some predefined value nv. D. Classifying Attached objects In the preceding surface classifications surfaces were classified and the result of the classifications were then transferred to the points. When classifying Attached Objects points are classified directly. As already mentioned every point carries in its label the classification of the configuration of its parent line segments. If at a point, the configurations of both parent line segments is raised, then it must mean that at that point there has to be an Attached Object, if the surface segment has the classification Terrain. All points belonging to Attached Objects are classified in this fashion. All remaining points are classified as Bare Earth. VII. RESULTS The algorithm has been tested on a number of different data sets but presented here are results from processing of an airborne laser point-cloud of the city of Stuttgart (point spacing of about 1 meter), which is of a relatively high resolution. The point-cloud is made up of about 32000 points. The point-cloud is an abstraction of a landscape, composed of a wide variety of features (complex roofs, multiply elevated

Fig. 6. Nine possible configurations of line segments based on height differences at their ends.

platforms, courtyards, ramps, vegetation, cars, etc.,) found in a typical urban environment. Some preliminary results are shown in Fig 7. A. Segmentation In most parts the segmentation was successful. However, it failed in areas where there are an insufficient number of points for line creation (e.g., occlusions). Implications of different parameters on the segmentation are shown in Table I. B. Classification Terrain, Detached Object, Unclassified After the first classification the points classified as Detached Object were set aside and the point-cloud was resegmented without these points. This iteration was necessary because after the first iteration some Building points are classified as UnClassified, and in the iterative step the classification of these points is ascertained. This was necessary for surfaces off multi-level buildings and dense vegetation.
TABLE I SEGMENTATION Param. Profile spacing d Value used 1.5 * the point spacing 5 * the point spacing Comment Too small a profile spacing increases the algorithms run time (many unnecessary line segments). Determines the largest gaps that can be bridged in the point-cloud. Too small a value and too many disconnected lines are generated in vegetated areas. Too large a value and objects are incorrectly connected resulting in the merging of objects in the segmentation. The higher the point spacing the more reliable the generation of line segments. High point spacing, cause segmentation failure in steep terrain. The higher the point spacing the more reliable the generation of line segments. Occlusions cause segmentation failure in steep terrain.

eh

0.5m OR 0.5 * the point spacing. Which ever is smaller 0.3

es

#010 C. Classification Bridges, Depressions Shown in Fig 7. is the classification of a bridge. Most points on the bridge have been correctly classified. However, a few points on the Bare Earth have also been classified as Attached Object. These are corrected by placing a minimum on the number of points in a segment that can be classed as Attached Object. VIII. CONCLUSION Preliminary results from the segmentation and classification have been good. More importantly the algorithm is extensible and future work will further strengthen the algorithm. Some notable advantages of the algorithm are: Global and local context - A whole city block is processed at a time, in which both the global and local context of space is analyzed. Because of this the algorithm only offers meaningful results when a large city block is analyzed at a time.
Example 1 Example 2

5 Performance - Because most of the processing is done in two dimensions (the profiles), the algorithm is fast. The test data set (32000 points) was processed in less than 30s (using a Pentium III 1.5 GHz and 256 MB of memory). Single data structure - The data structure, represented by the profiles and line segments, is used for both the segmentation and all classifications. There is no fall back on other data structures, and no other support data is derived. This also serves to speed up the algorithm. In future work on classification, besides improving the generation of line segments, the following features will be added (i) Profiles in more than two directions (ii) Multiple laser pulses, and (iii) Data fusion use of airborne imagery REFERENCES
[1] [2] P. Axelsson, Processing of laser scanner data - algorithms and applications. ISPRS JPRS, 54 (1999). pp. 138 147, 1999. M. Elmqvist, Ground Estimation of Laser Radar Data using Active Shape Models. Paper presented at the OEEPE workshop on airborne laserscanning and interferometric SAR for detailed digital elevation models 1-3 March 2001, paper 5 (8 pages). Royal Institute of Technology Department of Geodesy and Photogrammetry 100 Stockholm, Sweden, 2001. S. Filin, Surface Clustering from Airborne Laser Scanning Data. Proceedings of the Photogrammetric Computer Vision ISPRS Commission III, Symposium 2002 September 9-13, 2002, Graz, Austria. pp.119-124, 2002. A. Hoover, G. Jean-Baptiste, X. Jiang, P.J. Flynn, H. Bunke, D.B. Goldgof, K. Bowyer, D.W. Eggert, A. Fitzgibbon, R.B. Fisher, An Experimental Comparison of Range Image Segmentation Algorithms. IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 7, pp. 673-689, July 1996. I. Lee, T. Schenk, 3D perceptual organization of laser altimetry data. IAPRS, Vol. 343/W4. pp. 57 65, 2001. K. Kraus, N. Pfeifer, Determination of terrain models in wooded areas with airborne laser scanner data. ISPRS JPRS. Vol. 53, pp. 193-203, 1998. B. Petzold, P. Reiss, W. Stossel, Laser scanning surveying and mapping agencies are using a new technique for the derivation of digital terrain models. ISPRS JPRS. Vol. 54 (1999). pp. 95 104, 1999. M. Roggero, Airborne Laser Scanning: Clustering in raw data. IAPRS, Vol XXXIV 3/W4 Annapolis, MD, 22-24 Oct, 2001. pp. 227232, 2001. M. Roggero, Object Segmentation with Region Growing and Principal Component Analysis. Proceedings of the Photogrammetric Computer Vision ISPRS Commission III, Symposium 2002 September 9-13, 2002, Graz, Austria. pp.289-294, 2002. G. Sithole, Filtering of laser altimetry data using a slope adaptive filter. IAPRS, Vol. XXXIV 3/W4 Annapolis, MD, 22-24 October 2001. pp. 203-210, 2001. G. Sithole, "Filtering Strategy: Working Towards Reliability". Proceedings of the Photogrammetric Computer Vision ISPRS Commission III, Symposium 2002 September 9-13, 2002, Graz, Austria. pp.330-335, 2002. G. Sohn, I.J. Dowman, "Terrain Surface Reconstruction by the Use of Tetrahedron Model with the MDL Criterion". Proceedings of the Photogrammetric Computer Vision ISPRS Commission III, Symposium 2002 September 9-13, 2002, Graz, Austria. pp.336-344, 2002. G. Vosselman, Slope based filtering of laser altimetry data. IAPRS, Vol. 33/B3, pages 935-942, 2000. R. Wack, A. Wimmer, "Digital Terrain Models from Airborne Laserscanner Data - a Grid Based Approach". Proceedings of the Photogrammetric Computer Vision ISPRS Commission III, Symposium 2002 September 9-13, 2002, Graz, Austria. pp.293-296, 2002.

[3]

[4]

[5] [6] [7] [8] [9]

[10] [11]

[12]

[13] [14] Fig. 7. (top) Shaded relief (middle) Classification into Terrain (light grey), Detached Objects (dark grey) and Unclassified (white), (bottom) Classification of Attached Object (bridges shown in dark grey)