Assessing lidar coverage and sample density

Resource Center Professional Library Extensions 3D Analyst Guide books Lidar solutions in ArcGIS

One basic QA/QC process when you receive lidar data is to ensure the lidar points delivered by your data provider have the coverage and density expected. Checking any coverage and density problems and resolving them early in your lidar processing is important. Two geoprocessing tools are useful in this regard: Point File Information, found in the 3D Analyst toolbox, and Point To Raster, located in the core Conversion toolbox.

Point File Information

The Point File Information geoprocessing tool reports basic statistics about one or more point data files on disk. The primary purpose of the Point File Information geoprocessing tool is to help you review and summarize the data before loading it into your geodatabase. LAS (the industry-standard format for lidar data) and ASCII format files are supported as input. Since lidar projects often utilize collections of data files, sometimes in the hundreds or even thousands, the tool lets you specify folder names in addition to individual files. When given a folder, it reads all files inside it that have the suffix you specify.

For each input point file,the tool outputs one polygon with accompanying attribution to a target feature class. The polygon graphically depicts the x,y extent, or bounding box, of the data in the file. Attributes include file name, point count, z-min, z-max, and point spacing.

The point spacing reported by Point File Information is not exact and deserves some discussion. For the sake of performance, it uses a rough estimate that simply compares the area of the file's bounding box with the point count. It's most accurate when the rectangular extent of the file being examined is filled with data. So files with significant numbers of points excluded over large water bodies or on the perimeter of a study area, only partially occupied with data, will not produce accurate estimates. Therefore, the reported point spacing is more meaningful as a summary when looking at trends for collections of files. Something useful to do with the output feature class is to display it in ArcMap, open its attribute table, and sort the point spacing field in ascending order. You can also symbolize on the point spacing field using a graduated color ramp. In the

image below, the LAS extent polygons are symbolized using a color ramp based on the point spacing estimate. The polygons symbolized in red indicate a tighter point spacing.

The Point File Information geoprocessing tool works quickly on LAS files because it only needs to scan their headers to obtain the information it's looking for. It takes significantly longer with ASCII files because, with them, the tool actually has to read all the data. You can also examine the classification conducted on the source points in the LAS files. Lidar data can be classified by the lidar provider into different class codes depending on the surface feature returned by the lidar system. These class codes can represent ground and nonground lidar returns. When you select the Summarize by class code option in the Point File Information dialog box, the output attribute table will contain summary statistics for each class code encountered in each LAS file. Use this option to examine the class codes in each LAS file and the corresponding point information associated to each class code. Assuming the Point File Information review process determines everything to be acceptable, the next step is to load your lidar points into a multipoint feature class with the LAS To Multipoint or the ASCII 3D To Feature Class tool. Put this feature class in a feature dataset if you intend to build a terrain dataset from the points. Though you have the choice between using LAS or ASCII format files, LAS is generally a better way to go. LAS files contain more information and, being binary, can be read by the importer more efficiently. Once the points are loaded into a multipoint feature class, you can use the Point To Raster geoprocessing tool to get a more in-depth view of the point distribution.

Point To Raster
The Point To Raster geoprocessing tool creates rasters from points and also supports multipoints. It's a generic tool with many options and uses. For evaluating lidar point density, the tool's COUNT option should be employed. This uses the number of points falling in a raster cell as the cell value. Being able to look at this graphically over the extent of the project area is revealing. There are a few parameters for the Point To Raster tool whose values for this exercise are not obvious. First is the Value Fieldparameter. It does not matter what this is set to, because the value field is ignored when Cell Assignment type is set to COUNT. Second is the Cellsize parameter. You might think the average point spacing is a good cell size for the output raster, but this typically results in too many empty, or NoData, cells because lidar points are not evenly spaced. Also the output raster could end up being unnecessarily large. Instead, it is better to go with a cellsize that is several times larger than the average point spacing but small enough to identify gaps or voids that warrant further investigation. A reasonable size is four times the point spacing. As an example, if your data is sampled at 1 meter and you set cellsize to 4, you can expect, on average, to get 16 points in a cell.

You can also evaluate the density for different types of points. While most of the time you'll probably just check the density for all returns, it can be useful to look at those that fall in a certain class, like ground. For example, this can give you an idea of how good your ground penetration is in vegetated areas. The Point To Raster tool doesn not make a distinction between point types. So you control what points are used by how you create the multipoint feature class with the LAS To Multipoint tool. It provides options for loading points by class code and return number. Once your raster has been created, look at it in ArcMap. Use a color ramp renderer to display it so it is easy to distinguish between cells with high counts and those with low. You can also set the NoData color to something that stands out. Look for variance in density and data voids. Have your vendor explain anything that doesn't look right.

In the image below, the black areas represent areas where no data exists, while the red areas represent areas where the lidar sample density is higher.

Copyright 1995-2012 Esri. All rights reserved.

Creating raster DEMs and DSMs from large lidar point collections
Resource Center Professional Library Extensions 3D Analyst Guide books Lidar solutions in ArcGIS

Summary
Raster, or gridded, elevation models are one of the most common GIS data types. They can be used in many ways for analysis and are easily shared. Lidar provides you with the opportunity to make high-quality elevation models of two distinct flavors: first return and ground. A first return surface includes tree canopy and buildings and is often referred to as a digital surface model (DSM). The ground, or bare earth, contains only the topography and is frequently called a digital elevation model (DEM).

These graphics show hillshaded representations of a first return surface, or DSM, on the left and a bare earth model, or DEM, on the right.

Coming up with a plan

Before beginning to create a raster from lidar, some basic factors need to be evaluated:

Extent of lidar coverage Number of lidar points and point density Desired output raster resolution Extent of output raster(s) Format of output raster(s)

Consideration of these factors will help determine whether you will be producing one raster or a collection of rasters. Part of this process requires that you figure out how many rows and columns you're willing to have in one raster. This depends on what you intend to do with the raster in terms of analysis, display, and potential sharing or distribution of the data. The desire to work off one dataset for analysis can be in conflict with practical constraints associated with dataset size. Another thing to consider is the amount of lidar you have. Trying to process 10 billion lidar points as one dataset, while possible, is

likely to prove unwieldy. In this situation you would want to make multiple rasters from this volume of lidar data, so consider splitting up the lidar processing as well. Not only does this keep individual datasets at reasonable sizes, it also keeps process duration on those datasets shorter. The longer a process takes to execute, the more likely something's going to go wrong (for example, power outage). If you have determined you need to split up your data, the next question is how can this be accomplished. Will it be based on a regular grid system, political boundaries, or an anticipated application? Since lidar collections tend to have multiple uses, splitting them up based on a regular grid system or political divisions like county boundaries makes the most sense. An engineer can mosaic the different pieces he or she needs for an individual project. If the intended use is weighted heavily for one type of application, such as hydrology, then use divisions logical for the application. For example, in the case of hydrology, watershed boundaries are a good candidate. ArcGIS supports many raster formats, so you have a choice of what format to write to. This decision is best based on the intended use of the product. If it's to be shared with the general public you might think about distributing in either TIFF or JPEG format. For analysis using the ArcGIS platform, consider using the file-based geodatabase format. The first step in getting from lidar points to raster is loading the points into a geodatabase. To load your lidar points into a multipoint feature class, use either the LAS To Multipoint or the ASCII 3D To Feature Class geoprocessing tool, depending on the source data format of the lidar data. Put the multipoint feature class in a feature dataset if you intend to build a terrain dataset from the lidar points. Although you have the choice between using LAS or ASCII format files, LAS is a more acceptable binary file format. LAS files contain more information and, being binary, can be read by the importer more efficiently. For more information on importing lidar source measurements into the geodatabase, see Data import and load tools.

Using the Point To Raster geoprocessing tool

If your only source of data is the lidar, you can use the Point To Raster geoprocessing tool to produce raster elevation models. ThePoint To Raster tool does not produce the highest quality result possible. Lidar tends to be so dense that for many applications the accuracy produced with the Point To Raster geoprocessing tool is good enough and the convenience and speed of this tool make it worthwhile. If producing a bare earth raster surface, or DEM, use just the ground lidar points to produce the raster. Set the Value field parameter on the tool to Shape to use the z-values from the multipoint vertices. Also, set Cell assignment type to either MIN or MEAN. MIN will bias output heights to local lows, while MEAN is more of a general purpose option. To

produce a first return surface, or DSM, use the first return lidar points with the MAX option of the tool since you want to bias the output to local highs.

The Point To Raster geoprocessing tool produces gridded elevation models from lidar point sets. While Point To Raster offers the easiest and fastest way to produce a raster from lidar, it has a significant drawback. You can end up with many NoData cells since it only returns values for cells that have one or more points in them. The problem is exacerbated when only using ground points to make a DEM because gaps in the data occur where there's vegetation and buildings obscuring the ground. To reduce the effect of NoData versus data cells, you can increase the output cellsize relative to the average point spacing. You can also reduce the number of NoData cells after execution of Point To Raster by using the sample below in the Python window if:

The Spatial Analyst extension is on The output name of the last focal stats expression executed is 'outfocalmNN' (outfocalm03, outfocalm04, outfocalm05 etc.) The output (in this example, the output from Point To Raster is called point2ras) is a layer in the table of contents The valid path is: C:\data\

from arcpy.sa import * outfocalm01 = FocalStatistics("point2ras", NbrRectangle(3, 3, "CELL") outfocalm02 = FocalStatistics(outfocalm01, NbrRectangle(3, 3, "CELL") #Repeat using output (temporary raster object) in the next processes until all nodata is gone out = Con(IsNull("point2ras"), outfocalmNN, "point2ras") #Save the result to disk out.save("C:\data\myfinalDEM")

You can run the Python sample multiple times to fill in larger NoData areas, but it is not recommended to run it more than a couple times. It is better to just accept larger void areas as a consequence of using this approach.

Original points on left, output from Point To Raster in the middle (NoData cells colored white) in the middle, post-processed raster with NoData filled in on the right.

Using the terrain dataset to create a raster DEM

If you have photogrammetric breaklines to go along with your lidar, or need higher-quality results than can be produced with the Point To Raster tool, use the terrain dataset. For an overview of the terrain dataset, see What is a terrain dataset?

On the left is a surface made without breaklines along the river banks. The right-hand side has breakline enforcement. Breaklines are important for maintaining the definition of water-related features in the elevation model. Breaklines are used to capture linear discontinuities in the surface. The most common types are edge of pavement, lake shorelines, single-line drains for small rivers, and double-line drains for large rivers. Sometimes breaklines are also collected to help define and sculpt the surface without necessarily representing discontinuities. Examples of these applications include contourlike form lines and the crests of rounded ridges. Breaklines, while frequently used in bare earth models, tend to be detrimental when used with first return surfaces because they can be in conflict with the aboveground points. For example, breaklines capturing road edge of pavement can be coincident in x,y but different in z to points in the tree canopy overhanging the road. Because of this, consider excluding breaklines from your first return surface or at least those where you know there's potential conflict. The most efficient means of organizing breaklines for use in a terrain dataset (see table below) is to separate them into different feature classes based on surface feature type (SFType). Surface feature types control how the features are enforced in the model and how the natural neighbor interpolator, used during rasterization, interprets the surface as it crosses over these features. A distinct break in slope will occur across "hard" features but not across "soft" features.
SFType for terrain datasets Measurement type Feature class type SFType

Lidar points Edge of pavement, single- and double-line drains for rivers, sharp ridgelines Lakes, reservoirs Eroded/Rounded ridgelines, contourlike form lines Study area boundary

3D multipoint feature class 3D line feature class 2D polygon feature class with z-value stored as attribute 3D line feature class 2D polygon feature class;no z-value

Mass Hardline Hardline or hardreplace Softline Softclip

It is best for the sake of terrain performance to place all hardlines together in one feature class. It is understood this might not be possible, for example, if you need to keep road and water features separate. Keep in mind, the fewer feature classes used to define a terrain, the better. Replace SFType is used to set everything inside a polygon at a constant height. It is used mostly for lakes when there's inadvertently other data inside them, such as lidar points, whose heights are not exactly the same as the shoreline and therefore prevent the water bodies from being flat. Replace SFType does incur more processing cost than normal hard- or softlines, so it is best to avoid using it in a terrain dataset. Ideally, there should not be lidar samples in water bodies (consider adding this as a stipulation in the contract with your data provider), but if there are, you can use the Delete Terrain Points geoprocessing tool to handle them after the terrain dataset is built. Otherwise, you can eliminate any offending points before building your terrain using the Erase Point geoprocessing tool. If you will be producing both bare earth and first return surfaces using terrain datasets, load the lidar points into two different multipoint feature classes, a feature class for the ground points and a feature class for the aboveground points. The bare earth terrain is defined with a reference to only the ground points. The first return terrain dataset references the same ground point feature class as the bare earth terrain with the additional reference to the aboveground points. This means two different terrain datasets can reference the same feature class. Starting with ArcGIS 9.3, terrain datasets can be pyramided using one of two point thinning filters: z-tolerance and window size. For DEM production, you can use either of these pyramid types. If you intend to rasterize from the full-resolution point set, use the window size filter for terrain construction because it is significantly faster. If you are willing to use thinned data for analysis, which is reasonable if the lidar data is oversampled for your needs, use the z-tolerance filter. While more time consuming, it is most appropriate because it provides an estimate of vertical accuracy of the thinned representation. For DSM production, use the window size filter with the MAX option.

Use the Terrain To Raster tool to produce your rasterized elevation model. This provides options for interpolation, output cellsize, and which pyramid level to use from the terrain dataset.

The Terrain To Raster tool produces gridded elevation models from terrain datasets. For interpolation, the natural neighbors options is the best option. While not as fast as linear interpolation, it generally produces better results both in terms of aesthetics and accuracy. Set the output cellsize relative to the lidar point sample density. You will not gain any accuracy by using a cellsize that is substantially smaller than the average point spacing. Also, make sure to set the analysis extent, as set through the Environments, for the extraction of subsets where appropriate. The use of a snap raster can also be of use for the sake of alignment of raster outputs. Overview steps to generate a raster DEM surface from lidar point data in ArcGIS are described below. First, you will create a terrain dataset in ArcCatalog or the Catalog window inside your appliation, then use geoprocessing tools to convert this terrain dataset to a raster DEM. 1. Create a terrain dataset. Steps:

1. Determine the source data and how that data will contribute to the terrain dataset. For more information on representing terrain source data, see Representing terrain source data in feature classes and Types of source data supported in terrain datasets. Note: All source data must have the same spatial reference to build a terrain dataset. 2. Create a file geodatabase in ArcCatalog or in the Catalog window. Right-click the folder where the terrain is to be built, point to New, then click File Geodatabase in the context menu. 3. Create a feature dataset. Right-click the file geodatabase, point to New, then click Feature Dataset in the context menu. For more information on correctly generating a feature dataset, see Creating a feature dataset. 4. Import the source measurements into feature classes. These feature classes must be produced inside the feature dataset created in step 3. For more information on how to import source data for a terrain, see Importing terrain dataset source measurements. 5. Build a terrain dataset in ArcCatalog or the Catalog window using the New Terrain wizard. To access the New Terrain wizard, right-click the feature dataset to display the menu, point to New, then click Terrain. For more information about using the New Terrain wizard, see Building a terrain dataset with the Terrain wizard. 2. Use the Terrain To Raster geoprocessing tool. Steps: 1. From the 3D Analyst Tools, double-click the Terrain To Raster geoprocessing tool to open it. 2. Click the Input Terrain browse button to add the terrain dataset. 3. Click the Output Raster browse button to specify the location where the raster dataset is to be created.

4. Set the optional Output data type parameter to either 32-bit floating point or 32-bit integer. Floating point is the default value. 5. Set the interpolation method to either Linear or Natural Neighbors. These methods are TIN-based interpolation methods applied through the triangulated terrain surface. The Linear option finds the triangle encompassing each cell center and applies a weighted average of the triangle's nodes to interpolate a value. TheNatural Neighbors option uses the Voronoi neighbors of cell centers. Consider the natural neighbors method for interpolating a terrain surface. Natural neighbor interpolation exhibits a longer processing time; however, the generated surface is much smoother than that produced with a linear interpolation. It is also less susceptible to small changes in the triangulation. 6. Set Sampling Distance to either Observations or Cellsize, which controls the horizontal resolution of the raster. Indicate the value next to the option once you have selected the desired method. The Observations method calculates the cell size for you based on the set value this number represents and the number of cells you want on the longest edge of the raster surface. You can set the cell size explicitly using the Cellsize option. 7. Set the resolution to use. The resolution parameter indicates which pyramid level of the terrain dataset to use for conversion. To output a raster dataset at full resolution, set this parameter to 0. Pyramid levels are defined using ztolerance or window size, which represents the approximate resolution of the terrain dataset relative to the full resolution data. 8. Consider using the environment settings to explicitly control the extents of the DEM you want to generate. To extract a subset of the terrain, click the Environments button on the bottom of the geoprocessing tool. Click the General Settings tab and define the extent of the output DEM. Using either Point To Raster or Terrain To Raster geoprocessing tool, you can process hundreds of millions, even billions, of lidar points into high-resolution gridded DEMs and DSMs. These surface models can then be used with the large collection of raster tools available in ArcGIS for analysis. They are also great for making maps (see graphic below) and, due to their simple data structure, are easy to share.

Copyright 1995-2012 Esri. All rights reserved.

Data area delineation from lidar points

Resource Center Professional Library Extensions 3D Analyst Guide books Lidar solutions in ArcGIS

It is common for lidar or photogrammetric data for a survey to be delivered without a detailed data area boundary. Often, the x and y extents of the survey area are defined by a tile system that covers an area of interest, and the data fills these tiles. The image below depicts lidar data tiles for a project. The extent of these tiles is a gross approximation of the actual study area boundary.

As is often the case, the actual lidar data does not fully cover the extent of tiles that are on the perimeter of the project area. The data is only guaranteed to cover some minimal extent, and there is no explicit or absolute boundary other than what can be inferred, as shown in the image below. This graphic is centered on one such tile.

Either way, the area of coverage is usually not a cleanly filled rectangle.

The problem
If a surface is made without declaring the data area up front (in other words, by including a clip polygon when defining a terrain dataset or TIN), some of what are actually voids around the perimeter are treated as data areas. Analytic results in these areas are unreliable because height estimates are based on samples that can be far away. The graphic on the left below depicts a dense collection of lidar points shown in green. The gaps in the interior are water bodies (where lidar is typically omitted). The irregularly shaped data boundary is easy to see, but unless an explicit extent is provided in the form of a clip polygon, TIN and terrain dataset-related tools will fill in voids, greatly oversimplifying the actual data extent.

You know areas outside the data collection extent should be excluded from the surface. The problem is coming up with the polygon that provides an accurate representation of this extent.

The solution
The solution is to synthesize a data boundary from the points that can be used to enforce a proper interpolation zone in the surface. Below, the left image depicts the lidar points. The middle image displays a polygon synthesized boundary from the points. The right image is a surface made from the lidar points and clip polygon.

The point spacing is the primary variable to use when going after the data area. Surveys usually have explicit minimums on point spacing to provide control for interpolators. Areas that do not meet density requirements are exceptions. They usually fall in one of the following categories: water bodies, obscured areas, and holidays (send the latter back to the data provider for repair). The vast majority of the data will meet sample density specifications. Point spacing is usually reported in metadata. If you do not know the point spacing of the lidar data, use the 3D Analyst Point File Information geoprocessing

tool. Alternatively, approximate the point spacing using the Measure tool in ArcMap. To learn more about point spacing, see Average point spacing. Data area delineation from lidar points Once you know the point spacing of the lidar data, follow these steps to delineate the data area: Steps: 1. Rasterize the lidar points using the Point To Raster geoprocessing tool. Rasterization of the lidar points helps aggregate the area covered by the lidar points. It provides a good data structure to work with for subsequent steps. You just need to tell the geoprocessing tool what cell assignment type to use and the output cell size. Use COUNT as the Cell assignment type value. Specify a value for cellsize that is several times larger than the average point spacing of the lidar data. Otherwise, you will get a lot of noise because the points are not evenly spaced. From the standpoint of processing efficiency and noise reduction, the larger cell size you use, the better, but there will be a trade-off with the tightness of fit in the result. A good place to start is four times the average point spacing.

2. Assign one value to all data cells using the Con geoprocessing tool.

Using the Con geoprocessing tool in this workflow is to simply turn any and all data cells of the raster into cells with one value. This value defines a raster zone that will be expanded in step three. All that is needed is to take the output from the Point To Raster tool and provide a constant value for a positive expression. All nonzero value cells will be considered positive and be assigned the constant value. Since COUNT was used as the cell assignment method during rasterization, any cell with a point in it must have a value greater than zero.

3. Fill small NoData areas using the Expand geoprocessing tool. Unless you used a very coarse cell size relative to your average point spacing, there is a likelihood of many NoData cells remaining. Most of these can be eliminated using the Expand geoprocessing tool. You want to remove most of these so the polygon produced during vectorization in a later step is not full of holes. That would be unnecessarily expensive.

The Expand tool pushes the zone of interest outward. In this case, the zone is all the data cells coded with a value of 1. This effectively eliminates small gaps found in the interior.

The left image shows many individual cells and some small clusters of NoData cells (in white). The right image shows the results from using the Expand geoprocessing tool where the NoData cells (in white) are removed, for the

most part. It is okay if some isolated NoData areas remain in the output. The remaining NoData cells will be handled in the last step. 4. Reduce the overall extent of data cells using the Shrink geoprocessing tool. While Expand eliminates isolated NoData cells, it also extends the data area outward. It needs to actually be brought in a little. Clip polygons need to be smaller than the actual point extent, so when terrain datasets or TINs try to estimate z-values along the polygon boundary, points can be found on both sides. This is needed to get good z estimates. To reduce the raster's data boundary, use the Shrink geoprocessing tool as shown below. Reduce the extent somewhat so the polygon produced in step 5 is smaller than the actual data extent of the points. This enables the software to estimate better z-values along the polygon boundary.

At this point, you have a relatively clean raster with the extent of its data cells slightly within the lidar point extent. 5. Vectorize the raster with the Raster To Polygon geoprocessing tool.

The Raster To Polygon geoprocessing tool converts the raster to a polygon feature class. Make sure the Simplify polygonsoption is checked. If it is not, the output will be stairstepped rather than smooth and contain more vertices than necessary. The Raster To Polygon tool outputs a polygon feature class. The result is representative of the data extent of the points used at the beginning of the process.

At this point, the process is almost complete. You need to review the output for correctness. Chances are there is one more step remaining, which is the removal of remaining holes inside your clip polygon. 6. Remove any remaining small holes using the Eliminate Polygon Part geoprocessing tool. The Eliminate Polygon Part geoprocessing tool eliminates any internal rings, leaving just the exterior boundaries. A clip polygon now exists that can be used when defining a terrain dataset or TIN. It should conform to the data extent of the lidar points but fall slightly inside them. The left image below shows the resulting clip polygon. The right image is a zoomed-in view showing the extent of the polygon relative to the source points. Note how it lies slightly inside the source point boundary.

Copyright 1995-2012 Esri. All rights reserved.

Estimating forest canopy density and height

Resource Center Professional Library Extensions 3D Analyst Guide books Lidar solutions in ArcGIS

Forest canopy density and height are used as variables in a number of environmental applications. These applications include the estimation of biomass, forest extent and condition, and biodiversity. Canopy density, or canopy cover, is the ratio of vegetation to ground as seen from the air. Canopy height measures how far above the ground the top of the canopy is. Lidar can be used to determine both of these variables. The following are steps to calculate canopy density and height from lidar points. First, you need lidar that has been classified into ground returns (bare earth) versus nonground returns. This type of point classification is usually performed by your data provider. Secondly, you need to consider when the lidar was collected and the type of vegetation in the study area. If there are a lot of deciduous trees and the collection was performed during autumn (leaf off),the density calculation is not going to work.

Loading points into the geodatabase

To calculate canopy density, load the ground, or bare earth, lidar points into one multipoint feature class and aboveground points into another multipoint feature class. Assuming your data is in LAS format, you can load the lidar points into the geodatabase using the LAS To Multipoint geoprocessing tool. Make sure to specify the proper class codes to filter the two multipoint feature classes with. The following table contains the LAS class codes as defined in the LAS 1.1 standard:

Class code 0 1 2 3 4 5 6 7 8 9

Classification type Created, never classified Unclassified Ground Low vegetation Medium vegetation High vegetation Building Low points (noise) Model key Water

Note: Any LAS file made in the last few years should use these codes if the points have been classified. Unfortunately, there's still some ambiguity. For example, class 2 is ground, but class 8 is ground as well. Class 8, or model key, points are a special set of ground points used for contouring or other applications requiring a thinned set of ground points. Whether you have them depends on how the data was processed. If you don't know, specify both classes. If it turns out there are not any model key points, it won't hurt. Vegetation has a similar issue. Sometimes vendors place everything that's above ground into class 1 because they haven't performed a more detailed classification on them. So if you are unsure of the specifics of your data's classification, load nonground points using classes 1, 3, 4, and 5. That's a reasonable catch-all to get your vegetation points. Note, if buildings or other man-made nonground features are in class 1, you will get them too, and they'll skew the results somewhat. Calculating the density

The most effective way to determine the canopy density is to divide the study area into many small equal-sized units through rasterization. In each raster cell, you compare the number of aboveground returns to the total number of lidar returns. The important technique to remember here is to determine an appropriate cellsize for this analysis. It needs to be at least four times the average point spacing. You can go larger but not smaller with the cell size. Steps: 1. Use the Point To Raster geoprocessing tool on the aboveground points with the COUNT option.

2. Convert any resulting NoData cells to 0 so that subsequent operations treat a cell with no points as 0. This is accomplished using the IsNull geoprocessing tool followed by the Con geoprocessing tool.

3. Repeat steps 1 and 2 with the lidar ground multipoints. 4. Add the aboveground and bare earth rasters together to get a total count per cell using the Plus geoprocessing tool.

5. All the rasters you've made so far are long data types. You need one raster to be floating point to get floating point output from the Divide geoprocessing tool that you will use in step 6. To generate a float raster, use the output raster from the Plusgeoprocessing tool as input to the Float geoprocessing tool.

6. Now use the Divide geoprocessing tool to compare the aboveground count raster and the floating point total count raster. This gives you the ratio from 0.0 to 1.0, where 0.0 represents no canopy and 1.0 very dense canopy.

The following image represents canopy density. The lightest areas have little to no vegetation. These are areas where a large percentage of lidar shots could "see" the ground. The dark green areas, where lidar could not penetrate to ground as well, indicate denser vegetation canopy.

Calculating the height

To determine canopy height, you will need to subtract the bare earth surface (DEM) from the first return surface (DSM). Follow the steps in the Creating raster DEMs and DSMs from large lidar point collections topic to generate these two surfaces. Steps: 1. Once you have generated the first return and bare earth rasters, use the Minus geoprocessing tool to determine the difference between these two raster datasets. The difference results represent, over forest, the canopy height.

The image below represents height above ground. It ranges from blue (little to no height) to orange, which is the tallest.

Lidar can be used to calculate the density and height of vegetation. This is useful for a variety of purposes, including biomass and carbon estimates, as well as forest management.
Copyright 1995-2012 Esri. All rights reserved.

Creating intensity images from lidar in ArcGIS

Resource Center Professional Library Extensions 3D Analyst Guide books Lidar solutions in ArcGIS

Intensity is a measure, collected for every point, of the return strength of the laser pulse that generated the point. It is based, in part, on the reflectivity of the object struck by the laser pulse. Other descriptions for intensity include return pulse amplitude and back scattered intensity of reflection. Keep in mind, reflectivity is a function of the wavelength used, which is most commonly in the near infrared. Intensity is used as an aid in feature detection and extraction, in lidar point classification, and as a substitute for aerial imagery when none is available. If your lidar data includes intensity values, you can make images from them that look something like black-and-white aerial photos. ArcGIS provides the ability to creating intensity imagery from lidar data.

1. Loading points with intensity

The first step in creating an intensity image in ArcGIS is to import your source points into the geodatabase as multipoints. The source lidar dataset must be in either LAS or ASCII XYZI file format. If your data is in LAS format, use the LAS To Multipoint geoprocessing tool with intensity selected as an attribute to import, as shown in the image below. If your data is in ASCII XYZI format, use the ASCII 3D To Feature Class geoprocessing tool. It is most common to use first return lidar data to create intensity images. If your lidar data is in ASCII format, format the ASCII files representing first return point data. If your lidar data is in LAS format, specify the class code on the LAS to Multipoint dialog box to represent first return lidar data. The tool will then filter the points and only import the first return points into the multipoints.

The data loading process results in a multipoint feature class with an attribute field containing the intensity values (see below). You cannot read these attributes directly because they are packed into binary large objects (BLOBs) in the multipoint feature class. Every vertex of a multipoint has intensity mapped into the BLOB and is stored for that multipoint record.

2. Generating an intensity image

Use the Point To Raster geoprocessing tool to generate an intensity image from the multipoint feature class. Populate Input Featureswith the name of the multipoint feature class and specify Intensity BLOB for Value field. Select MEAN from the Cell assignment typeoptions for a rasterization method. Set a cell size that is approximately four times larger than the average point spacing. Once the geoprocessing tool has been run, the resulting raster is the lidar intensity image. You might find one of the other Cell assignment type options useful for analysis, for example, RANGE of intensity as a variable used in feature detection.

Finally, review the presence of NoData areas. Significant numbers of NoData cells will result if the output cell size you specified is too small relative to the density of the lidar points. You can see NoData cells by assigning a color to them on the Symbology tab of the raster's Layer Properties dialog box. If there are several areas with no data available, the easiest thing to do is go back and rerun thePoint To Raster geoprocessing tool with a larger cell size. Alternatively, you can fill in missing values using an expression in the Spatial Analyst calculator. See Creating raster DEMs and DSMs from large lidar point collections.

Image display
The range of values in the image is hard to predict without knowing details about how your data was collected and processed. For one thing, the original intensity values are sensor dependent. Secondly, the values may have been adjusted by the vendor (for example, normalized to a range of 0255). Due to these variables, it is hard to determine what the best display options are for intensity imagery. Experiment with the intensity raster layer stretch type and contrast. Turning on bilinear resampling is probably a good idea. If you are looking for more display possibilities, consider combining intensity with another variable such as hillshade. The intensity image below is displayed with 50 percent transparency on top of a hillshade of the first return surface.

The return intensity collected for every lidar point can be used to make images. These images have a variety of uses in GIS applications including feature detection and extraction. ArcGIS provides tools to make these images.
Copyright 1995-2012 Esri. All rights reserved.

Updating a portion of a terrain dataset with new measurements

Resource Center Professional Library Extensions 3D Analyst Guide books Lidar solutions in ArcGIS

The ability to update a surface is important to provide accurate, current surface information, available for conducting analysis. Surface updates can come in different forms:

Adding ancillary data (for example, breaklines) Removing or replacing bad data Using newer or more accurate data Increasing extent with additional data Inserting design/modeled data to perform what-if analyses

These kinds of updates are best performed on the measurements used to construct a surface rather than on derivatives like raster DEMs. Those can be re-created as needed after the measurement edits have taken place. Terrain datasets support this editing model because they maintain a direct link to the source measurement data. When you modify the measurements, you are automatically modifying the terrain in the same process. For more basic information on the terrain dataset, see What is a terrain dataset?

How terrains are edited

Editing a terrain dataset is really about editing the source measurements. Using standard feature edit tools, you can manipulate the measurements that reside in feature classes that participate in a terrain. Terrain datasets are made from one or more feature classes, with simple rules for each feature class that control how they are used to shape the terrain surface. For example, a multipoint feature class containing lidar points can be added as mass points, a line feature class containing streams and lakeshores is used as a source of breaklines, and a polygon feature class can control the data area boundary. Most feature classes used to define a terrain are what we call referenced. This means that the terrain maintains a pointer, or handle, to them. The terrain prevents its referenced feature classes from being deleted and pays attention to any edits that occur on them including the addition, deletion, or modification of feature geometry. You can use the feature editor in ArcMap as well as geoprocessing tools to modify these feature classes. A terrain will automatically flag itself as dirty in areas where edits were made. Then the terrain can be rebuilt to bring its pyramid scheme in sync with the updated features. It does this based on the dirty areas. It is a way to conduct localized processing, and the entire terrain does not need to be reconstructed. Multipoint feature classes have the option of being embedded. When multipoints are embedded, the terrain build process copies the points into pyramid tables held private by the terrain and it becomes the container for the points. The terrain does not reference the source feature class. This source feature class can be deleted, allowing you to retrieve what is typically a substantial amount of disk space, approximately 1GB per 150 million points. Terrain-specific tools, Append Terrain Points (which can both add and replace) andReplace Terrain Points, are used to edit the embedded points based on an area of interest. These geoprocessing tools also offer the benefit of being BLOB attribute aware, which is necessary if you have any LAS attributes stored with those multipoints. For more information on BLOBs and lidar attributes, see Creating intensity images from lidar.

These terrain dataset processing tools keep the BLOB-based values in sync with the points relative to the edits. For example, if a few vertices of a multipoint are deleted from an embedded feature class, the terrain will delete the corresponding BLOB-based attribute values for those points.

Appending measurements
Measurements can be added to a terrain dataset through the Append and the Append Terrain Points geoprocessing tools. The Appendgeoprocessing tool operates on terrain referenced feature classes. The Append Terrain Points tool is used to add or replace points in embedded feature classes. You can also add a feature class to an existing terrain using the Add Feature Class To Terrain geoprocessing tool. Adding measurements using the Add Feature Class To Terrain geoprocessing tool creates a schema edit that invalidates the entire terrain dataset, requiring a full terrain rebuild. If data is to be added incrementally, it is best to append it to a feature class that already participates in the terrain dataset than add a new feature class to the terrain dataset for each new set of data. For example, sometimes data is provided in phasesthe bare earth lidar points are made available first and the breaklines come later in several deliveries. Knowing this schedule, you can create a terrain dataset referencing the lidar multipoint feature class containing the bare earth points, plus an empty line feature class held in anticipation of the breaklines. The image below displays a zoomed-in view of a terrain dataset made solely with bare earth lidar points.

When portions of the breaklines are made available, they are added to the terrain dataset by adding them to the line feature class referenced by the terrain dataset. This is done using the Append geoprocessing tool, as shown below.

After running the Append geoprocessing tool, the terrain dataset will become dirty in the areas where the lines were added. To determine the dirty areas, add a dirty area renderer from the Symbology tab of the terrain layer's Layer Properties dialog box. The following image displays the Add Renderer dialog box from the terrain dataset Layer Properties. For steps describing how to add the dirty area renderer for a terrain dataset, see Displaying dirty areas of a terrain.

The purple lines represent the dirty area tiles of a terrain dataset. The terrain layer draws the dirty area depicting a rectangular boundary that surrounds the edited area.

The terrain dataset now must be rebuilt. This is done using either the Build Terrain geoprocessing tool or the Build Terrain button on the Update tab of the terrain Properties dialog box in ArcCatalog or the Catalog window. Once the terrain dataset is rebuilt, the improvement made by the breaklines is evident. The definition of the water features on the terrain dataset has been improved with the addition of the breaklines as shown in the image below.

Replacing features
With lines and polygons in referenced feature classes, the replacement of measurements is a two-step process. First you delete the old features, then append the new features. If you are only dealing with a handful of features, consider selecting and deleting them using the Editor tools in ArcMap. For larger collections, rely on geoprocessing tools. For example, use Select By Location followed byDelete Features and Append. It is easiest to replace lidar points if they are embedded in the terrain dataset. The Replace Terrain Points geoprocessing tool has an area of interest or feature layer option to replace points. This will replace all the points within a given area of interest. So if you discover something was wrong with a few source point files that were used to build a terrain dataset, you can replace them without needing to rebuild the entire terrain from scratch. The images below show an example of one area of a bare earth model that was inadvertently loaded with first return lidar data. The images are hillshades derived from a terrain dataset. The one on the left clearly shows the mistake of either first or all return data being included for a subarea. On the right is the image of the correct data.

To correct this problem, load the replacement data into a new multipoint feature class, then run the Replace Terrain Pointsgeoprocessing tool to update the terrain dataset with the correct lidar point data. By default, the replacement area will come from the extent of the input feature layer, as displayed in the image below.

Once the points have been replaced, the terrain dataset needs to be rebuilt to update the affected area. Run the Build Terraingeoprocessing tool or use the Build Terrain button on the Update tab of the Terrain Properties dialog box in ArcCatalog or the Catalogwindow. Both options will update the terrain area affected by the new lidar points. There are times when updates to a surface model are needed, whether for quality improvement or what-if scenario analysis. It is hard to make these types of updates to derivative products like raster DEMs without ending up with some anomalies around the update area. It is more appropriate to modify the source measurement data from which the surface model is derived. For larger datasets, like those coming from lidar, it is also desirable that datasets be reprocessed only where the updates occur rather than rebuilding everything. Terrain datasets support this by maintaining links to their source measurements in the geodatabase and their use of dirty areas.
Copyright 1995-2012 Esri. All rights reserved.

Minimizing noise from lidar for contouring and slope analysis

Resource Center Professional Library Extensions 3D Analyst Guide books Lidar solutions in ArcGIS

Lidar is promoted as an accurate form of elevation data. To a large degree, this comes from its dense sample interval. Some refer to lidar as "painting the ground with elevation measurements." Indeed, it is common to acquire submeter sampling with lidar. This facilitates canopy penetration, which improves the accuracy of ground models in forested areas. The high sample density also improves results for certain applications such as floodplain delineation. However, lidar is not always the most optimal source for all surface modeling activities. Two areas that tend to be problematic are contour derivation and slope analysis.

Messy contours and steep slopes

Getting a computer to produce nice-looking contours from traditional data sources is hard enough. It is even more difficult to produce them from lidar. Contours produced from full-resolution lidar tend to be jagged with many twists and turns and isolated closed rings.

Slope assessment with lidar data is also problematic. If you examine the average slope from full-resolution lidar, you will see that it is unusually high. This is true even over relatively flat ground, as shown in the image below depicting the slope generated from full-resolution lidar. Green is little to no slope, yellow is moderate slope, and red is steepest. Note the high amounts of yellow and red sprinkled throughout the surface.

Some people misinterpret these issues with contours and slope as simply a result of lidar being more accurate than other forms of surface data. At large enough scales, almost any surface is fairly rough. While there is some truth to this statement, a significant part of the problem really stems from the relationship between horizontal sample density and vertical accuracy. As with any measurement technology, lidar is not perfectly accurate. Its vertical accuracy generally ranges from 12 to 15centimeters. That provides a random height difference between two adjacent points of 24 to 30centimeters. When the points are only 1 meter or less apart horizontally, that height differential becomes significant. From a signal processing perspective, this is called high frequency noise.

Techniques to reduce noise

To get less jagged contours and more reasonable slope estimates, you have to remove the noise from the lidar and lose as little real information as possible while doing so. Although there is no way to prevent all loss, harm can be minimized. The terrain dataset offers a couple of tools to facilitate this. One is intelligent point thinner; the other is a high-quality interpolator. Point thinning occurs during the terrain pyramid building process. Some people think pyramids are strictly a visualization tool, used solely to speed up drawing. While this is true for raster pyramids, the same does not apply to terrain datasets. Terrains were designed knowing that lidar is better suited for some applications when it has been generalized.

There are two terrain dataset point thinning algorithms available in ArcGIS: z-tolerance pyramiding and window size pyramiding. Both algorithms have something specific to offer and are arguably better than random point filters used by other solutions that, while fast and reasonable for visualization, are not what you would want to use for analysis. The z-tolerance filter employs a TIN-based algorithm to find a subset of points sufficient to create a surface that is within a given vertical distance to the full-resolution surface. The use of this filter is recommended when a quantifiable measure of vertical accuracy is needed. The window size filter selects points within a given horizontal sample distance. Every so many units in x,y (the sampling window size), the points within that area are examined and one or two are selected depending on which option you chose. The point selection method can be the point closest to the mean of the other points in the sample window, the highest or lowest of the points in the sample window, or both the highest and lowest. As stated earlier, much of the noise problem results from a bad ratio of high horizontal sample density to vertical accuracy. The window size filter enables you to improve that ratio. While it is hard to quantify the accuracy of the thinned data, empirical evidence has shown this approach works well.

Thinning the data

Lidar points are thinned when a terrain dataset is built. The filter algorithm applied is based on the type of pyramid selected for the terrain dataset. Fortunately, the same name is used for pyramid type as filter algorithm so there is no ambiguity. The pyramid type is selected through the Terrain wizard found on the feature dataset context menu in ArcCatalog or the Catalog window.

Next, you would need to specify the pyramid levels. For the sake of noise reduction, you are interested in that first pyramid level, which is one step removed from full resolution. A z-tolerance equal to the vertical accuracy of the data is reasonable for that level. This will eliminate as much noise as possible while decreasing the accuracy of the result as little as possible. If you start out with 15 centimeters vertical accuracy, you end up with approximately 30 centimeters accuracy; if you go on to produce contours, set their interval to at least double this, namely 60 centimeters or approximately 2 feet. When building a window size pyramid, a sample distance equal to twice the nominal point spacing is reasonable.

After having built a terrain dataset, the thinned points reside in its pyramid. Use the Surface Contour and Surface Slope geoprocessing tools to generate contours and slope. The results will be less noisy than if made from the unthinned points. One drawback is that contours and slope estimates made from these TIN-based surfaces will still be more angular and discontinuous than necessary. A recommended alternative is to go through a raster where the results will be smoother.

Raster interpolation
Contour and slope outputs can be improved by deriving them from a raster that is made from a terrain dataset. Engineers have a traditional bias toward working directly on TINs, so they may not like the idea of going through a raster, but when dealing specifically with contour and slope production from lidar data, this bias is unwarranted. For one thing, lidar points are essentially collected in a random distribution, and the resulting triangles are not handpicked or guaranteed to fit some mathematical model made inside a CAD package (for example, for road design). Additionally, the linear piecewise surface defined by planar triangle faces is not smooth. A smoother surface can be made using theTerrain To Raster geoprocessing tool with the natural neighbors interpolation option. An added benefit of using Terrain To Raster is that you can rasterize an entire terrain dataset at once and avoid the size constraint associated with TIN extraction.

Once you have generated a raster surface model of the terrain dataset, you can then use the Contour and Slope raster geoprocessing tools on the derived raster. The following images illustrate the difference between full resolution and generalized contour and slope derivatives.

Contours made from full-resolution lidar on the left and a rasterized version of the thinned set on the right. The result, while obviously not cartographic quality, still represents a significant improvement.

Slope made from full-resolution lidar is on the left and slightly thinned and rasterized is on the right. Lidar is considered noisy due to a low vertical accuracy relative to horizontal sample distance. This noise translates into poorquality contours and excessively high average slope rates. Noise can be reduced through point thinning and smoothing without losing much of the accuracy and detail. The terrain dataset provides these tools via pyramiding and natural neighbor interpolation.
Copyright 1995-2012 Esri. All rights reserved.