Geospatially Coping with a Shifting Landbase

By Quince Lunde, Helix Water District

elix Water District provides quality services for 260,000 people in a 50-square-mile area of sunny Southern California. The Helix service area includes several suburban municipalities east of the city of San Diego.

USERS PERSPECTIVE
These days, updated and more accurate land data are becoming more available with increasing frequency. When the landbase in a GIS is replaced, in many areas, the existing utility network and asset data no longer fit in the same relative relationship to the new landbase as they did to the old landbase. The network and assets need to be adjusted to line up accurately with respect to the new landbase. A simplified view of this concept is seen in the following diagram. The terms shift vectors, conflation, and constraint snooping will be explained later in this paper.

time, leading to some key business drivers for updating the original landbase. The introduction of orthophotography as a backdrop to the GIS map data immediately highlighted areas where the original landbase, water network, and assets did not line up with the physical features on the photography. More spatially accurate assessors information was added to the GIS and it also did not line up with the existing landbase. A third-party regional standard landbase became available, and the ability to use that would be less expensive to purchase than the cost of maintaining adjusted utility network the nonstandard Helix landbase in-house. Effective data sharing with municipalities and other agencies was new landbase dependent on a standard landbase to ensure ease of data transfer and alignment of third-party facilities with respect to Helixs data. GPS data points were being captured for certain assets in the field and integrated into the GIS, 3

Lunde

Helix is an industry leader in the treatment and distribution of potable water. Helix uses a stateof-the-art water treatment plant that can treat and store up to 109 MGD (million gallons per day), of which 95% is imported. Helix distributes water through over 700 miles of pipe. Helix is considered substantially built out, meaning there are limited areas for growth in new water services. Helix utilizes the GE Energy Smallworld portfolio of geospatial software products. These products enable users to access and analyze GIS data from the desktop as well as in the field. Helix uses the robust data storage capabilities of Smallworld to support multiple edit staff in separate facilities.

Utility Network Adjustments

utility network

Constraints from CONSTRAINT SNOOPING

shift vectors from CONFLATION

old landbase

Business Drivers Helix implemented its GIS in 1999 with the primary objective being management of its water system assets. Many new uses and users of the GIS have developed over

The Problem in Simple Terms www.gita.org

NETWORKS

further highlighting misalignments and inaccuracies. The combined effect of these drivers was beginning to erode user confidence in the data. Something needed to be done. Landbase and Orthophoto Mismatches Manual Solution

Helix became interested in ways to automate the process of adjusting their network to fit the new landbase. They were looking for an automated adjustment tool that would work from within their existing GIS environment, without the need for ETL middleware. They wanted it to have built-in project management features that would facilitate the

need to be user-friendly, with all GIS staff being able to use it after a short training period. Last but not least, it would have to prove cost effective with efforts being reduced and shifted from numerous hours of manual adjustments with questionable accuracy to automated adjustments with consistently high accuracy. GIS operators would shift their focus to managing the automated processes and QA/QC of the results. Attendance at a GITA annual conference led to the discovery of a suitable software tool called adjust.IT. It was developed by a company named we-do-IT based in Melbourne, Australia and is distributed in North America by Ubisense Consulting. It worked within Helixs GE Smallworld GIS environment and appeared to meet the requirements, so a pilot project was undertaken to prove the claims and benefits.

New landbase was acquired, and manual methods were attempted to realign the network and assets using rubber sheeting methods and heads-up digitizing. This was found to be very timeconsuming, expensive in terms of resource requirements, and not consistently accurate. There was no easy way to measure or ensure accuracy. Also, when another new landbase of higher accuracy becomes available in the future, there would be no automation or repeatability in the process, which would require similar manual adjustments all over again. Requirements for an Automated Adjustment Tool

work processes involved. They also wanted the adjustments to be constrained by the existing shape of the network so that it maintained straight lines, right angles, and parallel lines with offsets as well as symbol and annotation placement in the resulting adjusted data. Ideally there would be little or no need for further manual data cleanup once the adjustment was made. The tool would also

The product was subsequently purchased, and a project is under way to systematically adjust all of Helixs service territory Automated Adjustment Process

www.gita.org

NETWORKS

The overall process is depicted in the previous diagram.The process consists of five steps within two defined stages. Automatic landbase Conflation Stage This stage compares the new landbase to the old landbase and generates shift vectors that show the angle and magnitude of shift for points and lines located in each of the old and new landbase datasets. This is known as the Conflation Stage. Conflation is defined as the automatic detection of identical points within different datasets. In some jurisdictions, these shift vectors are provided by third-party landbase service providers along with their updates. However, this is not usually available in North America, so a Conflation Stage would be required. Step 1: Preparation of the Automatic Base Map Comparison This would be initiated by selection of common old and new base map objects along with definitions of the areas and objects to be compared. Step 2: Determination of Shift Vectors The shift vectors are found through comparison of the old and new base map and are stored as persistent objects for easier management, visualization, and navigation. Thousands of shift vectors are generated automatically from an initial manual selection of a few seed object locations www.gita.org

common to both the old and the new landbases. Statistics for the generated shift vectors indicate the match rate for the overall shift, so that the operator can decide if the match is sufficient for the subsequent asset and network adjustments. Specific areas with a high level of mismatch can be run separately with a higher number of initial seed object locations, resulting in an improved match rate. Not all shift vectors will be uniform in magnitude throughout a service area. The varying degrees of shift are displayable as a thematic map depicting the degree of shift geographically. With a bit more work, they can also be displayed as a TIN with the degree of shift depicted as the Z coordinate, providing a 3-D thematic view of the shift over an area. These can help to determine project areas for adjustment and the amount of additional effort and quality control that might be required for certain areas. Analyzing Shift Variance

sources, they can be read in directly as data files in various formats and stored as shift vector objects. Automatic Adjustment of the Assets and Network This stage uses the previously generated landbase shift vectors and a predefined set of network constraints to adjust the network and assets using a rigorous least squares mathematical adjustment in a high-performance processing environment. Step 3: Project Management This component utilizes the version management capabilities of the GIS. Each area for adjustment is set up as a project, and all project-specific parameters and constraints are defined. Parameters and constraints can be associated with a parent (project) object making creation, analysis, and deletion as well as merging of projects possible. Data can be broken up into adjustment regions based on statistical node counting, an established map sheet system, or any other gridbased system. Step 4: Preparation of Adjustment Using the previously detected (or supplied) shift vectors, a systematic GIS trace will snoop through the network to find all network constraints (right angles, straight lines, distances, parallel lines, GPS points, etc.) based on project5

If shift vectors can be supplied by external (usually government)

NETWORKS

specific parameters. There are a set of default constraints, but the user is also able to change, add, or delete constraints manually. In addition, it is possible to assign individual weights to each constraint to overwrite the projectwide parameters. The system also supports additional constraints such as fixed points, arbitrary angles, bearings, distances, areas, or any user-defined constraints that are a function of relevant points in either the landbase or the network. Step 5: Execution of Adjustment As a last step, the network, its assets, dimensions, and free text are all automatically adjusted and shifted to their correct position relative to the new landbase. The actual least squares adjustment task is passed to a highly efficient processing environment. The adjustment produces a full set of statistics for analysis and allows the operator to accept or reject the results. Following visual inspection and statistical analysis, the operator decides on the quality of the adjustment outcome. It is possible to go back and change individual constraints or project parameters to repeat the process and/or store the final result. Additional projects (Steps 3 to 5) could be undertaken, or several projects could be combined (e.g., several subdivisions or suburbs forming a larger city area). Since the version management capabilities of the GIS are utilized, the project can be easily rolled back to any step in the process and run again with changes that may help to provide a better quality end result. www.gita.org

The only requirement in addition to this process is to establish procedures for quality control of the outcomes. The tool provides exceptional reporting and statistics for assisting with this effort, during both the conflation and adjustment stages. Original Data Quality and Expectations The automatic adjustment of geometric arcs and splines was problematic due to the difficulty of deriving the proper shift for the control points. A decision was made to stroke the arcs and splines into smaller straight segments. Geometries with segments shorter than 2 feet had to be cleaned up prior to adjustment. In most cases these were data quality issues. Staff Training GIS staff was sufficiently trained on-site by Ubisense consultants using Helixs environment. The pilot project fine tuned staff competencies and work flow processes. Life After Adjustment Using Safe softwares FME product, Helix will be able to isolate changes (deltas) between subsequent landbase deliveries from the third party. From these isolated deltas, shift vectors can be created and directly imported into the adjust.IT software for additional automated adjustments. Conclusion For Helix, the main benefits of automated adjustment are:

Minimal training required for GIS operators to use the tool and operate the adjustment process Labor cost savings Minimal impact to end users as the data does not have to be inaccessible for long periods of time

If there is anything we would do differently, it would be to spend more time scrubbing our data. The adjust.IT software may help you find data problems, but it wont fix them for you.

Quince Lunde is a GIS programmer/analyst at Helix Water District and can be reached at quince.lunde@helixwater.org.

NETWORKS