TR 515 PDF

Q
C
C - C
op V1
y
op V1
C -
y
C
C
Q
Document No: TR-530

First Edition
December 2016
Department of Municipal Affairs and Transport
PO Box 20
Abu Dhabi, United Arab Emirates
© Copyright 2016, by the Department of Municipal Affairs and Transport. All Rights Reserved. This
document, or parts thereof, may not be reproduced in any form without written permission of the
publisher
ROAD PERFORMANCE MONITORING SYSTEMS
TABLE OF CONTENTS
List of Figures .............................................................................................................................. iv
List of Tables ................................................................................................................................. v
Abbreviations and Acronyms ..................................................................................................... vi
1 INTRODUCTION ..................................................................................................................... 1
1.1 Overview........................................................................................................................... 1
1.2 Purpose and Scope .......................................................................................................... 1
1.2.1 Purpose ..................................................................................................................... 1
1.2.2 Scope ........................................................................................................................ 2
op V1
1.3 Application of this Guide ................................................................................................... 3
1.3.1 Goals ......................................................................................................................... 3
1.3.2 Stakeholders.............................................................................................................. 3
C -
y
1.4 Content and Format .......................................................................................................... 4
1.4.1
C
Strategic and Requirements Section .......................................................................... 4
C
1.4.2 Tactical and Process Section ..................................................................................... 5
1.4.3 Operational and Implementation Section ................................................................... 6
Q
2 USAGE DATA REQUIREMENTS ............................................................................................ 7

2.1 Overview........................................................................................................................... 7
2.1.1 Data vs. Measures ..................................................................................................... 7
2.1.2 Traffic vs. Events ....................................................................................................... 7
2.2 Traffic Data Typology ........................................................................................................ 8
2.2.1 Measure .................................................................................................................... 8
2.2.2 System ...................................................................................................................... 9
2.2.3 Granularity ............................................................................................................... 10
2.2.4 Scope ...................................................................................................................... 10
2.2.5 Sampling.................................................................................................................. 11
2.2.6 Accuracy .................................................................................................................. 11
2.2.7 Latency .................................................................................................................... 11
2.3 Event Data Typology....................................................................................................... 11
2.3.1 Control ..................................................................................................................... 12
2.3.2 Incidents .................................................................................................................. 12
2.3.3 Work Zones ............................................................................................................. 12
2.3.4 Weather ................................................................................................................... 12
2.3.5 Special Events ......................................................................................................... 12
2.4 Traffic Data Technologies ............................................................................................... 12
2.4.1 Infrastructure Sensors.............................................................................................. 13
2.4.2 Vehicle-Based Sensors............................................................................................ 15
Page i
TOC First Edition -December 2016
2.5 Usage Monitoring Strategy ............................................................................................. 18

2.5.1 Strategy Methodology .............................................................................................. 18
2.5.2 Key Data Items ........................................................................................................ 18
2.5.3 Summary ................................................................................................................. 19
3 CONDITION DATA REQUIREMENTS .................................................................................. 21
3.1 Overview......................................................................................................................... 21
3.1.1 Pavement Management Context .............................................................................. 21
3.1.2 Traffic vs. Pavement Monitoring ............................................................................... 22
3.2 Condition Reporting Requirements ................................................................................. 22
op V1
3.2.1 Traffic Monitoring Data............................................................................................. 22
3.2.2 Pavement Monitoring Data ...................................................................................... 28
3.2.3 Other Data Required to Perform Condition Reporting .............................................. 32
3.3 Pavement Management Implementation ......................................................................... 33
C -
y
4
C
LOCATION DATA REQUIREMENTS .................................................................................... 34
4.1 Overview......................................................................................................................... 34
C
4.2 Spatial Reference ........................................................................................................... 34
4.2.1 Geographic Referencing .......................................................................................... 34
Q
4.2.2 Transport Network Assumptions .............................................................................. 35

4.3 Linear Reference ............................................................................................................ 36
4.4 Topology ......................................................................................................................... 37
4.5 Attributes ........................................................................................................................ 38
5 DATA COLLECTION ............................................................................................................. 40
5.1 Methods .......................................................................................................................... 40
5.1.1 Traffic ...................................................................................................................... 40
5.1.2 Event Usage ............................................................................................................ 44
5.1.3 Condition ................................................................................................................. 46
5.1.4 Location ................................................................................................................... 48
5.2 Procurement ................................................................................................................... 49
5.2.1 Usage Data Procurement ........................................................................................ 49
6 DATAPROCESSING ............................................................................................................. 52
6.1 Approach ........................................................................................................................ 52
6.2 Technologies .................................................................................................................. 52
6.2.1 Using Databases ..................................................................................................... 52
6.2.2 Data Warehousing ................................................................................................... 52
6.2.3 Hosting .................................................................................................................... 53
6.3 Storage ........................................................................................................................... 53
6.3.1 Location Data .......................................................................................................... 54
Page ii
6.3.2 Condition Data ......................................................................................................... 55

6.3.3 Traffic Usage Data ................................................................................................... 55
6.4 Processing ...................................................................................................................... 57
6.4.1 Computation ............................................................................................................ 57
6.4.2 Quality Control ......................................................................................................... 58
6.4.3 Imputation ................................................................................................................ 60
6.4.4 Aggregation ............................................................................................................. 61
6.4.5 Fusion Engine .......................................................................................................... 62
7 DATA DISTRIBUTION .......................................................................................................... 64
op V1
7.1 Approach ........................................................................................................................ 64
7.1.1 End Users ................................................................................................................ 64
7.1.2 DATA USES ............................................................................................................ 65
7.2 Technologies .................................................................................................................. 67
C -
y
7.2.1
C
Web-based Information Systems ............................................................................. 68
7.2.2 Web Services .......................................................................................................... 71
C
7.2.3 Mobile Services ....................................................................................................... 72
7.2.4 Personal Navigation Devices ................................................................................... 73
Q
7.2.5 Broadcast Radio/TV ................................................................................................. 75

7.2.6 Variable Message Signs .......................................................................................... 75
7.2.7 Alerts ....................................................................................................................... 76
7.2.8 Kiosk........................................................................................................................ 76
7.3 Process........................................................................................................................... 77
7.3.1 Distribution System Performance ............................................................................. 77
7.3.2 Access Control......................................................................................................... 77
7.3.3 Data Liability Issues and Copyrights ........................................................................ 78
7.3.4 Presentation of Data ................................................................................................ 78
7.3.5 Data Presentation Tools .......................................................................................... 79
8 APPLICATIONS .................................................................................................................... 82
8.1 Monitoring and Measurement ......................................................................................... 82
8.1.1 Fundamental Relationship ....................................................................................... 82
8.1.2 Coupling .................................................................................................................. 82
8.1.3 Methods ................................................................................................................... 82
8.2 Types of Applications ...................................................................................................... 85
8.2.1 Measures ................................................................................................................. 85
8.2.2 Evaluations .............................................................................................................. 87
8.2.3 Analytics .................................................................................................................. 87
8.2.4 Modelling ................................................................................................................. 87
Page iii
8.2.5 Decision Support ..................................................................................................... 88

8.3 Example Application ....................................................................................................... 88
8.3.1 Goal and Strategy .................................................................................................... 88
8.3.2 Key Performance Indicators ..................................................................................... 88
8.3.3 Leveraging Monitoring ............................................................................................. 88
8.3.4 Refining Monitoring .................................................................................................. 88
8.3.5 Conclusion ............................................................................................................... 90
9 ACTIVITIES ........................................................................................................................... 91
9.1 Approach ........................................................................................................................ 91
op V1
9.2 Initial ............................................................................................................................... 91
9.2.1 Collection ................................................................................................................. 91
9.2.2 Processing ............................................................................................................... 95
9.2.3 Distribution............................................................................................................. 107
C -
y
9.3
C
Operational ................................................................................................................... 112
9.3.1 Collection ............................................................................................................... 112
C
9.3.2 Processing ............................................................................................................. 113
9.4 Periodic......................................................................................................................... 117
Q
9.4.1 Collection ............................................................................................................... 117

9.4.2 Processing ............................................................................................................. 118
9.4.3 Distribution............................................................................................................. 120
9.5 Triggered ...................................................................................................................... 122
9.5.1 Collection ............................................................................................................... 122
9.5.2 Processing ............................................................................................................. 123
9.5.3 Distribution............................................................................................................. 123
Cited References....................................................................................................................... 124
Other References ...................................................................................................................... 124
LIST OF FIGURES
Figure 1: Data Scope ...................................................................................................................... 2
Figure 2: RPMS Systems Diagram ................................................................................................. 5
Figure 3: Ramp and Mainline Detection Model .............................................................................. 10
Figure 4: Traffic Data Technology Tree ......................................................................................... 13
Figure 5: Example Distribution of Commercial Vehicles on Two Roads ........................................ 26
Figure 6: Polyline Example ........................................................................................................... 35
Figure 7: Picture of a Simple Linear Referenced Route ................................................................ 36
Figure 8:A Topology...................................................................................................................... 38
Figure 9: PeMS, a Web-Based Information System Displaying Traffic Speed, Variable Message
Signs, Incidents, and Lane Closures in Real-Time ........................................................................ 69
Page iv
Figure 10:Regional Integrated Transport Information System (RITIS) 3D Incident Visualization ... 70
Figure 11:PeMS, Displaying Traffic Congestion and Incidents (Blue Circles) Across Space and
Time.............................................................................................................................................. 71
Figure 12:Real-Time Traffic Conditions on a Mobile Device .......................................................... 73
Figure 13: Garmin nüvi® 3490LMT Personal Navigation Device ................................................... 74
Figure 14:Variable Message Sign Displaying Real-Time Delay Information .................................. 76
Figure 15:Dashboard Depicting Peak Period Travel Time Reliability and Sensor Health over the
Entire System ............................................................................................................................... 80
Figure 16: RITIS Visualization of Speeds on a Freeway Segment During an Incident, with Distance
on the Y-Axis and Time on the X-Axis. A Queue Can Be Seen ..................................................... 81
Figure 17: Detailed Anatomy of an Incident................................................................................... 89
Figure 18: Dashboard of an Automated Monitoring System, Google Analytics ............................ 117
op V1
LIST OF TABLES
Table 1: Traffic Data Items ............................................................................................................ 19
C -
y
Table 2: Event Data Items............................................................................................................. 20
C
Table 3: FHWA Vehicle Classification ........................................................................................... 25
Table 4: Example Attributes .......................................................................................................... 39
C
Table 5: Location Data Tables ...................................................................................................... 97
Table 6: Condition Data Tables ..................................................................................................... 99
Table 7: Traffic Usage Data Tables ............................................................................................. 101
Q
Table 8: Event Usage Data Tables ............................................................................................. 103

Table 9: Metadata Tables ........................................................................................................... 104
Table 10: Service Performance ................................................................................................... 107
Table 11: Timestamp Codes ....................................................................................................... 110
Table 12: Sensor Alarms............................................................................................................. 114
Page v
ABBREVIATIONS AND ACRONYMS

DMAT Department of Municipal Affairs and Transport
DMA Department of Municipal Affairs
DSRC Dedicated Short Range Communication
ADM Abu Dhabi City Municipality
ADTMC Abu Dhabi Transport Management Centre
ADUPC or UPC Abu Dhabi Urban Planning Council
ATC Automatic Traffic Count
AADT Average Annual Daily Traffic
AASHTO American Association of State Highway and Transportation Officials
ATIS Advanced Traffic Information System
AVI Automated Vehicle Identification
AVL Automated Vehicle Location
op V1
ATMS Advanced Transportation Management System
AUH Abu Dhabi International Airport
ADUS Archived Data User Service
API Application Programming Interface
CAD
C -
Computer aided dispatch
y
CMN Congestion Management Network
CMP
C
Congestion Management Plan
CMPP Congestion Management Policies and Procedures
EIA Environmental Impact Assessment
C
ESAL Equivalent Single Axle Loadings
ETC Electronic Toll Collection
Q
HOV High Occupancy Vehicle

KPI Key Performance Indicators
ISP Information Service Providers
ITS Intelligent Transport Systems
IRI International Roughness Index
GIS Geographic information systems
GPS Global Position System
LiDAR Light Detection and Ranging (LiDAR)
LPR License Plate Readers
MOE Measures of Effectiveness
OCR Optical Character Recognition
PMS Pavement Management Systems
PCI Pavement Condition Index
PDO Property Damage Only (related to crashes)
PHD Person Hours of Delay
PHT Person Hours Travelled
PKMT or PKT Person Kilometres Traveled
PPP Public-Private Partnership
PT Public Transport
RFID Radio-Frequency Identification
ROW Right-of-way (land used for roadway)
RPMS Roadway Performance Monitoring System
SOS State of the System
SOV Single Occupancy Vehicle
SQL Structured Query Language
STEAM DMAT’s Strategic Transport Evaluation and Assessment Model
STMP Surface Transport Master Plan
TDM Travel Demand Management
TSM Transportation System Management
UAE United Arab Emirates
UTMC Urban Traffic Management and Control
VHD Vehicle Hours of Delay
Page vi
VHT Vehicle Hours Travelled

V/C Ratio Volume to Capacity Ratio
VKMT or VKT Vehicle Kilometres Travelled
WIM Weigh-in-Motion
CM/Cum Cubic Meter
EAM Environmental Assessment Manual
FM-RDS FM Radio Data System
FWD Falling Weight Deflectometre
GUI Graphical user interface
HPMS Highway Performance Monitoring System
HTML Hyper Text Mark-up Language
LiDAR Light Detection and Ranging
LPFM Low-power broadcasting
op V1
PND Portable navigation devices
ReST Representational State Transfer
RPMS Roadway Performance Monitoring System
SOAP Simple Object Access Protocol
TMC
C -
Traffic Management Centre
y
WIM Weigh In Motion
WSDL
C
Web Services Description Language
DTMF Dual Tone Multiple Frequencies
C
PDA Personal Digital Assistant
Q
Page vii
1 INTRODUCTION
1.1 Overview
In 2010, the Abu Dhabi Department of Transport (DOT) commenced with the “Unifying and
Standardizing of Road Engineering Practices” Project. The objective of the project was to enhance
the management, planning, design, construction, maintenance and operation of all roads and
related infrastructures in the Emirate and ensure a safe and uniform operational and structural
capacity throughout the road network.
op V1
To achieve this objective a set of standards, specifications, guidelines and manuals were
developed in consultation with all relevant authorities in the Abu Dhabi Emirate including the
Department of Municipal Affairs (DMA) and Urban Planning Council (UPC). In future, all authorities
or agencies involved in roads and road infrastructures in the Emirate shall exercise their functions
and responsibilities in accordance with these documents. The purpose, scope and applicability of
C -
y
each document are clearly indicated in each document.
C
It is recognized that there are already published documents with similar objectives and contents
C
prepared by other authorities. Such related publications are mentioned in each new document and
are being superseded by the publication of the new document, except in cases where previously
published documents are recognized and referenced in the new document.
Q
1.2 Purpose and Scope

1.2.1 Purpose
Most national departments of transport have collected systematic data on their roadway systems
since the middle of the twentieth century. These processes are typically referred to as
performance monitoring systems. This document examines these existing processes and
leverages best practices from several other national departments of transport, in order to create a
comprehensive Roads Performance Monitoring System (RPMS) for the Abu Dhabi Emirate.
Although this document will perform a similar function in the Abu Dhabi Emirateas it does in other
nations, there are some differences. In some countries, this guide is the result of a legislative
action and, as such, defines mandatory collection processes, encoded into law. InAbu Dhabi, this
document is an internal agency guide that defines best practice within the agency, and specifies a
clear implementation framework for performance monitoring. It does not mandate specific
technologies or collection formats. It does not impose collection requirements on other
stakeholder agencies (such as the Municipality of Abu Dhabi), nor does it define any agency
consequences for lack of monitoring data.
Because it is developing its performance monitoring regime in 2011, Abu Dhabi has the
opportunity to leverage the experience of legacy agency practice, and incorporate the advances in
monitoring technologies that have taken place since most agency RPMS processes were crafted.
This guide takes advantage of these opportunities by keeping only the essential portions of
existing process, and incorporating advanced monitoring technologies and techniques into its
recommendations. For references, see Appendix A.
Page 1
01-INTRODUCTION First Edition -December 2016
1.2.2 Scope
This section details the scope of the monitoring efforts detailed in this guide.
1.2.2.1Data Scope
The scope of data collected for monitoring purposes often includes three major categories of
inventories:
 Locations
 Conditions
 Usage.
op V1
C -
y
C
C
Q
Figure 1: Data Scope
Locations typically take the form of detailed geographic representations of all roadway centre-lines,
with associated attribute information (i.e. number of lanes), encoded in a standardized,
documented geographic information system, including linear referencing. Condition reporting
typically associates another set of attribute information (i.e. pavement condition) to this locational
framework. Usage information associates attribute information about the travel that occurs on the
roadway (i.e. number of vehicles). All three of these categories of information are essential to a
functional RPMS. Location data gives an agency the consistent framework required to reference
its attribute information. Condition information gives an agency the ability to optimize system
maintenance. Usage information allows an agency to effectively operate the existing system and
plan the future system. In most agencies, data collection for these three categories of inventories
is prioritized in the order above: first locations, then conditions, and then usage. This guide follows
this standard practice.
1.2.2.2Process Scope
The scope of processes that are required to effectively monitor these categories of data items are
also threefold:
Page 2
 Collection
 Storage
 Access.
Data collection is the core activity in monitoring. The processes outlined in this guide encompass
a variety of technologies, approaches, and measures. Typical monitoring processes can include
systems as low-technology as sampled manual counts and as complex as continuous, real-time
sensor or probe data. This guide will describe when a given process is appropriate and how to
apply it effectively, consistently, and efficiently.
Data management has become an increasingly important component of monitoring, as the scope
and scale of data collection activities have increased. This additional data volume has created
data management challenges for data monitoring systems. This guide will describe best practice
op V1
for data management.
Data access encompasses all the avenues through which an agency transmits its data to
stakeholders. Over the last few decades of agency practice, access methods have evolved from
C -
manual, and request-based, to automated, pushed data access. This guide will give guidance on
y
the best approach to data access.
C
1.3 Application of this Guide
C
1.3.1 Goals
Q
The purpose of this document is to detail a framework for understanding the performance of the
roadway system in the Abu Dhabi Emirate. This framework will include a clear accounting of the
types of data items that will be recorded for the roadway network, including the type, frequency,
and scope of data collection activities.
This document is structured as a staff implementation guide. As such, it details specific processes
that agency staff can follow in order to implement an effective RPMS. Upon reading this
document, agency staff can expect to have the knowledge required to implement the Abu Dhabi
EmirateRPMS.
1.3.2 Stakeholders
Performance measurement, performance monitoring, and performance management are broad
and active topics at all levels of infrastructure and institutional management. Often, within
transport agencies, performance measurement practices fall into different silos of institutional
management. For example, one a core set of agency performance metrics relate to project
delivery and the monitoring of contractor performance. Another performance practice within the
agency may involve optimizing the delivery of projects throughout an organization. This guide
focuses on the traditional scope of performance monitoring processes within agencies, namely: the
system for understanding the state of the transport network over time. While this key performance
monitoring process informs most other performance measures within an agency, it does not
encompass them.
Agency stakeholders for this RPMS are broad, because the data collection processes defined
within are core to agency operation. Typically, the scope of stakeholders mirror the three main
categories of data required for effective monitoring: stakeholders that use location data; those that
use condition data; and those that require the facility usage data. The first set of stakeholders,
those that use location information, is the broadest group within agency practice, as location data
Page 3
is ubiquitous. The second group, stakeholders that require condition data, are typically asset
managers at various levels of the agency. The final group, those that require facility usage data,
are typically facility operators or transport planners. This guide will specify processes that assist all
three of these stakeholder groups.
Because the data items that it covers are core to agency practice, this guide contains fundamental
connections to several other manuals within the Abu Dhabi DMAT. The primary interactions
between this guide and other agency manuals are threefold, corresponding to the three primary
types of monitoring data. Usage measures are closely related to the Congestion Management
Procedures Manual (TR-523) (1). Location data necessarily interacts with Land Surveying and
Mapping Guide for Road Projects (TR-539)(2). Condition measures are further detailed within the
Pavement Design Manual (TR-527)(3). Where possible, the interactions between this guide and
these other manuals are detailed explicitly in the sections that follow.
op V1
1.4 Content and Format
This guide follows a three-part structure. The first part takes a strategic view toward monitoring
C -
and lays out high-level requirements for monitoring data. The second part takes a more tactical
y
C
approach to understanding the processes required to monitor performance. The final section takes
an operational approach and details the implementation of these processes through specific
C
activities and applications.
1.4.1 Strategic and Requirements Section

Q
At the highest strategic level, the DMAT staff must understand which data items can be measured
and which data items should be measured. The first three chapters of this document lay out the
three main categories of data to be collected: usage, condition, and location.
1.4.1.1Usage Data Requirements

This chapter creates data requirements topology, with a special focus on usage monitoring data.
The three core data elements include location data, condition data, and usage data. Usage data is
further classified with six categories:
 Measure. The fundamental quantities that define system performance.

 System. The mode and network of the measure.
 Granularity. The level of spatial and temporal detail of the measure.
 Scope.The spatial and temporal reach of the measure.
 Accuracy.The degree to which the measure reflects reality.
 Latency. The speed at which the measure is delivered, starting with real-time.
This chapter will focus on the collection of core traffic usage data items. While it will survey the
higher-level performance measures that this data can calculate, other manuals and processes will
specify the congestion performance measures to be used within the agency, such as the
Congestion Management Procedures Manual (TR-523)(1).
1.4.1.2 Condition Data Requirements

This chapter delineates condition reporting requirements, processes, and methods. Condition
reporting encompasses a wide array of asset management data.
Page 4
This guide will focus on traditional monitoring data system items, such as pavement condition. It
will touch on other asset management data, but will reserve most of the detail on these items to
other agency guidelines, such as the Pavement Design Manual (TR-527).
1.4.1.3 Location Data Requirements

This chapter delineates location reporting requirements, processes, and methods. Location data
reporting encompasses a wide array of geographic data. This guide will focus on the location data
required to enable the monitoring data system to function correctly. It will touch on other geo-
referenced data, but will reserve most of the detail on these items to other agency guidelines, such
as the Land Surveying and Mapping Guide (TR-539)
1.4.2 Tactical and Process Section
op V1
The preceding chapters focused on data requirements; the next three focus on the tactical,
process level. These previous chapters were focused on strategic issues, such as: What data
should I collect? This section is focused largely on tactical issues. What technologies should I
use to collect data? What systems do I need to develop for effective data management? How do I
C -
give my customers/stakeholders access to this data? These three questions define the
y
C
fundamental processes required to execute effective monitoring: collection, management, and
distribution.
C
First, the Department must collect each data item. All collection activities are performed by a
combination of people and technologies. Second, the department must manage the data that it
Q
receives from its collection apparatus. Data management involves several district sub-processes
in order to effectively use the data for performance monitoring. Finally, the DMAT must effectively
distribute this data to its customers. A rough system diagram of these three processes is shown in
Figure 2 below.
Figure 2: RPMS Systems Diagram
Page 5
1.4.2.1Data Collection
Updating location, condition, and usage data are key processes within a performance monitoring
system. This chapter details the methods and systems that will be used to update the monitoring
data for all of these categories. In keeping with the emphasis of this guide, it details the
technologies that agencies can use to automate data updates. One key issue in this process is
how to effectively sample data; a special sub-section outlines methods for effective sampling.
1.4.2.2Data Processing
Data processing is a key topic within the monitoring process. The evolution of practice has
mirrored the evolution of data management within the information technology community in
general: data management has become more comprehensive, more structured, and more robust.
op V1
This section will review and recommend agency best practices for data processing.
1.4.2.3Data Distribution
As information technology tools have evolved, opportunities for seamless data access have greatly
C -
improved. Because of this, this section details standards for access, with a focus on leveraging
y
C
distributed technologies and cloud-based approaches.
1.4.3 Operational and Implementation Section

C
The final two chapters focus on how to take the requirements and processes of the preceding two
sections and make them operational. As such, it is divided into two chapters that detail the ways to
Q
implement monitoring.
1.4.3.1Application
This chapter details the methods for taking monitoring data and applying it to agency practice.
Because monitoring data is used throughout an organization, there is a delicate balance between
this continual monitoring process and more ad-hoc measurement exercises. This section details
the types of uses for data.
1.4.3.2Activities
This final chapter discusses the implementation details required for robust monitoring practice.
What types of facilities should be monitored? What values should be collected? How frequently?
Where? This is where the processes and requirements are translated into concrete
implementation.
Page 6
2 USAGE DATA REQUIREMENTS
2.1 Overview
This section presents a framework for understanding core monitoring data, surveys traffic data
collection technologies, and lays out a strategic vision for monitoring data collection within the
DMAT. In order to set the stage for this four part framework, the following sub-sections detail the
connection between data and measures and divide core data into two distinct categories (traffic
and events).
op V1
2.1.1 Data vs. Measures
In order to effectively implement a monitoring program, monitoring staff must understand the
relationship between higher-level measures (such as vehicle kilometres travelled or average daily
traffic) and lower-level core data (such as point-measured speeds). Higher-level measures are all
C -
derived from lower-level core data. These measures are typically derived from department
y
C
business practices; because these practices evolve over time, so do the higher level measures
used to track them. A laundry list of high-level congestion measures is detailed in Chapter 5 of the
C
Congestion Management Policy and Procedures(1).
It is unrealistic to develop a future proof set of data items: as measures change, so must the data
Q
items used to derive those measures. However, a clear understanding of the core data items and
associated collection technologies can help to facilitate a strategic approach to monitoring system
development. The goal of a monitoring system, with respect to usage data, is to efficiently collect
the subset of core data items required to generate the higher level measures required by agency
stakeholders. This section focuses exclusively on providing a clear understanding of the core data
items that can be collected, rather than the high-level measures that can be derived from those
data items.
Hence, this section defines usage data requirements at a strategic level, outlining the core
framework for understanding the two fundamental types of core data (traffic and event), surveying
the available collection technologies, and defining high level collection requirements for the DMAT.
Downstream sections will deal with the tactical approaches to efficiently acquiring this data
(Chapter 5: Data Collection) and the operational approaches to effectively implementing collection
within the department (Chapter 9: Activities).
2.1.2 Traffic vs. Events

For conceptual clarity, this guide divides core data items into two major categories: traffic data and
event data. These two categories represent two different types of activities: traffic data is typically
measuring continuous phenomena, while event data typically measures discrete phenomena.
These two types of data are parallel along another dimension: traffic data tends to be endogenous
to the analysis of a transport system, while event data tends to be exogenous. In other words, the
traffic data tends to measure the results of traffic impacts, while event data tends to be the cause
of those impacts. In general, traffic data is a primary monitoring focus, while event data is typically
a secondary or tertiary focus of monitoring systems. However, this event data is critical to
understanding the performance of a system and – since some of it is directly under operational
control of transport agencies – is often a valid focus of measurement efforts at advanced agencies.
Page 7
02-USAGE DATA REQUIREMENTS First Edition -December 2016
For these conceptual reasons, this guide splits these two types of core data items into separate
categories. In general, the attributes of traffic data defined below (scope, etc.) also apply to event
data.
2.2 Traffic Data Typology

This typology is a framework for understanding all fundamental traffic data measures. Core traffic
data requirements can be fully characterized using seven categories:
 Measure: The fundamental quantities that define system performance

 System: The transportation network of the measure
 Granularity: The level of spatial and temporal detail of the measure
 Scope: The spatial and temporal reach of the measure
op V1
 Sampling: The amount of a given activity that the measure captures
 Accuracy: The degree to which the measure reflects reality
 Latency: The speed at which the measure is delivered, starting with real-time.
2.2.1 Measure
C -
y
C
At their core, all transport systems consist of travellers traversing networks. Because of this, the
fundamental traffic quantities that can be extracted from transport systems, using existing
C
technologies, can be divided into three main categories:
 Points. Point measures occur at a point on a given link

Q
o Speed. The instantaneous velocity of vehicles at the point

o Count. The number of vehicles that traverse the point
o Presence. The amount of time a vehicle is directly above the point
o Attribute. Some classified aspect of the vehicles traversing the point
 Link Sets. Link measures cover a set of links, from an origin to a destination
o Travel Time. The time it takes to traverse from the origin to the destination
o Flow. Total number of vehicles traveling from the origin to the destination
o Density. A rarely calculated measure of vehicle spacing on a specific link
 Vehicles. Vehicle measures relate to a specific vehicle over space and time
o Attribute. Some classified aspect of the vehicle.
Point data are typically collected by fixed infrastructure sensors. The only fundamental data items
collected at point sensors are: speed, count, presence, and vehicle attributes. Speed, count, and
presence are substitutes for each other, when two of the three categories are known. For
example, speed can be estimated from vehicle counts and occupancies. (Point) Attributes are a
broad category that can encompass any aspect of the vehicle. These now typically include:
vehicle lane, vehicle length, vehicle weight, axle configuration, and person occupancy. Point
measures can be taken for transit vehicles, but are typically not relevant, as it is more economical
to instrument a limited set of vehicles than the infrastructure itself.
Link set data are typically collected by origin-destination matching schemes. For most link data,
the exact path that vehicles take between an origin and destination is unknown. Thus, the use of
the term ‘link set’ rather than ‘link’: this guide uses a more abstract definition of a link for this
classification, which may or may not specify a full path. Density is a measurement that is rarely
used; historically, it has been directly measured through aerial photography. Density can, of
course, be estimated using many methods, but this is the only way to directly measure it, using
practical systems. This guide includes it in this typology for completeness only. Most other density
metrics are estimates from occupancy detectors.
Page 8
Vehicle data is measured directly on the vehicle, and tracks the vehicle through the network. As
with all the core measures, this kind of data must have a time and location stamp. The attributes
that can be measured about the vehicle beyond these are essentially unlimited, as vehicle
information technologies continue to progress: speed, heading, acceleration, events, etc.
Currently, speed and heading are the most commonly measured attributes.
These data items fully define the quantities that can be directly measured from the system using
existing and planned technologies. All other advanced traffic information system (ATIS) related
measures are derived from these core measures. For example, delay can be estimated in several
ways. Least accurately, it can be estimated from travel times between two points (link measures),
using travel times and flows. More accurately, it could be calculated from a dense point sensor
network (point measures), using speeds and counts. Most accurately, it could be calculated from
connected vehicles (vehicle measures), with a large, known sample size and typical detailed global
op V1
position system (GPS) information (location, time, speed). In all cases, delay is derived by one or
more categories of core measures; in no case can it be directly measured.
This is a critical concept that bears repeating: all other traffic measures are derived from the
C -
core measures listed above.
y
C
Take another example: queue lengths. Junction queue lengths cannot be directly calculated. For a
derived measure, basic parameters must be established. For example, is a vehicle in a queue if it
C
is traveling at 10 KPH? What about 15 KPH? What if the vehicle in lane one is traveling 10 KPH,
but the vehicle in lane two is traveling 15 KPH? Are queues specific to lanes, or do they apply to
the entire roadway? Once these boundaries are determined, a queue can be defined as a
Q
concept. Then it must be measured. There is no device or technique that directly measures a
queue. Queue measures are typically constructed from either vehicle probe runs, detailed
inspection of a series of closely spaced traffic detectors, or video analytics. Thus, correctly
defining a user queue length requirement requires breaking this concept down into its core
measures.
2.2.2 System
Because all transport systems consist of vehicles traversing networks, the core metadata for any
system measure rests on describing the modal network and vehicles of interest. In this case,
metadata is data that describes the structure of the underlying measure. Modal network here
refers to the mode choice of the traveller.
For example, the traveller could be driving a vehicle, so they would be traveling using an auto
mode. They could be traveling via bus, so they would be traveling using a transit mode. This
typology considers all major surface transport modes, excluding non-motorized modes. This guide
excludes non-motorized modes from our analysis because they are outside the scope of this
monitoring process.
Systems can become complex and interconnected, defying easy categorization. Ramp data, at the
intersection of the freeway and arterial systems are a good example of this. Typical ramp data
collection infrastructure illustrates this complexity, in theFigure 3, below:
Page 9
op V1
C -
y
C
Figure 3: Ramp and Mainline Detection Model
C
2.2.3 Granularity
All measures must have corresponding spatial and temporal metadata. Most measurement
Q
equipment records measures from individual vehicle events. Frequently, for practical reasons,
these measures are aggregated as they move up the chain from the field and into the hands of
analysts or travellers. These aggregations are typically made in deference to limited (or pricey)
communication bandwidth, storage limitations, or processing constraints. Requirements must
specify the temporal granularity for any given set of measurements.
In addition, spatial granularity is required to fully define a traffic data requirement. For example,
spatial granularity is often defined as the density of the measurements along the linear network
links. Measures that are taken at half-kilometre spacing are twice as granular as those taken at
kilometre spacing. Spatial granularity is generally driven by some combination of sensor density
and aggregation.
2.2.4 Scope
All measures must have a spatial and temporal scope, in addition to granularity. The spatial scope
is simply the area of the network covered by the measurement systems. Once granularity is
specified – half-kilometre spaced freeway point sensors, for example – the scope is the number of
directional freeway centre-line kilometres covered at this level of granularity. Because data
collection is a costly process, measurements are generally targeted at areas of interest; often
these are congested segments.
Temporal scope is simply the start and end time for collection. Typically, measurement equipment
falls into two major categories: continuous collection equipment and temporary collection
equipment. For most intelligent transport system sensors, temporal scope is continuous. For
special project equipment, the scope must be defined.
Page 10
2.2.5 Sampling
Sampling applies to collection methods that, even for the geographies, times, and vehicles they
cover, selectively capture information. Sampling is a statistical practice and statistical methods
and concepts – such as confidence interval and bias – are used to define sampling scope. For the
purposes of this categorization, this sampling scope can also define reliability requirements for
continuous, real-time systems. In this sense, even continuous measurements are (in practical
applications) actually sampled, as sensors break. Because of this, most sampling scopes are
either narrow or broad. For selective measurements, sampling scope can be very small: below 1%
of the total population of possible measurements, for example. For continuous measurements, this
scope is typically very large: over 99% of the total population of possible measurements, for
example.
op V1
2.2.6 Accuracy
Measure accuracy refers to the level of reliable detail in a given sensor output. As with sampling,
accuracy is a statistically constructed concept. No traffic measurement system can measure a
quantity with 100% accuracy. Because of this, accuracy is typically modelled as a distribution of
C -
y
error terms. Typically, and in this guide, a measurement system is deemed within a specified
C
accuracy if the measured item is within the proscribed range 95% of the time. Creating a baseline
measurement of conditions for validation purposes is a major challenge in most transport systems.
C
For most traffic measurements, GPS probe runs typically serve as such for most sensor
evaluations. Of course, even GPS has measurement inaccuracy. As a consequence of this
complexity, actual accuracies are frequently estimated, or simply unknown.
Q
2.2.7 Latency
Latency refers to the speed at which the data is delivered from the field to the end user. It is a
continuous value, but can usually be put into two categories: real-time and archived. Real-time is a
misnomer: latency between the field and the end-user is always present. For transport data
applications, sub-second or even sub-minute field latency is typically considered real-time. Data is
typically considered archived if it is delivered with daily or greater latency. Latency is an important
vector for requirements analysis because real-time systems are typically more expensive than
archived systems.
2.3 Event Data Typology

In addition to core traffic measures – i.e., measures of the vehicles on various networks – there are
a variety of ancillary measures that impact system performance. These can generally be thought
of as events, which impact some set of points or links on the transport system for a given period of
time. These events can impact system performance, so this guide includes them in its analysis of
data requirements.
These high-level event categories are:
 Control. Signals and other devices that control vehicles movements

 Incidents. Unplanned network events that impact the capacity of the system
 Work Zones. Planned construction events that impact the capacity of the system
 Weather. Exogenous environmental events that impact the capacity of the system
 Special Events. Exogenous social events that impact the demand on the system.
Page 11
All of these event measures can be categorized and defined using the same traits as core traffic
measures: an associated system, granularity, scope, sampling, and latency.
2.3.1 Control
Control devices include signals at intersections and ramps, and other forms of control on a
network, targeted at vehicles. Control events tend to have a major impact on arterial and transit
system performance and a lesser impact on expressway performance. As such, they are more
important data requirements for non-freeway systems. An example control requirement:
continuous, real-time signal event data for the arterial system on all major connectors during the
AM peak period.
2.3.2 Incidents
op V1
Incidents most commonly involve vehicle crashes of various types, but can involve a wide variety
of unplanned activities on transport networks, mostly perpetrated by travellers. The vast majority
of incidents reduce capacity on a given link for some period of time. This category covers
unplanned construction closures. An example incident data requirement: an archive of all recorded
C -
incidents, on a specific road, for a month.
y
C
2.3.3 Work Zones
C
Work zones are a relatively specific term for a more general concept: any planned activities on the
transport system that impact (typically, reduce) capacity for a specified period of time. In general,
their impact is similar to incidents, but operators have more advance warning and precise detail of
Q
their exact impact. An example work zone data requirement: hourly real-time lane closure
information for a district, by lane for a day.
2.3.4 Weather
Weather activities are exogenous events from the perspective of a transport network: they occur
outside of the network’s internal function and activities. However, they can have major impacts on
(typically large) sets of network links: capacity reductions, and an increased likelihood of incidents.
They are also a realistic data requirement: several agencies provide free access to high quality
weather data. An example weather requirement: visibility for Abu Dhabi Emirate during February
2011, recorded to the meter, in hourly increments.
2.3.5 Special Events

Special events are also exogenous events, but they impact demand rather than capacity. They
can be understood as either planned or unplanned. Unplanned events typically fall under the
purview of emergency and evacuation management. Collecting detailed data on planned events is
not always realistic because event venues do not offer automated data feeds to agencies. More
commonly, planned events can be managed using existing tool sets, such as traffic models. Data
can be collected on these, but usually only manually. An example special event data requirement:
estimated attendance, date, and times for all football games at the Sheikh Zayed Stadium for 2011.
2.4 Traffic Data Technologies

This section focuses on traffic data collection technologies: the core sensors required for any
monitoring system. At a high level, these divide between sensors placed on the infrastructure, and
sensors placed on vehicles. At a secondary level, the vehicle sensors can be divided into sensors
that measure start and end points of a trip, and sensors that detail a full travel path. These three
Page 12
major sensor categories roughly parallel the three core types of traffic measures (defined
previously in Section 2.2.1). The technologies available to implement these measurements,
however, are quite varied. This technological variance is the focus of this section.
op V1
C -
y
C
C
Q
Figure 4: Traffic Data Technology Tree
2.4.1 Infrastructure Sensors

Infrastructure-based sensors are already a common component of traffic management systems in
many regions, and should be used a starting point for data collection in travel time reliability
monitoring programs. As shown in the diagram above, infrastructure sensors are point sensors,
which means that they only provide data values across a screen line, or one fixed location along
the roadway. Since they provide no information on an individual’s route or time travelled between
two points, the data they transmit requires some processing and interpolation before travel times
can be calculated. This also means that the travel time estimates that they produce will likely be
less accurate than those calculated by methods that match vehicles between two points or track
vehicles along a route. The following infrastructure sensors are options for measuring data within a
monitoring program.
2.4.1.1 Loop Detectors

All loop detectors can measure traffic volumes and occupancies (the percentage of time that the
detector had a vehicle over it during a given period). From this data, loop detectors can calculate
speed with reasonable accuracy, and from this, travel times can be computed. Double loop
detectors are sometimes used and transmit speed data directly; however, research has shown that
the speeds obtained from flow and occupancy data are more accurate than those reported by
double loop detectors.
Loop detectors have historically been the most common traffic monitoring tool due to their relatively
low cost of installation and high performance, and will thus be a key part of monitoring in many
jurisdictions and regions. Coverage, however, varies greatly between different cities and nations.
Page 13
Installing new loop detectors is problematic because of the intrusiveness of their installation and
maintenance requirements. Therefore, it is recommended that loop detectors be used for data
monitoring only where they will be used primarily for traffic operations. Monitoring is a secondary
use; deployments of loop detectors should be driven by operational needs.
2.4.1.2 Wireless Magnetometer Detectors

Advances in vehicle detection technologies have led to the development of smaller, wireless
magnetic vehicle sensors that can be placed in the roadway to collect the same data as loop
detectors. These dot detectors are faster and easier to install than loop detectors, require less
maintenance, and have data accuracy comparable to loops. Where agencies would like to install
additional in-road infrastructure detectors, wireless magnetometer detectors may have cost
advantages over loop detectors.
op V1
2.4.1.3 Radar Detectors
To overcome the shortcomings of loop detection, a number of regions have deployed radar
detectors, which are placed overhead or along the side of the road and measure volume and
C -
speed data. Radar detectors have the advantage of being less intrusive than in-road detectors, but
y
C
they can lose their speed calibration and are sensitive to bad weather conditions such as snow,
fog, or temperature change. Radar detectors are a viable option in locations where agencies want
to increase the frequency of data collection infrastructure along a roadway without installing more
C
loop detectors.
Q
2.4.1.4 Video Analytics

Another point-measure alternative is translation of video into point measures, via computer vision
algorithms. In the commercial marketplace, there are two alternative technology approaches for
video analytics. This first approach is a slightly more mature market for integrated hardware
systems. The second emerging technology is software-only systems that repurpose existing video
feeds into vehicle measures such as volume, occupancy, and speed.
2.4.1.5 Weigh-In-Motion
Weigh-in-Motion (WIM) are specialty sensors that provide vehicle (attribute) classification data of
vehicle streams at a given point. In addition, WIM equipment also provides planners and
designers with equivalent Single Axle Loadings (ESAL) that heavy vehicles place on pavements.
Motor vehicle enforcement officers use heavy truck axle load data to plan enforcement activities.
In summary, the uses of traffic and truck weight data include enforcement, pavement, bridge, and
legislative and regulatory issues. There are three main types of WIM technologies:
o Piezoelectric Sensor. In this technology a sensor is embedded in the pavement and

produces a charge that is equivalent to the deformation induced by thetyreloads on the
pavement’s surface. It is common to install two inductive loops and two piezoelectric
sensors in each monitored lane. A properly installed and calibrated Piezoelectric WIM
system can provide gross vehicle weights that are within 15% of the actual vehicle weight
for 95% of the measured trucks
o Bending Plate. The bending scale consists of two steel platforms. The plates are
instrumented with strain gauges, which measures tire load-induced plate strains. The
measured strains are then analyzed to determine the tire load. A properly installed and
calibrated bending plate WIM system can provide gross vehicle weights that are within 10%
of the actual vehicle weight for 95% of the measured trucks
Page 14
o Single-Load Cell. This device consists of two platforms placed adjacently to cover the
monitored lane. A single hydraulic load cell is installed at the center of each platform to
measure the tire load induced forces that are then transformed into tire loads. A properly
installed and calibrated single load cell WIM system can provide gross vehicle weights that
are currently? Within 6% of the actual vehicle weight for 95% of the measured trucks.
2.4.2 Vehicle-Based Sensors

There are two categories of vehicle-based sensors: Automated Vehicle Identification (AVI)
technologies and Automated Vehicle Location (AVL) technologies. These technologies are more
accurate in calculating measures such as travel times than infrastructure-based detection methods
as they provide direct travel times between two points for individual vehicles, but are not yet widely
implemented in real-time data collection due to the small percentage of vehicles currently equipped
op V1
for sampling. However, it is projected that vehicles equipped with AVI and AVL technologies will
become a far more prevalent part of the vehicle fleet, so all monitoring programs should
incorporate these technologies or plan on using them when their regional fleet has sufficient
vehicles from which to sample.
C -
y
2.4.2.1 Automated Vehicle Identification Technologies
C
Automated Vehicle Identification refers to technologies that detect a passing vehicle at a sensor
point, and then re-identify that same vehicle at a second sensor point. The advantage of AVI is that
C
it can provide data on the time it took for individual vehicles to get from one point to another (which
infrastructure sensors cannot). However, it cannot provide information on the route that each
Q
vehicle took. Since there are often multiple ways to travel between two points, especially in urban
areas, this presents some difficulty in accurately measuring route-based travel times that must be
overcome by deploying sensor-readers at frequent intervals and removing unrepresentative travel
times through a filtering process.
Bluetooth
Bluetooth receiver technology has only recently begun to be applied to traffic data collection, and
appears promising for inclusion in a travel time monitoring program. In the context of traffic,
Bluetooth detectors record the public Bluetooth wireless network identification (ID) of a driver’s cell
phone or other electronic device as the vehicle passes a point; this recorded ID number can then
be matched as the vehicle passes subsequent detectors, allowing travel times to be calculated.
This technology is advantageous in that it is accurate, low-cost, and portable. A drawback,

however, is that currently only a small percentage of drivers have Bluetooth-enabled devices in
their vehicles; estimates range from 1%-10% of the vehicle stream, depending on location. It is
projected that these percentages will grow as consumer Bluetooth market penetration increases,
making Bluetooth an important data collection alternative for future projects.
There are a few issues with Bluetooth measurements that need to be accounted for in the data
filtration process. First, Bluetooth readers frequently record the same wireless network ID more
than once as a vehicle passes, especially when vehicles are traveling slowly. These duplicate
addresses need to be removed to avoid counting a vehicle’s travel time more than once. Second,
the Bluetooth readers have a wide detection range that could collect travel times that do not reflect
actual conditions. For example, a Bluetooth sensor station at a freeway location could detect a
vehicle that is in a queue on an entrance ramp; as a result, a longer than accurate travel time
would be reported along the freeway from this vehicle. These unrepresentative travel times need to
be filtered out of the data. Additionally, on arterial streets, Bluetooth readers report travel times
Page 15
from non-vehicular modes like walking or cycling, so these times need to be removed in the data
filtering process to obtain an accurate representation of vehicular travel time reliability.
License Plate Readers

License Plate Readers (LPR) employs cameras that capture a digital image of a vehicle’s license
plate and use Optical Character Recognition (OCR) to read the plate number. While primarily used
for toll enforcement, LPR can also be to calculate travel times for vehicles that pass by two or more
cameras.
The advantage of LPR is that it can obtain travel time samples from vehicles without requiring the
presence of any specific device within the vehicle. This method, however, is not well suited for
data collection on high-speed freeways.
op V1
Additionally, plate matching is not always accurate, especially during adverse weather conditions.
The equipment needed is also costly, and there are some privacy concerns that come with tracking
a vehicle by its license plate number.
C -
y
Radio-Frequency Identification
C
C
Radio-Frequency Identification (RFID) technology is employed in electronic toll collection (ETC)
and can be used to re-identify vehicles for travel time purposes. Therefore, this technology will be
most applicable to urban areas that currently use or plan on implementing ETC. The percentage of
Q
drivers who have RFID tags varies depending on the region. The accuracy of the collected travel
time distribution increases with the number of samples collected, so the market penetration rate of
toll tags in a location is important in determining their role in travel time data collection. One
advantage of RFID is that its detection technology is very accurate; agency experience suggests
detection rates between 85% and 99%.
RFID is also an alternative for areas that do not have ETC if RFID tags get distributed to volunteer
drivers. Aside from sample size concerns, this technology also has associated some privacy
concerns since RFID provides data that is individually identifiable to a vehicle. Therefore, in using
this technology, the system will need to ensure that transmitted data is encrypted so as to remove
personal information.
Vehicle Signature Matching

Vehicle signature matching refers to recently developed methods that attempt to match the unique
magnetic signature of a vehicle as it passes over a loop to a signature from an upstream loop.
Single loop, double loop, and dot detectors all have this capability. While loops are not capable of
matching every vehicle, research and testing of this method has shown that it can match enough
vehicles to provide accurate travel time distributions for both freeways and arterials.
One advantage of this method is that it can use pre-existing detectors in new ways that provide
more accurate travel times than previously utilized spot flow and occupancy measurements. For
arterials, it is advantageous over traditional detector data since it estimates travel times without the
need for signal phase information. It also offers an additional benefit over other AVI technologies;
it avoids potential privacy concerns through anonymity. This technology has only seen limited use
in practice thus far, with projects in a few locations in California, but appears promising for
measuring travel times on both freeways and arterials.
Page 16
2.4.2.2Automated Vehicle Location

Automated Vehicle Location (AVL) refers to technologies that track a vehicle along its entire path.
These methods provide the most accurate and direct measurements of travel times, but have yetto
provide reliable regional travel time data. This is projected to change as more vehicles become
equipped with these technologies and agencies become more accustomed to using them for real-
time data collection.
Global Positioning Satellite

Any vehicle equipped with a GPS-based receiver can be tracked along its path of travel to
calculate route-based travel times and other traffic data, such as speeds. GPS technology is well-
suited for accurate travel time calculations because it can pinpoint a car’s location within a few
metres and its speed within 5kilometres per hour.
op V1
GPS has traditionally been used to calculate travel times with test probe vehicles equipped with
GPS receivers. The value of this data is limited because of the small number of test probe
vehicles typically deployed do not provide real-time data on a permanent basis. However, even in a
C -
more advanced system that monitors all GPS-equipped vehicles in-real time (via cellular modems
y
C
or other communication backhaul), the low market penetration rate of GPS technology will be a
constraint on its ability to accurately represent travel time and travel time variation. However, it can
be assumed that more vehicles and devices will have GPS capabilities in the future, so this method
C
should be incorporated into travel time reliability data collection.
Q
Connected Vehicle
Connected Vehicle (CV) programs, still in the initial stages of planning and development, aim to
provide vehicles with the capability to communicate data with one another and with roadside
transceivers through Dedicated Short Range Communication (DSRC). While these efforts are
primarily focused on improving travel safety, a side product is the ability of the roadside sensors to
measure travel times of equipped vehicles.
As an example of the future of this technology, Connected Vehicle California, an organization

comprised of a number of different agency, academic, and automotive industry partners, is
currently developing a test bed that includes collecting raw location, time, speed, and direction data
from vehicles to be processed into travel times and then distributed to a traveller information
system in the Bay Area. They are also testing efforts to collect weather data from probe vehicles,
which would be another valuable input in linking travel time variation with its causes. While this
technology is likely many years away from widespread implementation, it should be considered for
future travel time data collection efforts and used for such purpose in a region as the technology
becomes available.
Cellular Tower Triangulation

Travel times can be calculated by tracking vehicle locations through cell phones. Cellular
telephone networks track cell phones to know which base station the phone should be handed off
to as it travels. A traffic monitoring system could attain this information from the cellular telephone
networks to calculate travel times.
The advantage of this method is that cell phones are nearly ubiquitous in vehicles, leading to a
high sampling percentage. However, the accuracy of the data they transmit is very low; cell phone
triangulation methods only determine location with an accuracy of about 50-200 metres. Especially
in dense urban networks, this would create difficulties in assigning vehicles to a specific link and
Page 17
would result in inaccurate travel time estimation. Tests comparing travel times from cell phone
triangulation with those from probe vehicles showed that cell phone triangulation was only accurate
approximately 20% of the time.
For this reason, other AVL methods like GPS will be better suited for travel time data collection in
real-time.
2.5 Usage Monitoring Strategy

2.5.1 Strategy Methodology
Using the traffic and event typologies developed above, and referencing the collection technology
survey, this section develops a strategy for collection. It answers the question: which data items
op V1
should the department collect?
There are a variety of methods for determining which core data items to collect. At one end of the
spectrum, agencies can simply follow the state of the practice for monitoring. Using this approach,
C -
the agency would review the anticipated measures used by agency stakeholders, relying on the
y
C
accumulated wisdom of practice. At the other end, agencies could take a more top-down
approach. Using this process, they could generate stakeholder use cases, determine appropriate
measures to monitor each use case, and identify the associated core data items required to derive
C
those measures.
The methodology recommended in this guide to determine core monitoring data items falls
Q
between these two ends of the spectrum. It leverages monitoring best practice, reviews the
anticipated full set of possible measures envisioned in Chapter 5 of the Congestion Management
Procedures Manual (TR-523), and selects the core data items that can efficiently cover the largest
set of possible measures.
2.5.2 Key Data Items

Most traffic usage measures pivot around vehicle volume measurements. Most traffic congestion
measures can be derived from some combination of infrastructure sensors, producing speed and
occupancy data. Because of this, as previously discussed, infrastructure sensor technologies have
evolved to combine two or three of these core data items within a single, deployable unit. As with
other monitoring systems, these two values – speeds and volumes – form the core data units of
the DMAT’s monitoring system.
In addition, some key condition measures (pavements, per Manual TR-527) require attribute data
from vehicle flows. These are typically measured using classification or WIM sensors. These data
are critical to condition monitoring and should therefore be part of the DMAT’s monitoring system.
Increasingly, most monitoring systems include directly monitored travel times, as the technologies
for acquiring these have matured and the demand for more sophisticated measures – such as
travel time reliability – have increased. In accordance with the emerging state of the practice,
selective monitoring of travel times should be part of the DMAT’s long term monitoring vision.
The table below summarize these strategic choices, leveraging the previously developed typology
for traffic data. As with any monitoring program, more data is better than less data. This table,
therefore, does not exclude any data items from possible collection; rather, it prioritizes possible
data items for collection, with highest priority items highlighted.
Page 18
2.5.3 Summary
Table 1 shows a short summary for traffic data items.
Table 1: Traffic Data Items
Unit Data Item Priority Notes

A critical data item for building congestion
Speed High measures. Will be collected in real-time,
continuously, at numerous key locations.
The traditional basis for monitoring system usage.
Volume High Will be collected in real-time, continuously, at
numerous key locations.
Point
op V1
A proxy for density, presence is typically more
Presence
Low useful in operations than monitoring, given speeds
(Vehicle)
and volumes collection.
Some attributes of the flow are required for effective
C -
y
Attribute High condition monitoring, such as vehicle classifications.
C Will be sampled at selected key locations.
A state-of-the-practice monitoring data item. Can
C
be estimated from ubiquitous speed data, but
Travel Time High
should be collected in real-time, continuously, at
Q
selected key locations.

A state-of-the-art monitoring data item. This item
Link Set Flow Moderate can be leveraged for model calibration and
validation.
An obscure, rarely collected data item. It is included
Density Low in the typology for completeness only. Not
recommended for collection.
A key future data item, not yet required for effective
Attribute monitoring. Individual vehicle attribute data should
Vehicle Moderate
(Speed) be revisited as a possible key monitoring data item
in the next revision of this guide.
Page 19
Table 2performs the same summary for event data items that the above table performed on traffic
data items. In addition, this table notes key attributes of the event data. In all cases, location and
time attributes are required, and thus omitted from the explicit attributes column.
Table 2: Event Data Items
Unit Key Attributes Priority Notes

The key monitoring input, required for
Incidents Severity High stakeholders analysing the overlap between
safety and traffic.
A monitoring input required for stakeholders
Type analysing the intersection between maintenance
Work Zone Moderate
Lane impacts activities and traffic. A useful measure for
op V1
understanding construction impacts.
A monitoring input, required for stakeholders
analysing the intersection between special
demand situations and traffic. This measure
Events Participant Count Low
C - requires substantial monitoring effort and is
y
C difficult to automate. The results of this
monitoring are often difficult to interpret.
A monitoring input, required for stakeholders
C
analysing the intersection between signal
Signal status systems and traffic. Can include freeway
Control Moderate
Q
Control events signals (ramp meters) or arterial signals (at

intersections). A useful measure that is complex
to collect and archive effectively.
Visibility A monitoring input, required for stakeholders
Wind Speed analysing the intersection between weather and
Weather Moderate
Precipitation traffic. For the department, the key attributes for
Temperature collection are likely visibility and wind speed.
Page 20
3 CONDITION DATA REQUIREMENTS

3.1 Overview
This chapter examines asset management-related condition reporting as employed within the
context of pavement maintenance and rehabilitation. It details the importance of utilizing traffic
monitoring data as part of pavement management activities and provides recommendations for the
DMAT concerning the collection, analysis, and use of different data types within such a program.
3.1.1 Pavement Management Context

Condition reporting typically supports an asset management system. The principal purpose of a
op V1
transport asset management program is to minimize the life-cycle and user costs associated with
the on-going operation and maintenance of a wide range of transport-related assets, including:
 Roads
o Pavement
C -
y
 Major Structures
C
o Street Furniture
o Bridges
C
o Tunnels.
Asset management staff inventory and track the condition of a variety of assets. Pavement
Q
represents one of the most capital-intensive recurring investments most roadway-oriented

transport agencies are likely to operate and maintain. Maintaining pavement on a large roadway
network typically requires making complex decisions concerning how and when to resurface or
otherwise restore the roadway in order to ensure the preservation of an acceptable level of
roadway performance and user cost. Successful implementation of such a program requires the
appropriate allocation of resources and labour. Roadway supervisors have traditionally made many
of these decisions based on their knowledge and experience. Most agencies now use performance
measurement-oriented Pavement Management Systems (PMS), based on the analysis of high-
quality information, to enhance their decision-making processes. The use of total life-cycle
analysis considers the costs to road users (vehicle operation costs, travel time costs, accident
costs, etc.) and road agency costs for maintaining and improving the road network. The condition
of the road network is monitored and deterioration models are used to predict future life as a
function of maintenance options. Pavement Management Systems (PMS) perform these functions
and recommend repair alternatives that maximize economic value.
With the right data and analytical tools, an agency implementing a PMS should be able to identify
the ideal time to repair pavement using surface treatments or overlays, thereby avoiding the
extremely high costs associated with roadway reconstruction. With accurate user cost data, an
agency can also maximize the total economic return of maintenance spending for roadway users.
For example, allocating maintenance funds per roadway as a function of utilization. With that in
mind, this chapter provides an overview of the types and characteristics of data required for the
successful implementation of such a system.
Page 21
03-CONDITION DATA REQUIREMENTS First Edition -December 2016
3.1.2 Traffic vs. Pavement Monitoring

Deterioration of pavement occurs over time based on the cumulative effect of factors such as the
standard of original construction, frequency and type of maintenance employed and the nature of
traffic utilizing the roadway. Consequently, traffic data represents a major component of the
information required by PMS. Pavement condition data is a pre-requisite for understanding the
extent of a roadway’s current level of degradation. Traffic monitoring data is important because it
provides pavement managers with the ability to comprehend the type and level of utilization to
expect from roadway customers. Having access to both provides PMS users with animproved
ability to address the current level of deterioration, as well as take steps to more effectively inhibit
such deterioration in the future.
3.2 Condition Reporting Requirements
op V1
As indicated in the previous section, comprehensive pavement management requires two types of
data:

C -
Traffic Monitoring Data
y

C
Pavement Condition Data.
3.2.1 Traffic Monitoring Data

C
Many transport agencies have developed agency-wide traffic monitoring systems that can be used
to help assess the rate of pavement deterioration. Agency staff involved in pavement
Q
management-related activities should be aware of the capabilities of such systems in to effectively

integrate this data into their decision-making processes. In general, traffic data can be used to
help answer a number of questions related to the maintenance and rehabilitation of pavement,
focusing on optimizing life-cycle costs.
There are three fundamental types of traffic data that should be sought by pavement management
professionals as part of a comprehensive pavement management system. They are:
 Traffic Volume Data

 Vehicle Classification Data
 Truck (and Axle) Weight Data.
3.2.1.1 Traffic Volume Data

Traffic volume data is a vital component of traffic and transport-related planning. At the same time,
it also serves as an input for pavement management systems. When included as part of pavement
management decision-making, traffic volumes are typically described as Average Annual Daily
Traffic (AADT) volume. AADT represents the total number of vehicles, including passenger cars,
trucks, and buses, moving in both directions of travel, expected to use a roadway during a typical
day for a given year.
AADT can be used in conjunction with other variables such as average percentage of truck traffic
(obtained from vehicle classification data) to develop factors for assessing the deterioration of
roadway pavement based on vehicle loading. See the section on Vehicle Classification Data for
additional information concerning approaches for classifying vehicles.
AADT was traditionally computed as the simple average daily traffic volume for all 365 days of the
year, based on continuous count data collection. However, because in many cases traffic
Page 22
monitoring equipment is not operational 24 hours per day, 7 days per week, 365 days per year,
average volumes generated using this technique have the potential to be based on days having
atypical volumes, resulting in inaccurate AADT. For this reason, a new approach for calculating
AADT was developed by the American Association of State Highway and Transportation Officials
(AASHTO) during the mid-1990s. This approach is based on calculating seven monthly averages
(one for every day of the week) for each month of the year. The resulting 84 values (7 daily
averages x 12 months) are then averaged to obtain AADT. The United States Federal Highway
Administration (FHWA) has adopted this approach as its preferred method for calculating AADT in
its FHWA Traffic Monitoring Guide.(4)
Calculating Vehicle Kilometres of Travel

Multiplying AADT for a section of roadway by the section length generates estimated Vehicle
op V1
Kilometres of Travel (VMT). Some agencies have begun analysing the relationship between VMT
along various sections of roadway and the condition of pavement along those sections to establish
the percentage of travel occurring on pavement of differing conditions. In addition, understanding
the portion of VMT composed of truck traffic is important for pavement design and management
applications.
C -
y
C
Ensuring Traffic Volume Data Quality
While deploying and maintaining large numbers of continuous count stations ensures provision of
C
the most accurate AADT, doing so is both costly and time consuming. As most agencies only have
enough resources to monitor a very small percentage of the roadway network using this method,
Q
most AADTs are generated based on short-term sampling methods. In cases where such methods
are used, traffic count data is collected using portable sensors deployed for between for two weeks
days at least every three years along highways and other multilane facilities (including primary
arterials). All other locations, such as minor arterials, collectors, and ramps, should be assessed at
least once every six years. A number of methods are used to calculate AADT from such short-term
count data, but most methods attempt to remove seasonal and day-of-week biases by applying
adjustment factors created based on data from nearby continuous count stations. Finally, AADT
estimates for all sections of roadway not explicitly counted during the current year should be
estimated using an annual growth adjustment factor. Such growth adjustment factors are
statistically determined using historical data for the road segment. However, in cases where no
historical data is available, growth adjustment factors from similar road segments are typically
used.
3.2.1.2 Vehicle Classification Data

Just as traffic volumes can vary along different roads, so can the vehicle types using them. As a
result, vehicle classification data is required for use in accurately identifying variations in the
composition of the traffic stream; especially with regard to the percentage of heavy trucks. The
reason behind this is that although trucks and buses make up a minority of total motor vehicle
traffic, the weight of these vehicles produces the majority of deterioration and damage to the
roadway. The most common method for assessing the quantity of roadway deterioration
generated by traffic of different types is to characterize that damage as an equivalent number of
“standard” loads. The most frequently used standard load is the 8182 kg (18,000 lb.) Equivalent
Single Axle Load (referred to as an ESAL).
Roadway agencies utilize a variety of vehicle classification schemas. Amongst these, the most
basic distinction is one between cars, single unit commercial vehicles (i.e., trucks and buses), and
Page 23
combination commercial vehicles (i.e., tractor-trailer combinations). As the proportion of buses in

the traffic stream is usually small, commercial vehicle traffic is typically referred to as “truck traffic.”
In its simplest form, composition of the traffic stream can be expressed as the “percentage of
commercial vehicles” within AADT. However, for pavement management purposes, it is
insufficient to know only the number of commercial vehicles without having a further understanding
of the specific vehicle types. In contrast with some other agency approaches, the FHWA classifies
vehicles in terms of their configuration rather than weight. Overall, the FHWA Traffic Monitoring
Guide (TMG) recommends sorting vehicles into 13 different categories. These categories,
including a description of each, are listed in Table 3.(4)
op V1
C -
y
C
C
Q
Page 24
Table 3: FHWA Vehicle Classification
Class Type Description
All two- or three-wheeled motorized vehicles. Typical vehicles in

this category have saddle type seats and are steered by
handlebars rather than wheels. This category includes
1 Motorcycles
motorcycles, motor scooters, mopeds, motor-powered bicycles,
and three-wheel motorcycles. This vehicle type may be reported
at the option of the State.
All sedans, coupes, and station wagons manufactured primarily
2 Passenger Cars for the purpose of carrying passengers and including those
passenger cars pulling recreational or other light trailers.
op V1
All two-axle, four tire, vehicles, other than passenger cars.
Other Two-Axle, Included in this classification are pickups, panels, vans, and other
3 Four-Tire Single Unit vehicles such as campers, motor homes, ambulances, hearses,
Vehicles and carryalls. Other two-axle, four-tire single unit vehicles pulling
recreational or other light trailers are included in this classification.
C - All vehicles manufactured as traditional passenger-carrying buses
y
C with two axles and six tires or three or more axles. This category
includes only traditional buses (including school buses)
4 Buses
functioning as passenger-carrying vehicles. All two-axle, four-tire
C
single unit vehicles. Modified buses should be considered to be a
truck and be appropriately classified.
All vehicles on a single frame including trucks, camping and
Two-Axle, Six-Tire,
Q
5 recreational vehicles, motor homes, etc., having two axles and

Single Unit Trucks
dual rear wheels.
Three-Axle Single All vehicles on a single frame including trucks, camping and
6
Unit Trucks recreational vehicles, motor homes, etc., having three axles.
Four or More Axle
7 All trucks on a single frame with four or more axles.
Single Unit Trucks
Four or Less Axle All vehicles with four or less axles consisting of two units, one of
8
Single Trailer Trucks which is a tractor or straight truck power unit.
Five-Axle Single All five-axle vehicles consisting of two units, one of which is a
9
Trailer Trucks tractor or straight truck power unit.
Six or More Axle All vehicles with six or more axles consisting of two units, one of
10
Single Trailer Trucks which is a tractor or straight truck power unit.
Five or Less Axle All vehicles with five or less axles consisting of three or more
11
Multi-Trailer Trucks units, one of which is a tractor or straight truck power unit.
Six-Axle Multi-Trailer All six-axle vehicles consisting of three or more units, one of
12
Trucks which is a tractor or straight truck power unit.
Seven or More Axle All vehicles with seven or more axles consisting of three or more
13
Multi-Trailer Trucks units, one of which is a tractor or straight truck power unit.
The importance of dividing trucks into different categories, rather than working with a single truck
percentage, is illustrated in Figure 5 which compares the distribution of truck types obtained on two
rural roads, an Interstate (freeway), and a collector. As this figure illustrates, the majority (about 80
per cent) of commercial vehicles on the Interstate were 5-axle single trailer trucks (Class 9 in Table
3), while the majority of commercial vehicles (about 70 per cent) on the rural collector were 2-axle
trucks (Class 5 in Table 3).
Additionally, having accurate classification data enables the generation of more accurate Axle
Correction Factors, used to estimate the types of vehicles, and resultant axel counts, moving along
Page 25
a given section of roadway based on data collected from more conventional volume sensors;
updated each year based on current vehicle classification data.
It should be noted that some agencies aggregate these 13 categories into a smaller number,
generally between three and five, for pavement loading-related calculation purposes.
Percentage of Total Truck Traffic
op V1
C -
y
C
C
Vehicle Class
Q
Figure 5: Example Distribution of Commercial Vehicles on Two Roads
Ensuring Vehicle Classification Data Quality

It is generally recommended that traffic classification data be collected in a manner that parallels
that utilized for traffic volume data collection, described in subsequent sections. Similarly, limited
resources preclude the collection of continuous classification data on a network-wide basis. As a
result, a fairly large number of short duration (minimum of 48 hour) counts should be conducted to
monitor and capture both vehicle classification and overall volume data. Such data provides end
users with the basic truck traffic statistics necessary for pavement-related applications. However,
as with volume-based data collection, steps need to be taken to remove seasonal and day-of-week
biases associated with short-term data. Such adjustments are typically achieved via the
application of factors developed using data generated by continuous counters operating nearby. It
is recommended that classification data be collected, at a minimum, every three years along
highways and other multilane facilities (including primary arterials), with annual adjustment factors
utilized to estimate classification distributions for all sections of roadway not explicitly evaluated
during the current year. All other locations, such as minor arterials, collectors, and ramps, should
be assessed at least once every six years.
Although the combination of AADT and truck-oriented classification data meet many of the Abu
Dhabi DMAT needs for pavement-related analysis, it should be kept in mind that new pavement
guidelines will likely also seek data concerning both seasonal and time of day variations in truck
volumes. This additional data is being pursued due to differences in the behaviour of pavement
under varying environmental and temperature conditions having an impact on the useful life of the
road surface.
Page 26
3.2.1.3 Truck Weight Data

Data concerning truck class or type, volume, and weight (including axle type and load) are used as
primary inputs for a variety of roadway agencies’ most important tasks related to pavement
management. Examples include:
 Determining pavement overlay and reconstruction design thickness

 Selecting pavement maintenance and rehabilitation treatments
 Determining pavement-loading restrictions
 Estimating user cost benefits.
These data types are collected using Weigh-in-Motion (WIM) systems. WIM systems not only
assess the axle weight and type of all passing axles, they also classify vehicles. As compared with
op V1
other traffic monitoring activities, WIM data collection requires some of the most sophisticated
sensors, the most controlled operating environments (strong, smooth, level pavement in good
condition), and the most costly equipment set up and calibration.
Roadway agencies commonly summarize, report, and input truck weight data in the following
C -
y
ways:

C
Gross vehicle weight (GVW) per vehicle (usually by vehicle class)
 ESAL for specific vehicle types (as discussed in section 3.2.1.2)
C
 ESAL distribution for specific vehicle types.
Beyond the truck weight metrics described above, more advanced pavement management tools
Q
require data inputs concerning Axle Load Spectra, which characterizes loading directly based on
the number of axles (e.g., single, tandem, triple, and quadruple), axle configuration (based on the
10 FHWA Vehicle Classification Categories reserved for commercial vehicles), and vehicle weight.
Although the number of variables involved in calculating axle load spectra makes it significantly
more challenging to compute, it produces a more precise characterization of traffic (typically in
tabular format) that provides relative axle weight frequencies for each common axle combination
over a given period of time.
In general, the primary objective of a truck weight data collection program is to obtain accurate
estimates of the distribution of vehicle weight and axle loads according to roadway and vehicle
class. Statistics such as those enumerated above for a given vehicle class can be expressed as
distributions, as mean values, or as mean values with specified confidence intervals, depending on
the needs of the analysis that will use this information.
Furthermore, each of these summary statistics can be generated for a specific site, a group of
sites, or an entire geographic region, depending on the needs of the analysis and the data
collection and reporting procedures.
As a single average statistic for any of these measures is not likely applicable on a region-wide or
national basis, truck weight data collection should focus on the creation of accurate summary axle
load distributions that can be applied to different types of roads. The most important step in this
process is to group a given region’s roads into categories carrying different types of truck traffic, so
that each category reflects commercial truck and bus traffic having reasonably similar
characteristics. For example, roads that convey trucks carrying agricultural materials should be
grouped separately from roads carrying only goods for urban delivery. Categories such as “rural”,
“urban”, “residential”, and “industrial (ports)” are typically used. In general, these categories should
be easily applied within each region. They should also provide a logical means for discriminating
Page 27
between roads likely to have very high load factors and roads with lower load factors (i.e., between
roads where most trucks are fully loaded and roads where many trucks are only partially loaded or
empty).
Additionally, the DMAT should apply its knowledge regarding the movement of specific types of
heavy trucks to the truck weight grouping process, so that roads that carry those heavy trucks can
be grouped appropriately. For example, roads leading to/from port facilities might be grouped
together due to the nature of the load factors common to many of the vehicles likely to be found on
those roadways. One thing to keep in mind when implementing this process is that the number of
groups created impacts the number of WIM installations needed. As a result, it is recommended
that agencies just starting to pursue the collection of WIM data begin by defining a fairly small
number of groups in order to limit the number of WIM stations required.
op V1
Ensuring Truck Weight Data Quality
As indicated previously, calibration of WIM equipment is more demanding than the calibration of
other types of traffic monitoring equipment. Due to the fact that vehicle dynamics are impacted by
pavement roughness, the correct calibration value for a WIM scale is a function of the pavement
C -
y
condition at each site. As these conditions differ at each sensor location, a significant calibration
C
effort is required each time WIM equipment is deployed, as well as over time to ensure that an
appropriate level of accuracy is maintained.
C
Within each grouping of roads described in the previous section, a number of WIM stations should
be used to collect data; with at least one of these sites operating continuously throughout the year
Q
to track seasonal changes in the loads being carried. That said, operating additional WIM stations
continuously throughout the year should provide a more reliable assessment of the impact of
seasonal change, as well as facilitate an evaluation concerning whether the initial groups of roads
selected do, in fact, carry similar types of commercial traffic as anticipated.
3.2.1.4 Traffic Data Collection Summary

Roadway infrastructure (e.g., pavement) management professionals need to be familiar with the
nature of traffic data, in particular, its temporal and spatial variability; when utilized for pavement
management purposes, traffic data variability is typically expressed in terms of daily, monthly
(seasonal), or yearly variation. Having an understanding of this variation is important as it may
impact pavement performance in a number of ways, including:
 Hourly variation in traffic volume may be important for predicting rutting in asphalt concrete
mixes
 Monthly (seasonal) variation of various measures may exert a significant influence on
pavement damage occurring during different periods.
Finally, due to the difficulty of collecting high quality truck volume and weight data, it is not
recommended that enforcement of legal commercial vehicle weights be tied to data collection
carried out for pavement management purposes. Doing so has the potential to contribute to the
creation of long-term problems in obtaining unbiased and accurate data.
3.2.2 Pavement Monitoring Data

Asset management systems maximize the benefit derived from funds expended by selecting
optimal maintenance and rehabilitation alternatives and schedules. The focus here is on selection
Page 28
of those pavement condition data items that will provide a basis for long-term management
decision-making.
Collection, storage, and analysis of pavement condition data serves as the basis for identifying on-
going maintenance and rehabilitation needs and related management activities. The pavement
performance monitoring process consists of:
 Periodically evaluating pavement performance based on metrics collected over time

 Predicting future pavement performance using models and archived data
 Deriving maintenance and rehabilitation budgeting recommendations based on a
comparison of potential maintenance and rehabilitation alternatives.
3.2.2.1 Pavement Condition Data
op V1
Pavement maintenance and rehabilitation decisions are primarily driven by pavement condition
and user cost data. There are a number of methods for performing pavement condition monitoring,
with associated strengths and weaknesses. These methods are often used together:

C -
Visual inspection by condition class: roads are assigned to condition classes such as very
y
C
good, good, fair, and poor. This type of visual inspection is performed quickly and relatively
inexpensively
 Pavement roughness/smoothness: is measured directly and is standardized as the
C
International Roughness Index (IRI). The IRI is relatively inexpensive to collect and
analyse, and provides a measure of the level of service provided to roadway users. IRI
Q
measures the pavement profile and therefore does not necessarily include structural
damage that is accumulating at lower levels. IRI can therefore not be used alone to
manage pavements. Physical distresses must also be used for decision-making
 Measuring individual pavement distresses: modern pavement management systems
record individual distress data measured in the field per pavement section. Individual
distresses can then be modelled and used to predict pavement life. A number of measures
can be drawn from individual distresses, such as the structural condition index (SCI) and
Pavement Condition Index (PCI), discussed below. These indexes are calculated by the
pavement management system and useful for characterizing the condition of individual
pavement sections and for decision-making.
An example of a composite pavement condition index calculated from measured individual

distresses is the PCI. PCI is an analytical tool used to evaluate the pavement condition by
measuring and recording individual distresses. It was designed require a minimum amount of
equipment and is performed visually. It measures distresses that effect longitudinal profile (also
measured by IRI) and other distresses that don’t affect wheel path profile, such as longitudinal
cracking, alligator cracking, block cracking, ravelling, and others that are not in the wheel path. PCI
is expressed as a numerical rating (0 - 100) for each pavement section. It is used to monitor high
and low volume highways, municipal roads, parking lots, and airfields.
The most important practical advantage of the PCI is that it has been developed to represent the
collective judgment of experienced engineers regarding the overall condition of pavement.
Pavement supervisors use the overall PCI rating to select the category of maintenance and
rehabilitation needed. The PCI evaluation process assesses the extent, severity, and type(s) of
surface distresses, as well as the road ride quality. Surface distresses can be categorized as:
 Surface defects
Page 29
o Raveling of surface aggregate

o Flushing (or bleeding of asphalt)
 Surface deformations
o Rippling and shoving
o Wheel track rutting
o Distortion
 Cracks
o Longitudinal wheel track fatigue: single, multiple, alligator
o Centre-line: single, multiple, alligator
o Pavement edge: single, multiple, alligator
o Transverse: single, multiple, alligator
o Longitudinal: meander or mid-lane.
op V1
Distresses within statistically sampled sections of roadway are catalogued and scored for use in
calculating the PCI. This data is typically collected as part of a windshield survey or post-collection
analysis of digital video; collected by an instrumented survey vehicle driving at highway speeds.
Survey vehicles are often instrumented with profilometers to enable the collection of longitudinal
C -
roughness and transverse rutting data, as well as video equipment for use in subsequent PCI
y
evaluation.
C
The monitoring of distresses enables identification of performance trends over time and together
C
with distress models, provides the capability to forecast performance. A pavement management
system can associate maintenance activities with different distresses, their severities, and/or
Q
composite condition scores. Maintenance activities can be broken down into 4 categories:
 Preventative (Preservation)
 Routine maintenance
 Structural improvements
 Reconstruction.
Each of these activities can be triggered by a PMS when specific distresses reach threshold values
and/or composite condition scores reach threshold values. The thresholds selected should be
tailored to the DMAT based on agency needs and potential rehabilitation options.
In addition to assessing PCI (and by implication, individual distresses), it is recommended that

pavement managers also measure roughness/smoothness with IRI.IRI will measure the vertical
deflection caused by distresses in the wheel path, but will miss all other distresses and structural
problems. Therefore, IRI only provides a trailing indicator of structural performance. As a result,
IRI cannot be used by itself to drive the pavement management and rehabilitation process; it must
be used in conjunction with monitoring of distresses. An example of structural failure that IRI
would miss is low or medium severity alligator cracking and low level rutting.
Roadways can be classified as highways or municipal roads. Highways can be further classified as
primary and secondary motorways. Municipal roads can be classified as arterials, collectors, or
local roads. Considering only agency costs, the roadway class of each section of pavement
determines the trigger values that will drive the potential application of various maintenance
alternatives. For example, a PCI of 70 with 5 per cent alligator cracking may trigger the application
of an asphalt concrete overlay for a high use primary highway, but trigger nothing or only a
preservation treatment for a low volume municipal collector. A PMS that uses a life-cycle cost
model will make maintenance recommendations as a function of road agency costs and user
Page 30
costs. User costs would include vehicle operation costs, passenger time, and costs due to road
accidents.
Monitoring pavement using PCI and IRI facilitates the quantification of functional and structural
performance over time. In addition to surface performance, the monitoring and diagnosis of
subsurface layer properties is also recommended to more accurately establish the structural
capacity of heavily travelled highways and streets. This can be accomplished via the periodic use
of Falling Weight Deflectometre (FWD) non-destructive testing. FWD testing applies an impact load
to the pavement that resembles a moving heavy truck wheel/axle. The maximum deflection and
deflection basin data collected are then used to establish the load carrying capacity of the
pavement, which provides an estimate of its ability to carry heavy loads in the future. In addition,
FWD deflection data can also be used to design structural overlays and to support reconstruction
decision-making concerning the pavement.
op V1
Ensuring Pavement Condition Data Quality
Regardless of how pavement condition data is collected, it is important that the implementing
agency first develop a detailed quality management plan, including well-documented quality
C -
y
acceptance procedures. It is important that this quality management plan define the activities to be
C
conducted, as well as clear timelines, milestones, and evaluation criteria.
C
As concerns the frequency with which pavement condition data should be collected, the following
schedules are recommended:
Q
 PCI
o Primary Main Roads- initially, every year for three years and then every other year
o Secondary Main Roads - every year for three years and then every two to three
years
o Arterial and Smaller - one year and then every two to three years
 IRI
o Every one to two years
 FWD
o Entire network tested via statistical sampling process every five years (i.e., conduct
sampling every 20 metres within larger intervals of 300 metres).
In addition, sections of highway with poor PCI values (55 or lower) should be surveyed on an
annual basis.
3.2.2.2 Pavement Data Collection Summary

Measurement of pavement performance that includes individual distresses (surface and structural
condition), IRI (functional condition), and FWD (load carrying capacity) provides a comprehensive
basis for making effective road agency cost decisions concerning the type and timing of
maintenance and rehabilitation treatments. Combined with user cost data, these measures
provide the road agency with the tools necessary to both manage the present, as well as plan for
the future needs of a roadway network’s pavement management system.
Page 31
3.2.3 Other Data Required to Perform Condition Reporting

In addition to data attributes concerning traffic and pavement condition, PMS users must also deal
with other data types that have the potential to degrade the system’s effectiveness. The two likely
to have the greatest impact are Metadata and Location Referencing Data.
3.2.3.1 Metadata
Within the context of pavement management, metadata are used to describe data collection
procedures and post-processing techniques that may impact the consistency or quality of the data
available for analysis. For example, the U.S. DOT’s HPMS System contains metadata related to
the collection and reporting of:
 Traffic counts
op V1
o Per cent total section-level AADTs reported that are based on actual counts for the
reported data year or factored prior year AADT
o Number of permanent and portable counter locations that were counted for 24 hours
or more
C -
o Per cent of class AADTs reported that are based on actual counts for the reported
y
C
data year or factored prior year class AADT
 Vehicle classification
o Type of classification counts used for reporting purpose
C
o Per cent of permanent and portable classification count locations that were counted
for 24 hours or more
Q
o Per cent of permanent and portable classification count locations that were counted
for 48 hours or more
 Type of IRI equipment used to measure the International Roughness Index (IRI)
o Type of equipment used predominately for measuring the international roughness
index (IRI)
o IRI reporting interval (not to be confused with device sampling interval)
 Method and equipment used to collect rutting data
o Method (Manual or Automated) used to collect most of the rutting data
o Number of sensors for the equipment used predominately for collection of rutting
data.
o Type of equipment used predominately for collection of rutting data.
One of the biggest hurdles to overcome in the successful implementation and operation of a PMS
is ensuring consistency with legacy data. The use of appropriate metadata can help to facilitate
any such transition.
3.2.3.2 Location Referencing Data

Pavement management systems assess roadway condition both by location (spatially) and with
respect to time (temporally). This aspect of management is important because meaningful
analysis typically requires multiple years of data be collected and assessed for the same
location/segment before pavement deterioration trends can be wholly understood. Consequently,
the importance of having high quality location referencing data cannot be understated. This is
explained in more detail in the next section.
Page 32
3.3 Pavement Management Implementation

Previous sections of this chapter have detailed the types of data to be collected in support of a
PMS, as well as techniques for collecting that data and ensuring its quality. That said, the primary
question remaining is how to make the best use of this data. In addressing this question, the most
important concept to understand is that a PMS can be implemented on two primary levels, the
Network level and the Project level.
 Network Level Management. Is typically focused on the evaluation of all roadways within
agency’s jurisdiction, and is aimed at developing a prioritized maintenance and restoration
program anticipated to result in the greatest benefit given overall agency budget
constraints.
In general, network-level management makes use of significantly greater amounts of estimated
op V1
data than does project level management. As part of the development of a network level
management program, agency roadways should each be identified according to their functional
classification. This classification will guide monitoring and investment.
 Project Level Management. Focuses on the collection and analysis of data for a specific
C -
location or section of roadway after it has been identified as a candidate for maintenance or
y
C
repair. This level of analysis requires greater detail than that collected at the network level
in order to ensure that an adequate understanding exists regarding the current nature of the
pavements’ condition and the causes of deterioration.
C
Given that these approaches to pavement management are not mutually exclusive; the DMAT will
need to develop an implementation strategy that considers both.
Q
Finally, one of the goals of the DMAT’s asset management program should be to avoid duplication
of effort with regard to its data collection activities. However, successful coordination of these
efforts requires that a number of other issues first be addressed, including:
 Agency-wide agreement on the use of a common location referencing system.

 Development of overlapping data collection schedules.
 Design of compatible approaches for data processing, storage, and retrieval.
As these activities are nearly impossible to coordinate when approached on a project-by-project

basis, it is recommended that the Department seek to institutionalize them as part of its standard
practices to the greatest extent feasible.
Page 33
4 LOCATION DATA REQUIREMENTS

4.1 Overview
The underlying data set required to effectively monitor a transport system is a clear description of
the location and attributes of the network itself. Without a strong base of geographic location data,
the core datasets of condition and traffic information cannot be effectively leveraged. Thus,
location and descriptive attribute information serves as the core metadata for this system.
Four core data sets define transport networks:
 Spatial Reference
op V1
 Linear Reference
 Topology
 Attributes.
4.2 C -
Spatial Reference
y
C
4.2.1 Geographic Referencing
C
The spatial referencing required for performance monitoring is associated with the road network
itself. Shapes are geo-referenced geometric primitives that define the spatial location of a given
Q
transport network elements. The primary transport network elements within a performance
monitoring framework are the roads themselves. However, additional elements - mostly street
furniture, such as signals, signs, and cabinets as well as bridges - can also serve important roles
within performance monitoring.
Geographic information systems are a complex topic that is beyond the scope of this monitoring
document. Most of the references to geographic information fundamentals here are guided by
specifications from the Open Geospatial Consortium1. To effectively locate a roadway, shapes
require two primary components: a geometric primitive (i.e. a basic shape) and a spatial location.
Geometric primitives have fairly complex data models. An example of a geometric primitive
typology is given in below.
The Open Geospatial Consortium (OGC) is an international industry consortium of 479 companies,
government agencies and universities participating in a consensus process to develop publicly
available interface standards. OGC® Standards support interoperable solutions that "geo-enable"
the Web, wireless and location-based services and mainstream IT. The standards empower
technology developers to make complex spatial information and services accessible and useful
Page 34
04-LOCATION DATA REQUIREMENTS First Edition -December 2016
with all kinds of applications.
op V1
Figure 6: Polyline Example
C -
y
C
Next, each geographic primitive needs to be spatially referenced. The spatial reference standard
for performance monitoring should be latitude and longitude. Latitude and longitude are polar
coordinates, not Cartesian coordinates. They locate objects on a sphere (the earth), not on a flat
C
surface (a map). If they are interpreted and rendered as a flat surface, they distort the image to the
viewer in non-intuitive ways. For example, distances on the projected image will not be true.
Q
Coordinate Referencing Systems (CRS) translate these polar coordinates into Cartesian
coordinates for objects at or near the surface of the earth. A framework for translating between
polar and Cartesian coordinates is beyond the scope of this guide. The best practice for transport
agencies is to store this data in polar (X,Y) coordinates.
One implication of this recommendation is the need for a consistent geodetic datum. Because the
Earth is not perfectly spherical, geographers use models of the earth to represent this imperfection.
These models are called geodetic datums. The current world standard geodetic datum is the
World Geodetic System of 1984 (WGS84). The best practice for transport agencies that store
geographic data in polar coordinates is to use this datum.
4.2.2 Transport Network Assumptions

Objects with a geographic primitive and a spatial reference are typically termed geographic
features; for brevity, this guide will henceforth refer to them as features. The monitoring-specific
issues with transport features revolve around their reference location and directionality. Typically,
roadways are represented as polylines, which means they have length, but no width. Of course,
physical roadways do have widths, generally scaling by the number of lanes they contain. Thus,
the geographic representation of a roadway for monitoring must select some consistent point on
the roadway to represent. The best practice for most agencies is to the model the roadway using
centre-lines; this reference location is recommended practice for the Abu Dhabi DMAT.
The second issue is directionality. Roadways can be modelled using a single polyline to represent
both directions of travel (bi-directional), or two polylines, to represent each direction of travel. The
best practice for most agencies is to use two sets of polylines, one for each direction. This allows
for more accurate representation of the physical roadway, simplifies linear referencing
requirements, and sets the stage for more sophisticated topological representations. With bi-
directional representations, centre-lines should still be used as the reference point, but should
Page 35
correspond to the left (with respect to direction of travel) edge of the roadway.
4.3 Linear Reference

Linear referencing systems are an alternative to coordinate systems (such as latitude and
longitude) for locating objects in geographic space. These systems locate an object with reference
to an existing roadway. Thus, a roadway must be spatially referenced in order to be linearly
referenced. Once it is spatially referenced, additional metadata can be added and additional
computations can be performed, in order to create a linear referencing system on top of the road
network. This linear referencing system can then be used to uniquely locate features on the road
network, without using a spatial reference system.
The additional metadata required to develop a linear reference system is as follows. Each
op V1
geographic feature should contain the following information:
 Route Identification(ID)
 Start Post-kilometre

C -
End Post-kilometre.
y
C
The Route ID is simply a unique identification for a route, which can span multiple features.
Ideally, Route IDs will correspond to the agency route numbering system (see Manual AD-P-01),
C
but this is not required. What is required is that every Route ID is unique. In addition, the
difference between the start and end post-kilometres should correspond exactly to the length of a
given roadway feature. In addition, a given feature should have a start post-kilometre that is
Q
exactly equal to end post-kilometre of its upstream feature. For example, if Feature B is
downstream of Feature A, and upstream of Feature C and has a start post kilometre of 10 and an
end post kilometre of 20, then the end post kilometre of Feature A must be 10 and the start post-
kilometre of Feature C should be 20. The start post-kilometre of the first feature on a route should
be zero. See Figure 7, below, for an illustration of this example.
Figure 7: Picture of a Simple Linear Referenced Route
This approach has two major consequences. First, post-kilometres for the same location can
differ, based on the direction of travel. Second, post-kilometres for a given location can change
Page 36
over time, as roadway alignments change. This dynamism generally confounds the use of this
linear referencing system as a communications medium for agency stakeholders. For example,
post-kilometre signs erected using this linear referencing scheme require frequent updates. Other
types of signage (exit numbers that reference post-kilometres) also require expensive updates. In
fact, the only way to know the true post-kilometre of any given point on the roadway, is to query the
system in real-time.
Because of this, some transport agencies develop two post kilometre systems: an internal system,
used for performing calculations; and an external system, used for communicating location. The
internal system is dynamic and frequently updated. The external system is relatively static and
rarely updated. External systems allow agencies to maintain a consistent referencing system for
roadway stakeholders: other departments (such as maintenance); roadway users (travellers); and
sister agencies (such as law enforcement).
op V1
In order to effectively monitor performance, an internal linear referencing system is required.
Without it, systems cannot effectively perform basic calculations, such as transforming spot speeds
into travel times, or translating volumes into vehicle kilometres travelled. If an agency uses an
C -
external linear referencing system, best practice for most agencies is the ability to translate
y
C
seamlessly between the internal and external systems.
Best practice in linear referencing systems for performance monitoring requires a software utility to
C
translate a linear reference into spatial coordinates, and vice versa. Abu Dhabi DMAT should
develop such a system.
Q
4.4 Topology
Topological networks are a connected set of features that can be logically traversed in the same
way that travellers traverse the physical network they represent. There are many topological
representations, but - at core - they all involve connecting an upstream feature to a downstream
feature. Assigning unique IDs to all features, then attaching a vector of downstream ID pointers to
each feature typically accomplish this connection. For example, if Feature 1 connected to Feature
2 and Feature 3, then the vector of pointers for Feature 1 would be [2,3]. Figure 8 below illustrates
the topology that this data structure would model. In this topology, polyline A->B leads into polyline
B->C and polyline B->D.
Page 37
op V1
C -
y
C Figure 8:A Topology
C
As long as an agency implements a linear referencing system, topological representations are not
strictly necessary for effective performance monitoring. However, they are extremely useful and
are recommended for any routes where an agency may perform performance measurement or
Q
modelling.
4.5 Attributes
Attributes are simply numerical or text data elements that describe non-locational elements of the
roadway. Typically, they are associated with a homogenous section of the roadway, although they
can be individually linear referenced as well. Typical segment feature attribute data items are
shown in the table below:
Page 38
Table 4: Example Attributes
Attribute Type Example
Road width Decimal 10.9 m

Lane width Decimal 3.6 m
Inner shoulder width Decimal 1.5 m
Outer shoulder width Decimal 2.4 m
Design speed limit Decimal 110 kph
Functional class List Principal Arterial
Inner median type List Unpaved
Inner median width Decimal 14.0 m
Terrain List Flat
op V1
Population List Urbanized
Barrier Text Three Beam
Surface List Concrete
C -
Some segment attributes are environmental variables, associated with a segment for convenience.
y
For example, population is not a property of the segment; it is a property of the surrounding
C
environment. In a GIS system, it might properly be represented as a series of polygons with
specific densities. These environmental variables should be tied to segments only if the other GIS
C
layers needed to glean the information are not available for quick overlay and extraction.
In addition to segment-specific attributes, it can be useful to record lane specific attributes. These
Q
attributes are unique to both a segment and a lane in a direction of travel.
Page 39
5 DATA COLLECTION
Data collection is the core activity in monitoring. Typical monitoring processes can include low
technology systems such as sampled manual counts,or complex technologies such as continuous,
real-time sensor or probe data. Over time, however, for most agencies, the process has evolved
from labour intensive, sampling-oriented approaches to technology-driven continuous collection
systems. The current state of the practice in data collection is to install sensors, use
communications technologies to attach them to a data collection system, and collect continuous
data in real-time. From here, the data is managed and processed using methods described in
Chapter 6.
This data collection chapter advises on two key areas: The Methods section discusses the various
op V1
technologies and sources that can be used for data collection, and advises on when a given
method is appropriate. The Procurement section presents various implementation strategies for
effectively collecting data.
C -
y
5.1 Methods
C
This section describes the technologies and sources from which data in each of the focus areas-
C
traffic, event, location, and condition- can be collected. Traffic usage data is collected by sensors.
The array of sensor technologies available for traffic usage monitoring is described in Chapter 2;
this section advises on methods for selecting technologies and automating the collection of sensor
Q
data. The data collection mechanisms for the other data types are less automated; as such, their
sections focus on data collection options for different needs and budgets.
5.1.1 Traffic
Traffic information is the data item with the most collection options. It can be collected through a
variety of methods and technologies. At a low technology, temporary level, traffic data can be
collected through manual traffic counts. This is most commonly done at arterial intersections. The
majority of traffic data collection alternatives, however, rely on sensor and communications
infrastructure for more comprehensive, often real-time monitoring.
There are three points of consideration for creating an automated traffic usage data stream. The
first is selecting technologies to deploy that meet the agency’s data requirements. The second is
understanding the various ways that data is collected and distributed in the field. The third is
evaluating the different communications alternatives available for collecting data. Each is described
below.
5.1.1.1Technologies
The technologies used to monitor traffic usage data should align with the data requirements
established by the agency. Chapter 2 details the various sensor technologies available for
monitoring traffic usage data. At a high-level, these sensors can be divided into two categories:
sensors placed on the infrastructure and sensors placed within vehicles.
Within the vehicle-based category, sensors either measure the start and end points of a trip or the
vehicle’s full path. Chapter 2 details seven vectors of analysis for core traffic data requirements:
Page 40
05-DATA COLLECTION First Edition -December 2016
 Measure
 System
 Granularity
 Scope
 Sampling
 Accuracy
 Latency.
Of these seven vectors, measure, granularity, scope, sampling, and accuracy are relevant to an
agency’s selection between sensor technologies. System refers to metadata, which is required for
all technologies, and latency is more dependent on the communications infrastructure than the
sensor type. The five relevant vectors and their dependence on the technology types are described
op V1
below.
Measure
Measure refers to the data collected by each sensor. If an agency wants to collect data on traffic
C -
volumes, then the deployment of some infrastructure sensors is required. While speeds can also
y
C
be derived from infrastructure sensors, the accurate estimation of congestion measures from
infrastructure sensor speeds requires a dense spacing of sensors. If agencies want to monitor
congestion and travel time information, the deployment of some vehicle-based sensors is
C
encouraged.
Granularity
Q
Granularity refers to the temporal and spatial aggregation of the raw data. Temporally, most
sensors record measures from individual vehicle events. However, for infrastructure sensors, this
information is typically aggregated before being transmitted from the field, usually due to
communication bandwidth or processing restraints. Collecting individual vehicle data from
infrastructure sensors typically requires some hardware or software modifications to the sensor or
its controller. Spatial aggregation refers to the spacing of sensors along the roadway. Computing
accurate congestion performance measures from infrastructure sensors requires a dense spacing
of at least one sensor per kilometre. Vehicle-based sensors that monitor the end-points of trips
need to be spaced such that they inform upon travel times from well-travelled origins and
destinations.
Scope
Scope refers to the area of the network covered by measurement systems. Due to the cost of
widespread data collection, deployments are generally targeted at areas of interest; typically, these
are congested segments. Because vehicle-based sensors only require placement at key origin-
destination pairs, they can generally be deployed with a greater scope for less money than can
infrastructure sensors.
Sampling
Sampling applies to collection methods that do not sample the entire traffic stream.It primarily
applies to vehicle-based sensors, which only capture vehicles equipped with the appropriate
sampling technology, such as a Bluetooth-enabled device or a toll tag. If an agency wants to
pursue vehicle-based technologies, it needs to consider the penetration rate of a given sampling
technology in the traveling public. This will indicate the most appropriate vehicle-based technology
to use in a region, and whether there are enough devices in the driving population to generate the
Page 41
required amount of data. For most traffic applications, a sampling rate of 5% or more is adequate
to calculate robust measures.
Accuracy
Accuracy refers to the level of reliable detail in a given sensor output. It is difficult to measure the
accuracy of given technologies. The accuracy of infrastructure-based sensor data can be improved
through sensor calibration and maintenance. For vehicle-based sensors that only measure trip
endpoints, the biggest accuracy variable is the ability to filter out travel times from vehicles that
made stops or detours along their trip. For vehicle-based sensors that sample vehicles
continuously, the primary accuracy issue is locational errors in the sampling technology. For GPS,
these locational errors are minimal, but for cellular phone triangulation, they can be so large as to
make it difficult to assign a vehicle to the correct roadway.
op V1
5.1.1.2 Sources
There are four primary sources from which a performance monitoring system can obtain traffic
usage data:
C -
y


From the sensor
C
From a roadside controller
 From another system
C
 From a third-party vendor.
Q
The details of each source are described in this section. Because a single performance monitoring
system will likely have to acquire data from multiple source types, the best practice is to develop a
piece of software that lies in between the data sources and the performance monitoring system
(called “the collector” in the remainder of this chapter). This allows the collector to focus on the
complexities of interfacing with different data sources, and allows the performance monitoring
system to focus on the management, processing, and distribution of data.
The first alternative is to collect data directly from the sensor itself. Typically, device vendors
establish protocols for directly communicating with their sensors; these protocols detail the rules for
the data exchange. There is no industry standard for communicating with devices;, the collector
software must be developed to interface with each device type’s protocols. Typically, the collector
can poll devices for data at a regular interval. Alternatively, some sensors have the ability to
“push”, or send data to the collector at specified intervals.
The second alternative is to obtain data from a roadside controller that collects and aggregates
data from one or more sensors. Sensor types such as loop and radar detectors typically require a
controller cabinet to collect and process their data. The benefit of interfacing with the controller,
rather than the individual devices, is that there are fewer controller vendors than device vendors,
and controllers are already capable of collecting and standardizing data from multiple devices. This
reduces the complexity of the collector software. The drawback of communicating with the
controller is that controllers typically aggregate traffic data to at least the 20-second level, which
reduces the granularity of the raw data.
The third alternative is to collect data from a pre-existing system. For example, many agencies
have Advanced Traffic Management Systems (ATMS) that collect real-time traffic data. Where
these systems exist, leveraging their infrastructure greatly simplifies the performance monitoring
data collection process. There are a number of options for acquiring data from another system. At
the lowest level of technology, some systems may write their data to flat files, which can then be
Page 42
transferred to the performance monitoring system over the web. Other systems may generate data
feeds (for example, XML), that the performance monitoring system can consume. Another option is
to use Web Services (such as REST) to obtain the data. A more complicated option is to query the
other system’s database (if it has one) to obtain required information.
The final option is to obtain data from a third-party vendor. This is the most likely source for vehicle
data, such as that collected by GPS units or cellular telephones. In these cases, the data is
collected and acquired through third parties. The development of these partnerships is discussed
further in the Process section of this chapter.
Because traffic usage data cannot be meaningfully interpreted without associated metadata, it is
beneficial for agencies to automate the collection of metadata. There are a few alternatives for
collecting metadata. In some scenarios, an agency may deploy sensors specifically for
op V1
performance monitoring, and not attach them to any other system. In this case, the performance
monitoring effort should develop software that allows agency staff to create, maintain, and update
sensor configuration information. The user-facing side should be web-based, and let permitted staff
add sensors to the system at appropriate locations, change the properties of existing sensors, and
C -
activate and deactivate sensors. Mechanisms should be developed to send this information into
y
C
the performance monitoring system whenever it is updated. In cases where a region has another
system that maintains the configuration of the sensor network, the performance monitoring system
should be developed with the capability to incorporate this information. This can be done through a
C
low technology approach of uploading flat files to the performance monitoring system, or a high
technology approach of using web services to obtain updated configuration information.
Q
5.1.1.3Communications
Like most Intelligent Transportation Systems (ITS) devices, traffic usage sensors require
communications infrastructure and protocols to transmit data to a central hub or traffic
management centre. Similar infrastructure and protocols are needed to transfer data between
systems and between data management centres.
The infrastructure refers to the technology used to transfer data, and the protocol refers to the rules
that govern the data transmission.
Road monitoring communications architecture decisions are typically based on deployment-

specific criteria such as bandwidth and transmission speed needs, cost, scalability, and available
expertise. The main communications transmission technologies are wire line (cooper wire and
optical fibre) and wireless. A combination of all three of these technologies may be used, and each
can support several protocols for data communications. This section describes these three
technologies, but does not attempt to describe communications alternatives down to the protocol
level.
Copper wire
Traditional copper phone wire used in plain old telephone system (POTS) has very limited
bandwidth and quickly loses signal strength during transmission. Several protocols are commonly
used to transmit data over copper phone wires. Integrated Services Digital Network (ISDN) and
Digital Subscriber Line (DSL) are two protocols that work on copper phone wires to transmit larger
bandwidth digital data from road monitoring devices. Both ISDN and DSL require modem banks to
make the digital information analogue and transmit it over dedicated copper wire; these modems
can be either leased or owned by a transport agency. Internet Protocol (IP) and Ethernet protocols
are also increasingly used to transmit ITS data, and the technology to do so over copper wire is
Page 43
available. Transmission of closed circuit video signals is typically done over a coaxial cable or
copper wires for short distances.
Optical Fibre
Optical fibre is replacing copper wire for applications requiring higher bandwidth or spanning longer
distances. It is widely used in connecting field sensors and controllers with Traffic Management
Centres. Optical fibre is better suited than copper or coaxial wires to the complex ITS
communications architectures and can support many data transmission protocols. While optical
fibre is well suited for long-distance transmissions, it has historically not been cost-effective for
short distance transmissions. Fibre optic cables offer 1,000 times more bandwidth than copper
cable and can cover distances 100 times further. Fibre optic cables are also a fraction of the size
and weight of copper cable.
op V1
Optical fibre can be arranged in several types of networks to transmit data. An optical fibre ring
network is commonly used for road monitoring. In a ring network, a token containing information
requested by one or more computers on the network is passed around a ring of computers until the
requesting computer(s) have received the data. Two communications cables can send information
C -
in both directions around the ring, which creates a fault-tolerant network. A fibre optic network
y
C
requires an optical transmitter to convert an electrical signal into an optical signal, cables
containing bundles of optical fibres, amplifiers, and an optical receiver.
C
Wireless
Wireless transmission is increasingly replacing and complementing wire line technologies.
Q
Wireless transmission is much faster and easier to deploy than wire line. Current 3G and 3.5G
cellular wireless networks offer actual speeds of about 8 Megabytes per cell and offer broadband
access to the Internet. Wireless transmission is subject to interference by physical obstructions
such as buildings, hills, trees, and rain, and may also be subject to gaps in coverage.
Another technology increasingly used for transport data transmission is microwave, which
transmits data by the use of small radio wavelengths, widely used for point-to-point
communications. The high frequency of microwave gives it a very large information-carrying
capacity. Microwaves are limited to line-of-sight propagation; microwave radio relay is a technology
that transmits data between two locations on a line of sight radio path. Long daisy-chained series
of such links can form a secure data communication system without the need for copper or optical
fibre wires.
General Packet Radio Service (GPRS) is another data transmission technology used for road
monitoring data transmission. GPRS is a packet-based (no dedicated connection) mobile data
service that operates on the 2G and 3G cellular communication system’s global system for mobile
communications (GSM). The total number of frequencies available in the radio spectrum may
ultimately limit use of wireless technology for data transmission.
5.1.2 Event Usage

The data sources for events and the level of detail they provide vary greatly from site to site.
Weather and control data are the event types most likely to have automated, sensor-driven data
streams. Data regarding incidents and work zones is typically collected through communication
feeds that are populated by people, such as emergency dispatchers, traffic managers, or lane
closure management staff, but is often available in automated feeds. Event data is typically not
automated, and thus needs to be collected manually.
Page 44
Detailed event data collection methods, organized by event type, are described below.
5.1.2.1Control
Traffic control data is collected at the device’s roadside controller cabinet. In some cases, it is also
sent to a traffic control software system in a traffic management centre. For TMCs that archive
control data, a low cost alternative is to obtain it via flat files on a periodic basis. Typically,
however, control information is stored for no longer than a few minutes in the controller or at the
TMC. In these cases, the only alternative is to collect the signal data in real-time, before it is
erased from the controller or the TMC software. This can be done through the development of a
software interface with the controller or the traffic control software system. These interfaces are
costly to develop and typically only function to collect data from the same vendor, due to the lack of
standardization in control software systems. New technological developments are likely to facilitate
op V1
the real-time acquisition of continuous control data for future deployments. The next generation of
traffic signal controllers is being developed with improved mechanisms for storing and distributing
control data.
Additionally, recent research has produced off-the-shelf devices that can be plugged into certain
C -
y
types of controllers to collect and transmit data.
C
The low technology, low cost alternative to automated acquisition is to manually obtain control
estimates from timing plans, typically available from the TMC. While feasible for time-of-day plans,
C
this approach does not work well for signals that operate on traffic responsive control plans.
Q
5.1.2.2Incidents
Incident data is frequently available through an automated stream in near real-time. Most
emergency response agencies use Computer Aided Dispatch (CAD) systems to respond to
incidents; these systems have feeds that a transport agency can connect with. Additionally, many
TMCS use real-time close circuit video feeds to confirm incidents, then enter the incidents into a
log. This log is a potential source for incident data. Finally, private services may collect incident
data at a high level of specificity from various sources, including video, mobile units, and
emergency communication frequencies.
The above-described sources are real-time; for post-facto incident acquisition, some transport
agencies compile safety data on a quarterly or annual basis. Unless the data is needed in real-
time, this source offers the benefit of being quality-controlled and pre-processed.
5.1.2.3Weather
The low cost option for acquiring weather data is to obtain it from national meteorology centres or
other agencies that monitor weather conditions. Often, historical weather data is published online
at hourly or daily aggregations. Some agencies also offer weather information in real-time through
various feeds.
Agencies also have the option to directly install Environmental Sensor Stations (ESS) at roadway
locations; these systems can provide real time weather information at locations selected by the
agency. The data can then be sent to the agency via the communication methods outlined in the
Traffic Usage section of this chapter.
Page 45
5.1.2.4Work Zones
The business process of lane closure management often produces data that can be useful to a
traffic monitoring system. Many agencies have lane closure systems that serve as on-going
communication between the contractors and agency. These systems store detailed information on
lane closure locations, durations, and types of work. Another alternative is to obtain work zone
information from traveller information sources, such as Variable Message Sign logs that details
work zone impacts and detours.
5.1.2.5Special Events
Data about special eventsis typically collated by individuals rather than systems. Historic event
data can be collected by manually reviewing calendars for major event venues in the area.Another
op V1
alternative is to obtain event data from TMCs, many of which collect event logs to know when and
where to activate event-based signal timing plans.
5.1.3 Condition
C -
There are two types of condition information: traffic monitoring data and pavement condition data.
y
C
Traffic monitoring data is collected by sensors; therefore, collection methodologies are the same as
those described in the Traffic Usage section. The framework for collecting pavement condition
data, however, differs because it is typically collected no more than once per year, and it is
C
collected by people rather than sensors., this section advises on collection methods for the
required data types outlined in Chapter 3.
Q
5.1.3.1Visual Surveys
In some circumstances, on-the-ground visual surveys are the only way to collect quality pavement
condition data. Surveyors can visually assess and record pavement qualities and attributes while in
a moving vehicle, while stationary, or through high-resolution digital images. Surveys can collect
information on rutting, cracking, depression, patching, potholes, stripping, flushing, and bleeding.
However, visual surveys are time-consuming and can also risk surveyor safety. Additionally,
lighting, noise, and weather can all interfere with the manual collection of quality condition data.
5.1.3.2Condition Sensors
Survey vehicles equipped with various technologies are regularly used to collect several types of
condition data. Survey vehicles can be equipped with modules for roughness and rutting as well as
for geometry, photo, video, skid resistance, and GPS. This section describes available pavement
condition sensors.
Laser Profilometer
A profilometer is a measuring instrument used on a survey vehicle driving at highways speeds to
measure pavement surface roughness; the data collected is used to calculate the IRI. Survey
vehicles are instrumented with either contact or non-contact profilometers that enable the collection
of longitudinal roughness and transverse rutting data. A typical profilometer can measure small
vertical features ranging in height from tennanometres to one millimetre. Some profilometers also
measure cross slope, curvature, longitudinal gradient, and cracks, although instrument-collected
stress and crack data have a low level of accuracy. Most profilometers use GPS technology to
record position, and some systems include ground-penetrating radar that can record asphalt layer
thickness.
Page 46
A camera can be installed on a profilometer to collect video footage for use in subsequent
evaluation. A video-logging system consists of digital images displayed, compressed, and stored
digitally in real time. The images typically include a 120-degree composite view of the roadway and
right-of-way of the road. The images can be collected and stored at operator-selected intervals.
The operator can also choose to manually add data to the video log and/or synchronize the
footage with the profilometer distress and sensor data. The digital image files and pavement
condition data can be uploaded to a web-based server and accessed by multiple users.
Laser Crack Measurement System

The Laser Crack Measurement System (LCMS) allows the automatic detection of cracks and the
evaluation of rutting, macro-texture and other road surface features. The LCMS uses high-speed
cameras, advanced optics and laser line projectors to acquire both 2D images and high-resolution
op V1
3D profiles of the road. The collected data is analysed with software that detects and analyses
cracks, lane markings, ruts, macro-texture, curbs and drop-offs, raveling, and potholes. Cracks are
classified as transverse, longitudinal, or alligator, and their severity is evaluated. The LCMS can
operate in full daylight or in night time conditions with survey speeds of up to 100 km/hr.
C -
y
Laser Rut Measurement Systems
C
Laser Rut Measurement Systems (LRMS) measure the transverse (across the lane) profile of the
road using two scanning lasers. The data are used to detect and characterize pavement rutting
C
under a simulated straight edge. The system uses custom optics and a high-powered pulsed laser
projector, which allow the system to operate during daylight or at night. Software developed to
Q
work with the LRMS can be used to verify the calibration of the system, record 3D data, and extract
marking line position. Image processing algorithms detect line distortion based on the shape of the
pavement in order to determine the depth and width of ruts. The transverse profile data is
collected, analysed by software, and stored in real time.
Ultrasonic Sensors
An ultrasonic sensor can be used to measure the transverse profile of the road using high quality
ultrasonic sensors. This is one of the more cost-effective and efficient ways of collecting rut depth
data across an entire traffic lane. Optical technology and high-power pulsed laser line projectors
allow the system to operate in daylight ornight time conditions.
Bump Integrator
The bump integrator measures the riding quality of a road surface by measuring the relative
displacement of a vehicle’s suspension in relation to its body. It can measure both surface
irregularity and riding quality at normal road speeds. The raw roughness data can then be
converted into a calibrated roughness index such as IRI.
Falling Weight Deflectometer

Monitoring and diagnosis of subsurface layer properties at discrete locations can be accomplished
via the periodic use of Falling Weight Deflectometer (FWD) non-destructive testing. FWD testing
drops an impact load to the pavement that simulates a moving, heavily-loaded truck axle. It collects
data on the maximum deflection and deflection basin and uses it to establish the load carrying
capacity of the pavement; this indicates its ability to carry heavy loads in the future. In addition,
FWD deflection data can be used to design structural overlays and make reconstruction decisions.
Page 47
Rolling Weight Deflectometer

A Rolling Weight Deflectometer (RWD) is a specially designed tractor-trailer that measures
pavement deflections while traveling at highway speeds, using laser measuring devices mounted
under the trailer. At this time, RWD is costly.
5.1.4 Location
Location information changes infrequently but requires significant resources to effectively collect.
Road location data collection methods should be chosen based on the level of positional and
measurement accuracy needed. For many types of location data, automated collection methods
cannot produce the level of accuracy needed for traffic monitoring.
Traditionally, location data collection has been done manually in the field, using surveyors. This
op V1
process is labour- and cost-intensive, takes months or years to complete, and can be dangerous
because it places surveyors near traffic. However, this remains the most accurate data collection
method. An alternative is to use survey vehicles with equipment to measure road geometrics and
other location information. The use of digital surveys to obtain location data is not only more cost-
C -
efficient than manual methods, but it allows for the use of geo-referencing, which then enables
y
C
location data to be integrated with usage and condition data. Current research focused on
measuring road geometry and attributes using autonomous (machine-controlled) vehicles may
further improve the efficiency of location data collection.
C
5.1.4.1Global Position System
Q
Global Positioning System (GPS) technology is typically used in conjunction with another method
to collect location data; it collects spatial information, but not inventory information. Its ability to
collect needed data must be considered in relation to its sensitivity, satellite service availability
(coverage), and positioning accuracy. GPS availability is generally the highest in urban areas.
Currently, the accuracy of the most sophisticated GPS technologies ranges from one to two metres
(horizontal and vertical), 99% of the time.
5.1.4.2Digital Stereo Photogrammetry

Stereo photogrammetry estimates the three-dimensional coordinates of points on an object using
measurements made from aerial photographic images taken from different positions. Digital stereo
photogrammetry techniques use software to determine accurate geo-referenced measurements of
road width, as well as the dimensions of other objects on and adjacent to the roadway. Data on
roadway assets and features are extracted from geo-referenced digital images, but are limited by
the perspectives of the aerial photographs. Assets recorded and geo-referenced from the images
include bridges, signs, streetlights, barrier systems, pavement markings, and rumble strips. Data
quality depends on the resolution and spacing of the images, the accuracy of the GPS coordinates,
the system calibration, and the institution of a rigorous quality control program.
5.1.4.3Light Detection and Ranging (LiDAR)

Light Detection and Ranging (LiDAR) technology can measure the distance to and properties of a
target by illuminating it with light, often using pulses from a laser. Road topology and attribute data
can be derived from LiDAR technology fitted to aircraft or satellites. LiDAR data, in the form of
point clouds, are acquired, processed, and delivered in a digital format, which allows for automated
processing. LiDAR data can be used for classification (ground, vegetation, buildings, etc.),
vectorization, measurement of features, and 3D modelling and texturing.
Page 48
5.1.4.4 Mobile-mapping systems

A mobile-mapping system generally refers to a fully integrated 3D laser scanning and digital
imagery system that is mounted on a vehicle. The system uses redundant technologies to achieve
accurate position information, even when satellite information may be blocked. With a mobile-
mapping system, data is collected at normal roadways speeds, data collection can be completed
with one pass, and the surveyor remains in the vehicle.
Spherical 360 degree images registered with high-accuracy LiDAR point clouds allow visual
inspection and geospatial positioning of surfaces, areas and features and accurate measurements
of distances. Mobile systems can attain accuracies of around ten to 15 centimetres.
A mobile system typically includes data collection software, data processing software, and
op V1
analysis/extraction software. The system can produce geo-referenced shape files, which can be
exported to GIS for integration with other data sets, as well as geo-referenced photo and video
logs for future use.
5.2 Procurement
C -
y
C
An agency must decide how to execute the collection of data using the methods described in this
chapter, either using its own equipment and staff, a contractor, or a partner. An agency is likely to
C
use multiple types of arrangements in order to procure the full range of location, condition, and
traffic data that it requires. This section describes the considerations for the Abu Dhabi DMAT
when making these decisions. The majority of the discussion focuses on procuring traffic usage
Q
data, since this is the most complex data source.
5.2.1 Usage Data Procurement

Background
Historically, transport agencies have paid contractors to build, install, and maintain sensor
networks, but have retained ownership of the sensors themselves. Early providers negotiated
deals with infrastructure owners to franchise and manage large sensor networks in major urban
areas, mostly with payments from agencies at various levels of government. These early firms
established the market for this type of public-private traffic data partnership.
Changes in data acquisition models, however, have been driven by recent advances in mobile
technologies, which have led to the ability to crowd-source vehicle probe data on a large scale.
Crowd sourcing is the idea of collecting measurements from individuals as a way of understanding
the whole system. The increasing market penetration of smartphones has radically altered the
cost equation for data aggregators, who no longer have to build, install, and maintain sensor
networks. In this framework, customers, instead of contractors or agencies, pay for the GPS or
smartphone “sensors”. These customers are then enticed by intermediaries (device vendors and
mobile application developers) to purchase software that provides them a service, but also reports
customer traffic data back in real-time, leveraging the hardware that the customer owns.
This has led to a flood of new entrants to the traffic crowd-sourcing industry. Current vendors of
probe data products now tend to fall into two categories: major consumer technology firms, and
focused traffic data intermediaries; only the latter would typically sell data products to a public
agency.
Page 49
Outsourcing Data Collection

Outsourcing opportunities vary by sensor type. As explained above, the collection of vehicle-based
sensor data from GPS units or smartphones typically requires outsourcing or partnerships, since
these devices are consumer-owned with data collected by third-parties. The only alternative to
outsourcing is for an agency to develop its own crowd-sourcing mobile application that provides
some service (such as traveller information) in return for traffic data from the user. For other
sources, agencies typically have an option of whether to outsource or develop and maintain the
data collection infrastructure in-house.
There are a number of advantages to outsourcing data collection. It lowers software and hardware
costs for the agency, and it reduces staff time needed to install and maintain the data collection
infrastructure. It lets the agency leverage the data quality, processing, and fusion mechanisms
op V1
researched and developed by the vendor. It also generally provides enhanced network coverage
over what an agency could reasonably maintain on its own.
These benefits come at the cost of agency control over data collection, processing, and
distribution. Data quality from outside sources is difficult to validate. Agencies can mitigate this by
C -
y
purchasing data incrementally and monitoring deployments closely to ensure that the data meets
C
requirements. Additionally, because processing and fusion algorithms and specific data source
types are typically proprietary, agencies have little insight into how the data is derived. Agencies
C
may also be bound by contractual restrictions from supplying data to partners or using it for certain
purposes. Procured services are subject to what a provider is willing or able to offer and are
subject to change. The road network coverage is limited to the area that the provider serves, with
Q
the agency dependent on the provider to expand or modify coverage. Finally, the collection of data
is subject to provider business and technological issues; for example, if the provider goes out of
business, the data stream will likely end.
Owning Sensors
When an agency chooses to own its sensors rather than procure services, it gains more control
over its data stream but increases the internal resources required to sustain performance
monitoring. In this case, the agency is responsible for deploying the equipment, ensuring that
equipment is continuously providing quality data, and performing maintenance. Agencies can
conductthis maintenance themselves or pay a contractor to do it. When an agency owns its own
sensors, it becomes responsible for product management, repair, and the maintenance of the
communication network. This means that the agency needs to maintain staff with the required
knowledge to perform these tasks, no small effort.
Lifecycle analysis
The decision on whether to deploy collection infrastructure or contract with private providers for
data can be informed through life-cycle benefit/cost analysis. An agency should first clearly define
the user needs, data quality requirements, and geographic scope of the deployment. Based on
these requirements, it can then identify reasonable sets of technologies for the deployment. After
surveying the state and comprehensiveness of existing detection, it can develop a detailed set of
alternatives that can include: (1) fully replacing existing infrastructure with different sensor
technologies; (2) fully replacing existing infrastructure with the same sensor technology; (3)
supplementing existing infrastructure with additional technology deployments; or (4) contracting
with a private service to buy required data.
For the set of infrastructure alternatives, the following parameters should be identified:
Page 50
 Capital cost by major component

 Estimated life of major components
 Estimated annual operations and maintenance costs
 Inflation and discount rates to be applied.
From this, the agency can calculate the annualized cost in current Dirhams for the life cycle of
each infrastructure alternative, optionally applying weighting factors to account for differences in
data quality and reliability. These estimates can then be compared with the annual cost of private
data purchase for similar data sets.
5.2.1.1Location Data
Topology data is a set of standard geo-referenced data files that describe the underlying topology
op V1
of the road network. These are typically purchased from commercial GIS companies, although
many transport agencies maintain their own layers. There are instances where commercial
topology data is insufficient for transport agency uses. For example, most GIS data sets do not
describe the lane configuration on a link as it approaches an intersection, whether there are
C -
dedicated turn lanes, or which lanes can turn onto which links. Increasingly, open-source mapping
y
ventures are capable of providing comprehensive roadway networks, but often lack required details
C
such as the number of lanes at a given location. Agencies are often able to procure the base
roadway networks, but may have to augment them with the details required for performance
C
monitoring. These details can be acquired through the location data collection methods described
in this chapter.
Q
5.2.1.2 Condition Data

Although agencies are increasingly outsourcing pavement data collection, most are still collected
by the agency. Pavement distress and smoothness data are the most frequently outsourced, for
cost-effectiveness reasons as well as limits to agency data collection capabilities.
Page 51
6 DATAPROCESSING
6.1 Approach
Data Processing describes the process of storing data, identifying and correcting problems with it,
then performing computations and aggregations so that it can be easily leveraged to inform on
system performance. This chapter contains three sections. The Technologies section provides
recommendations on data storage considerations. The Storage section details the high-level tables
needed to store usage, condition, and location information in a digital database. Finally, the
Processing section describes the computational steps needed to turn raw data into usable,
accurate information. All of the systems described here must be implemented in harmony with the
op V1
DMAT ITS Architecture.
6.2 Technologies
C -
y
6.2.1 Using Databases
C
Typically, raw location, condition, and usage data are all stored in a digital database. Utilizing a
database to store road-monitoring data supports both real-time and long-term uses of data,
C
enables communication between different data systems, and allows data to be used by multiple
users through different interfaces.
Q
In practice, these database systems have large and complex physical data models, with hundreds
of relational tables.Each table has fields (columns) and records (rows). Each table requires a
primary key, which is a field or set of fields that uniquely identify each record. The primary key can
be composed of real data, or it can simply be a meaningless identifier attached to the record when
it is written into the database. Tables relate to one another through common fields. Specifically,
relationships are designated through “foreign keys” which are fields in tables that contain the same
values as the primary key of another table. Data is accessed through queries, which retrieve
information from specified tables and fields that meets specified conditions. The most commonly
used querying language is the Structured Query Language (SQL).
6.2.2 Data Warehousing

For road performance monitoring with continuous sensor data, it is recommended to use a data
warehouse to store data, instead of or in addition to an operational database. Operational
databases are optimized to record data, whereas data warehouses are optimized to analyse data.
In a data warehouse, data is stored in the database in a non-normalized format, at multiple levels
of aggregation and imputation., the data warehouse can relate data across time, sources, and
categories, enabling users to obtain integrated information about the road system. This framework
also makes it feasible to compile data from multiple sources, and then share usable data with the
many entities that need access to it. For example, the data warehouse can host corridor-level
information from multiple local transport agencies as well as private data providers, then provide
historical information on vehicle kilometres travelled (VKT) growth on that corridor to a regional
planning agency. Collaboration in the creation of a data warehouse for all traffic data can avoid the
need to maintain multiple small databases. Finally, data warehouses utilize sets of processing
routines that run at certain time points to create aggregate summary data. This read-only summary
data can be queried and analysed far more efficiently than an application database filled with raw
data.
Page 52
06-DATA PROCESSING First Edition -December 2016
6.2.3 Hosting
Historically, a data warehouse had to be hosted on-site by a transport agency and was accessible
only to that agency. Today, a data warehouse, both hardware and software, can be designed and
architected for secure multi-user hosting. Whether to host a data warehouse on-site or else use a
decentralized “cloud”-based data warehouse is a business decision. There are many large private
providers of cloud-based data warehousing; their ability to customize for transport data needs is
variable.
The advantages of a cloud-based data warehouse are its low initial investment costs, its elasticity
in terms of volume, usage, and price, and the fact that it reduces strain on internal IT resources.
There are some disadvantages to cloud-based models: they are newer technologies and less
mature. However, cloud-based products are quickly improving, so these challenges are becoming
op V1
minimal.
A cloud-based data warehouse can be hosted in a private cloud, public cloud, or a hybrid, in which
some applications and databases live within the agency while development and front-end
applications live in the public cloud. A private data warehouse can improve performance and
C -
y
increase privacy.
C
6.3 Storage
C
Data items collected by the system need to be stored in an organized manner that facilitates
processing and queries for information. There are two broad categories of information that need to
Q
be stored in the database: data and metadata. Data refers to the performance monitoring
information, and metadata refers to the ancillary information needed to make sense of the data
(such as sensor locations).
The requirements for data storage are as follows:
1. All raw sensor data should be stored permanently in the data warehouse when privacy
issues do not allow it.
2. If the format of the raw data raises privacy concerns (for example, if it uniquely identifies
vehicles or individuals), data should be stored at the lowest level of granularity possible that
resolves the conflict.
3. Raw and processed data should be stored in parallel; raw data should never be replaced.
This ensures that, as computation methods and technologies evolve, information can
always be re-computed from the raw data set.
4. Data must be stored with the ID of the device that collected it.
The requirements for metadata storage are as follows:
1. Metadata must be stored in separate tables from the actual data.

2. Metadata tables must be updated any time there is a change to the detection network.
3. Every piece of monitoring equipment must be assigned a unique ID. If the device is ever
deactivated, its ID must never be re-used.
4. The storage regime must be designed to allow the tracking of detection changes over time.
This allows changes in performance to be linked back to changes in the network.
5. The metadata tables must allow a device to be marked as inactive, with inactive devices
excluded from computations. This enables agencies to exclude a device’s data from being
stored if the device breaks or if there is construction at the location.
Page 53
This following section provides guidance for designing a data warehouse to store the data items
recommended for performance monitoring.
6.3.1 Location Data

Location data needs to be stored in a database so that it can be used to map monitored roadways
and make geographical sense of the traffic data. For transport networks, location information
consists of four core data sets:
 Spatial Reference
 Linear Reference
 Topology
 Attributes.
op V1
The Locations chapter describes the requirement to link a roadway’s spatial location (latitude and
longitude) with the linear referencing system used by the agency to place equipment and attributes
(kilometre-posts). The data model for roadway spatial reference information typically consists of
shapes, made up of a geometric primitive and a spatial location. Linear referencing information
C -
typically consists of the agency’s roadwaykilometre-posts and offsets. One approach for storing
y
C
these two types of location information in a database is to create a table with records defined for
every roadway by the road number, direction of travel, starting latitude and longitude,
C
startingkilometre-posts, and length. Records need to be defined such that they fully cover every
roadway from start to finish at a consistent and fine-grained granularity. This allows the records to
fully capture the curvature of the roadway, as well as allowingkilometre-posts and latitude and
Q
longitude to be approximated from one another through a developed algorithm. If the

agency’skilometre-posts do not represent actual, absolute distances along a roadway, an
internalkilometre-post system needs to be developed to represent each point’s location relative to
the roadway’s starting point (with distance typically computed along the roadwaycentre-line).
These internalkilometre-posts then need to be stored in a field to match the corresponding
agencykilometre-post. This set-up allows internalkilometre-post (required for spatial performance
measure aggregations) to be calculated for any agencykilometre-post input into the system (for
example, to locate sensors or incidents).
Topological networks are connected sets of roadway segments that can be logically traversed. For
database storage, it is recommended that agencies implement a table that defines a set of key
routes, across which performance measures will be calculated and stored to enable simple and
frequent access. Each route can be defined by a series of contiguous roadway segments, each
containing a road number, direction, and starting and endingkilometre-post.
The database should be designed to store all attributes that the agency collects along its roadway
segments. Attributes can be stored in the same table as the spatial and linear reference
information if the agency desires. However, this means defining attributes for homogeneous
sections of roadway multiple times over. If the agency prefers, attributes can be defined for each
homogeneous section of roadway in a separate table. These sections of roadway must be defined
by the road number, direction, and starting and endingkilometre-post.
The database should also have the capability to store roadway features (such as signs and
bridges), that are located at specific points along a roadway. It is recommended that these be
stored in a separate table from the attributes, to distinguish between characteristics of a roadway
segment (such as number or lanes and lane width) and infrastructure located at a specific point.
Features should be stored by their type, road number, direction, and kilometre-post.
Page 54
6.3.2 Condition Data

There are two types of condition data that need to be stored: traffic monitoring data and road
condition data. Each of these data types also requires the storage of metadata to spatially
reference sensors and define segments along which road condition is monitored.
Like the traffic usage data, the traffic monitoring data needs to be stored in real-time in a raw data
table, then quality controlled and aggregated in separate tables. Here, traffic monitoring data is
defined as vehicle classification and WIM sensor data. While measures such as AADT and VKT
are also important inputs for road condition monitoring, they can be gathered from information
stored in the traffic usage data tables. The database must also store relevant traffic monitoring
metadata. Metadata for condition traffic monitoring sensors is similar to that for traffic usage
sensors, in that it can be categorized into lane-by-lane sensors and multi-lane stations. To limit the
op V1
number of specialized sensors deployed, Chapter 3 recommends monitoring categories of
roadway such as rural, urban, residential, and industrial, rather than monitoring every single
roadway. In the metadata, sensors and stations can be assigned groups to facilitate the
aggregation of condition traffic monitoring data within each category. Stations can also be assigned
C -
to condition segments, which are used by the agency to report pavement survey and treatment
y
history information.
C
Road condition data has a different storage framework; it is not collected in a real-time feed, but
C
rather is entered into the database typically no more frequently than once per year. In storing road
condition information, agencies need to create a road condition metadata table that defines the
segments over which conditions are monitored and treatments are applied. The condition
Q
segments should be defined by a road number, direction, starting and endingkilometre-post,

functional class, and group. All road condition records can then be defined by the ID of the
condition segment. The exact data stored for condition segments can be flexible to agency needs.
At a minimum, the database should be capable of storing historical surveys on road condition
measures, the locations and types of observed condition defects, and historical treatments and
costs. It is also recommended that the database have the ability to store images and videos of field
conditions; most sophisticated, commercially available databases have this capability.
6.3.3 Traffic Usage Data

There are two relevant aspects to storing traffic usage data: storing real-time traffic data and
storing sensor metadata. Typically, sensor data is transmitted in real-time into a database using
data feeds. Metadata is usually acquired once at the beginning of system development, then
updated only occasionally as the deployed network expands and changes.
6.3.3.1Real-time Data
The Usage Data chapter of this document recommends collecting real-time traffic data in two
categories: point and link set. Because the raw data for these two types varies in form and
sampling frequency, each requires its own raw data table. Some point data sources are specialized
(such as vehicle classification or WIM sensors); these sensors require a separate raw database
table that is addressed in the condition section.
After the raw data is placed in the raw database tables, it should never be modified; the results of
all computations, aggregations, and fusions should be stored in separate database tables. The
processing section of this chapter describes five data processing steps for traffic usage data: (1)
computation; (2) quality control; (3) imputation; (4) aggregation; and (5) fusion. The results of each
of these steps must be stored in separate database tables. As the data is processed, it is important
Page 55
to also store ancillary information on the data quality of each record, such as the percentage of
data that is imputed for aggregated point data and the number of samples collected per time period
for link set data.
6.3.3.2Metadata
Traffic usage metadata describes relevant aspects of the detection network like sensor locations
and types. The correct storage structure for metadata depends on the source of the data being
collected.
For point data sources, which collect data in individual lanes, it is recommended that metadata be
organized into two separate entities: (1) a sensor, which is lane-specific; and (2) a station, which
groups sensors monitoring different lanes at the same location. The grouping of sensors into a
op V1
station makes it easier to spatially aggregate data across all lanes at a location, as well as along
the roadway. For sensors, the database needs to store the ID of its associated station, the device
type, and the type of lane and lane number that it monitors. For stations, the database must store
its location information in the form of road number, direction, kilometre-post, and latitude and
longitude, the number of lanes and type of lane that it monitors, the length of roadway it monitors,
C -
y
the type of data that it can collect, and whether it is active. The length of the roadway monitored by
C
each station is a required input for spatially aggregating point sensor data along a roadway
segment, to compute measures like VKT and delay. It is recommended that each station be
C
assigned to monitor the segment of the roadway halfway up to the neighboring upstream and
downstream sensors.
Q
In cases where sensors are sparsely spaced, the length of an individual station should be limited to
avoid applying its data to an overly-long stretch of roadway, along which traffic conditions are likely
to vary significantly. It is recommended that the agency limit the length of an individual station to no
more than 4kilometres in the upstream and downstream directions (an 8kilometre maximum
length). The station database table must also include fields to mark the version number and active
data for each record; this allows a single station to have multiple records that show how its
configuration (for example, the number of lanes it monitors) has changed over time. To assist in
data fusion, the station metadata table can also store information that identifies nearby vehicle
identification sensors and links.
The data collected by link set sources is typically not lane-specific; it is used only to match vehicle
records at upstream and downstream sensors to monitor travel times along a roadway section. As
such, metadata for link set sensors resembles that of stations, but does not require storing the lane
type, number of lanes, or types of data monitored. Beyond the sensor metadata, link set data
requires a table to define the links monitored by the link set network. These links are made up of
any origin-destination pair equipped with link set sensors. The links are defined by attributes such
as roadway, direction, startingkilometre-post, starting sensor ID, endingkilometre-post, ending
sensor ID, and length.
6.3.3.3 Event Data

The data warehouse should have separate tables for each category of events that the agency is to
collect data on. Each table should be designed according to the local availability of data. For most
event types, some data items (such as start time and location) are required for storage in the
database, but other data items (such as wind speed for weather) may be designated as optional if
they are not consistently reported by the event data source. In designing the event usage data
tables, it is also important to consider what, if any, additional metadata is required to use the
information to interpret monitored traffic data.
Page 56
Incidents and work zones are located on a particular roadway, direction, andkilometre-post, and
this information must be stored for each event record. Because this location information can be
used to link the event with nearby sensors, no additional metadata is required.
There are a number of possible formats and variables that can represent traffic control information;
the data available depends on the type of intersection controller, the timing used, and the data
collection framework. For example, in some situations, it may be possible to collect real-time
information from the field; in other cases, it will only be possible to store static green time estimates
derived from signal timing plans. For this reason, there are no specific storage requirements for
traffic control information; tables should be structured according to the local availability of data. A
traffic control metadata table is required to identify each signalized intersection, the turning
movements that it serves, and the phase numbers that correspond to each turning movement. This
table can also identify the ID of sensors monitoring each phase, if present.
op V1
Weather conditions are often measured by specialized sensors, frequently located at airports. In
some cases, these sensors may be located on the roadside, but the measured conditions still span
a wider geographical area. The real-time weather data must be stored in a table by the sensor ID
and timestamp.
C -
y
C
Weather data requires a metadata table to store the locational information (latitude and longitude,
as well as road number, direction, andkilometre-post if on the roadside). The agency can optionally
C
store information linking a weather station with a nearby route, or with roadway segments likely to
be impacted by the conditions monitored at the station.
Q
Special events generally occur at specific venues. The database should be capable of storing a
special event’s venue, date, start time, duration, and attendance. It should also include a metadata
table to define each special event venue and location (by latitude and longitude). Similar to
weather, the agency can optionally store information linking a specific venue with defined routes or
roadway segments likely to be impacted by its event traffic.
6.4 Processing
The raw data requires significant processing prior to being turned into usable and informative
performance measures. Using automated monitoring systems to perform data processing saves
agencies resources and increases the speed at which raw data can be turned into actionable
information. This section presents a five-step framework for processing raw data. The first step-
computation- calculates required values from the raw data. The second step- quality control-
diagnoses errors in the data. The third step- imputation- replaces missing or bad data with
estimated values. The fourth step- aggregation- summarizes data into useful units for analysis. The
final step-fusion- leverages multiple data sources to increase the quantity and quality of provided
information.
The steps outlined above apply only to the real-time data acquired by the system. It is assumed
that other forms of information, such as location and road conditions, are processed and quality-
controlled prior to being stored in the system.
6.4.1 Computation
In this step, the system performs calculations on the raw data to compute needed information. It is
recommended that, for infrastructure-based sensors, raw data first be aggregated up to an atomic
temporal aggregation (e.g. five-minutes) prior to computation, quality control, and imputation. For
infrastructure-based sensors, this processing step computes speed from count and presence data.
Page 57
For link set data sources, this step computes travel times. Typically, no computation is needed for
the raw road condition traffic monitoring data. The outputs of these computations are stored in a
separate table.
6.4.1.1Point Data
For point sensors that report count and presence data, speed needs to be computed from the raw
data. The best practice for performing this computation is to use an estimate of the average vehicle
length at a sensor to compute the missing value. The governing equation for this computation is:
Here, (1/g) is the average effective vehicle length, which is the sum of the actual vehicle length and
op V1
the width of the detection zone.
Estimates of the average vehicle length can be obtained from a number of sources, including field
surveys, nearby vehicle classification sensors, or nearby sensors that directly observe counts,
C -
presence, and speed. To increase the accuracy of the computation, the average vehicle length
y
C
assumption can be lane- and location-specific, and can also be calibrated by time of day and day
of week. In this case, a database table is needed to store the vehicle length assumption for each
sensor, day of week, and time of day. The values in this table can be updated as frequently as
C
desired. As stated above, it is recommended that this computation be performed on the raw data
aggregated up to the five-minute level, to ensure that there are enough vehicle samples to
Q
generate an accurate estimate.
6.4.1.2 Link Set Data

For link set data, the primary computational steps are to match vehicles between sensors and
compute travel times for each sensor pair. The calculations involved are relatively trivial; when a
vehicle ID is seen at two sensors, the starting and ending sensors need to be matched with the
applicable link, as stored in a metadata table. Then, the travel time can be computed and stored for
that link and timestamp. Best practice is to store the timestamp of when the vehicle passed the first
reader.
6.4.2 Quality Control

Data quality is one of the most important factors that dictate the usability of the data. In this
processing step, the system diagnoses bad data and removes it from further processing. It is
important to note that this step does not remove data points from the raw database tables, but
rather populates new tables with validated data and summary statistics on the percentage of data
removed. It is a required step for all of the raw data sets.
6.4.2.1Point Data
Point sensors can experience a wide range of errors. Common errors include hardware problems
with the sensor or its roadside control cabinet and failures in the chain of communications
infrastructure that brings a sensor’s data to a centralized location. Each of these errors causes
unique, diagnosable signs of distress in the data transmitted by the sensor. There are two broad
categories of methods for identifying bad sensors based on their data: (1) real-time data quality
measurement; and (2) post-hoc data quality measurement. While this section briefly describes
real-time data quality measurement, post-hoc methods are recommended for performance
monitoring data processing.
Page 58
Real-time data quality methods run algorithms on each sample transmitted by an individual sensor.
Sensors typically send data samples every 20- to 30-seconds. Because traffic conditions can vary
so widely from one 30-second period to another, however, it is difficult to assign “acceptable”
regions within which a sensor can be considered good and outside of which it can be diagnosed as
broken. Additionally, depending on the time of day, day of week, and other factors like the facility
type, geographical location, and lane number, sensors record different levels of traffic. This method
requires the calibration of diagnostics for each individual sensor, making it impractical for large
scale deployments.
The best practice is to use post-hoc data quality measurement methods. These methods make
diagnostic determinations based on the time series of a sensor’s data over an entire day. This
methodology works because typically, once sensors go bad, they stay bad until the problem can be
resolved. They are usually bad for multiple days at a time. Under this suggested framework, the
op V1
following four symptomatic errors can be detected to mark a sensor as bad during a given day:
1. Too many zero presence and count values reported;

2. Too many non-zero presence and zero count values reported;
3.
C -
Too many very high presence samples reported; or
y
4.
C
Too many constant presence and count values reported.
The agency is advised to only run quality control algorithms on the data collected during hours
C
when the facility carries significant traffic volumes (for example, between 5:00 AM and 10:00 PM).
Otherwise, low volume samples collected during the early morning hours could cause a false
positive quality control diagnosis, particularly under the first category listed above. Additionally, the
Q
agency should establish threshold levels for each error symptom that reflect the unique traffic
patterns in its regions. For instance, the thresholds should be set differently for sensors that are in
urban versus rural areas. Working sensors in rural areas may submit a high number of data
samples that fall into the first category simply because there is little traffic. Alternatively, working
sensors on congested urban facilities may submit many samples that fall into the third category
due to queuing.
6.4.2.2 Link Set Data

For link set data, the quality control step identifies and removes outlier travel times measured from
vehicles that likely made stop or detours between the origin and destination readers. The simplest
approach to reducing the impact of outliers on the magnitude of reported performance measures is
to store only the median travel time measured over a given time period (for example, a fifteen-
minute period), then use this summary metric in performance analysis. The median travel time is
less impacted by outliers than the average travel time. More sophisticated algorithms have also
been developed; these typically remove travel times that differ by more than that a set percentage
from the average travel time of the previous period, or smooth measured travel times with historical
average data. In general, these practical methods are relatively ad hoc and lack the statistical rigor
of the point sensor quality control methods described in the previous section.
6.4.2.3Condition Traffic Monitoring Data

Quality control for condition data is important because of the key role condition data plays in
making expensive infrastructure investment decisions. In the field, quality control is achieved by
calibrating and verifying equipment prior to data collection and continually validating data collected
on known control segments.
Page 59
From a data management perspective, the system must implement software routines that check
the reasonableness, completeness, and consistency of condition traffic monitoring data, which
typically consists of volumes, vehicle classifications, and weights. To quality control these
measures, the system can perform daily checks to ensure that they fall within expected ranges.
Example checks include:
 Maximum number of vehicles per hour per lane

 Maximum percentage of vehicles by class by day
 Minimum number of axles by class
 Axle weight range check
 Axle spacing range check.
6.4.3 Imputation
op V1
In this step, missing data or data that has been diagnosed as bad is imputed, to fill in any holes in
the database prior to aggregation. This step is required for point data. Point data is unique from link
set data in that, when everything is functioning, it samples the entire vehicle stream. It is the only
C -
data source that can be used to generate volume-based performance measures, such as VKT and
y
C
vehicle-hours of delay. However, accurately computing these volume-based performance
measures requires a complete data set, without any temporal or geographic holes. Conversely,
there is no need to impute data for link set sensors; the majority of the time, if there are no travel
C
times reported between reader pairs, it means that no equipped vehicles passed between the
sensors during the time period. This missing data sample will not impact the ability of the link set
Q
data to inform on historical travel time and travel time reliability trends. Similarly, there is little value
in imputing vehicle classification or weight information; for its uses, it is much more important to
have an accurate dataset than a complete dataset.
Missing raw data points for a given point sensor have two causes: (1) the data warehouse does not
receive data for that sensor for a particular time period; or (2) the data was received, but was
identified as bad in the quality control step. In either case, it is important to use imputation methods
to fill in the missing data. It is recommended that imputation be performed in real-time for all point
sensors located in mainline roadway lanes. Imputation is less critical and more difficult for ramp
and connector facilities that see less predictable traffic patterns.
Different imputation routines require different types of information; as such, it is best practice to
implement sequences of imputation routines that can be utilized depending on the availability of
acquired data. If the data requirements for the optimal routine are not met, then the second routine
can be used, with the process continuing until the sensor’s data is filled in.
In practice, effective imputation regimes follow the below structure, listed in order of accuracy:
1. Local Neighbors. Impute from data measured at neighboring sensors, based on historical
relationships developed between the data at the broken sensor and data at sensors in
adjacent lanes, as well immediate upstream and downstream locations.
Imputation by local neighbours is performed as follows:
Where (i,j) is a pair of sensors, q is count, k is presence, t is the time interval, and α0, α1, β0,
and β1are coefficients estimated through linear regression.
Page 60
2. Global Neighbors. Impute from data at neighboring sensors, using global coefficients.
When a particular sensor has never reported data, this method is recommended because it
does not rely on measured relationships between the broken sensor and its neighbors. In
this method, global coefficients are computed based on relationships between pairs of
sensors in different configurations (for example, sensors in different lanes at the same
location, or sensors upstream and downstream of one another) within the region. These
coefficients are used to estimate values at the broken sensor from data at neighboring
sensors.
Imputation by global neighbours is performed as follows:
op V1
Where (i,j) is a pair of sensors, q is count, k is presence, t is the time interval, =0 if iand jare
at the same spot on the roadway, 1 if they are upstream/downstream, lis the lane number of
C -
the sensor, and the α*0, α*1, β*0, and β*1 are coefficients estimated through linear regression.
y
C
3. Temporal Medians. Impute from temporal median. When adjacent sensors are also
broken, this method is recommended. A temporal median is the median at the same
C
location for the same time of day and day of week over a historical period (for example, the
last eight weeks). Imputation by temporal median is performed by assigning imputed count
and presence values at a sensor as the median of all the count and presence values
Q
measured at the sensor over the most recent set of weeks.
4. Cluster Medians. Impute from cluster medians. This method is recommended when no
other methods are feasible with available data. A cluster is defined as a group of sensor
stations that have the same macroscopic count and presence patterns over the course of a
typical week. A certain number of clusters (for example, 10) can be identified within a
region using cluster analysis, then the median data values for each time of day and day of
week can be computed. When a sensor in a particular cluster is broken, its data is imputed
as the cluster median. Imputation by temporal median is performed by assigning imputed
count and presence values at a sensor as the median of all the count and presence values
measured at the sensor over the most recent set of weeks.
The coefficients and values estimated through each of these methods should be stored in the
database.
6.4.4 Aggregation
Data aggregation is the process by which sensor data is summarized into larger spatial and
temporal units that can be accessed more quickly than the raw data.
6.4.4.1Temporal Aggregation
Point Data
It is recommended that point data first be aggregated prior to the computation, quality control, and
imputation steps. Raw data is typically consumed at the 20- or 30-second levels. For counts, the
five-minute data sample is computed by simply summing up the individual data samples. For
presence, the five-minute value is the average over the samples received. For speeds, it is
Page 61
recommended that the five-minute value be computed as the flow-weighted average across all
samples. These five-minute summary values can be stored in a data table of five-minute sensor
aggregates, from which all further traffic data measures can be derived. Data can also be further
aggregated to hourly and daily values using the same methods.
Link Set Data

The raw data from link set sensors typically consists of a vehicle ID and a timestamp marking when
that vehicle was read at a particular sensor. Following the computation and quality control steps,
the data set consists of validated travel time measurements for each origin and destination sensor
pair. In this case, the initial temporal aggregation process computes summary metrics from all of
the individual vehicle travel times computed within a certain time period (for example, every five-
minutes or every hour). It is recommended that the base temporal level of aggregation for link set
op V1
data match that used to aggregate point data, so that the two data sources can ultimately be fused.
Because link set data is aggregated up from individual vehicles sensors, the aggregation can also
include rich information on the variability of travel times within each time period. It is recommended
that the aggregated link set data table include the following summary metrics, to provide insight
C -
into both the magnitude and variability of travel times measured within each time period, as well as
y
the quantity of data collected:
C
 Minimum travel time
C
 Maximum travel time
 Average travel time
 Standard Deviation
Q
 Percentile travel times (for example, the 25th, 50th, and 95th percentile travel times)
 Number of samples
 Percentage of samples filtered.
Unlike the aggregation of point data, in which hourly data can be aggregated up from five-minute
data, and daily data can be aggregated up from hourly data, the aggregation of link set data must
always be done from sets of individual travel times.
This is because recommended metrics such as the median travel time cannot be readily computed
from summary metrics, and must always refer back to the individual data samples. In general,
travel times do not need to be aggregated to temporal levels higher than hourly, as daily average
travel times are not meaningful.
6.4.4.2 Spatial Aggregation

Spatial aggregation applies primarily to point data; it refers to the aggregation of lane-by-lane
sensor data to the station level. It can be performed at any level of temporal granularity. Some
agencies may choose not to store lane-specific data at high aggregations (such as daily or
monthly), in which case station-level data can be stored instead. For counts, data is aggregated as
the sum across lanes. For presence, data is aggregated as the average across lanes. For speed,
data is aggregated as the flow-weighted average across lanes.
6.4.5 Fusion Engine

Data fusion is the integration of data from multiple sources. A data fusion system, part of a data
warehouse, can integrate data from point sensors and link set sensors from multiple vendors or
agencies. As with sensor technologies (and all emerging technologies), early adopters created
their own data fusion systems, either through internal efforts or by hiring outside contractors to
Page 62
develop custom systems. As vertical transport information markets have matured – advanced
transport management systems, traveler information systems, performance measurement systems
– most turnkey data providers have developed internal fusion engines. When done effectively,
data fusion capitalizes on the strengths of each data source. For example, since link set data
sources sample percentages of the traffic stream, they cannot inform on count information. Fusing
link set data with point sensor data provides a more comprehensive view of link traffic patterns.
There are a few important points of consideration for agencies implementing data fusion. The first
is selecting the data sources and measures to fuse. As in the example above, a common approach
is to fuse counts from point sensors with travel times from link set sensors. A similarly common
technique is to fuse speeds from point sensors with speeds from link set sensors to derive fused
travel times. The second is the level of aggregation- both spatial and temporal- at which to fuse.
For travel time fusion, it is common to link point speeds with link set speeds at a fifteen-minute
op V1
granularity. This is high enough to ensure that sufficient link set samples are received, but low
enough that the travel times are still meaningful. In terms of spatial aggregation, fusion is usually
performed on a link level, rather than a point sensor level. The third is the relative weight to give to
each data source in the computation. This may differ over time depending on a number of factors,
C -
such as the quantity and quality of data from each source acquired during the time period. The
y
C
fourth is the computation of a confidence interval for the fused data point that represents the
accuracy of the estimate. This is typically based on the accuracy of each individual data source
and the weight that the source is given in the fusion computation.
C
Q
Page 63
7 DATA DISTRIBUTION
7.1 Approach
Distribution to end users is the final stage of the road performance data supply chain. Having the
right distribution technology and process empowers agency managers and stakeholders to
efficiently access the data they need to make good decisions. The DMAT is in a position to have
one single integrated information system for users to access its data, as opposed to configuring
access to several legacy databases.All of the systems described here must be implemented in
harmony with the DMAT ITS Architecture.
op V1
7.1.1 End Users
Data distribution should be designed for specific end users and the questions these users need to
answer. The traditional customers for road data are government asset managers, who rely on
C -
simple data about traffic volume, classification, mass and growth rates. Today, road operations
y
C
managers and road users demand more real-time information on roads, incidents and traffic. The
following are typical user groups for usage, condition and location data. The users are divided into
internal (within the agency) and external (outside of the agency).
C
7.1.1.1 Internal
Q
Roadway System Managers: Roadway system managers are responsible for the daily operations
of the roadway network, and often work at a Traffic Management Centre (TMC). These users are
concerned with improving journey time information and reliability, reducing congestion, providing
information to support modal shift away from cars, improving road safety, enhancing road policing,
and changing travel behaviour in response to weather.
Public Safety Staff: Public Safety personnel, typically police and fire, are responsible for
coordinated response to incidents where the safety of people or property needs to be ensured. The
speed with which public safety staff responds to an incident impacts their ability to minimize injury,
death or property damage
Policy-makers: Policy-makers are primarily focused on executing decisions on long-term transport

spending and facility improvements. Their responsibilities also include planning preventative
maintenance (performed to improve or extend the functional life of a pavement) and corrective
maintenance (performed after a deficiency occurs in the pavement), and urgent maintenance
(emergency road maintenance required to ensure safe roadway conditions).
7.1.1.2 External
Information Service Providers: Information Service Providers (ISPs) are in the business and
providing information directly to travellers or to private data providers who add value to the data
before distributing it to travellers.
Media: Media includes radio, TV and new media (Internet) news services that provide real-time
traffic and event information directly to travellers. Traveller information is provided by media as part
of an array of news services or as a public service. Some media is available by subscription only.
Travellers: Travellers are concerned with trips: specifically, what routes they should take, when
they should travel, and how much extra time they should allow for on-time arrival. Travellers make
Page 64
07-DATA DISTRIBUTION First Edition -December 2016
two types of trips: constrained trips (which require on-time arrival) and unconstrained trips (which
are more flexible). Information for constrained trips is of the most value for travellers.
Transit System Managers: Transit system managers care about roadway performance because it
is strongly linked with ridership and revenue. They need to know where poor performance is
caused by overall variable traffic conditions, situations such as bad weather or lane closures, poor
scheduling, or issues with driver performance. Within transit system managers, there are planners
(who plan alignments, create schedules, and consider ways to improve ridership and on-time
performance) and operators (who use real-time information to better manage the system).
Logistics and Fleet operators: Logistics and fleet operators need information to better schedule
shipments and trips, and manage the excess costs of building buffer times into schedules.
Logistics people are primarily concerned with determining the routes and time periods that will
op V1
minimize late deliveries and choosing appropriate delivery windows based on historical reliability
trends. Fleet operators are concerned with ensuring vehicles are in the right locations at the right
times.
Software Developers: Commercial software developers create applications, which are utilized by
C -
y
all of the above end users. Software developers do not administer infrastructure or fleets but
C
instead provide new ways of accessing and applying existing data to those who do. Software
developers may create these applications independently (out of a personal interest or perceived
C
business opportunity), or as a contractor to a transport agency.
7.1.2 DATA USES

Q
This section outlines the needs of each of the user groups for road performance monitoring
information.
Roadway systems managers: Roadway system managers are the primary consumers of road
monitoring data. They utilize the data to obtain a uniform and comprehensive assessment of the
performance of their roads, and then take action based on that information. Benefits of real-time
traffic management include reduced travel times, reduced fuel consumption, and lowered exhaust
emissions. Roadway system managers typically use an Advanced Traffic Management System
(ATMS) as their interface with real-time road performance data. Roadway system managers’
specific operational responsibilities that rely on road monitoring information include:
 Coordinate and adapt to maintenance activities, closures, and detours;

 Detect and verify incidents, and provide incident information to partner agencies and drivers
 Manage traffic and transport resources to support partner agencies in responding to, and
recovering from, incidents ranging from minor traffic incidents through major disasters.
When required, special traffic management strategies are implemented to support
evacuation and re-entry
 Support high occupancy vehicle (HOV) lane management and coordination, road pricing,
and other demand management policies that can alleviate congestion and influence mode
selection
 Manage reversible lane facilities and barrier and safeguard systems that control access to
transport infrastructure
 Coordinate traffic information and control strategies in neighbouring jurisdictions.
 Coordinate with rail operations to support safer and more efficient highway traffic
management at highway-rail intersections
 Exercise control over those devices utilized for automated highway system (AHS) traffic
and vehicle control
Page 65
 Use WIM and vehicle identification data to enforce traffic laws

 Use traffic usage and condition data to determine to diagnose loop detectors and determine
optimum times for lane closures and other roadway maintenance.
Some of these operational uses of data rely on decision support systems, which are software-
based systems intended to help roadways systems managers integrate raw data with models.
Example decision support systems are dynamic ramp metering algorithms and automatic incident
detection.
Public safety staff: Public safety staff needs to quickly access real-time traffic usage and event
information in order to dispatch police and fire personnel to incidents as quickly as possible, and to
re-route traffic at incidents to avoid bottlenecks and delay. Public safety staff and their dispatchers
access typically monitoring information through the ATMS at the TMC or else through
op V1
communications from the TMC. The EOC needs access to monitoring data and location data in
order to plan its emergency response. In case of emergency, the EOC monitors roadways so that it
can dynamically adjust its strategies. The EOC may pre-empt the current traffic control strategy;
activate traffic control and closure systems such as gates and barriers, activate safeguard
C -
systems; use driver information systems to support evacuation traffic control plans; coordinate
y
C
information and controls with other traffic management centres; and coordinate execution of
evacuation strategies, including activities such as setting closures and detours, establishing routes,
updating areas to be evacuated, and timing the process.
C
Policy-makers: Policy-makers use road monitoring data to inform local transport plans, project
prioritization, traffic modelling, journey time studies, planning studies, safety analysis, and air
Q
pollution studies. Policy-makers also need usage and condition data to inform decisions about
maintenance or reconstruction projects. For example, they can use data to determine whether
congestion bottlenecks can be alleviated by improving operations or by minor capital
improvements.
Policy-makers typically have staff that query or mine detailed archived road monitoring data for
analysis; these staff then develop summary reports that aid the policy-maker in decision-making. ·
Travellers: Travellers typically use real-time road monitoring information in order to make
decisions about when to travel, what mode to use, and what route to take. Travellers often receive
the data through web-based and mobile traveller information systems, through media
communications, and through personal navigation devices. Typical performance information that is
shared with travellers includes:
 Map displaying real-time road congestion and construction data

 Construction and closure reports
 Congestion and travel time reports
 Historical travel time reporting
 Dynamic message sign reporting
 Incident reports
 Links to video images available in the corridor
 Links to transport-related sites in the corridor
 Real-time route planning based on traffic conditions.
Information Service Providers: ISPs typically have a formal relationship with a transport agency
or its data warehouse manager, and access road monitoring data by querying the data warehouse
through a server. Increasingly, ISPs and the media acquire agency road monitoring data and
bundle it with other data or services to provide real-time information and/or predictive routing to
Page 66
logistics and fleet operators and to travellers through various technologies. Predictive routing
utilizes real-time, historical, predictive and incident data to suggest optimal routes to drivers.
Private data providers also acquire agency location information to populate their GIS databases
and support their products. An example ISP is INRIX, a private company who currently provides
traffic data to Google, who then combines this data with crowd-sourced and other third-party data
to provide live traffic conditions on its maps in many parts of the world.
Media: Media are interested in the same measures as the travellers they serve but access the
information differently. Media generally access road performance information through ISPs (speed
and event data, live camera images) and communications from TMCs.
Transit System Managers and Logistics and Fleet Operators: Transit System Mangers and
Logistics and Fleet Operators are interested in studying both real-time and historical road
op V1
monitoring data to optimize their resources and decrease travel times. Transit System planners
generally access road monitoring information through a web-based interface; operators may use a
web-based interface or mobile- or location-based interfaces. The data these users need are
speeds and volumes, event information, and in some instances road condition.
C -
y
Software developers: Software developers typically access road monitoring data by querying the
C
data warehouse through a server. The primary concerns of software developers are that the
transport data their software consumes is reliable, standardized to the greatest extent possible,
C
and well documented. Reliable data makes these software systems more useful. Standardization
reduces the cost of including new data feeds into existing software systems. Thorough, clear, and
up-to-date data documentation also reduces the cost of development and can give greater
Q
meaning to the data. Furthermore, the more current the usage data is, the more relevant and
advanced the software systems created by developers can be. The data developers are most
interested in providing are real-time roadway conditions; developers also need location information
to populate their maps and databases. As more data becomes available to third-party developers,
the breadth of software applications is expected to grow.
7.2 Technologies
Historically, road performance data was housed in a database that was only accessible to a few
users. Today, performance data is provided to a variety of end users through many technologies.
The types of data distributed through each technology are dependent on its capabilities, as well as
where and when users typically utilize the technology. The main technologies are:
 Web-based information systems

 Web services
 Mobile services
 Personal navigation devices
 Broadcast radio/TV
 Variable message signs
 Interactive Voice Response
 Alerts
 Kiosks.
These technologies are evolving rapidly and are increasingly integrated. For example, a single
device can now access web-based information systems through broadband, receive location-
based routing information through a private data provider, and receive SMS text alerts on traffic
Page 67
conditions. From the agency’s perspective, web-based information systems and web services are
the most fundamental data distribution technologies listed here (see Section 9.2.3: Distribution).
7.2.1 Web-based Information Systems

Web-based information systems are the primary means of distribution for road performance
monitoring information. They are the core interface through which most end users, internal and
external to the agency, interact with system data. These systems allow transport agencies to
distribute real-time and archived performance data simultaneously over the web to multiple end
users, often in real-time. By definition, these information systems must be accessed through a local
network or remotely over the Internet. Figure 9shows a basic web-based interface for monitoring
real-time road network performance. Figure 10shows a web-based 3D incident visualization used
for active traffic management.
op V1
Web-based information systems are well suited for transport systems performance monitoring for
several key reasons.
Firstly, because transport systems are often distributed over a large geographic region, transport
C -
data sets can be large and thus difficult to evaluate quickly without automated tools. Furthermore,
y
C
transport data can only be interpreted within the spatial context of the physical transport network,
necessitating advanced mapping and visualization techniques. Web-based information systems
meet these needs by giving users tools to navigate and consume data, such as search,
C
visualization, mapping, comparison, and computational analysis.
Q
Web-based information systems can be customized and personalize to support a range of use
cases. Agencies typically maintain different web interfaces to serve the needs of various internal
and external users (see Section 7.3.2: Access Control). Users of web-based applications can
request specific types and aggregations of data directly, reducing the burden on agencies to
manually prepare and distribute data themselves. Web applications can also be personalized at
the individual user level to support varying needs and preferences.
Web-based information systems can be large and complex, as they are often used as access
points for tasks related to the collection, filtering, processing, and storage of data, in addition to the
distribution of data to end-users. To accomplish these tasks, web applications connect to various
components of a traffic management system, such as data warehouses, traffic stations, and
camera systems. At the data distribution level, end-users typically use these systems to obtain
processed data or generate reports to be used for decision-making. These systems can also
increase the efficiency of data distribution, by allowing analysis tasks to be automated and
scheduled for common or recurring reporting needs.
Page 68
op V1
C -
Figure 9: PeMS, a Web-Based Information System Displaying Traffic Speed, Variable
y
C
Message Signs, Incidents, and Lane Closures in Real-Time
Web-based information systems are made up of several discrete components:

C
 Server(s). A server is a machine that is dedicated to executing a specific operation in
support of a larger process. Servers are typically connected to a network and their activity
Q
results in inputs and outputs of information over that network. For example, a server could
be set up to pull traffic data from the field, aggregate it, and then pass it on to another
server. Servers can be secured, allowing agencies to deploy applications in a secure
environment.
 A Graphical User Interface (GUI). Users must interact with web-based information
systems through a GUI. GUIs facilitate the visual navigation and display of information. A
GUI can include elements such as buttons, scroll bars, text boxes, and tabs. Most web-
based information systems have a GUI that is written in HyperText Mark-up Language
(HTML).
 Middleware. Middleware, sometimes referred to as “plumbing”, is a type of computer
software that serves minor operations involving connecting various software and hardware
components to one another. In a transport network, a layer of middleware could be used to
standardize the format of data coming in from the field.
 Geographic Information Systems. Geographic information systems (GIS) store, analyse,
and present geographically referenced data.
 Other Software. There may be a need for other software beyond that running on the
server, the GUI, the middleware, and the GIS software.
Page 69
op V1
C -
y
Figure 10:Regional Integrated Transport Information System (RITIS) 3D Incident
C Visualization
The following sections provide examples of web-based transport information systems.

C
7.2.1.1 Advanced Transportation Management System
Q
An Advanced Transportation Management System (ATMS) is an example of an information system

used within an agency, at a TMC. An ATMS integrates and processes real-time information from
sensors, cameras, incident reports, and other sources for use by roadway system managers.
ATMSs perform functions beyond the scope of this document, such as controlling CCTV cameras
and logging incidents. However, from a performance monitoring standpoint, their interfaces
typically provide a map with traffic data overlays, as well as the ability to examine real-time traffic
data and events in-depth. While ATMSs have not historically been web-based, the advantage of
implementing a web-based ATMS is that it can facilitate access and data sharing across multiple
TMCs and operation centres.
7.2.1.2 Traveller Information Services

Traveller Information Services assemble data from the TMC and distribute useful information to
travellers through the web or mobile devices (described below). The agency or market decides
how much of the network will be included in a traveller information service based on how much
data is available and the needs of end users. Traveller Information Services use performance
monitoring data to display real-time traffic information over a map and provide travellers with
expected trip times. They also typically provide information beyond the scope of this document,
such as multi-modal trip planning.
7.2.1.3 Archived Data User Service

Archived Data User Services (ADUS) are web-based platforms from which users can access
historical performance monitoring information. Through ADUS, users can access data and
performance measures at multiple levels of spatial and temporal granularity and in different formats
via GUI reports. These reports let users query the database to obtain information for selected
geographic segments and date ranges. Figure 11 is a sample web-based data plot of traffic
congestion and incidents generated through an ADUS.
Page 70
op V1
C -
y
C
C
Figure 11:PeMS, Displaying Traffic Congestion and Incidents (Blue Circles) Across Space
and Time
Q
7.2.2 Web Services

Web Services is a way for ISPs and software to access road monitoring data. The term web
services broadly describes software systems designed to support two-way communication
between a pair of electronic devices over a network, typically involving a request from a client and
a response from a server. Web services enable the large-scale dissemination of query able
information (such as freeway speeds at a location) and services (such as routing). For example, a
smartphone application could request freeway congestion information at a particular location and
time from a web service, and the web service would respond with the requested data. These
requests are typically initiated by the user through the client application’s graphical user interface
(GUI) but are executed entirely by the application over a network.
The functionality of web services differs from that of web-based information systems. In short, web
services facilitate application-to-application communication while web-based information systems
are built for user-to-application communication. Web services are a platform for the exchange of
small amounts of information without a GUI. They are used by computer programs to communicate
with a server “behind the scenes”. Web-based information systems, on the other hand, are more
fully featured, containing a GUI and typically presenting a large amount of navigable information
and interactivity. The two are related in that web-based information systems can connect to web
services (and/or directly to local information sources).
Web services have become widespread largely because they allow information to be disseminated
directly, eliminating the overhead of a browser-based interface. This means that a transport agency
interested in providing data to the public can focus their efforts on the data itself. Agencies that
adopt this approach benefit by freeing themselves of the cost and complexity associated with
developing and maintaining the applications that present this data to the public. The growth of web
services harmonizes with the Open Government movement, growing global interest in publicly
accessible government data (sometimes referred to as Open Government), allowing government to
Page 71
fill the role of data providers (by maintaining web services) while the private sector builds
applications to deliver this data to the public.
7.2.2.1 Standards in Web Services

The World Wide Web Consortium (W3C), an international organization, maintains various
standards (defined in the W3C Web Services Activity Statement) which govern the procedures and
information formats by which applications and web services communicate. These standards are
necessary because of the large (and increasing) diversity of web services and applications that
interact. Although these standards are not formally enforced, web services that adopt them benefit
from increased use and visibility due to the large number of existing software libraries developed to
access web services based on these standards. These standards create uniformity across web
services at multiple levels: in the way the web service advertises its functionality to client
op V1
applications, in the exchange protocol by which requests and responses are sent over the network,
and in the format of the exchanged information.
According to the W3C, all web services must describe their functionality in a language called Web
Services Description Language (WSDL). Applications interested in interacting with the web service
C -
can refer to the service’s WSDL document to see what operations the service is capable of
y
C
performing. The WSDL document for a web service describes exactly how each operation should
be called, as well as the variable types it expects to receive and the parameters to be returned for
C
each operation. WSDL is based on Extensible Mark-up Language (XML), which is a widely used
machine-readable text format.
Q
The interface for a web service is referred to as its Application Programming Interface (API). These
interfaces are most often developed to conform to one of two main communication protocols:
Representational State Transfer (REST) or Simple Object Access Protocol (SOAP). Web services
which implement a REST-based interface can use standard HTTP, which makes them simpler to
implement and access than those built to use SOAP. REST also allows transfer of data in a variety
of formats, for example JavaScript Object Notation (or JSON), whereas SOAP can only transfer
data using XML. SOAP has more extensive security features than REST and built in “retry” logic
for communication failures.
7.2.2.2Application Programming Interface (API)

While the communication with the web-service can be carried out according to the REST or SOAP
protocols, that communication takes places through the web service’s Application Programming
Interface.
A transport agency can set up a real-time web-based API that allows third-party programs to query
the API and allows the server to respond with information.
API users need good documentation, reliable and up-to-date data, clear policies about what
agency branding they can use, and clear policies about data ownership. While provision of real-
time data through an API has become prevalent, agencies are only now providing more static
system information such as geospatial information and network topology through APIs.
7.2.3 Mobile Services

The proliferation of computationally powerful portable devices with access to the internet through
3G (UMTS) networks, 4G broadband networks, or Wi-Fi networks means that users have access
to real-time road performance data wherever they are. These devices are often equipped with a
sophisticated array of sensors that allow the user’s experience to be tailored to their current
Page 72
conditions. For example, users can view traffic conditions around them without having to first find
their location on a map. The most common sensors in mobile devices include three-axis
accelerometers, GPS chips, magnetometers, microphones, touchscreens, gyroscopes, digital
compasses, and cameras. This diverse array of sensors as well as powerful on-board computing
ability and network connectivity make mobile devices an important part of the transport data
system.
Web browsers that can access and properly display web content are now included in most mobile
devices, allowing their users access to the same web-based information systems available to users
of desktop computers and other “non-mobile” devices. Even when they are accessed through the
mobile device’s browser, these information systems can still determine the location of the mobile
device based on the geo-location of the device’s IP address. This allows them to present data
op V1
C -
y
C
C
Q
specific to the user’s current location, if desired.
Figure 12:Real-Time Traffic Conditions on a Mobile Device
Challenges in working with mobile devices include their much smaller screens as compared to
other types of devices. This usually means that less information and fewer UI controls can be
presented on a mobile device at once. Often, additional effort is taken to develop mobile-versions
of traveller information tools that are specially formatted for the smaller screens of mobile devices.
Mobile applications are often developed by transport agencies for use by travellers. For example,
ONE.MOTORING is a program in Singapore with a mobile application that offers real-time traffic
conditions in addition to allowing users to pay taxes and fines, and sell and buy vehicles.
7.2.4 Personal Navigation Devices

Portable navigation devices (PNDs) include navigation-enabled cellular phones, in-car navigation
systems and portable GPS units, all of which use a GPS navigation device to acquire position data
Page 73
to locate the user on a road in the unit’s map database. Because various types of navigation
devices are beginning to overlap in their usage and capabilities, they are all treated as PNDs in this
section.
PNDs can now be used to receive and display real-time traveller information. For example, Garmin
PNDs in the United Sates provide traffic information sourced from: data from millions of Garmin
owners; data from millions of cellular phone owners; incident reports; radio feeds of live
information; news stations; historic traffic data from NAVTEQ Traffic Supply; historic traffic data
from Garmin owners; and, fixed traffic sensors on major roads. Figure 13 shows a typical PND
interface displaying location information and estimated arrival time. Use of a PND to acquire real-
time information typically requires a traffic receiver and monthly subscription to a data service.
op V1
C -
y
C
C
Q
Figure 13: Garmin nüvi® 3490LMT Personal Navigation Device
7.2.4.1 Data Transmission to PND

Commercial providers of PNDs provide traffic data directly to users of their devices using cellular
broadband, Bluetooth, FM radio or Satellite radio. What is transmitted to the traveller through these
media is personalized, based on the traveller’s location and projected route, but generally limited to
key intersections or trouble spots and sometimes delayed because the information is transmitted
intermittently.
Bluetooth: With Bluetooth, a cellular phone receives the traffic information, which is beamed to the
PND via Bluetooth technology. There are limited phones that are compatible with Bluetooth.
FM Radio Data System: FM Radio Data System (FM-RDS) is an extremely low-bandwidth

communications protocol for embedding small bits of digital information in conventional FM radio
broadcasts. FM-RDS is increasingly used to transmit traffic information to travellers through their
PNDs that are equipped with Traffic Message Channel (TMC) receivers. More than 200 FM-RDS
messages can be transmitted per hour. Each message, usually a detailed warning about incidents
and traffic congestion, is binary-encoded and sent as a TMC message according to the Alert C
standard and translated by a TMC receiver into the language of the user. Location code tables
assign numerical values to specific road segments or intersections. In Europe, location code tables
are maintained on a national level and integrated with commercial in-vehicle navigation systems or
automobile manufacturers. In the U.S. and Canada, location tables are maintained and marketed
privately. The European FM TMC is one of the most advanced FM-RDS for traffic information.
Page 74
Satellite radio: Many satellite radio services have dedicated traffic and weather channels that
cover metropolitan areas and play a continuous loop of local weather information and detailed
traffic data. Satellite radio can also be used to transmit traffic information such as traffic incident
and average speeds to compatible PNDs, which overlay the current traffic information for a driver’s
chosen route on top of a GPS road map. The information can also be used to estimate travel
times. Satellite radio is generally only available by subscription. FM-RDS information can also be
transmitted using satellite radio.
7.2.5 Broadcast Radio/TV

One of the oldest and most common sources used by travellers to obtain information for their
commute is commercial radio and TV, which provide periodic quick snapshot reports of travel
conditions. Radio and TV are ubiquitous, making it accessible to most commuters including while
op V1
they are in their vehicle. One of the challenges with radio and TV is the limited time allocated for
traffic reports and a lack of personalization based on where a traveller is located and what their
information needs are.
7.2.5.1Low-Power Broadcasting
C -
y
C
Low-power broadcasting (LPFM) is the use of low-powered radio stations to provide localized
travel information. LPFM is used by governments across the world. These stations are generally
active near highways, airports and some tourist attractions like parks.
C
7.2.6 Variable Message Signs
Q
Variable message signs are typically LED signs that display text messages received from the TMC,
where VMS control software is located. The software uses a client-server architecture to control a
VMS, and communicates with a VMS through many technologies such as optical fibre, radio, or
Ethernet. Even though variable message signs (VMS) are used mainly to report critical information
on road closures, incidents, and other non-predictable situations, transport agencies develop travel
time information and use the signs to display it when no other critical information takes priority.
Most transport agencies that display travel time information have an automated algorithm that
calculates the distance and the speed from sensors and provides travel times for specific corridors
and/or landmarks. Commuters can decide if an alternate route should be taken based on the travel
time displayed or, at the very least, the information helps remove the uncertainty or worry about
how long a trip might take. Figure 14 shows an example of a VMS that communicates real-time
traffic monitoring information.
Page 75
op V1
Figure 14:Variable Message Sign Displaying Real-Time Delay Information
C -
y
C
7.2.6.1Interactive Voice Response
A voice-activated travel information system is a type of interactive voice response system that
C
provides travel information on-demand using speech recognition and DTMF tones entered via the
telephone keypad. A voice-activated travel information system can be accessed by calling a
specific phone number from a landline or cellular phone. In the United States, a single phone
Q
number, 511, has been designated by the Federal government for accessing local traveller
information services across the nation. The service can be provided in different languages and also
understand different user languages or accents. These systems provide:
 Emergency messages
 Real-time traffic conditions
 Information for common congestion points
 Event information
 Weather conditions
 Public transit information.
7.2.7 Alerts
An increasing number of transport agencies are offering alerts to their customers on a subscription
basis (free or paid). Commuters can select specific routes for which they want to receive e-mail
alerts sent to a PDA, mobile phone, or an Internet account. Alerts can be about traffic conditions,
construction projects, or weather. The advantage of alert services is that the commuter doesn't
have to actively seek traveller information because it is automatically sent to the subscriber, who in
turn can decide to change the route or timing of a trip to avoid problem areas. The use of Twitter
(Internet-based social networking software) or other Web 2.0 technology is a similar way for
agencies to communicate traffic information to travellers.
7.2.8 Kiosk
A kiosk is a structure housing a walk-up information station or posting keyboard. A traveller
information kiosk can provide a variety of travel information to users, including dynamic traffic and
transit information, travel directions, fare card dispensing, travel reservations, and directory
information.
Page 76
7.3 Process
In order to successfully distribute data, the data distribution system must maintain acceptable
levels of performance, typically expressed in terms of reliability, responsiveness, and security.
Distributing data also requires considering who should have access to what data; data liability
issues; and presentation of data.
7.3.1 Distribution System Performance

Independent of the technology platform, users will interact with the data distribution system by
connecting through a client to a server. Since server design is a common problem in many
disciplines, guidelines exist which can be followed to ensure acceptable performance of the
distribution system.
op V1
It is important for the agency to establish reliability goals for web services and web-based
information systems for several reasons. For example, these services will likely be relied upon for
public safety functions such as responding to traffic incidents where system availability is critical.
Data distribution systems should also be responsive (to ensure a positive user experience) and
C -
y
secure.
C
System reliability is most often conceptualized in terms of time: an unreliable server is one that has
a higher likelihood of being unreachable at any given time. Particularly in highly distributed
C
systems, it is possible that individual service interruptions may affect only a portion of the service’s
entire user base. Service reliability can also be quantified by the number of users affected by a
Q
particular outage. For every disruption of service, the start time (time when the interruption began),
recovery time (time when service was fully restored), and the percentage of users affected should
be recorded.
A server handling the distribution of transport data should not only be reliable, it should respond
quickly. Slow response times are one of the most frequently cited sources of user frustration with a
service. For a transport agency, responsiveness is also important to ensure that the user receives
data as close to real-time as possible.
Lastly, the data distribution system must be secure. An unsecure system is vulnerable to many
types of malicious attacks including code injection and denial of service (DoS) attacks. These
attacks can lead to data loss, compromised data security, or a loss of access to the distribution
system.
7.3.2 Access Control

Firewall and security issues should be addressed as early as possible when designing platforms
for road performance data distribution. Use cases can be placed in one of three main access
categories:
1. Public access: available to all interested parties (public, academia, private firms and public
institutions), typically free of charge through a simple registration process. An example is a
full set of archived sensor data provided through a public website
2. Subscriber access: restricted to data contributors and other parties, included as fee-
paying users or on the basis of meeting the conditions of a public interest test. This may
include inter-agency sharing. An example is real-time predictive travel times provided to an
ISP via a restricted-access data feed.
Page 77
3. Private access: restricted to the internal use of the providing organisation, no pricing. An
example is the detailed provision of sensor metadata detailing diagnostic thresholds and
imputation parameters.
Access to metadata may not be necessary for all users, but would be important to those
performing analysis of performance data.
Operationally, access is controlled using firewalls and/or user passwords. For example, sensor
data from a traffic management centre may have to pass through a firewall before being added to a
data warehouse. This allows the TMC agency to control which traffic data can be shared with end
users and can also be used to control the privacy of the TMC’s data sources. Another possible
use of a firewall is between the web server and the Internet. Passwords and Virtual Private
Networks (VPN) are Internet-based methods of controlling access.
op V1
It may be useful to create a regional data clearing house to streamline access to a region’s full
range of transport data for planning and operations users. Partnership agreements may be
required with other public agencies like transit agencies and private sector sources such as freight
operators.
C -
y
C
A key distribution issue, particularly to ISPs, is the need for a service level agreement incorporating
requirements such as a guaranteed minimum availability level, a response time in the event of
breakdown, and a protocol for notice to change characteristics such as the data format.
C
Protecting the privacy of vehicle users is another consideration when managing access to data.
Q
For example, if a traffic video is made available streaming online (as is common in some parts of
the world), it is possible that sensitive personal information (related to a traffic accident, for
example) may be inadvertently broadcast. In this case, there may be a legal case made against
the agency that provided the video. Furthermore, agencies have learned over time that traffic video
should not be stored permanently in order to avoid subpoenas.
7.3.3 Data Liability Issues and Copyrights

Transport agencies must consider their data ownership rights and those of its private partners
when distributing data. Private partners may incur risks in data sharing with a transport agency and
its users, and private purveyors of data may be distrustful of government access policies and
controls. There may also be a potential conflict between the public benefit of open access to
transport data and the government/private interest in restricted access to data for competitive
advantage.
Data ownership and copyrights can be ambiguous, particularly when data has been aggregated or
derived from multiple sources. The views on ownership and copyrights have been slowly evolving
given that the ultimate purpose of a responsible service provider (both government and private
sector) is to deliver a benefit to their customers.
7.3.4 Presentation of Data

At the end of the data distribution process lies the nontrivial task of actually presenting data to the
end-user. This often involves difficult, open-ended decisions such as what data to include and how
to best represent it. For example, should vehicle speeds on a freeway be communicated using the
average speed, the median speed, or as a distribution? Should this information be compiled for 3-
kilometer freeway segments, 5-kilometer segments, or segments of arbitrary length, and at what
level of temporal aggregation? Should this data be presented in a chart, a table, or on a map? How
should gaps in the data be represented? The answers to these questions depend on the use case,
Page 78
however there will likely be uncertainties along the way, even in the most prescribed
circumstances.
Effective presentation of data is critical to the effectiveness of the performance monitoring practice
as a whole. Presentation decisions can inappropriately influence the way the data is interpreted
and what actions are taken based on its review. This happens when graphs are disproportionately
skewed or when the scale of a graph is misleading. However, more subtle cases of data distortion
also exist. For example, imagine an agency that wants to rank the top freeway segments of
arbitrary length based on number of traffic incidents. It would seem natural to rank the segments by
average incident density (incidents per kilometre), but this would artificially rank long segments
with a sustained high number of incidents lower than short segments with a brief spike in incidents.
Distributors must also consider what data to include, and how to aggregate it, since presenting raw
op V1
transport data is often undesirable. Summary statistics can be used to aggregate large quantities
of raw data into meaningful metrics. When this is done, care must be taken to ensure that the
summary statistics themselves harmonize with intuition about their meaning. For example, the
mean and median vehicle speed on a freeway can often be very different depending on the
C -
homogeneity of the sample group, but they are often interpreted as having roughly the same
y
C
meaning. Summary statistics can also vary widely based on criteria like which weekdays and
times-of-day they include.
C
7.3.5 Data Presentation Tools
Since data is often presented with a specific use case in mind, the format for that data should also
Q
be chosen according to that use case. Different data presentation formats have different
characteristics in terms of how they are perceived by users visually, and each format highlights
different things about the data it presents. The efficient and accurate performance of the
anticipated task for which the data will be used can be encouraged by the choice of data
presentation tool. One way to do this is by choosing tools that facilitate the use of visual operators
to complete the anticipated task rather than more demanding logical operators.
The efficiency with which a visual task is performed depends on the job required of the user and
the presentation format. In the case of transport visualization, a user may have to first orient
themselves to the visualization in order to interpret it correctly, then they may visually identify
patterns in the data, then they may decide on a suitable course of action based on the patterns
they have found. An intuitively presented visualization reduces the cognitive energy required for
the initial orientation stage, and visualizations that highlight trends “at-a-glance” allow patterns to
be recognized quickly and easily. The formats most commonly used in transport data are
dashboards, maps, graphs, tables, video, and animations.
7.3.5.1Dashboards
Transport data dashboards are a natural metaphor for a vehicle’s dashboard: they display the most
important, high-level performance metrics of a transport system at-a-glance. A performance
monitoring dashboard often contains system-wide metrics such as the total number of vehicle-
hours of delay in a system, the total number of incidents in the day, the average travel time
reliability during the peak period for the day, or system-wide sensor health. Figure 15 is an
example dashboard for a roadway sensor system.
Page 79
op V1
Figure 15:Dashboard Depicting Peak Period Travel Time Reliability and Sensor Health over
the Entire System
7.3.5.2Maps
C -
y
C
Maps use location data to plot transport metrics in space geographically. Maps are most useful for
communicating vehicle speeds at a location, variable message sign data, incidents, lane closures,
and road maintenance activities. Maps can also show behaviour over time such as average traffic
C
volumes, occupancy, congestion, and common areas for bottlenecks, incidents, and sensor health.
Web-based traffic maps commonly utilize third-party maps such as those provided by Google, due
Q
to their low cost and ease of implementation. Maps are the primary user interface for road system
managers and travellers.
Map users benefit from a familiar interface, and allow users to easily find data at specific locations
that they recognize visually. Moreover, maps include the configuration of the roadway network in
the presentation of the data itself, which could help to highlight a large number of incidents
resulting from a blind corner, for example.
When traffic data is plotted on a map, spatial trends are highlighted. However, maps are limited in
the amount of data they can show at once. For example, for clarity, freeway data and arterial data
are typically not shown simultaneously on maps at the same scale.
7.3.5.3Graphs
Data that is encoded into points, lines, shapes, and colours is commonly referred to as a graph.
Computers enable direct interaction with graphs, allowing users to explore the data themselves. In
a printed visualization, the scale and range at which the data is displayed must be fixed. However,
using computers, it is possible for users to scroll through the data continuously, zoom and pan
smoothly without reloading, highlight individual data points, and turn layers of data on and off.
Printed graphs are still be valuable for displaying smaller ranges of data.
Along with maps, graphs form the primary means through which users interact with transport
system data. Due to their technical nature and ability to show greater detail, graphs are most
commonly used by those internal to the agency.
Page 80
op V1
C -
Figure 16: RITIS Visualization of Speeds on a Freeway Segment During an Incident, with
y
C
Distance on the Y-Axis and Time on the X-Axis. A Queue Can Be Seen
7.3.5.4Tables
C
Tables are a useful format to export data in for use in other databases or applications. Data tables
should be made available as a convenience to users, allowing them to dynamically produce and
Q
download reports, and to obtain a copy of a data set for their own purposes. Tables are also useful
for cases where the viewer may need to see the actual values, instead of a symbolic
representation of them in a graph. Additionally, smaller data sets are often well-suited for tables,
when a graph would be unnecessary.
7.3.5.5Video
A video stream may be used as part of a camera-based traffic monitoring system. This consists of
a live or recorded video from a camera that is often stationed at a fixed location along a roadway.
The video itself can be adjusted to highlight processing tasks related to machine-vision, such as
automatic vehicle detection. This allows viewers of the video to see whether the software properly
detects all vehicles.
7.3.5.6Animations
Web-based distribution systems can leverage high-bandwidth network connections and/or vector-
based graphics techniques (e.g., SVG, HTML5 Canvas) to present animated data. Animations
differ from static graphs in that they evolve over time. A map with roadways that change colours
with conditions and a chart that evolves to show the time-dependant effects of a traffic incident are
both examples of animations. The value of animations lies in their inclusion of time as an additional
degree of freedom in visualizing data. Following intuition, data animations most often evolve in time
in a way that correlates with the time dimension of the data being displayed. However, animations
could also assign another data type to the time dimension of the animation. For example, imagine
an animation that scrolls down a freeway displaying congestion across the roadway measured at a
single moment in time.
Page 81
8 APPLICATIONS
This section details the relationship between the process of monitoring (the focus of this guide) and
the applications of monitoring in two sections. The first section gives an overview of this
relationship. The second section lays out the types of applications built on top of monitoring
systems.
8.1 Monitoring and Measurement

8.1.1 Fundamental Relationship
Ultimately, the purpose of all monitoring is measurement. To apply monitoring data to agency
op V1
business processes, low-level data must be translated into high-level measures that track
performance. However, monitoring is not the same as measurement. System monitoring is an on-
going process. It must be implemented continuously over the course of years, to build a base of
knowledge about the performance of a transport network. Performance measurement is a
C -
purpose-driven task that is subordinate to the business process it is used to enable.
y
C
Business processes change frequently within agencies. New processes emerge with changes in
leadership, in response to legislative mandates, and as staff strive to implement new technologies
C
and methods. Basic transport system behaviour, on the other hand, is relatively static. Entire new
models of transport infrastructure do arise with the emergence of disruptive technologies: rail,
Q
bicycles, and automobiles. However, the introduction of new transport technologies is relatively
infrequent; once every few decades. Because of this, the key to system monitoring is consistency
in the measurement of a slowly evolving infrastructure. Understanding this distinction is the key to
understanding the application of monitoring data to performance measurement.
System monitoring is, in other words, a mediation layer between a physical infrastructure of relative
permanence and an ephemeral, overlapping network of business processes. Any suitable
performance monitoring system broadly considers the requirements of both the physical world and
agency business processes, but does not slavishly model itself after the current business process.
This business process, inevitably, will change and any monitoring system that does not recognize
this fact will become quickly obsolete.
8.1.2 Coupling
The art of effective performance measurement is to tie the measures that DMAT chooses to
consider to an underlying set of monitoring data that DMAT is collecting. Failing this,
measurements will be infrequent, inaccurate, and unsustainable.
8.1.3 Methods
Assuming the fundamental building blocks of the monitoring system are in place, generating
measures from the smallest units of monitoring data is a straightforward, relatively mechanical
process. The building blocks, such as diagnostics and imputation ensure that the data underlying
the measures are valid and complete.
The algorithms used to generate these measures are available in publications such as the
Highway Capacity Manual. The measures on the system can be divided into geographic
components that match the atomic data units defined in Chapter 4 (Traffic). Some common
higher-level measures are defined below, as examples.
Page 82
08-APPLICATIONS First Edition -December 2016
8.1.3.1 Screen Line Measure Methods

Spatial measures track the performance of the system at a specific point. Speed is one of the
three fundamental screen line measures (along with vehicle count and vehicle occupancy). With
any combination of two of these measures, the third, and all derivative measures can be
computed.
If the sensors are reporting speed (e.g. radar detectors deployed head-on, or double loop
detectors) speed can be measured directly. In all other cases (e.g. single loop detectors), speed
must be computed. Speed computations are varied, but traditionally have been computed by using
a G-factor in combination with the count and occupancy. The G-factor is a value that represents
the effective length of the vehicle. It is a combination of the average length of the vehicles in the
traffic stream and the calibration of the traffic sensor itself. Typically, a constant value for the G-
op V1
factor is used for each detector location. This leads to inaccurate speeds because the G-factor
varies by lane, time-of-day, as well as the detector sensitivity. The DMAT recommended approach
estimates a G-factor for each loop for every 5 minutes over an average week to provide accurate
speed estimates. The algorithm has been tested and validated against ground truth data from
C -
double loop detectors and floating cars.
y
C
This standard method estimates speed using flow, occupancy, and a G-factor that represents the
average effective length of a vehicle according to the following equation:
C
Q
1
Where: g is the average effective car length (the sum of the car length and the width of the loop
detector).
Because g has been shown to vary with a number of factors, including location and time of day,
this relationship can be refined. One method assumes that vehicles travel at a certain free-flow
speed when traffic conditions fall below a certain occupancy level. This free-flow speed is not
constant for all locations, but varies by the number of lanes at the location, the lane number, and
the lane type (mainline or HOV). The method then works backwards to estimate an average
vehicle length for uncongested conditions based on the free-flow speed and the measured flow
and occupancy values at the detector for that time of day and day of week. Plotting the obtained
values and fitting a regression line to the scatter plot gives an estimate of effective vehicle length
by time of day.
Mean effective vehicle length does not just depend on the time of day, but can also vary by day of
the week, lane, location of the detector station, and detector sensitivity. To account for these
differences, effective length estimates can be stored at 5-minute intervals, for every day of the
week and every lane at every detector station.
These values can then be retrieved in real-time, multiplied by the observed volume to occupancy
ratio, and then filtered to obtain speed estimates. These further steps have been employed to
estimate speed from single-loop detectors. These calculate the G-factor using the previously
described method for a number of different time periods of every day of the week using a defined
set of free-flow speeds.
This G-factor is used to compute the initial estimate of speed for each loop in real-time. The speed
estimate is then passed through an exponential filter with weights that vary as a function of flow
Page 83
(the smoothing is severe when flows are low and weak when flows are high). The resulting
estimate is the speed reported for that detector during that time period.
8.1.3.2Journey Measure Methods

Journey measures track the performance of the system from an origin to a destination over a
specific route. The primary journey measure is travel time. Methods for calculating travel time
depend fundamentally on the type of sensor used to measure the journey. For infrastructure-
based sensors, this step requires (1) estimating mean speeds from the raw data; and (2)
transforming these spot-speed estimates into link travel times. For vehicle-based sensors, this step
requires (1) calculating a mean travel time from the samples collected during the time period and
(2) quantifying the error in the travel time estimation. The detailed methodologies for these steps
are presented in the rest of this section. To illustrate methods, an example of the former algorithm
op V1
is detailed below.
The algorithm starts with the speed estimates, developed in the previous section. The outputs of
these speed calculations are spot-speed estimates, or speed estimates at multiple fixed locations.
Since no data is provided on vehicle speeds in between the fixed detectors, efforts must be made
C -
y
to connect the various speed points and approximate a link travel time. This methodology
C
assumes that speeds measured at a detector apply to the stretches of roadway between that
detector and halfway to the nearest upstream and downstream detectors.
C
After the speed distribution along the roadway has been approximated, average travel times can
be calculated. One option for calculating travel times from spot-speeds is to simply sum up the
Q
travel times between each detector using the speeds measured at the trip start time. This allows
for the prediction of travel times along a route in real-time, but is less accurate for estimating travel
times on long routes, where speeds are likely to change during the journey. A better estimation
technique for long trips is to calculate the time that it takes a vehicle to traverse a segment, get the
speed at the next segment at the time the vehicle would arrive there, calculate the travel time for
the next segment, and repeat so as to arrive at the total travel time for the entire route. The
disadvantage of this method is that it cannot calculate travel times for the current time period, only
for trips that just ended.
This sort of aggregation introduces a number of biases into the travel time reliability calculations.
Aggregation of infrastructure-based data usually occurs at two different levels: data is first
aggregated to 20-, 30-, or 60-second intervals during data collection and transmission, then is
frequently aggregated further for storage in the archive. With respect to travel time calculations,
recent research has suggested that there are three errors introduced in the sampling process from
fixed detectors:
Sampling Error: This error is the difference between a vehicle’s true travel time and the travel time
calculated from its speed at a fixed detector. This error is due to the assumption that a vehicle’s
speed remains at its detected speed until the next downstream detector. This error increases as
the distance between fixed detectors grows.
Grouped Speed Error: This error is the difference between individual travel time estimates and
the group-averaged travel time estimates. The grouped speed error is the result of applying a
single travel time value to a group of vehicles that actually experiences a range of travel times.
Arithmetic Mean Speed Error: Individual travel time estimates are accurately calculated from the
harmonic mean speed. Since the detector speed estimates are arithmetic mean speeds, this error
Page 84
is the difference between the actual group-averaged travel time and the group travel time estimate
calculated from the arithmetic mean speed.
In reliability monitoring with infrastructure-based sensors, the only one of these biases that can be
controlled is the sampling error, which is partially a function of the distance between detectors.
Having closely spaced detectors reduces the probability that large variations in speed will occur in
between the detectors. For example, with detectors spaced far apart (for example, every 2
kilometres), free-flow conditions might exist at two detectors with a bottleneck in between them. In
this case, the average travel time would be reported as free-flow, when in reality, conditions are
much slower. In contrast, it is also possible for two detectors to be placed at bottleneck locations
(for example, downstream of busy entrance ramps). Even though speeds might be free-flow in
between them, their reported speeds and thus travel times would be very slow.
op V1
Detector spacing and location is key to maximizing the accuracy of reliability measurements. One
study on monitoring delay on urban freeways found that detector spacing of one-half kilometre are
needed to capture delay with an accuracy of 10%. Unfortunately, spacing this close is not
commonly seen in practice, especially in rural areas. Methodologies for handling the limitations of
C -
sparse infrastructure-based detection are presented in the Assembling Travel Time Distributions
y
section of this chapter.
C
8.2 Types of Applications
C
8.2.1 Measures
Q
8.2.1.1Overview
The core application of system monitoring data is to track measures of agency performance.
These measures typically derive from a list of objectives, which, in turn, derive from a list of agency
goals. Most commonly, they are considered the key performance indicators (KPIs) of agency
practice.
8.2.1.2 Characteristics of Useful Measures

Generally speaking, measures should share several common characteristics. They should be:
observable, actionable, endogenous, diverse, and important. Although counter-intuitive, it is
common practice to develop KPI's that cannot be observed, and - therefore - measured. As a
practical matter, this is rarely useful: if a KPI cannot be effectively measured, however
inadequately, it should not be tracked.
Agency measures should also be actionable: the agency should be able to impact the measure.
Occasionally, agencies select measures over which they have minimal impact. These should be
avoided. For example, measures of economic growth, while important, are only marginally
impacted by the day-to-day activities of a department of transport. Generally speaking, non-
actionable measures can be made actionable by narrowing their scope to just the phenomena over
which the agency does have purview. For example, economic goals can be narrowed to freight
mobility goals, or general infrastructure health goals, two actionable slices of economic
development.
Measures should be endogenous. In economic terms, variables in economic models are often
divided into endogenous and exogenous. Endogenous variables are internal to the system.
Exogenous variables are external the system. In effect, endogenous measures are those that can
be controlled by the agency; exogenous measures are largely beyond agency control. This criteria
Page 85
is a complement to actionability. It is not enough for a measure to be actionable; it must also be

largely within agency control, not primarily driven by outside factors. For example, most measures
of congestion (delay, journey times) are problematic, because they are driven both by agency
practice and by larger societal forces, such as economic growth.
Measures should be diverse: they should examine as many sides of agency performance as
possible. Diversity has a large scope and can cover many vectors of analysis. Some diversity of
stakeholders impacted is desirable. Diversity of measures over internal department division
responsibilities is also desirable. Another desirable form of diversity is a mix of input and output
measures. Output measures track the end performance of the transport system, often from a
traveller perspective. An example of an output measure is journey time reliability. Input measures
track the internal efforts that the department makes to improve output measures. An example of a
related input measure is incident clearance time.
op V1
Measures should be important. Often agencies clutter their goal tracking with an abundance of
measures, only some of which track the core issues. Tracking each additional measure (for a
single goal or objective) comes at a cost of clarity of understanding and action. Tracking a very
C -
limited set of measure per goal often provides a clear outcome that is lacking when a plethora of
y
C
measures show differing trends.
8.2.1.3Deriving Measures from Data

C
In practice, it is a major challenge to develop a monitoring regime that can support department
goals. Often, there is a mismatch between the broad aspirations of an agency toward its mission
Q
and the limited resources that an agency devotes to monitoring those goals. At a practical level,
this is typically the core challenge: the monitoring system is inadequate to the task of
measurement. To mitigate this, DMAT will adopt two principles: incrementalism and realism.
Incrementalism implies that measures inadequately monitored are superior to measures that are
not monitored. Implementers of monitoring systems shall prefer imperfect monitoring to absent
monitoring. For example, measuring journey times is best performed through direct measurement
with an automated vehicle identification system, such as those identified in Chapter 5. However,
lacking resources to deploy such a system, they can be estimated through point detection.
Lacking resources for either automated vehicle re-identification or point detection, manual (floating
car) surveys are preferable to no monitoring. Each measure should be monitored, however
crudely.
This implies that measures that cannot be effectively monitored will not be tied to agency goals. In
absence of resources to implement any of these monitoring approaches, it is critical that the
department cull measures that cannot be monitored. At its core, this is a reflection of the
philosophy that what gets measured gets managed. This realism requires extreme discipline, in
order to narrow department focus on goals that can be measured and achieved, over goals that
are admirable, but not actionable. In short, the connection between measurement and monitoring
must be maintained. Maintaining the connection between department performance measures and
the monitoring system is critical to the credibility of both. Without a derivation from monitoring
data, performance measures are toothless. Without a clear role in the generation of performance
measures, monitoring systems are purposeless.
Page 86
8.2.2 Evaluations
8.2.2.1Evaluation vs. Measurement

While measurement is typically an on-going process, tied to mission-oriented goals, evaluation is
typically a more ad hoc process, dedicated to understanding the outcome of a specific action.
Often, evaluations are focused on the impacts of projects. Did the new road the DMAT built
reduce delay on a corridor? Did a signal retiming reduce delay on the minor approaches?
Evaluations and on-going measurement are critical, interrelated tools. Without performing
evaluations during a year, there is no way to assess progress toward a measured goal.
Evaluations help the department understand whether or not a specific approach or tool will
positively impact performance on a specific goal.
op V1
8.2.2.2Evaluation and Monitoring Data
As previously discussed, measurement should be closely coupled with monitoring. The coupling
between evaluations and monitoring system is much looser. Typically the goal of evaluations with
respect to data should be to leverage the most monitoring data possible, without limiting
C -
y
themselves to monitored data. Evaluations typically tend to be more specialized and often require
C
larger data sets than an agency typically monitors. Because they are ad hoc and generally
unforeseen, it is unrealistic to attempt to craft a monitoring system that fully supports all possible
C
evaluations. Rather, the goal of monitoring data within an evaluation context should be to provide
a base of support, not an exhaustive set.
Q
8.2.3 Analytics
Analytics are higher-level measures that are used to identify patterns in transport data, to support
decision-making. Applications of analytics include the study of data using statistical analysis in
order to discover and understand historical patterns, with focus on prediction. The field of analytics
includes the use of operations research, statistics, and probability. Common transport analytics
include time space diagrams, such as speed-contour plots, and automated bottleneck
identification.
8.2.4 Modelling
Modelling is an emerging area of monitoring data application. Transport models of all varieties
(travel demand, operational simulation) require massive amounts of data for calibration and
validation. Most of this data can now be acquired using automated systems. Because of this,
efforts are increasingly being made to integrate monitoring systems into predictive traffic models.
These integrations take two forms: loose coupling and full integration.
Transport models of all types – travel demand, micro-simulation – were developed before ITS
sensor-based monitoring systems. Because of this, they typically require non-monitoring data sets
for calibration and validation. Some agencies have developed tools and systems for transforming
sensor data from monitoring processes to fulfill a subset of the calibration needs of these traditional
models.
An emerging class of integrated models allow for closer coupling with monitoring systems. These
types of models are typically run in real-time and used for both prediction and planning. They are
designed from the ground up to integrate with monitoring systems and fully utilize sensor data for
calibration.
Page 87
8.2.5 Decision Support

Decision support systems combine analytics and modelling to provide automated answers to basic
tasks. For example, a decision support package might combine elements of performance
measures, analytics, and modelling to present a fully integrated toolkit for making real-time
operational decisions. These tend to be the most sophisticated and demanding applications of
monitoring systems.
8.3 Example Application

8.3.1 Goal and Strategy
As an example of this measure application process, say a transport agency was interested in
op V1
improving safety. Further suppose that this agency had purview over freeways and arterials. One
possible strategy would be to improve operational safety by clearing major incidents more quickly
than current practice. Major incidents often cause secondary incidents, particularly in low-visibility
situations, so this is plausible. The question for this guide is: How does this goal interact with the
RPMS?
C -
y
C
8.3.2 Key Performance Indicators
C
The first step in this process is to develop a set of key performance indicators (KPI) to better
monitor the transport system. As per above, this KPI should be actionable, endogenous, diverse,
and important. One KPI to monitor is clearly simply the number of accidents. Underneath this,
Q
should be a database that also shows, for each incident, where they occurred, when they occurred,
their type, and the period of time they have been occurring. From this data, another KPI could be
developed to identify average accident time length. At a first cut, these two KPI’s are a good
starting place for implementing a process to clear incidents more quickly, in order to prevent
secondary incidents. So far, the KPI’s are:
 Number of incidents on the network. (Units: incidents.)

 Average incident clearance time. (Units: hours.)
8.3.3 Leveraging Monitoring

Any agency that followed the process detailed in this guide would already collect the data required
to report these KPI’s (see Table 7 in the following Chapter). This means that any user of the
system should be able to request or build reports to detail all required KPI’s, sliced up
geographically or temporally as needed. As such, the connection between monitoring and this set
of KPI’s is clear. An analyst could use any set of applications – evaluations, modelling, or decision
support – to provide management with the tools they need to view the incident trends. This is the
outbound linkage between KPI’s and monitoring.
8.3.4 Refining Monitoring

However, there is another, equally important inbound linkage between KPI’s and monitoring.
These are management or operational needs that refine the way that monitoring is performed. For
example, the two KPI’s identified here may be adequate to characterize the incident clearance
response, but it is not clear how actionable they are. It is possible to attempt new operational
clearance strategies, such as running frequent service patrols, and then measure subsequent
clearance time, but this is a relatively crude KPI to understand the anatomy of each incident.
Page 88
For example, every incident has a measurable set of activities associated with it. Each incident
also has a set of measurable impacts on the performance of the freeway system. For every
incident, there are essentially two theaters of activity, or locations where response is taking place.
In the traffic management center, operators and dispatchers are detecting, verifying, and
coordinating incident response. In the field, at the incident, response personnel are directly
assisting travelers impacted by the incident.
In addition to these activities, incidents impact corridor operations. These impacts include
bottlenecks upstream from the incident, and occasional bottlenecks in the opposite direction of the
freeway. Figure 17 shows a conceptual diagram of the detailed anatomy of an incident in the form
of an incident timeline. In this timeline, the dots represent actions by individuals in the traffic
management center or in the field. The lines represent durations of first responders on scene.
The call outs in the diagram represent major, systematized, measured time points within an
op V1
incident.
C -
y
C
C
Q
Figure 17: Detailed Anatomy of an Incident
Many of these measures could greatly assist in determining the cause of slow incident clearance
times. Is it that vehicles are not arriving quickly enough? Are they spending too much time on
scene? Is there a stakeholder that is blocking clearance? Are operators performing the correct
actions?
An agency dedicated to this reduced clearance time goal would be well served to develop more
KPI’s around incident clearance and then feed the requirements from these KPI’s back into the
agency RPMS. This can be a challenge. Perhaps the activity logging systems used by operators
in the traffic management centre will need to be recoded. Perhaps operators and first responders
will need to be retrained. First responders might need new vehicle-based sensors or equipment to
assist with this monitoring.
Page 89
8.3.5 Conclusion
This sequence of steps represents the process that should be applied for any given KPI
application, within the context of the RPMS. First, leverage the existing RPMS data to understand
a given problem. Then, look for gaps in this data and develop new KPI’s to support an operational
process. If warranted, feed these requirements back into the RPMS process and develop a new
set of monitoring process to support these KPI’s.
op V1
C -
y
C
C
Q
Page 90
9 ACTIVITIES
9.1 Approach
A complete road performance monitoring program involves activities that take place on four
different time scales:
1. Initial: These are the initial sets of activities required to create the baseline monitoring
system
2. Operational: These are activities that take place on a frequent (likely daily) basis. They
are on-going activities, not infrequent
op V1
3. Periodic: These activities occur with a very slow frequency: monthly to annually
4. Triggered: Triggered activities occur in response to specific events.
This chapter describes the specific key activities that the DMAT should undertake in each of these
categories.
C -
y
9.2 Initial
C
C
9.2.1 Collection
Q
9.2.1.1Install Sensors
The DMAT should install infrastructure-based sensors and deploy vehicle-based sensors. Ideally,
the location and density of these sensors should be based on the data types and quality that are
needed by end-users. In practice, it is difficult to determine optimal detector location and density. It
is recommended that sensors be placed at minimum prescribed spacing that vary by roadway
functional classification and region (urban or rural) The following are general roadway
classifications that can be used to plan sensor infrastructure.
1. Expressway: Limited access roadways with grade-separated junctions and barrier-

separation between directions of travel. High Level of service
2. Arterial: Some at-grade junctions, with paved islands separating traffic directions. High
level of service
3. Collector: Provides a less highly developed level of service at a lower speed for shorter
distances by collecting traffic from local roads and connecting them with arterials. 10,000-
20,000 AADT
4. Local: Consists of all roads not defined as arterials or collectors; primarily provides access
to land with little or no through movement. 1-10,000 AADT.
For each road functional classification, recommended measures and sensor spacing are provided
below. Where urban and rural roads have different monitoring requirements, they are described
separately.
Urban Expressway
Priority 1: Volume, Speed, Occupancy (at least two of these)
Priority 2: Travel Times
Deployment Requirements:
Page 91
09-ACTIVITIES First Edition -December 2016
 Infrastructure-based sensors at a one-kilometre spacing in mainline lanes and on

freeway-to freeway interchanges
 Infrastructure-based sensors at every on-ramp and off-ramp
 Vehicle re-identification sensors at key origin-destinations pairs, spaced no further than
30 kilometres apart
 Vehicle classification sensors placed at a spacing of 50 kilometres, or where high truck
volumes are known to be present.
Rural Expressway
op V1
Infrastructure-based sensors at a three-kilometre spacing in mainline lanes and on
expressway interchanges
 Infrastructure-based sensors on major on-ramp and off-ramps (AADT> 1,500)
 Vehicle Classification sensors placed at a spacing of 50 kilometres, or where high truck
C -
volumes are known to be present.
y
Urban Arterial
C
C
Priority 2: Travel Times along key corridors
Q
 Infrastructure-based sensors in every lane placed, within 3 metres of each intersection

 Vehicle re-identification sensors placed every four intersections along segments of
arterial that have AADT>15,000.
Rural Arterial
 Infrastructure-based sensors in every lane, placed within 3 metres of each intersection.
Collector
Priority 1: Volume, Speed, Occupancy (at least two of these) at signalized intersections with
arterials
 Infrastructure-based sensors in every lane, placed at the mid-block between

intersections.
Local
Priority 1: Volume
 Bi-annual hourly traffic counts taken over a period of at least one week.
A higher density of sensors may be desirable in areas with high accident rates, frequent
congestion or bottlenecks, high traffic travel/recreational traffic, construction sites, rapidly
Page 92
growing/urbanized areas, jurisdictional boundaries, and routes with high goods movement
volumes.
9.2.1.2Collect Metadata
Metadata must be acquired and assembled prior to the creation of a performance monitoring
database. The following types of metadata and attributes must be collected manually from the field:
Condition Data
Condition Traffic Sensors
 Road Number
 Direction

op V1
Kilometre-post
 Latitude
 Longitude
 Lane Number
 Lane Type
C -
y


Sensor Type
Jurisdiction.
C
C
Traffic Usage Data
Infrastructure-based Sensors
Q
 Road Number
 Direction
 Kilometre-post
 Latitude
 Longitude
 Lane Number
 Lane Type
 Sensor Type
 Reports presence?
 Reports speed?
 Jurisdiction.
Vehicle Identification Sensors

 Road Number
 Direction
 Kilometre-post
 Latitude
 Longitude
 Sensor Type
 Number of lanes
 Jurisdiction.
Event Usage Data
Traffic Signals
 Intersection Name
 Main Street Road Number
Page 93
 Cross Street Road Number

 Signal timing phases by movement
 Jurisdiction.
Weather Stations
 Latitude
 Longitude
 Sensor Type
 Road Number (optional)
 Direction (optional)
 Kilometre-post (optional).
Special Event Venues


op V1
Venue Name
 Latitude
 Longitude.
Following the collection of all metadata from the field, agencies need to perform the following steps
C -
to prepare the metadata for storage in the database:
y
C
Condition Data
1. Identify stations into which condition traffic sensors can be grouped
C
2. Identify roadway segments along which pavement condition information is collected, by
road number, direction, starting kilometre-post, and ending kilometre-post
Q
Traffic Usage Data

1. Identify stations into which infrastructure-based sensors can be grouped
2. Calculate the length of roadway monitored by each infrastructure-based station
3. Define links monitored by vehicle identification sensors, by the road number, direction,
starting and ending sensors, and starting and ending kilometre-posts
Event Usage Data

1. Define sections of roadway impacted by weather conditions measured at each weather
station, by road number and starting and ending kilometre-post(optional)
2. Define sections of roadway impacted by special events at each venue, by road number,
direction, and starting and ending kilometre-post(optional)
9.2.1.3Collect Location Data

The performance monitoring system requires a well-formed GIS network with specific attributes
that can fully define the agency’s entire roadway system. Chapter 4 describes the data needed to
represent the roadway network, and Chapter 5 lists alternative sources for acquiring location data.
Existing GIS data may be organized in databases for other uses such as traffic modelling, some
may be from commercial map providers, and the remaining GIS data should be collected expressly
for the purpose of creating a road performance monitoring system.
Other transport data that should be collected are railroads, ferries, aviation, non-motorized, port,
jurisdictional boundaries; some of this may come from other transport agencies.
Page 94
9.2.2 Processing
9.2.2.1Construct GIS Network

The complexity of constructing a GIS network depends on:
 Location referencing systems (LRS). Is a standard LRS being used by the agency?
 GIS tools. Is the agency already using GIS databases and software?
 Quantity and quality of data. Do new data items need to be collected?
 Management systems. What systems are already in place for managing pavements,
bridges, safety, signs, pavement markings, etc.?
 Hardware and software. Are legacy components sufficient to support the task of
integrating large and complex sets of information?
op V1
Depending on the array of GIS data sources used by Abu Dhabi DMAT, work is needed to
integrate heterogeneous data. Geo-referenced data may be linear, point or tabular and stored in
different geo-referenced file types (Shape, CAD, Coverage, etc.) and need to be synchronized to a
C -
common schema. Some other data maybe not be geo-referenced at all, a may be stored in a flat
y
file or hierarchical database.
C
Efforts to homogenize data should be organized around improving data quality. Following the
C
acquisition of a GIS data set, the agency must perform the following steps to ensure that the
network definitions can support performance monitoring:
Q
 Homogenize attributes
 Create relationships between features
 Synchronize data with a linear referencing system
 Create topology or connectivity between road segments
 Referencing to geographic coordinate systems
 Create custom user interface to access existing databases.
Quality assurance and quality control (QA/QC) must be performed for location data at the time of
creating the GIS network and every time new data is added. The majority of QA/QC functions can
be automated and fit into the following four categories:
1. Topological – Checks regarding connectivity of the line work at intersections, overpasses

and bridges represented as separate features, arcs meeting at jurisdictional boundaries,
etc.
2. Scale/Spatial – Does the location accuracy meet the planned business use of the data;
does the “aesthetic” representation of the transport feature meet the business
requirements?
3. Attribute – Are the minimum required fields included, are the field descriptions met, how
many of the attributes are populated, are the attribute values valid?
4. Metadata – Concerns regarding metadata include: has the required metadata been
provided, is it complete, does it conform to established metadata standards; does the
metadata match the layer?
Because this GIS database should be integrated with other agency functions and process,
stakeholders that should be involved in this process are: data users, asset managers, policy-
makers, information technology professionals, data management professionals and external
experts and data collectors. The geo-database used for road performance monitoring can be
integrated with an enterprise GIS which is used for GIS applications across agency departments,
Page 95
with a GIS-based asset or work management system, Customer Relation Management software,
or with enterprise financial software.
Appropriate personnel should be trained in the requirements and uses of the new system. Some
challenges to address in developing a GIS network are working with agency staff who manage
databases that will now need to be integrated; agency staff who do not wish for the data they
collect to be transparently visible to policy-makers; and costs that are difficult to anticipate,
including labour costs for planning and data acquisition, software and hardware purchases,
technology changes and data storage and maintenance.
9.2.2.1.1 Create Database

The performance monitoring system requires a relational database to store location, condition, and
op V1
usage information, as well as associated metadata. The database and the operating system it runs
on must be capable of storing 5 terabytes of data. The database should include, at a minimum, the
tables and fields listed in Tables 5 through 9. Optional means that the record may contain null in
the field if it does not apply and was not collected.
C -
y
C
C
Q
Page 96
Table 5: Location Data Tables
Table Name Purpose Fields Requirements

Roadway Links spatial reference Road Number Must contain a record for
Locations information Road Name every 0.1 km of roadway.
(latitude/longitude) with Must contain both external
linear referencing Direction and internal starting kilometre-
information (kilometre- posts for each record.
posts). Links external Start External
and internal kilometre- Kilometre-post
posts.
Start Internal Kilometre-
post
op V1
Start Latitude
Start Longitude
Length
C -
y
Jurisdiction
C
Routes Defines key segments Route ID Must contain records for each
C
across which Route Name strategic route. If route spans
performance indicators multiple roadways, multiple
will be measured Start Point Name segments must be defined.
Q
Road Number and Direction

End Point Name must link to Roadway
Locations table. If routes
Segment ID match with links monitored by
vehicle identification sensors,
Road Number record must include the
corresponding link set link ID
Direction (see Link Set Links metadata
table).
Start Kilometre-post
End Kilometre-post
Link Set Link ID

(optional)
Roadway Defines key roadway Road Number Must be defined for each
Attributes segment characteristics Direction homogenous roadway
tracked by the agency section. Road Number and
Start Kilometre-post
Direction must link to
Length Roadway Locations table.
Speed Limit
Functional Class
Page 97

Features Locates structural Feature Must contain records for each
assets (such as bridges Road Number key asset to be tracked in the
or signs) system. Road Number and
Direction Direction must link to
Roadway Locations table.
Kilometre-post
Description (optional)
op V1
C -
y
C
C
Q
Page 98
Table 6: Condition Data Tables

Survey Records pavement Condition Segment ID Condition Segment ID must
History condition information Date link to Condition Segments
recorded during periodic metadata table.
surveys PCI
IRI
FWD
Treatment Records dates and Condition Segment ID Must locate treatments by

History types of treatments Date condition segment ID. May
performed to address also locate treatments by
op V1
deficiencies Treatment Type kilometre-post (applicable
for spot treatments)
Cost
Kilometre-post(optional)
C -
y
Defects
C
Records locations of Condition Segment ID Must locate defects by
roadway defects Defect Type condition segment ID and
kilometre-post. Must store
C
Kilometre-post the date the defect was first
recorded, as well as the
Date Recorded date that it was treated (if
Q
applicable).
Date Treated (optional)
Treatment Type
(optional)
Raw Records raw condition Timestamp Sensor ID must link to

Condition data from sensors (such Sensor ID Condition Sensors metadata
Traffic as WIM) table.
Speed
Weight
Vehicle Class
Axle Weight
Filtered Records condition data See Raw Condition Sensor ID must link to
Condition following data quality Traffic Condition Sensors metadata
Traffic processing table.
Page 99

Five-minute Records condition data Timestamp Sensor ID must link to
condition aggregated to the five- Sensor ID Condition Sensors metadata
traffic minute level. table.
Vehicle Class
Flow
Presence
Average Speed
Average Weight
Average Axle Weight
op V1
Hourly Records condition data See Five-minute See Five-minute Condition
condition aggregated to the hourly Condition Traffic Traffic
traffic level
C -
y
Daily
condition
C
Records condition data
aggregated to the daily
See Five-minute
Condition Traffic
See Five-minute Condition
Traffic
traffic level
C
Monthly Records condition traffic See Five-minute See Five-minute Condition
condition data aggregated to the Condition Traffic Traffic
Q
traffic monthly level.
Page 100
Table 7: Traffic Usage Data Tables
Raw Point Store raw data Timestamp Must store raw, unaltered sensor
collected from Sensor ID data. Stores speed and presence
infrastructure Count data only if directly reported by the
based sensors Presence sensor. Sensor ID must link to
(optional) Sensor metadata table.
Speed (optional)
Raw Aggregated Stores five- Timestamp Must store raw data aggregated up
Point minute data Sensor ID to the five-minute level. Stores
aggregated from Count speed and presence data only if
the raw data Presence directly reported by the sensor.
op V1
(optional) Sensor ID must link to Sensor
Speed (optional) metadata table.
Computed Point Stores raw and Timestamp Must store computed values (such
computed data Sensor ID as speed) with the raw data, for
collected from Count sensors that do not directly report
C -
infrastructure Presence required values. Sensor ID must link
y
C
based sensors at
the five-minute
Speed to sensor metadata table.
level
C
Imputed Point Stores raw and Timestamp Must mark which data points are
imputed point Sensor ID directly measured and which are
data at the five- Count imputed. If imputed, it must mark
Q
minute level Presence the method of imputation. Sensor ID

Speed must link to Sensor metadata table.
Imputed?
Imputation method
(optional)
Hourly Point Stores hourly Timestamp Must store the % of data imputed
sensor data Sensor ID during the hour. Sensor ID must link
Count to Sensor metadata table.
Presence
Speed
% Imputed
Hourly Station Stores hourly Timestamp Must store station-level performance
data aggregated Station ID measures such as delay and VKT,
across all lanes Count which must be computed based on
of a station Presence the station’s length (see Station
Speed metadata table). Station ID must link
Delay to the Station metadata table.
VKT
% Imputed
Daily Station Stores daily data See Hourly Station See Hourly Station
aggregated
across all lanes
Monthly Stores monthly See Hourly Station See Hourly Station
data aggregated
across all lanes
Raw Link Set Stores raw Timestamp Must store raw, unaltered link set
vehicle reads at Sensor ID data.
vehicle-based Vehicle ID
sensors.
Page 101
Computed Link Stores raw Timestamp Must store travel times associated
Set vehicle travel Link ID with link IDs. Link ID must link to the
times along a link Vehicle ID Link Set Link ID metadata table.
Travel Time
Quality Controlled Stores filtered See Computed See Computed Link Set
Link Set travel times Link Set
along a link
Five-minute Link Stores five- Timestamp Must store information on average
Set minute Link ID travel time, as well as the variability
aggregated Valid Sample Total of travel times during the interval.
travel times % Filtered Must store number of valid samples
along a link Average Travel used to calculate the travel time, as
op V1
Time well as the percentage of travel
Standard times filtered during the quality
Deviation control step. Link ID must link to the
10th percentile Link Set Link ID metadata table.
50th percentile
C - 95th percentile
y
Hourly Link Set
C
Stores hourly
aggregated
See Five-minute
Link set
See Five-minute Link Set
travel times
C
along a link
Five-minute Fused Stores traffic Timestamp Must store a confidence indicator
Traffic Usage data fused Link ID based on data quality. Link ID must
Q
between point Count link to the Link Set Link ID metadata

and link set at Speed table.
the five-minute Travel Time
level. Confidence
Hourly Fused Stores data See Five-minute See Five-minute Fused Traffic
Traffic Usage aggregated up to Fused Traffic Usage
the hourly level Usage
from the five-
minute traffic
usage data table.
Page 102
Table 8: Event Usage Data Tables

Incidents Stores data for individual Incident ID Must indicate incident start
incidents Road Number time and duration and/or
Direction end time, as well as a
Kilometre-post specific location. Road
Latitude Number and Direction
Longitude must link to Roadway
Start Timestamp Locations Table.
Response Time
Duration
Injuries
Fatalities
op V1
Severity (optional)
Work Zones Stores data for individual Work Zone ID Must store work zone start
work zones Road Number time and duration and/or
Direction end time, and the roadway
Start Kilometre-post section. Road Number
C - End Kilometre-post and Direction must link to
y
C Start Timestamp
Duration
Roadway Locations table.
Lanes Closed
C
Type (optional)
Control Stores signal timing Signal ID Signal ID must link to
status for each phase of Timestamp Signals metadata table.
Q
an intersection Phase Number Phase Number must link to

Phase Status Signals metadata table.
Weather Stores weather Weather Station ID Weather Station ID must
information from Timestamp link to Weather Stations
weather stations Visibility metadata table.
Precipitation
Temperature
Wind Speed (optional)
Events Stores special event Event ID Venue Name must link to
information Start Time Venues metadata table.
Duration
Venue Name
Attendance
Page 103
Table 9: Metadata Tables

Sensors Stores information about Sensor ID Must link sensors with their
traffic usage sensors. Station ID associated station. Station
Type ID must link to the Stations
Lane Type metadata table.
Lane Number
Measures Presence?
Measures Speed?
Stations Stores information about Station ID Must store stations by
traffic usage stations Version Number version number to track
over time. Active Date activity over time. Must
Road Number allow a station to be
op V1
Direction marked as inactive. Must
Kilometre-post contain a field to mark the
Latitude current record for each
Longitude station. Must store station
Number of Lanes length to support the
C - Lane Type spatial aggregation of
y
C Station Type
Length
performance measures.
To support fusion, can
Active? optionally store the link ID
C
Measures speed? that the station is on (if
Current? applicable). This link ID
Jurisdiction must link to the Link Set
Q
Link ID (optional) Links metadata table.

Name (optional)
Link Set Stores information about Sensor ID Must store sensors by
Sensors automatic vehicle Version Number version number to track
identification sensors. Active Date activity over time. Must
Sensor Type allow a sensor to be
Road Number marked as inactive. Must
Direction contain a field to mark the
Kilometre-post current record for each
Latitude sensor.
Longitude
Active?
Current?
Jurisdiction
Number of Lanes
Name (optional)
Link Set Stores information about Link ID Must contain a record for
Links links monitored by Link Name (optional) every link monitored by
automatic vehicle Road Number vehicle identification
identification sensors. Direction sensors. Must store the
Start Kilometre-post link length to allow speeds
Start Sensor ID to be computed from travel
Start Name (optional) times for data fusion
End Kilometre-post purposes.
End Sensor ID
End Name (optional)
Length
Page 104

Condition Stores information about Sensor ID Must link sensors with their
Sensors condition traffic Station ID associated station. Station
monitoring sensors Type ID must link to the
Lane Number Condition Stations
metadata table.
Condition Stores information about Station ID Must store condition
Stations condition traffic Version Number stations by version number
monitoring stations Active Date to track activity over time.
Road Number Road Number and
Direction Direction must link to the
Kilometre-post Roadway Locations table.
Latitude Must allow a station to be
Longitude marked as inactive. Must
op V1
Number of Lanes contain a field to mark the
Lane Type current record for each
Station Type station. Must store the
Active? roadway group. Condition
Current? Segment ID must link to
C - Condition Segment ID the Condition Segments
y
C Group
Jurisdiction
metadata table.
Name (optional)
C
Condition Defines segments on Condition Segment ID Must store a record for
Segments which pavement Road Number each segment on which
condition is monitored Direction the agency collects
Q
Start Kilometre-post condition surveys. Must

End Kilometre-post store the group and the
Group functional class of the
Functional Class segment. Road Number
Name (optional) and Direction must link to
the Roadway Locations
table.
Signals Defines traffic signals for Signal ID Must contain a record for
which data is collected Intersection Name each signal from which
Main Street Road Number timing data is to be
Cross Street Road Number collected. Main Street and
Phase Number Cross Street Road
Movement Numbers must link to the
Sensor ID (optional) Roadway Locations table.
Jurisdiction Must store the movements
associated with each
phase of the intersection.
Weather Locates weather Weather Station ID Must contain a record for
Stations stations and associates Station Type each monitored weather
them with roadway Latitude station. May link weather
segments Longitude stations with segments of
Weather Station Name roadway affected by
(optional) measured conditions
Road Number (optional) (through either kilometre-
Start Kilometre- posts or route ID). Road
post(optional) Number must link to the
End Kilometre- Roadway Locations table.
post(optional) Route ID must link to the
Route ID (optional) Routes table.
Page 105

Venues Locates special event Venue Name Must store all regional
venues. Latitude special event venues. May
Longitude associate them with
Jurisdiction roadway sections affected
Road Number (optional) by the event (through
Direction (optional) either kilometre-posts or
Start Kilometre- route ID). Road Number
post(optional) and Direction must link to
End Kilometre- the Roadway Locations
post(optional) table. Route ID must link to
Route ID (optional) the Routes table.
op V1
During the database creation process, the agency must select units along which to constrain
values stored in the database. The following units are recommended for key data items:
 Cost (dirham)
 Date (YYYY-MM-DD)
C -
y
 Delay (vehicle-hours)
C
 Length (kilometres)
 Precipitation (millimetres)
C
 Presence (% during time interval)
 Road Width, Lane Width (metres)
Q
 Speed, Speed Limit, Wind Speed (kilometres per hour)

 Temperature (degrees Celsius)
 Time/Duration (minutes)
 Timestamp (YYYY-MM-DD HH:MM:SS, in local time)
 Travel time (minutes)
 Weight (kilograms).
To enable relations between data tables, the agency must also generate standardized sets of
acceptable inputs for the following data items (for example, direction as either N, S. E, or W).
Inputs should align with standard agency asset management and data collection terminology.
 Defect Type
 Direction
 Feature
 Functional Class
 Group
 Incident Severity
 Intersection Movement
 Road Number
 Sensor Type
 Signal Status
 Station Type
 Terrain
 Treatment Type
 Venue Name
 Visibility
 Work Zone Type.
Page 106
9.2.3 Distribution
Due to the many different use cases for transport system data (described in Chapter 7), creating a
single data access point that meets every possible distribution need is not recommended. Instead,
agencies should focus on two fundamental groups of end users for road performance data:
1. Users who only need unformatted data

2. Users who need elements such as a graphical user interface (GUI), maps, cross-
referencing, and inline documentation in addition to the data itself.
The first end user group includes developers of third party mobile applications and operators of
variable message signs, and is satisfied through web services. Data users such as information
service providers and fleet operators can also use agency-developed web services to incorporate
op V1
transport data from the agency into their systems. The second end user group includes roadway
system managers and policy makers and is satisfied through web-based information systems.
Because web services and web-based information systems are the basis for all distribution
technologies, specific actions recommended will be made for these types of systems rather than all
C -
of their variations described in Chapter 7. These recommended activities are specific to the
y
C
technology platform where appropriate.
Web services and web-based information systems (and all of their variations) will run on one or
C
more servers connected to a database which stores transport data streaming in from the field.
These servers must first be designed to ensure acceptable performance given the expected load
Q
they will see. Proper server design takes into account estimates of concurrent connections and
request rate to ensure that the servers can deliver acceptable reliability and responsiveness.
Ground rules for data clarity must be established, including defining data formatting rules and
documenting the distribution system. This step minimizes the effort required to consume the data.
Finally, rules for user consumption of data must be established. These rules govern issues such as
data ownership and acceptable user behaviour, and are typically expressed in a license
agreement.
9.2.3.1.1 Establish Service Performance Goals

As established in Chapter 7, it is important to establish clear performance goals for the data
distribution system. These goals will lead to concrete parameters from which the distribution
system will be designed. System reliability, responsiveness, and security should be considered
when building a data distribution system. Table 10 summarizes service performance values; each
metric is explained below.
Table 10: Service Performance
Guide Guide Guide

Area Metric Scoring Formula Value: Value: Value:
Red Amber Green
Service Annual # 5MinPeriodsDown 98% 99% 99.99%
Reliability uptime 100%-
percentage
105190
Concurrent Maximum N/A 1,200 2,400 6,000
Users concurrent
connections
Request Rate Maximum N/A 6,000/ min 12,000/ 30,000/
supported min min
Page 107
request rate
Responsiveness Average N/A 10 seconds 7 seconds 2
response seconds
time
Reliability
Reliability refers to uninterrupted availability of the distribution system and is measured in terms of
“uptime percentage” (aggregated monthly or annually). This value is the total percentage of time
that the service was reachable to end users within a given month (or year). Uptime can be
calculated by subtracting the percentage of five-minute periods in the year during which the service
was unreachable from 100 percent.
As described in Section 6.2.3, cloud hosting eliminates the need for service providers to house and
op V1
operate their own servers. This specialization has improved standards for the reliability of web
services. As of 2011, most cloud hosting services promise uptime percentages exceeding 99.9%
annually (10.1 minutes of downtime per week on average), with the most reliable services
promising 99.9999% annual uptime (0.605 seconds of downtime per week on average).
C -
y
Transport agencies that deploy their distribution system in the cloud are not necessarily
C
guaranteed to achieve the uptime percentage promised by their hosting service. Rather, that
percentage represents the expected uptime percentage of the server hardware. If the distribution
C
software running on that hardware is unreliable, the actual uptime percentage will drop.
Transport agencies that host their own distribution systems may find achieving very high uptime
Q
percentages challenging, particularly while these systems are under active development.
To accommodate self-hosted systems, and because of the complexity of transport data distribution
systems, an acceptable annual service uptime percentage is 99% (an average of 1.68 hours of
downtime per week). If the service is outsourced, it may be appropriate to define a policy through
which the customers are refunded based on the amount of downtime in a given month.
Concurrent Connections
In order to avoid service outages, the server should be designed to withstand the most demanding
conditions expected. The first step in estimating those conditions is to estimate the maximum
number of expected concurrent active users. This is particularly important for self-hosted systems,
as cloud hosting services typically scale capacity up and down automatically with demand.
In web-based information systems, a user is active from the time their session begins to the time
their session expires or is terminated. For web services, a user is active from the beginning of their
request for data to the time when the response is delivered. Note that concurrent connections are
only as demanding as the load that each connection places on the server. With that caveat, a
transport data distribution system should be able to support at least 2,400 concurrent users. These
recommendations are made based on typical 2011 server technology and expected user activity
but can vary widely by application.
Users do not submit requests continuously while connected to a service. Rather, a user requests
information from the server, the server processes that request, and then the server returns the
result to the user. At this point, some time elapses before the user’s next request. The time
between user requests is called think time. The maximum number of expected concurrent
connections and the expected average think time can be used to arrive at the number of requests
per minute that the server should support. This value, when combined with the cost of each
request in terms of memory allocation, database access, etc., defines the necessary load capacity
Page 108
of the server. Since the cost of each request varies greatly depending on local factors (e.g.,
database structure) as well as the distribution technology used, guidance is only provided on
request rate. Peak request rates vary greatly, but at a minimum, a peak (unsustained) rate of
12,000 requests/min should be designed for. Note that these figures take into account the more
demanding database access and processing tasks common in transport data distribution systems.
Think time is inversely proportional to request frequency; therefore users of a data distribution
system with very high request frequencies can cause service interruptions. This means that a
single user can potentially bring down the system for all users. In order to prevent this, a maximum
allowable request frequency should be defined for each distribution channel. For web-based
information systems, this can be done by tracking the user sessions associated with each request
and denying requests from users that exceed the threshold. For web services, this can be done by
granting users of the web service a private alphanumeric key code that must be included in every
op V1
request.
A single key whose requests exceed the threshold should be ignored. These keys should be
unique, and should be stored by the agency along with that user’s name and email address.
C -
y
This threshold must be defined based on the processing and input/output (I/O) capacity of the
C
server as well as the typical complexity of each request. It may be necessary to monitor user
activity on data distribution system and manually limit certain users.
C
Responsiveness
Q
Responsiveness is measured in units of average response time and is defined as the time elapsed
between the user’s initial request and the server’s response.
Average response time is affected by factors such as network bandwidth, number of users, number
and type of requests submitted, and average think time. Some of these elements (such as the
bandwidth of the user’s connection) are uncontrollable. In transport applications, a core means for
improving responsiveness is by optimizing database queries. Additional improvements can be
made by improving the server’s network bandwidth or optimizing the server’s process management
program. Web usage studies show that average response time should be less than seven seconds
in all cases, regardless of the complexity of the request.
Security
All servers connected to an open network are vulnerable to malicious attacks. However, steps can
be taken to defend against the most common types of attacks. One of the most damaging types of
attacks is code injection. Code injection targeting an SQL database, called SQL injection, can give
a malicious user full control over the database, compromising information privacy, and allowing the
user to modify database contents. Other attacks, such as a distributed denial of service, attempt to
make a server unavailable to its intended users.
SQL injection is done by including special characters in web input fields which trigger database
actions. For example, a web-based information system might contain a form for user account
creation, and SQL injection could be carried out through one of the form’s text fields. Because SQL
uses special characters (such as the semicolon) to interact with the database, it is possible for
malicious users to include these special characters in their input, effectively ending the intended
database queries and allowing the user to run their own database queries. Fortunately, SQL
injection can be guarded against by ensuring that all user-generated data is properly escaped
before adding it to a database.
Page 109
Another relatively common malicious server attack is a denial-of-service (DoS) attack. In general,
DoS attacks attempt to overload a server with such a large number of requests that the server
becomes unable to respond adequately. DoS attacks can be difficult to prevent because the
malicious requests are often indistinguishable from legitimate requests. There is no universal
defence against DoS attacks. Because of the wide range of types of DoS attacks, there exist a
large number of possible defence techniques.
9.2.3.2 Establish Rules for Clarity

The fundamental role of a transport data distribution system is to distribute intelligible data. To
make certain that all delivered data meets this standard, several initial actions must be taken:
 Establish strict system-wide data formatting rules


op V1
Plan for appropriate reporting of “null values”
 Write thorough and clear documentation.
Consistency in data formatting and the reporting of “null values” is important because of the
possibility that third party applications will consume transport data over web services. These
C -
applications parse data according to fixed rules, and inconsistent reporting of data can cause
y
C
values to be incorrectly interpreted. Inconsistent data formats also cause difficulty for human data
consumers.
C
Data Formatting Rules
Data distribution lies at the end of the road performance data supply chain, it is the last chance to
Q
ensure that all reported values follow consistent formatting rules. It may not be desirable to keep
data formats consistent throughout the entire data processing system. For example, storing
timestamps as epoch time (seconds since Jan 1, 1970) may make certain data processing
operations faster and more direct. However, epoch time is generally not considered to be “human
readable”. For most web-based information systems, a more intelligible time format should be
selected and enforced at the data distribution level. The international standard ISO 8601 specifies
“YYYY-MM-DD hh:mm:ss” as the preferred complete timestamp format, where each element is
defined as follows:
Table 11: Timestamp Codes
Code Interpretation
YYYY 4-digit year (e.g., 1985)
MM 2-digit month (e.g., 02)
DD 2-digit day (e.g., 06)
hh 2-digit hour (00-24)
mm 2-digit minute (00-59)
ss 2-digit second (00-59)
However, a localized timestamp format may be preferred to account for regional preferences.
Regardless of format, a clear policy on the inclusion of leading zeros in the timestamp should be
established and followed carefully. Data formatting rules should also include unit preferences that
should be followed universally by all data, which is distributed over web services or web-based
information systems. Units may be chosen based on regional preference.
Page 110
Reporting of Empty Values

In road performance data, it is common for an individual sensor to have no measurement to report
at a particular time. For example, a double loop detector may be configured to report speed and
occupancy every five minutes, however there may be some five-minute periods late at night during
which no vehicles were observed. If no value is reported at times like these, ambiguity arises:
either an error could have occurred or no vehicles were observed. This principle holds in data
distribution as well. A distinction should be made between “no observed value” and zero. One way
this can be done is by displaying “N/A” when no value exists.
Documentation
All distribution services provided by the agency should be accompanied by thorough online
documentation describing their use. This is particularly true for web services, which (lacking a GUI
op V1
and inline documentation) rely entirely on accompanying documentation to describe their use.
Documentation for web services should be organized at three levels: (1) The functions provided by
the web service, (2) The input variables required by each function, and (3) The output provided by
C -
each function. Examples of web service requests and responses should also be included to
y
C
demonstrate use of the system.
Web-based information systems should be documented thoroughly in a similar way. Typically, a

C
“Help” section of the web site is developed and serves as a resource for answers to common
questions, or instruction for further exploration. Similarly, including a means of contact for users to
Q
ask additional questions can help users to make the most of the available data.
For both web services and web-based information systems, it may be necessary to include a
glossary of terms. While this effort does not affect the quality or availability of the data, it is
valuable in that it raises awareness of services and increases the quality of applications built with
data from the web service.
9.2.3.3 Formalize Rules Governing Data Services

License Agreements
A license agreement must be established that formalizes the rules governing use of the distribution
systems. The license agreement is to be made between the agency providing the data and the
users. Users indicate their compliance with the license agreement by signing it electronically before
beginning to use the services.
The license agreement defines the rules by which the service can be accessed and the data can
be used. Important elements to include in a license agreement are:
 Explicit instructions not to use the agency’s copyrighted intellectual property (for
example, the agency’s official logo) in any applications that make use of the data. This
is often done so that only official data sources carry the agency’s official logo, branding
them as first-party data sources
 A clear warning that there is no guarantee of data availability or data accuracy. Without
this element, it may be possible for data consumers to bring complaints against the
data-provider for damages caused by inaccurate data or data outages. For example, if
a user of a third-party application that displays highway speeds is unable to reach a
hospital in time because of inaccurate travel time predictions, they won’t be able to lay
fault on the agency
Page 111
 A statement which allows the agency to change or discontinue service at any time
without warning
 A statement clarifying the ownership of the data. Some agencies retain official
ownership of transport data; others release ownership into the public domain, viewing
the transport data as being inherently communal.
The license agreement should ideally be concise and clear, to maximize the number of users who
read and understand it before agreeing to it and consuming data through the web service.
9.3 Operational
9.3.1 Collection
op V1
9.3.1.1 Monitor Sensor Health
The functionality of traffic sensors, also known as sensor health, should be monitored continuously.
Detector health indicates whether a sensor actually measures what occurs on the ground and
whether that data is sent to the controller and then on to the TMC. The health of each sensor in the
C -
system should be evaluated every day, once a day. It is recommended that the system be
y
C
maintained such that at least 70-75% of detectors provide quality data at any time. Some stations
may need to be maintained to a higher standard (working 95% of the time) if located at key
monitoring locations, such as near an expressway to expressway connection. These locations
C
should be prioritized for corrective maintenance.
Q
Sensor failure usually occurs at one of three system levels:
 Sensor (card is broken, loose, or failing; sensor is obstructed, etc.)

 Controller Cabinet (hardware or software is malfunctioning, power is down)
 Communications (modem, telephone line, etc. is broken).
The diagnosis of broken traffic sensors, and an estimate of the cause, can be automated in the
system by taking into account the quantity and quality of data samples received for each sensor.
This is similar to the Processing step described below for monitoring data feeds, but it is on a
sensor, controller, and communication line level. The following diagnoses can be implemented for
malfunctioning infrastructure:
 Sensor software configuration error or bad wiring: No samples received from a

sensor over a 24-hour period. Samples received for sensors attached to the same
controller and/or communication line
 Sensor insufficient data: Less than 60% of expected data samples received from a
sensor over a 24-hour period. Sensors attached to same controller or communication
line sent more samples
 Sensor stuck on: Certain percentage of sensor data samples have a presence above
0% (roadway) or count above 1 vehicle/30-seconds (ramps) over a 24-hour period. See
Periodic Processing section for suggested thresholds
 Sensor hanging on: Certain percentage of sensor data samples have a count of zero
and a non-zero presence. See Periodic Processing section for suggested thresholds
 Sensor hung on a value: Certain number of consecutive sensor data samples have
the same presence value. See Periodic Processing section for suggested thresholds
 Sensor card is off: Certain number of consecutive samples have zero presence and/or
flow. See Periodic Processing section for suggested thresholds
Page 112
 Controller power or communication down: None of the sensors attached to the

same controller report samples over a 24-hour period
 Malfunctioning communication line: None of the sensors attached to the same
communication line report sample over a 24-hour period.
Agencies must both perform corrective maintenance as soon as they find that equipment is broken,
as well as perform preventative maintenance. Preventative maintenance requirements are
described in the Periodic Collection section of this chapter. Processing routines should be able to
detect whether a corrective repair to a non-functioning detector was complete or successful by
comparing new data with the detector’s archived data.
9.3.2 Processing
op V1
9.3.2.1 Monitor Incoming Feeds
Each real-time data feed needs to be monitored on a daily basis. The agency should implement
automated alarms, which are software scripts that check the quantity of incoming data, then alert
appropriate staff of potential data feed issues that require further investigation. The following lists
C -
recommended alarms and triggers for each type of data feed:
y
C
C
Q
Page 113
Table 12: Sensor Alarms
Sensor Type Alarm/Trigger/Reaction
Traffic Usage Alarm 1: No data received

Infrastructure-based Trigger: No data samples received over a single hour
Sensors Reaction: Email staff every time this alarm is triggered
Alarm 2: Too few data samples received

Trigger: Less than 75% of expected data samples received over a
single day
Reaction: Email staff at the end of the day
Traffic Usage Alarm 1: No data received
Vehicle Identification Trigger: No data samples received over a four hour period
op V1
Sensors Reaction: Email staff every time this alarm is triggered

Trigger: Less than 25% of the average number of data samples for
that same day of week are received over a single day
C -
y
Condition Sensors
C
Alarm 1: No data received
Trigger: No data samples received over a 24 hour period
C
Incidents Alarm 1: No data received
Trigger: No incidents received over a 24 hour period
Reaction: Email staff at the end of the day.
Q

Trigger: Less than 25% of the average number of weekly incidents
is received for a given week.
Reaction: Email staff at the end of the week
Weather Alarm 1: No data received
Trigger: No data samples received over a 24 hour period
Reaction: Email staff at the end of the day.
9.3.2.2 Monitor Processing System

The processing system can also be monitored through automated alarms. A software script that
indicates whether the processing task was successful or whether it failed should accompany every
processing task. An alarm should be set up to email the appropriate staff on a daily basis of which
processes failed. The following processes should be logged, and the results summarized in a
nightly email to appropriate staff:
 Writing to raw database tables from data feeds

o Raw point
o Raw link set
o Raw condition traffic
o Incidents
o Control
o Weather
 Calculations
o Speed from count and presence (point data)
o Travel times from vehicle identifications (link set data)
 Quality Control
o Identification of bad point data
Page 114
o Filtration of bad link set data

o Filtration of bad condition traffic data
 Imputation
o Imputation for bad point data
 Aggregation
o Point, link set, and condition traffic data to the five-minute temporal level
o Point, link set, and condition traffic data to the hourly temporal level
o Point and condition sensor data to the station level
o Point and condition traffic data to the daily temporal level
o Point and condition traffic data to the monthly temporal level.
Alerts should also be established to inform staff of the following situations:
op V1
1. Network Outage
2. Disk space Full
a. Alert when disk space is 85% full (warning)
b. Alert when disk space is 95% full (critical)
3. RAM Memory Overload
C -
y
a. Alert when there the number of available Bytes falls below 5 MB
4. Database read error
C
5. Database write error.
C
Over time, thresholds should be modified from those suggested here to ensure that only useful
alarms are being generated and distributed. It is recommended that the database logs be
evaluated to identify a day with a baseline level of performance. Then, alarms can be adapted to
Q
report when conditions are 10% (warning) and 20% (critical) worse than those observed on the
baseline day.
9.3.2.3 Distribution
The day-to-day activities of a transport data distribution system are primarily aimed at maintaining
the acceptable service quality (in terms of reliability, responsiveness, and security) that was initially
designed. The most widespread and efficient way to do this is through the use of automated
quality-checking routines.
9.3.2.4 Monitor Outgoing Feed Performance

A critical activity in the day-to-day operations of a transport data distribution system is to verify that
the end users are able to access the data. To do this, each data distribution platform should be
monitored regularly by an automated process. A custom monitoring process can be written to
perform a simple test query, either to a web service or to a web-based information system, at a
regular interval. If the query returns an error or an unexpected result, that distribution channel
should be flagged and its error logged. Alternatively, many open source IT infrastructure monitoring
tools exist that can be easily implemented to monitor the availability of web services and generate
automated reports.
A good practice is to use an online dashboard to display the status of the agency’s data distribution
channels. With the health of each distribution channel monitored and displayed in this way,
problems can be identified more easily and quickly.
The specific action taken to correct a problem will depend on the particular error detected.
Sophisticated monitoring tools commonly provide detailed diagnostic information on server metrics
such as:
Page 115
 Disk usage
 Network health
 Response latency
 I/O times
 Relative server load.
9.3.2.5 Collect Web Usage Data

Similarly to the monitoring of the channels’ availability, usage data can be monitored through
automated processes. Collecting usage data is important for identifying ways to improve
distribution channels in the longer term.
Feedback
op V1
A simple but effective way to identify potential areas of improvement in the data distribution system
is through the collection of user feedback. In web services, documenting an email address for
comments, questions or suggestions can do this. In web-based information systems, a particular
page might contain a feedback form that can be completed by users. Web-based information
C -
systems, because of their full GUI, can also collect feedback in the form of numeric ratings, which
y
can be processed and displayed on an internal dashboard automatically.
C
Automated Web Usage Data
C
Automated usage data can also be collected. Private services like Google Analytics offer free,
detailed statistics on web usage. Some quantities measured by Google Analytics include:
Q
 Information on individual visits. This includes number of visits, number of unique

visitors, number of page views, average time on site, bounce rate, and percentage of
visits that are from new visitors
 Information on individual visitors. This includes demographic information such as
language, location, browser, operating system, internet service provider, and device
type (e.g., mobile)
 Information on user activity. This includes users engagement on social networks,
pages visited, flow of visits through individual pages, and popularity of individual pages
or services
 Information on sources of traffic. This includes the percentage of users who arrive
directly, via web search, or through links on other websites, as well as the referring
search terms and referring sites.
Page 116
op V1
C -
y
C
Figure 18: Dashboard of an Automated Monitoring System, Google Analytics
C
9.4 Periodic
9.4.1 Collection
Q
9.4.1.1 Preventative Maintenance

Periodic site, sensor and communications inspections are required parts of preventive
maintenance. The following general preventative maintenance schedule can be used to organize
preventative maintenance. Actual maintenance schedules should be based on sensor-specific
needs, environmental conditions and agency resources. Older sensor equipment may require more
frequent maintenance in order to maintain data quality.
 Daily: Self-test pass/fail status, sensor status report

 Monthly: Inspect general condition of site
 Quarterly: Inspect mounting, hardware, status indicators, guy lines, sensor ground
connections, flexible conduits, cables and connections, exterior cabinets, cabinet exterior
maintenance, and overall cleanliness
 Each Maintenance: Update maintenance log, schedule next maintenance
 After any severe weather event: Inspect sensor for wind damage, and flooding or soil
erosion; fill soil erosion.
Corrective maintenance should usually be prioritized before preventative maintenance.

Maintenance may be combined with management of other equipment such as VMS
troubleshooting, repair, verification, acceptance of repair or referral, reporting on repair progress
and network status. Maintenance should be designated for in-house staff or contracted out;
detailed requirements and activities should be detailed for each type of equipment.
Maintenance Log
A digital maintenance logbook should be used to record all preventive and corrective maintenance
performed on the sensor. Problems observed and their resolutions should be recorded. Printouts of
diagnostic results should be with the log, and are helpful for documenting trouble. Every log entry
Page 117
should be dated and signed. A date for the next scheduled preventive maintenance may also be
entered in the log.
9.4.1.2 Evaluate New Sensor Technologies

The agency should evaluate its data collection technologies on a yearly basis, as costs change
over time and new technologies are constantly being developed. The steps involved in this process
are:
 Identify performance measurement goals for the year

 Identify new technologies not previously evaluated that can support performance measure
goals
 Update costs for technologies previously evaluated, and assess their ability to meet
op V1
performance measure goals
 Develop deployment plan that can support performance measurement goals. This may
involve replacing sensors and/or supplementing the existing network to fill in gaps.
9.4.2 Processing
C -
y
C
9.4.2.1Evaluate System Performance
On a yearly basis, the agency should investigate the performance of the system by evaluating the
C
following metrics:

Q
Usage statistics on most commonly queried tables and fields

 Response times to common queries
 Resource allocations for common queries.
By evaluating these statistics, staff can prioritize components, processes, or queries to optimize.
Queries can be optimized by either modifying the query, implementing a new database table to
pre-summarize retrieved data, or indexing the queried tables to lower response times.
9.4.2.2Update Diagnostic Thresholds

Diagnostic thresholds are run on point data to detect when infrastructure-based sensors are
broken, so that their data can be imputed. These thresholds should be evaluated and adjusted
every year. As a starting point, the following thresholds are recommended to mark an
infrastructure-based sensor as bad:
Urban Highway Sensors

 2% of samples have zero count and non-zero presence; or
 20% of samples have presence greater than 70%; or
 50% of samples have zero presence and non-zero count; or
 500 consecutive samples have the same presence value; or
 59% of samples have zero presence.
Rural Highways Sensors

 2% or more samples have zero count and non-zero presence; or
 20% or more samples have presence greater than 70%; or
 50% or more samples have zero presence and non-zero count; or
 500 or more consecutive samples have the same presence value; or
Page 118
 90% or more samples have zero presence.
Arterial Sensors
 99.5% of samples have zero count
 2% of samples have counts of greater than 20 vehicles per 30 seconds.
Based on agency experience in the first year of performance monitoring, these thresholds can
be refined, then revisited on a yearly basis. Alternative thresholds can be developed and
implemented based on the following categories:
 Station-specific thresholds, based on observed deviations from the average values
measured over the first year
 AADT-specific thresholds, to refine the urban versus rural framework
op V1
 For arterials, turning-movement specific thresholds, to account for the fact that turn lanes
likely have more samples with zero count than through-lanes.
9.4.2.3Update Imputation Parameters

C -
y
Imputation parameters need to be initially established and then updated with a certain frequency.
C
Recommendations for calculating and updating imputation parameters within each regime are as
follows:
C
Local Neighbour
Q
Imputation by local neighbour is performed as follows:
Where (i,j) is a pair of sensors, q is count, k is presence, t is the time interval, and α0, α1, β0, and
β1are coefficients estimated through linear regression.
The coefficients used in local neighbour imputation should be estimated for each five-minute period
of each day of the week between every neighbouring sensor pair using at least ten good data
samples. The parameters should be re-estimated every year.
Global Neighbours
Imputation by global neighbours is performed as follows:
Where (i,j) is a pair of sensors, q is count, k is presence, t is the time interval, =0 if iand jare at
the same spot on the roadway, 1 if they are upstream/downstream, lis the lane number of the
sensor, and the α*0, α*1, β*0, and β*1 are coefficients estimated through linear regression.
The coefficients used in local neighbour imputation should be estimated for each five-minute period
for each day of the week between each possible combination of sensors by freeway location (same
location versus upstream/downstream) and lane number. Each parameter should be generated
using at least ten data samples from each of at least twenty different sensor pairs in the given
configuration. Each parameter should be re-estimated every year.
Page 119
Temporal Medians
Imputation by temporal median is performed by assigning imputed count and presence values at a
sensor as the median of all the count and presence values measured at the sensor over the most
recent set of weeks.
For each sensor, the temporal median should be estimated for each five-minute period for each
day of the week, and should be estimated from the most recent ten data samples that were directly
observed. As such, each temporal median parameter should be re-estimated on a weekly basis.
Cluster Medians
Imputation by cluster median is performed by assigning imputed count and presence values at a
sensor as the median of count and presence values measured over that sensor’s cluster over the
op V1
most recent weeks.
The agency should define at least two clusters for each facility type, assigned according to whether
they experience higher traffic volumes in the morning or in the evening. Clusters can be further
C -
refined through cluster analysis, to define the optimal set of clusters for each facility type. Each
y
cluster’s median count and presence values should be estimated for each five-minute period of
C
each day of the week, and should be estimated using at least ten good data samples from each
sensor in the cluster. Clusters definitions should be re-evaluated and medians re-estimated on a
C
yearly basis.
9.4.2.4 Update Vehicle Length Assumptions

Q
The assumptions of the average vehicle length used to compute speeds for sensors that do not
directly measure it should be updated on a yearly basis. The requirements for the average vehicle
length assumptions (G-factors) are as follows:
 G-factors should be sensor- (i.e. lane- and location-) specific

 G-factors should be estimated for every five-minute period of every day of the week.
The optimal way of measuring G-factors is to use vehicle length data collected from nearby WIM
stations. Alternatively, if there are sensors nearby that directly measure count, presence, and
speed, G-factors can be back-calculated and applied to the sensors that do not report speed. The
final, most-costly alternative is to estimate vehicle length values during site visits.
9.4.3 Distribution
To maintain a level of stability in the system and avoid user confusion, significant modifications to
the distribution channels should generally occur no more frequently than monthly.
9.4.3.1Evaluate and Update Distribution Channels

It is likely that on a monthly or annual basis, subtle changes in the data needs of users will be
observed. Users of a web service or web-based information system, whether internal or external to
the agency, will be more engaged and productive when using a system that is perceived as being
under active development. Changes to distribution systems should be made based on the
performance and usage data collected by the agency, as well as the current state of the road
performance monitoring practice.
Page 120
Performance
While emergency measures such as restoring a distribution system from a service outage should
take place immediately, longer-term improvements to system performance should be considered
on a semi-annual or annual basis. Decisions on how to improve performance should be based on
information gained from the monitoring of outgoing feed performance. The latest technologies
should be used to ensure that the system performs at a high level. Activities that should be
considered for improving the performance of a distribution system include:
 Optimizing data distribution software. Typically, software optimization should be

prioritized over hardware optimization. For transport data distribution systems,
common software optimization activities include indexing databases, simplifying
database queries, and refactoring inefficient data processing routines.
op V1
 Scaling hardware up. If performance monitoring of an outgoing feed reveals
performance bottlenecks in processing power or memory capacity, it may be
appropriate to “scale up” the server hardware. Adding additional processors,
upgrading processor speed, adding additional memory, and upgrading memory
C -
speed and capacity are all common ways to improve distribution system
y

performance.
C
Scaling hardware out. “Scaling out” is another technique to improve performance
and involves distributing core activities across multiple nodes. If core server
C
functions (e.g., the processing of incoming data feeds and the handling of user
requests) are moved to separate servers (or to separate partitions on the same
Q
server), catastrophic system failures become less likely. When hardware is “scaled
out”, a single process is less likely to cause the entire system to fail, leading to
reliability improvements.
Usage
As road performance monitoring techniques continue to evolve, new features will need to be
implemented into the distribution system. For web-based information systems, this could mean
calculating different performance measures, developing new transport system reports, or refining
the control users have over existing reports. For web services, this could be the implementation of
a new function to access a previously unavailable type of information. Additional features such as
these should be added to keep pace with the state of the practice, as well as in response to user
feedback.
As user activity across the distribution system is tracked, some features of the system will emerge
as more or less popular than others. In general, resources should be focused on improving the
most-used functions of a data distribution system. Web-based information systems can improve
their usability by visually highlighting the most popular functions while featuring unpopular functions
less prominently.
Demographic information should also be considered to ensure that the distribution system fully
supports the needs of its users. For example, a web-based information system should be tested on
all of the browsers from which it receives a significant number of requests. Similarly, support for
multiple languages may be considered depending on demographic information.
Page 121
9.5 Triggered
9.5.1 Collection
9.5.1.1Update Database
Whenever relevant usage, location and condition data have been collected manually or otherwise
acquired, the performance monitoring database should be updated. These updates can be
scheduled or else performed each time the data is collected, depending on how essential the data
is to continuous performance management. Usage data should be updated whenever each of the
following occurs:
 Manual traffic counts have been collected.
op V1
The GIS network/location data should be updated whenever any of the following occur:
 New location data has been collected by the agency, particularly linear referencing data
 Construction activity has resulted in a change to spatial or linear referencing data
 C -
y
Other location data such as topology or road attributes have changed

C
New sensors have been installed or removed
 Another agency has updated data that is part of the GIS database
C
 A map provider has updated data that is being used by the agency.
Condition data should be updated whenever the following occurs:

Q
 Road condition has been collected using manual or digital methods.
9.5.1.2Continuity during Construction

Construction activity often results in increased congestion, which needs to be monitored. However,
sensors are often removed during construction. When construction is planned, temporary detection
systems should be required when permanent systems are removed. Sensors can be combined
with other traffic management devices such as CCTV and CMS to substitute for the permanent
detection system. Construction schedules should be coordinated with the TMC; TMC staff should
update the database with construction information and monitor data availability, particularly when
data for a decision support system such as ramp metering is involved.
Construction contracts should also require replacement of sensors that are removed during
construction; persons capable of proper sensor installation and field-testing will be needed.
Operations and planning staff should be involved in the planning and design phase of construction
projects. There may be an opportunity to incorporate improved technology and more efficiently
coordinate the requirements of multiple users.
9.5.1.3Add Sensors
When new roadway is constructed, the appropriate new sensors should be installed. Corridor-wide
or regional sensor plans should prioritize future deployments and provide a basis for incorporating
these deployments into capital projects. The construction schedule needs to allow for testing in the
middle of the project to make sure that communication with the TMC and other technology issues
are identified and solved before the end of the project. Installing, turning on, and testing
electronics at the end of a project leaves no room for error or problem solving (troubleshooting)
unless it is built explicitly into the project schedule.
Page 122
Construction projects need to include funding for sensor installation, communications

infrastructure, and modifications (data integration) at the TMC. Detection deployment may require
hardware or software upgrades at the TMC to handle additional detection stations. Integration of
data into the TMC is essential if the information is to be useful.
9.5.2 Processing
9.5.2.1Update Metadata
Metadata should be collected and updated every time there is a change to the roadway or
deployment network. Metadata should be collected and updated within three days of the following
changes:
op V1
Addition of a sensor
 Removal of a sensor
 Deactivation/reactivation of a sensor
 Movement of a sensor

C -
Change in the number of lanes monitored by a sensor
y

C
Opening of a new section of roadway.
9.5.2.2Create New Database Tables

C
As the system matures, new performance measure and reporting requirements will necessitate
changes to the new database, to ensure that processing is done accurately and needed
Q
information can be retrieved quickly. New database tables should be added and populated in
response to the following triggers:
 Need for a new performance measure in reports

 Need for a new level of spatial granularity in reports
 Need for a new level of temporal granularity in reports.
9.5.3 Distribution
Since the transport data distribution system is a primary platform for communication from the
transport agency to the public, it is well served to distribute general news and advisories. This type
of information is traditionally disseminated through the media, however as more people continue to
receive transport information through web-connected devices, it makes sense to broadcast these
announcements over the data distribution system as well. This can include the following types of
information:
 Planned construction activities

 Significant traffic incidents
 News on transport planning activities.
Page 123
CITED REFERENCES
1. Department of Municipal Affairs & Transport .Congestion Management Policy and
Procedures. Abu Dhabi :, 2013.
2. —. Land Surveying and Mapping Guide. Abu Dhabi :, 2013.
3. Department of Municipal Affairs & Transport.Pavement Design Manual. Abu Dhabi : s.n.,
2013.
4. FHWA.Traffic Monitoring Guide. 2001.
OTHER REFERENCES
op V1
American Association of State and Highway Transportation Officials. Mechanistic-Empirical
Pavement Design Guide, 2008.
C -
y
American Public Works Association. ASTM D6433-09: Standard Practice for Roads and Parking
C
Lots Pavement Condition Index Surveys, & Asphalt/Concrete Distress Manuals, 2009.
Chin, C., T. Martin, P. Robinson, T. Thoresen, and C. Evans. Network Performance Indicators –
C
Next Generation. Austroads Publication No. AP-T176/11. Austroads, 2011.
Q
Crow, M., M. Halladay, S. Martinovich, D. DeLucia, D. Harkey, D. McNamara, J. Lacy, B. Serian,

M. Griffith, S. MacGregor, B. Ellison, and J. Kevin. Traffic Safety Information Systems in Europe
and Australia. FHWA Report No. FHWA-Pl-04-010. FHWA, US Department of Transportation,
2004.
DUK Department for Transport Publications. http://www.dft.gov.uk/publications. Accessed

September 16, 2011.
Europe-Wide Traffic Management Services. http://www.easyway-its.eu/. Accessed September 14,

2011.
Gleave, S., Mid-term evaluation of the TEN-T Programme (2007-2013). DM 28 – 0/110. European
Commission Directorate General Mobility and Transport, 2011.
Hajek, J. and Selezneva, O. “New Developments in the Use of Traffic Data for Pavement
Management,” 5th International Conference on Managing Pavements, 2001.
Highway Performance Monitoring System Field Manual, FHWA, US Department of Transportation,

2010.
HMPS Reassessment 2010+. FHWA, US Department of Transportation, 2008.
Intelligent Transportation Systems – Road-Deployment on the Trans-European Transport Network

(TEN-T) http://ec.europa.eu/transport/its/road/deployment_en.htm. Accessed September 16, 2011.
Intelligent Transportation Systems in Action: Action Plan and Legal Framework for the Deployment
of Intelligent Transport Systems (ITS) in Europe. European Commission, 2011.
Jeddah Strategic Plan. Municipality of Jeddah, 2009.
Page 124
REFERENCES First Edition -December 2016
Karl, C. Tools that Impact Network Performance: Measurement and Evaluation Tools. Austroads
Publication No. AP-R310/01. Austroads, 2007.
Kaysi, I., H. Al-Hathlool, S. Al-Kathani, M. Whiteway, and F. Darwish. Prioritization of National

Road Projects in Saudi Arabia: Framework and Implementation. Presented at 12th World
Conference on Transportation Research, Portugal, 2010.
Land Transport Authority Annual Report 2009/10. Land Transport Authority, Republic of Singapore,
2011.
Litzka,J., B. Leben, F. La Torre, A. Weninger-Vycudil, et al. The Way Forward for Pavement
Performance Indicators across Europe. Publication 08/32, European Cooperation in Science and
Technology, 2008.
op V1
Luk, J., and G. Kazantaidis. Implementation of National Network Performance Indicators: First
Round Results. Austroads Publication No. AP-T122/09. Austroads, 2009
MacDonald, D., C. Yew, R. Arnold, J. Baxter, R. Halvorson, H. Kassoff, M. Meyer, K. Philmus, J.
Price, D. Rose, M. Walton, and W. White Transportation Performance Measures in Australia,
C -
Canada, Japan, and New Zealand. FHWA Report No. FHWA-PL-05-001. FHWA, US Department
y
of Transportation, 2004.
C
Mirshahi, M, Obenberger, J., Fuhs, Charles, Howard, Charles, Krammes, R., Kuhn B., Mayhew,
C
R., Moore, M., Sahebjam, K., Stone, C., Yung, J. Active Traffic Management: The Next Step in
Congestion Management. FHWA Report No. FHWA-PL-07-012. FHWA, US Department of
Q
Transportation, 2007.
Non-Federal Applications of the Highway Performance Monitoring System (HPMS).

http://www.fhwa.dot.gov/policyinformation/hpms/nahpms.cfm. Accessed September 9, 2011.
Northwest Technology Transfer Center. Local Agency Pavement Management Application Guide.
OneMotoring Fast Track to Complete Motoring. http://www.onemotoring.com.sg/. Accessed

September 14, 2011.
Saher cuts down traffic violations by 70% in Jeddah.

http://www.zawya.com/story.cfm/sidZAWYA20110508031448/Saher_cuts_down_traffic_violations_
by_70_in_Jeddah. Accessed September 12, 2011.
Saher is is an automated traffic control and management system.http://www.saher.gov.sa/.

Accessed September 16, 2011.
Saudi Arabia to implement a new transportation

policy.http://www.saudiembassy.net/latest_news/news04030803.aspx. Accessed September 12,
2011.
Singapore Land Transport Statistics in Brief 2009. Land Transport Authority,

Republic of Singapore, 2010.
SmartRoads: Connecting Communities.

http://www.vicroads.vic.gov.au/Home/TrafficAndRoadConditions/HowWeManageTraffic/Smartroad
s/. Accessed September 14, 2011.
The Ninth Development Plan (2010-2014). Kingdom of Saudi Arabia Ministry of Economy and
Planning 2010.
Page 125
Traffic England. http://www.dft.gov.uk/publications. Accessed September 16, 2011.
Traffic Monitor 2009/2010. VicRoads Victoria State Government, 2011.
Troutbeck, R., M. Su, and J. Luk. National Performance Indicators for Network Operations.
Austroads Publication No. AP-R305/07. Austroads, 2007.
VicRoads — Digital and Mobile Applications.

http://www.vicroads.vic.gov.au/Home/Moreinfoandservices/DigitalAndMobileApplications/
Accessed September 14, 2011.
VicRoads — How we manage traffic.

http://www.vicroads.vic.gov.au/Home/TrafficAndRoadConditions/HowWeManageTraffic/ Accessed
September 16, 2011.
op V1
Vis, M.A. and Van Gent, A.L. (Eds.) Road Safety Performance Indicators: Country Comparisons.
Deliverable D3.7a of the EU FP6 project SafetyNet, 2007.
C -
y
C
C
Q
Page 126

TR 515 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TR 515 PDF

Uploaded by

Copyright:

Available Formats

Q

Document No: TR-530

2 USAGE DATA REQUIREMENTS ............................................................................................ 7

2.5 Usage Monitoring Strategy ............................................................................................. 18

4.2.2 Transport Network Assumptions .............................................................................. 35

6.3.2 Condition Data ......................................................................................................... 55

7.2.5 Broadcast Radio/TV ................................................................................................. 75

8.2.5 Decision Support ..................................................................................................... 88

9.4.1 Collection ............................................................................................................... 117

Table 8: Event Usage Data Tables ............................................................................................. 103

ABBREVIATIONS AND ACRONYMS

HOV High Occupancy Vehicle

VHT Vehicle Hours Travelled

1.2 Purpose and Scope

Figure 1: Data Scope

1.4.1 Strategic and Requirements Section

1.4.1.1Usage Data Requirements

 Measure. The fundamental quantities that define system performance.

1.4.1.2 Condition Data Requirements

1.4.1.3 Location Data Requirements

1.4.2 Tactical and Process Section

Figure 2: RPMS Systems Diagram

1.4.3 Operational and Implementation Section

2 USAGE DATA REQUIREMENTS

2.1.2 Traffic vs. Events

2.2 Traffic Data Typology

 Measure: The fundamental quantities that define system performance

 Points. Point measures occur at a point on a given link

o Speed. The instantaneous velocity of vehicles at the point

2.3 Event Data Typology

These high-level event categories are:

 Control. Signals and other devices that control vehicles movements

2.3.5 Special Events

2.4 Traffic Data Technologies

Figure 4: Traffic Data Technology Tree

2.4.1 Infrastructure Sensors

2.4.1.1 Loop Detectors

2.4.1.2 Wireless Magnetometer Detectors

2.4.1.4 Video Analytics

o Piezoelectric Sensor. In this technology a sensor is embedded in the pavement and

2.4.2 Vehicle-Based Sensors

This technology is advantageous in that it is accurate, low-cost, and portable. A drawback,

License Plate Readers

Vehicle Signature Matching

2.4.2.2Automated Vehicle Location

Global Positioning Satellite

As an example of the future of this technology, Connected Vehicle California, an organization

Cellular Tower Triangulation

2.5 Usage Monitoring Strategy

2.5.2 Key Data Items

Table 1: Traffic Data Items

Unit Data Item Priority Notes

selected key locations.

Table 2: Event Data Items

Unit Key Attributes Priority Notes

Control events signals (ramp meters) or arterial signals (at

3 CONDITION DATA REQUIREMENTS

3.1.1 Pavement Management Context

represents one of the most capital-intensive recurring investments most roadway-oriented

3.1.2 Traffic vs. Pavement Monitoring

3.2 Condition Reporting Requirements

3.2.1 Traffic Monitoring Data

management-related activities should be aware of the capabilities of such systems in to effectively