~ronrpn. &,.-A Vol. 19A. No. I. Pp. 73-82. 1985 0191-2607185 53 oO+ .

@I
Rmed III the U.S.A. 0 1985 Pergamon FTcssLtd.

PERFORMANCE EVALUATION FOR BUS TRANSIT

GORDON J. FIELDING,
TIMLYNN T. BABKSKY and MARY E. BRENNER
School of Social Sciences and Institute of Transportation Studies, University of California,
b-vine, CA 92717, U.S.A.

(Received 4 May 1984; in revised form 20 September 1984)

Abstract-Performance analysis has gained increased attention now that transit organizations are emphasizing
efficiency objectives. Debates over both the number of indicators required and their method of selection are
reviewed. Research upon which this paper is based uses FY 1980 Section 15 data to first identify and then test
a set of performance indicators which are useful for evaluation of fixed route, motor bus transit. Four parallel
data sets, based on transformations of the original data, and several exploratory factor analyses were used to
detect the underlying structure of the data. Rigorous testing verified that the structure represented the most
salient performance dimensions. A small subset of seven performance indicators was identified and tested as
representative of these underlying dimensions. These indicators can be used together or individually to assess
transit performance for a single system or for cross-sectional comparisotis.

After two decades of expansion, the American transit
industry has begun to consolidate its rejuvenation and
emphasize more efficient production of service. In part,
this has resulted from the desire of federal, state and
local governmental sponsors to evaluate relative per-
formance in order to account for public assistance, and
in part, it has come from a desire by managers to improve
performance within their agencies.
Many conference sessions, workshops and profes-
sional papers have been devoted to the merits of different
methods of performance analysis and the merits of spe-
cific performance measures. Miller (1980) has summa-
rized this literature and suggested the need for a simple
framework using three or four measures of efficiency
and effectiveness. This paper develops such a framework
and advances the thesis that there exists a consistent set
of performance concepts relevant to fixed route transit
operations. And further, that a small, unique set of in-
dicators can be selected from these concepts for the mea-
surement of transit performance.
Not all researchers agree that multiple measures of
transit performance are needed. Several authors have
advanced claims that a single measure is sufficient. Tul-
ley and Andersen (1981) examine transit performance
from the perspective of a fii seeking to maximize rid-
ership within allowable deficit limits. They suggest that
deficit per passenger is the appropriate governmental
objective for transit. Nash (1980) prefers cost per pas-
senger or passenger mile when analyzing alternative in-
vestments for management. Patton (1983) has suggested
that transportation statistics on performance can be in-
tegrated about the indicator of cost per passenger. Each
assumes that the overarching goal for transit is trans-
porting passengers for the minimum cost. Kneafsey (1975)
refers to this as efficiency-in-the-small or cost min-
imization to the firm.
Allocative efficiency-what Kneafsey calls effi-
ciency-in-the-large, is more suitable for transit per-
formance analysis. This defines efficiency in terms of
resources used to produce service. By this definition,
73
efficiency is a statement about the achievements of an
agency in transforming a set of inputs into a set of out-
puts. Others have used technological efficiency for
this concept in public sector analysis in contrast to eco-
nomic efficiency (Burkhead and Herrigan, 1978).
Confusion over defining efficiency has reduced the
value of many studies. Public transit agencies cannot
focus on a single objective function as they must respond
to the objectives of various publics. Advocacy of
single measures of transit performance integrating effi-
ciency and effectiveness has not aided transit perform-
ance. Separate measures provide more useful results.
These can be grouped into indicators of system efficiency
and indicators of system effectiveness. Efficiency indi-
cators define the relationship between resource input and
produced output. They include indicators of overall cost
efficiency, labor utilization and vehicle utilization. Ef-
fectiveness indicators generally reflect the ability of transit
operations to meet certain goals. These include service
utilization, service quality and accessibility of service.
PERFORMANCE EVALUATION USING SECTION 15 DATA
Section 1.5 of the Urban Mass Transportation Act of
1964, as amended, provides for the collection of a unique
set of comparable transit statistics by requiring all transit
applicants for federal operating assistance to report a
uniform set of information about their system. Regula-
tions to implement the legislation were published in the
Federal Register on September 17, 1975. The methods
used originated from Project FARE (Uniform Financial
Accounting Reporting Elements) which the Urban Mass
Transportation Administration (UMTA) and the transit
industry had initiated in 1972. The reporting system is
designed to meet the needs of: (1) individual transit sys-
tems for comparing their performance with other transit
systems with similar characteristics; (2) transit industry
associations for monitoring industry performance and (3)
Federal, State and local government agencies for transit
industry analysis and for financial assistance program
74 G. J.
administration. Data is sent to UMTA and edited and
compiled by the Transportation Systems Center. Copies
are available in magnetic tape version or as hard copy
in annual editions of the National Urban Mass Trans-
portation Statistics.
The first year of Section 15 reported statistics (FY
1979) was used by Anderson and Fielding (1982) to test
the performance concept model developed by Fielding,
Glauthier and Lave (1978). The results were not satis-
fying. Serious questions were raised about the validity
and completeness of the first years data, although the
method of using factor analysis to identify the underlying
structure in the data held promise. If the data set could
be improved, mote rigorous factor-analytic solutions could
be applied on different versions of the data to test the
validity of the performance model.
This paper analyzes data from the second year (FY
1980) of reported statistics (U.S. Department of Trans-
portation, 1982). It replicates the methods and revises
the results from analysis of the first year (FY 1979)
statistics. Because the thesis that a highly consistent set
of performance concepts exists and that they can be
represented by a small, unique set of performance in-
dicators remains controversial, emphasis is given to ex-
plaining how:
(i) variables were selected, verified and groomed;
(ii) performance indicators were selected and calculated
in alternative ways to minimize bias;
(iii) different methods of factor analysis were used to
explore the structure of performance concepts;
(iv) tests were used to verify the structure of perform-
ance concepts.
Categories of indicators
A wide variety of Section 15 statistics were evaluated
as possible performance indicators. Three categories of
statistics-service inputs, service outputs and service
consumption-provided the framework to organize the
much larger set of data. Figure 1 portrays the organizing
framework originally advanced as a performance concept
model by Fielding et al. (1978). Cost-efficiency indi-
cators measure service inputs (labor, capital, fuel) to the
amount of service produced (service outputs: vehicle
hours, vehicle miles, capacity miles, service reliability).
Cost-effectiveness indicators measure the level of service
consumption (passengers, passenger miles, operating
revenue) against service inputs. Finally, service-effec-
tiveness indicators measure the extent to which service
outputs are consumed.
Data preparation
The Section 15 data base includes information from
15 separate required forms and three voluntary levels of
reporting. At the required level of reporting, information
is available for revenues, subsidies, expenses, wages and
benefits, service schedules, maintenance performance,
energy consumption, accidents, employee counts, ser-
vice supplied and service consumed. For many purposes
the level of detail is excessive and key pieces of infor-
mation are often fragmented into tiny units which must
be aggregated before analysis. Many of the variables
relevant to the performance concept model refer to dif-
FIELDINGef al.
ferent units of analysis; e.g. financial data is fiscal year
based while service consumed and service supplied in-
formation is reported for an average weekday, Sat-
urday and Sunday. Further, revenue data and subsidy
data are not disaggregated by mode; they are reported
for the entire system, This requires the development of
weighting strategies for single mode performance anal-
ysis. Data organization, verification and correction tech-
niques used in this research are described in more detail
elsewhere (Fielding et al. 1984).
The first task of analysis entailed compilation and
assessment of each of the basic variables that would later
be included as performance indicators. Some of the basic
variables available in the Section 15 data base are better
than others; some present unique problems for multi-
variate statistical analysis. Revenue and expense data are
fairly complete while operators wage, state and local
subsidies, maintenance and passenger data present par-
ticular problems. Systems differ as to how local versus
state subsidy is defined; the same funding source may
be designated differently by several transit systems.
Maintenance data is often suspect; systems using outside
contract maintenance labor often claim zero maintenance
personnel and cost. Finally, passenger data is the weakest
part of the Section 15 data base; between 13% to 25%
of the data is missing.
The problems identified here have not gone unnoticed.
Suggestions from industry personnel and academic re-
searchers are continually solicited and evaluated to im-
prove the data base. Although agreement on definitions
and methods for refining the data collection process has
been slow in coming, the section 15 data base is becom-
ing increasingly sound. Both the quality of the data and
the consistency with which it is reported are improving.
Missing data procedures
Missing information in Section 15 data poses a unique
analytical problem. Both valid zeros and no informa-
tion reported codes are represented by zeros. Whenever
possible, other information available in the data base
was pieced together to provide for missing data or to
distinguish between valid zeros and failure to report.
Missing values encountered at any point in the com-
putation of basic and ratio variables and during the mul-
tivariate statistical procedures cause a snowball effect
of missing information. If any case is missing even one
piece of information it is automatically deleted from the
computations and subsequent analyses. The missing val-
ues problem has a cumulative effect as cases are dropped
from the analysis. Thus, from a total of 304 transit sys-
tems running fixed route, motor bus service, only two-
thirds of the cases-198 systems-had enough infor-
mation available to use in the final analyses.
SELECTING PERFORMANCE INDICATORS
There were many potential performance indicator ra-
tios available from the Section 15 data base. In selecting
the set of performance indicators to be used, the data
had to include variables that would relate to the con-
ceptual model, i.e. those that would best represent the
three categories of performance concepts-cost-effi-
 Performance evaluation for bus transit
Service
Inputs
Labor
Capital
Fuel
Service- Effectiveness
Service
outputs
Vehicle Hours
Vehicle Miles
Capacity Miles
Fig. 1. Framework for a transit Performance concept model.
ciency, cost-effectiveness and service-effectiveness.
Particular attention was given to the availability and re-
liability of the Section 15 reported data from which the
ratios would be calculated. As noted, some of the Section
1.5 data variables were more complete or more reliable
than others.
Table 1 lists the initial set of 48 variables selected for
multivariate analysis. The variables are organized under
the performance concept to which they relate. This set
of 48 variables in most cases (other than passenger data)
represents the most complete, generally reliable and non-
redundant performance indicators available in the Sec-
tion 15 data set.
Variables based on revenue-capacity miles were not
included because of the inconsistency in measurement
of that variable across systems. Ratios based on popu-
lation data were not included because available popu-
lation information reflected total urban population rather
than service area population. Otherwise, performance
indicator ratios comparable to the 1979 data analyses
were selected for use. This facilitated comparison with
previous results and identification of shifts due to the
better-collected, cleaner and more complete data.
Distribution of data
One of the first tasks for exploring the data set was
to search for extreme outliers and to remove them from
the analysis. Extreme outliers could force the analysis
75
Service
Consumption
Passengers
Passenger Miles
Operating Revenue
to focus on the inflated variance due to the presence of
an outlier, rather than the more true-to-data variance
present across the range of the other cases. Four cases
were dropped from the analysis because of the outlier
quality of their reported statistics.
The next task was to check the univariate descriptive
statistics for each of the selected performance indicator
ratios to evaluate the distribution of the case values across
the variable range. Two descriptive statistics that provide
information on how far a variable deviates from a nor-
mal-like distribution of values are skewness and kurtosis.
For a normal distribution of data, both skewness and
kurtosis equal zero; for each statistic the further from
zero the value, the less normal-like is the data distri-
bution. The less normal-like the distribution, then the
more questionable are the statistical results.
The skewness and kurtosis values for the list of 48
variables ranged from - 5.212 to 16.098 and from 1.373
to 263.908 respectively, indicating that the distributions
were far from normal. The proposed multivariate pro-
cedures to be used on the performance indicator data set
were considered relatively robust, i.e. valid even un-
der deviations from normality. Robustness is of greatest
concern when using inferential statistical techniques.
However, even descriptive techniques, like the ones used
here, could be affected by highly skewed data. As the
goal of this research was to provide a highly reliable set
of consistent analytical findings that could serve as a
benchmark for cross-year comparisons, it was important
76 G. J. FIELDING PI ul.

Table 1. Performance indicators by concept

CUST EFFICIENCV -S SERVICEEFFECTIVENESS

MslJRt3
Labor Efficiency Utilization of Service

TVH/Dp Vehicle Hrs Per Employee TPAWRVH Passngr Trips Per Rev Veh Hrs
TPAS/RVM Passngr Trips per Rev Veh Ri
TPAS/W Passngr Trips per Peak Veh
PASWTPS Passngr Mi per Passngr

(beratina Safety

TVWACC l.DOO,OOD Veh Mi per Accident

RVH/KC Rev Veh Hrs per Accident
Vehicle Efficiency
Revenue Gerwation
TVWAVEH Vehicle Hrs Per Active Veh
WPVEH Vehicle Hrs per Peak Veh Req REWPVEH Passngr Rev per Peak Veh
TVWAVEH Vehicle Mi per Active Veh REV/RW Passngr Rev per Rev Veh Hr
TVIWVEH Vehicle Hi per peak Veh Req OREWRVH (per Rev per Rev Veh Hr
RVWTVM Revenue Veh Mi Per Veh Wi REV/TPAS Passngr Rev per Passngr

Fuel Efficiency Public Assistance

RVWFUEL Revenue Veh Ni per Cal Diesel RVlVLSUB Rev Veh Hrs Per Local Cap,
TVWFLEL Vehicle li (Bus) per Gal Diesel
RVWSSLB &?Lh*% per State Cap,
Mintenance Efficiency @er Asst
RWUOSUB Rev Veh Hrs Per Tot @er Asst
TVIVEXP Total Veh Mi per Naint Expense RVH/TSUB Rez;hA;g per Tot cap.
TV?l/!NT Veh Mi per Ralnt brployee
TVR/RCAL l.ODO,OOO Veh fli Per Roadcall TPA!YLOA Passngn Per Local Cper Asst
TPASJTSUI Passngrs per Tot @er. cap Asst
OutLNJt wr Dollar Cost REWTSUB Passngr Rev Per Tot cper.
Cap Asst
RVWDEXP Revenue Veh Hrs per REV/OSlB Passngr Rev per Tot @er Asst
Over Expense PAS/OSuB Passngrs per Tot @eer Asst
TVWOEXP Veh Mi Per Cper Expense
WlW Rev Veh Hrs Per lot Labor, COST EFFECTIVENESS MEASURES
Fringe Expense
RWU(wG Rev Veh Hrs per @er Labor, Service Ccmsmvtim LW Exwnse
Fringe Expense
RVHAWG Rev Veh Hrs Per Veh Mint Labor, PAUDEXP Passngrs pr LQeer Expense
Fringe Expense PASWOEX Passngr I41 r;orzrpense
RWUADU; Rev Veh Hrs per A&in Labor, PAS/W Passcgrs
Fringe Expense K nefits
PAS/FUEL Pa!xiE per Gal Diesel
PASWTEX Passngt rnl per Tot Expense

Revenue Generation per Exoense

OREVKEXP Dper Rev tooper Expense

TREWTEX Tot Rev to Tot Expense

to begin with a set of data that had a minimum of dis-
tributional problems.
Logarithmic and weighted data sets
To counter any possible bias in the analyses and to
provide a comparable set of more normally distributed
performance indicator variables, the base 10 logarithms
of the 48 performance indicators were calculated to shift
the distribution of the data to a less skewed curve. This
provided two sets of comparable data--the 48 perfotm-
ante indicator ratios calculated from the Section 15 re-
reported data; (4) logs of the weighted data variables.
The purpose for developing these four sets was to ensure
that when final results from multivariate analyses were
reported, most contingencies for possible bias in the data
had been addressed. Consistent results across the four
data sets would provide evidence that, indeed, a stable
performance concept structure had been found in the
data.
EXPLORATORY ANALYSIS

ported data and a set of 48 logarithm variables.
A third and fourth set of indicators were also devel-
oped. As mentioned above, revenue data is reported as
a total for the whole system; it is not broken down by
mode when more than one mode exists. Capital and
operating assistance are also not consistently separated
by mode. Therefore, a third set of performance ratios
was developed using basic variable data and revenue
statistics that were weighted to eliminate revenue and
subsidies from modes other than bus transit. Then, a full
set of 48 base 10 logarithms was calculated on the weighted
data: again, to provide more normally-distributed data.
As a result of the cleaning, verifying and grooming,
four somewhat different sets of performance indicator
data were available: (1) ratios from reported data; (2)
logs of reported data variables; (3) ratios from the weighted
Multivariate analyses were used to search for a highly
consistent set of performance concepts relevant to fixed
route transit and for a small, unique subset of perform-
ance indicators. Factor analysis is ideal for detecting the
structure within a set of data and for determining those
few key variables with which a whole range of infor-
mation can be represented. The prime objective in this
research was to search for the minimum amount of data
necessary to convey the maximum amount of perform-
ance information. Parsimony and consistency were the
key criteria; factor analysis was the most efficient means.
Factor analysis
The most distinctive characteristic of factor analysis
is its ability to reduce a large set of data to a smaller set
 Performance evaluation for bus transit
of components or factors which portray the un-
derlying structure of relationships among a set of vari-
ables. Based upon the correlation patterns of a large
number of variables, the objective of the factor analytic
technique is to group together those variables which are
highly correlated with each other. The analyst then in-
terprets each factor according to the variables belonging
to the group. The idea is to summarize many variables
by using a few representative factors.
Factor analysis not only provides information on the
number of factors underlying the data, it also determines
which variables grouped on a particular factor are most
highly related or representative of the identified factor.
The factor loading of each variable on the respective
factors can be interpreted as the correlation of the var-
iable with the factor; high factor loadings represent high
correlations.
In performing any factor analysis, Comrey (1973) sug-
gests that there are three problem areas that could exist
in the data, and obscure the underlying data structure:
(1) two variables carry highly redundant information
(collinearity); (2) a variable loads across several factors
equally well (poorly defined structure in the variable)
and (3) one factor has all or most of the variables weight-
ing heavily on it (poorlydefined structure in the data
set).
The first exploratory factor analysis was begun with
the most complete set of performance indicator ratios
available in the Section 15 data. It remained necessary
to assess how well these variables measured the target
information and how relevant the indicators were for
cross-system analysis. The next task involved determin-
ing from the set of 48 variables which subset of variables
provided the best cross-system measures, and best de-
fined the structure in the data while testing the data for
the three possible contaminating problems listed above.
As four parallel sets of performance indicators were
available, the same type of exploratory factor analysis
was carried out on each set. After each exploratory factor
analysis was performed, the resulting factor loading ma-
trix was evaluated. Finding similar results across the four
data sets would signal detection of the consistency in the
data which would point to the true underlying struc-
ture in the variables.
Variable elimination
In the first exploratory analysis on the full set of 48
performance indicators, a total of 128 cases were in-
cluded in the analysis. As mentioned earlier, factor anal-
ysis will drop from the analysis every case missing any
piece of information. Because the missing values were
scattered throughout the 48 variable set, the snowball
effect had eliminated nearly two-thirds of the cases from
the analysis. Thus, in the next exploratory pass through
the data, it was decided to eliminate those variables that
compounded the missing data problem and those that
were still somewhat questionable as to the quality and
comparability of reported information. This increased
the number of cases to approximately 198.
Fuel related variables (RVMIFUEL, TVMIFUEL) were
eliminated because with four different types of fuel listed
77
for motor bus operations it was difficult to validly com-
pare fuel efficiency across systems. Local and state sub-
sidy related variables (e.g. RVH/LSUB, RVH/SSUB)
were dropped because definitions of local versus state
subsidies were inconsistent. Capital subsidy variables
were dropped because they can greatly shift from year
to year.
The passenger miles (PASM) variable was missing
from almost 20% of the cases. To increase the number
of cases entering into the analysis, variables based on
PASM (e.g. PASM/OEX, PASM/TPS) were eliminated
from the data set.
Variables related to active vehicle counts were also
deleted. Distinctions were not always made between school
buses, charter buses and other motor buses. Some agen-
cies listed more active vehicles than total vehicles while
for others, vehicle inventories were incomplete. Varia-
bles using peak vehicle requirements (PVEH) were
retained.
The variable RVMlTVM was eliminated because 65
of the cases had revenue vehicle miles equal to total
vehicle miles, a strong indication of a definitional prob-
lem, which greatly inflated the kurtosis value of the
variable. The roadcall related variable, TVM/RCAL,
was dropped because the definitions for what makes a
true roadcall were unreliable. The variables related to
total expense (e.g. PASMITEX, TREVITEX) were de-
leted because total expense is not truly comparable across
systems; there are no set parameters for depreciating
capital costs. Finally. REV/RVH was so highly corre-
lated with OREVlRVH that it was eliminated.
With each exploratory factor-analytic pass through the
data sets, the variables were checked against the factor
structure to determine if remaining variables presented
any of the structural problems mentioned above. With
each pass through the data, the underlying structure be-
came more clearly defined. The number of cases entering
into the analysis had increased from 128 to 198 and the
same general solution appeared across the four different
sets of data.
The final set of 30 performance indicators that emerged
after the fourth pass through the data reflected a strong
set of performance indicator variables. These portrayed
such highly consistent factor loadings across all data sets
that it was evident that the most salient features of the
performance concept model had been identified.
Table 2 lists the 48 performance indicator variables
selected for analysis from the Section 15 data base. They
are portrayed within the framework of the conceptual
model. Those variables eliminated prior to the final anal-
ysis are marked with an asterisk to offset them from the
final set of 30 performance indicators used in subsequent
analyses.
FINAL FACTOR ANALYSIS ON 30 PERFORMANCE
INDICATOR RATIO VARIABLES
The final factor analysis was carried out on the cleaned
set of 30 performance indicator ratio variables. After all
the data cleaning and verifying strategies, after all the
exploratory passes through the data and after all the
78 G. J. FIELDINGet al.

Table 2. Forty-eight performance indicator variables used in The patterns of factor loadings were so similar be-
analyses tween the reported data, weighted data and the two sets
COSTEFFICIEKY Kk3JRE.s of logs that it appeared very convincing that the under-
lying structure in the data set had, indeed, been found.
Seven factors, accounted for approximately 83% of the
variance. Table 3 shows the pattern of factor loadings
for the final weighted data set. Factors one, two and
three represent output per dollar cost, utilization of ser-
vice and revenue generation per expense, respectively.
These first three factors directly relate to the three major
categories of the performance concept model-cost ef-
SERVICEEFFICIENCYIEASlREs ficiency, service effectiveness and cost-effectiveness.
Factors four, five and six represent labor efficiency,
vehicle efficiency and maintenance efficiency, respec-
tively; each a refinement of cost-efficiency. Factor seven
is clearly related to safety.

VERIFYING THE FINAL 1980 FACTOR ANALYSIS

COSTEFFECTIVENESS
MSLRES
Wariable anitted prior to final
considerations for data quality, these 30 variables were
chosen to represent the best possible information on per-
formance currently available in the Section 15 data base.
Principal component factor analysis with varimax or-
thogonal rotation was carried out on the four different
sets of 30 performance indicator variables. Two different
computer routines were used-SPSS-PA1 and BMDP-
P4M. The latter was used to compare as closely as pos-
sible the current analyses with the previous work.
The adequacy and strength of the final solution were
determined by Thurstones five criteria for detecting sim-
ple structure solutions in factor analysis results (Harmon,
1967). The rotated factor loading structure was compared
against Thurstones criteria for evaluating structure for
its simpleness and met each of the qualifying con-
analysis ditions. This was convincing evidence that a clear, un-
derlying structure in the data had been found.
In interpreting and portraying the factor loading pat-
tern, i.e. in Table 3, an arbitrary cutoff of .5 was used
to determine the representative variables for each factor.
It was felt that a high cut off value would make for easier
and clearer interpretation of the factors.
The next point investigated was how much of the
variance of the final factor solution was not being ac-
counted for by those identified high-loading varia-
bles. The data were tested and it was found that for each

Table 3. Rotated orthogonal factor loading matrix

FACTOR 1 FACTOR 2 FACTOR 3 FACTOR 4

OmpT PER DOLLAR COST UTILIZATION OF SERVICE REVENUE GENERATICNI USOREFFICIENCY
EXPENSE

TvMiOEXP .90 PAS/OEXP .m REWOSUS .s2 lVH/EMP .91*

RVMlTWG .67 PAMWAG .66 OREWOEXF -91. RVHiOEMP .66
RVH/OEXP .67 TPASlRVH .66 OREWRV .M TVH/PVE 53
RVH/OWAG .63 TPASIPVH .a4 REWPVEH .n TVM/EMP .Sl
TVMIMEXP .,I TPASlRVM .83 REWTPAS .70
RVH/OSUS .61 PASOSUS .67 RVHlOSUB .66
RVWVMWG .s6 PAS/OSUS 61

Percnlo,Vtianc8Explalrad: Percent of Vananca E@aind: P,,ro,VananceEx~d: Percemofvarianw Elqlsined:

26.7 16.6 12.6 9.1

FACTOR 5 FACTOR 6 FACTOR?

VEHCLE EFFICIENCY MAlNTENANCEEFFlClENCY SAFEI-Y

TVM/PVEH .92 lvM/MNT .w* TVMIACC .93

PVEH/OP -.77 PVEH/MNT .9Q RVHIACC .63
NIPVEH .77

penen, of Varianca Ex$4med: PercantdVdEd: pacwndvwianceEap4ained:

7.2 6.6 6.6

TOTAL AMOUNT OF VARlANCE EXFIAINED: 63%

 Performance evaluation for bus transit
factor, approximately 95% of the information was still
being represented while overall, 86% of the variance of
the original factor structure was represented in the subset
of high-loading variables.
Reliability
A third question regarding the set of high-loading var-
iables defining the factor structure centered on the re-
liability-in a statistical sense-of the grouped varia-
bles. Cronbachs Alpha was calculated for each group
of variables gathered together on a particular factor.
Cronbachs Alpha can be used to evaluate the internal
consistency of a group of variables to see if they essen-
tially target the same underlying information (Carmines
& Zeller, 1979). Alpha values range from zero to one
with a value equal to one representing perfect reliability,
or internal consistency in this case. An alpha value of
.8 is considered very reliable.
A standardized item alpha was calculated for each
group of high-loading variables on each factor, across
each of the four sets of slightly different data. The alpha
values hovered around the .8 criterion on the weighted
data set, and were all well above .8 on the log set of
the weighted data. This was true on all Factors except
Factor 5 which produced an uninterpretable alpha value.
Factor 5 measures the positive and negative poles of the
vehicle efficiency concept as shown in the negative and
positive factor loadings. Thus, it confounds the calcu-
lation of a standardized item alpha.
Factor structure stability
Two final questions were raised regarding the 1980
final factor analysis. They both focused on a single con-
cem-how globally relevant was the final factor struc-
ture? Would the underlying structure of the data remain
stable over different theoretical assumptions or an in-
crease in data cases?
Classical inferential factor analyses were carried out
on the four performance indicator variable sets. This type
of analysis assumes that the data comes from a sample
of cases from a larger population. All solutions and re-
ported statistics are mathematically adjusted to predict
values as they would exist in a larger population. Thus,
it is conceivable that, if a factor structure is somewhat
weakly defined, a different structure could emerge from
an inferential solution than from a principal components
analysis. However, results from both the inferential and
principal components analyses were consistent across the
four data sets.
To test whether the final structure in the analyses would
remain stable over an increase in data cases, an esti-
mation procedure for missing data was used. The BMDP
statistical computing package includes a program whereby
missing data values can be estimated. Multiple regres-
sion on the variables with data is used to predict a most
likely estimate for any case missing data on some subset
of the variables in the analysis. When no prediction can
be made from other available data, the mean of the
variable of interest is used to replace the missing value.
When any case is missing too much of its data, it is not
used in the estimation procedure.
79
A final set of factor analyses was carried out on the
four sets of performance indicators where missing values
had been replaced with estimates. The number of cases
then being analyzed increased from 194 to 280.
It was plausible that an increase in the number of cases
being analyzed could shift a weak or unstable factor
solution to a different factor structure. The final set of
factor-analytic solutions carried out from the data sets,
which included estimated values, were entirely consist-
ent with the earlier results.
Thus, after rigorous testing of the final 1980 factor
analysis. It was found that: (1) the same general under-
lying structure had consistently appeared across all
checking routines; (2) not only the same factors ap-
peared, but they also appeared in the same order and (3)
with minor fluctuations, the factor loading patterns were
generally the same. Therefore, it was concluded that
a stable, consistent and reliable simple structure had
been detected out of the larger group of performance
indicators.
SELECTING REPRESENTATIVE MARKER VARIABLES
The objective for this research was the establishment
of a small, unique subset of performance indicators that
is particularly useful for performance evaluation by sys-
tems of all sizes. The goal was to identify the minimum
amount of data necessary to convey the maximum amount
of performance information. To accomplish this, the fac-
tor loading data in the rotated factor-structure solutions
on the final variable sets were used. High factor loadings
represent a high correlation of a particular variable with
a particular factor. When a variable has a high factor
loading on only one factor, it can be said to represent
that factor both statistically and conceptually.
To select a small subset of easily accessible perform-
ance indicators from the final factor structure, five cri-
teria were used: (1) Representativeness of a variable vis-
a-vis a factor was reflected in a high factor loading on
only one factor; (2) the distribution of values in the
variable had to be as close to normal-like as possible;
(3) ease of collection of the variable was assessed via
the percentage of data missing; (4) the variable had to
have been well captured by the factor structure in general
(high communality) and (5) the variable selected had to
be easily understood by transit managers.
Seven representative or marker variables were se-
lected from the final factor structure-one variable rep-
resenting each factor. Seven alternate markers were
also identified. These alternates can be used equally well
for assessing performance: it is the performance dimen-
sion that is important rather than the marker variable.
Table 4 lists the selected marker variables and their
alternates.
The markers and the alternate set of markers are highly
reliable (alpha range is from .802 to .937). Thus, with
a maximum of seven variables from a much larger data
set, performance of a transit system can be assessed. To
assess the three major categories represented in the con-
ceptual model, the first three marker variables would
be sufficient. Further, any one of the seven factor con-
 80
Table 4. Performance indicators best representing the underlying performance concept
1 output per $ cost
2 Utilization of
service
3 Revenue Generation
per Expense
4 Labor Efficiency
5 Vehicle Efficiency
6 Mintenance Efficiency
7 safety
cepts could be assessed via the relevant marker
variable.
KEY INDICATORS FOR PERFORMANCE EVALUATIONS
The seven performance indicators identified in the
previous section represent the key underlying dimensions
of performance as found in the Section 15 data base.
These performance variables can be validly used for
cross-sectional comparisons of transit agencies.
The first three indicators-Revenue vehicle hours per
operating expense (RVH/OEXP), unlinked passenger trips
per revenue vehicle hour (TPAS/RVH) and operating
revenue per operating expense (OREV/OEXP)-are the
most informative global measures available in the data
base. They represent the first three factors which account
for most of the variance in the underlying data. Further,
they each relate to one of the performance model con-
cepts noted in Fig. l-cost efficiency, service effec-
tiveness and cost effectiveness. The four additional in-
dicators can be used to evaluate more specific areas of
cost and service which have important impacts on overall
performance.
The set of seven alternate marker performance in-
dicators can be thought of as the second best list for
conducting performance evaluations. They also represent
the underlying dimensions of performance. These have
been identified to assist transit analysts in selecting equally
valid cross-sectional measures when local or state re-
quirements request reporting of particular statistics. For
example, vehicle miles rather than vehicle hours are
preferred by some governmental agencies as a measure
of service produced. The dimensions identified are more
important than the indicators chosen to represent the
structure.
The evaluation process
One of the most important functions of any perform-
ance evaluation is to identify the major strengths and
weaknesses of the object being evaluated. In transit, this
involves measuring the amount of output produced by
an agency per unit of input and whether the output is
consumed. Evaluations can be static snapshots of the
agency at one point in time, or they can be trend analyses
of the same system over months or years. One of the
G. J. FIELDINGet al.
RVlVCfZXP TVWCEXP
TPM/RVH TPAWRVR
(*(EV/ou(P REV/ON3
TMEW RVwoEw
TVIwJVEH wwVEH
mvm1 PVEIWWT
TVWACC RVHIAaz
most revealing types of analysis comes about when per-
formance achievements for an agency are compared not
only against those of previous years, but also against
those achieved by other agencies which are regarded as
its peers. Through performance comparisons within a
peer group, the achievement of any one system can be
demonstrated in sharp relief (Fielding et al., 1985).
Figure 2 demonstrates the effectiveness of conducting
single system performance comparisons to a peer group.
Jacksonville Transit Authority (JTA) was compared against
its peers across the seven performance indicators iden-
tified in this research for fiscal years 1980, 1981 and
1982. The graphic display of Jacksonvilles performance
information reveals that JTA is above its peer group
average on nearly all performance indicators over the
three years of data.
For each year, safety (TVMIACC) is the strongest
area and maintenance efficiency (TVMIMNT) is the
weakest area of system performance. Labor efficiency
(TVHIEMP) is consistent and appears to be improving,
while vehicle efficiency (TVM/PVEH) shows a sharp
decline between 1980 and 1981. The drop in utilization
of service (TPAS/RVH) over the three years is somewhat
reflected in the dip in revenue generation per expense
(OREVIOEXP) but seems to have had little effect on
JTAs output per dollar cost (RVHIOEXP). The low
performance on maintenance efficiency and the decline
over time in vehicle efficiency are performance trends
Nhich may require managerial intervention.
Using Section 15 data and the performance indicators
identified and tested above, transit managers can monitor
system performance. Unusual performance vis-a-vis other
members of the peer group, or evidence of trends within
one system over time, can alert the manager to the need
for corrective action. Thus, Section 15 data and the set
of performance indicators listed here can provide the
basis for insightful managerial control over the transit
operations. The indicators provide vital signs for the
major dimensions of transit performance: they may not
indicate the cause of change, but they should alert man-
agement to the need for analysis.
Research that has led to the development of the per-
formance evaluation methodology has been based on the
Section 15 data tape available through Transportation
Systems Center. However, for individual agency per-
 Performance
Standard Scorns
2
fotmance evaluations or for peer group comparison, the
Section 15 Annual Report would be more useful. The
Annual Report is a compilation of financial and operating
statistics which have been reported under Section 15
requirements. Data is both aggregated by size and mode
categories, and reported for individual transit systems.
To facilitate single agency performance evaluations, an
instructional manual has been prepared and is available
through the National Technical Information Service of
Springfield, Virginia (de la Rocha et al., 1984). The
performance instructional manual is a self-contained sys-
tem for performance evaluation based on use of the Sec-
tion 15 Annual Report and the seven key performance
indicators identified in this article.
CONCLUSION
The FY 1980 Section 1.5data has been used to identify
and test the most easily accessible and parsimonious set
of performance indicators for fixed-route transit. The
research had two objectives: (1) to find the minimum
amount of data necessary to provide solid and stable
performance evaluation capability, and (2) to test the
validity of the methodology developed from the previous
analysis of FY 1979 data.
The use of factor analysis on a large set of performance
indicator ratios gleaned from the data the structure of
the underlying performance concepts. From the factor
structure, a subset of seven variables was identified and
tested against the larger data structure. These seven var-
iables are the most salient performance indicators cur-
rently available in the Section 15 data base. They can
be used together or individually to assess fixed-route
transit performance.
There is a great deal of confidence in the data used
evaluation for bus transit
1981
1982
and in the final results. Rigorous cleaning. verifying and
grooming procedures carried out before analysis insured
that the input data was as complete as possible. Careful
decisions regarding which variables to keep andor drop
from the analysis provided the best possible set of per-
formance indicators available in the Section 15 data. The
use of four parallel data sets and several exploratory
factor analyses detected the simple underlying structure
of the data. Finally, the rigorous testing and validation
of that underlying factor structure was convincing that
the most salient performance indicator concepts had been
identified. The consistent and stable structure in the data
enabled the selection of seven marker variables for
evaluation. These too measured up to testing and veri-
fying procedures. Given the quality and content of the
Section 15 data, it is felt that the most salient features
for performance evaluation have been determined, and
that the marker variables can be used with confidence
by transit managers and governmental officials.
Acknow/edgemenr-Research for this paper has been supported
by the Urban Mass Transportation Administration. U.S. De-
partment of Transportation, Research Contract No. CA-I I-0026.
The U.S. Government assumes no liability for the contents or
use thereof.
REFERENCES
Anderson S. C. and Fielding G. J. (1982) Comparative Analysis
of Transit Performance. University of California. h-vine. In-
stitute of Transportation Studies. Final Report, UMTA-CA-
11-0020-1.
Carmines E. G. and Zeller R. A. (1979) Reliobilit) and Validity
Assessmen?. Sage Publications, Beverly Hills.
Comrey A. L. (1973) A First Course in Factor Andyis. Ac-
ademic Press, New York.
de la Rocha 0.. Brenner M. E. and Fielding G. J. (1984) The
82 Cl. J. FIELDING ef al.
Irvine Evaluation Method: An Instructional Manual for Use
in Conducting Internal Evaluations of Transit Performance.
University of California, Irvine. Institute of Transportation
Studies. Final Report, UMTA-CA-11-0027-l.
Fielding G. J., Glauthier R. E. and Lave C. A. (1978) Per-
formance indicators for transit management. Transporration
7, 365-379.
Fielding G. I., Brenner M. E., de la Rocha 0.. Babitsky T.
T. and Faust K. (1984) Using Section I5 Data for Transit
Performance Analysis. University of California, Irvine. In-
stitute of Transportation Studies, FiiaI Report, UMTA-CA-
I I-0026-2.
Fielding G. J., Brenner M. E. and Faust K. (1985) Typology
for bus transit. Trans Res. (forthcoming).
Harmon H. H. ( 1967) Properries of Different Types of Factor
Solutions: Modern Factor Analysis. University of Chicago
Press, Chicago.
Kneafsey J. T. (1975) Transport Economic Analysis. Lexington
Books, Lexington, MA.
Miller J. H. (1980) The use of performance-based methodol-
ogies for the allocation of transit operating funds. Transpn
Quarterly 34, 574.
Nash C. A. (1980) Management objectives, fares and service
levels in bus transport. /. Transpn Econ. Policy 12, 70-85.
Patton T. A. (1983) Transit Performance Indicators. U.S. De-
partment of Transportation, Transportation Systems Center,
Cambridge. Staff Study (SS-67-0.3-01).
Talley W. K. and Becker A. J. (1982) A single measure for
evaluating public transit systems. Transpn Quarterly 36, 423-
431.
U.S. Department of Transportation, Urban Mass Transportation
Administration (1982) National Urban Mass Transportation
Statistics: Second Annual Report, Section 15 Reporting Sys-
tem. Report No. UMTA-MA-O6-0107-82-1. Washington, D.C.
Vaziri M. and Deacon J. A. (1983) Application of Section 15
and Census Data to Transportation Decision Making. Uni-
versity of Kentucky, Lexington. Dept of Civil Engineering,
Final Report. UMTA-KY- 1 I-0002-83.