Professional Documents
Culture Documents
Proceedings of 2011 NSF Engineering Research and Innovation Conference, Atlanta, Georgia Grant # CMMI-0927790
using the probability of damage detection when 2.1 Cost of accident investigation: Figure 1 shows
uncertainty originates from the initial crack distribution. the government budget for NTSB’s ―Aviation safety
However, there has not been similar work on the (Ref. 8).‖ It can be seen that the budget has increased
contribution of accident investigation in terms of cost over the years. The budget spike increase seen in 1997
effectiveness. is because NTSB required a one-time cost for TWA
The objective of this paper is to quantify the 800 accident investigation which started in 1996.
contribution of accident investigation and to model its When it comes to the cost of accident investigation,
cost effectiveness. First, we analyze the cost of accident we cannot ignore the money spent by industry and other
investigation and a trend of accident causes in order to parties. Aircraft makers and airliners are involved in
shed light on a key factor—the probability of investigation and play crucial roles with military
recurrence of an accident. Then, we introduce cost participation, if needed. As we mentioned previously,
effectiveness measures, cost per life saved and the purpose of accident investigation is to prevent a
threshold of cost effectiveness. Past accidents with similar accident from occurring in the future, so
different types of causes are selected as examples, and implementation cost should be taken into account to
we examine how the probability of recurrence affects improve safety. For instance, implementation cost
cost effectiveness. Furthermore, we generalize these includes a one-time redesign cost, in addition to
cost effectiveness measures in order to make them operational cost for procedures such as additional
applicable to general safety improvement measures. We periodical inspection. Thus, the cost of NTSB
then perform a comparison in cost effectiveness investigation should be considered as only one part of
between accident investigation and a structural design the entire cost for safety improvement.
change intended to reduce the probability of failure due
to fatigue of a fuselage panel. 50
45
The paper is organized as follows. In Section 2, the
40
analysis methodology of past accidents in terms of cost 35
and accident causes is described. Then, in Section 3, the
Budget [$M]
30
definition of cost effectiveness is discussed. Section 4 25
Proceedings of 2011 NSF Engineering Research and Innovation Conference, Atlanta, Georgia Grant # CMMI-0927790
160
Other accidents
1400 These categories are key factors in determining a
140 Fatal accidents 1200 likelihood of recurrence of an accident, and in turn the
Fatalities
120 cost effectiveness of investigation. If an accident cause
Number of accidents
1000
Number of fatalities
100 is likely to happen to other similar airplanes, i.e., error,
800
80
recommendations based on investigation will
600 potentially prevent many accidents in the future. On the
60
400
other hand, if large amounts of money are spent for an
40
investigation about an extremely rare event accident, it
200
20 contributes less effectively to future safety. Note that
0 0 we did not distinguish human-related accidents between
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
error and rare event, since human-related causes can be
Figure 2: Number of accident from 1999 to 2009 considered as either an error or a rare event, and
sufficient information is not available from the reports
to distinguish between them.
9 Table 1 shows the categorized accident causes of
Part 121/135 major accidents from 1996 to 2005. It can be seen that
Accident rates [per million flight hours]
8 Part 121
while rare events are less dominant (9%), 46% are due
7 to errors. This indicates that a significant amount of
6 accidents had potential to recur, and investigation
prevented them from happening again.
5
Human-related cause is the most dominant factor,
4 representing 65% of the accidents. Human-related
3 causes could be categorized as both error and rare event.
In terms of providing proper improvement measures, it
2
is crucial to carefully distinguish these causes from
1 error and rare event through accident investigation.
0
Moreover, we figured out that 35% of the accidents are
due to a combination of several causes. Our
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
Proceedings of 2011 NSF Engineering Research and Innovation Conference, Atlanta, Georgia Grant # CMMI-0927790
per accident and the likelihood of recurrence of an The advantage of having a threshold value is that
accident are taken into account. Since there is a after we determine the probability of recurrence by
possibility that a similar accident may happen to aircraft accident investigation, we can judge whether or not the
of the same model, the number of airplanes in the fleet implementation plan for a recommendation is cost
is also considered. Moreover, recommendations issued effective. Even during investigation, once the
by NTSB are applied not only to the same aircraft probability of recurrence is determined, we can see if
model but also to related fleets which have the same the investigation is cost effective and whether or not we
failure potential. Thus, the number of airplanes in the need to go further into detail of the accident.
related fleets should be considered for estimating
potential future fatalities. 4. Examples of Cost Effectiveness of Accident
Consequently, we define the equation of the cost Investigation: As examples of the measurement of
effectiveness of the accident investigation as investigation cost effectiveness, we use two past
accidents with different types of causes: error and rare
(1) event.
where is ―cost per life saved‖. This value 4.1 Evaluation of cost effective of accident
represents how much is invested to prevent one fatality investigation (Error case): The example of an
in the future. , , , and are the invested cost, accident due to error is from Alaska Airlines Flight 261,
the number of fatalities in that accident, the number of which occurred on January 31, 2000. Casualties
airplanes in fleet (or related fleets), and the probability included two pilots, three cabin crewmembers, and 83
of recurrence of an accident in lifetime per airplane, passengers on board. The airplane, MD-83, was
respectively. destroyed by impact forces (Ref. 12).
The NTSB concluded that the probable cause was
3.2 Effective cost for future safety and threshold of ―a loss of airplane pitch control resulting from the in-
cost effectiveness: How much society should invest in flight failure of the horizontal stabilizer trim system
preventing fatalities or extending life is controversial jackscrew assembly’s acme nut threads. The thread
and there are many ongoing discussions about this issue failure was caused by excessive wear resulting from
in different social communities, e.g., health care, fire, Alaska Airlines’ insufficient lubrication of the
natural disaster, transportation, etc. In aviation, jackscrew assembly.‖ According to this report, several
economic values used in investment and regulatory factors contributed to the accident.
decisions of the Federal Aviation Administration (FAA) First, lubrication of the nut threads was not
were analyzed (Ref. 10). Reference 10 specified that adequately performed. Second, there were inappropriate
values of life and injury should be based on the wider lubrication and inspection intervals for the wear
―willingness to pay‖ by society for reduced risks of condition; because of this, the wear exceeding its
fatalities and injuries. The latest guidance in 2004 critical condition could not be discovered before the
provided a minimum value of $3 million per fatality following lubrication or inspection point. Lastly, there
averted. This means that people would be willing as a was no fail-safe mechanism to prevent the catastrophic
group to pay $3 million to prevent one fatality. effect from the nut thread loss. These observations lead
Similarly, in Europe, an aviation fatality saved is valued us to conclude these inappropriate maintenance
4.05 million EURO by the European Transport Safety procedures and the aircraft design as errors, meaning
Council in 2003 (Ref. 11). Thus, a value of several that once insufficient lubrication happens, a similar
million dollars per fatality adverted can be considered catastrophic accident is likely to happen to other
as a common estimate currently in the aviation airplanes.
community. Furthermore, looking at the cause of inadequate
Here we introduce the concept of the cost lubrication, it was concluded that ―more than just the
effectiveness threshold. By using $3M per fatality for last scheduled lubrication was missed or inadequately
in Eq. (1), we can obtain corresponding performed.‖ The report also notes that ―inadequate
investigation cost as the threshold of cost being lubrication of the accident jack screw assembly was not
invested. Once we estimate a probability of recurrence an isolated occurrence‖ and ―deficiencies continue to
of an accident, we can calculate the threshold for exist in Alaska Airlines’ lubrication practices,‖
effectiveness of the cost being invested, indicating that there was a possibility of other locations
_th . Notice
that the cost expressed in Eq. (2) is in dollars. with insufficient lubrication. In fact, NTSB issued
recommendations on improving Alaska Airlines’
_th 3 × 106 (2)
lubrication process.
Based on these facts, we conclude that the accident
cause can be categorized as error, and similar accidents
Proceedings of 2011 NSF Engineering Research and Innovation Conference, Atlanta, Georgia Grant # CMMI-0927790
are likely to happen again. We estimate that the Since this accident was due to a combination of
probability of recurrence is between one per hundred several human errors, we considered it as an extremely
and one per thousand in the lifetime of an airplane. By rare event; it is not likely to happen to other airplanes.
using Eq. (2), the corresponding cost effective threshold In fact, some recommendations about evacuation after a
range is determined to be $2,950M - $295M (Table 2). collision were issued, but there was no recommendation
If the cost for the accident investigation and intended to prevent the direct causes, i.e., the failure to
implementation is less than this cost range, these safety ―see and avoid" and the interrupted transmission. It is
improvement activities can be considered as cost not easy to clearly determine the probability of
effective. Note that the design lifetime of MD-83 is recurrence of such a human-related rare event, but we
50,000 flight hours (Ref. 13) and the number of estimated the range of it from 10-5 to 10-7 in the life
delivered airplanes in the fleet as of 2010 is 1,191. time of an airplane. This range is equivalent to an
accident rate of 3.3×10-4 to 3.3×10-6 per million flight
Table 2: Cost effectiveness threshold hours. The corresponding cost effectiveness threshold is
(Alaska Airlines Flight 261) shown in Table 3. Note that design lifetime of the
Probability of reoccurence Cost effectiveness threshold Beech 1900 is 30,000 flight hours (Ref. 15) and the
(P1r) (Cinv_th) number of delivered airplanes in the fleet as of 2010 is
Case-1 10 -2 $2,950M 695. As in the previous example, if the cost being
-3
invested for investigation and implementation are less
Case-2 10 $295M
than this cost range, they were held cost effectively.
4.2 Evaluation of cost effective of accident Table 3: Cost effectiveness threshold
investigation (Rare event case): An example of a rare- (United Express Flight 5925)
event case is from United Express Flight 5925 which Probability of reoccurence Cost effectiveness threshold
occurred on Nov 19, 1996. The airplane, a Beechcraft (P1r) (Cinv_th)
1900, crashed while landing at the Quincy airport in
Case-1 10-5 $0.292M
Illinois, colliding with a Beechcraft King Air. All 12
aboard the Beech 1900 and 2 aboard the King Air were Case-2 10-7 $0.003M
killed in the accident (Ref. 14).
NTSB concluded that ―contributing to the severity As seen from these two examples, the proposed
of the accident and the loss of life were the lack of model is useful for evaluating cost effectiveness of
adequate aircraft rescue and firefighting services and accident investigation and implementation of
the failure of the air stair door on the Beech 1900C to recommendations.
be opened‖, but the direct cause of the collision was a
combination of human errors. 4.3. Comparison in cost effectiveness with different
While the Beech 1900 was approaching runway 4, types of safety improvement measures: In this
Air King was in its takeoff roll on the same runway. subsection, we compare cost effectiveness between
The pilot on the Beech 1900 mistook an interrupted accident investigation and another reliability
transmission from another airplane; Piper Cherokee improvement measures that are intended to reduce the
which was holding for departure on runway 13, as a probability of failure. Kale et al. (Ref. 16) traded off
response from the King Air to their request for King the reliability improvement for fatigue of a fuselage
Air’s intention. Therefore, the pilots of the Beech 1900 panel against lifecycle cost. In the study, variability in
mistakenly believed the King Air was not planning to initial crack size, material properties for crack growth,
take off after the Beech 1900 finished landing and and pressure load in flight was considered as an
cleared the runway. uncertainty. For design variables, the thickness of panel,
The response from the Piper Cherokee to Beech inspection type, and inspection interval in service were
1900 was unnecessary and inappropriate. Furthermore, optimized to find the minimum lifecycle cost for a
the pilots on the King Air failed to announce their given probability of failure. The lifecycle cost is
intention to take off and missed their duty to ―see and calculated based on material cost, fuel consumption,
avoid‖ other traffic. The pilot of the King Air did not and inspection cost in service. The Appendix shows the
hear the transmissions from the Beech 1900; it was details of the cost estimate.
reportedly likely that either they did not properly We selected a case in which the probability of
configure the radio receiver switches, or they were failure of a panel is improved from 10-7 to 10-8 in the
preoccupied, distracted, or inattentive. As a result of lifetime and additional lifecycle cost per panel is
these pilot’s actions, the collision occurred on runway 4 $0.81M. We expanded this model to an entire airplane
when the King Air had begun its takeoff while the (Boeing-747). We assumed to have 1,350 panels in the
Beech 1900 was landing. entire airplane. Then, reliability improvement of an
Proceedings of 2011 NSF Engineering Research and Innovation Conference, Atlanta, Georgia Grant # CMMI-0927790
airplane was converted into from 1.4×10-4 to 1.4×10-5 in fraction of these costs. Thus, we can conclude that the
its lifetime, and additional lifecycle cost was calculated accident investigation was much more cost effective
as $1,149M. In the reliability-based design optimization than the structural design change.
(RBDO) framework, reliability improvement is applied This comparison can be considered as evaluating
from a certain probability of failure to another. two types of investments for future safety. While
Accordingly, we generalize Eq. (1) for cost accident investigation reveals and prevents
effectiveness as unrecognized failure modes, which are discovered by
accident, from recurring in the future, the stricter
(3) structural design requirements are equivalent to
− t )
imposing a stricter design rule on already-known failure
modes. In this particular case, the investment for
where is a probability of failure in the lifetime protecting against unknown failure modes is much
of each airplane before improvement is applied, and more cost-effective than applying more severe design
t is the probability of failure in the lifetime of each rules.
airplane after improvement is applied. This equation is
identical to Eq. (1) when t is zero, where a similar Table 5: Comparison of cost corresponding to the same
accident is assumed to be completely prevented after cost effectiveness
accident investigation. Cost per life saved Invested cost
Table 4 shows the cost per life saved by the (C1life) (Cinv)
structural design change. For the number of fatalities, Accident investigation $14,700M (when P 1r = 10-2)
(Alaska Airlines Flight 261) $ 1,470M (when P 1r = 10-3)
we consider two extreme cases. The first case is that $15M
fatigue failure leads to a fatal accident; the number of Structural design change
$1149M
for fatigue
fatalities is the same as the number of all passengers of
Boeing-747 (about 450 passengers). The other case is
that there is a single fatality, like Aloha airlines’ 5. Concluding Remarks: A methodology of
accident due to metal fatigue in 1988. Cost per life quantifying cost effectiveness of accident investigation
saved is finally calculated as $15M for 450 fatalities for saving lives is proposed. We showed that the
and $6,667M for one fatality. probability of recurrence of an accident is a key factor
for determining cost effectiveness. Through examples,
Table 4: Cost per life saved we illustrated the evaluation of cost effectiveness for
(Structural design change for fatigue life) different types of accident causes, such as error and rare
event. We concluded that distinguishing accident
Quantity
causes, i.e., error or rare event, is a crucial role not only
Additional cost (Cinv) $1,149M for issuing appropriate recommendations but also for
evaluating cost effectiveness of investigation and
Number of airplanes in the fleet 1,418
implementation. Furthermore, we performed a
Assumed fatalities 450 1 comparison in cost effectiveness between accident
investigation and a structural design change, intending
Cost per life saved (C1life) $15M $6,667M to reduce the probability of fatigue failure. We found
that for the example of Alaska Airlines case, the safety
Since we do not know the actual cost for the Alaska improvement by the accident investigation was clearly
Airlines Flight 261 case, we estimated a cost which more cost effective.
performs the same cost effectiveness as the structural
design change case, i.e., cost per life saved = $15M. 6. Appendix - Cost model of structural design
Table 5 shows the comparison of cost effectiveness change for fatigue: The total lifecycle cost is
between the accident investigation of Alaska Airlines calculated based on material cost, fuel consumption,
and the structural design change. The cost of the Alaska and inspection cost in service. While a fatigue-tolerant
Airlines case is determined to be $14,700M if = 10-2, structure with thicker panels can reduce a frequency of
-3
and $1,470M if = 10 . the inspection, a cost penalty on weight is imposed. Eq.
According to the accident report, recommendations (4) is the model of total lifecycle cost, t t , used in
issued mainly consist of two improvement measures, (Ref. 16).
such as revising lubrication and maintenance t t (4)
procedures, and design correction for the catastrophic
single point of the wear. Considering implementation where is material manufacturing cost per unit
plans and the amount of NTSB’s annual budget weight, is structural weight, is fuel cost per unit
previously discussed, it is apparent that investigation weight per flight, is service lifecycle of airplane, is
and implementation for this accident were only a small
Proceedings of 2011 NSF Engineering Research and Innovation Conference, Atlanta, Georgia Grant # CMMI-0927790
cost of inspection, and is the number of inspections, Available from
respectively. The total lifecycle cost in lifetime per http://www.ntsb.gov/info/inv_guides.htm
panel is calculated as $18.66M when = 10-7, [8] Budget of the United States Government: Appendix,
-8
and $19.47M when t = 10 so that the additional Other Independent Agencies, Fiscal year 1996-2005,.
cost for the improvement is obtained as $0.81M. [9] National Transportation Safety Board, Aviation
Accident Statistics [Online database],
7. Acknowledgements: This research was partly http://www.ntsb.gov/aviation/stats.htm, as of 8 June,
supported by National Science Foundation (Grant 2010.
CMMI-0927790). The authors gratefully acknowledge [10] GRA, Incorporated, (2004) ―Economic values for
this support. FAA investment and regulatory decisions, a guide,‖
Draft Final Report prepared for FAA Office of Aviation
8. References: Policy and Plan
[1] Dhillon, B.S., Fashandi, A.R.M., Liu, K.L. (2002) [11] European Transport Safety Council, (2003) ―Cost
―Robot systems reliability and safety: a review,‖ Effective EU Transport Safety Measures,‖ Brussels,
Journal of Quality in Maintenance Engineering, Ottawa, Belgium, ISBN 90-76024-16-2
Canada, 8(3):170-212 [12] National Transportation Safety Board, (2000)
[2] Garbatov, Y., and Guedes Soares, C., (2001), ―Cost ―Airplane Accident Report Loss of Control and Impact
and reliability based strategies for fatigue maintenance with Pacific Ocean Alaska Airlines Flight 261
planning of floating structures,‖ Reliability Engineering McDonnell Douglas MD-83, N963AS About 2.7 Miles
and System Safety 73:294-301. North of Anacapa Island, California January 31, 2000,‖
[3] Kale, A.A., Haftka, R.T., and Sankar, B.V., (2008) NTSB/AAR-02/01
―Efficient Reliability Based Design and Inspection of [13] Arthur Pearcy, (1999) ―McDonnell Douglas MD-
Stiffened Panels against Fatigue,‖ Journal of Aircraft, 80 & MD-90‖, Motorbooks International, ISBN 0-
45 (1): 86-97 7603-0698-2
[4] Acar, E., Kale, A., and Haftka, R.T., (2007) [14] National Transportation Safety Board, (1997)
―Comparing Effectiveness of Measures that Improve ―Aircraft Accident Report Runway Collisio United
Airplane Structural Safety,‖ Journal of Aerospace Express Flight 5925 And Beechcraft King Air A90
Engineering, 20(3): 186-199 Quincy Municipal Airport QUINCY, ILLINOIS,
[5] Kulkarni, S.S., and Achenbach, J.D. (2008) November 19,1996‖ NTSB/AAR-97(04)
―Structural health monitoring and damage prognosis in [15] Docket No. FAA-1999-5401, 14 CFR Parts 119,
fatigue, Structural Health Monitoring‖ 7: 37-49. 121, 129, 135, and 183, (2002) ―CFR Final Rule: Aging
[6] 49 CFR Part 831, ―Accident/Incident investigation Airplane Safety.‖
procedures‖ [16] Kale, A., and Haftka, R.T., (2008), ―Tradeoff of
[7] National Transportation Safety Board, (2002) Weight and Inspection Cost in Reliability-Based
―National Transportation Safety Board Aviation Structural Optimization,‖ JOURNAL OF AIRCRAFT,
Investigation manual Major Team Investigations.‖ 45(1):77-85
Proceedings of 2011 NSF Engineering Research and Innovation Conference, Atlanta, Georgia Grant # CMMI-0927790