Stochastic Preservation

Florida State University Libraries
Electronic Theses, Treatises and Dissertations The Graduate School
2011
Stochastic Preservation Model for

Transportation Infrastructure
Omar St. Aubyn Alexander Thomas
Follow this and additional works at the FSU Digital Library. For more information, please contact lib-ir@fsu.edu
THE FLORIDA STATE UNIVERSITY
COLLEGE OF ENGINEERING
STOCHASTIC PRESERVATION MODEL FOR TRANSPORTATION
INFRASTRUCTURE
By
OMAR ST. AUBYN ALEXANDER THOMAS
A Dissertation submitted to the

Department of Civil and Environmental Engineering
in partial fulfillment of the
requirements for the degree of
Doctor of Philosophy
Degree Awarded:
Summer Semester, 2011
The members of the committee approve the dissertation of Omar St. Aubyn Alexander
Thomas defended on June 16, 2011.
John Sobanjo
Professor Directing Dissertation
Eric Chicken
University Representative
Lisa Spainhour
Committee Member
Primus Mtenga
Committee Member
Approved:
Kamal Tawfiq, Chair, Department of Civil and Environmental Engineering
John Collier, Dean, College of Engineering
The Graduate School has verified and approved the above-named committee members.
ii
Ill like to dedicate this dissertation to my darling wife, Tamika, my parents Audley and
Ivy Thomas, and the rest of my family who have been supportive throughout the entire
process. Thanks to Almighty God, Who through all things are possible!
iii
ACKNOWLEDGMENTS
I would like to thank my major professor, Dr. John Sobanjo, for his guidance throughout my
PhD. process. I had the tremendous opportunity to learn from his wealth of experience and
mentorship over the years. I also would like to thank the rest of my committee members,
Dr. Lisa Spainhour, Dr. Primus Mtenga and Dr. Eric Chicken. I appreciate the knowledge
I have gained from their instructions and interactions throughout my years of research and
studies at the Florida State University. The knowledge gained has not only enabled me to
be successful in my studies, but has equipped me for life.
I acknowledge my colleagues and friends who have all been a source of strength and
encouragement to me every step of the way. Thanks to the faculty and staff of the Depart-
ment of Civil and Environmental Engineering, who I have gotten to know and respect over
the years. I am grateful to the staff for their kindness and genuine interest in my success,
and I acknowledge the faculty for the high standard of education they have imparted to me
and others over the years, which is priceless.
Love and much thanks to my beautiful wife, Tamika, who shared this dissertation jour-
ney with me from beginning to end, and who have been a constant support in every way.
Many thanks to my wonderful parents Audley and Ivy Thomas, who have always believed
in me. I am also grateful to my siblings Carolin, Kim, Shaun and Karen, for their prayers
and words of encouragement. Thanks to Almighty God, the Alpha and Omega.
iv
TABLE OF CONTENTS
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi
1 INTRODUCTION 1
1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Background on Pavement Management . . . . . . . . . . . . . . . . . . . . . 3
1.4 Background on Bridge Management . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Stochastic Preservation Models . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 LITERATURE REVIEW 10
2.1 Stochastic Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Ng (1996)s Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 DeStefano (1998)s Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 Ng and Moses (1999)s Model . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5 Black et al. (2005a,b)s Model . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.6 Cox Proportion Hazards Model . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.7 Rehabilitation-Cost Effectiveness: Existing Methodologies/Theories for Op-
timization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.7.1 Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.7.2 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.7.3 Multiobjective optimization . . . . . . . . . . . . . . . . . . . . . . . 18
2.8 Adaptive Control Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 19
3 COMPARISON OF SEMI-MARKOV AND TRADITIONAL MARKOV

CHAIN MODELS 21
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2.1 Data Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2.2 Training and Test Data . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2.3 Markov Chain Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2.4 Semi-Markov Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.5 Simulation Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
v
3.2.6 Actual Pavement Deterioration based on Test Data . . . . . . . . . . 32
3.3 Discussion of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.3.1 Transition Probabilities based on the Markov Chain Model . . . . . 35
3.3.2 Transition Probabilities based on the Semi-Markov Model . . . . . . 35
3.3.3 Comparison of the Markov Chain Model, Semi-Markov Model and
Actual Pavement Deterioration Behavior . . . . . . . . . . . . . . . . 45
3.4 Summary of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4 SEMI-MARKOV APPROACH FOR MODELING THE DETERIORA-

TION OF BRIDGE ELEMENTS 50
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.2 Methodology based on Black et al. (2005a,b) . . . . . . . . . . . . . . . . . 52
4.2.1 Filtering Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2.2 Model Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2.3 Markov Chain Model based on Regression Method . . . . . . . . . . 55
4.3.1 Bare Concrete Deck . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.3.2 Pourable Joint Seal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3.3 Reinforced Concrete Abutment . . . . . . . . . . . . . . . . . . . . . 78
5 BAYESIAN UPDATE TECHNIQUE FOR INTERVAL DEPENDENT

TRANSITION PROBABILITIES 83
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.2 Bayesian Update of Transition Probabilities: A Theoretical Explanation . . 83
5.2.1 Likelihood function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.2.2 Prior Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2.3 Posterior Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.4.1 Prior Transition Probabilities . . . . . . . . . . . . . . . . . . . . . . 89
5.4.2 Observed (Likelihood) Transition Probabilities . . . . . . . . . . . . 90
5.4.3 Updated (Posterior) Transition Probabilities . . . . . . . . . . . . . 91
5.4.4 Bridge Element Deterioration Predictions using Updated Transition
Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.4.5 Pourable Joint Seal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.4.6 Bare Concrete Deck and Reinforced Concrete Abutment . . . . . . . 96
6 CONDITION INDEX-BASED SEMI-MARKOV MODELS OF BRIDGE

ELEMENT DETERIORATION 102
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.2.1 Expected Condition State . . . . . . . . . . . . . . . . . . . . . . . . 103
6.2.2 Yearly Transition Probabilities based on the Semi-Markov Process . 104
vi
7 ADAPTIVE CONTROL APPROACH TO BRIDGE MANAGEMENT

SYSTEM 119
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.2.1 Network Level Optimization . . . . . . . . . . . . . . . . . . . . . . . 120
7.2.2 Project Level Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.2.3 Modeling of Deterioration Using Semi-Markov Processes . . . . . . . 121
7.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
7.3.1 Discount Coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
7.3.2 The Laplace transform (s-transform) . . . . . . . . . . . . . . . . . . 122
7.3.3 Semi-Markov Decision Process with Discounting . . . . . . . . . . . 123
7.3.4 Do-nothing Action (Action d) . . . . . . . . . . . . . . . . . . . . . 125
7.3.5 Maintenance Action (Action m) . . . . . . . . . . . . . . . . . . . . 126
7.3.6 Rehabilitation Action (Action r) . . . . . . . . . . . . . . . . . . . 128
7.3.7 Action Unit Cost for SM DP and M DP models . . . . . . . . . . . 128
7.4.1 The Inputs for Do-nothing Actions . . . . . . . . . . . . . . . . . . 131
7.4.2 The Inputs for Maintenance Action . . . . . . . . . . . . . . . . . . 133
7.4.3 Results of the Semi-Markov Decision Processes Model to F HW A
Cost Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.4.4 Results of the Semi-Markov Decision Processes Model to Florida (Pontis R
)
Cost Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
8 MODELING SERVICE LIVES OF PAVEMENT AND BRIDGE ELE-

MENTS 141
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
8.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
8.2.1 Pavement: Service Life . . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.2.2 Bridge Element: Service Life . . . . . . . . . . . . . . . . . . . . . . 142
8.2.3 Parametric Model: Weibull Model . . . . . . . . . . . . . . . . . . . 142
8.2.4 Semi-Parametric Model: Cox Model . . . . . . . . . . . . . . . . . . 143
8.2.5 Nonparamteric Analysis: Kaplan-Meier . . . . . . . . . . . . . . . . 143
8.3.1 Survival Analysis: Service Life for Flexible Pavement . . . . . . . . . 144
8.3.2 Survival Analysis: Service Life for Concrete Bare Deck . . . . . . . . 147
9 DISCUSSION AND CONCLUSION 152

9.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
9.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
vii
Bibliography 156
Biographical Sketch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
viii
LIST OF TABLES
2.1 Number of transformers in each paintwork class grouped by the number of

years since the last refurbishment (Source: Black et al. (2005b)) . . . . . . . 14
3.1 The Range of Crack Indices and Corresponding Condition States . . . . . . . 23
3.2 Transition Probabilities of the Embedded Markov Chain of the Semi-Markov

Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Results based on Maximum Likelihood Estimation of the Scale () and Shape
() Parameters for the Holding Time (Weibull) Distributions in Condition
State i . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4 Means and Standard Deviations of Sojourn Times (Weibull) . . . . . . . . . 42
3.5 Goodness-of-Fit Test on Pavement Sojourn Times (Weibull) . . . . . . . . . 43
4.1 Example of a condition record for Pourable Joint Seal (element 301) . . . . . 51
4.2 Estimated Scale and Shape Parameters, Mean and Standard Deviation for the
Sojourn Times in Each Condition States for Bare Concrete Deck . . . . . . . 58
4.3 Computed Condition State 1 probabilities for Bare Concrete Deck . . . . . . 59
Sojourn Times in Each Condition States for Pourable Joint Seal . . . . . . . 69
4.5 Computed Condition State 1 probabilities for Pourable Joint Seal . . . . . . 69
Sojourn Times in Each Condition States for Reinforced Concrete Abutment . 78
5.1 Number of Pourable Joint Seal in each condition state for years 2007 and 2008 90
5.2 Transition Probabilities and Proportions in Condition State i for Pourable

Joint Seal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.3 Prior Transition Probability Weights and Updated Transition Probabilities for
Pourable Joint Seals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
ix
5.4 Updated Transition Probabilities for Pourable Joint Seal (Condition State 1)
based on the Sojourn Time Fitted to a Weibull Distribution and Extended
over 25 years . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Bare Concrete Deck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Reinforced Concrete Abutment . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.1 Determination of the Expected Condition State based on the Modified Element
Condition Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.2 Sample Condition Record and Condition Estimation for Pourable Joint Seal 107
6.3 The Number and Duration of Element 301 in Each Condition State . . . . . 109
6.4 M LE of the Parameters of the Sojourn Time (Weibull) Distributions for each
Condition State (Pourable Joint Seal) . . . . . . . . . . . . . . . . . . . . . . 110
6.5 Mean and Standard Deviations of the Sojourn Time Distributions for each
Condition State (Pourable Joint Seal) . . . . . . . . . . . . . . . . . . . . . . 110
6.6 Goodness-of-Fit Test on Sojourn Times for Element 301 (Weibull) . . . . . . 110
6.7 The Computed Transition Probabilities from Condition State 1 . . . . . . . . 112
6.10 Mean, Variance and Weibull Parameter Estimates for Element 12 (Bare Con-
crete Deck) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.11 Mean, Variance and Weibull Parameter Estimates for Element 215 (Reinforced
Concrete Abutment) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.1 Summary of F HW A Action Unit Cost for Element 12 (Bare Concrete Deck)
in Environment 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.2 The MLE of the Parameters of the Sojourn Time (Weibull) Distributions for
each Condition State for Do-nothing Action for Bare Concrete Deck . . . . . 131
7.3 Mean and Std. Dev. of the Sojourn Time Distributions for Do-nothing Action
for Bare Concrete Deck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.4 Numerical Solution for the Laplace Transform of the Sojourn Time Distribu-
tion for Do-nothing Action for Bare Concrete Deck . . . . . . . . . . . . . . . 133
x
7.5 Matrix Representing the Embedded Markov chain for Maintenance Action for
Bare Concrete Deck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.6 The MLE of the Parameters of the Sojourn Time (Weibull) Distributions for
Maintenance Action for Bare Concrete Deck . . . . . . . . . . . . . . . . . . 134
7.7 Mean and Std. Dev. of the Sojourn Time Distributions for Maintenance
Action for Bare Concrete Deck . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7.8 Numerical Solution for the Laplace Transform of the Sojourn Time Distribu-
tion for Maintenance Action for Bare Concrete Deck . . . . . . . . . . . . . . 135
7.9 Comparison of Results: SMDP vs. Traditional MDP for Bare Concrete Deck
using F W HA Cost data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.10 Summary of Floridas Action Unit Cost for Element 12 (Bare Concrete Deck)
in Environment 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
7.11 Comparison of Results: SMDP vs. Traditional MDP for Bare Concrete Deck
using Floridas (Pontis
R
) Cost data . . . . . . . . . . . . . . . . . . . . . . . 138
7.12 Summary of Floridas Action Unit Cost for Element 301 (Pourable Joint Seal)
in Environment 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.13 Comparison of Results: SMDP vs. Traditional MDP for Pourable Joint Seal
using Floridas (Pontis
R
) Cost data . . . . . . . . . . . . . . . . . . . . . . . 139
8.1 Results of Weibull Regression for Flexible Pavement . . . . . . . . . . . . . . 144
8.2 Results of Cox Regression for Flexible Pavement . . . . . . . . . . . . . . . . 144
8.3 Results of the Kaplan-Meier Method for Flexible Pavement (Life Table) . . . 145
8.4 Description of the Main Material on Bridge . . . . . . . . . . . . . . . . . . . 147
8.5 Results of Weibull Regression for Concrete Bare Deck . . . . . . . . . . . . . 148
8.6 Results of Cox Regression for Concrete Bare Deck . . . . . . . . . . . . . . . 148
8.7 Results of the Kaplan-Meier Method for Concrete Bare Deck (Life Table) . . 149
xi
LIST OF FIGURES
1.1 Conceptual illustration of a pavement condition life-cycle (Source: Shahin

(2006)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Health Index for a bridge showing increased life due to a major project in
program year 5 (Source: Shepard and Johnson (2001)). . . . . . . . . . . . . 7
1.3 Element Condition Index for a bridge element showing increased life due to
two periodic maintenance and a major rehabilitation actions. . . . . . . . . . 7
2.1 Single-step Transition Process of a Markov Chain (Source: Ng and Moses (1999)) 12
2.2 A Typical Sample Function of the Deterioration (Semi-Markov) Process (Source:

Ng and Moses (1999)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1 Transition diagram that represents the possible single transitions between
condition states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2 Schematic of Pavement Segment Divisions into Equal Units of One-Tenth of a

Mile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 An Example of the Change in the Condition States of Pavement Segments

Over Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4 Flowchart for a single run of the simulation over 20 years. . . . . . . . . . . . 33
3.5 Frequency of Sojourn Times in Condition State 10 Before Transitioning to 9 37
3.6 Frequency of Sojourn Times in Condition State 10 Before Transitioning to 8 38
3.7 Frequency of Sojourn Times in Condition State 9 Before Transitioning to 8 . 39
3.8 Frequency of Sojourn Times in Condition State 9 Before Transitioning to 7 . 40
3.9 Monte Carlo Simulation Results: Comparison of the Markov chain, Semi-
Markov and Actual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.10 Monte Carlo Simulation Results: 95% C.I. Bands on the Markov Chain
Model and Actual Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
xii
3.11 Monte Carlo Simulation Results: 95% C.I. Bands on the semi-Markov Model
and Actual Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.12 Survivor Curve: Proportions of pavements in best condition state (state 10) . 48
3.13 Proportions of pavements in condition state 9 and higher . . . . . . . . . . . 49
4.1 Graphical representation of the possible paths between condition states sub-
sequent to becoming new. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.2 Graphical representation for p2,3 (3). . . . . . . . . . . . . . . . . . . . . . . . 56
4.3 Graphical representation for determining pE

3 (5, 2). . . . . . . . . . . . . . . . 57
4.4 Comparison of the actual number with the estimated number of Bare Concrete
Deck for Condition State 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.7 Probability Density Function (Weibull) of sojourn times for Bare Concrete
Deck for all Condition States. . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.8 Cumulative Distribution (Weibull) Function of sojourn times for Bare Con-
crete Deck for all Condition States. . . . . . . . . . . . . . . . . . . . . . . . 63
4.9 The Proportion of Bare concrete Deck in Each Condition State. . . . . . . . 64
4.10 The Number of Bare concrete Deck in Each Condition State. . . . . . . . . . 65
4.11 Expected Element Condition Index for Bare Concrete Deck over time based
on the Blacks (Semi-Markov) Approach and Markov Chain. . . . . . . . . . 68
4.12 Comparison of the actual number with the estimated number of Pourable Joint
Seal for Condition State 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.15 Probability Density Function (Weibull) of sojourn times for Pourable Joint
Seal for all Condition States. . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
xiii
4.16 Cumulative Distribution (Weibull) Function of sojourn times for Pourable
Joint Seal for all Condition States. . . . . . . . . . . . . . . . . . . . . . . . . 74
4.17 The Proportion of Pourable Joint Seal in Each Condition State. . . . . . . . 74
4.18 The Number of Pourable Joint Seal in Each Condition State. . . . . . . . . . 75
4.19 Expected Element Condition Index for Pourable Joint Seal over time based on
the Blacks (Semi-Markov) Approach and Markov Chain. . . . . . . . . . . . 77
4.20 Expected Element Condition Index for Reinforced Concrete Abutment over
time based on the Blacks (Semi-Markov) Approach and Markov Chain. . . . 80
5.1 Prediction of Element Condition Index for Pourable Joint Seal for Do-nothing
Actions over 12 years. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2 Prediction of Element Condition Index for Pourable Joint Seal for Do-nothing
5.3 Health Index vs. Age of Element: Expansion Joints (Source: Sobanjo and
Thompson (2011)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.4 Prediction of Element Condition Index for Bare Concrete Deck for Do-nothing
5.5 Prediction of Element Condition Index for Reinforced Concrete Abutment for
Do-nothing Actions over 12 years. . . . . . . . . . . . . . . . . . . . . . . . . 98
5.6 Prediction of Element Condition Index for Bare Concrete Deck for Do-nothing
5.7 Prediction of Element Condition Index for Reinforced Concrete Abutment for
Do-nothing Actions over 25 years. . . . . . . . . . . . . . . . . . . . . . . . . 99
5.8 Health Index vs. Age of Element: Decks and Slabs (Source: Sobanjo and
Thompson (2011)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.9 Health Index vs. Age of Element: Substructures (Source: Sobanjo and Thomp-
son (2011)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.1 Flow Chart for Determining Sojourn Time in a Particular Condition State for
Do-nothing Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.2 Probability Density Function of the Sojourn Time in each Condition State
(Element 301) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.3 Cumulative Density Function of the Sojourn Time in each Condition State
(Element 301) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
xiv
6.4 Deteration Curves for Element 301 (Pourable Joint Seal) . . . . . . . . . . . 116
6.5 Deteration Curves for Element 12 (Bare Concrete Deck) . . . . . . . . . . . . 117
6.6 Deteration Curves for Element 215 (Reinforced Concrete Abutment) . . . . . 117
7.1 Probability Density Functions of the Sojourn Time in each Condition State
for Do-nothing Action for Bare Concrete Deck . . . . . . . . . . . . . . . . . 132
7.2 Cumulative Density Functions of the Sojourn Time in each Condition State
for Do-nothing Action for Bare Concrete Deck . . . . . . . . . . . . . . . . . 132
7.3 Probability Density Function of the Sojourn Time in any Condition State for
Maintenance Action for Bare Concrete Deck . . . . . . . . . . . . . . . . . . 134
8.1 Survival Curves of Pavement Segments Service Life (3 models) . . . . . . . . 146
8.2 Survival Curves of Bridge Elements Service Life (3 models) up to 12 years . . 150
8.3 Survival Curve of Bridge Elements Service Life (Weibull) up to 100 years . . 150
xv
ABSTRACT
In this dissertation new methodologies were developed to address some of the existing needs
as it relates to Transportation Asset Management Systems (T AM S). The goal of T AM S
is to model the performance and preservation of transportation infrastructure. Currently,
traditional Bridge Management Systems (BM S) such as Pontis R
and BRIDGIT R
utilize
Markov chain processes in their performance and preservation models. Markov models have
also been suggested and used at some State transportation agencies for modeling the perfor-
mance of highway pavement structures. The Markov property may be considered restrictive
when modeling the deterioration of transportation assets, primarily because of the memo-
ryless property. In other words, the Markov property assumes that the sojourn times in the
condition states follows an exponential distribution for the continuous-time Markov chain,
and a geometric distribution for the discrete-time Markov chain. This research addresses
some of the limitations that arise from the use of purely Markov chain deterioration and per-
formance models for transportation infrastructure, by introducing alternative approaches
that are based on the semi-Markov process and reliability functions.
The research outlines in detail an approach to develop semi-Markov deterioration mod-
els for flexible highway pavements and American Association of State Highway Transporta-
tion Officials (AASHT O) Commonly Recognized (CoRe) Bridge Elements. This takes
into consideration the probability of transitions between condition states and the sojourn
time in a particular condition state before transitioning to another condition state. The
proposed semi-Markov models are compared against the traditional Markov chain models.
With Weibull distribution as the assumed distribution of the sojourn time in each condition
state, for both the pavement and bridge deterioration models, Maximum Likelihood Esti-
mation (M LE) was used to determine the estimates of the distribution parameters. For
the pavement deterioration, the comparison of the semi-Markov and Markov chain models
is presented, based on a Monte Carlo simulation of the condition. For the bridge element
deterioration, the proposed semi-Markov model is compared against another semi-Markov
approach outlined by Black et al. (2005a,b). A Bayesian-updated model was also compared
to the proposed semi-Markov model. The research findings on the semi-Markov modeling
validates the hypothesis that the rate of deterioration of pavements and bridge elements
tends to increase over time. The results obtained from this study outlined a feasible alter-
native method in which historical condition data can be used to model the deterioration of
pavement and bridge elements based on semi-Markov processes.
For pavement deterioration, the semi-Markov model appeared to be superior to that
of the Markov chain model in predicting the pavement conditions for the first five years
subsequent to a major rehabilitation. The approach by Black et al. (2005a,b), which was
xvi
applied to bridge element deterioration, assumes that the proportion of asset in state i at
interval t is equal to the total probability of that asset being in state i after the tth interval.
It was discovered that this may not be true when the sample size of the asset being analyzed
gets relatively small. Black et al. (2005a,b) used a least squares optimization technique to
estimate the parameters of the (Weibull) sojourn time distribution, obtaining local optimal
values, which may not best estimate the condition of the asset.
An adaptive control approach for modeling the preservation of CoRe Bridge Elements
based on Semi-Markov Decision Processes (SM DP ) is also outlined in this dissertation.
The methodology outlined in this study indicated that the use of SM DP can be used to
determine the minimum long-term costs for the preservation of bridge elements from the
CoRe Bridge Element data. The use of semi-Markov process to model deterioration re-
laxes the assumption of the distribution of the sojourn time between condition states for
deterioration and improvement works, and therefore the SM DP model is less restrictive
than Markov Decision Process (M DP ) model. Also, Reliability (survival) functions were
developed for both pavement segments and bridge elements to estimate their service lives.
The Weibull regression and Cox Proportional Hazards models developed showed the associ-
ation between factors, such as Average Daily Traffic (ADT ) and the environment, and the
condition of the asset over time.
The proposed methodology outlined above is being researched at a time when there
is a need for increased efficiency in the spending of government resources, while ensuring
the preservation of the nations transportation assets and network. The proposed stochas-
tic models are based on the principles of semi-Markov processes, and address some of the
limitations of the traditional Markov chain model. The survival analyses using the histor-
ical condition data allows for quick estimations as it relates to the service lives for bridge
segments and bridge elements.
xvii
CHAPTER 1
INTRODUCTION
This dissertation presents the development of a set of methodologies for the preservation of
key transportation assets at the network level. The term preservation typically refers to
a customer-focused program of activities undertaken to provide and maintain serviceable
roadways (Primer, 1999). It encompasses reconstruction, rehabilitation, and preventive
maintenance, as well as minor rehabilitation activities (Primer, 1999). The goal of in-
frastructure preservation is to cost-effectively and efficiently improve asset performance, as
measured by attributes such as ride quality, safety, and service life (Primer, 1999). Al-
though this definition refers to roadways, the term preservation has been also used to
encompass those activities aimed at the maintenance, repair and rehabilitation (MR&R)
of bridge assets and their elements such that their conditions remain above a minimum
threshold condition value.
Currently, Pontis R
and BRIDGIT R
Bridge Management systems utilize Markov chain
processes in their Bridge Management Systems. The Markov property may be considered
to be restrictive when modeling the deterioration of transportation assets, since the history
of the process is not taken into account when predicting the future (Ross, 1996). For
this research, semi-Markov models are developed, using historical condition data, to model
the deterioration of pavements and bridge elements. The semi-Markov models are then
compared against the traditional Markov chain models.
A main component of an Infrastructure Management System (IM S) is a network preser-
vation model, in which the optimal policy as it relates to the Maintenance, Repair and
Rehabilitation (M R&R) of the infrastructure is determined. The policy is a set of recom-
mended actions that minimizes the long-term M R&R cost requirements, while keeping the
infrastructure out of risk of failure. Ng and Moses (1999) outlined how the Semi-Markov
Decision Process (SM DP ) could be used to determine the minimum-cost long-term policy
for National Bridge Inventory (N BI) Bridge Elements. Ng and Moses (1999) did a case
study, using N BI data from the State of Indiana. However, the cost structure under each
option and the decision processes under rehabilitation or replacement alternatives were hy-
pothetical. In this study, the SM DP is used to determine the minimum-cost long-term
policy in accordance with the American Association of State Highway Transportation Of-
ficials (AASHT O)s Commonly Recognized (CoRe) Bridge Elements, in which the cost
structure under each option and the decision processes for M R&R actions are based on
actual data.
1
The service lives of flexible pavements and bridge elements are determined using Survival
Analyses or Reliability Theory techniques. Cox Proportional Hazards method and Weibull
regression are considered. The advantage of the Cox PH method is, it is semiparametric, i.e.
no assumption is made for the distribution of the baseline failure rate or hazard function, yet
the model allows for predictors within the model. The Cox PH method was previously ap-
plied to modeling flexible (asphalt concrete) pavements in Ohio by Yu (2005). Also, Mauch
and Madanat (2001) used Cox P H models to predict the distribution of times between
changes in condition states for reinforced concrete bridge decks in the N BI database. In
this dissertation the Cox Proportional Hazards method was used to develop survival curves
based on historical Crack Index (CRK) data, from which the estimated service lives of the
pavements and bridge elements were determined.
A Weibull regression for the service life was also done and a Wald test was performed
on the shape parameter to determine if the failure rate or hazard function was constant.
In addition to the semiparametric and the Weibull (parametric) models, a nonparametric
analysis was also done based on the Kaplan-Meier method (Kaplan and Meier, 1958). The
proposed methodology outlined above is unique and is being researched at a time when there
is a need for increased efficiency in the spending of government resources, while ensuring
the preservation of the nations transportation assets and network.
1.1 Problem Statement

There is a growing need to maintain and preserve transportation infrastructure by mak-
ing the best use of available funds, which are typically limited. The traditional Markov chain
models are more restrictive than other stochastic models, such as the semi-Markov models.
The transition probabilities used in current BM Ss, such as Pontis R
, are partially based
upon expert elicitation, which is relatively subjective. The Markov chain model specifies
that the transition probability of going from one state to another is independent on how the
item arrived at the current state or the extent of time it has been there. On the contrary,
if the asset condition levels can be described as condition states, then it is expected that
the longer a pavement or bridge element remains in a particular condition state, the more
likely that asset drops to a lower condition state. It therefore means that the deterioration
process of transportation assets may be described by a semi-Markov process in which the
distribution of the sojourn time in each condition state is more flexible than that assumed
by use of the traditional Markov chain model. In addition to the use of stochastic processes
to model the performance of pavements and bridge elements, the service lives of these assets
are associated with a number of factors such as traffic volume, environmental conditions
and the age of the asset, which also can be taken into consideration.
1.2 Research Objectives

The overall objective of this dissertation was to use stochastic techniques, such as semi-
Markov processes, to better model the performance and preservation of transportation
infrastructure. Furthermore, this research seeks to prove that the use of semi-Markov
models are superior to the traditional Markov chain models for modeling the performance
2
and preservation of transportation infrastructure . Below is a list of the individual research
objectives:
1. To develop a semi-Markov deterioration model for highway pavement in Florida, based

on their historical condition. Monte Carlo simulations will be used in the model to
predict future conditions due to deterioration based on the proposed semi-Markov
model and the traditional Markov chain model.
2. To develop semi-Markov deterioration models for AASHT O Commonly Recognized

(CoRe) bridge elements, based on historical condition of the bridges in Florida.
3. To determine the maintenance-effectiveness of transportation infrastructure network.

For the CoRe Bridge Elements, this will be done by combining and analyzing data
from Floridas Maintenance Management System (M M S) and the CoRe Bridge Ele-
ment inspection data.
4. To develop a network preservation model for transportation infrastructure, using an

Adaptive Control approach based on Semi-Markov Decision Process (SM DP ), which
can determine the minimum-cost long-term policy.
5. To determine the expected service lives for pavement and bridge elements using the
Cox Proportional Hazards model and other reliability functions.
1.3 Background on Pavement Management

Since 1977, California Department of Transportation (Caltrans) has been routinely
collecting condition data for its pavement network and using a pavement management
system (P M S) to manipulate this data in order to aid in the management of the network
(Lea, 2004). There is an increasing demand for pavement networks to be managed versus
just being only maintained. For example, it is likely to be more economical for money to
be spent to rehabilitate a pavement that is about to begin to fail rapidly, than one that
has already failed significantly. Figure 1.1 (Shahin, 2006) is a conceptual illustration of a
pavement condition life-cycle. The ability to determine both the current condition of the
pavement and to predict future conditions is of importance for any Pavement Management
System (P M S). The Pavement Condition Rating (P CR) is used as a measure of pavement
condition in the state of Florida. This PCR value is the minimum of the following three (3)
indices, which are also measured on 0 to 10 scales: 1) Crack Index; 2) Rut Index; and 3)
Ride Index. For all the indices, 10 represent the best condition and 0 represents the worst.
3
Figure 1.1: Conceptual illustration of a pavement condition life-cycle (Source:
Shahin (2006))
4
1.4 Background on Bridge Management
In 1967, the Silver Bridge between Point Pleasant, West Virginia, and Gallipolis, Ohio,
collapsed during rush hour traffic causing 46 civilians to die. The United States Congress
and the President established a law, which was the beginning of uniform standards for bridge
inspections. The law includes mandates for States to inspect all public road bridges of a
certain length and requires the U.S. Secretary of Transportation, in consultation with the
States, to maintain an inventory of these bridges, known as the National Bridge Inventory
(N BI) (Small et al., 2000; Dekelbab et al., 2008). Collapse of the I-35W Mississippi River
bridge in Minneapolis, MN, killing 13 people and injuring nearly 100 more, served as another
reminder that the nation cannot take bridge safety for granted (Dekelbab et al., 2008). The
National Bridge Inspection Standards (NBIS), established since the early 1970s, require
biennial safety inspections for highway bridges that are longer than 6.1 meters (20.0 feet)
on public roads. State and local Department of Transportations (DOT s) collect and report
information on the conditions and compositions of the structures (Dekelbab et al., 2008).
This information includes descriptions of the functional characteristics, location, geometry,
ownership, and maintenance responsibilities for each bridge (Dekelbab et al., 2008).
There are two main ways in which the conditions of bridges and their elements are
recorded. First, the federal requirement is to store the data inspected according to the
N BIS in the National Bridge Inventory (N BI). Secondly, many state agencies also inspect
and store data based on the American Association of State Highway Transportation Offi-
cials (AASHT O)s Commonly Recognized (CoRe) Bridge Elements. For the N BI, each
bridge is divided into four (4) main components: Deck, Superstructure, Substructure and
Channel and Channel Protection. The N BI also considers Culverts. Each of the four main
components and the culvert is rated on a 0 9 scale by severity of deterioration. For the
Deck, Superstructure and Substructure, 9 represents EXCELLENT CONDITION and 0
represents FAILED CONDITION. Although there have been some research in the develop-
ment of preservation models for bridges using National Bridge Inventory (N BI) data, not
many researchers have attempted to improve or develop new performance models using the
AASHT O CoRe Bridge Elements data.
For the CoRe Bridge Elements, approximately 160 elements were originally established,
with each bridge having an average of approximately 10 elements. The elements were
established during the 1990-91 development of the Pontis R
Bridge Management System by
F HW A and California Department of Transportation (Caltrans). The proportion of an
element in a particular condition state is determined from visual inspections of the bridge
elements. A CoRe Bridge Element can have either 3, 4 or 5 possible condition states plus
the failure state. Condition state 1 always represents the best condition for each element,
and the failure state represent the worst condition. The condition states for a CoRe bridge
element can be defined as: 1, 2, ..., N S, F , where N S is either condition states 3, 4 or 5,
depending on the element type, and F represents the failure state. Some examples of CoRe
bridge elements are listed below, with the number of condition states excluding the failure
state in parenthesis:
12 - Bare Concrete Deck (5 condition states)
13 - Concrete Deck/Slab with Asphalt Concrete Overlay (5 condition states)
5
215 - Reinforced Concrete Abutment (4 condition states)
301 - Pourable Joint Seal (3 condition states)
302 - Compression Joint Seal (3 condition states)
321 - Reinforced Concrete Approach Slab (4 condition states)
Caltrans developed the California Bridge Health Index (HI), a numerical rating of 0%
to 100% that reflects element inspection data in relation to the asset value of a bridge or
network of bridges (Shepard and Johnson, 2001). Figure 1.2 (Shepard and Johnson, 2001)
illustrates how the HI may change over time, in which there is a major project in program
year 5. The major project in program year 5 is reflected by an increase in the HI from 62%
to 100%. The life of the bridge in Figure 1.2 is extended by approximately eight (8) years.
The Element Condition Index (ECI) for a particular CoRe Brdige Element can also be
determined. When maintenance works are done to bridge elements the ECI also increases
to some condition index less than 100%. It is often the goal for transportation agencies to
ensure that the condition of their bridge elements do not fall below a minimum threshold
condition. Figure 1.3 gives a schematic of how the ECI of a bridge element may change over
time. In Figure 1.3 the bridge element at year 0 is new and it is not allowed to deteriorate
below a minimum ECI threshold value of 49%, by performing either a maintenance action
or a rehabilitation action. The maintenance action causes an increase in ECI to 86%, and
an extension of life of five (5) years, while rehabilitation increases the ECI to 100%.
6
Figure 1.2: Health Index for a bridge showing increased life due to a major project
in program year 5 (Source: Shepard and Johnson (2001)).
Figure 1.3: Element Condition Index for a bridge element showing increased life
due to two periodic maintenance and a major rehabilitation actions.
7
1.5 Stochastic Preservation Models
The Federal Highway Administration (F HW A) has the responsibility to work with var-
ious states to ensure that the nations highway system operates safely, including the approx-
imately 590,000 bridges that cross roadways, waterways and that link the transportation
system together. Stochastic preservation involves the use of probabilistic techniques to op-
timize policies that establish a desired level of service, minimize overall cost, while ensuring
that the assets are not at risk of failure. The traditional method used for the preserva-
tion of bridge elements is Markov Decision Processes (M DP ). Deterministic models have
also been developed to model the performance of bridge elements in accordance with the
N BI data (Yanev, 2007; Kim and Yoon, 2010). However, the uncertainties as it relates
to deterioration parameters, the subjectivity and inaccuracy of infrastructure inspection,
and the stochastic nature of the deterioration process led to a wide spread of stochastic
models (Morcous and Akhnoukh, 2006). The advantage of stochastic models is that they
are able to capture the physical and inherent uncertainty, model uncertainty, and statistical
uncertainty while predicting the future performance of infrastructure facilities (Lounis and
Mirza, 2001).
The M DP is based on the Markovian assumption, which states that the next state to
be visited depends only on the present state of the system. Costs are incurred as a result
of the decisions that are sequentially made at fixed epochs, as the systems evolves over
time. An infinite planning horizon can be assumed, in which the long-term cost per unit
time can be determined. The advantage of the SM DP model over the traditional M DP
model is that SM DP does not assume that decisions are necessarily made at fixed epochs.
The proposed SM DP model is an extension of the semi-Markov process in which the state
transitions are controlled by a set of actions that are associated with a set of costs. For the
semi-Markov process, subsequent to selecting the next state, the process spends a time in
the current state before transitioning (Ibe, 2009). The time that is spent prior to transition
is known as the sojourn time, which may be assumed to follow a particular distribution.
One of the key components of a Bridge Management System (BM S) is a network preser-
vation model, in which Maintenance, Repair and Rehabilitation (M R&R) action categories
are taken into consideration. To ensure that the preservation model remains consistently
optimal, the adaptive control approach is utilized. If new information is provided, the
control laws of the preservation model can be adjusted to ensure that the model remains
optimal. A do-nothing action occurs when no improvement action is done to the bridge
element, and the bridge element is allowed to deteriorate. For the do-nothing action it
is assumed that the deterioration behavior of each bridge element can be described by a
semi-Markov process, in which the time spent in each state except the failure state can be
defined by a Weibull distribution. The Weibull distribution is widely used in engineering,
and was originally proposed for the interpretation of fatigue data (Ross, 2002). The Weibull
distribution has been found to be an appropriate distribution to describe the sojourn time
between successive condition states for do-nothing actions (Black et al., 2005a,b; Sobanjo,
2011; Sobanjo et al., 2010), where a condition state is the state that represents a particular
condition of an asset. For this research, an Adaptive Control approach based on semi-
Markov decision processes (SM DP ) was used to model the preservation of bridge elements
over time, and the results are compared against that produced from the traditional Markov
8
Decision Processes model.
1.6 Dissertation Outline

In Chapter 2 the literature review outlines the previous work done in Transportation
Asset Management Systems (T AM S) that is pertinent to this research. In Chapter 3 a pro-
posed semi-Markov model for pavement deterioration is compared against the traditional
Markov chain model, using Monte Carlo simulation. Black et al. (2005a,b)s approach is
outlined and applied to the Core bridge element data in Chapter 4. The results from Chap-
ter 4 are used in Chapter 5 to develop an updated semi-Markov model, using a Bayesian
technique. Chapter 6 outlines another proposed semi-Markov model, which is compared
against the models outlined in Chapters 4, 5 and the Markov chain model used in Pontis R
.
The Adaptive Control methodology using semi-Markov decision processes (SM DP ) is de-
scribed and applied to the CoRe bridge element data in Chapter 7. Lastly, in Chapter 8
parametric, semiparametric and nonparametric techniques are used to estimate the service
lives of highway pavements and bridge elements. The pavement dataset used in Chapters 3
and 8 were obtained from the Florida Department of Transportation (F DOT ) condition
database. The bridge element dataset used in Chapters 4 to 8 were obtained from the
Floridas CoRe Bridge Element data records used in the Pontis R
BMS.
9
CHAPTER 2
LITERATURE REVIEW
2.1 Stochastic Models

Stochastic models include models that are based on Markovian techniques. Pontis R

R
and BRIDGIT Bridge Management systems utilize Markov chain models. A Markov
deterioration model is based on the assumption that the condition of a bridge element is
described in terms of a limited number of condition states, and transition probabilities are
used to link the current state to a future state with a maintenance action. The transition
probability represents the probability of the condition of an element changing from one state
to another when an action (including do-nothing action) is done. A Markov chain can be
defined as a discrete-time stochastic process Xn ; n = 0, 1, 2, . . . for which the Markovian
property hold (Ibe, 2009). The Markovian property refers to the memoryless property of
a stochastic process in which the future condition only depends on the present condition
and not on the past condition. Therefore for a Markov chain, the conditional probability
of moving into condition state j at time n + 1, given that at the current time n the object
is in condition state i can be mathematically expressed as follows (Ross, 1996):
Pij = P r {Xn+1 = j|Xn = i} (2.1)
Recently, researchers have been considering the applicability of semi-Markov process to

model the deterioration of assets. A semi-Markov process is a process in which the holding
times can have an arbitrary distribution and can depend on both the current state and the
state to be visited next (Ibe, 2009). A continuous-time Markov process is a special case of
a semi-Markov process, in which the sojourn time follows an exponential distribution.
2.2 Ng (1996)s Model

In Ng (1996) the semi-Markov process was expressed in terms of the transition matrix
P and the holding time Matrix H(t):
P = pij , i = 1, 2, .., N ; j = 1, 2, .., N

(2.2)
H(t) = {hij (t)} , t 0 (2.3)
10
where,
0 1 0 ... 0
0 0 1 ... 0
P = ... .. .. . . ..

(2.4)

. . . .
0 0 0 ... 1
0 0 0 ... 1
Note that pij = 1 for j = i + 1 and pij = 0 for all other values except the worst state. The
worst state is an absorbing state and therefore has a transition probability of 1.
Ng (1996) discussed a methodology in which the holding time probability density func-
tion (pdf ) hij (t) could be derived from the distributions of failure time T using the survival
analysis. Let Tj be the time to reach state j, then the hij (t), which is pdf Tij , can be
obtained from the knowledge of the distributions of Ti and Tj . Ng (1996) explained the dif-
ficulty in determining hij (t) by considering the difference of two arbitrary random variables
Z = X Y . The pdf of Z is:
Z
fZ (z) = fX,Y (x, y z)dx (2.5)

or Z
fZ (z) = fX,Y (y + z, y)dy (2.6)

and the joint distribution of X and Y is given by
fX,Y (x, y) = fY (y) fX|Y (x|y) (2.7)
where, fx|y (x|y) is the conditional distribution of X given Y . Therefore the pdf of Z is
given by Z
fz (z) = fY (x z) fX|Y (x, x z)dx (2.8)
z
The integration starts at z because x must be greater than z in order that f (x z) is
positive-valued. To derive an expression for the random variable z from the pdf s of X and
Y , the common assumption related to semi-Markov model is for Y and Z to be independent.
Therefore, Z x
fZ (z) = fY (x z) fZ (z)dz (2.9)
0
which is the integral convolution based on X = Y + Z, where Y and Z are independent.
The pdf of Z, fZ (z) is found by solving the above equation 2.9, but the solution is rather
complex (Ng, 1996).
2.3 DeStefano (1998)s Model

DeStefano (1998) focused on methodological enhancements for an infrastructure man-
agement system (IM S) for the New York State Thruways (N Y ST s) highway network, by
considering duration-based stochastic modeling. DeStefano (1998) looked at improving the
following four (4) models used for bridge network level maintenance decision process: 1)
11
condition assessment, 2) deterioration, 3) condition prediction, and 4) performance. The
IM S for the N Y ST s highway network also records each bridge component in a single
condition state like the N BI, however it is measured on a 1 to 7 scale. The bridge compo-
nents used are also similar to that used in N BI. In addition to deck, superstructure and
substructure, the IM S for the N Y ST s highway network also has Wearing Surface as a
bridge component. Uniform and Weibull distribution functions were approximated from
the condition data on the bridge components. Probability plotting and maximum likeli-
hood estimation were use to approximate the parametric distribution functions (DeStefano,
1998).
2.4 Ng and Moses (1999)s Model

As mentionted in Chapter 1, Ng and Moses (1999) outlined a Semi-Markov Decision
Process (SM DP ) based on a case study, using National Bridge Inventory (N BI) data
from the State of Indiana. Ng and Moses (1999) also outlined the difference between a
deterioration process that is based on a Markov chain and one that is based on a semi-
Markov process by the graphical representations in Figures 2.1 and 2.2 respectively. In
Figure 2.1, there is a single-step transition process with fixed transition probabilities between
states at each epoch (Ng and Moses, 1999). In Figure 2.2, Tij represents the sojourn time
in state i before dropping to the next worst state j (Ng and Moses, 1999). There is also a
probability of transition at Ti , known as the embedded Markov chain of the semi-Markov
process.
Figure 2.1: Single-step Transition Process of a Markov Chain (Source: Ng and

Moses (1999))
12
Figure 2.2: A Typical Sample Function of the Deterioration (Semi-Markov) Pro-
cess (Source: Ng and Moses (1999))
13
2.5 Black et al. (2005a,b)s Model
Black et al. (2005a,b) developed a Semi-Markov Approach for modeling asset deterio-
ration. One of the datasets used in Black et al. (2005a,b) to assess the semi-Markov model
was that of paintwork on 11kV ground-mounted transformers. A sample of 309 transformers
that were inspected between 1998 and 2000, were taken from a United Kingdom (UK) elec-
tricity companys database and used to develop the model. The condition of the paintwork
was categorized into the following four states: good, some peeling, noticeable rust and
extensive rust and according to age in years. The quantity of transformers in each state
for each time interval in years was recorded in a dataset, shown in Table 2.1.
Table 2.1: Number of transformers in each paintwork class grouped by the number
of years since the last refurbishment (Source: Black et al. (2005b))
No. in each state

Time 1 2 3 4
(years)
1 5 1 0 0
2 1 1 0 0
3 25 3 0 0
4 20 2 0 0
5 11 3 0 0
6 12 6 3 0
7 21 9 1 0
8 14 14 0 1
9 21 14 0 0
10 21 10 1 0
11 27 19 5 0
12 7 8 3 0
13 6 5 1 0
14 5 2 1 0
The implementation of the semi-Markov model by Black et al. (2005a,b) was based on
the following modifications to the more general semi-Markov models.
Time is divided into intervals and items can only transition once in each interval.
During a time interval, items can only either stay or move to the next state.
The time items spend in a given state is assumed to follow a Weibull Distribution.
The probability that specified the semi-Markov model was pi,j (m), which is the probability
of changing from state i to state j during the mth time interval after entering state i,
conditional on having been in state i for m 1 time intervals.

1 pi,j+1 (m) if j = i;
pi,j (m) = (2.10)
0 if j 6= i or j 6= i + 1 except for worst.
14
After the parameters of the Weibull Distribution for time spent in state i was estimated,
the pi,j (m) was calculated. Based on the dataset of the condition of the paintwork in
Table 2.1, the number of time intervals since refurbishment, t, is known rather than the
number of time intervals since entering state, m. Therefore, the underlying probability for
the observation is qi (t), the probability of being in state i after the tth time interval since
last refurbishment. qi (t) can be expressed in terms of pi,j (m) and values of the Weibull
parameters can be selected so as obtain values of qi (t) that are as closely as possible to
the fractions needed to obtain the values in Table 2.1, the least squares method is used to
obtain the best values for qi (t). The following series of equations are used to express qi (t)
in terms of pi,j (m):
n1
X
pi,i+1 (t) = pE
i (t 1, k) pi,i+1 (k + 1) (2.11)
k=0
k
Y
pE
i (t, k) = pi1,i (t k) (1 pi,i+1 (m)) (2.12)
m=1
t
X
qi (t) = pE
i (t, k) (2.13)
k=0
where,
pE
i (t, k) is the probability of being in state i after interval t and having been in state i for
k intervals.
pi,i+1 (m) is the probability of changing from state i to state i + 1 on the k th interval after
entering state i, conditional on having been in state i for m 1 intervals.
qi (t) is the total probability of being in condition state i after the tth interval since
becoming new.
The following formula is used to calculate pi,i+1 (m):
Fi (m) Fi (m 1)
pi,i+1 (m) = (2.14)
1 Fi (m 1)
The Blacks Approach assumes that the total probability of being in condition state i after
the tth interval since becoming new, is equal the overall proportion of bridge element in
condition state i in interval t.
2.6 Cox Proportion Hazards Model

The Cox Proportional Hazards (P H) Model is a semiparameteric model used in Survival
Analysis. Consider the hazard function with the following form:
h(t) = h0 (t)exp[G(X, )] (2.15)
15
where h0 (t) is a baseline hazard function, G is an arbitrary known function of influential
factors, X, that could affect pavement service life, and is the coefficient vector of parameter
X, then the associated survival is called a PH model (Berthold and Hand 2002).
Yu (2005) used the Cox Proportional Hazard (PH) Model to predict pavement condition
based on influential factors using a dataset on asphalt overlays on flexible pavements in
Ohio. Based on the proportionality assumption of the Cox PH, if there are two pavements
with different values of influential factors, then the ratio of the hazards to fail for these
two pavements at any time does not depend on time. The Cox PH model is based on the
assumption that there is a log-linear relationship between the influential factors and the
underlying hazard function. The hazard function is an indication of how likely a pavement
will fail at time t, given that it has not failed before time t. The hazard function of pavement
is typically expected to increase with time based on the aging of pavement material and
the cumulative effect of traffic loadings. The h0 (t) is frequently called the baseline hazard
since for any given pavement segment with known parameter x, the corresponding hazard
can be obtained by multiplying function G by h0 (t). The base shape of the hazard function
of a survival model is determined by the baseline hazard.
If function G has the form,
G(X, ) = 1 x1 + 2 x2 + ... + n xn (2.16)
and the form of h0 (t) is arbitrary, then
h(t) = h0 (t)exp(1 x1 + 2 x2 + ... + n xn ) (2.17)
is called Cox Proportional Hazards Model (Berthold and Hand, 2002). Equation 2.15 is
frequently called a semi-parametric model since h0 (t) is estimated at each point in time
without explicit expression and G is parameterized. Based on equation 2.15, the corre-
sponding survival function can be expressed as:
S(t|x) = S0 (t)exp(x) (2.18)
where, x are values of the variables and S0 (t) is the baseline survival function, which is
sometimes called the Flemington-Harrington Estimate (Cleves et al., 2001). Yu (2005) used
the Wald (Chi-square) statistics to check the Proportional Hazard assumption for each
predictor variable.
2.7 Rehabilitation-Cost Effectiveness: Existing

Methodologies/Theories for Optimization
Juni et al. (2008) developed a method to estimate the cost of ongoing routine operations
and the maintenance cost components of total highway life-cycle cost using historical data
by: 1) conducting a correlation study to characterize the relationship between maintenance
cost and maintenance condition; 2) Develop model equations for estimating the recurring
annual cost for routine, county-level highway maintenance.
Sobanjo et al. (2002) estimated the cost of Maintenance, Repair and Rehabilitation of
Florida Bridges. One of the major problems expressed was matching the historical cost
16
data record for MR&R action on a bridge with the deteriorated state (Pontis R
definition)
of the bridge when this action was performed. The solution to this problem was to elicit
expert opinions and prorate the same cost between different states, using the ratio in the
Pontis R
cost data from the Clemson study (Elzarka et al., 1996).
A number of optimization techniques are used in IM Ss, including linear programming,
dynamic programming and multiple objective analyses. If a problem can be modeled as a
linear objective function subject to a set of linear constraints, linear programming can be
used to develop an optimum strategy for the system described by the model (Jewel, 1986).
Dynamic Programming is used to optimize systems possessing a serial structure. A serial
structure consists of components connected head to toe with no recycle. For those problems
in which no one dominant objective can be identified, a Multiobjective analysis approach is
required (Jewel, 1986). Multiobjective analysis is a framework for optimization of systems
to meet these multiple objectives (Jewel, 1986).
2.7.1 Linear Programming

Linear Programming is an optimization tool in which the objective function and the
constraints are linear functions of the variables involved (Kaplan, 1982). The tool has
been used in infrastructure management, sometimes in conjunction with other optimization
techniques (Stukhart, 1991; Chassiakos et al., 2005). In Chassiakos et al. (2005) a weighted-
objective linear programming model was used to optimize the maintenance schedule of
highway concrete bridges.
2.7.2 Dynamic Programming

Dynamic Programming is used in a number of Infrastructure Management Systems,
including Pontis R
Bridge Management System. Robelin and Madanat (2006) developed a
reliability-based optimization model for the maintenance and replacement of bridge decks,
which was also based on dynamic programming. In Pontis R
the Maintenance, Repair and
Rehabilitation (MR&R) optimization model uses policies to minimize long term mainte-
nance costs while keeping the bridge elements out of risk of failure at the network level
(Golabi et al., 1993). The Dynamic Programming formulation is shown below:

X
V (i) = minaA C(i, a) + d Pij (a)V (j) (2.19)
j
where,
V (i) = total expected long-term discounted cost,
i = the condition state of an element,
a = set of feasible Maintenance, Repair and Rehabilitation (M R&R) actions for condition
state i,
C(i, a) = the expected cost for an M R&R action a in condition state i
17
d = the discount factor or present worth factor for a single time period, [defined as d =
1/(1+int), where int = interest rate],
j = successor condition state of bridge element one year after action a is taken, and
Pij (a) = the probability that the bridge element will transition from state i to state j one
year after action a is taken.
2.7.3 Multiobjective optimization

Multi-objective optimization (or programming), also known as multi-criteria or multi-
attribute optimization, is the process of simultaneously optimizing two or more conflicting
objectives subject to certain constraints (Steuer, 1986; Sawaragi et al., 1985). Patidar
et al. (2007) is a recent study in which multiobjective optimization was considered for
application in bridge management systems. The following techniques are applicable under
Multiobjective optimization:
Incremental Benefit-Cost (IBC). The IBC heuristic maintains a list of investment
candidates sorted by their benefit-cost ratio. Benefit can be any measure that is additive
over the bridge inventory, so an increase in benefit on one bridge reflects a benefit by the
same amount for the entire inventory. For each bridge, a set of alternative candidates is
defined, and the rule of diminishing marginal returns is essential to the heuristic, such that
candidates failing to satisfy the rule are eliminated from consideration.
The Project Level Analysis and Network Analysis Tools (Sobanjo and Thompson, 2004,
2007; Thompson and Sobanjo, 2006a,b) were developed to fulfill the needs to make project-
level and network-level decisions. The Network Analysis Tool was developed as a bridge
decision support tool for network analysis. It uses multi-objective benefit/cost analysis
capable of: a) predicting system wide performance based on a given budget level and b)
funding requirement to meet a performance target level. Performance measures that are
compatible with the Project Level Analysis Tool (P LAT ), the F DOT Deficient Bridge List
(DBL) and other models inside the Pontis R
bridge management system were developed. An
optimization model using the incremental-benefit algorithm, with the performance measures
serving as constraints and objectives, was formulated and developed.
The Incremental Utility-Cost (IU C). IU C heuristic is based on similar concepts
to those of the IBC method, in which the benefit or reward of a project is measured
in terms of the total utility. The utility is a function of multiple performance measures.
The IU C heuristic orders candidates in decreasing order of IU C values and selects the
candidate which has a high IU C. The National cooperative Highway Research Program
(N CHRP ) (Patidar et al., 2007) outlines a multiobjective approach for bridge management
systems at the Network-level that uses IU C. The problem was treated as a multi-choice
milti-dimensional knapsack problem (M CM DKP ). The basis for which alternative bridge
actions could be evaluated was developed by establishing the following set of goals and
performance criteria: 1) Preservation of bridge condition, 2) Traffic safety enhancement, 3)
Protection from extreme event, such as earthquake, collision, over-load and other human-
made hazards, 4) Agency cost minimization, 5) User cost minimization.
For decision-making problems, a class of solution technique predicated on utility theory,
was utilized. It is based on the premise that the decision makers preference structure can
18
be represented by a utility function. The three major steps that are generally involved in
the construction of this utility function are: weighting, scaling and amalgamation. The
underlining assumptions and appropriate assessment methods for this approach depends
on whether the decision making problem is one of certainty or uncertainty. The next step
was optimization, where the researchers sought to identify that which yields the maximum
value for the objective function.
Genetic Algorithm. Genetic Algorithm (GA) is a robust technique formulated on
the mechanics of natural selection and natural genetics (Holland, 1975). GAs operate by first
manipulating a pool of feasible solutions to identify and explore properties simultaneously.
Secondly, GAs employ probabilistic transition rules which enable the generations of solutions
from the existing pool of previous solutions. Thirdly, GAs select better solutions in each
step by the comparison of the objective function values of the generated solutions.
Fwa et al. (2000) used GA to generate and identify better solutions for pavement man-
agement until convergence was reached. Fwa et al. (2000) developed a genetic-algorithm
(GA) based procedure for solving multiple objective network level pavement problems. The
objective of his study was to present an approach to maximize work production, minimize
total maintenance cost and maximize overall network pavement condition. The constraints
were production requirements, budget constraint, manpower availability, equipment avail-
ability, and rehabilitation schedule constraints. Fwa et al. (2000) had a rank-based approach
for defining the fitness of a solution. A Pareto frontier was generated by all solutions with
a ranking of 1. This set of solutions are such that none are superior in all attributes (or
objective function) to each other. For two objectives, the Pareto frontier is defined by a
curve, and for three or more objectives the Pareto frontier is defined by a surface. Liu and
Frangopol (2006) and Miyamoto et al. (2002) also used Genetic Algorithm in multiobjective
problems for infrastructure management. Liu and Frangopol (2006) had three minimization
objective functions: maintenance cost, bridge failure cost, and user cost. In Miyamoto et al.
(2002) prioritizations are placed on the individual objective functions that are considered,
so each objective function has a level of importance.
2.8 Adaptive Control Methodology

An Adaptive Control process can be described by three (3) main functions: identifi-
cation, decision, and modification (Cooper et al., 1960; Eveleigh, 1967). Identification is
defined as the process by which the system is characterized (Eveleigh, 1967), which in the
case of an asset preservation model, it is the set of Maintenance, Repair and Rehabilitation
(M R&R) actions and their effectiveness. The decision process is the means by which mea-
surements are used to decide how the system performance relates to the desired optimum
(Eveleigh, 1967). Modification is the process of changing parameters of the system toward
the optimum setting, as controlled by the identification and decision processes (Eveleigh,
1967). Research in adaptive control has a long and vigorous history. In the 1950s, it was
motivated by the problem of designing autopilots for aircraft operating at a wide range of
speeds and altitudes (Sastry and Bodson, 1989).
Adaptive Control formulations have been developed for application in IM Ss. Durango
and Madanat (2002) looked at its application by taking into consideration the uncertainty
19
involved in characterizing a facilitys deterioration rate in the process of formulating main-
tenance and repair policies. The optimization model used in Durango and Madanat (2002)
minimizes the user and maintenance and repair costs of pavement segments over a period.
Guillaumot et al. (2003) improved on the model by Durango and Madanat (2002) by consid-
ering the uncertainty as it relates to the inspections or measurements done to the pavement
facilities. Gonzalez et al. (2006) looked at its application to railway lines to also develop
maintenance and repair policies. However, Gonzalez et al. (2006) optimization model is
based on minimizing the risk cost over a period; risk is treated as a function of the dete-
rioration level and criticality. Madanat et al. (2006) used systematic probing for selecting
optimal M R&R policies for pavement infrastructure. Some of the main BM Ss that are
being used, such as Pontis R
, has an updating model for the set of transition probabili-
ties used in its preservation model, and hence could also be considered adaptive (Madanat
et al., 2006). The adaptive control methodologies outlined by Durango and Madanat (2002),
Guillaumot et al. (2003), Gonzalez et al. (2006), and the techniques currently used in the
traditional BM Ss are all based on M DP , in which dynamic and or linear programming
are used as the optimization tool.
In this dissertation SM DP is used as the optimization and decision making tool for
the adaptive control model in accordance with the American Association of State High-
way Transportation Officials (AASHT O)s defined CoRe (Commonly Recognized) Bridge
Elements Standard, using actual condition and cost data. The proposed SM DP model ac-
counts for the uncertainties in the deterioration rate and the time it takes for maintenance
works to get done.
20
CHAPTER 3
COMPARISON OF SEMI-MARKOV AND

TRADITIONAL MARKOV CHAIN MODELS
3.1 Introduction
The preservation of flexible (asphalt) pavements involves developing models that are able
to estimate the performance of pavement networks. Markov chain has been used to model
the performance of pavements in a number of Pavement Management Systems (P M Ss),
such as The Arizona Department of Transportation (ADOT ) Network Optimization Sys-
tem (N OS) (Wang, 1992; Wang et al., 1994). Wang et al. (1994) outlined an approach
to generate transition probabilities of a Markov chain model based on current pavement
performance data. Nasseri et al. (2009) used the application of Markov chain to investigate
the crack histories of flexible pavements and to determine the cause of rapid deterioration
of surface cracks. Markov chain has also been used to model the deterioration of other
infrastructures, including bridge elements, storm water pipes and wastewater pipes (Golabi
et al., 1993; Micevski et al., 2002; Baik et al., 2006). The use of semi-Markov processes for
modeling the crack performance of flexible pavements was mentioned by Yang et al. (2005,
2009) as a precursor to outlining how recurrent Markov chains can be used to model crack
performance of flexible pavements. Semi-Markov processes have been used in deterioration
models for other assets such as bridge elements and transformers (Ng and Moses, 1998;
Sobanjo, 2011; Black et al., 2005a,b). The main difference between semi-Markov processes
and Markov chains is that a Markov chain assumes that the sojourn time in one state before
transitioning to another follows an exponential distribution for continuous-time and a geo-
metric distribution for discrete-time, while for a semi-Markov process the sojourn time can
have any continuous-time distribution (Ng and Moses, 1998; Howard, 1971). The sojourn
time is the time from when a unit of pavement segment first enters a particular state to the
time it enters another state. In this chapter the states are based on the condition level of
the pavement and are here after referred to as condition states. Also, a Weibull distribution
is assumed for the sojourn times in the proposed semi-Markov deterioration model.
In this chapter, Monte Carlo Simulation will be used to model the deterioration of
flexible pavements using transition probabilities derived from both Markov chain and semi-
Markov deterioration models. The results are compared against the actual or observed
pavement deterioration behavior obtained from historical pavement condition data. The
transition probabilities for do-nothing actions derived from the semi-Markov deterioration
21
model takes into consideration the probability of transitions of the embedded Markov chain
and the sojourn time in a particular condition state before transitioning to another. The
Markov chain approach, as outlined by Wang et al. (1994), was applied in this chapter to
determine the general transition probabilities for do-nothing actions between condition
states, where do-nothing actions means that there was no intervention in the deterioration
process. It is assumed that once the asset falls out of a condition state, that condition state
is not visited again. A comparison between the Markov chain model, semi-Markov model
and the actual pavement deterioration behavior in predicting the decrease in condition state
over time will be done. Based on the results of the simulation, survival curves for particular
condition states will also be determined. The methodology outlined in this chapter can
be used in conjunction with additional engineering properties, by categorizing pavement
segments with similar engineering properties. The use of statistical tools, such as semi-
Markov modeling, which takes into consideration the age of the infrastructure, can provide
insights into the expected behavior of transportation structures at the network level. It also
takes into consideration the overall inherent uncertainties as it relates to structure of the
pavement.
3.2 Methodology
3.2.1 Data Description
The Florida Department of Transportation (F DOT ) uses a Pavement Condition Rating
(P CR), on a 0 to 10 scale, as a measure of the overall condition of a segment of pavement.
This PCR value is the minimum of the following three (3) indices, which are also measured
on 0 to 10 scales: 1) Crack Index; 2) Rut Index; and 3) Ride Index. For all the indices, 10
represent the best condition and 0 represents the worst. The F DOT Pavement Manage-
ment System (P M S), called Florida Analysis System for Targets (F AST ), is a Statistical
Analysis System (SAS) based software that is used to predict future pavement condition.
In F AST , regression equations based on the historical performance of pavements in a par-
ticular district are used to predict the performance of pavements within that same district
(Dietrich, 2010). The Crack Index (CRK) is the most critical indicator that typically gov-
erns the overall roadway condition. The CRK is obtained by visual inspections carried
out by a survey crew and inspection vehicle, which collects data as it traverses a section
of pavement (Yang, 2004). It is known that the accurate prediction of crack condition is
integral to any P M S (Yang et al., 2005, 2009; Yang, 2004).
For the purpose of this study, the Markov chain and Semi-Markov models were based on
the analysis of the CRK values of flexible pavements extracted from the F DOT pavement
condition database. The data was organized such that the condition of pavements were
tracked from the year they became new or overlaid, regardless of the thickness of the
asphalt layer, and while the conditions in the subsequent years either remained the same or
dropped to a lower value. For this chapter, a network level analysis of pavement condition
using CRKs as a measure for condition is done. One thousand, five hundred and eighty
(1,580) pavement segments were selected from the F DOT pavement condition database
containing 8,597 pavement segments. This selection of pavement segments were selected
to ensure that all pavements were flexible (asphalt concrete) pavements and that there
22
were no missing data as it relates to the following attributes for the years 1986 to 2005:
Beginning Mile Post, Ending Mile Posts, the Year overlaid, Crack Index (CRK), Rut Index
(RU T ), Ride Index (RIDE), Average Daily Traffic (ADT ), Pavement Condition Rating
(P CR) and Truck Percentage. From the 1,580 pavement segments selected, the Average
Daily Traffic (ADT ) ranged from 500 to 183,500 and the truck percentage ranged from 0.68
to 59.28. The pavement segments in the database consists mainly of interstates and state
roads throughout Florida.
3.2.2 Training and Test Data

Fifty percent (50%) of the 1,580 selected pavement segments were sampled randomly
and used as training data to develop the Markov chain and semi-Markov models. The
results from the models were compared against the actual behavior of the remaining 790
pavement segments, obtained from the test data. In order to reflect solely do-nothing
actions, condition data that reflected improvements to pavement segments were filtered out,
that is, for both the training and test data, the tracking was stopped at the point where
the CRK either increased or at the year at which the study ended. For this study, seven
(7) condition states were formulated as described in Table 3.1.
Table 3.1: The Range of Crack Indices and Corresponding Condition States
Crack Index (CRK) Range Condition State

9.5 CRK 10 10
8.5 CRK < 9.5 9
7.5 CRK < 8.5 8
6.5 CRK < 7.5 7
5.5 CRK < 6.5 6
4.5 CRK < 5.5 5
CRK < 4.5 4
3.2.3 Markov Chain Model

A stochastic process, known as a Markov chain (Ross, 1996), can be described as follows:
If Xn = i describes a process such that the process is in state i at time n, and the process
in state i has a fixed probability Pi,j of being in state j, after a transition, then
P {Xn+1 = j|Xn = i, Xn1 = in1 , . . . , X0 = i0 } = Pi,j (3.1)

for all states i0 , i1 , , in1 , i, j and all n 0
The following equation, obtained from Wang et al. (1994), can be used to generate
transition probabilities for the Markov chain model:
mi,j(ak )
pi,j (ak ) = (3.2)
mi (ak )
for i, j = 10, 9, 8, 7, 6, 5 & 4 where,
23
k = k th rehabilitation action, in this case, the do-nothing action, i.e. k = 1.
pi,j (ak ) = transition probability from state i to j after action k is taken.
mi,j (ak ) = total number of miles of pavement for which the state prior to action k was i
and the state after the action k was j.
mi (ak ) = total number of miles of pavement for which the state prior to action k was i.
For this study, a MATLAB R

Computing software program was formulated to perform
the calculations of Equation 3.2 for do-nothing actions, using CRKs of 790 pavement
segments from the training data. This resulted in the generation of a set of transition
probabilities that was used to model the yearly transitions. Details of the simulation are
outlined in section 3.2.5.
3.2.4 Semi-Markov Model

To describe the semi-Markov process, consider a stochastic process with states 0, 1, 2, . . . ,
which is such that whenever it enters state i, i 0, then: 1) it will enter the next state j with
probability Pij , i, j, 0 , and 2) given that the next state is j, the sojourn time form i to j has
distribution Fij . For a semi-Markov process, the sojourn times can follow any distribution.
The method of Maximum Likelihood Estimation (M LE) is used to estimate the Weibull
distribution parameters. Before discussing the details of the semi-Markov model, it is
prudent to define the basic concepts of the M LE method. The Maximum Likelihood is
one of the most popular techniques in statistics for deriving estimators (Casella and Berger,
2001).
Maximum Likelihood Estimation. Consider an identical and independently dis-

tributed (iid) sample, X1 , ..., Xn from a population with probability density function (pdf)
or probability mass function (pmf) f (x|1 , , k ), the likelihood function is defined by
L(|X) = L(1 , ..., k |x1 , ..., xn ) = ni=1 f (xi |1 , ..., k ) (3.3)

The pdf of the Weibull distribution is defined by (Billington and Allan, 1983; Tobias and
Trindade, 1995):
1
t t
f (t) = e ( ) (3.4)

where and are scale and shape parameters respectively. t is the number of years each
unit of a mile of pavement segment takes to sojourn from one condition state to another.
If = 1/, then

f (t) = (t)1 e(t) (3.5)
it follows from Equation 3.3 (Birolini, 2007) that the likelihood is as follows:
24
n

L(t1 , ..., tn , , ) = e t1 + ... + tn ni=1 t1
i (3.6)
After differentiating the log likelihood and setting it to zero, solve for the M LE of the
parameters and , which are:
"P n
#1
n
i=1 ti ln (ti ) 1X
= ln (ti ) (3.7)
Pn n
i=1 ti i=1
and
" #1/
n
= P (3.8)
n
i=1 ti
However, if there are n units of pavement in a particular condition state and k units
have transitioned to a lower condition state (with complete individual sojourn times t1 <
t2 < . . . < tk ) and the sojourn time for n k units of pavement are unknown, then the
sojourn times T1 , ..., Tn k for the n k units of pavement should also be accounted for in
the evaluation. For the Weibull distribution, assuming that the incomplete sojourn times
of T1 , ..., Tnk have been observed in addition to the complete sojourn times t1 , ..., tk , then
the likelihood function is defined by:
L = ki=1 f (ti |) nk
j=1 f (1 F (Tj , |)) (3.9)
where i sums over all completed sojourn times, j sums over all incomplete sojourn times,
and can be a vector.
k
(Tj )
L(t1 , ..., tn , , ) = e t1 + ... + tk ki=1 t1
i nk
j=1 e (3.10)
yielding the M LE of the parameters and below:

P 1
k Pnk k
i=1 ti ln (ti ) + j=1 Tj ln (Tj ) 1 X
= ln (ti ) (3.11)
Pk Pnk
Tj k
i=1 ti + j=1 i=1
and
1/
k
= P (3.12)

k
ki=1 Tj
Pnk
i=1 ti +
25
Semi-Markov Kernel. To understand how the semi-Markov can be developed by
knowing the sojourn times in a particular condition state before transitioning, one may
consider the semi-Markov kernel in the form shown in Equation 3.13. Ibe (2009) defines
the one-step transition probability Qi,j (t) of the semi-Markov process as:
Qi,j (t) = P [Xn+1 = j, Gn t|Xn = i] t0 (3.13)

where Qi,j (t) is the conditional probability that the process will be in state j next, given
that it is currently in state i and the waiting time in the current state i is no more than t.
Gn is the time the process spends in i before making a transition to j. It also follows that:
Qi,j (t) = pi,j Hi,j (t) (3.14)

where pi,j is the transition probability of the embedded Markov chain, and
Hi,j (t) = P [Gn t|Xn = i, Xn+1 = j] (3.15)
Semi-Markov Process. Howard (1971) presented the following formulation to de-

termine the probability that a continuous-time semi-Markov process will be in state j at
time n given that it entered state i at time zero.
N
X Z n
>
ij (n) = ij wi (n)+ pik hik (m)kj (nm) i = 1, 2, ..., N ; j = 1, 2, ..., N ; n = 0, 1, 2, ...
k=1 0
(3.16)

1 i = j;
ij = (3.17)
0 i=6 j.
where,
ij (n) is the probability that a continuous-time semi-Markov process will be in state j at

time n given that it entered state i at time n = 0, and is called the interval transition
probability from state i to state j in the interval (0, n).
>w
i (n) is the probability that the process will leave its starting state i at a time greater
than n.
The second term in Equation 3.16 describes the probability of the sequence of events, in
which the process makes an initial transition from state i to some state k at some time m
and thereafter proceeds from state k to state j in the remaining time n m. To consider all
possible scenarios, the probability is summed over all states k to which the first transition
could have been made and over all times of the initial transition, m, between 1 and n. pik
is the probability from transitioning from i to k, and hik (m) is the probability distribution
of the sojourn time from i to k at time m. The matrix formulation of Equation 3.16 is:
26
Z n
(n) => W (n) + [P H(m)](n m) n = 0, 1, 2, ... (3.18)
0
Also, let
C(m) = [P H(m)] (3.19)

and C(m) is defined as the core matrix (Howard, 1971). The elements of C(m) are cij (m) =
pij hij (m), where pij is the transition probability of the embedded Markov chain and hij (m)
is the probability distribution of the sojourn time in state i, before transitioning to j at
time m.
It therefore means that the interval transition matrix representing a single transition at
time m, 0,m (m), for do-nothing actions can be represented as:
1 9j=4 p10,j H10,j (m) . . . . . . . . . . . . p10,5 H10,5 (m) p10,4 H10,4 (m)
P
..

0 . ... ... ... p9,5 H9,5 (m) p9,4 H9,4 (m)
. .

0 0 . ... ... p8,5 H8,5 (m) p8,4 H8,4 (m)
0,m (m) =

. .

0 0 0 . ... p7,5 H7,5 (m) p7,4 H7,4 (m)
..
.

0 0 0 0 p6,5 H6,5 (m) p6,4 H6,4 (m)
0 0 0 0 0 1 p5,4 H5,4 (m) p5,4 H5,4 (m)
0 0 0 0 0 0 1
(3.20)
where pij is the probability for the embedded Markov chain of the semi-Markov process,
and Hij is the cumulative distribution of the sojourn time between condition state i and
condition state j at time m.
As the number of years, m, increases it can be seen by Equation 3.16 that a number
of permutations must be taken into consideration to compute the overall transition proba-
bilities for an interval (0, n), and so another approach is suggested in this study to model
the overall transition over time. In the context of modeling asset deterioration, this study
is focused on the do-nothing scenarios, where there is no intervention in the deterioration
process. It is assumed that once the asset falls out of a condition state, that condition
state is not visited again, and so the semi-Markov process occurs in one direction only.
There exists also the probability of the pavement segments skipping condition states in a
single transition; however the probability of skipping two (2) or more condition states is
extremely small and therefore not considered in the proposed model. It is assumed that
only one condition state may be skipped in a single transition, therefore the transition
probability (for the embedded Markov chain), pij , of skipping a condition state, k, is one
minus the transition probability (for the embedded Markov chain) of transitioning to the
next state, pik . In other words,
pij = 1 pik i = 10, 9, 8, 7, 6, 5; j = i 2, k = i 1, min(j, k) = 4 (3.21)

This is true because the overall probability of eventually leaving a particular condition state,
other than the failure state, to a lower condition state is 1, since only deterioration is being
considered.
27
It is also assumed that the sojourn time in condition state i before transitioning to j is
the time from the pavement segment first entered condition state i to the time the pavement
segment first entered condition state j. The transition diagram below in Figure 3.1 explains
the assumptions for possible single transitions in developing the semi-Markov model, where
pij is the probability for the embedded Markov chain from condition state i to condition state
j, and hij (m) is the probability density function of the sojourn time between condition state
i and condition state j at time m. The term single transition is based on the assumption
that only a single transition can occur in a year. For the transition labeled p10,9 h10,9 (m) the
pavement segment spends some time in condition state 10 before transitioning to condition
state 9, and for the transition labeled p10,8 h10,8 (m) the pavement segment spends some time
in condition state 10 before transitioning to condition state 8 without visiting 9.
Figure 3.1: Transition diagram that represents the possible single transitions
between condition states
Instead of determining the interval transition probabilities for the interval (0, n), the
conditional transition probabilities for each yearly interval (m 1, m] for m = 1, 2, . . . , n
is determined and multiplied by each other to approximate the transition probabilities for
the interval (0, n). It is assumed that only a single transition can occur in a year. In other
words, let
0,n (n) = 0,1 1,2 . . . n1,n (3.22)

where m1,m is a one year single transition probability matrix from time m 1 to m (i.e.
mth interval), m = 1, 2, . . . , n. Based on Equation 3.20, if it is assumed that condition states
can have a maximum drop of two condition states, then the interval transition probability
for the first year results to:
1 9j=8 p10,j H10,j (m) p10,9 H10,9 (m) p10,8 H10,8 (m) 0
P
0 0 0
..

0 . . . . . . . 0 0 0

..

0 0 . . . . . . . 0 0

0,m (m) =
..
0 0 0 . . . . . . . 0

..

0 0 0 0 . . . . . . .

. . . . . .

0 0 0 0 0
0 0 0 0 0 0 1
(3.23)
28
where m = 1. However to determine the transition probability matrix for the subsequent
intervals, a different formulation is used, in which it is assumed that the sojourn time is left
truncated at the start of each interval.
Survival Analysis and Left Truncation. Statistical analyses of lifetime, i.e. sur-
vival time or failure time data, have been developed into useful topics, for workers in the
field of engineering and biomedical sciences. For the biomedical field, the term Survival
Analysis is typically used, and in engineering science, the term Reliability Theory is more
prominent. In survival analysis, sometimes individuals are selected and followed prospec-
tively until failure or censoring, but their current lifetime at the point of selection is not
t = 0 (i.e not at birth), but some value t = t0 > 0. It therefore means that the lifetime or
censoring time of the individual, Ti , is greater than t0 (Cleves et al., 2001; Castillo et al.,
2005). As a result the lifetime Ti is considered to be left truncated at t0 where,
(
0 if t0 t;
FT |T >t0 (t) = FT (t)FT (t0 ) (3.24)
1FT (t0 ) if t0 < t.
To determine the probabilities associated with the sojourn time for the interval (1, 2], we
assume that from Equation 3.24 t0 = 1 and 1 < t 2. It therefore means that only sojourn
times greater than t = 1 is being considered at this point and the cumulative distribution
of the sojourn time in the interval can be considered truncated (Castillo et al., 2005). It is
therefore described as:
Hi,j T (t) Hi,j T (1)

Hi,j T |T >1 (t) = , 1<t2 (3.25)
1 Hi,j T (1)
Similarly for interval (m 1, m] the cumulative distribution of the sojourn time in the
interval can be described as:
Hi,j T (t) Hi,j T (m 1)

Hi,j T |T >m1 (t) = , m1<tm (3.26)
1 Hi,j T (m 1)
At t = m the cumulative distribution of the sojourn time becomes
Hi,j T (m) Hi,j T (m 1)

Hi,j T |T >m1 (m) = , m1<tm (3.27)
1 Hi,j T (m 1)
Therefore, the transition probability for interval (m 1, m] is given by:
1 9j=8 p10,j H10,j T |T >m1 (m) p10,9 H10,9T |T >m1 (m) . . . 0

P
0 0 0
..

0 . . . . . . . 0 0 0

..

0 0 . . . . . . . 0 0

m1,m (m) =
..
0 0 0 . ... ... 0
..

0 0 0 0 . . . . . . .

. .
0 0 0 0 0 . . . .
0 0 0 0 0 0 1
(3.28)
29
Equation 3.27 have been previously used to determine transition probabilities for one-
step state-based yearly transitions in deterioration models (Black et al., 2005a,b), where
the transition probability of the embedded Markov chain was assumed to be 1. It therefore
means that the probability obtained from Equation 3.27 can be used to describe the prob-
ability associated with the sojourn time to the end of the period, given that it survived up
to the start of the period.
Sojourn Time of Pavements between Condition States. For the proposed semi-
Markov model, the number of miles for a particular segment was rounded to the nearest
one-tenth of a mile and the sojourn time distribution for each one-tenth unit of a mile of
pavement segment was analyzed. Figure 3.2 gives a schematic of how the pavement segment
is divided into one-tenth of a mile sub-sections. A MATLAB R
program was written to
organize and analyze the data to estimate the parameters of the sojourn time distributions
of pavement segments in each condition state. The following outlines the steps performed
by the MATLAB R
program are outlined as follows:
1. The 790 pavement segments, used as training data, were imported with the following
attributes: Beginning Mile Post, Ending Mile Posts, the Year overlaid, and the CRKs
over twenty (20) years (1986-2005).
2. The length of each segment to the nearest one-tenth of a mile was determined.
3. The yearly decreases in CRKs for each segment subsequent to the year being newly
constructed or overlaid were extracted, such that a series of decreasing CRKs for each
segment could be used to model do-nothing actions on the pavement segments over
time.
4. The pavement segments were assigned to condition states over time, in accordance
with Table 3.1, as outlined earlier in this chapter.
5. At some point in time, all the pavement segments tracked, were either just becoming
new or entered the current condition state from a higher or equal condition state.
If a unit of pavement segment exits the current condition state to a lower condition
state at a known time, then the sojourn time of that unit of pavement segment in the
current condition state is essentially known. However, if a unit of pavement segment
is in a particular condition state and the tracking of the pavement ended because the
condition state either increased or the study was terminated, the sojourn time of
that unit of pavement segment is not precisely known and is therefore right-censored
(Lee, 1992). See Figure 3.3 for an example of the change in the condition states of
pavement segments over time, outlining the complete and censored times spent in
particular condition states.
6. Based on the complete and censored durations obtained from the data, the distribution
of the sojourn time from condition state i to condition state j (Hi,j (t)) was determined,
where j is either i 1 or i 2, i = 10, 9, 8, 7, 6, 5 min(j) = 4. The proportion of the
number of units of assets that left condition state i and went to condition state j,
to the total number of units of assets that left condition state i and went to all
condition states other than itself (pi,j ) was determined, for each condition state i.
30
From Figure 3.3, segment 3 spends 6 years in condition state 10, has a one state
drop and then spends 7 years in condition state 9, and then another one state drop
to condition state 8. Segment 3 then spends 1 year in condition state 8, before
transitioning to condition state 7, where the pavement segment spent at least 6 years
in condition state 7. In Figure 3.3, for segment 4, it can seen that the segment spends
10 years in condition state 10 before having a two states drop to condition state 8.
The sojourn times for segment 4 for the lower condition states can also be inferred
from Figure 3.3.
Figure 3.2: Schematic of Pavement Segment Divisions into Equal Units of One-
Tenth of a Mile
Figure 3.3: An Example of the Change in the Condition States of Pavement Seg-
ments Over Time
Bar charts representing the frequency distributions of sojourn times were produced to
give an indication of the likely probability density functions of the sojourn times in each
condition state before transitioning to another lower condition state. The distribution of
the sojourn times were assumed to follow a Weibull distribution, as have been done in pre-
vious research in which semi-Markov deterioration models were developed (Sobanjo, 2011;
Black et al., 2005a,b). According to Ross (2002), the Weibull Distribution was originally
proposed for the interpretation of fatigue data and is widely used in engineering models
because of its versatility. The maximum likelihood estimate of the scale () and shape
() parameters of the Weibull distribution, used to describe the sojourn time distributions,
31
were computed. Another MATLAB R
program was used to generate a series of transition
probability matrices, based on the Weibull parameters ( and values) obtained for the
sojourn times between condition states. The M LE of the and values serves as inputs
for this program. The transition probabilities according to Equations 3.23 and 3.28 are
then determined, and used to simulate the survival curves and the expected deterioration
over time.
3.2.5 Simulation Approach

Jiang and Sinha (1992) outlined a Monte Carlo simulation approach for predicting the
condition of pavements based on transition probabilities from a Markov chain deterioration
model. A modified approach was done in this study for both the semi-Markov and Markov
chain deterioration models. The Monte Carlo method is one of the most commonly used
simulation techniques in engineering modeling (Jiang and Sinha, 1992).
The MATLAB R
programs formulated were used to do the following simulations using
transition probabilities derived from both the Markov chain and semi-Markov model:
The average condition states over 20 years.
Survival curves of pavement segments remaining above a threshold condition state

were also determined from the simulation results.
A 20-year period simulation was performed using 10,000 trials. The steps for one run of
the simulation are outlined in the flow chart in Figure 3.4. The first step of the simulation
was to establish the yearly transition probability matrices, which were for the Markov
chain model and for the semi-Markov model. For the Markov chain case, the transition
probability matrix were assumed to be independent of the interval, t, and thus remains
the same for each interval. However, for the semi-Markov case, the transition probability
matrix changes for each interval. Condition states i and j ranges from 10 (best condition)
to 4 (worst condition). The next step was to create another matrix that has the cumulative
probabilities of transition from condition state i to condition state j for each i, denoted as
the N (t)-matrix. The ith , j th term (ni,j ) in the Ni,j -matrix is the cumulative probability
from condition state i of the respective i,j terms in the i,j matrix for the Markov model.
Likewise, ni,j in the Ni,j (t)-matrix is the cumulative probability from condition state i of
the respective i,j terms in the matrix for the semi-Markov model. For the Markov chain
case, i,j is shown later in Equation 3.30 and the corresponding N -matrix is shown in
Equation 3.31.
The initial pavement condition state was set at i = 10. A random number, u, was then
generated from a uniform distribution in the interval [0, 1]. If u was less than ni,i , then the
pavement remained in the same condition. If u fell between ni,y and ni,y1 , where condition
state y = 5, . . . , 10, then the pavement transitioned to condition state y 1.
3.2.6 Actual Pavement Deterioration based on Test Data

To reflect the actual behavior of the pavement segments due to do-nothing actions,
the test data was used to do the following: 1) determine the average condition state over
time; 2) generate survival curves for the pavements in the network, which were estimated
32
Yearly transition probability matrices, Pi,j(t), were established,
where Pi,j(t) = i,j (Markov chain model) or i,j(t) (semi Markov model)
i = 10, t = 1
Create N(t) matrix,

i
where ni , y (t ) = p i , j (t )
j= y
i, j, y = 4, , 10 ;
ji;yi
Create u (random prob. between 0 and 1)

y =i
t=t+1
t < 20 no
years
yes
y=y 1
u < ni , y (t ) no no
ni , y (t ) u < ni , y 1 (t )
yes yes
t = t+1
i=y
t = t+1
Condition state i at each interval t

is determined over 20 years.
`
Figure 3.4: Flowchart for a single run of the simulation over 20 years.
33
by computing the proportion of pavements that was greater than or equal to a particular
condition state. The average condition state is synonymous with the weighted average
pavement Crack Index over time (avgCRK). To determine the avgCRK for each year, the
sum-product of CRK and length of pavement segment was computed for each year and
divided by the total miles of the pavement segment for that year as outlined below:
Pn
(li,y CRKi,y )
avgCRKy = i=1 Pn , i = {1, 2, 3, 4, . . . , n} ; y = 1, 2, (3.29)
i=1 li,y
where,
avgCRKy = the weighted average pavement Crack Index in year y.
i = a numerical indicator given to a unique pavement segment in a particular year, for

which the pavement condition due to a do-nothing action subsequent to becoming new
is known.
n = the number of pavement segments in a particular year, for which the pavement
condition due to a do-nothing action subsequent to becoming new is known.
y = the corresponding number of years subsequent to the segment becoming new.
li,y = the length of a pavement segment i in year y.
CRKi,y = the Crack Index of a pavement segment i in year y.
34
3.3 Discussion of Results
Results from both the Markov chain and semi-Marov models were compared against the
actual test data on the CRKs. Seven hundred and ninety (790) pavement segments from
the training data were used to develop both the Markov chain and semi-Markov models.
The terminal state, 4, was chosen due to the relatively small sample size of pavements with
CRKs 3.5 and less, particularly as it relates to the semi-Markov model. This limitation is
explained further in this chapter. For comparative reasons, the same number of condition
states used for the semi-Markov model was used for the Markov chain model.
3.3.1 Transition Probabilities based on the Markov Chain Model

The set of transition probabilities generated from the Markov chain model are repre-
sented as a transition probability matrix, and is shown below in Equation 3.30. To the left
and top of the transition probability matrix in Equation 3.30 are the condition states i and
j respectively, where ij is transition probability matrix used in the Markov chain model.
10 9 8 7 6 5 4
10 0.905 0.072 0.017 0.006 0 0 0

9 0
0.737 0.157 0.090 0.016 0 0
8 0 0 0.660 0.274 0.042 0.014 0.010
ij = (3.30)
7 0
0 0 0.707 0.188 0.086 0.019 .
6 0
0 0 0 0.724 0.112 0.164

5 0 0 0 0 0 0.582 0.418
4 0 0 0 0 0 0 1
The corresponding N-matrix used in the simulation and described in Figure 3.4 is shown in
Equation 3.31.
10 9 8 7 6 5 4
10 0.905 0.977 0.994 1.0 1.0 1.0 1.0

9 0
0.737 0.894 0.984 1.0 1.0 1.0
8 0 0 0.660 0.934 0.976 0.990 1.0
Nij = (3.31)
7 0
0 0 0.707 0.895 0.981 1.0 .

6 0
0 0 0 0.724 0.836 1.0

5 0 0 0 0 0 0.582 1.0
4 0 0 0 0 0 0 1.0
3.3.2 Transition Probabilities based on the Semi-Markov Model

The transition probability of the embedded Markov chain of the semi-Markov process
was estimated for each transition and the results are shown in Table 3.2.
35
Table 3.2: Transition Probabilities of the Embedded Markov Chain of the Semi-
Markov Process
Transitions i to j Transition Probability
10 to 9 0.707
9 to 8 0.752
8 to 7 0.645
7 to 6 0.468
6 to 5 0.214
5 to 4 1.000
10 to 8 0.293
9 to 7 0.248
8 to 6 0.355
7 to 5 0.532
6 to 4 0.786
36
The frequency distribution of the observed sojourn time for each unit mile before tran-
sition was determined. Figures 3.5 to 3.8 show the following frequencies of sojourn times
or durations in condition state (uncensored and right-censored):
Frequency of Sojourn Times in Condition State 10 Before Transitioning to 9 (Fig-

ure 3.5)
Frequency of Sojourn Times in Condition State 10 Before Transitioning to 8 (Fig-

ure 3.6)
Frequency of Sojourn Times in Condition State 9 Before Transitioning to 8 (Figure 3.7)
Frequency of Sojourn Times in Condition State 9 Before Transitioning to 7 (Figure3.8)
Figure 3.5: Frequency of Sojourn Times in Condition State 10 Before Transitioning to 9
Although the unit of pavement analyzed is for every one-tenth of a mile, the unit for the
y-axes in Figures 3.5 to 3.8 is represented in terms of miles of pavement. From the graphs in
Figures 3.5 and 3.6 it can be seen that there are more units of pavement that transitioned
completely from condition state 10 to 9, than from 10 to 8, which is expected. The total
length of pavement that are right-censored in Figures 3.5 and 3.6 are the same, since it is
not known whether each unit of pavement would have transitioned from condition state 10
to 9 or from 10 to 8. Also, the complete (uncensored) durations of units of pavements in
37
38
39
40
condition 9 before transitioning to 8 in Figure 3.7 are generally higher than the complete
(uncensored) durations in condition state 9 before transitioning to 7 in Figure 3.8.
It is also important to note that the sample sizes of the sojourn time for each unit
of pavement segment in the higher condition states are generally more than that in the
lower condition states. It therefore means that for the semi-Markov deterioration model,
there is likely to be an increase in the uncertainty in the predictions for the lower condition
states. A Weibull distribution was assumed for the sojourn times in a condition state before
transitioning to the lower condition state. Tables 3.3 and 3.4 summarize these results.
Table 3.3: Results based on Maximum Likelihood Estimation of the Scale () and
Shape () Parameters for the Holding Time (Weibull) Distributions in Condition
State i
Transition i to j 95% C.I. Limits for 95% C.I. Limits for
Lower Upper Lower Upper
10 to 9 9.432 9.332 9.533 2.128 2.094 2.163
9 to 8 4.887 4.777 4.999 1.579 1.539 1.62
8 to 7 3.496 3.394 3.602 1.345 1.304 1.387
7 to 6 5.039 4.811 5.278 1.257 1.208 1.308
6 to 5 6.304 5.754 6.906 1.523 1.412 1.641
5 to 4 3.164 3.03 3.304 2.062 1.94 2.193
10 to 8 13.126 12.904 13.351 3.182 3.088 3.278
9 to 7 6.103 5.845 6.372 1.249 1.204 1.295
8 to 6 9.672 8.744 10.697 1.465 1.35 1.591
7 to 5 9.103 8.355 9.918 1.236 1.165 1.312
6 to 4 5.417 5.092 5.763 1.693 1.591 1.802
The 95% C.I. limits of the scale and shape parameters are outlined in Table 3.3. Gen-
erally, the ranges of the C.I. limits are not excessive, and shape parameter estimates are
significantly greater than 1. In Table 3.4 the means and standard deviations for each so-
journ time (Weibull) distribution is shown. The standard deviations for the sojourn time in
condition state 8 before transitioning to 6, and that of condition state 7 before transitioning
to 5 seems relatively high in comparison to the others. The high standard deviations are a
function of the available data. It is believed that as more data for each transition becomes
available, the standard deviations associated with those transitions would become less.
A Goodness-of-fit test was done on the distribution of the complete sojourn times using
the Anderson-Darling (AD) test and the results are shown in Table 3.5. The AD test
measures how well the data follow a particular distribution. The hypotheses for the AD
test are:
H0 : The data follow a specified distribution
Ha : The data do not follow a specified distribution

If the p-value for the AD test falls below the chosen significance level (typically 0.05 or
0.10), it can be concluded that the data does not follow the specified distribution. Also, the
41
distribution with the smallest AD statistic has the closest fit to the data. Minitab R
was
used to compute the AD statistic and corresponding p-value to test how good the complete
durations fitted the predetermined Weibull distribution used to describe the sojourn time.
One of the reasons why good fits were generally not obtained is partially due to the fact that
the AD test was done only on the complete durations and the dataset is highly censored,
and the predetermined Weibull distributions were based on both the complete and censored
durations. In addition, the durations are collected in a discrete manner. Although the fitted
Weibull distributions are generally not a good fit statistically, the Weibull distribution still
proves to be a flexible distribution that can be used to describe the sojourn times in each
condition state, in which the deterioration rate is monotonically increasing. Condition state
4 was selected as the terminal or absorption state because the existence of condition states 3
or lower would not produce meaningful results, which is particularly due to the sample size
of pavement segment that have CRK values less than and equal to 3.5. The first five yearly
transition probability matrices for the semi-Markov model are shown in Equations 3.32
to 3.36. For Equation 3.32, 10,10 = 0.991 means that there is a 0.991 probability that
the pavement segment remains in condition state 10. The other i,j terms represent the
probability of transition from i to j in a given year.
Table 3.4: Means and Standard Deviations of Sojourn Times (Weibull)
Transitions i to j Mean (years) Standard Deviation

10 to 9 8.35 4.13
9 to 8 4.39 2.84
8 to 7 3.21 2.41
7 to 6 4.69 3.75
6 to 5 5.68 3.80
5 to 4 2.80 1.43
10 to 8 11.75 4.05
9 to 7 5.69 4.58
8 to 6 8.76 6.08
7 to 5 8.50 6.91
6 to 4 4.84 2.94
42
Table 3.5: Goodness-of-Fit Test on Pavement Sojourn Times (Weibull)
Transition i to j Data size Goodness of Fit on Complete Durations

Complete Censored AD coefficient p - value
10 to 9 17791 10065 5555.223 <0.001
9 to 8 5794 2794 1345.947 <0.001
8 to 7 3939 1523 555.612 <0.001
7 to 6 3712 2331 1380.054 <0.001
6 to 5 1572 1273 1494.283 <0.001
5 to 4 539 990 212.34 <0.001
10 to 8 11738 10065 17880.053 <0.001
9 to 7 4359 2794 1807.013 <0.001
8 to 6 1883 1523 1471.112 <0.001
7 to 5 2946 2331 2210.766 <0.001
6 to 4 1693 1273 1359.002 <0.001
43
Year 1
10 9 8 7 6 5 4
10 0.991 0.008 0.001 0 0 0 0

9 0
0.822 0.078 0.100 0 0 0
8 0 0 0.795 0.170 0.035 0 0
ij (1) = (3.32)
7 0
0 0 0.814 0.123 0.063 0 .
6 0
0 0 0 0.886 0.059 0.056

5 0 0 0 0 0 0.911 0.089
4 0 0 0 0 0 0 1
Year 2
10 9 8 7 6 5 4
10 0.970 0.028 0.002 0 0 0 0

9 0
0.716 0.150 0.134 0 0 0
8 0 0 0.690 0.249 0.061 0 0
ij (2) = (3.33)
7 0
0 0 0.749 0.166 0.085 0 .
6 0
0 0 0 0.773 0.107 0.120

5 0 0 0 0 0 0.744 0.256
4 0 0 0 0 0 0 1
Year 3
10 9 8 7 6 5 4
10 0.944 0.049 0.007 0 0 0 0

9 0
0.652 0.197 0.151 0 0 0
8 0 0 0.633 0.290 0.077 0 0
ij (3) = (3.34)
7 0
0 0 0.717 0.188 0.095 0 .

6 0
0 0 0 0.695 0.138 0.167

5 0 0 0 0 0 0.602 0.398
4 0 0 0 0 0 0 1
Year 4
10 9 8 7 6 5 4
10 0.915 0.071 0.014 0 0 0 0

9 0
0.603 0.234 0.163 0 0 0
8 0 0 0.591 0.319 0.090 0 0
ij (4) = (3.35)
7 0
0 0 0.694 0.203 0.103 0 .
6 0
0 0 0 0.631 0.163 0.206

5 0 0 0 0 0 0.484 0.516
4 0 0 0 0 0 0 1
44
Year 5
10 9 8 7 6 5 4
10 0.883 0.093 0.024 0 0 0 0

9 0
0.562 0.265 0.173 0 0 0
8 0 0 0.557 0.343 0.100 0 0
ij (5) = (3.36)
7 0
0 0 0.676 0.215 0.109 0 .
6 0
0 0 0 0.577 0.183 0.240

5 0 0 0 0 0 0.388 0.612
4 0 0 0 0 0 0 1
3.3.3 Comparison of the Markov Chain Model, Semi-Markov Model

and Actual Pavement Deterioration Behavior
Using the transition probabilities generated from both the Markov chain and semi-
Markov model, Monte Carlo simulations were done for 10,000 trials. The overall expected
(average) condition state over time was determined from the Markov chain and semi-Markov
models. Both models were compared against the actual condition states over 20 years. The
condition criteria as outlined in Table 3.1 was also used to determined the actual condition
states based on the avgCRK values. The avgCRK for each year was computed by finding
the sum-product of the actual CRK values and length of pavement segment for each year
and dividing by the total miles of the pavement segment for that year, as outlined previously
in Equation 3.29. Figure 3.9 shows the comparison of the actual condition states over time
with the expected condition states from the Markov chain and the semi-Markov models. To
demonstrate statistically how close the prediction models are to the actual behavior of the
flexible pavements and if the models are good fits, the Pearsons Chi-Square test statistic
for each model is determined. The Chi-Square test statistic is computed by the following
formula (Montgomery and Runger, 2006):
k
X (Oi Ei )2
20 = (3.37)
Ei
i=1
where,
Oi = the observed value or the actual value describing the pavement behavior
Ei = the expected value or the predicted value

There are k 1 degrees of freedom, where k = 20. To determine if the models are a
good fit consider the following hypotheses:
H0 : The predicted values reflect the actual pavement behavior.
H1 : The predicted values do not reflect the actual pavement behavior.

where = 0.05 Therefore H0 is rejected if 20 > 20.05,19 = 30.14. The 20 values of the
results from the respective models in Figure 3.9 are as follows:
20 (semi-Markov model) = 0.633
45
20 (Markov chain model) = 0.191
Therefore, both models are good fits based on the Chi-Square test.
Figure 3.9: Monte Carlo Simulation Results: Comparison of the Markov chain,
Semi-Markov and Actual
Based on a visual inspection of the curves in Figure 3.9 it is clear that the semi-Markov
model closely fits the actual condition state curve generated from the test data for the first
five (5) years. This indicates that the semi-Markov model appears to be a better model than
the Markov chain model for predicting the onset of deterioration in this case, particularly up
to the end of year 5. Figures 3.10 and 3.11 shows the 95% C.I. intervals for the simulated
values from the respective models. Figure 3.10 indicates that the Markov chain model
slightly under predicts the actual pavement behavior at years 2 to 6, as the actual condition
states falls above the upper 95% C.I. interval bound. Although the semi-Markov model
slightly over predicts the actual pavement behavior at years 7 to 11, it provides good
predictions for the first five (5) years, as the actual condition states falls within the 95%
C.I. interval bounds for the first five years (See Figure 3.11). The limitations of the semi-
Markov model, particularly beyond year 5 could be partially due to the sample size of the
available data used to develop the model. However, the accuracy of the predictions in the
earlier years is more critical than the later years, where higher levels of uncertainty are
typically expected.
In addition to estimating the expected condition states for the pavement segments,
survivor curves for pavements in the best condition (condition state 10) and pavements in
condition states 9 and greater were also produced and are shown in Figures 3.12 and 3.13
respectively (Wang et al., 1994). These curves show the expected proportion of pavements
above a predetermined threshold value, based on the actual pavement behavior of the test
46
Figure 3.10: Monte Carlo Simulation Results: 95% C.I. Bands on the Markov
Chain Model and Actual Condition
Figure 3.11: Monte Carlo Simulation Results: 95% C.I. Bands on the semi-
Markov Model and Actual Condition
47
data and that due to the Markov chain and semi-Markov prediction models respectively.
The bar chart represents the sample size (in miles) from the test data that was used to
generate the actual behavior curve (Figures 3.12 and 3.13).
Figure 3.12: Survivor Curve: Proportions of pavements in best condition state (state 10)
For the proportions of pavements in best condition state (state 10), the test statistic,
20 , based on the results of the respective models in Figure 3.12 are shown below:
From the observations, the curve generated from the semi-Markov model closely matches
the curve obtained from the test data. However, both models have a good fit, since the
20 values for each model is less than 30.14.
For the proportions of pavements in condition state 9 and higher, the survival curve
generated from the semi-Markov model also fits the curve representing the test data better
than that produced from the Markov chain model (See Figure 3.13).
The semi-Markov model closely matches the actual behavior in the earlier years, particularly
up to year 6. Again, both models are good fits, having 20 values less than 30.14. It can be
seen from both Figures 3.12 and 3.13 that the sample size reduces over time in the test
data. A similar reduction in sample size of the training data results in an increase in the
uncertainties of the expected condition states generated from the prediction models.
48
Figure 3.13: Proportions of pavements in condition state 9 and higher
3.4 Summary of Results

This chapter has outlined a feasible approach in which historical pavement condition
was used to develop a semi-Markov deterioration model that is applicable in modeling
network level performance. This methodology can be used on any group of pavements
segments based on their engineering properties and or external factors such as environment
conditions and traffic loading. Monte Carlo simulations were used to generate deterioration
curves based on the Markov chain and semi-Markov deterioration models. A comparison
of results produced from the traditional Markov chain model, the approach based on the
Semi-Markov process, and the actual pavement condition behavior for the deterioration of
flexible pavements were outlined.
Both the Markov chain and semi-Markov models were good fits for actual pavement
deterioration behavior based on the Chi-Square Test. However, the Markov chain model
slightly under predicted the actual pavement behavior at years 2 to 6, while the semi-
Markov model slightly over predicted the actual pavement behavior at years 7 to 11 at
a significance level of = 0.05. The semi-Markov model appeared to be superior to the
Markov chain model counterpart for predicting the onset of deterioration. It is expected
that as the sample size of the data used to develop the model decreases over time, there is
a higher degree of uncertainty in the predictions. Therefore, the fact that the semi-Markov
slightly over predicted in years 7 to 11 does not necessarily reflect the inadequacy of the
model, but could have been a function of the size of the data for those years.
It can be concluded, that if sufficient data is available, the use of semi-Markov process
may be an improved alternative to the traditional Markov chain process in modeling the
deterioration of flexible pavements. The deterioration model only forms a part of a P M S,
and so the future work involves integrating the semi-Markov deterioration model with a
cost-effectiveness model for maintenance, repair and rehabilitation for various alternatives
to obtain a P M S that can be applicable for use in the industry.
49
CHAPTER 4
SEMI-MARKOV APPROACH FOR MODELING

THE DETERIORATION OF BRIDGE
ELEMENTS
4.1 Introduction
The purpose of this chapter is to investigate the semi-Markov methodology outlined
by Black et al. (2005a,b) herein after referred to as Blacks Approach, for modeling the
deterioration of CoRe Bridge Elements data. Blacks Approach is also compared with the
Markov chain model. Pairs of historical condition data reflecting deterioration are used
to develop the Markov chain model by using a regression method. There are a number
of challenges when using historical condition data to model the deterioration of bridge
elements. Firstly, the size of the required data must be adequate to accomplish the main
objective. The F DOT bridge element inspection data used in this study spans twelve
(12) years (1997-2008 inclusive). The first step is to extract data reflecting deterioration
and or do-nothing actions. Do-nothing actions mean that no improvement was done to
the bridge element, and therefore allowed to deteriorate. However, majority of the bridge
elements in the inventory were not allowed to deteriorate with no improvement actions.
This research contributes significantly to the development of new deterioration models
for the CoRe Bridge Elements data. There has not been a significant level of research done
on the deterioration of bridges using the CoRe Bridge Elements data. This is partially due
to the fact that the use of the CoRe Bridge Elements data is more recent than the N BI, and
only a few states in Unites States of America (USA) have a fair size dataset to undertake
a meaningful study. Furthermore, it is easier to determine the time that bridge elements
spend in condition states when the bridge elements are based on the N BI data format. For
the N BI data, each bridge element can only be in one condition state at a time, and hence
the time a bridge element spends in a particular condition state can be easily determined.
The challenge with the CoRe Bridge Elements data is that each bridge element can have
proportions in different condition states, for example, the condition record for Pourable
Joint Seal (element 301), which has three (3) condition states, may show that 47.1% of
the total quantity is in condition state 1, 8.8% of the total quantity is in condition state
2, and the remaining 44.1% is in condition state 3 as outlined in Table 4.1. In Table 4.1,
BRKEY is Bridge Key (an identification number assigned to each bridge), ELEM KEY is
50
Element Key (a number assigned to each bridge element), QU AN T IT Y is Total Quantity,
QT Y ST AT E# is Quantity in Condition State # (feet) and P CT ST AT E# is Percentage
in Condition State#. The overall condition of a CoRe bridge element is therefore based
on the quantity of that element in each condition state (Thompson and Shepard, 2000;
Shepard and Johnson, 2001). Being able to describe the condition level in proportions may
allow for better estimates to be made as it relates to maintenance and replacement costs
of the bridge elements; however, this can pose a challenge when using the data to develop
deterioration models. An approach is suggested in this chapter to estimate the expected
condition state.
Table 4.1: Example of a condition record for Pourable Joint Seal (element 301)
BRKEY ELEMKEY QUANTITY PCTSTATE1 QTYSTATE1 PCTSTATE2 QTYSTATE2 PCTSTATE3 QTYSTATE3
750207 301 68 47.1 32 8.8 6 44.1 30
Condition data with the format as outlined in Table 4.1 have been used to develop
Markov chain deterioration models. The Regression Method, using matrices computations,
in which the change in the pairs of succeeding inspection records are observed, was used
to develop the deterioration model for AASHT O CoRe Bridge Elements in California
(Thompson and Johnson, 2005). Thompson and Johnson (2005) considered a number of
bridge elements and found out that in the case of bridge elements once a bridge deck
reaches condition state 3 it receives a highly effective rehabilitation action, thus minimizing
the number of elements that transitions to condition state 4. It should be noted that bridge
decks have five (5) condition states. Also, in California maintenance actions are condition
response. As a result, the sample size of the data of bridge elements in the lower condition
states will be much lesser than in the higher condition states. This is also expected for the
dataset used in this study, which are of bridges in Florida. The method used by Thompson
and Johnson (2005), applied the Markov chain and was not based on observing the sojourn
time of bridge elements in particular condition states. The semi-Markov model outlined in
this dissertation takes into consideration the estimated distribution of the sojourn time of
bridge elements in particular condition states, which is based on a set of computations and
assumptions explained later in this chapter.
As mentioned earlier, the condition of AASHT O CoRe Bridge Elements are recorded
in such a way that the proportions of bridge elements in each condition state are recorded
in a dataset. For this reason, Blacks Approach to modeling the deterioration of the CoRe
Bridge Elements is appropriate. Essentially, Blacks Approach allows for historical data
to be used to develop a semi-Markov process to predict deterioration of bridge elements
subsequent to the element becoming new. Here, becoming new means that the bridge
element is either new or just undergone major rehabilitation.
As was done in Black et al. (2005a,b), a Weibull distribution is assumed for the time
spent in a particular condition state prior to transition to the next state, defined as the
sojourn time. Sobanjo (2011) also assumed Weibull distribution for the distribution of
sojourn times in a semi-Markov model developed for the condition of bridge components
based on the National Bridge Inventory (NBI) data. For Blacks Approach, the parameters
of the Weibull distribution are determined, using a least squares optimization technique.
For this technique, initial values were selected for the scale and shape parameters of the
51
Weibull distribution. An iterative process was then utilized to optimize these parameters
by minimizing the difference between the actual number of assets in each state and the
estimated number of assets. Black et al. (2005a,b) tried different initial values for the
Weibull parameters, which gave different end results, which indicates that the Weibull
parameters obtained were local optimal values.
4.2 Methodology based on Black et al. (2005a,b)

The data used in Black et al. (2005a,b) is a snap shot of the number of transformers in
particular states based on their age or years since last refurbishment, and not the tracking
of conditions on particular transformers. This data is used to determine the number of
transformers in state i in a given year after refurbishment, t, which is assumed to be
equal to the total probability of the transformer being in state i after the tth year since
last refurbishment. A similar approach is done for this study, but with some modifications.
Bridge elements are typically inspected every two (2) years. This means that if a snap shot
of the number of bridge elements were to be done, as in the case of Black et al. (2005a,b),
then only two (2) years of data would be required. However, to maximize the use of the
available data, a snap shot of the number of bridge elements in each condition state over
twelve (12) years were done. As a result, the bridge elements were tracked over the twelve
(12) years of available data, and the time since the bridge elements became new were
estimated based on the element inspection dates and condition levels recorded. Therefore,
new is defined as the time (or the year) at which 100% of that bridge element became in
condition state 1, for which the previous inspection reflected it having less than 100% in
condition state 1 or if the bridge is in fact newly constructed. A bridge is determined to be
newly constructed if the year built for the bridge coincides with the most recent element
inspection date, and if this is the case, all the bridge elements associated with that bridge
is assumed to be new.
For all practical purposes, the bridge element may be assigned to the bridge on a unit
basis, e.g. a bridge has one deck instead of 5,000 square feet of deck. However, since
the CoRe Bridge Element data does not always have 100% of the bridge element in a
particular condition state, the proportion in the condition state was used to estimate the
overall number of bridge elements in the network that falls into a particular condition state.
Therefore, the proportion of bridge element that can be in a particular condition state per
bridge in any given year is from 0 to 1. Furthermore, since the time from the bridge element
became new is determined for each bridge, there is the possibility that a particular bridge
element on a bridge may be noted as becoming new more than once within the twelve
(12) years period (1997-2008 inclusive). It is assumed that each event of a bridge element
becoming new is independent of each other.
For this chapter a snap shot of the condition over the twelve (12) years was done.
However, if a snap shot of the inspection records are done, there is a strong tendency that
the condition data obtained from two consecutive years will be on separate sets of bridges.
This is so because each bridge inspection is typically done on a two (2) years cycle. To
overcome this problem, it is proposed that the missing data for a given year be accounted
for. One way of accounting for the missing data was to first determine if an inspection
52
was conducted on the same bridge element at least two (2) years apart. If this was true,
then there was a strong possibility that the condition of that element in the interim year
(in which no inspection was done) was between the condition level in the prior year and
that of the subsequent year. In general, it is typically not assumed that the deterioration of
bridge elements is linear, but within an increment of two years, and within the context of
developing a network deterioration model, it is reasonable to assume linear deterioration in
this case. Linear interpolation for estimating conditions midway between inspection dates
for transportation assets have been done previously by Mishalani and Madanat (2002).
However, if the interval between inspections exceeded two (2) years, the missing data was
treated as missing, and the tracking of that bridge element stopped at the element inspection
date prior to which there was two (2) or more consecutive years of no inspection records.
In this paper, Bare Concrete Deck (element 12), Pourable Joint Seal (element 301) and
Reinforced Concrete Abutment (element 215) were used as examples for the application of
Blacks Approach.
4.2.1 Filtering Process

The first step is to filter the data and organize it into a usable format. The Pontis R
element data essentially has records of the data with attributes such as BRIDGE ID or
BRKEY (Bridge Key), ELEM KEY (Element Key), ST RU N IT KEY (Structural Unit
Key), ELINSPDATE (Element Inspection Date), P CT ST AT E# (Percentage in State#),
QT Y ST AT E# (Quantity in State#) amongst other attributes. The records are sorted
by ELIN SP DAT E, ST RU N IT KEY , and BRIDGE ID for each bridge element. This
allows for the observation of the change in condition of the bridge element over time. For this
model, major rehabilitation is assumed to have taken place when the P CT ST AT E1 value
for a bridge element was increased from a lower value to 100% or the Element Condition
Index was increased to 100%. The formula for Element Condition Index was derived from
the California Bridge Health Index formula (Thompson 2000) and is shown in Equation 4.5.
The formula below is used to compute bridge Health Index (HI) (Roberts and Shephard,
2000): P
CEV
HI = P 100 (4.1)
T EV
where, T EV is the total element value and CEV is the current element value.
T EV = T otal element quantity f ailure cost of element(F C) (4.2)
X
CEV = (Quantity in Condition State i W Fi ) F C (4.3)
The condition state weighting factor (W F ) is given mathematically as:

1
W F = 1 (State no. 1) (4.4)
State Count 1
To determine the condition index of an element, consider Equation 4.1 with only one bridge
element. The failure cost of element (F C) terms cancels out, resulting in a simplified
53
formula:
P
(Quantity in Condition State i W Fi )
Element condition index = (4.5)
T otal element quantity
StateCount is the number of possible condition states for that bridge element (i.e. 3, 4 or
5).
When it was observed that the Element Condition Index had increased to 100%, the
conditions of the bridge element for the subsequent years were tracked until there were
either no more observations or until Element Condition Index increased to 100% again,
which meant that the bridge element had undergone a subsequent rehabilitation. The
number of years since the element became new was determined based on the subsequent
element inspection dates. For inspections that are two (2) years apart, the proportion
of bridge elements in each state for the intermediate year are estimated by averaging the
proportions in each state from the inspection records for prior and subsequent years. As the
time since becoming new increased the number of usable data tended to decrease. This is
because bridge elements are generally not allowed to deteriorate to an extreme condition,
and more so, not to deteriorate to the failure state. This poses a problem as it relates
to obtaining data to model the conditions of bridge elements that are in lower conditions
(such as, condition states 4, 5 and the failure state for 5-state bridge elements). It should be
noted that for the AASHT O CoRe Bridge Elements the best condition is condition state 1
and the condition worsens as the state number increases. The condition state prior to the
failure state for bridge elements in the CoRe Bridge Element database is either 3, 4, or 5,
depending on the specified bridge element. Yearly transition probabilities are determined
based on Blacks Approach.
4.2.2 Model Implementation

The implementation of the semi-Markov approach is based on the modifications as
outlined in Section 2.5 (Black et al., 2005a,b). The probability that specifies the semi-
Markov model is pi,j (t), which is the probability of changing from state i to j in interval t
given that the item was in state i after interval t 1. j = i or i + 1. Therefore,
pi,j (t) = pi,i+1 (t) = 1 pi,i (t) (4.6)
The probability of changing from state i to i + 1 in interval t given that the item was in
state i after interval t 1, pi,i+1 (t), is given by the following formula.
pi,i+1 (t) Fi (m) Fi (m 1)

pi,i+1 (t) = = (4.7)
pi (t 1) 1 Fi (m 1)
where,
pi,i+1 (t) is the probability of changing from state i to i + 1 at interval t since becoming
new, that is unconditional on the current condition state.
Fi (m) is the cumulative distribution function for the sojourn time in condition state i.
54
Figure 4.1 is a graphical representation of the possible paths between condition states
for a 5-condition state element. Therefore, based on Figure 4.1 and Equation 2.11, where
Ci,i+1 (m) = pi,i+1 (m), we see that:
p2,3 (3) = pE E
2 (2, 0) C2,3 (1) + p2 (2, 1) C2,3 (2) (4.8)
pE
2 (2, 0) in Figure 4.2 (cloud 1) is the total probability for which the last leg is represented
by the bold arrow. For this example it has only one path. Similarly, pE 2 (2, 1), in cloud 2, is
the total probability for which the last leg is represented by the bold arrow, which is also
one path in this case. C2,3 (1) and C2,3 (2) are both denoted by bold dashed arrows. Based
on Equation 2.12,
pE
3 (5, 2) = p2,3 (3) (1 C3,4 (1)) (1 C3,4 (2)) (4.9)
In Figure 4.3, p2,3 (3) is the total probability represented by the cloud (which is the sum-
mation of the two paths). The probabilities 1 C3,4 (1) and 1 C3,4 (2) are represented by
the bold arrows. Based on Equations 2.11 to 2.13, and using initial values of 1 for both
the scale () and shape () parameters of the Weibull distribution, the initial estimated
distribution was determined.
After the parameters of the Weibull distribution for time spent in condition state i is
estimated, the pi,j (t) is calculated. The number of years since becoming new is known
rather than the number of years since entering a condition state. Therefore, the underlying
probability for the observation is qi (t), the probability of being in state i after the tth interval
since last replacement. qi (t) can be expressed in terms of pi,j (t) and values of the Weibull
parameters can be selected so as obtain values of qi (t) that are as closely as possible to the
actual overall proportion of bridge element in state i after the tth interval since becoming
new by finding the sum of least squares between the actual and estimated number of
bridge elements in each condition state.
4.2.3 Markov Chain Model based on Regression Method

To apply the Markov Chain model, the regression method based on matrices calculations
was utilized, using pairs of condition data that was recorded two (2) years apart. The
improvement actions done to the bridge elements were filtered out. Pairs of condition
records for only do-nothing action were observed. Consider Equation 4.10.

x1 . . . x F p1,1 (ad ) . . . p1,F (ad ) y1 . . . yF
.. . .. .. .. .
. . . . .. . ... . = . . . . .. (4.10)
x1 . . . xF pF,1 (ad ) . . . pF,F (ad ) y1 . . . yF
in which xi is the proportion of element in condition state i before do-nothing action,
ad ; yi is the proportion of element in condition state i after do-nothing action, ad , for
i = 1, 2, N S, F . N S represents the number of condition states except the failure state,
which is denoted as F . Each row of the respective matrices with the xi and yi values
represents a different inspection record of a particular bridge element. If xi and yi are
known for a sample of the same bridge element, then the transition probability matrix for
55
Figure 4.1: Graphical representation of the possible paths between condition
states subsequent to becoming new.
Figure 4.2: Graphical representation for p2,3 (3).
56
Figure 4.3: Graphical representation for determining pE
3 (5, 2).
action ad can be determined by the regression method using matrices computations, based
on the Equation 4.11.
1 T
Pi,j (ad ) = X T X

X Y (4.11)
The transition probabilities obtained, in the form of a transition probability matrix,

represent the transition over a two year period, and hence the square root of that transition
probability matrix is needed to estimate the set of transition probabilities for a one year
period. Mathematical programs, such as MATLB, can be used to determine the square
root of some matrices. Sobanjo and Thompson (2011) suggested a method to determine the
yearly transition probability matrix. After the regression method was done, the transition
matrix representing the two-year transition was normalized such that values left of the
diagonal were set to zero, any diagonal value less than 0.01 are set to 0.01, and each row is
adjusted such that it sums to 1 (Sobanjo and Thompson, 2011). After the square root of
the matrix was determined, it was normalized again. The resulting transition probabilities
were used to develop the Markov chain deterioration model.
The deterioration models were applied to Bare Concrete Deck (element 12), Pourable
Joint Seal Joint (element 301) and Reinforced Concrete Abutment (element 215). For
the semi-Markov model (Blacks Approach), the sum of least squares between the actual
number of bridge elements and the estimated number was done for each condition state,
to obtain optimal values for the scale and shape parameters of the Weibull distribution used
to describe the sojourn times for each condition state for each bridge element.
57
4.3.1 Bare Concrete Deck
The values of the scale and shape parameters of the Weibull distribution were each
initialized to 1. For the Bare Concrete Deck, local optimal values for the scale and shape
parameters were determined for condition states 1 and 2, in which the estimated parameters
were in the prescribed limits of [0.01, 1000]. For the first run in condition state 3, the
estimates for the scale and shape parameters were 0.239 and 0.728 respectively. When the
shape parameter is less than 1, it means that the rate of deterioration decreases with time,
which is not logical for the deterioration of transportation assets. It is generally expected
that deterioration should increase with time. As a result, a minimum value of 1 for the
shape parameter was imposed for this scenario, and the analysis done a second time. For the
second run for condition state 3, both the scale and shape parameters remained unchanged,
when the initial values were 1 respectively. The initial value for the shape parameter was
changed to 1.5, and the analysis done for a third time. The result of the third run, although
still not completely logical, was used in the model. A similar procedure was followed
for condition states 4 and 5, which gave similar results as obtained for condition state 3.
Table 4.2 gives the summary of the results of the Weibull distribution used to describe
the sojourn time in each condition state. In Table 4.2 it can be seen that the resulting
distribution of the sojourn time in condition state 1 has a relatively high standard deviation,
which is a function of the data and the assumptions that governs Blacks Approach.
Table 4.2: Estimated Scale and Shape Parameters, Mean and Standard Deviation
for the Sojourn Times in Each Condition States for Bare Concrete Deck
Condition States Scale Shape Mean Std. Dev.

1 8.86 1.07 8.62 8.02
2 9.31 2.69 8.28 3.32 a
3* 0.4 1 0.4 0.4
4* 0.4 1 0.4 0.4
5* 0.4 1 0.4 0.4
a
* an additional constraint was added for the shape parameter to be greater than or equal to 1. The initial
values for the scale and shape parameters were 1 and 1.5 respectively.
The results of the transition probabilities based on the sojourn time distribution for
condition state 1 are shown in Table 4.3. Figures 4.4, 4.5 and 4.6 each show the comparison
of the actual number of bridge elements with the estimated number of bridge elements in
condition states 1, 2, and 3 respectively, for Bare Concrete Deck. The differences between
the actual and estimated values are reasonably close for condition states 1 and 2. For
condition state 3, the differences between the actual and estimated number of Bare Concrete
Deck are not very close for durations up to 6 years.
58
Table 4.3: Computed Condition State 1 probabilities for Bare Concrete Deck
a F1 (a) p1,2 (a) p1,1 (a) q1 (a) p1,2 (a) pE

1 (a, a) pE
1 (a, z) z 6= a
1 0.091 0.091 0.909 0.909 0.091 0.909 0
2 0.183 0.101 0.899 0.817 0.091 0.817 0
3 0.268 0.104 0.896 0.732 0.085 0.732 0
4 0.346 0.107 0.893 0.654 0.078 0.654 0
5 0.418 0.109 0.891 0.582 0.071 0.582 0
6 0.482 0.11 0.89 0.518 0.064 0.518 0
7 0.54 0.112 0.888 0.46 0.058 0.46 0
8 0.592 0.113 0.887 0.408 0.052 0.408 0
9 0.638 0.114 0.886 0.362 0.046 0.362 0
10 0.68 0.115 0.885 0.32 0.042 0.32 0
11 0.717 0.116 0.884 0.283 0.037 0.283 0
12 0.75 0.116 0.884 0.25 0.033 0.25 0
59
Figure 4.4: Comparison of the actual number with the estimated number of Bare
Concrete Deck for Condition State 1.
Figure 4.7 show the probability density functions (pdfs) of the sojourn times for all the
condition states, superimposed on the same graph. All the pdfs, except that for condition
state 2 are right-skewed. Figure 4.8 shows the cumulative distribution functions (cdfs) for
all condition states, superimposed on the same graph. It should be noted that this method
does not take into consideration right-censoring of durations in condition state. Since this
AASHT O CoRe Bridge Element dataset tends to be highly right-censored, particularly as
the condition deteriorates, some of the results may not accurately represent the deterioration
behavior of the bridge element.
Figure 4.9 gives the proportion in each condition state over time, and it can be seen
that the proportions in condition states 3, 4 and 5 are relatively small, which is based on
the available data on Bare Concrete Deck for those condition states. There is a noticeable
transition of the Bare Concrete Deck from condition state 1 to state 2, reflected in the
change in the proportion of the bridge element in the respective condition states over time.
60
61
62
Figure 4.7: Probability Density Function (Weibull) of sojourn times for Bare Con-
crete Deck for all Condition States.
Figure 4.8: Cumulative Distribution (Weibull) Function of sojourn times for Bare
Concrete Deck for all Condition States.
63
The transition from condition state 2 to state 3 is not noticeable, and it is felt that the
time spent in condition state 2 for a large number of Bare Concrete Decks might have been
right-censored. Right-censoring occurs when the actual ending time of an observation is not
precisely known, because the termination of follow-up occurs before the event would have
occurred. In this case the event is the transition from condition state 2 to 3. Figure 4.10
gives an indication of the sample size or number of Bare Concrete Decks over time in each
condition state, which also reinforces this point that the sample of Bare Concrete Deck in
condition states 3, 4 and 5 is relatively small.
Figure 4.9: The Proportion of Bare concrete Deck in Each Condition State.
64
Figure 4.10: The Number of Bare concrete Deck in Each Condition State.
65
Transition Probabilities for Bare Concrete Deck (element 12) based on Blacks
Approach. The yearly overall transition probabilities based on Blacks Approach were
computed for all the possible condition states for Bare Concrete Deck (element 12). The
general form of the equation is as follows in Equation 4.12, where N S is the number of
condition states, F is the failure state and m represent the interval or year. Equations 4.13
to 4.17 show the the overall yearly transition probabilities in matrices format. For Equa-
tion 4.13, the 1,1 term means that the probability of remaining in condition state 1 is 0.909
and the probability of transitioning to condition state 2, 1,2 , is 0.091. The i,j terms below
are equal to pi,i+1 (m) term in Equation 2.14.

1,1 (m) 1,2 (m) 0 0 ... 0
0 2,2 (m) 2,3 (m) 0 ... 0
.. ..

.. ..
. .

(m) =
. . . . . 0

0 0 0 N S1,N S1 (m) N S1,N S (m) 0

0 0 0 0 N S,N S (m) N S,F (m)
0 0 0 0 0 1
(4.12)
Year 1

0.909 0.091 0 0 0 0
0 0.998 0.002 0 0 0

0 0 0.084 0.916 0 0
i,j (1) =
0
(4.13)
0 0 0.084 0.916 0
0 0 0 0 0.084 0.916
0 0 0 0 0 1
Year 2

0.899 0.101 0 0 0 0
0 0.987 0.013 0 0 0

0 0 0.084 0.916 0 0
i,j (2) =
(4.14)
0 0 0 0.084 0.916 0
0 0 0 0 0.084 0.916
0 0 0 0 0 1
Year 3

0.896 0.104 0 0 0 0
0 0.969 0.031 0 0 0

0 0 0.084 0.916 0 0
i,j (3) =
(4.15)
0 0 0 0.084 0.916 0
0 0 0 0 0.084 0.916
0 0 0 0 0 1
66
Year 4
0.893 0.107 0 0 0 0
0 0.946 0.054 0 0 0

0 0 0.084 0.916 0 0
i,j (4) =
(4.16)
0 0 0 0.084 0.916 0
0 0 0 0 0.084 0.916
0 0 0 0 0 1
Year 5
0.891 0.109 0 0 0 0
0 0.919 0.081 0 0 0

0 0 0.084 0.916 0 0
i,j (5) =
(4.17)
0 0 0 0.084 0.916 0
0 0 0 0 0.084 0.916
0 0 0 0 0 1
Transition Probabilities for Bare Concrete Deck (element 12) based on Markov
Chain. 3567 pairs of condition data were used in the Regression Method to obtain the
transition matrix for do-nothing action based on the Markov chain model for Reinforced
Concrete Abutment. The overall deterioration rate produced from the Markov chain model
appears to relatively slow. However, this is as a result of the lack of data that is available for
Bare Concrete Deck at low condition levels. Based on the results of the Regression model
shown in Equation 4.18, it can be seen that condition state 4 is functioning as an absorption
state. This is due to the fact that there is not adequate data at the lower condition levels
to inform the Markov chain model of the deterioration behavior beyond condition state 3.
Each Year
0.882 0.118 0 0 0 0
0 0.992 0.008 0 0 0

0 0 0.996 0.004 0 0
i,j =
(4.18)
0 0 0 1 0 0
0 0 0 0 1 0
0 0 0 0 0 1
The following graph in Figure 4.11 shows the expected Element Condition Index over
time for Bare Concrete Deck (element 12) based on Blacks Approach and the Markov chain
method.
67
Figure 4.11: Expected Element Condition Index for Bare Concrete Deck over time
based on the Blacks (Semi-Markov) Approach and Markov Chain.
68
4.3.2 Pourable Joint Seal
For Pourable Joint Seal, local optimal values were determined for condition states 1,
as the scale and shape values converged within the prescribed limits of [0.01, 1000], using
initial values of 1 for the scale and shape parameters respectively. For condition state 2,
the scale and shape values after the initial run were 6.065 and 0.615 respectively. Since the
shape value resulted in a value less than 1, an additional constraint was added to ensure
that the shape parameter is either greater than or equal to 1 for this scenario. This will
ensure that the resulting deterioration model will have a deterioration rate that is either
constant or increases with time. The results of the second run were used, which is given
in Table 4.4, which gives the summary of the results of the Weibull distribution used to
describe the sojourn time in each condition state.
for the Sojourn Times in Each Condition States for Pourable Joint Seal

1 11.13 1.16 10.55 9.09
2 4.54 1 4.54 4.54
3 4.63 2.03 4.49 2.12
The results of the transition probabilities based on the sojourn time distribution for
condition state 1 are shown in Table 4.5. Figures 4.12, 4.13 and 4.14 each show the
comparison of the actual number of bridge elements with the estimated number of bridge
elements in condition states 1, 2, and 3 respectively, for Pourable Joint Seal. The differences
between the actual and estimated values are reasonably close for condition states 1 and 2.
For condition states 3, there are some differences between the actual and estimated number
of Pourable Joint Seals that have the durations of 1 to 5 years.
Table 4.5: Computed Condition State 1 probabilities for Pourable Joint Seal
a F1 (a) p1,2 (a) p1,1 (a) q1 (a) p1,2 (a) pE

1 (a, a) pE
1 (a, z) z 6= a
1 0.059 0.059 0.941 0.941 0.059 0.941 0
2 0.127 0.072 0.928 0.873 0.068 0.873 0
3 0.195 0.079 0.921 0.805 0.069 0.805 0
4 0.262 0.083 0.917 0.738 0.067 0.738 0
5 0.326 0.086 0.914 0.674 0.064 0.674 0
6 0.386 0.089 0.911 0.614 0.06 0.614 0
7 0.442 0.091 0.909 0.558 0.056 0.558 0
8 0.494 0.093 0.907 0.506 0.052 0.506 0
9 0.542 0.095 0.905 0.458 0.048 0.458 0
10 0.587 0.097 0.903 0.413 0.044 0.413 0
11 0.627 0.098 0.902 0.373 0.041 0.373 0
12 0.664 0.1 0.9 0.336 0.037 0.336 0
69
Figure 4.12: Comparison of the actual number with the estimated number of
Pourable Joint Seal for Condition State 1.
70
71
72
Figure 4.15 show the probability density functions (pdfs) for all the condition states,
superimposed on the same graph. Figure 4.16 shows the cumulative distribution functions
(cdfs) for all condition states, superimposed on the same graph.
Figure 4.15: Probability Density Function (Weibull) of sojourn times for Pourable
Joint Seal for all Condition States.
Figure 4.17 gives the proportion in each condition state over time, and it can be seen
that as the proportions in condition state 1 decrease the proportions in condition states
2 and 3 increases over time. After year 9, the proportions in condition state 3 continues
to rise while that in condition state 2 starts to decline. Figure 4.18 gives an indication of
the sample size or number of Pourable Joint Seals over time in each condition state, which
indicates that the sample of Pourable Joint Seal generally decreases over time.
73
Figure 4.16: Cumulative Distribution (Weibull) Function of sojourn times for
Pourable Joint Seal for all Condition States.
Figure 4.17: The Proportion of Pourable Joint Seal in Each Condition State.
74
Figure 4.18: The Number of Pourable Joint Seal in Each Condition State.
75
Transition Probabilities for Pourable Joint Seal (element 301) based on
Blacks Approach. The yearly overall transition probabilities based on Blacks Approach
were also computed for all the possible condition states for Pourable Joint Seal (element
301), and are shown in Equations 4.19 to 4.23.
Year 1
0.941 0.059 0 0
0 0.802 0.198 0
i,j (1) =
0
(4.19)
0 0.956 0.044
0 0 0 1
Year 2
0.928 0.072 0 0
0 0.802 0.198 0
i,j (2) =
0
(4.20)
0 0.872 0.128
0 0 0 1
Year 3
0.921 0.079 0 0
0 0.802 0.198 0
i,j (3) =
0
(4.21)
0 0.793 0.207
0 0 0 1
Year 4
0.917 0.083 0 0
0 0.802 0.198 0
i,j (4) =
0
(4.22)
0 0.720 0.280
0 0 0 1
Year 5
0.914 0.086 0 0
0 0.802 0.198 0
i,j (5) =
0
(4.23)
0 0.654 0.346
0 0 0 1
Transition Probabilities for Pourable Joint Seal (element 301) based on
Markov Chain. 4506 pairs of condition data were used in the Regression Method to
obtain the transition matrix for do-nothing action based on the Markov chain model for
Pourable Joint Seal. The results are shown in Equation 4.24.
Each Year
0.938 0.062 0 0
0 0.977 0.023 0
i,j =
0
(4.24)
0 1 0
0 0 0 1
The following graph in Figure 4.19 shows the expected Element Condition Index over
time for Pourable Joint Seal (element 301) based on Blacks Approach and the Markov chain
method. The overall rate of deterioration for Pourable Joint Seal based on the Markov Chain
model is relatively slow. This is based on the fact that the model is treating condition state
3 as an absorption state as can be seen by the transition matrix in Equation 4.24. The
76
results of the Regression Model for condition state 3 is based on the limited data at this
condition level. There is no information of the Pourable Joint Seals transitioning to the
failure state.
Figure 4.19: Expected Element Condition Index for Pourable Joint Seal over time
based on the Blacks (Semi-Markov) Approach and Markov Chain.
77
4.3.3 Reinforced Concrete Abutment
For Reinforced Concrete Abutment (element 215) the local optimal values were not
determined for the first condition state for the first trial, as the scale and shape values did
not converge within the prescribed limits of [0.01, 1000]. However, different initial values
were attempted. Local optimum values were obtained, when initial values of 1 and 1.5
for the scale and shape parameters, respectively, were attempted. A similar procedure was
followed for condition states 2 and 3 using random initial values, and pairs of local optimum
values was found for each scenario. For condition state 4, the result of the initial run gave
a shape parameter less than 1, and so an additional constraint of the shape parameter
being greater than or equal to 1 was added. However, local optimum values still were not
reach within the prescribed limits of [0.01, 1000]. The results for condition state 3 were
assumed to also apply to condition state 4. Table 4.6 gives a summary of the results of
the Blacks Approach outlining the parameters of the sojourn times in each condition state.
The resulting series of transition probabilities for the first five (5) years are also shown in
Equations 4.25 to 4.29.
for the Sojourn Times in Each Condition States for Reinforced Concrete Abutment

1* 13.33 9.84 12.68 1.55
2* 1.44 1.88 1.28 0.71 a
3* 1.15 1.79 1.02 0.59

4** 1.15 1.779 1.02 0.59
a
* Scale and Shape parameters did not converge with constraints using initial values of 1 respectively.
** The Scale and Shape parameters obtained for the previous condition state was used in the deterioration
model.
78
Transition Probabilities for Reinforced Concrete Abutment (element 215)
based on Blacks Approach. Year 1

1.000 0.0 0 0 0
0 0.604 0.396 0 0

i,j (1) = 0
0 0.460 0.540 0 (4.25)
0 0 0 0.460 0.540
0 0 0 0 1
Year 2

1.000 0.0 0 0 0
0 0.260 0.740 0 0

i,j (2) = 0
0 0.149 0.851 0 (4.26)
0 0 0 0.149 0.851
0 0 0 0 1
Year 3

1.000 0.0 0 0 0
0 0.121 0.879 0 0

i,j (3) = 0
0 0.058 0.942 0 (4.27)
0 0 0 0.058 0.942
0 0 0 0 1
Year 4

1.000 0.0 0 0 0
0 0.059 0.941 0 0

i,j (4) = 0
0 0.024 0.976 0 (4.28)
0 0 0 0.024 0.976
0 0 0 0 1
Year 5

1.000 0.0 0 0 0
0 0.029 0.971 0 0

0
i,j (5) = 0 0.011 0.989 0 (4.29)
0 0 0 0.011 0.989
0 0 0 0 1
Transition Probabilities for Reinforced Concrete Abutment (element 215)
based on Markov Chain. 7286 pairs of condition data were used in the Regression
Method to obtain the transition matrix for do-nothing action based on the Markov chain
model for Reinforced Concrete Abutment. The results are shown in Equation 4.30.
Each Year

0.994 0.006 0 0 0
0 0.994 0.006 0 0

i,j = 0
0 1 0 0 (4.30)
0 0 0 1 0
0 0 0 0 1
79
Figure 4.20: Expected Element Condition Index for Reinforced Concrete Abut-
ment over time based on the Blacks (Semi-Markov) Approach and Markov Chain.
80
Again the Markov chain model gives an overall slower rate of deterioration. The results
of the Regression Model for the Reinforced Concrete Abutment in Equation 4.30 show the
probability of remaining in condition state 3 to be equal to 1. This result indicates that there
is no record of the Reinforced Concrete Abutment deteriorating to condition levels lower
than condition state 3. However in reality, if any bridge element is allowed to deteriorate, it
will deteriorate to failure. In Figure 4.20, the deterioration subsequent to year 10 based on
Blacks Approach is exaggerated. This is due to the relatively high transition probabilities,
ij (t), in condition states 2, 3 and 4.

This chapter investigated the semi-Markov approach outlined by Black et al. (2005a,b)
and its application to analyze bridge deterioration in accordance with the AASHT O Com-
monly Recognized (CoRe) Elements data. One of the limitations in developing the dete-
rioration model is the size of the available data for condition states representing the lower
condition levels. This is due to the fact that once Bare Concrete Decks are in the lower
condition states, such as states 3, 4 or 5, rehabilitation is typically recommended.
Blacks Approach assumes that the proportion of asset in state i at interval t is equal
to the total probability of that asset being in state i after the tth interval, which is not
particularly true when the sample size of the asset being analyzed gets relatively small.
Blacks Approach is based on expressing the proportion of assets in a particular state at a
particular interval, in terms of the transition probability of that asset from one condition
state to another for a particular interval. In this chapter, graphical representations of all
associated probabilities were developed and used to explain in detail the mechanics of the
equations and assumptions made by Black et al. (2005a,b).
For each of the bridge element analyzed in this study, it was found that less data was
available to determine the transition probabilities from the lower condition states, which
made it more challenging for the optimal Weibull parameters for the sojourn times to be
estimated. In addition, Blacks Approach uses a least square method that produces local
optimal values for the Weibull parameters, which lends itself to some degree of error, since
the best fit for the sojourn time distributions may not be achieved.
Based on the investigation done in this chapter, the Blacks Approach was modified. In
Black et al. (2005a,b) a snap shot of the database at one point in time was used. However
in the investigation done in this chapter, a snap shot over twelve (12) years of data was
done, which allowed for better use of the data than if only the most recent records were
used. To overcome the challenges of missing data in the year no inspections were done,
the condition records for the missing inspection year was imputed if there were existing
condition records for the years immediately before and after that of the missing data. As a
result, better estimates of the overall network condition in a given year could be determined.
The Markov chain model used in this report was based on a regression of the historical
condition data, using matrices computation. The limitation in developing this model is
also based on the fact that there are not much information on bridge elements at the lower
condition levels. This is understandable, since bridge elements are not typically allowed to
deteriorate to the point of failure. It therefore means better inferences from the results can
81
be made when the bridge elements are fairly new and are at higher condition levels. It
follows that there is a higher degree of uncertainty as the bridge elements become older and
the condition level drops to lower condition sates.
82
CHAPTER 5
BAYESIAN UPDATE TECHNIQUE FOR

INTERVAL DEPENDENT TRANSITION
PROBABILITIES
5.1 Introduction
The chapter also looks at the application of the Bayesian technique, in which the prior
information is based on the traditional Markov chain model and the new information is
based on a semi-Markov deterioration model as developed by Black et al. (2005a,b). The
use of Bayesian update is not new as it relates to bridge management. Actually, the Pontis
R
Bridge Management System (BM S) uses a Bayesian technique for the updating transition
probabilities (Golabi et al., 1993). Golabi et al. (1993) explains that two estimates of a
transition probability can be combined by using the weighted average of the two estimates.
Hu et al. (2002) describes the use of a pseudo-Bayes technique, within the context of con-
tingency table analysis. The technique is similar to that applied in Golabi et al. (1993).
This involves first specifying an appropriate prior, and secondly updating it with a new
estimator based on observed data.
5.2 Bayesian Update of Transition Probabilities: A

Theoretical Explanation
The Bayesian technique is used for updating the transition probabilities based on the
following formula (Hu et al., 2002; Golabi et al., 1993; Bulusu and Sinha, 1992):
pui,j = ci pi,j + (1 ci )pi,j (5.1)
where,
pui,j is the updated (posterior) transition probability of the bridge element transitioning
from condition state i to j.
pi,j is the prior transition probability of the bridge element transitioning from condition
state i to j.
83
pi,j is the observed mean probability of the bridge element transitioning from condition
state i to j.
ci is the weight given to the prior transition probability of the bridge element transitioning
from condition state i to j.
Essentially, in Equation 5.1 the updated (posterior) transition probability is simply a

weighted average of prior transition probability and observed (likelihood) mean probabil-
ity. In the context of modeling transition probabilities based on a semi-Markov model it
should be noted that in Equation 5.1, the weights given to the prior and observed transi-
tion probabilities are time dependent. The observed transition probabilities are also time
dependent.
The formulation expressed in Equation 5.1 can be derived based on assumptions as it
relates to the likelihood function and the prior distribution, similar to that done by Bulusu
and Sinha (1992). Lu and Madanat (1994) outlined the statistical methodology of Bayesian
Analysis, and presented how it may be used to update facility deterioration models. If
there exists an unknown parameter, , the Bayesian approach can be used to estimate this
parameter by combining prior information and sample or observed information (x) into
what is termed the posterior distribution of given x, from which decisions and inferences
can be made (Lu and Madanat, 1994; Box and Tiao, 1992; Berger, 1985). Consider the
following Bayes formula:
L(x, )()
(|x) = R + (5.2)
L(x, )()d()
where
L(x, ) = f (x|) = likelihood of experimental outcome x, that is, conditional probability

of obtaining a particular experimental outcome assuming the parameter is ;
() = prior probability of , that is, before availability of experimental information;
(|x) = posterior probability of , that is, probability that has been revised in the light
of experimental outcome x.
The denominator on the right-hand-side of the equation is a normalizing constant.
From Equation 5.2 it can be observed that both the prior distribution and the likelihood
function contribute to the posterior distribution of . The prior probability density function
(pdf ) contains information about the prior, and the likelihood function contains information
on the observed data.
5.2.1 Likelihood function

Consider a set of observed values X1 , X2 , ....., Xn , which represent a random sample
from a population of x with underlining density function fX (x). If the parameter of the
distribution is , then the probability of observing the particular set of values is:
84
n
Y
L(x, ) = f (x|) = fX (xk |) (5.3)
k=1
where the likelihood function, f (x|), is the product of the density function of x evaluated
at X1 , X2 , ....., Xn (Lu and Madanat, 1994). Let us assume that each bridge element in
condition state i, Xk , for a given year follows a Bernoulli Distribution, where each bridge
element can either transition out of condition state i to condition state j, or remain in condi-
tion state i. Therefore X1 , X2 , ....., P
Xn can be assumed to be independently and identically
distributed Bernoulli(p), and Y = nk=1 Xk is Binomial(n, p). The likelihood function can
therefore be expressed as the Binomial Distribution (Casella and Berger, 2001):
n!
L(y|) = f (y|n, p) = py (1 p)ny (5.4)
y!(n y)!
where the probability of success at each trial is p and the probability of failure is q = 1 p.
Let the probability of remaining in condition state i be the probability of success where
p = pi,i and the probability of transitioning to the next condition state j be the probability
of failure q = pi,j = 1 pi,i . In classical statistics pi,j is constant. However, pi,j is the
probability of failure or transitioning to condition state j in the mth interval. Therefore
the likelihood function is given as:
n!
L(y|) = f (y|n, pi,j ) = (pi,j )y (1 pi,j )ny (5.5)
y!(n y)!
where
y is the number of bridge elements that transitioned to condition state j for the mth
interval.
n is the total number of bridge elements that were in condition state i at the start of
the mth interval. Let us assume that the estimated number of bridge elements that will
transition at the end of the mth interval, y, is given by:
y = pi,j n (5.6)
Therefore let
y
pi,j = (5.7)
n
For the AASHT O CoRe Bridge Element data, a proportion of a bridge element may
be recorded as being in a particular condition state for a single bridge, as explained in
Chapter 4. As a result, the number of bridge elements in the network in each condition
state is estimated by the summation of the proportions of bridge element in each condition
state across all bridges.
85
5.2.2 Prior Information
The prior information is typically based on the previous knowledge of the data. In the
case of this study, the prior information is based on transition probabilities used in the
traditional Markov chain deterioration model, which is an accepted model despite some
suspected assumptions; it is being used by over 40 states in the United States (Golabi
and Shepard, 1997; Sobanjo and Thompson, 2011). The prior information is also based on
the current number of bridge elements in each condition state, obtained from the most
recent available bridge element inspection data. The assumptions made for the prior in
this chapter is based on the distribution that is associated with the transition of the bridge
element from one condition state to another based on do-nothing actions. This is not to be
confused with the distribution of sojourn times used in developing the semi-Markov model.
If the prior distribution, (), is of the same family as the posterior distribution, (|y),
then it is called the conjugate prior. A conjugate prior simplifies the Bayes theorem when
determining the posterior distribution (Lu and Madanat, 1994). In general, there is a
natural family of prior distributions called the conjugate family, and the Beta family is
conjugate for the Binomial family. As a result, the Beta Distribution is used as a prior
distribution for binomial proportions in Bayesian Analysis (Evans et al., 2000). Let it be
assumed that the prior information has a Beta Distribution, such that:
(pi,j ) (1 pi,j )1
(pi,j |, ) = (5.8)
B(, )
where
()()
B(, ) = (5.9)
( + )

E(pi,j |, ) = (5.10)
+

V ar(pi,j |, ) = (5.11)
( + )2 (+ + 1)
represents the total number of bridge elements that will transition to condition state j
from i at the end of a given year, based on the prior model.
represents the total number of bridge elements that will remain in condition state i at
the end of the said year, based on the prior model.
Therefore + represents the total number of bridge elements that are in condition state
i at the start of the said year.
Note that total number here may not be an exact whole number, as it represents the
sum of proportions in a particular condition state.
86
5.2.3 Posterior Distribution
As outlined in Equation 5.2, the posterior density is the combination of the prior infor-
mation and the likelihood function. The two main descriptions of the posterior distribution
are its mean and its variance (Lu and Madanat, 1994). The formulas for the mean and the
variance of the posterior distribution are shown below:
Z
(|x)
E () = (|x)d() (5.12)

Z
V (|x) () = 2 (|x)d() [E (|x) ()]2 (5.13)

If the prior distribution is a conjugate of the likelihood function, then the posterior
distribution has the same form as the prior distribution. The Bayesian approach has its
advantages in engineering planning and design in that it provides a formal procedure for the
systematic updating of information which may result in an increase in prediction precision
(Lu and Madanat, 1994). Based on our discussion above, since the posterior distribution
also follows a Beta Distribution, then its parameters can be estimated. The formulation for
the posterior distribution results in Equation 5.14 (Casella and Berger, 2001):
(n + + )
(|y) = f (pi,j |y) = (pi,j )y+1 (1 pi,j )ny+1 (5.14)
( + y)( + n y)
Equation 5.14 is a Beta Distribution with parameters ( , ), where
= + y (5.15)
= + n y (5.16)
and the mean of the Beta Distribution is given as Equation 5.17

+y + n y
E(pi,j | , ) = = + (5.17)
++n ++n + ++n n
which can be reduced to Equation 5.18

y
E(pi,j | , ) = ci + (1 ci ) (5.18)
+ n
where
+
ci = (5.19)
++n
Therefore,
E(pi,j | , ) = ci pi,j + (1 ci )pi,j (5.20)

where,
87
+ represents the total estimated number of bridge elements that is in condition state
i at the start of a given year, based on the prior data.
n is the total number of bridge elements that were in condition state i at the start of the
mth interval, based on the new observed information.
5.3 Methodology
The Bayesian technique was used for updating the transition probabilities of the tradi-
tional Markov chain model, using new information obtained from a semi-Markov approach
outlined by Black et al. (2005a,b), herein referred to as Blacks Approach. Blacks Approach
was developed using historical bridge condition data for the years 1995 to 2008 inclusive, as
outlined in the previous chapter, in which Equation 5.21 was used to determine the yearly
conditional transition probabilities:
Fi (t) Fi (t 1)
pi,j (t) = pi,j (t) = (5.21)
1 Fi (t 1)
where,
pi,j (t) is the overall transition probability from condition state i to j at interval t since
becoming new, conditional on the current condition state. In this case, it represents the
observed data.
Fi (t) is the cumulative distribution function for the sojourn time in condition state i.
The results from Blacks Approach were used as the new information, which were used to
update the traditional Markov Chain deterioration model used in Pontis R
. The traditional
Markov Chain model was obtained for the Program Year 2005, and it was assumed that a
similar model represented the Program Year 2008. The transition probabilities used in the
Pontis R
BMS were considered the prior information.
The following steps were taken:
1. The Prior Transition Probabilities, pi,j s, for the bridge element were obtained from
Pontis
R
.
2. The number of the bridge elements in each condition state for the two most recent
years were determined. For this study, the number of bridges associated with the years
2007 and 2008 were used, in which each bridge was only considered once. Two years
of data was considered, because the inspection records covering the entire inventory
of bridges in Florida spans two (2) years. It was assumed that the records for 2007
and 2008 represented the values for 2008.
3. The Observed Transition Probabilities, pi,j s, representing the new information was
determined. These are the yearly transition probabilities derived from Blacks Ap-
proach.
88
4. The number in each condition state, based on the time since becoming new, was used
to develop this model. A bridge element was considered new if it was in fact new or
had just undergone major rehabilitation. During the twelve (12) years period a bridge
element on a particular bridge may have became new more than once, and each time
it was treated as an independent event. For example, it might have been observed
that 100% of the Pourable Joint Seal on a particular bridge was in condition state 1,
one (1) year after rehabilitation. After having undergone some major rehabilitation
in subsequent years, it might have been observed that 80% of the Pourable Joint Seal
on the same bridge is in condition state 1, one (1) year after the recent rehabilitation.
In this case, the proportions are summed to 180% or 1.8 in condition state 1, one (1)
year after the rehabilitation.
5. The weight given to the prior transition probability of the bridge element remaining in
condition state i, ci was determined by the proportions of bridge elements associated
with the prior and posterior information, based on Equation 5.19.
6. Equation 5.1 was used to determine the updated (posterior) transition probability of
the bridge element transitioning to condition state j, pui,j .
The updated transition probabilities of the bridge element transitioning to condition

state j, pui,j , determined was based on 12 years (1997 - 2008 inclusive) of data. These tran-
sition probabilities may be used to estimate bridge element condition for the first twelve
years after becoming new. However, what is needed is for inferences to be made much
further into the future. To be able to do this, an Updated Semi-Markov model was deter-
mined based on these set of updated transition probabilities. One way of formulating such a
model is to fit a Weibull distribution for the sojourn time of the bridge element, which will
produce transition probabilities that closely matches the updated transition probabilities
obtained from the Bayesian Update technique. To do this Equation 5.21 was used again,
where pi,j (t) is closely matched with pui,j , using the Sum of Least Squares Method, and the
resulting sojourn time (Weibull) distribution was determined from the corresponding Fi (t)
values.

5.4.1 Prior Transition Probabilities
To demonstrate the application of the Bayesian update technique, the bridge element
Pourable Seal Joint is used as an example. The following matrix represents the original
probability transition matrix for a do-nothing action for the Pourable Joint Seal (element
301) in environment 1 (benign), obtained from Pontis R
4.4.0 BMS, which is based on the
Program Year 2005 for the Florida bridge network.

0.8938 0.1062 0 0
0 0.8372 0.1628 0
P =
0
(5.22)
0 0.7009 0.2991
0 0 0 1
89
where, P is taken to be the prior information expressed as a transition probability
matrix that describes the yearly deterioration of Pourable Joint Seal. In Pontis R
, transition
matrices are formulated to determine the performance of bridge elements in four (4) different
environments: benign - 1, low - 2, moderate - 3 and severe - 4. In a benign environment
the rate of deterioration is the least. Based on Equation 5.22, p1,2 = 0.1062, p2,3 = 0.1628
and p3,F = 0.2991, where pi,j is the probability of transitioning from condition state i to
condition state j. The probability of remaining in the failure state is always 1, since failure
represents an absorption state.
A MATLAB R
program was written to determine the total number of bridge elements
in each condition state for the years 2007 and 2008. These number of bridge elements are
used as part of the prior information on the bridge elements, for which P is based. The
total number of Pourable Joint Seal in each condition state is shown in Table 5.1.
Table 5.1: Number of Pourable Joint Seal in each condition state for years 2007
and 2008
State 1 State 2 State 3
3,424.20 1,232.20 667.4
5.4.2 Observed (Likelihood) Transition Probabilities

Blacks Approach was applied to the CoRe bridge element data used in this dissertation.
The Results from applying Blacks Approach for Pourable Joint Seal (element 301) for the
first twelve (12) years, after major rehabilitation, are shown in Table 5.2.
Table 5.2: Transition Probabilities and Proportions in Condition State i for

Pourable Joint Seal
Interval Transition Probabilities Number in Condition States i
(years) p1,2 p2,3 p3,F 1 2 3
1 0.0587 0.1979 0.0436 5575.1 56.5 29.4
2 0.0723 0.1979 0.1283 3563.4 369.6 175.0
3 0.0785 0.1979 0.2072 2975 608.1 286.9
4 0.0829 0.1979 0.2797 1955.7 518.9 293.4
5 0.0862 0.1979 0.3462 1632.5 581.9 298.7
6 0.089 0.1979 0.4068 1060.9 395.6 226.5
7 0.0913 0.1979 0.4621 888.5 369.9 211.6
8 0.0934 0.1979 0.5125 595.7 270.0 163.3
9 0.0953 0.1979 0.5583 502.9 226.9 137.2
10 0.0969 0.1979 0.6000 237.9 101.0 78.1
11 0.0985 0.1979 0.6378 165.4 63.6 55.0
12 0.0999 0.1979 0.6721 10.3 3.7 3.0
The results in Table 5.2 may be considered to be a series of likelihood transition

probabilities at different time intervals, which are the observed data used to update the
90
original transition probabilities. For year 1, p1,2 = 0.0587, p2,3 = 0.1979, p3,F = 0.0436
(These are the pi,j values from Table 5.2).
5.4.3 Updated (Posterior) Transition Probabilities

The number of Pourable Joint Seal in each condition state for year 1 from Table 5.1
are 3242.2, 1232.2 and 667.4 for condition states 1, 2 and 3 respectively. An example of
the computation of the ci values and the updated probabilities for remaining in condition
states 1, 2 and 3 for the first year, using Equation 5.1, is shown below:
For condition state 1,
3424.2
c1 = = 0.3805 (5.23)
3424.2 + 5575.1
therefore,
0.3805 0.1062 + (1 0.3805) 0.0587 = 0.0768 (5.24)
1232.2
c1 = = 0.9562 (5.25)
1232.2 + 56.5
therefore,
0.9562 0.1628 + (1 0.9562) 0.1979 = 0.1643 (5.26)
667.4
c1 = = 0.9578 (5.27)
667.4 + 29.4
therefore,
0.9578 0.2991 + (1 0.9578) 0.0436 = 0.2883 (5.28)
5.4.4 Bridge Element Deterioration Predictions using Updated

Transition Probabilities
Based on the sample calculation in the previous section, pu1,2 (1) = 0.0768, pu2,3 (1) =
0.1643 and pu3,F (1) = 0.2883, which is the probability of transition to condition state j in
the first interval. Table 5.3 gives the weights for the prior transition probabilities, ci , and the
updated transition probabilities, that are used to estimate future element conditions based
on do-nothing actions for Pourable Joint Seal, where pi,j = 1 pi,i . Figure 5.1 shows the
predicated Element Condition Index for Bare Concrete Deck over twelve (12) years, based
on the results from Blacks Approach, the traditional Markov chain model from Pontis R
,
and the Updated Model (based on a combination of Blacks Approach and the traditional
Markov Chain model). The Updated Model is limited to twelve (12) years, because the
number of bridge elements in each state for each year is limited to the twelve (12) years of
available data.
Sobanjo and Thompson (2011) developed two (2) new deterioration models for AASHT O
CoRe Bridge Elements in Florida. The first model is a New Markov deterioration model,
based on the regression method using historical conditions (as outlined in Chapter 4). The
second is a Markov + Weibull deterioration model, in which the time to deterioration of
the bridge element in condition state 1 is assumed to follow a Weibull distribution, and
the time to deterioration in the remaining condition states are based on the New Markov
91
model, which assumes the time in each condition state to be exponentially distributed. In
Sobanjo and Thompson (2011), graphs of the predicted Element Condition Index over time
were produced for the New Markov model, the Markov + Weibull model and the Old
Markov model for expansion joints, decks and slabs, and substructures. Graphs for the
respective element categories are shown in Figures 5.3, 5.8 and 5.9. The Old Markov
model is the traditional Markov Chain model. Although the models developed in Sobanjo
and Thompson (2011) are based on other methodologies than which are done in this disser-
tation, except for the Old Markov model, the graphical results obtained in Sobanjo and
Thompson (2011) can be compared with the results obtained in this chapter.
The Markov + Weibull model in Sobanjo and Thompson (2011) takes into consider-
ation the Weibull distribution of the time spent in condition state 1, whereas the Blacks
Approach takes into consideration the Weibull distribution of the time spent in all condition
states. The Old Markov model in Sobanjo and Thompson (2011) is synonymous with the
traditional Markov Chain model used in this study. The Graphs for Expansion Joints in
Sobanjo and Thompson (2011) in Figure 5.3, showed the New Markov and Markov +
Weibull models being very close to each other and having a slower initial rate of deteri-
oration than the Old Markov model. However, in this dissertation, the graphs produced
from the results of the Blacks Approach, the traditional Markov Chain, and the Updated
Model (Bayesian Updated Model) were relatively close to each other for the Pourable Joint
Seal. (See Figure 5.1).
Figure 5.2 shows similar graphs to that of Figure 5.1, but they are extended over a
25 year period. In Figure 5.2, the result of the Updated Semi-Markov Model is shown
along with Blacks Approach and Markov Chain model. The Update Semi-Markov Model
is derived from the Updated Model using the Sum of Least Squares Method between the
transition probabilities for the first twelve (12) years from both models (i.e. the Updated
Model and the Update Semi-Markov Model) to fit a Weibull distribution for the sojourn
times in each condition state. This allows for the model to be extrapolated beyond the 12
years for which data is available. Table 5.4 gives the results of the Least Squares Method,
which was used to fit the sojourn time for condition state 1, in which Equation 5.21 is used
to determined the transition probabilities based on the fitted model.
92
5.4.5 Pourable Joint Seal
Table 5.3: Prior Transition Probability Weights and Updated Transition Proba-
bilities for Pourable Joint Seals
Time Prior Transition Probability Weights Updated Transition Probabilities
(years) c1 c2 c3 pu1,2 pu2,3 pu3,4
1 0.3805 0.9562 0.9578 0.0768 0.1643 0.2883
2 0.4900 0.7693 0.7922 0.0889 0.1709 0.2636
3 0.5351 0.6696 0.6993 0.0933 0.1744 0.2715
4 0.6365 0.7037 0.6946 0.0977 0.1732 0.2932
5 0.6772 0.6792 0.6908 0.0997 0.174 0.3137
6 0.7635 0.7570 0.7466 0.1021 0.1713 0.3264
7 0.7940 0.7691 0.7593 0.1031 0.1709 0.3383
8 0.8518 0.8203 0.8034 0.1043 0.1691 0.3411
9 0.8719 0.8445 0.8295 0.1048 0.1683 0.3433
10 0.9350 0.9242 0.8952 0.1056 0.1655 0.3306
11 0.9539 0.9509 0.9239 0.1058 0.1645 0.3249
12 0.9970 0.9970 0.9955 0.1062 0.1629 0.3008
Figure 5.1: Prediction of Element Condition Index for Pourable Joint Seal for
Do-nothing Actions over 12 years.
93
Figure 5.2: Prediction of Element Condition Index for Pourable Joint Seal for
Figure 5.3: Health Index vs. Age of Element: Expansion Joints (Source: Sobanjo
and Thompson (2011))
94
Table 5.4: Updated Transition Probabilities for Pourable Joint Seal (Condition
State 1) based on the Sojourn Time Fitted to a Weibull Distribution and Extended
over 25 years
2
Time (t) (yrs.) pu1,2 fitted pu1,2 fitted pu1,1 pu1,2 f itted pu1,2 Fi (t)
0 0 0 1 0
1 0.077 0.078 0.922 2.07E-06 0.078
2 0.089 0.089 0.911 1.38E-09 0.16
3 0.093 0.093 0.907 5.16E-09 0.239
4 0.098 0.096 0.904 1.54E-06 0.312
5 0.1 0.099 0.901 8.84E-07 0.38
6 0.102 0.101 0.899 2.01E-06 0.442
7 0.103 0.102 0.898 6.83E-07 0.5
8 0.104 0.104 0.896 3.58E-07 0.551
9 0.105 0.105 0.895 2.01E-08 0.598
10 0.106 0.106 0.894 2.04E-07 0.641
11 0.106 0.107 0.893 1.47E-06 0.679
12 0.106 0.108 0.892 3.23E-06 0.714
13 0.109 0.891 0.745
14 0.11 0.89 0.773
15 0.11 0.89 0.798
16 0.111 0.889 0.821
17 0.112 0.888 0.841
18 0.112 0.888 0.859
19 0.113 0.887 0.875
20 0.113 0.887 0.889
21 0.114 0.886 0.901
22 0.115 0.885 0.913
23 0.115 0.885 0.923
24 0.115 0.885 0.932
25 0.116 0.884 0.94
Minimum Least Square value = 1.25E-05
95
5.4.6 Bare Concrete Deck and Reinforced Concrete Abutment
Tables 5.5 and 5.6 give the prior weights and updated transition probabilities for Bare
Concrete Deck (element 12) and Reinforced Concrete Abutment (element 215) respectively.
Figures 5.4 and 5.5 show the predicted Element Condition Index for Bare Concrete Deck and
Reinforced Concrete abutment, respectively, over twelve (12) years based on the different
methods. Figures 5.6 and 5.7 show the predicted Element Condition Index for Bare Concrete
Deck and Reinforced Concrete abutment, respectively, over 25 years.
In Sobanjo and Thompson (2011), the graphs representing Decks and Slabs showed that
the New Markov and Markov + Weibull models gave a rate of deterioration that was
faster than the Old Markov model (See Figure 5.8). The rate of deterioration based on
the Updated Model, in this study, was faster than the traditional Markov Chain model over
the twelve years evaluated.
In Sobanjo and Thompson (2011), the graphs representing Substructures for the first
ten (10) years showed a rate of deterioration based on the Markov + Weibull model to be
generally slower than Old Markov (See Figure 5.9). For this study, the Blacks Approach
produces a rate of deterioration that is initially slower than the traditional Markov Chain
mode for the Reinforced Concrete Abutment, which is a substructure. The results are
somewhat consistent.
bilities for Bare Concrete Deck
(years) c1 c2 c3 c4 c5 pu1,2 pu2,3 pu3,4 pu4,5 pu5,F
1 0.310 0.963 0.982 1.000 1.000 0.071 0.034 0.065 0.106 0.192
2 0.384 0.789 0.776 0.824 0.600 0.072 0.030 0.244 0.249 0.482
3 0.439 0.693 0.716 0.808 0.667 0.070 0.034 0.296 0.262 0.434
4 0.529 0.712 0.748 0.778 0.667 0.064 0.040 0.268 0.286 0.434
5 0.570 0.692 0.743 0.724 0.667 0.062 0.049 0.273 0.330 0.434
6 0.638 0.729 0.769 0.875 0.75 0.057 0.056 0.25 0.207 0.373
7 0.665 0.731 0.794 0.875 0.857 0.055 0.065 0.228 0.207 0.296
8 0.734 0.773 0.82 0.875 1.000 0.049 0.068 0.206 0.207 0.192
9 0.778 0.792 0.838 0.857 1.000 0.046 0.073 0.190 0.222 0.192
10 0.929 0.908 0.883 0.875 1.000 0.033 0.055 0.151 0.207 0.192
11 0.964 0.939 0.954 0.955 1.000 0.03 0.051 0.089 0.143 0.192
12 0.997 0.996 0.994 1.000 1.000 0.027 0.036 0.055 0.106 0.192
96
bilities for Reinforced Concrete Abutment
(years) c1 c2 c3 c4 pu1,2 pu2,3 pu3,4 pu4,F
1 0.451 0.940 0.961 1.000 0.007 0.058 0.067 0.067
2 0.494 0.743 0.849 0.793 0.008 0.217 0.17 0.229
3 0.504 0.618 0.753 0.889 0.008 0.358 0.269 0.163
4 0.535 0.645 0.768 0.889 0.009 0.358 0.263 0.167
5 0.545 0.634 0.700 0.889 0.009 0.378 0.331 0.169
6 0.577 0.673 0.772 0.751 0.01 0.346 0.264 0.298
7 0.587 0.665 0.774 0.687 0.01 0.356 0.263 0.358
8 0.626 0.734 0.822 0.960 0.012 0.292 0.217 0.104
9 0.658 0.743 0.830 0.801 0.016 0.284 0.21 0.252
10 0.846 0.863 0.890 1.000 0.02 0.168 0.153 0.067
11 0.901 0.893 0.954 1.000 0.023 0.139 0.093 0.067
12 0.990 1.000 0.989 1.000 0.018 0.037 0.059 0.067
Figure 5.4: Prediction of Element Condition Index for Bare Concrete Deck for
97
Figure 5.5: Prediction of Element Condition Index for Reinforced Concrete Abut-
ment for Do-nothing Actions over 12 years.
Figure 5.6: Prediction of Element Condition Index for Bare Concrete Deck for
98
Figure 5.7: Prediction of Element Condition Index for Reinforced Concrete Abut-
ment for Do-nothing Actions over 25 years.
Figure 5.8: Health Index vs. Age of Element: Decks and Slabs (Source: Sobanjo
and Thompson (2011))
99
Figure 5.9: Health Index vs. Age of Element: Substructures (Source: Sobanjo and
Thompson (2011))
100
This chapter demonstrated how a Bayesian technique may be used to update transition
probabilities that were originally based on a Markov chain model to transition probabilities
that are based on a semi-Markov process. The chapter outlined a methodology in which
the results of the semi-Markov approach outlined by Black et al. (2005a,b) were used as
new information used to enhance or update the current Markov Chain deterioration model
used in Pontis R
. Although the new information spans 12 years of data, this data was
used to extrapolate future conditions based on the current trend. Again, as more years of
data are acquired, the accuracy of the extrapolation will improve. When the results of this
study were compared with another recent study done by Sobanjo and Thompson (2011),
the results were found to be fairly consistent.
Sobanjo and Thompson (2011) used an improved model in which the time to deteri-
oration in condition state 1 is based on a Weibull distribution and the modeling of the
remaining condition states was based on a New Markov Chain model that was based on
regression analysis. Once yearly transition probabilities are produced from a deterioration
model that is based on historical data, the Bayesian technique outlined in this chapter can
be used to develop an updated deterioration model.
From the historical condition data, it is observed that less AASHT O CoRe Bridge
Elements are at the lower condition levels in comparison to the higher condition levels. This
results in less condition data reflecting do-nothing actions at the lower condition levels. It
therefore means that better inferences can be made at the relatively new condition levels
of bridge elements, which are generally in the earlier years of the life of the bridge element.
On the contrary, there is an increase in the uncertainty in the deterioration behavior of
bridge elements at the lower condition levels.
101
CHAPTER 6
CONDITION INDEX-BASED SEMI-MARKOV

MODELS OF BRIDGE ELEMENT
DETERIORATION
6.1 Introduction
The objective of this chapter was to investigate and develop a deterioration model that is
based on the principles of semi-Markov process, in which Maximum Likelihood Estimation
(M LE) was used to estimate the parameters of a Weibull distribution that describes the
sojourn times of bridge elements in each condition state. The record of bridge elements are in
accordance with CoRe Bridge Elements. The data utilized in this study were obtained from
the bridge element condition report at Florida Department of Transportation (F DOT ).
As outlined in Chapter 4, it is easier to determine the time that bridge elements spend in
condition states when the bridge elements are based on the N BI data format. The challenge
with the CoRe Bridge Elements data is that each bridge element can have proportions in
different condition states. Being able to describe the condition level in proportions may
allow for better estimates to be made as it relates to maintenance and replacement costs
of the bridge elements; however, this can pose a challenge when using the data to develop
deterioration models. An approach is suggested in this chapter to estimate the expected
condition state. The semi-Markov approach developed by Black et al. (2005a,b) used a
Sum of Least Squares method for the parameter estimation for the distribution of the
sojourn times, which was assumed to be Weibull. However, in this chapter the Maximum
Likelihood Estimation method is used to estimate parameters for the distributions of the
sojourn times. The durations of a bridge element type in the expected condition state over
a twelve (12) year period are used to estimate the sojourn time distributions. The term
expected condition state will be explained later in this chapter.
6.2 Methodology
To overcome the issue of having proportions of a particular bridge element in different
condition states, an expected condition state value is estimated. Bridge element inspection
data for the years 1995 to 2008 are used. The objective here is to first determine the
durations of bridge elements in each expected condition state.
102
6.2.1 Expected Condition State
The formula used to determine the Element Condition Index was outlined in Chapter 4,
Equations 4.1 to 4.5. However, the current computation of the element condition index does
not take into consideration the last condition state (before the failure state). For example,
element 12 (concrete bare deck) has five (5) condition states plus the failure state. Based
on the computation of the element condition index as expressed in Equations 4.4 and 4.5, if
100% of this element is in condition state 5, it results in an element condition index of zero,
although it still is not in the failure state. It is therefore proposed that a linear relationship
amongst the condition states that incorporates the last condition state (prior to failure),
be used to express the element condition index. To do this, Equation 4.4 must be modified
accordingly. Therefore, let:

1
W Fmod = 1 (State no. 1) (6.1)
State Count
where the failure state is assigned the condition state no. which results in W Fmod = 0. For
example for a five (5) condition state element, such as element 12 (concrete bare deck) the
failure state is assigned the number 6. Using Equations 4.5 and 6.1, where W Fi = W Fmod ,
a Modified Element Condition Index (M ECI) can be computed. Sobanjo and Thompson
(2011) developed the following formula to give a computed condition index equivalent to
the M ECI discussed above:
ni
1 X pctij
ci = (ni j + 1) (6.2)
ni 100
1
where,
ci = the computed condition index (or M ECI)of element i, 0 ci 1.
ni = total listed number (in Pontis R

statecnt field or State Count) of condition states j
for bridge element i, with j = 1, 2, ..ni .
pctij = percentage of bridge quantity in state j for element i.
The expected condition state is assumed to be proportional to the M ECI as outlined

in the Table 6.1 below. It is important to note that in Table 6.1 the linear association
is between the Expected Condition State column and the value columns. The range
is determined by taking the mid-values between each state, where 100% is the maximum
possible value (for condition state 1) and 0% is the minimum possible value (for the failure
state). The syntax for computing the expected state is:
(100 M ECI)
ES = round(N S + 1) (6.3)
100
where,
ES = Expected Condition State
N S = Number of states that bridge element has, excluding the failure state.
103
M ECI = Modified Element Condition Index (%)
round = Round to the nearest integer.
Table 6.1: Determination of the Expected Condition State based on the Modified
Element Condition Index
Expected State Modified Element Condition Index
5-state index 4-state index 3-state index
value range value range value range
1 100 (90,100] 100 (87.5,100] 100 (83.5,100]
2 80 (70,90] 75 (62.5,87.5] 67 (50,83.5]
3 60 (50,70] 50 (37.5,62.5] 33 (16.5,50]
4 40 (30,50] 25 (12.5,37.5] 0 (0,16.5]
5 20 (10,30] 0 (0,12.5] - -
6 0 (0,10] - - - -
As shown in the following sections, after the expected condition state is determined,
the transition probabilities of the bridge element in each condition state are estimated and
used to create deterioration curves. For the CoRe Bridge Elements data format, an increase
in condition state number reflects a deterioration in the bridge element. Therefore if the
expected condition state shows an increase in value, then the bridge element is assumed to
have transitioned out of the previous condition state for do-nothing actions.
6.2.2 Yearly Transition Probabilities based on the Semi-Markov Process

In this chapter it is assumed that the condition of the bridge element can only drop one
level at a time, since the deterioration of bridge elements are generally relatively slow, and
condition states are not expected to drop more than one condition state in a given year. The
probability that a semi-Markov process will be in state j at time n given that it entered
state i at time zero based on Howard (1971) was presented in Equation 3.16. A similar
approach to that done in Chapter 3 was done to determine the transition probabilities for
the deterioration of the CoRe Bridge Elements, in which the change in condition states
was limited to one drop per interval. The yearly transition probabilities has the general
form as previously outlined in Equation 4.12, where i,j (m) is the value obtained from the
individual equations in the matrix of Equation 3.27.
Matlab R
Program for Bridge Deterioration. The bridge element database has
inspection records of the data with attributes such as BRIDGE ID or BRKEY (Bridge
Key), ELEM KEY (Element Key), EN V KEY (Environment Key), ST RU N IT KEY
(Structural Unit Key), ELIN SP DAT E (Element Inspection Date), QU AN T IT Y (Total
Quantity), P CT ST AT E# (Percentage in Condition State#), QT Y ST AT E# (Quantity in
Condition State#), Y EARBU ILT (Year the Bridge was Built) amongst other attributes.
For each bridge key and element key, the inspection records are sorted by increasing
element inspection dates, since there are typically many element inspection dates to
one element key and to one bridge key.
104
The expected condition state is calculated for each bridge element (element key) at
each element inspection and the total quantity noted. If the total quantity of bridge
element at a succeeding element inspection date varies by more than one (1) unit from
that of the former element inspection date, then it is assumed that there has been a
change in the total quantity of the bridge element, and that record is not used. Total
quantities can change when one bridge element type is replaced by another, and also
in the case of bridge widening. If the total quantity of a bridge element is allowed to
change significantly in a succeeding record, it may affect the result of the expected
condition state, which may lead to a misinterpretation of the actual deterioration
behavior of the bridge element.
The percentage in each condition state was used to determine the modified element
condition index for each bridge element (element key) associated with the same bridge
(bridge key).
Since inspections are typically done every two (2) years in Florida, it is expected
that a number of inspections for a given bridge for a given year will be missing.
The missing condition data is estimated, using linear interpolation, if the modified
element condition index in the alternate year is the same or less. Linear interpolation
for estimating conditions midway between inspection dates for transportation assets
have been done previously by Mishalani and Madanat (2002). Table 6.2 gives an
example of how the condition records for Pourable Joint Seal (element 301), for a
particular bridge, was arranged. In Table 6.2, the row in which Record is referred as
Estimated represents the condition record that was estimated by linear interpolation.
This is essentially averaging the proportion in each condition state from the inspection
records prior and subsequent to the year of the missing record, and then computing
the corresponding M ECI.
The duration of time for which the total number of a bridge element spends in an
expected condition state before transitioning to another state is determined. It
should be noted that one bridge element is assigned to each bridge. If the bridge
element transitioned to a higher condition state (i.e. lower state #), it is assumed
that its life in that condition state had in fact come to an end and a maintenance,
repair or rehabilitation (MR&R) action was done to the bridge element that caused
the noticeable increase in condition. If the bridge element transitioned to a lower
condition state, then it is also assumed that its life in that condition state had in
fact came to an end. However, if neither of these scenarios took place and no more
records of the bridge element condition is available, then the duration time is treated
as right-censored.
Taking into consideration both the uncensored (complete) and right-censored (incom-
plete) durations in each expected condition state, the Maximum Likelihood Estimation
(M LE) method was used to determine parameters of a Weibull Distribution. Since
bridge element inspections are typically done every two (2) years, the entire network
of bridges are generally inspected over a two (2) year period, and so it made sense to
determine the sojourn times based on two-year intervals. Therefore, if a bridge ele-
ment transitioned from a particular condition state in an odd number of years, it was
105
taken to have transitioned within the next higher even number of years. Figure 6.1
outlines the flowchart for determining the sojourn time for do-nothing actions.
106
Table 6.2: Sample Condition Record and Condition Estimation for Pourable Joint Seal
Records BRKEY ELINSPDATE QUANTITY PCTSTATE1 PCTSTATE2 PCTSTATE3 MECI YEAR ES

Actual 010004 6/3/1997 62 1 0 0 100.0 1997 1
107
Estimated 010004 62 0.5 0 0.5 66.7 1998 2
Actual 010004 5/25/1999 62 0 0 1 33.3 1999 3
Actual 010004 5/30/2000 62 0 0 1 33.3 2000 3
Import Bridge Element Data: BRKEY (Bridge Key), ELEMKEY (Element Key), ENVKEY (Environment Key),
STRUNITKEY (Structural Unit Key), ELINSPDATE (Element Inspection Date), PCTSTATE# (Percentage in
State#), QTY STATE# (Quantity in State#) (1995/2008)
Sort records by ELINSPDATE, then by STRUTUNITKEY, then by BRKEY

then by BRKEY
x = x+1 Determine the Modified Element Condition Index, MECI, for record x for Interpolated x = x +1
a particular ELINSPDATE and BRKEY. Record x represent a row of data.
Interpolate condition between the 2 yrs.

to estimate records 1 yr. later. (i.e.
Compute the Expected generate an .)
State, i, based on current
record, x. B: Overall
bridge element quantities yes
must not change by more
than 1 unit.
Current record is not

1st record in state i an
for a set BRKEY or no and ELINSPDATE for no
has same BRKEY as next record is 2 years
previous record and from current
still in state i. ELINSPDATE.
yes
State changes to j. Time in
state i is COMPLETE.
Time in state i increases based
on ELINSPDATE or expected Next record has the
date of next record. same BRKEY and
yes ELINSPDATE of
next record 2 yrs.
no
Time in state i is
CE SORED.
Maximum Likelihood Estimation is used to determine the
scale and shape parameters of the Weibull distribution (the
sojourn time in state i).
Figure 6.1: Flow Chart for Determining Sojourn Time in a Particular Condition
State for Do-nothing Action
108
The methodology described above was used to analyze the following three (3) bridge
elements: 1) Pourable Joint Seal; 2) Bare Concrete Deck; 3) Reinforced Concrete Abutment.
For this section, the results for element 301 (Pourable Joint Seal) are looked at in more
detail. The first set of results is shown in Table 6.3. The first column in Table 6.3 shows the
durations or sojourn times in years that element 301 spends in each condition state before
transition. The durations for both uncensored (complete) and censored (incomplete) times
are shown. The differentiation between uncensored and right-censored is expressed in the
second column, where 0 represents uncensored and 1 represent right-censored. The next
three (3) columns in Table 6.3 give the number of element 301 in a particular condition
state for a specified duration.
Table 6.3: The Number and Duration of Element 301 in Each Condition State
Number in Condition State
Durations (years) Censoring State 1 State 2 State 3
2 0 1015 658 179
4 0 353 164 121
6 0 78 63 47
8 0 34 12 11
10 0 5 4 0
12 0 1 0 0
2 1 1695 597 313
4 1 1011 344 166
6 1 630 248 109
8 1 378 221 131
10 1 345 108 99
12 1 166 5 3
Table 6.4 shows the parameters of the sojourn time (Weibull) distributions, along with
the corresponding 95% confidence interval bounds, obtained from the M LE computations.
Based on the 95% C.I in Table 6.4, all the estimated shape parameters for the sojourn
time (Weibull) distribution are significantly greater than 1. If the shape parameter of the
Weibull distribution is less than 1, it suggests that the rate of deterioration decreases with
time (Birolini, 2007), which is not realistic for bridge elements experiencing only do-nothing
actions. Therefore, for the purpose of predicting deterioration, a minimum scale parameter
of 1 is used in the prediction model to ensure that the deterioration rate either remains the
same or increases with time.
Table 6.5 gives the mean and variance of the resulting distribution based on the es-
timated parameters. The probability density function (pdf ) and cumulative distribution
function (cdf ) curves for all condition states for element 301 are shown in Figures 6.2
and 6.3 respectively. The Anderson-Darling (AD) statistic and corresponding p-value for
each sojourn time is outlined in Table 6.6. The fitted Weibull distributions are generally not
a good fit statistically. However as was discussed in chapter 3, the Weibull distribution is
109
Table 6.4: M LE of the Parameters of the Sojourn Time (Weibull) Distributions
for each Condition State (Pourable Joint Seal)
Condition State 95% C.I. 95% C.I.

lower upper lower upper
State 1 10.59 10.12 11.08 1.36 1.31 1.41
State 2 7.65 7.28 8.04 1.44 1.37 1.51
State 3 9.44 8.74 10.2 1.55 1.43 1.69
still a flexible distribution that can be used to describe the sojourn times in each condition
state, in which the deterioration rate is monotonically increasing.
Table 6.5: Mean and Standard Deviations of the Sojourn Time Distributions for
each Condition State (Pourable Joint Seal)
Condition State Mean Standard Deviation

State 1 9.7 7.21
State 2 6.95 4.9
State 3 8.49 5.58
Table 6.6: Goodness-of-Fit Test on Sojourn Times for Element 301 (Weibull)
Condition State Data size Goodness of Fit on Complete Durations

Complete Censored AD coefficient p - value
State 1 1486 4225 1324.478 <0.001
State 2 901 1523 606.989 <0.001
State 3 358 821 241.056 <0.001
It is assumed that a bridge element can only transition one condition state in a given
year, and the transition probability for each year can be determined based on Equation 3.27.
The results of the computations using Equation 3.27 for condition states 1, 2 and 3 are
outlined in Tables 6.7, 6.8 and 6.9 respectively.
110
Figure 6.2: Probability Density Function of the Sojourn Time in each Condition
State (Element 301)
Figure 6.3: Cumulative Density Function of the Sojourn Time in each Condition
State (Element 301)
111
Table 6.7: The Computed Transition Probabilities from Condition State 1
Intervals (years) H1,2 (m) 1,2 (m) 1,1 (m)

0 0 0 1
1 0.04 0.04 0.96
2 0.098 0.061 0.939
3 0.165 0.073 0.927
4 0.234 0.083 0.917
5 0.303 0.09 0.91
6 0.37 0.096 0.904
7 0.434 0.102 0.898
8 0.495 0.107 0.893
9 0.551 0.112 0.888
10 0.604 0.116 0.884
11 0.651 0.12 0.88
12 0.694 0.124 0.876
13 0.733 0.128 0.872
14 0.768 0.131 0.869
15 0.799 0.134 0.866
16 0.827 0.137 0.863
17 0.851 0.14 0.86
18 0.872 0.143 0.857
19 0.891 0.145 0.855
20 0.907 0.148 0.852
21 0.921 0.150 0.850
22 0.933 0.153 0.847
23 0.944 0.155 0.845
24 0.952 0.157 0.843
25 0.960 0.160 0.840
112
Intervals (years) H2,3 (m) 2,3 (m) 2,2 (m)

0 0 0 1
1 0.052 0.052 0.948
2 0.135 0.087 0.913
3 0.229 0.109 0.891
4 0.325 0.125 0.875
5 0.419 0.138 0.862
6 0.506 0.15 0.85
7 0.585 0.161 0.839
8 0.656 0.17 0.83
9 0.717 0.179 0.821
10 0.77 0.187 0.813
11 0.815 0.194 0.806
12 0.852 0.201 0.799
13 0.883 0.208 0.792
14 0.908 0.214 0.786
15 0.928 0.22 0.78
16 0.944 0.226 0.774
17 0.957 0.232 0.768
18 0.967 0.237 0.763
19 0.975 0.242 0.758
20 0.981 0.247 0.753
21 0.986 0.252 0.748
22 0.990 0.256 0.744
23 0.992 0.261 0.739
24 0.994 0.265 0.735
25 0.996 0.269 0.731
113
Intervals (years) H3,F (m) 3,F (m) 3,3 (m)

0 0 0 1
1 0.03 0.03 0.97
2 0.086 0.057 0.943
3 0.155 0.076 0.924
4 0.231 0.091 0.909
5 0.311 0.103 0.897
6 0.39 0.115 0.885
7 0.466 0.125 0.875
8 0.538 0.135 0.865
9 0.605 0.144 0.856
10 0.665 0.152 0.848
11 0.719 0.16 0.84
12 0.766 0.168 0.832
13 0.807 0.175 0.825
14 0.842 0.182 0.818
15 0.872 0.188 0.812
16 0.897 0.195 0.805
17 0.917 0.201 0.799
18 0.934 0.207 0.793
19 0.948 0.213 0.787
20 0.96 0.218 0.782
21 0.969 0.223 0.777
22 0.976 0.229 0.771
23 0.981 0.234 0.766
24 0.986 0.239 0.761
25 0.989 0.244 0.756
114
Since the condition of bridge elements are based on the percentages in each condition
state then the condition can be represented as a vector of the corresponding proportions.
For example, the condition of Pourable Joint Seal (element 301) as in Table 4.1 may be
represented as:

A(m) = 0.471 0.088 0.441 0.000 (6.4)
where zero proportions are in the failure state. It therefore means that if vector A(m)
represents the condition for interval m, then the predicted condition for interval m + 1 is

1,1 1,2 0 0
0 2,2 2,3 0
A(m + 1) = 0.471 0.088 0.441 0.000 0
(6.5)
0 3,3 3,F
0 0 0 1
The results in Tables 6.7, 6.8 and 6.9 gives the transition probabilities for each epoch,
which can be formulated into a series of transition probability matrices over time. Assuming
the bridge element is new at the end of interval (year) 0, a series of transition probability
matrices over 25 years were formulated and used to generate prediction curves over the 25
years. The result for element 301 is shown in Figure 6.4 and is labeled Proposed Semi-
Markov. Three (3) other models are also shown on the same graph. In this chapter,
the M ECI for all the models are computed in accordance with Equation 4.5, in which the
weighting factors are based on Equation 6.1 and not Equation 4.4, as was done in Chapters 4
and 5. The Proposed Semi-Markov model is compared against the following three (3)
other models:
Markov chain (Pontis R

) model is based on the transition probabilities used in

R
Pontis for Program Year 2005 for the Florida bridge network.
Blacks Approach is a semi-Markov deterioration model based on the methodologies

outlined in Black et al. (2005a,b). The application of this methodology to the CoRe
Bridge Elements was outlined in Chapter 4
The Updated Semi-Markov model is based on combining the values of transition

probabilities obtained from the traditional Markov chain model used in Pontis
R
for
Program Year 2005 and that the Blacks Approach, as outlined in Chapter 5.
Tables 6.10 and 6.11 outlines the mean and variance for the sojourn times in each
condition states for elements 12 and 215, respectively, based on the Proposed Semi-
Markov model. The estimates of the Weibull parameters are also shown in Tables 6.10
and 6.11 respectively. It is observed that the shape parameter estimates are greater than
1, which indicates that the sojourn times are generally not exponential and hence the use
of Markov Chain to model the deterioration of bridge elements is somewhat restrictive. For
Reinforced Concrete Abutment, the sojourn time distribution for condition state 4 could
not have been determined due to the small sample of reinforced concrete abutments in that
condition state. It was assumed that the sojourn time in condition state 4 was identical to
that of condition state 3 for the purpose of developing the model. The respective prediction
curves for Bare Concrete Deck (element 12) and Reinforced Concrete Abutment (element
115
215) based on the Proposed Semi-Markov model are shown in Figures 6.5 and 6.6 along
with the results of the Markov chain (Pontis
R
), Blacks Approach and Updated
Semi-Markov models.
Table 6.10: Mean, Variance and Weibull Parameter Estimates for Element 12
(Bare Concrete Deck)
Mean Standard Deviation Scale Shape

State 1 7.58 5.61 8.28 1.37
State 2 22.01 17.04 23.84 1.30
State 3 10.16 7.76 11.04 1.32
State 4 5.50 3.19 6.18 1.78
State 5 5.13 3.20 5.74 1.65
Table 6.11: Mean, Variance and Weibull Parameter Estimates for Element 215
(Reinforced Concrete Abutment)
Mean Standard Deviation Scale Shape

State 1 175.22 172.66 176.29 1.01
State 2 11.30 9.37 12.04 1.21
State 3 9.55 7.21 10.40 1.34
State 4 9.55 7.21 10.40 1.34
Figure 6.4: Deteration Curves for Element 301 (Pourable Joint Seal)
116
Figure 6.5: Deteration Curves for Element 12 (Bare Concrete Deck)
Figure 6.6: Deteration Curves for Element 215 (Reinforced Concrete Abutment)
117
This chapter outlined a methodology in which semi-Markov processes was used to model
the deterioration of CoRe bridge elements. The traditional Markov chain deterioration
models used in BM Ss are initially based on expert elicitation, which is somewhat subjective.
Also, the Markov chain deterioration model used in BM Ss assumes that the sojourn times
between successive condition states is exponential, which is not an accurate assumption,
as outlined in this study. The semi-Markov model proposed in this chapter addresses the
limitations associated with the traditional Markov chain model. The semi-Markov model
validates the hypothesis that the rate of deterioration of bridge elements tends to increase
over time.
A Weibull distribution was assumed for the distribution of the sojourn times of bridge
elements in each condition state. However, there is sometimes a limitation in obtaining
adequate data as it relates to the deterioration of CoRe bridge elements, since bridge
elements are typically not allowed to deteriorate to the point of failure. Nevertheless, the
results shown in this chapter outlines a feasible alternative method in which historical
condition data can be used to model the deterioration of bridge elements based on semi-
Markov Processes.
118
CHAPTER 7
ADAPTIVE CONTROL APPROACH TO

BRIDGE MANAGEMENT SYSTEM
7.1 Introduction
This chapter outlines a methodology in which an adaptive control approach is used for
bridge management, in which the underlining decision model is based on semi-Markov deci-
sion process (SM DP ). Adaptive control is a term that originated in the field of electronics.
Extensive research on adaptive control was done in the early 1950s as it relates to the
design of autopilots for high-performance aircraft. Intuitively, an adaptive controller can
be defined as a controller that has the capability of modifying or updating its behavior in
response to changes in the dynamics of the process and the character of the disturbances. A
more pragmatic definition is that an adaptive controller is a controller with adjustable pa-
rameters and a mechanism for adjusting the parameters (Naidu et al., 2003). The concept
of adaptive control has been used by researchers in the field of infrastructure management,
particularly as it relates to pavement and rail infrastructures (Durango and Madanat, 2002;
Guillaumot et al., 2003; Gonzalez et al., 2006; Madanat et al., 2006). Madanat et al. (2006)
also describe the methodology used in the popular Pontis R
Bridge Management System as
being adaptive, since transition probabilities are updated over time. The proposed method-
ology outlined in this chapter differs from the established Bridge Management Systems,
such as Pontis R
, because it is based on the application of SM DP , versus the traditional
discrete-time Markov decision process (M DP ).
The maintenance of a bridge element can encompass a wide range of types of work at
different durations. The Weibull distribution is also assumed for the time it takes for main-
tenance actions to be completed. However, unlike the do-nothing action, it is assumed that
the sojourn times for maintenance activities are irrespective of the current and subsequent
condition states of the bridge element. For this study, it is assumed that rehabilitation
works predominantly encompass the replacement of the respective bridge element, which
has a more predictable duration time than that of maintenance works. Based on information
obtained from Floridas Statewide Bid data, a fixed interval was assumed for the duration
of rehabilitation works, irrespective of the current condition states. The time taken for
improvement (maintenance and rehabilitation) works to be completed, can be interpreted
to start from the point at which the problem to the bridge element was first identified,
including the time for the required works to be sent out for bid if necessary, followed by the
119
time taken to undertake the works.
7.2 Background
7.2.1 Network Level Optimization
One of the key goals of a Bridge Management System (BM S) is to be able to determine
the minimum-cost long-term policy for each bridge element (Thompson and Harrison, 1993).
This policy is a set of recommended actions that minimizes the long-term Maintenance,
Repair and Rehabilitation (M R&R) cost requirements, while keeping the bridge element
out of risk of failure. If the minimum-cost long-term policy can be determined, it can be
defined as the most cost efficient set of actions to the bridge element. In other words, if
any of these actions are delayed, it will result in more expense in the long-term. Likewise,
if more improvement actions other than what is recommended are done, then it will result
in higher costs in the long-term. This is based on the concept of steady state. Bridges are
expected to remain in service for long periods of time providing transportation connectivity
on a continuous basis. As such it is very important to be able to have an optimal policy
that is sustainable far into the future.
In a BM S the following three (3) things happen each year: 1) Bridge elements deterio-
rate when no improvement actions are done to them, otherwise called do-nothing actions;
2) Improvement actions are done to some of the bridge elements, which results in corre-
sponding costs; 3) The improvement actions cause an overall improvement in conditions.
On a network level, any given condition state will have elements passing in and out of
condition states based on the M R&R actions. For steady state to occur across the entire
bridge network, the total quantity of bridge element entering a particular condition state is
equal to the total quantity of that same bridge element leaving that same condition state.
As a result the network-wide distribution of a bridge element among its condition state
remains constant from year to year, and the policy becomes sustainable over the long-term
(Thompson and Harrison, 1993). The optimal policy is the one policy that satisfies the
requirements of a steady state and therefore minimizes annual expenditure of the agency in
charge of M R&R works.
For the traditional BM S, M DP is used to determine the optimal policy. One of the
limiting assumptions of the M DP is that the effects of deterioration (do-nothing action)
are described by a Markov chain, which assumes a constant rate of deterioration. As a
result, the traditional BM Ss also have a means of updating its transition probabilities. A
constant rate of change between condition states is identical to having the sojourn time in
each state being exponentially distributed. Regression methods have been used to model
bridge performance, in which the rate of deterioration does not have to be constant. How-
ever, Markov chain modeling has been shown to give better results than some regression
methods (Thompson and Harrison, 1993; Jiang, 2010). On the other hand, the semi-Markov
model can be used to facilitate deterioration rates that may change with time. Since do-
nothing actions represent the deterioration of bridge elements, it is expected that the rate
of deterioration will increase over time. In this chapter, it is assumed that the sojourn time
in a particular condition state follows a Weibull distribution and the deterioration process
can be represented as a semi-Markov process. The Weibull distribution is more flexible
120
than the exponential distribution, and so the proposed semi-Markov deterioration model
relaxes the assumption of the distribution of the sojourn times when compared against the
Markov chain model. The probability density function (pdf ) of the Weibull distribution is
shown below (Evans et al., 2000):
1
t t
f (t) = e ( ) (7.1)

where and are scale and shape parameters respectively. When the shape parameter
equals 1, the resulting distribution becomes exponential; this means that the rate deteri-
oration is constant. If the shape parameter exceeds 1, the rate of deterioration increases
with time, and if the shape parameter is less than 1, the rate of deterioration decreases
with time. Since the latter is not expected, a minimum shape parameter of 1 is used in the
proposed SM DP model for modeling do-nothing actions. This restriction is not required
when determining the sojourn times for improvement works.
7.2.2 Project Level Selection

Determining the minimum-cost long-term policy based on steady state conditions rep-
resents the first stage, at the network-level. The second stage involves prioritizing M R&R
works on bridges at the bridge or project level, particularly when the budget does not
permit for steady state conditions for the entire network. Although the second stage is
not considered in this dissertation, it is important to explain what it involves, since the
steady state results serves as a form of constraint for project-level analyses. Traditionally,
benefit-cost ratios are determined for each bridge. The benefit-cost ratios are based on the
benefit of doing M R&R works on a bridge in the current year versus doing it at a later
date (Thompson and Harrison, 1993). Also, the amount of money by which the sum of the
project level recommended actions exceeds the long-term total cost is considered a backlog.
The backlog represents the extra work that is required now to restore or achieve long-term
steady state conditions. The benefit-cost ranking is used to select bridges that are in the
most need of M R&R works. If over time the available funding is adequate the backlog
shrinks, and if there is not adequate funding the backlog grows (Thompson and Harrison,
1993). Other factors that can be used to determine benefit-cost ratios are user-costs related
to increased bridge capacity, expected delays for alternate routes and number of accidents
experienced on bridges.
7.2.3 Modeling of Deterioration Using Semi-Markov Processes

Semi-Markov processes have been used before to model the deterioration of bridges
using National Bridge Inventory (N BI) data, but not using CoRe Bridge Elements data.
Ng and Moses (1998) and Sobanjo (2011) looked at the use of semi-Markov processes in
modeling bridge deterioration based on N BI data. Ng and Moses (1999) outlined how
the Semi-Markov Decision Process could be used to determine the minimum-cost long-
term policy for N BI Bridge Elements. Ng and Moses (1999) did a case study, using N BI
data from Indiana state. However, the cost structure under each option and the decision
processes under rehabilitation or replacement options were hypothetical. In this study, the
Semi-Markov Decision Process (SM DP ) is used to determine the minimum-cost long-term
121
policy in accordance with the CoRe Bridge Elements, in which the cost structure under
each option and the decision processes for M R&R actions are based on actual data.
As discussed in the Chapter 4, one of the reasons why there has not been a significant
level of research done on the deterioration of bridges using the CoRe Bridge Elements data
is partially due to the fact that the CoRe Bridge Elements database is more recent than
that of the N BI. As discussed in Chapter 4, it is easier to determine the time that bridge
elements spend in condition states when the condition of bridge elements are based on the
N BI data format. For the N BI data, each bridge component can only be in one condition
state at a time, and hence the time a bridge component spends in a particular condition
state can be easily determined. A methodology was developed in Chapter 6 to convert
CoRe bridge element conditions to a single Modified Element Condition Index (M ECI).
7.3 Methodology
One component of the adaptive control model for bridge elements is the deterioration
model, which was explained in the Chapter 6, where Maximum Likelihood Estimation of
the parameters of the Weibull distribution, used to describe the sojourn time from one
condition state to a lower condition state, was determined. A similar approach is also used
in this chapter to determine the sojourn time distribution for maintenance works.
7.3.1 Discount Coefficient

In the M DP formulation used in the traditional BM S, there is a discount coefficient
that is used to model the discount in money over time. The discounting of money in the
semi-Markov decision model is treated differently. Consider continuous-time discounting
at a rate s > 0. Therefore, the present value of one unit received t times units in the
future equals est . For discounting over one year, let t = 1, therefore, es = d, where d
represents corresponding discrete-time discount rate, used in M DP . So d = 0.9 corresponds
to s = log(0.9) = 0.105, which is the corresponding discount factor based on continuous-
time (Puterman, 2005).
7.3.2 The Laplace transform (s-transform)

In order to take into consideration discounting for continuous-time in the SM DP model,
the Laplace transform (s-transform) of the distribution of the sojourn times between states
has to be computed (Ibe, 2009). It is therefore prudent to look at Laplace transform prior
to discussing the proposed semi-Markov decision model. Consider if fX (x) is the pdf of a
continuous random variable X that takes only non-negative values; therefore fX (x) = 0 for
x < 0. Therefore the Laplace transform of fX (x), denoted by Mx (s) (Ibe, 2009):
Z
sX
esx fx (x)dx

Mx (s) = E e = (7.2)
0
An essential property of a Laplace transform is that when it is evaluated at s = 0, its value

is equal to 1:
122
Z
Mx (s)|s=0 = fx (x)dx = 1 (7.3)
0
The Laplace transform of the Weibull distribution is rather complex, as shown in Equa-
tion 7.4 (Sagias and Karagiannidis, 2005).
!
1k 2k pk
pk q/p
p
tX 1 , , . . . , p p
E e = k k q+p2 Gq,pp,q
p
0 1
p
q1
p
| q (7.4)
t 2 q, q,..., q (qk tk )
where G is the Meijer G-function.

It was therefore solved numerically. The numerical solutions are determined by substi-
tuting the scale and shape (Weibull) parameter estimates and the continuous-time discount
rate, s, into the expression given in Equation 7.5.
1
t
Z
t
sX sx
e( ) dx

Mx (s) = E e = e (7.5)
0
In the case of when the bridge element is in the failure state, it can be assumed that
the bridge element spends one year in that state before transitioning back into the failure
state. If a particular transition takes one year every time, then the equivalent to the Laplace
transform in this scenario is given by Equation 7.6 (Puterman, 2005):
Mx (s) = E esX = es

(7.6)
7.3.3 Semi-Markov Decision Process with Discounting

The computation of the present values based on the SM DP is given by the following
formulation (Howard, 1971; Ibe, 2009):
N
X Z
vi (a, ) = ri (a, ) + pij (a) vj (a, )e fHij (, a)d i = 1, . . . , N (7.7)
j=1 =0
where, vi (a, ) = total expected long-term discounted cost; i = the condition state of a
bridge element; a = set of feasible M R&R actions for condition state i; ri (a, ) = the
expected cost for an M R&R action a in condition state i; = the continuous-time discount
factor; pij (a) = the transition probability of the embedded Markov chain that the bridge
element will transition from condition state i to condition state j after action a is taken;
fHij is the probability density function of the sojourn time, , from condition state i to
condition state j, when action a is taken. Therefore,
N
X
vi (a, ) = ri (a, ) + pij (a)MijH (a, )vj (a, ) (7.8)
j=1
where MijH (a, ) is the Laplace transform (s-transform) of fHij (, a) evaluated at s = , and
transition probability matrix, Pi,j is given by,
123

p1,1 (a) p1,2 (a) ... ... p1,N (a)
p2,1 (a) p 2,2 (a) ... ... p2,N (a)

Pi,j (a) = . . .
... ... ... ...
(7.9)
pN 1,1 (a) pN 1,2 (a) ... . . . pN 1,N (a)
pN,1 (a) pN,2 (a) ... . . . pN,N (a)
If we let
qij (a, ) = pij (a)MijH (a, ) (7.10)

we obtain
N
X
vi (a, ) = ri (a, ) + qi (a, )vj (a, ) i = 1, . . . , N. (7.11)
j=1
According to (Ibe, 2009), the long-term value for ri (a, ) is given by

N
X Z Z t
= Bi (a) + pij ex bij (x, a)fHij (, a)dxd (7.12)
j=1 =0 x=0
where Bi (a) is the immediate cost and is determined by

N
X
Bi (a) = pij (a)Bij (a) i = 1, . . . , N Bij (a) < (7.13)
j=1
The second term in Equation 7.12 represents the cost that accumulates at the rate per
unit time until the transition to state j occurs, bij (x, a) < . For the proposed SM DP
model in this dissertation, it is assumed that only immediate costs are incurred when there
is a transition from one condition state to another. Therefore it is assumed that the second
term in Equation 7.12 equals zero. Equation 7.11 can be represented in a matrix form (Ibe,
2009). Let
T
V (a, ) = v1 (a, ) v( a, ) . . . v( a, ) (7.14)
T
R(a, ) = r1 (a, ) r2 (a, ) . . . rN (a, ) (7.15)
T
q1,1 (a, ) q1,2 (a, ) . . . q1,N (a, )
q2,1 (a, ) q2,2 (a, ) . . . q2,N (a, )
Q(a, ) =
...
(7.16)
... ... ...
qN,1 (a, ) qN,2 (a, ) . . . qN,N (a, )
From Equation 7.11, considering steady state conditions,
V (a, ) = R(a, ) + Q(a, )V (a, ) (7.17)

It follows that,
124
V (a, ) = [I Q(a, )]1 R(a, ) (7.18)
Therefore, there is a Q(a, ) that results in the minimum long-term cost, which is based
on an optimum policy. In cases where the number of possible policies is exhaustive, it is
arguably more efficient to use a process called Policy Iteration to determine the minimum
long-term cost. However, since that is not the case for this study, the matrix method can
be used to obtain the result by searching all possible policies and then selecting the most
optimal one (Ibe, 2009). The minimum long-term cost can be defined by Ibe (2009) as:

N
X
vi = min ri (a, ) + qij (a, )vj (a, ) i = 1, . . . , N (7.19)

j=1
7.3.4 Do-nothing Action (Action d)

For do-nothing actions (ad ) it was assumed that the sojourn time in a condition state
before transition, follows a Weibull distribution, except when sojourning from the failure
state. Using Equation 7.5, the numerical solution for the Laplace of the Weibull distribution
for each scenario, except failure, is determined. Therefore let:
MijH (ad , ) = L {f (, ad |ij , ij )} i, j = 1, . . . , N S (7.20)
where N S represents the number of states, except the failure state. The failure state is a
terminal state or an absorption state. Therefore for do-nothing action, a bridge element
will remain in the failure state unless rehabilitated. Therefore,
MijH (ad , ) = es = e s = ; i=j=F (7.21)
where F represents the failure state. From Equations 7.10, 7.16 and 7.20,

p1,1 L {f (, ad |1,1 , 1,1 )} . . . . . . p1,F L {f (, ad |1,F , 1,F )}
.. ..
Q(ad , ) =
. ... ... .

pF,1 L {f (, ad |F,1 , F,1 )} . . . . . . pF,F L {f (, ad |F,F , F,F )}
0 0 0 e
(7.22)
where ad represents do-nothing action.
It is assumed that the deterioration of bridge elements is a one-step process and the
bridge element can only drop by one condition state at a given point in time. In other
words, all the bridge elements in a particular condition state will eventually transition to the
next lower condition state. It is also assumed that for do-nothing actions, once the bridge
element leaves a condition state, it will not be visited again. The transition probability
for the embedded Markov chain of the semi-Markov process, pij , for do-nothing action is
assumed to be always 1 between succeeding condition states, and for the failure (absorption)
state. Therefore, if the bridge element has five (5) condition states plus the failure state,
125

0 1 0 0 0 0
0 0 1 0 0 0

0 0 0 1 0 0
Pi,j (ad ) =
0
(7.23)
0 0 0 1 0
0 0 0 0 0 1
0 0 0 0 0 1
and so Equation 7.22 can be simplified to:

0 L {f (, ad |1,2 , 1,2 )} 0 0 0 0
0 0 L {f (, a | ,
d 2,3 2,3 )} 0 0 0

0 . ..
0 0 0 0
Q(ad , ) = ..

.

0 0 0 0 0

0 0 0 0 0 L {f (, ad |5,F , 5,F )}
0 0 0 0 0 e
(7.24)
7.3.5 Maintenance Action (Action m)

For the maintenance action (am ) it is also assumed that the sojourn time distribution
is Weibull. It is also assumed that each time a maintenance action is done it may affect
the condition of each bridge element in the network differently. In other words, the same
maintenance action may result in the condition state of a bridge element to either increase,
stay the same or decrease at a slower rate than it would have normally done without
the action being done. To best estimate the transition probability of the embedded Markov
chain due to the maintenance action, one can observe the changes that take place in a sample
of bridge elements to which the action was done. To do this, the method of least squares
using matrices computation is used to determine the expected transition probability matrix
between pairs of observations (Thompson and Johnson, 2005). Consider Equation 7.25:

x1 . . . x F p1,1 (am ) . . . p1,F (am ) y1 . . . yF
.. . .. .. .. .
. . . . .. . ... . = . . . . .. (7.25)
x1 . . . xF pF,1 (am ) . . . pF,F (am ) y1 . . . yF
in which each row of xi s are the proportions of bridge element (in each state) in condition
state i before maintenance action, am ; each row of yi s are the proportions of the same
bridge element in condition state i after maintenance action, am , for i = 1, 2, . . . N S, F .
Therefore, each row of the respective matrices with the xi s and yi s values represents a
different inspection record of a particular bridge element. If the xi s and yi s are known
for a sample of the same type of bridge element, then the transition probability matrix for
action am can be determined by the method of least squares using matrices computations,
using Equation 7.26. The closest pair of inspection dates before and after a maintenance
action is typically two (2 years), however it is assumed that for the maintenance action, the
resulting transition probability matrix represent the instant transition probability when
126
the action is done, and therefore reflects the embedded Markov chain of the semi-Markov
process.
1 T
Pi,j (am ) = X T X

X Y (7.26)
To determine when maintenance activities were done and the estimated durations of
the activities, the Maintenance Management System (M M S) dataset was merged with the
bridge element inspection data. The M M S dataset contains the description of maintenance
activities done to transportation infrastructure in Florida and their expected completion
dates, including that for bridges. Based on the merged data, the bridge element conditions
before and after maintenance activities could also be estimated, and Equation 7.26 was used
to estimate Pi,j (am ). The resulting transition matrix does not take into consideration the
uncertainties as it relates to the time it takes for the maintenance works to be undertaken,
which can be described by the distribution of the sojourn time in each condition state.
The sojourn time distribution for the maintenance action is assumed to follow a Weibull
distribution. Using Equation 7.5, the numerical solution for the Laplace of the Weibull
distribution of the sojourn times irrespective of the current and subsequent condition states
was determined for N S states, excluding the failure state. Since the solutions for N S states
are numerical and the scale () and shape () estimates are not state specific, then the
Laplace of the Weibull distribution for each i, j can be represented by the following:
MijH (am , ) = L {f (, m |, )} f or condition states 1, . . . , N S (7.27)

where N S represents the number of states, except the failure state. The failure state is
again considered to be a terminal state and Equation 7.21 can also be used to determine
qF,F (am , ). Therefore for a five (5) condition state (plus failure state) element,

p1,1 L {f (, am |, )} ... ... ... ... p1,F L {f (, am |, )}
p2,1 L {f (, am |, )} ... ... ... ... p2,F L {f (, am |, )}

p3,1 L {f (, am |, )} ... ... ... ... p3,F L {f (, am |, )}
Q(am , ) =
p4,1 L {f (, am |, )}

... ... ... ... p4,F L {f (, am |, )}
p5,1 L {f (, am |, )} ... ... ... ... p5,F L {f (, am |, )}
0 0 0 0 0 e
(7.28)
127
7.3.6 Rehabilitation Action (Action r)
It is assumed that the Rehabilitation action (ar ) takes two (2) years to get done every
time. The Florida Statewide Bid Cost data, for the years 2005 to 2008, has information on
the major rehabilitation works on bridges in Florida. It was difficult to extract the items
specifically relating to the major works done to Bare Concrete Deck (element 12), as these
items are described in terms of quantities of concrete and reinforcement. However, items
relating to bridge widening was used as an assumption. Based on a sample of the records,
it was observed that major works as it relates to bridge widening had an average duration
of 1.86 years from the bid year to the completion year. Items associated with major reha-
bilitation works as it relates to Joints were observed to have an average duration of 2 years.
Based on these observations it was assumed that the Rehabilitation action takes 2 years to
get done. Therefore let time of duration, t = 2 for this action. The transition probability
of the embedded Markov chain for rehabilitation action is also fixed; it is assumed that for
any bridge element in any condition state, the resulting condition after the two (2) years
will be condition state 1 when this action is done. The transition probability for the em-
bedded Markov chain of the semi-Markov process, pij , for rehabilitation action for a five
(5) condition state element (plus failure state) is represented as:

1 0 0 0 0 0
1 0 0 0 0 0

1 0 0 0 0 0
Pi,j (ar ) =
1
(7.29)
0 0 0 0 0
1 0 0 0 0 0
1 0 0 0 0 0
Based on Equations 7.6, 7.10 and 7.16,
e2

0 0 0
e2 0 0 0
2
e 0 0 0
Q(ar , ) =
e2
(7.30)
0 0 0
e2 0 0 0
e2 0 0 0
7.3.7 Action Unit Cost for SM DP and M DP models

The use of actual cost data have been found to be challenging because actual M R&R
consists of a myriad of hidden costs, which sometimes can not be easily accounted for.
As outlined in Chapter 1, Sobanjo et al. (2002) estimated the cost of M R&R actions for
Florida bridges. One of the major problems expressed was matching the historical cost data
record for M R&R action on a bridge with the deteriorated state (Pontis R
definition) of the
bridge when this action was performed. The solution to this problem was to elicit expert
opinions and prorate the same cost between different states, using the ratio in the Pontis R
cost data from the Clemson study (Elzarka et al., 1996). The cost used in the SM DP model
was obtained from the (FHWA, 2007) website, and assumed to be applicable to the Florida
bridge network for the purpose of demonstrating the SM DP model. The traditional BM S
128
has four environmental categories: 1 - benign; 2 - low; 3 - moderate; 4 - severe. The costs
for the benign (environment 1) are used in this study and element 12 (Bare Concrete Deck)
is used as an example. The first four columns of Table 7.1 gives a summary of the possible
actions and corresponding unit costs for element 12 (FHWA, 2007; AASHTO, 2005). The
5th column of Table 7.1 gives the revised action description for both the M DP and SM DP
models based on the assumptions in developing the proposed preservation model.
In the traditional M DP model a minimum failure cost is recommended that ensures that
the model remains optimal (FHWA, 2007), which is a function of the environment, discount
rates and probability of failure. An assumed environment of 1, discrete-time discount rate
of 0.9 and probability of failure (when in condition state 5) of 3.56% (FHWA default value)
was assumed to determine the minimum failure cost. This minimum failure cost was then
used in both the SM DP model and the traditional M DP model.
129
Table 7.1: Summary of F HW A Action Unit Cost for Element 12 (Bare Concrete
Deck) in Environment 1
States Possible Actions Pontis

R
Direct Cost Revised Action
Action per sq.ft. Description
State 1 No Dam- Do Nothing 0 $0.00 Do-Nothing (ad )
age
Add a protective system 1 $89.02 Rehabilitation (ar )
State 2 Distress Do nothing 0 $0.00 Do-Nothing (ad )
2%
Repair spalls and delamina- 1 $284.17 Maintenance (am )
tions
Add a protective system 2 $73.63 Rehabilitation (ar )
State 3 2 to 10% Do Nothing 0 $0.00 Do-Nothing (ad )
distress
tions
Repair spalls and delam. and 2 $381.37* Rehabilitation (ar )
add a prot. system
State 4 10 to 25% Do Nothing 0 $0.00 Do-Nothing (ad )
distress
tions
Repair spalls and delam. and 2 $231.64* Rehabilitation (ar )
add a prot. system
State 5 Distress Do Nothing 0 $0.00 Do-Nothing (ad )
over 25%
Repair spalls and delam. and 1 $242.19 Maintenance (am )
add/or a prot. system
Replace Deck 2 $301.50 Rehabilitation (ar )
Failure state Replace Deck 2 $1,249.78 Rehabilitation (ar )
a
a
* the cost was readjusted to be equal to the cost of Repair spalls and delaminations (Pontis action 1)
in the current state, plus the cost of Add aprotective system (Pontis action 2) as described for state 2.
Therefore, for state 3 rehabilitation cost = $307.74 + $73.63 = $381.37, and for state 4 rehabilitation cost =
$158.01 + $73.63 = $231.64.
130
The first step for the results is to determine the Inputs of the Adaptive Control model:
(a) The inputs for do-nothing actions: the scale () and shape () parameter estimates
of the Weibull distribution of the sojourn time for do-nothing actions based on current
condition states.
(b) The inputs for maintenance action: a) the set of transition probabilities of the embed-
ded Markov chain for maintenance actions; b) the scale () and shape () parameter
estimates of the Weibull distribution of the sojourn time for maintenance actions, irre-
spective of condition states.
7.4.1 The Inputs for Do-nothing Actions

Table 7.2 shows the Maximum Likelihood estimates of the scale () and shape () pa-
rameters of the sojourn time (Weibull) distributions, along with the corresponding 95%
confidence interval bounds. The estimates of the shape () parameters for all the condition
states, except condition state 5 were significantly greater than 1.0, which suggests that gen-
erally the rate of deterioration increases with time. Table 7.3 gives the mean and standard
deviations of the resulting distribution based on the estimated parameters. The relatively
high standard deviation for condition state 2 is due to the variability in the raw data. A plot
of the pdf s and cdf s of the sojourn times in each condition state can be seen in Figures 7.1
and 7.2 respectively.
Table 7.2: The MLE of the Parameters of the Sojourn Time (Weibull) Distribu-
tions for each Condition State for Do-nothing Action for Bare Concrete Deck
Condition State 95% Confid. Int. 95% Confid. Int.

State 1 8.28 7.97 8.61 1.37 1.31 1.42
State 2 23.84 20.9 27.21 1.3 1.2 1.41
State 3 11.04 9.05 13.47 1.32 1.11 1.57
State 4 6.18 4.65 8.22 1.78 1.26 2.53
State 5 5.74 3.76 8.76 1.65 0.98 2.76
All the estimated parameters were substituted in Equation 7.5, along with a continuous-
time discount factor of 0.11 to determine the Laplace of the distribution (Weibull) of the
sojourn times numerically for each scenario. The assumed continuous-time discount factor of
0.11 corresponds to a 0.9 discount factor in the discrete-time M DP model. A mathematical
software, Maple 14, was used to do the numeric computations. The inputs and outputs of
the numerical computations are summarized in Table 7.4. The numerical solutions of the
Laplace transform in Table 7.4 are substituted in Equation 7.22 to determine the Q(ad , )
matrix that represents do-nothing actions.
131
Table 7.3: Mean and Std. Dev. of the Sojourn Time Distributions for Do-nothing
Action for Bare Concrete Deck
Condition State Mean Std. Dev.
State 1 7.58 5.61
State 2 22.01 17.04
State 3 10.16 7.76
State 4 5.50 3.19
State 5 5.13 3.20
Figure 7.1: Probability Density Functions of the Sojourn Time in each Condition
State for Do-nothing Action for Bare Concrete Deck
Figure 7.2: Cumulative Density Functions of the Sojourn Time in each Condition
State for Do-nothing Action for Bare Concrete Deck
132
Table 7.4: Numerical Solution for the Laplace Transform of the Sojourn Time
Distribution for Do-nothing Action for Bare Concrete Deck
Condition Continuous-Time Laplace -

States Discount Factor Numeric Solution
State 1 8.28 1.37 0.11 0.505
State 2 23.84 1.3 0.11 0.233
State 3 11.04 1.32 0.11 0.427
State 4 6.18 1.78 0.11 0.578
State 5 5.74 1.65 0.11 0.601
7.4.2 The Inputs for Maintenance Action

In Florida there is a M M S database which keeps the record of maintenance activities
done to transportation infrastructure across the State. This database is separate from the
Florida bridge element inspection records, however both databases have Bridge IDs which
were used to merge the data. From the merged data, successive pairs of inspection records
were determined for which a maintenance action was done to the respective bridge ele-
ment in between the inspection dates. Equation 7.26 was used to determine the embedded
Markov chain for the semi-Markov process that describes maintenances actions. The matri-
ces computation gave results for condition states 1 to 5. The results are shown in Table 7.5;
note that the values are in percentages (%).
Table 7.5: Matrix Representing the Embedded Markov chain for Maintenance
Expected Percentage of Elements that will transition into State j
Condition State i State 1 State 2 State 3 State 4 State 5 Failure State
State 1 51.22 36.59 12.2 0 0 0
State 2 13.04 80.43 6.52 0 0 0
State 3 5.88 5.88 76.47 11.76 0 0
State 4 0 0 0 100 0 0
State 5 0 0 0 0 100 0
Failure State 0 0 0 0 0 100
In addition to the embedded Markov chain, the sojourn time for maintenance works was
also taken into consideration. In reality, the actual duration of maintenance activities are
due to numerous scenarios. Table 7.1 gives the general descriptions for maintenance works
(action am ) for each condition state. However, in the M M S database specific descriptions
are given for each maintenance work undertaken. The following are examples of some of
these descriptions: Repair two spalls in right travel lane of span 5; Repair the spall in the
deck underside bay 4 of span 12; Clean and patch the spall in Span 4. The duration of
each activity was determined by estimating the difference in days between the estimated
completion date of the activity and the expected element inspection date that the defects
133
in the bridge element would have been first identified. These durations were used to esti-
mate the sojourn time distribution, irrespective of the condition state, by determining the
Maximum Likelihood Estimation of the parameters of the Weibull Distribution. Tables 7.6
and 7.7 outline the results. A plot of the pdf s of the sojourn times in any condition state
can be seen in Figure 7.3. Using the estimated Weibull parameters, the Laplace transform
of the sojourn time distribution was determined numerically, which is needed to determine
the qi,j (am , ) terms for maintenance action, for i = 1, 2, 3, 4, 5; j = 1, 2, 3, 4, 5. The result
of the Laplace transform is shown in Table 7.8.
Table 7.6: The MLE of the Parameters of the Sojourn Time (Weibull) Distribu-
tions for Maintenance Action for Bare Concrete Deck
Condition State 95% Confid. Int. 95% Confid. Int.
All States 0.73 0.65 0.82 1.47 1.3 1.66
Table 7.7: Mean and Std. Dev. of the Sojourn Time Distributions for Maintenance
Condition State Mean Std. Dev.
All States 0.662 0.459
Figure 7.3: Probability Density Function of the Sojourn Time in any Condition
State for Maintenance Action for Bare Concrete Deck
134
Table 7.8: Numerical Solution for the Laplace Transform of the Sojourn Time
Distribution for Maintenance Action for Bare Concrete Deck
Condition Continuous-Time Laplace -
States Discount Factor Numeric Solution
All States 1.47 0.73 0.11 0.846
135
7.4.3 Results of the Semi-Markov Decision Processes Model to F HW A
Cost Data
The SM DP determines the minimum-cost long-term policy based on Equation 7.18
using matrices computations. A program was written in Matlab R
to solve the problem
in accordance with Equation 7.19, where the optimal set of alternatives was selected to
determine the matrix Q(a, ). The matrix equation representing the optimized solution
is shown as Equation 7.31, where the optimum set of alternatives has been selected to
determine matrices V (a, ), R(a, ) and Q(a, ). Equation 7.31 is written in the same
format as Equation 7.17. The first four rows of the V (a, ), R(a, ) and Q(a, ) matrices
are based on do-nothing actions (ad ), and the last two rows are based on rehabilitation
actions (ar ). For condition state 5, Action ar means replace deck (See Table 7.1). Of
course if the bridge deck fails it has to be also replaced. The values in vector V (a, ) are
the long-term unit cost to keep bare concrete bridge decks, in particular states across the
network, out of risk of failure.

8.97 0 0 0.505 0 0 0 0 8.97
17.76 0 0 0 0.233 0 0 0 17.76

76.20 0 0 0 0 0.427 0 0 76.20 (7.31)

178.46 = 0 + 0

0 0 0 0.578 0 178.46

308.76 301.50 0.810 0 0 0 0 0 308.76
1257.04 1249.78 0.810 0 0 0 0 0 1257.04
Based on the results shown in Table 7.9, the SM DP model gives higher minimum long-
term costs, and recommends that concrete bridge decks in condition state 5 and the failure
state be rehabilitated or replaced. The SM DP model recommends do-nothing actions for
the remaining condition states. The M DP model also recommends that rehabilitation or
replacement of the deck be done when the element is in condition state 5 and the failure
state, and do-nothing actions for all other condition states.
Table 7.9: Comparison of Results: SMDP vs. Traditional MDP for Bare Concrete
Deck using F W HA Cost data
Condition State Semi-Markov Decision Process Traditional Markov Decision Process

Optimized Policy Long-term Costs Optimized Policy Long-term Costs
State 1 Action ad $8.97 Action ad $1.41
State 5 Action ar $308.76 Action ar $302.77
Failure State Action ar $1,257.04 Action ar $1,251.05
136
7.4.4 Results of the Semi-Markov Decision Processes Model to Florida
(Pontis
R
) Cost Data)
The SM DP model was applied again, using CoRe Bridge Elements cost data obtained
from Pontis R
for Program Year 2005 for the Floridas bridge network. This was done for
Concrete Bare Deck (element 12) and Pourable Joint Seal (element 301). The discrete-time
discount rate for the M DP model is 0.95, and the corresponding discount factor based on
continuous-time is s = log(0.95) = 0.051. The Summary of Floridas Action Unit Cost
for Element 12 (Bare Concrete Deck) are outlined in Table 7.10, and the optimized SM DP
policy and corresponding long-term costs are shown in Table 7.11. The main difference
between the SM DP model and the M DP model is that for the SM DP model, when the
Bare Concrete Deck is in condition state 1, a do-nothing action is recommended, while
the M DP model recommends a maintenance action. There are slight differences in the
costs for each action for each condition state. The Summary of Floridas Action Unit
Cost for Element 301 (Pourable Joint Seal) are outlined in Table 7.12, and the optimized
SM DP policy and corresponding long-term costs are shown in Table 7.13. There are no
differences as it relates to the recommended policy for each of the methods. However there
are significant costs differences for condition states 1 and 2.
137
Table 7.10: Summary of Floridas Action Unit Cost for Element 12 (Bare Concrete
Deck) in Environment 1

R
Action per sq.ft. Description
State 1 Distress Do Nothing 0 $0.00 ad
2%
Add a protective system 1 $89.03 ar
Miscellaneous Maintenance 2 $0.00 am
State 2 2 to 10% Do Nothing 0 $0.00 ad
distress
Repair spalled/ delamaintion 1 $5.00 am
Add a protective system 2 $73.64 ar
State 3 10 to Do Nothing 0 $0.00 ad
25% distress
Repair spalled areas 1 $10.00 am
Repair spalls & add prot. sys- 2 $129.18 ar
tem
State 4 25% to Do Nothing 0 $0.00 ad
50% distress
Repair spalls & add prot. sys- 2 $151.25 ar
tem
over 50%
Replace Deck 2 $30.00 ar
Failure state Agency and User Failure Cost 2 $166.12 ar
Table 7.11: Comparison of Results: SMDP vs. Traditional MDP for Bare Concrete
Deck using Floridas (Pontis
R
) Cost data

State 1 ad $0.89 am $1.52
State 2 ad $1.77 ad $4.40
State 3 ad $7.60 ad $10.69
State 4 ad $17.81 ad $21.45
State 5 ar $30.81 ar $31.52
Failure State ar $166.93 ar $167.40
138
Table 7.12: Summary of Floridas Action Unit Cost for Element 301 (Pourable
Joint Seal) in Environment 1

R
Action per ft. Description
2%
Miscellaneous Maintenance 1 $4.00 am
State 2 2 to 10% Do Nothing 0 $0.00 ad
distress
Clean Joint and Replace 1 $26.00 ar
State 3 10 to Do Nothing 0 $0.00 ad
25% distress
Clean joint, patch 1 $74.00 am
Replace Joint 2 $122.01 ar
Failure state Agency and User Failure Cost 2 $1,324.66 ar
Table 7.13: Comparison of Results: SMDP vs. Traditional MDP for Pourable
Joint Seal using Floridas (Pontis
R
) Cost data

State 1 ad $4.52 ad $85.27
State 2 am $11.11 am $125.31
State 3 ar $126.09 ar $207.34
Failure State ar $1,328.74 ar $1,401.32
139
An adaptive control approach for modeling the preservation of bridge elements based
on SM DP is presented. The main inputs for the model are: a) the scale and shape
parameter estimates of the Weibull distribution used to describe do-nothing actions, and
b) the transition probabilities of the embedded Markov chain and the scale and shape
parameter estimates of the Weibull distribution for maintenance actions. The limitations
in this study stems from the fact that there is less data available on bridge elements in
lower condition states, since typically bridges are not allowed to deteriorate to failure.
Another limitation was also in obtaining the pairs of condition records used to determine
the embedded Markov chain for maintenance actions. The M M S database is separate from
that which contains the bridge element inspection records. Records from both databases
were merged to obtain the pairs of condition record that shows when a maintenance action
was done. The size of the merged data was limited by the size of the available records from
the M M S database.
Regardless of the limiting factors, the methodology outlined in this chapter shows that
the use of SM DP can be used to determine the minimum long-term costs for the preser-
vation of bridge elements. The use of semi-Markov process to model deterioration relaxes
the assumption of the distribution of the sojourn time between condition states for do-
nothing actions and is therefore less restrictive than Markov chain deterioration models.
The SM DP model accounts for uncertainties as it relates to the rate of deterioration with
time. It also takes into consideration uncertainties as it relates to the time taken for main-
tenance activities. The model outlined in this study is also applicable for modeling the
preservation of other assets. Future research involves conducting a project level analyses
using the results from the Adaptive Control model.
140
CHAPTER 8
MODELING SERVICE LIVES OF PAVEMENT

AND BRIDGE ELEMENTS
8.1 Introduction
Survival Analyses is based on analyzing the time to the occurrence of an event. In
this case we are looking at the time to the end of service life of a pavement or bridge
element, which is the expected time the infrastructure has acceptable use to the motorized
public. There are three (3) main approaches as it relates to Survival Analyses, which
are parametric modeling, semiparametric modeling and nonparametric analysis. The three
approaches are considered in this chapter. For parametric modeling, the time to failure is
assumed to have an underlying distribution, while adjusting for predictors in the model. For
this research the Weibull Distribution was assumed. The semiparamteric modeling has an
advantage when compared against the parametric model, because no assumption is made
of the underlining distribution of the time to failure while adjusting for predictors in the
model. This model is done as a comparison to the parametric model. In this chapter,
the Cox Proportional Hazards (P H) model is used (Cox, 1972) for the semiparamteric
modeling. The Cox Proportional Hazards method was previously applied to modeling
flexible pavements in Ohio by Yu (2005) to develop survival curves based on historical
Pavement Condition Rating (PCR) data. The condition data used for analyzing the service
life of pavement in this chapter is based on the Crack Index (CRK) of a sample of pavement
segments in Florida. Mauch and Madanat (2001) looked at using the Cox P H model to
determine the time spent in different conditions states, in which the condition states are
based on the National Bridge Inventory (N BI) database. The use of the Cox P H model in
this chapter is different than that done by Mauch and Madanat (2001), because it is used
to determine the overall service life a bridge element, based on the American Association
of State Highway Transportation Officials (AASHT O)s Commonly Recognized (CoRe)
Bridge Elements. The Kaplan-Meier method is used for the nonparametric analysis.
8.2 Methodology
The first step was to organize the data for the Survival Analysis. Since the consideration
here is the service life of the infrastructure, the threshold values that defines the end of
service life had to be established.
141
8.2.1 Pavement: Service Life
The Florida Department of Transportation (F DOT ) uses a threshold value of 6.4 for
the Crack Index (CRK). In other words, if the CRK is equal to or less than 6.4, the crack
condition is considered deficient (Nasseri et al., 2009). It is therefore reasonable to assume
that the service life of the pavement ends once the CRK is equal to or less than 6.4. The
pavement data was organized such that the CRK of each pavement segment, regardless
of their lengths, could be tracked from the time they were new to the time just prior to
having a CRK 6.4. Sometimes the time to an event may not occur before the end of the
study, and so only a lower bound on the time to the event is known. It therefore means that
the information available on the time to the event is partial. When only the lower bound on
the time to the event is known, the data is said to be right-censored. It therefore means
that if a pavement segment did not deteriorate to the point where the CRK is equal to or
less than 6.4, then the service life is right-censored.
8.2.2 Bridge Element: Service Life

To conduct a Survival Analysis on the Concrete Bare Decks (element 12), it was assumed
that the service life of the concrete bare deck ends when the deck falls out of condition state 2
into condition state 3 or lower condition. This is a reasonable assumption since maintenance,
repair and rehabilitation (MR&R) works are typically recommended to concrete bare decks
that are in condition state 3. Survival models were generated using the CoRe bridge element
data for Florida for the years 1997 to 2008 inclusive. The survival models generated can
be used by bridge management officials to easily estimate the expected service life of the
bridge element.
8.2.3 Parametric Model: Weibull Model

The Weibull distribution is presented here as follows (Cleves et al., 2001):
f (t) = ptp1 exp(tp ) (8.1)

The hazards function based on the Weibull regression has the following format:
h(t) = ptp1 exp(0 + Xj BX ) (8.2)

and the baseline hazard is as follows:
h(t) = ptp1 exp(0 ) (8.3)

where p is some ancillary shape parameter estimated from the data and the scale parameter
is parametrized as exp(0 ) (Cleves et al., 2001). The resulting survival function based on
the Weibull regression is as follows:
S(t|xj ) = exp {exp(0 + xj x )tp } (8.4)

The expected scale parameter is obtained by exponentiating the estimated intercept coef-
ficient. If the ancillary shape parameter, p, is positive, then the failure rate is monotone
increasing (Cleves et al., 2001). Also, when a Weibull regression is done, a Wald test for
142
H0 : ln(p) = 0 can be done to test whether the hazard is constant. This is equivalent to
testing H0 : p = 1. If the null hypothesis is rejected, then we reject that the failure rate is
constant (Cleves et al., 2001). The Wald Test involves determining the z-statistic for ln(p)
and its standard error. The baseline survival curve for the Weibull model can be determined
by making the covariates xj = 0.
Median Service Life. The median service life, T is defined as the 50th percentile of
the survival time. Therefore, for continuous distributions, T is the value such that F (T )
= S(T ) = 0.5 and can be obtained directly from the quantile function:
T = Q(0.5) (8.5)
8.2.4 Semi-Parametric Model: Cox Model

The Proportional hazards model has the following format, given a set of covariates Xj :
h(t|Xj ) = h0 (t)exp(Xj x ) (8.6)
The Cox proportional hazards regression model (Cox, 1972) states that failure rate for the
j th subject in the data can be defined by Equation 8.7, where the regression coefficients x
are to be estimated from the data (Cleves et al., 2001).
h(t) = h0 (t)exp(xj x ) (8.7)
The survival function for the Cox model is defined as (Cleves et al., 2001):
S(t|x) = S0 (t)exp(xj x ) (8.8)
However, when there are no covariates, or = 0, the estimate of the Survival Function
is based on the Fleming-Harrington estimate (Lawless, 2003), which serves as the baseline
survival function.
8.2.5 Nonparamteric Analysis: Kaplan-Meier

The Kaplan-Meier estimate (Rabe-Hesketh and Everitt, 2004) is a nonparametric esti-
mate of the Survivor function, S(t), and it is defined as:
Y dj

S(t) = 1 (8.9)
nj
j|t(j) t
where, all the failure times, or times at which the event occurs in the sample, are ordered
and labeled t(j) such that t(1) t(2) . . . t(n) . The number of bridge elements that experi-
ence the event at time t(j) , is denoted by dj . nj represents the number of bridge elements
that have not yet experienced the event at that time and are still at risk of experiencing it,
including those censored at time t(j) . The product is for all failure times less than or equal
to t. The Kaplan-Meier estimate is sometimes referred to as the Product Limit estimate.
143
8.3.1 Survival Analysis: Service Life for Flexible Pavement
The estimated service life durations and the corresponding sectional Average Daily
Traffic (ADT ) are shown below. For the Weibull and Cox Proportional Hazards regression
analyses, only the ADT was found to be a significant predictor for the time to failure.
Tables 8.1, 8.2, 8.3 gives the results from the paramteric, semiparametric and nonparametric
analyses respectively. Table 8.3 gives the results of the Kaplan-Meier in the form of a Life
Table. The sectional Average Daily Traffic (ADT ) has a negative coefficient, which suggests
that as ADT increases the failure rate decreases, which is not logical. However, for this
data there were limited covariates and thus the model does not capture the main factors
that are associated with the service life. One possible explanation of this could be that
higher quality pavements exists where there are higher ADT. The result of the Wald test
for H0 : ln(p) = 0 for the Weibull model is such that the test statistic is 32.23, and we can
reject the null hypothesis that the failure rate is constant, and conclude that the failure
rate is not constant. The value of p is 4.472, and hence it can be concluded that the failure
rate of the flexible pavement segments is monotone increasing.
Table 8.1: Results of Weibull Regression for Flexible Pavement
Covariates Coefficient Standard Error z P-value 95% Conf. Interval

SECTADT -1.3E-05 4.72E-06 -2.66 0.008 -2.2E-05 -3.31E-06
cons -12.9566 0.6002 -21.59 0 -14.1330 -11.7803
Table 8.2: Results of Cox Regression for Flexible Pavement

SECTADT -1.3E-05 4.75E-06 -2.63 0.008 -2.2E-05 -3.20E-06
Equations 8.10 and 8.11 show the resulting equations based on the Weibull and Cox
Proportional Hazards regressions respectively.
S(t|xj ) = exp exp(12.957 1.3 105 SECT ADT )t4.472

(8.10)
5 SECT ADT )
S(t|xj ) = S0 (t)exp(1.310 (8.11)
The survival curves from the three methodologies are superimposed on the same graph
for comparison (See Figure 8.1). The survival curves from the three (3) methodologies are
relatively close. Based on the Weibull baseline survival curve, the median service life is
approximately 17 years, which is also the approximate median survival time as outlined by
Equation 8.5 (Cleves et al., 2001). This is comparable with the design period stipulated for
pavement construction of 20 years in Florida (FDOT, 2008).
144
Table 8.3: Results of the Kaplan-Meier Method for Flexible Pavement (Life Table)
Time Beginning Service Life Censored Survivor Standard 95% Conf. Interval
(years) Total Ended (failed) Function Error
1 1580 0 106 1
2 1474 0 127 1
3 1347 0 117 1
4 1230 1 107 0.9992 0.0008 0.9942 0.9999
5 1122 1 143 0.9983 0.0012 0.9932 0.9996
6 978 3 122 0.9952 0.0021 0.9885 0.998
7 853 2 72 0.9929 0.0027 0.9851 0.9966
8 779 3 66 0.9891 0.0035 0.9797 0.9941
9 710 16 116 0.9668 0.0065 0.9514 0.9774
10 578 25 91 0.925 0.0103 0.9021 0.9427
11 462 18 72 0.8889 0.0129 0.8608 0.9117
12 372 7 44 0.8722 0.0141 0.8416 0.8973
13 321 21 29 0.8151 0.0179 0.7771 0.8473
14 271 31 29 0.7219 0.0223 0.6754 0.763
15 211 18 9 0.6603 0.0247 0.6094 0.7062
16 184 13 19 0.6137 0.0261 0.5603 0.6626
17 152 12 12 0.5652 0.0276 0.5094 0.6172
18 128 10 27 0.5211 0.0287 0.4633 0.5756
19 91 7 13 0.481 0.0302 0.4206 0.5387
20 71 29 42 0.2845 0.0333 0.2213 0.3508
145
Figure 8.1: Survival Curves of Pavement Segments Service Life (3 models)
146
8.3.2 Survival Analysis: Service Life for Concrete Bare Deck
Tables 8.5, 8.6, 8.7 gives the results from the paramteric, semiparametric and nonpara-
metric analyses respectively. Table 8.7 gives the results of the Kaplan-Meier in the form
of a Life Table. The result of the Wald test for H0 : ln(p) = 0 for the Weibull model is
such that the test statistic is 3.72, and we can reject the null hypothesis that the failure
rate is constant, and conclude that the failure rate is not constant. The value of p is 1.197,
and hence it can be concluded that the failure rate of the concrete bare deck is monotone
increasing. The following are the meaning of the predictors used in the model:
adttotal - Total Average Daily Traffic (ADT ).
truckpct - Percentage of ADT that is truck traffic, that does not include vans, pickup
trucks and other light delivery trucks.
envkey - Environment key. There are four environmental categories: 1 - benign; 2 -

low; 3 - moderate; 4 - severe.
age - Age of the bridge in years. Computed by subtracting the year built from the
year of the inspection date at which the condition was noted.
IM AT ERIAL# - An indicator variable for the main material on bridge. The # repre-
sents a code, which is outlined in Table 8.4 (FDOT, 2011).
LENGTH - Length of the roadway carried by the structure.
All the significant predictors, except truckpct have positive signs. The negative sign for
truckpct is not logical, since the failure rate is expected to increase with an increase in
truck percentage. Based on the sign for the envkey predictor, it can be seen that the
more severe the environment, the more the failure rate increase. Also, older bridges (with
higher values for age) tends to have a faster failure rate. Other predictors that positively
contribute to an increase in failure rate are the total ADT (adttotal), length of the bridge
(LEN GT H) and if the main material of the bridge is made of steel (IM AT ERIAL3 ).
Table 8.4: Description of the Main Material on Bridge
Codes Description
1 Concrete
2 Concrete continuous
3 Steel
4 Steel continuous
5 Prestressed concrete*
6 Prestressed concrete continuous*
7 Wood or Timber
8 Masonry
9 Aluminum, Wrought Iron, or Cast Iron
0 Other
147
Table 8.5: Results of Weibull Regression for Concrete Bare Deck

adttotal 4.49E-06 1.45E-06 3.11 0.002 1.66E-06 7.33E-06
truckpct -0.0336 0.0097 -3.46 0.001 -0.0525 -0.0146
envkey 0.2115 0.0862 2.45 0.014 0.0425 0.3805
age 0.0086 0.0038 2.29 0.022 0.0012 0.0159
IM AT ERIAL2 -13.8283 745.2051 -0.02 0.985 -1474.4040 1446.7470
IM AT ERIAL3 0.8971 0.3760 2.39 0.017 0.1601 1.6340
IM AT ERIAL4 0.3956 0.4106 0.96 0.335 -0.4092 1.2003
IM AT ERIAL5 -0.0682 0.3662 -0.19 0.852 -0.7859 0.6495
IM AT ERIAL6 -0.1262 0.5389 -0.23 0.815 -1.1824 0.9301
IM AT ERIAL7 -0.0082 0.6218 -0.01 0.989 -1.2269 1.2104
LENGTH 0.00026 0.00012 2.25 0.024 0.00003 0.00049
cons -5.4625 0.4644 -11.76 0 -6.3727 -4.5524
Table 8.6: Results of Cox Regression for Concrete Bare Deck

adttotal 3.88E-06 1.46E-06 2.66 0.008 1.02E-06 6.75E-06
truckpct -0.0329 0.0096 -3.41 0.001 -0.0518 -0.0140
envkey 0.2303 0.0862 2.67 0.008 0.0613 0.3992
age 0.0124 0.0037 3.34 0.001 0.0051 0.0197
IM AT ERIAL2 -35.2446 38100000 0 1 -7.46E+07 7.46E+07
IM AT ERIAL3 0.8737 0.3763 2.32 0.02 0.1362 1.6111
IM AT ERIAL4 0.4306 0.4108 1.05 0.295 -0.3745 1.2357
IM AT ERIAL5 -0.0192 0.3664 -0.05 0.958 -0.7373 0.6989
IM AT ERIAL6 -0.1221 0.5389 -0.23 0.821 -1.1783 0.9342
IM AT ERIAL7 0.2110 0.6218 0.34 0.734 -1.0076 1.4297
LENGTH 0.0002 0.0001 2.03 0.043 7.93E-06 0.00046
148
Table 8.7: Results of the Kaplan-Meier Method for Concrete Bare Deck (Life Table)
Time Beginning Service Life Censored Survivor Standard 95% Conf. Interval
(years) Total Ended (failed) Function Error
1 3922 51 419 0.987 0.0018 0.9829 0.9901
2 3452 74 247 0.9658 0.003 0.9594 0.9713
3 3131 80 618 0.9412 0.004 0.9328 0.9485
4 2433 16 182 0.935 0.0043 0.9261 0.9428
5 2235 48 389 0.9149 0.0051 0.9044 0.9243
6 1798 21 143 0.9042 0.0055 0.8928 0.9145
7 1634 26 361 0.8898 0.0061 0.8772 0.9012
8 1247 8 199 0.8841 0.0064 0.8709 0.896
9 1040 5 701 0.8799 0.0066 0.8662 0.8922
10 334 0 139 0.8799 0.0066 0.8662 0.8922
11 195 1 186 0.8753 0.008 0.8587 0.8901
12 8 0 8 0.8753 0.008 0.8587 0.8901
Equations 8.12 and 8.13 show the resulting equations based on the Weibull and Cox
Proportional Hazards regressions respectively.
exp(5.463 + 4.49 106 adttotal 0.0336 truckpct + 0.212 envkey

S(t|xj ) = exp .
+0.00859 age + 0.897 IM AT ERIALM AIN 3 + 0.000261 LEN GT H)t1.197 .
(8.12)

3.88 106 adttotal 0.0329 truckpct + 0.230 envkey
exp .
+0.0124 age + 0.874 IM AT ERIALM AIN 3 + 0.000235 LEN GT H.
S(t|xj ) = S0 (t)
(8.13)
The survival function from the three methodologies are superimposed on the same graph
for comparison (See Figure 8.2). Twelve (12) years were initially plotted since that was the
extent of our data, however the Weibull curve may be extended for any number of years
(See Figure 8.3). The median service life based on the Weibull baseline survival curve is
approximately 70 years, which is also the approximate median survival time as outlined by
Equation 8.5 (Cleves et al., 2001). For comparison, the Federal Highway Administration
(F HW A) guidelines outlines a design life of 50 to 100 years for a typical road bridge
(FHWA, 2001).
149
Figure 8.2: Survival Curves of Bridge Elements Service Life (3 models) up to 12 years
Figure 8.3: Survival Curve of Bridge Elements Service Life (Weibull) up to 100 years
150
In this chapter the techniques of Survival Analyses were applied to both pavement seg-
ments and bridge elements to estimate survival curves for service life times. The parametric
models confirm that that the failure rate of pavement segments and bridge elements are typ-
ically monotone increasing. This means that the rate of deterioration of these infrastructure
tends to increase with time. The Weibull and Cox regression models indicated that there
are other factors that are associated with the deterioration of these infrastructures. For
pavement, the number of possible covariates were limited and only the sectional ADT was
found to be significant, although intuitively its sign was not logical, and thus may be due to
some other underlining factor, such as the quality of pavements that accommodates higher
ADT .
For the bridge element, Concrete Bare Deck, six (6) predictors were found to be sta-
tistically significant which have, for the most part, intuitively correct signs. Two of the
significant predictors are the environment key (envkey) and the age of the bridge element
(age), which suggests that more severe environments and older bridges contributes to a
faster failure rate. Other predictors that have positive association with the increase in fail-
ure rate are the total ADT , length of the bridge, and if the main material of the bridge is
made of steel.
151
CHAPTER 9
DISCUSSION AND CONCLUSION
9.1 Discussion
This dissertation outlined a feasible approach in which historical pavement condition
data was used to develop a semi-Markov deterioration model that was used to model net-
work level performance as it relates to the deterioration of flexible (asphalt) pavement. A
methodology was outlined in which yearly transition probabilities were estimated from a
semi-Markov deterioration model and used to estimate various interval transition probabil-
ities. Monte Carlo simulations were used to generate deterioration curves based on Markov
chain and semi-Markov deterioration models, by determining the average conditions over
time, based on 10,000 runs for each interval. The comparison of both models with the
observed pavement behavior showed that in some cases, the use of the semi-Markov model
for predicting the deterioration of flexible pavements appeared to be superior to the Markov
chain model counterpart, particularly for predictions up to five (5) years after rehabilitation.
This dissertation also outlined a methodology in which semi-Markov processes was used
to model the deterioration of the American Association of State Highway Transportation
Officials (AASHT O)s Commonly Recognized (CoRe) Bridge Elements. The traditional
Markov chain deterioration models used in BM Ss are initially based on expert elicitation,
which is somewhat subjective. Also, the Markov chain deterioration model used in Bridge
Management Systems (BM Ss) assumes a constant deterioration rate, which is not the best
assumption, as outlined in this research. The semi-Markov model proposed in this disser-
tation addressed some of the limitations that are associated with the traditional Markov
chain model. This study validates the the hypothesis that the rate of deteriorate tends
to increase with time. This was evident by the resulting shape parameters of the Weibull
distributions of the sojourn times in each condition state, which were generally greater than
1. The Weibull distribution is more flexible than the exponential distribution in describing
the sojourn times in condition states. The use of the semi-Markov process in modeling the
deterioration of transportation infrastructure is a reasonable alternative to the traditional
models based on Markov chain.
An adaptive control approach for modeling the preservation of AASHT O CoRe Bridge
Elements based on Semi-Markov Decision Processes (SM DP ) was presented in this disser-
tation, which has not been done to the extent at which it has been done in this dissertation.
The main inputs for the model are: a) the scale () and shape () parameter estimates of
152
the Weibull distribution used to describe do-nothing actions, and b) the transition proba-
bilities of the embedded Markov chain and the scale () and shape () parameter estimates
of the Weibull distribution for maintenance actions. The methodology outlined in this dis-
sertation showed that the use of SM DP can be used to determine the minimum long-term
costs for the preservation of bridge elements from the AASHT O CoRe Bridge Element
data. The use of semi-Markov process to model deterioration relaxes the assumption of the
distribution of the sojourn time between condition states for Maintenance, Repair and Re-
habilitation (M R&R) works, and is therefore less restrictive than Markov Decision Process
model. The SM DP approach can also be applied to other transportation assets, such as
highway pavements.
Also, Reliability Theory (Survival Analyses) were applied to both pavement segments
and bridge elements to estimate survival curves for service life times. Weibull and Cox
regression models were developed which showed the factors that are associated with the
deterioration of these infrastructures. The results of the Survival Analyses of the pavement
segments and bridge elements confirmed the general results of the semi-Markov deterioration
models, such as, the rate of deterioration of transportation assets are generally monotone
increasing. The estimated service lives based on the survival models were comparable with
the design lives for pavement and bridge structures.
9.2 Conclusion
The overall objective was to use stochastic techniques such as the semi-Markov processes,
to better model the performance and preservation of transportation infrastructure, and to
also show that semi-Markov models are superior to Markov chain models in this regard. The
overall objective was met, but some of the individual objectives could not be accomplished
at this time due to limitations in the availability of the required data, and will be considered
for future research work. The main conclusions of the work done in this dissertation are
listed:
1. A semi-Markov deterioration model for highway pavement segments in Florida was

developed. The semi-Markov deterioration model gave better predictions for the onset
of deterioration, up to five years subsequent to a major rehabilitation.
2. The use of Monte Carlo simulations for modeling the deterioration of transporta-
tion assets were explored further than done by previous researchers in this area, by
the application of a semi-Markov deterioration model that models multiple drops in
condition states for a given year.
3. A Bayesian technique was used for updating transition probabilities. The Bayesian
technique provides a practical means in which transition probabilities of the tradi-
tional Markov chain model were updated based on new information on transition
probabilities to obtain an updated semi-Markov model.
4. A semi-Markov deterioration model was developed for the AASHT O CoRe bridge
element data in Florida. One of the main limitations was that the availability of data
on bridge elements in lower condition states was limited.
153
5. An Adaptive Control methodology based on Semi-Markov Decision Process (SM DP )
was developed for transportation infrastructure, using the AASHT O CoRe Bridge El-
ements data as an example. The SM DP model was used to determine the minimum-
cost long-term policy at the network level and compared against the traditional M DP
model.
6. Survival curves representing the expected service lives of pavement and bridge ele-
ments were developed using parametric, semiparametric and nonparametric methods.
This study validates the hypothesis that the rate of deterioration of transportation
assets increases with time.
9.3 Future Work

There is scope for future work as it relates to developing simulation models for trans-
portation asset deterioration. The Monte Carlo simulation for pavement deterioration done
in this study could be further enhanced by bootstrapping the training data for each run
of the simulation. For the simulation done in Chapter 3, 10,000 runs were done produc-
ing 10,000 estimated condition values for each interval. The mean and standard error of
the 10,000 estimated condition values were determined for each interval. The simulation
based on the semi-Markov and Markov chain models were based on a set of 790 pavement
segment condition records, which reflected 50% of the 1580 pavement segments that were
selected for the analyses. Better use of the 1580 pavement segment records can me made by
bootstrapping the 790 pavement segment condition records used to develop the respective
models for each run of the simulation.
Traditionally, bootstrapping is used to estimate the statistic such as mean and median
by resampling with replacement from the gathered data many times (Abe, 2001). Consider
M data x1 , . . . , xM , where we want to estimate the mean by the bootstrap. We sample L
data K times from the original M data with replacement, i.e. n allowing multiple
o selection of
th j j
one datum (Abe, 2001). If we let the j sampled data set be x1 , . . . , xL , then the mean
of the j th sampled data set is as follows (Abe, 2001):
L
j 1X j
x = xl (9.1)
L
l=1
and the estimate of the mean is given by the mean of Equation 9.1:
K
1 X j
x = xk (9.2)
K
k=1
with standard error:

v
u K
1 X j
(x x)2
u
K =t (9.3)
K 1
k=1
154
To apply the bootstrap technique to the pavement deterioration model, 790 pavement
segment records will be selected at random for each run of the simulation. Based on the
790 pavement segment records, the Markov and semi-Markov models will be developed as
outlined in Chapter 3, however for the next run, another 790 pavement segment records will
be selected again at random. Each time 790 pavement segment records are selected and used
to inform the semi-Markov and Markov chain models, the remaining 790 pavement segment
records will be used to determine the average actual pavement condition for comparison.
There is also scope for further research to be done as it relates to using SM DP to
determine minimum-cost long-term policies for highway pavements at the network level.
For this to be done, the effects of improvement works done on pavements will have to be
determined along with the associated costs. Project level decision making is another vital
component of any Transportation Asset Management System (T AM S). Traditionally, at
the project level both bridge and pavement projects are ranked in order of importance. The
techniques used for ranking includes the use of Incremental Benefit-Cost and Incremental
Utility-Cost methods. However, researchers have also looked at the use of Geographical
Information Systems (GIS) for project level selection. The term Geographical Information
Systems (GIS) has different meanings, depending on the context (UN, 2000). It sometimes
relate to the overall system of hardware and software that is used to work with spatial data.
It also refers to particular software packages that are designed to handle information on
geographic features. Lastly, the term is used to describe the field of study that is associated
with the methods, algorithms and procedures for working with geographic data (UN, 2000).
This leads us to the next issue of establishing integrated T AM S at the project level using
GIS.
If T AM Ss are integrated, it may serve to optimize the use of the overall funds that are
available for transportation projects. Although some work have been done in this area, there
is still scope for more work to be done (Ofosu, 2010; Gharaibeh, 1997). The use of Geo-
graphical Information Systems (GIS) is a useful tool that can be used to integrate M R&R
works at the project level. A GIS-based project level integrated system would involve
the identification of adjacent transportation projects, such as that for highway pavements,
bridges, culverts and signs etc. The optimal combination of transportation projects at the
project level, based on their geographical proximity could be determined, while estimat-
ing the best detour routes during the period of the maintenance and rehabilitation works.
Gharaibeh (1997) outlined some advantages that may arise from project level integration,
which includes the reduction of vehicle-miles that would travel through work zones, as result
of the coordinating the implementation of adjacent projects. It is expected that this would
result in a reduction in both agency and user costs.
155
BIBLIOGRAPHY
AASHTO (2005). Pontis bridge management systems software.
Abe, S. (2001). Pattern Classification: Neuro-fuzzy Methods and Their Comparisons.

Springer-Verlag London Limited, Great Britain.
Baik, H.-S., Seok, H. J., and Abraham, D. M. (2006). Estimating transition probabilities in
markov chain-based deterioration models for management of wastewater systems. Journal
of Water Resources Planning and Management, American Society of Civil Engineers
(ASCE), 132(1):1524.
Berthold, M. R. and Hand, D. J. (2002). Intelligent Data Analysis. Springer-Verlag New

York, Inc., NY, USA.
Billington, R. and Allan, R. N. (1983). Reliability Evaluation of Engineering Systems:

Concepts and Techniques. Pitman Books Limited, London.
Birolini, A. (2007). Reliability Engineering, Theory and Practice, Fifth Edition. Springer.
Black, M., Brint, A. T., and Brailsford, J. R. (2005a). Comparing probabilistic methods for
the asset management of distributed items. Journal of Infrastructure Systems, American
Society of Civil Engineers (ASCE), 11(2):102109.
Black, M., Brint, A. T., and Brailsford, J. R. (2005b). A semi-markov approach for mod-
elling asset deterioration. Journal of the Operational Research Society, 56:12411249.
Bulusu, S. and Sinha, K. C. (1992). Simulation approach to prediction of highway structure

conditions. Transportation Research Record: Journal of the Transportation Research
Board, 1347:1117.
Casella, G. and Berger, R. L. (2001). Statistical Inference, Second Edition. Duxbury, Pacific
Grove, CA 93950 USA.
Castillo, E., Hadi, A. S., Balakrishnan, N., and Sarabia, J. M. (2005). Extreme Value and
Related Models with Applications in Engineering and Science. John Wiley and Sons, Inc.,
New Jersey.
Chassiakos, A. P., Vagiotas, P., and Theodorakopoulos, D. D. (2005). A knowledge-based

system for maintenance planning of highway concrete bridges. Advances in Engineering
Software, Elsevier, 36(1112):740749.
156
Cleves, M. A., Gould, W. W., and Guitierrez, R. G. (2001). An Introduction to Survival
Analysis Using Stata, Revised Edition. Stata Press, 4905 Lakeway Drive, College Station,
Texas 77845.
Cooper, G. R., Gibson, J. E., Eveleigh, V. W., Lindenlaub, J. C., Meditch, J. S., and Railbe,
R. H. (1960). A survey of the philosophy and state of the art of adaptive systems.
Cox, D. R. (1972). Regression models and life-tables (with discussion). Journal of the Royal
Statistical Society, Series B(34):187220.
Dekelbab, W., Al-Wazeer, A., and Harris, B. (2008). History lessons from the national
bridge inventory. Public Roads, 71(6).
DeStefano, P. D. (1998). Performance prediction and decision analysis in bridge manage-

ment.
Dietrich, B. (2010). New developments in pavement design and pavement management in

florida.
Durango, P. L. and Madanat, S. M. (2002). Optimal maintenance and repair policies in

infrastructure management under uncertain facility deterioration rates: and adaptive
approach. Transportation Research Part A, 36:763778.
Elzarka, H., Bell, L., and Sanders, S. (1996). Cost Data Generation for the PONTIS Bridge
Management System, Research Final Report and DAGS User Manual.
Evans, M., Hastings, N., and Peacock, B. (2000). Statistical Distributions, Third Edition.
John Wiley and Sons.
Eveleigh, V. W. (1967). Adaptive Control and Optimization Techniques. McGraw-Hill,

USA.
FDOT (2008). Florida Department of Transportation: Flexible Pavement Design Manual.
FDOT (2011). Florida Department of Transportation Bridge Management System Coding

Guide.
FHWA (2001). Load and Resistance Factor Design (LRFD) for Highway Bridge Substruc-
tures.
FHWA (2007). website: http://knowledge.fhwa.dot.gov/tam/aashto.nsf/home.
Fwa, T. F., Chan, W. T., and Hoque, K. Z. (2000). Multiobjective optimization for pave-
ment maintenance programming. Journal of Transportation Engieering, American Society
of Civil Engineers (ASCE), 126(5):367374.
Gharaibeh, N. G. (1997). Improving highway infrastructure management practices using

gis and optimization techniques.
157
Golabi, K. and Shepard, R. (1997). Pontis: A system for maintenance optimization and im-
provement of us bridge networks. Institute for Operations Research and the Management
Sciences, 27(1):7188.
Golabi, K., Thompson, P. D., and Hyman, W. A. (1993). Pontis Version 2.0 Technical
Manual, A Network Optimization System for Bridge Improvements and Maintenance.
Gonzalez, J., Rosario Romera, J. C., and Perez, J. M. (2006). Optimal railway infrastruc-
ture maintenance and repair policies to manage under uncertainty with adaptive control.
UC3M Working Papers. Statistics and Econometrics, 5.
Guillaumot, V. M., Durango-Cohen, P. L., and Madanat, S. M. (2003). Adaptive optimiza-

tion of infrastructure maintenance and inspection decisions under performance model
uncertainty. Journal of Infrastructure Systems, American Society of Civil Engineers
(ASCE), 9(4):133139.
Holland, J. H. (1975). Adaptation in natural and artificial systems.
Howard, R. A. (1971). Dynamic Probabilistic Systems. Volume II: Semi-Markov and Deci-
sion Processes. John Wiley and Sons Inc., Canada.
Hu, Y.-T., Kiesel, R., and Perraudin, W. (2002). The estimation of transition matrices for
sovereign credit ratings. Journal of Banking and Finance, Elsevier, 26(7):13831406.
Ibe, O. C. (2009). Markov Processes for Stochastic Modeling. Elsevier Academic Press,
Massachusetts.
Jewel, T. K. (1986). A Systems approach to Civil Engineering Planning and Design.
Jiang, Y. (2010). Application and comparison of regression and markov chain methods in
bridge condition prediction and system benefit optimization. Journal of the Transporta-
tion Research Forum, 49(2):91110.
Jiang, Y. and Sinha, K. C. (1992). Comparison of methodologies to predict bridge deterio-

ration. Transportation Research Record: Journal of the Transportation Research Board,
1597:3442.
Juni, E., Adams, T. M., and Sokolowski, D. (2008). Relating cost to condition in routine
highway maintenance. Transportation Research Record: Journal of the Transportation
Research Board, 2044:310.
Kaplan, E. L. (1982). Mathematical Programming and Games. John Wiley and Sons, Inc.,
Canada.
Kaplan, E. L. and Meier, P. (1958). Nonparametric estimation from incomplete observa-

tions. Journal of the American Statistical Association, 53:457481.
Kim, Y. J. and Yoon, D. K. (2010). Identifying critical sources of bridge deterioration in cold
regions through the constructed bridges in north dakota. Journal of Bridge Engineering,
American Society of Civil Engineers (ASCE), 15(5):542552.
158
Lawless, J. F. (2003). Statistical Models and Methods for Lifetime Data, Second Edition.
John Wiley and Sons, Inc., USA.
Lea, J. (2004). Data Mining of the Caltrans Pavement Management System (PMS)
Database.
Lee, E. T. (1992). Statistical Methods for Survival Data Analysis, Second Edition. John
Wiley and Sons, Inc., USA.
Liu, M. and Frangopol, D. M. (2006). Optimizing bridge network maintenance management

under uncertainty with conflicting criteria: Life-cycle maintenance, failure, and user costs.
Proceedings of the IEEE Conference on Evolutionary Computation, 132(11):18351845.
Lounis, Z. and Mirza, M. S. (2001). Reliability-based service life prediction of deteriorating

concrete structures.
Lu, Y. and Madanat, S. (1994). Bayesian updating of infrastructure deterioration mod-

els. Transportation Research Record: Journal of the Transportation Research Board,
1442:2110114.
Madanat, S. M., Park, S., and Kuhn, K. (2006). Adaptive optimization of infrastructure
maintenance and inspection decisions under performance model uncertainty. Journal of
Infrastructure Systems, American Society of Civil Engineers (ASCE), 12(3):192198.
Mauch, M. and Madanat, S. (2001). Semiparametric hazard rate models of reinforced

concrete bridge deck deterioration. Journal of Infrastructure Systems, American Society
of Civil Engineers (ASCE), 7(2):4957.
Micevski, T., Kuczera, G., and Coombes, P. (2002). Markov model for storm water pipe
deterioration. Journal of Infrastructure Systems, American Society of Civil Engineers
(ASCE), 8(2):4956.
Mishalani, R. G. and Madanat, S. M. (2002). Computation of infrastructure transition prob-

abilities using stochastic duration models. Journal of Infrastructure Systems, American
Society of Civil Engineers (ASCE), 8(4):113120.
Miyamoto, A., Kawamura, K., and Nakamura, H. (2002). Bridge management system and
maintenance optimization for existing bridges. Computer-Aided Civil and Infrastructural
Engineering, 15(1):4555.
Montgomery, D. C. and Runger, G. C. (2006). Applied Statistics and Probability for En-
gineers, Fourth Edition. John Wiley and Sons, Inc., 111 River Street, Hoboken, NJ
07030-5774.
Morcous, G. and Akhnoukh, A. (2006). Stochastic modeling of infrastructure deterioration:

An application to concrete bridge decks.
Naidu, D. S., Ozcelik, S., and Moore, K. L. (2003). Modeling, sensing and control of gas
metal arc welding. Elsevier Science Ltd., The Boulevard, Langford Lane, Kidlington,
Oxford OX5 1GB, UK.
159
Nasseri, S., Gunaratne, M., Yang, J., and Nazef, A. (2009). Application of improved crack
prediction methodology in floridas highway network. Transportation Research Record:
Journal of the Transportation Research Board, 2093:6775.
Ng, S.-K. (1996). Survival analysis and semi-markov bridge deterioration modeling.
Ng, S.-K. and Moses, F. (1998). Bridge deterioration modeling using semi-markov theory.
A.A. Balkema Uitgevers B.V , Structural Safety and Reliability., 1:113120.
Ng, S.-K. and Moses, F. (1999). Optimal policy for civil infrastructure systems using semi-
markov decision process. Committee Report Paper: Case Studies in Optimal Design
and Maintenance Planning of Civil Infrastructure Systems, American Society of Civil
Engineers (ASCE), pages 202215.
Ofosu, K. (2010). An integrated approach to transportation infrastructure management.
Patidar, V., Labi, S., Sinha, K. C., and Thompson, P. (2007). NCHRP Report 590
Multi-Objective optimization for Bridge Management Systems.
Primer, A. M. (1999). Asset Management Primer, U.S. Department of Transportation,

Federal Highway Administration, Office of Asset Management.
Puterman, M. L. (2005). Markov Decision Processes: Discrete Stochastic Dynamic Pro-

gramming. John Wiley and Sons, New Jersey.
Rabe-Hesketh, S. and Everitt, B. (2004). A Handbook of Statistical Analyses using Stata,

Third Edition. Chapman and Hall/CRC.
Robelin, C. A. and Madanat, S. M. (2006). Dynamic programming based maintenance and

replacement optimization for bridge decks using history-dependent deterioration models.
Applications of Advanced Technology in Transportation Proceedings of 9th International
Conference, American Society of Civil Engineers (ASCE).
Roberts, J. and Shephard, R. (2000). Bridge management for 21st century. Transportation
Research Record: Journal of the Transportation Research Board, 1696:197203.
Ross, S. M. (1996). Stochastic Processes, Second Edition. John Wiley and Sons, Inc., USA.
Ross, S. M. (2002). First Course in Probability, Sixth Edition. Prentice Hall.
Sagias, N. C. and Karagiannidis, G. K. (2005). Gaussian class multivariate weibull distribu-

tions: theory and applications in fading channels. Institute of Electrical and Electronics
Engineers. Transactions on Information Theory, 51(10):36083619.
Sastry, S. and Bodson, M. (1989). Adaptive Control Stability, Convergence, and Robustness.
Prentice-Hall.
Sawaragi, Y., Nakayama, H., and Tanino, T. (1985). Theory of Multiobjective Optimization
(vol. 176 of Mathematics in Science and Engineering). Academic Press Inc., Orlando,
FL, USA.
160
Shahin, M. Y. (2006). Pavement Management for Airports, Roads and Parking Lots
Second Edition. Springer.
Shepard, R. W. and Johnson, M. B. (2001). California bridge health index - a diagnostic

tool to maximize bridge longevity, investment. Transportation Research Record: Journal
of the Transportation Research Board, 215:611.
Small, E. P., Philbin, T., Fraher, M., Fraher, M., and Romack, G. P. (2000). Current
status of bridge management system implementation in the united states. Transportation
Research Record: Journal of the Transportation Research Board, 498:16.
Sobanjo, J., Mtenga, P., and Rambo-Roddenberry, M. (2010). Reliability-based modeling of

bridge deterioration hazards. Journal of Bridge Engineering, American Society of Civil
Engineers (ASCE), 15(6):671683.
Sobanjo, J. O. (2011). State transition probabilities in bridge deterioration based on weibull

sojourn times. Structure and Infrastructure Engineering: Maintenance, Management,
Life-Cycle Design and Performance, 7(10):747764.
Sobanjo, J. O. and Thompson, P. D. (2004). Project Planning Models for Florida Bridge
Management System. Technical Report.
Sobanjo, J. O. and Thompson, P. D. (2007). Decision Support for Bridge Programming and
Budgeting. Technical Report.
Sobanjo, J. O. and Thompson, P. D. (2011). Enhancement of the FDOTs Project Level and
Network Level Bridge Management Analysis Tools. Final Report. Contract No. BDK83
977-01. Florida Department of Transportation.
Sobanjo, J. O., Thompson, P. D., Lewis, M., and Kerr, R. (2002). Estimating agency cost
of maintenance, repair, and rehabilitation for florida bridges. Transportation Research
Record: Journal of the Transportation Research Board, 1795:6673.
Steuer, R. E. (1986). Multiple Criteria Optimization: Theory, Computations, and Applica-

tion. John Wiley and Sons, Inc., NY, USA.
Stukhart, G. (1991). Study for a Comprehensive Bridge Management System for Texas.
Texas Transportation Institute, TX, USA.
Thompson, P. D. and Harrison, F. D. (1993). Pontis Version 2.0 Users Manual, A Network
Optimization System for Bridge Improvements and Maintenance.
Thompson, P. D. and Johnson, M. B. (2005). Markovian bridge deterioration: developing

models from historical data. Structure and Infrastructure Engineering, 1:8591.
Thompson, P. D. and Shepard, R. W. (2000). AASHTO Commonly-Recognized Bridge

Elements: Successful Applications and Lessons Learned.
Thompson, P. D. and Sobanjo, J. O. (2006a). Network Analysis Tool Users Manual Release
2.0.
161
Thompson, P. D. and Sobanjo, J. O. (2006b). Project Level Analysis Tool Users Manual
Release 2.0.
Tobias, P. A. and Trindade, D. C. (1995). Applied Reliability, 2nd Edition. Chapman and
Hall/CRC Press, Florida.
UN (2000). Handbook on geographic information systems and digital mapping.
Wang, C. P. (1992). Pavement network optimization and analysis.
Wang, K. C. P., Zaniewski, J., and Way, G. (1994). Probabilistic behavior of pavements.
Journal of Transportation Engineering, American Society of Civil Engineers (ASCE),
120(3):358375.
Yanev, B. S. (2007). Bridge Management. John Wiley, Hoboken, New Jersey.
Yang, J. (2004). Road crack condition performance modeling using recurrent markov chains
and artificial neural networks.
Yang, J., Gunaratne, M., Lu, J. J., and Dietrich, B. (2005). Use of recurrent markov chains
for modeling the crack performance of flexible pavements. Journal of Transportation
Engineering, American Society of Civil Engineers (ASCE), 131(11):861872.
Yang, J., Lu, J. J., Gunaratne, M., and Dietrich, B. (2009). Modeling crack deterioration of
flexible pavements: Comparison of recurrent markov chains and artificial neural networks.
Transportation Research Record: Journal of the Transportation Research Board, 1974:18
25.
Yu, J. (2005). Pavement service life estimation and condition prediction.
162
BIOGRAPHICAL SKETCH
Omar Thomas was born in the beautiful island of Jamaica. He pursued a Bachelor of Science
degree in Civil Engineering at the University of the West Indies, St. Augustine, Trinidad.
Subsequently, he worked for some years in Jamaica as a professional civil engineer, after
which he pursued a Masters degree in Civil Engineering at Cornell University, Ithaca. He
returned to his home country where he continued to work in the industry for a few years.
In Fall 2007, Omar began to pursue a doctoral degree at Florida State University, Tal-
lahassee, under the supervision of Dr. John Sobanjo. Omar was awarded second place for
the Big Bend Florida Chapter of the Institute of Transportation Engineers (ITE) Book
Scholarship presentation competition in Fall 2008, where he presented on a manuscript
entitled Modeling Traffic Flow in Construction Work Zones. In Spring 2011, Omar re-
ceived a Dissertation Research Grant award at Florida State University, for a poster and
oral presentation done on another manuscript entitled, An Adaptive Control Methodology
for Infrastructure Management System based on Semi-Markov Decision Processes, at the
52nd Annual Transportation Research Forum, Long Beach California. Omar successfully
completed his Doctoral degree in Summer 2011 at the Florida State University.
163

Stochastic Preservation

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stochastic Preservation

Uploaded by

Copyright:

Available Formats

Florida State University Libraries

Electronic Theses, Treatises and Dissertations The Graduate School

Stochastic Preservation Model for

STOCHASTIC PRESERVATION MODEL FOR TRANSPORTATION

OMAR ST. AUBYN ALEXANDER THOMAS

A Dissertation submitted to the

Kamal Tawfiq, Chair, Department of Civil and Environmental Engineering

John Collier, Dean, College of Engineering

3 COMPARISON OF SEMI-MARKOV AND TRADITIONAL MARKOV

4 SEMI-MARKOV APPROACH FOR MODELING THE DETERIORA-

5 BAYESIAN UPDATE TECHNIQUE FOR INTERVAL DEPENDENT

6 CONDITION INDEX-BASED SEMI-MARKOV MODELS OF BRIDGE

7 ADAPTIVE CONTROL APPROACH TO BRIDGE MANAGEMENT

8 MODELING SERVICE LIVES OF PAVEMENT AND BRIDGE ELE-

9 DISCUSSION AND CONCLUSION 152

2.1 Number of transformers in each paintwork class grouped by the number of

3.1 The Range of Crack Indices and Corresponding Condition States . . . . . . . 23

3.2 Transition Probabilities of the Embedded Markov Chain of the Semi-Markov

3.4 Means and Standard Deviations of Sojourn Times (Weibull) . . . . . . . . . 42

3.5 Goodness-of-Fit Test on Pavement Sojourn Times (Weibull) . . . . . . . . . 43

4.3 Computed Condition State 1 probabilities for Bare Concrete Deck . . . . . . 59

4.5 Computed Condition State 1 probabilities for Pourable Joint Seal . . . . . . 69

5.2 Transition Probabilities and Proportions in Condition State i for Pourable

6.7 The Computed Transition Probabilities from Condition State 1 . . . . . . . . 112

6.8 The Computed Transition Probabilities from Condition State 2 . . . . . . . . 113

6.9 The Computed Transition Probabilities from Condition State 3 . . . . . . . . 114

8.1 Results of Weibull Regression for Flexible Pavement . . . . . . . . . . . . . . 144

8.2 Results of Cox Regression for Flexible Pavement . . . . . . . . . . . . . . . . 144

8.4 Description of the Main Material on Bridge . . . . . . . . . . . . . . . . . . . 147

8.5 Results of Weibull Regression for Concrete Bare Deck . . . . . . . . . . . . . 148

8.6 Results of Cox Regression for Concrete Bare Deck . . . . . . . . . . . . . . . 148

1.1 Conceptual illustration of a pavement condition life-cycle (Source: Shahin

2.2 A Typical Sample Function of the Deterioration (Semi-Markov) Process (Source:

3.2 Schematic of Pavement Segment Divisions into Equal Units of One-Tenth of a

3.3 An Example of the Change in the Condition States of Pavement Segments

3.4 Flowchart for a single run of the simulation over 20 years. . . . . . . . . . . . 33

3.5 Frequency of Sojourn Times in Condition State 10 Before Transitioning to 9 37

3.6 Frequency of Sojourn Times in Condition State 10 Before Transitioning to 8 38

3.7 Frequency of Sojourn Times in Condition State 9 Before Transitioning to 8 . 39

3.8 Frequency of Sojourn Times in Condition State 9 Before Transitioning to 7 . 40

3.13 Proportions of pavements in condition state 9 and higher . . . . . . . . . . . 49

4.2 Graphical representation for p2,3 (3). . . . . . . . . . . . . . . . . . . . . . . . 56

4.3 Graphical representation for determining pE

4.9 The Proportion of Bare concrete Deck in Each Condition State. . . . . . . . 64

4.10 The Number of Bare concrete Deck in Each Condition State. . . . . . . . . . 65

4.17 The Proportion of Pourable Joint Seal in Each Condition State. . . . . . . . 74

4.18 The Number of Pourable Joint Seal in Each Condition State. . . . . . . . . . 75

6.5 Deteration Curves for Element 12 (Bare Concrete Deck) . . . . . . . . . . . . 117

8.1 Survival Curves of Pavement Segments Service Life (3 models) . . . . . . . . 146

1.1 Problem Statement

1.2 Research Objectives

1. To develop a semi-Markov deterioration model for highway pavement in Florida, based

2. To develop semi-Markov deterioration models for AASHT O Commonly Recognized

3. To determine the maintenance-effectiveness of transportation infrastructure network.

4. To develop a network preservation model for transportation infrastructure, using an

1.3 Background on Pavement Management

12 - Bare Concrete Deck (5 condition states)

13 - Concrete Deck/Slab with Asphalt Concrete Overlay (5 condition states)

301 - Pourable Joint Seal (3 condition states)

302 - Compression Joint Seal (3 condition states)

321 - Reinforced Concrete Approach Slab (4 condition states)

1.6 Dissertation Outline

2.1 Stochastic Models

Pij = P r {Xn+1 = j|Xn = i} (2.1)