UDOT Audit

UTAH DEPARTMENT OF TRANSPORTATION
I-15 UTAH COUNTY CORRIDOR EXPANSION
Findings and Recommendations
Report No. 11-DOT-A
Keeping Utah AUSTON G. JOHNSON, CPA

Financially Strong UTAH STATE AUDITOR
STATE OF UTAH DEPUTY STATE AUDITOR:
Office of the State Auditor Joe Christensen, CPA
UTAH STATE CAPITOL COMPLEX AUDIT DIRECTORS:
EAST OFFICE BUILDING, SUITE E310
P.O. BOX 142310 Van H. Christensen, CPA
SALT LAKE CITY, UTAH 84114-2310 Deborah A. Empey, CPA
Auston G. Johnson, CPA (801) 538-1025 Stan Godfrey, CPA
FAX (801) 538-1383 Jon T. Johnson, CPA
UTAH STATE AUDITOR
MANAGEMENT LETTER NO. 11-DOT-A
April 11, 2011
Gary R. Herbert, Governor

State of Utah
The Transportation Commission, Audit Committee,

and
John R. Njord, Executive Director
Utah Department of Transportation
At the request of Governor Herbert, the Office of the Utah State Auditor has performed a review of
the Utah Department of Transportation’s (UDOT) evaluation and selection process, technology
usage, and human resource management related to UDOT’s I-15 Utah County Corridor Expansion
(CORE) project. Below is a description of the CORE evaluation and selection process, followed by
the procedures we performed and a summary of our findings. The detailed findings and
recommendations can be found where indicated in the table of contents.
I-15 CORE PROJECT EVALUATION AND SELECTION PROCESS
UDOT elected to procure a design-builder for the CORE project to design and construct the best
value project possible within the $1.725 billion budget. This was accomplished through a best value
design-build procurement approach, which gives the design-builder a fixed price and encourages
them to propose innovative and creative solutions for achieving the goals of the project. The contract
is awarded to the proposer offering the best qualitative proposal for the established price. This
approach is relatively new and differs from the traditional best-price procurement approach where a
low bid is the typical goal. The CORE project is the first project of this kind for UDOT and the
biggest project of this kind in the country.
In April 2009, UDOT initiated the procurement process to select a design-builder for the CORE
project and in June 2009, issued a Request for Proposal (RFP) for the project. UDOT then
conducted weekly confidential one-on-one meetings with the proposers to answer questions related to
the requirements of the RFP and to discuss technical concepts. On November 19, 2009, the
proposers delivered their proposals to UDOT. Proposals were received from three different
proposers: Flatiron/Skanska/Zachry Joint Venture, Provo River Constructors, and Timpanogos
Transportation Constructors. Once the proposals were received, the evaluation and selection process
began, which is the focus of our review. On December 9, 2009, the Selection Official selected Provo
River Constructors as the winning proposer of the CORE project.
The flowchart on the following page gives an overview of the Evaluation and Selection Process.
i
I‐15 CORE PROJECT
EVALUATION AND SELECTION PROCESS OVERVIEW
REQUEST FOR PROPOSAL (RFP) EVALUATION AND SELECTION
1. Outlined project goals and values. MANUAL
1. Defined the Adjectival Ratings
2. Divided the proposal into four main categories. 1) Project Definition
and set numeric scoring ranges
(Scope/Pavement & Bridges); 2) Maintenance of Traffic; 3) Schedule;
within each rating: Exceptional
4) Pass/Fail
(85‐100); Very Good (70‐84);
3. Defined the specific criteria and factors to be considered in the evaluation and Good (55‐69); Acceptable (< 54).
selection process. 2. Detailed written instructions for
4. Determined the significance of each specific criteria and combined them into the evaluation and selection
groups (High, Medium, Low) within the first three categories according to their process.
importance to the overall project. These are referred to as combined groups of
3. Defined roles of various teams,
criteria in the findings. Pass/fail category was only rated as “pass” or “fail.”
committees, etc.
TECHNICAL EVALUATION TEAMS
(Approximately 65 members)
Team members were assigned to categories based on their expertise.

PROJECT MAINTENANCE SCHEDULE PASS/FAIL
DEFINITION OF TRAFFIC
1. Evaluated the technical elements of the proposal. Rated whether

the proposer
2. Compared each proposal to the RFP. Assessed each
proposer’s strengths and weaknesses in meeting the met the
specific criteria of the project. applicable EVALUATION
requirements. OVERSIGHT
ADVISORS 3. Recommended an adjectival rating for each combined
(3 people) group of criteria based on how well the proposer met TEAM
(3 people)
the qualifications for that criteria.

Provided 4. Completed forms in Appendices of Manual. Provided an
guidance at
5. Made presentation to Selection Recommendation independent
all levels of
Committee. observation
the
6. Wrote up the Proposal Rating and Ranking Summary of the
evaluation
Reports. evaluation
and
and selection
selection
process by
process to
comparing it
help
to the
ensure the
SELECTION RECOMMENDATION COMMITTEE (SRC) Evaluation
process
(7 members) and Selection
was
1. Reviewed information presented by the Technical Evaluation Teams and evaluated Manual and
followed.
the proposals in their entirety. the

Instructions
2. Discussed each point of the proposals, obtained a consensus.
to Proposers.
3. Determined the final adjectival ratings for combined groups of criteria.
4. Assigned numeric scores to each of the adjectival ratings within the predetermined
ranges.
5. Summarized results and presented final recommendation to the Selection Official.
SELECTION OFFICIAL
(Deputy Director of UDOT)
Made final determination of winning proposer. GOVERNMENT OFFICIALS
Observed the Selection Official
during the final stage of the
procurement process.
ii
PROCEDURES PERFORMED AND SUMMARY OF FINDINGS
We performed the following procedures for the period related to the 2009 CORE evaluation and
selection process:
1. We reviewed the procedures related to the evaluation and selection of the CORE contract for
adequacy and compliance with UDOT’s policies and procedures. Our testwork also included
comparing the CORE process to the bid evaluation and selection processes used for other
projects, interviewing employees and others involved in the CORE process, and reviewing and
testing the adjectival ratings and numeric scoring summaries on a sample basis for accuracy and
supporting documentation.
 We identified various weaknesses in the CORE evaluation and selection process related to:
a. Blinding as reported in Finding No. 1.
b. Scoring as reported in Finding Nos. 2 and 3. Because of subjectivity and lack of

documentation, we were unable to conclude regarding the actual legitimacy of each
particular score or determine how the Selection Recommendation Committee (SRC)
arrived at the scores, other than they fit into pre-determined ranges. We also identified
factors such as scope restriction, condensed evaluation criteria, and inadequately
defined ratings that might have contributed to the proximity of the scoring results
and/or influenced the results.
c. Roles, responsibilities, and reporting as reported in Finding Nos. 4 through 6.
Despite the identified weaknesses with the process, all evaluators we interviewed indicated
that they agreed with the decision to award the contract to Provo River Constructors and
that they did not experience undue influence in their decisions.
 Initially we heard allegations that UDOT changed scores of at least one proposer during the
evaluation and selection process, implicating improprieties in their process. However, we
found no evidence that scores were changed. Although we did find that some adjectival
ratings recommended by the technical evaluation teams were changed by the SRC during
their part of the evaluation and selection process, the changes appeared reasonable and are
in compliance with the Evaluation and Selection Manual as noted in Finding No. 2a.
 UDOT followed the CORE evaluation and selection process in accordance with the CORE
Evaluation and Selection Manual and the Instructions to Proposers. However, although the
Evaluation and Selection Manual is a documented process, it is not part of UDOT’s
approved policies and procedures. We recommend that UDOT incorporate its new
documented process for Best Value Design-Build projects into its Policies and Procedures
Manual (see Finding No. 7).
2. We reviewed access to the Project Development Business System (PDBS) and the CORE Portal
to determine if any unauthorized access to the information occurred in either system. The PDBS
was used to record the price allocations of each CORE proposal, and the CORE Portal is the
computer system used to accumulate and store CORE proposal and evaluation information.
iii
a) The Project Development Business System
 A weakness in PDBS resulted in a breach of confidentiality and information was

improperly disclosed by a UDOT employee to a proposer resulting in improper
disclosure of information as addressed in Finding No. 8.
b) The CORE Portal
Based on our review of two areas within the CORE Portal computer system, we found the
following:
 Within the “Procurement Portal,” which allows close collaboration between internal
and external project teams, access is closely controlled by limiting or granting access to
individuals for specific pages, directories, and/or documents on the site.
 Within the “Evaluation Portal,” which is a separate security-controlled portal for the
proposal evaluation and selection process set up for exclusive use by the evaluation
teams, individual access was approved by project leadership and included only those
persons with a direct role in the evaluation and selection process. These access controls
appeared reasonable to ensure that only appropriate personnel had access to the
evaluation site. We were unable to determine who had actual access during the CORE
evaluation and selection process because security tables are only a “snapshot” of
current user access at a point in time and past logs have been overwritten with recent
logs. Maintaining only current logs is a normal business practice and is considered
reasonable. We were, however, able to obtain a list of persons who had been given
access during the evaluation and selection process (based on client representations) and
determined persons on the list to be reasonable.
3. We reviewed the adequacy of UDOT’s policies and procedures over bid protests and settlements
and the applicability of the Utah State Settlement Act, and we performed other related
procedures as considered necessary. The scope of our review did not include determining
whether it was appropriate for UDOT to agree to a settlement.
 We found that the applicability of the Utah State Settlement Act (the Settlement Act) to
UDOT’s settling of its bid protest was unclear. UDOT settled the bid protest under the
Utah Procurement Code which gives the chief procurement officer of the agency the
authority to settle and resolve a bid or award protest prior to the filing of a lawsuit. Based
on the authority granted in the Procurement Code, UDOT did not consider the Settlement
Act applicable to the bid protest settlement. However, we believe that the Settlement Act
may have been applicable. No formal opinion by the Attorney General’s Office has been
requested to make a determination since the issue is moot based on 2011 HB 34 substitute
which was passed, resolving the issue.
 We determined that UDOT did not obtain complete and adequate documentation to support
the settlement amount as reported in Finding No. 9. We also determined that UDOT does
not have formal policies and procedures for bid protests and any resulting settlements as
reported in Finding No. 10.
iv
4. We reviewed UDOT’s policies and procedures regarding confidentiality agreements and conflict
of interest statements and tested the agreements/statements to determine if they were completed
and any identified potential conflicts were adequately addressed by UDOT management. We
also investigated specific conflict of interest/confidentiality situations as considered necessary.
 UDOT’s policies and procedures over conflict of interest and confidentiality issues were
inadequate. We sampled conflict of interest and confidentiality agreements for selected
employees associated with the CORE project and found all agreements sampled were
signed. However, we noted other problems associated with conflict of interest/
confidentiality issues, such as incomplete and inconsistent policies and procedures,
inadequate documentation for contracting decisions, and inadequate handling of an
identified conflict, as discussed in Finding Nos. 11, 12, and 13.
5. We reviewed the stipends paid to the unsuccessful CORE proposers to determine their purpose,
propriety, and compliance with State law and UDOT policies and procedures.
 Stipends are used to compensate unsuccessful proposers and if accepted by them, entitle
UDOT to use the proposer’s ideas. We found that the CORE stipends were reasonable,
properly paid and budgeted, and were in compliance with state law and UDOT policies and
procedures.
Our procedures were more limited than would be necessary to express an audit opinion on
compliance or on the effectiveness of UDOT’s internal control or any part thereof. Accordingly, we
do not express such opinions. Alternatively, we have identified the procedures we performed and the
findings resulting from those procedures. Had we performed additional procedures or had we made
an audit of the effectiveness of UDOT’s internal control, other matters might have come to our
attention that would have been reported to you.
Our findings resulting from the above procedures are included in the attached findings and
recommendations section of this report.
This report is intended solely for the information and use of the Governor and UDOT and is not
intended to be and should not be used by anyone other than this specified party. However, the report
is a matter of public record and its distribution is not limited.
By its nature, this report focuses on exceptions, weaknesses, and problems. This focus should not be
understood to mean there are not also various strengths and accomplishments. We appreciate the
courtesy and assistance extended to us by the personnel of UDOT during the course of the
engagement, and we look forward to a continuing professional relationship. If you have any
questions, please call Debbie Empey, Audit Director, at (801) 538-1342.
Sincerely,
Auston G. Johnson, CPA

Utah State Auditor
cc: Carlos Braceras, UDOT Deputy Director

Derek Miller, Chief of Staff, Governor’s Office
v
TABLE OF CONTENTS
Page
MANAGEMENT LETTER:
I-15 CORE PROJECT EVALUATION AND SELECTION PROCESS i
FLOWCHART SHOWING OVERVIEW OF SELECTION PROCESS ii
PROCEDURES PERFORMED AND SUMMARY OF FINDINGS iii
FINDINGS AND RECOMMENDATIONS:
EVALUATION AND SELECTION PROCESS:

1. EVALUATION PROCESS NOT BLINDED 1
2. ISSUES WITH SCORING BY THE SELECTION RECOMMENDATION COMMITTEE (SRC):
a. Lack of Proper Documentation for the SRC’s Conclusions and Recommendations 2
b. Risk of Inconsistent/Misleading Information Given to Proposers 3
c. Concerns Over Extended Warranty 5
d. Sensitivity Analysis Not Part of Policies and Procedures 7
3. POSSIBLE CONTRIBUTORS TO CLOSE SCORES 8
4. EVALUATION OVERSIGHT TEAM ROLES AND RESPONSIBILITIES
NOT ADEQUATELY DEFINED 12
5. GOVERNMENT OFFICIALS ROLES AND RESPONSIBILITIES NOT
ADEQUATELY DEFINED 13
6. UNNECESSARY PRELIMINARY MEETING 15
7. WEAKNESSES IN THE NEW BEST VALUE DESIGN-BUILD PROCESS 15
8. INADEQUATE SECURITY AND PROTECTION OVER BID INFORMATION 17
SETTLEMENT:
9. INCOMPLETE AND INADEQUATE DOCUMENTATION TO SUPPORT SETTLEMENT AMOUNT 18
10. LACK OF POLICIES AND PROCEDURES OVER BID PROTESTS 21
CONFLICTS OF INTEREST:
11. INADEQUATE CONFLICT OF INTEREST/CONFIDENTIALITY POLICIES AND PROCEDURES 22
12. LACK OF CONTRACT DECISION DOCUMENTATION AND POTENTIAL
CONFLICT OF INTEREST 24
13. INAPPROPRIATE HANDLING OF CONFLICT OF INTEREST 25
AUDIT RESPONSE LETTER Attachment 1

FINDINGS AND RECOMMENDATIONS
EVALUATION AND SELECTION PROCESS:
1. EVALUATION PROCESS NOT BLINDED
The Utah Department of Transportation (UDOT) did not “blind” (conceal the identity of)
the contractors submitting the proposals and, as a result, there is the possibility that the
proposals might not have been evaluated objectively. It was necessary for the Technical
Evaluators to know the identity of the proposers since exchange of information was
necessary between those parties during the technical portions of the process. However,
once the Technical Evaluators formulated their recommendations, blinding should have
occurred throughout the rest of the evaluation process. It was also necessary for both the
Project Director and the Deputy Project Director to know the identity of the contractors
because they were heavily involved in the entire procurement process. Therefore, they
should not have been members of the Selection Recommendation Committee (SRC) so that
the proposals could have been blinded from the SRC.
Blinding helps to ensure that proposals are reviewed objectively and that the possibility of
bias, whether real or perceived, is avoided. UDOT uses blinding on other projects as
evidenced by other UDOT projects we reviewed. Using blinding on the CORE project
would have prevented the appearance of bias in the awarding of the contract and might have
prevented the protest and eventual settlement.
Recommendation:
We recommend that UDOT include blinding in evaluation and selection procedures,

wherever possible.
UDOT’S Response to Finding 1:
 Regarding the audit finding that “Blinding . . . might have prevented the protest and
eventual settlement,” UDOT does not believe that this is supported nor that blinding is
likely to have prevented the protest or to have impacted the settlement, which was
reached for pragmatic reasons regarding moving the project forward and preventing
greater costs to the public. UDOT agrees that blinding could have “prevented the
appearance of bias in awarding the contract;” however, UDOT disagrees with the
emphasis the audit has placed on that single element of the process.
 Blinding of proposals is typically beneficial, and historically UDOT has used it to
isolate cost considerations from technical evaluations on more typical best-value
selections. On the I-15 CORE project, the cost was fixed and only the technical
elements evaluated, so blinding at the SRC level was not comparable to other projects.
1
UDOT’S Response to Recommendation 1: UDOT agrees with the recommendation as a

general principle but does not believe blinding should necessarily override all other
considerations. However, since the I-15 CORE procurement process, UDOT developed and
implemented a Best Value Design-Build Selection Manual of Instruction that requires
blinding on projects. In the event there is a situation in which blinding would not be
advisable, UDOT senior leadership will carefully consider the options before making a
decision.
2. ISSUES WITH SCORING BY THE SELECTION RECOMMENDATION

COMMITTEE (SRC)
Refer to the flowchart on page ii for a summary of the scoring process.
a. Lack of Proper Documentation for the SRC's Conclusions and Recommendations
Per the Evaluation and Selection Manual, the SRC was responsible for making the final
determination of the adjectival ratings and assigning corresponding numeric scores to
each of the combined groups of criteria; however, they did not provide adequate
documentation to sufficiently support the adjectival ratings and scoring. While there
were some factors like highway miles paved, number of structures replaced, etc. which
could be measured quantitatively, other factors like phasing or design strategies were
more inherently subjective and could not be measured quantitatively. Because of the
inherent subjectivity, some conclusions might have been based on personal preference,
which is a part of any evaluation process. For instance, one evaluator might value
project completion in entirety over project efficiency or one design or traffic flow
pattern over another. Regardless of the nature of the subjectivity, we felt that the
conclusions or recommendations of the SRC weren’t always documented completely
and thoroughly in the Final Recommendation Report. Although they did specify
scoring considerations, they did not document the details as to how they arrived at the
score, but rather included only a broad overall summary for each proposer. Therefore,
it was not obvious what criteria or strength(s) they valued over another. Another factor
contributing to this weakness was that only general guidelines were provided in the
Evaluation and Selection Manual regarding how scores were to be awarded (as reported
in Finding No. 3.c.), which allowed for additional subjectivity in the scoring process.
As a result of subjectivity and inadequate documentation, we were unable to evaluate
the specific scores given, but we were able to determine that the scores given were
reasonable and applied within numeric ranges assigned to the adjectival ratings.
In addition, there were allegations made that scores were changed in order to award the
contract to Provo River Constructors. For 3 of the 27 combined groups of criteria, the
SRC changed the adjectival ratings to be one level higher than that of the Technical
2
Evaluators. The SRC documented that they disagreed with some of the adjectival
ratings of the Technical Evaluation Teams but did not document the reasons and/or the
significant strengths that supported their change in ratings. However, since the SRC
was responsible for determining the final adjectival rating and the Technical Evaluators
were only responsible for recommending adjectival ratings to the SRC, the changes to
the adjectival ratings were made within the purview of the SRC and were found to be
reasonable.
Because subjectivity is inherent in any evaluation and selection process of this nature, it
is critical that the SRC adequately document their conclusions and recommendations.
Recommendation:
We recommend that UDOT provide adequate documentation to support

adjectival ratings and scoring.
UDOT’S Response to Finding 2a:
 Regarding the audit finding that the SRC “did not provide adequate
documentation to sufficiently support the adjectival ratings and scoring,” UDOT
responds that the SRC relied on the work of the ETs (Technical Evaluation
Teams), which provided detailed information and analysis of the proposals. The
ETs’ reports, developed in communication with the SRC, comprise the
foundational documentation for the SRC recommendation.
 There are advantages and disadvantages to providing the type of detail
recommended in the audit regarding scoring and documentation. UDOT notes the
“general guidelines” and ratings process employed on the I-15 CORE project are
consistent with national best practices, that they served their function in
identifying the best-value proposal, and that the audit finding concluded that the
scoring was both “reasonable” and applied as specified.
UDOT’s Response to Recommendation 2a: UDOT believes the documentation was

adequate, it was consistent with existing best practices, and it met its intended purpose.
However, UDOT will add language to the Best Value Design-Build Selection Manual
of Instruction to emphasize the importance of adequate documentation with respect to
adjectival ratings and numeric scoring.
b. Risk of Inconsistent/Misleading Information Given to Proposers
UDOT might have given instruction in one-on-one meetings that was inconsistent with
scoring by the SRC. Provo River Constructors (Provo River) were given scoring value
for lane-extension miles beyond (south of) Spanish Fork; however, in the one-on-one
3
meeting minutes kept by UDOT, Flatiron/Skanska/Zachry Joint Venture (Flatiron Joint

Venture) were instructed to stay within the “limits” identified in the Instructions to
Proposers, which were “lengthening the Project to extend as far as Spanish Fork Main
Street.” This instruction seems to imply that the proposers should not include designs
extending beyond Spanish Fork, yet the scoring evidences that the SRC positively
valued the extension proposed by Provo River. Therefore, Flatiron Joint Venture’s
statement in their protest that that they were misled when advised by UDOT that “lane
extension south of Spanish Fork had ‘no value’” seems plausible given the
inconsistency in the UDOT meeting minutes and the final scoring.
This inconsistency or misunderstanding between UDOT and the proposers could have
occurred because both UDOT and the proposers kept their own sets of minutes for one-
on-one meetings and no attempt was made to ensure that both parties agreed with what
was documented in both minutes. Only one set of minutes should have been kept, and
the minutes should have been approved at the next meeting as is customary. In
addition, the approved minutes should have been reviewed to ensure that instructions
given to each proposer were consistent and properly followed in the scoring process.
One approved set of minutes would help to prevent disagreements and
misunderstandings.
Recommendation:
We recommend that UDOT provide instruction and/or answers given in one-on-

one meetings that are consistent with the Instructions to Proposers and the
Evaluation and Selection Manual and do not provide misleading or inconsistent
information to proposers. We also recommend that UDOT keep only one set of
minutes for one-on-one meetings and ensure they are approved by both parties,
are consistent with instruction given to other proposers, and are properly followed
in the scoring process.
UDOT’S Response to Finding 2.b:
The audit states that Provo River Constructors (PRC) was “given scoring value for
lane extension miles beyond (south of) Spanish Fork” but omits why. PRC was given
credit for the lane miles as a transition into the CORE reconstruction project, not for
the reconstruction itself. The extension was assessed as a “minor strength” as a
transition that improved traffic flow into the project.
The I-15 CORE team made a concerted effort to provide consistent information to all
proposers regarding the RFP. UDOT does not believe Flatiron/Skanska/Zachry Joint
Venture (FSZ) was “misled” in discussions of the southern terminus of the project or
that FSZ provided sufficient evidence to support such an assertion. Certainly there was
4
no intent to misdirect any of the proposers. Meeting minutes taken by UDOT reflect a
different interpretation of the discussions.
UDOT accepts that FSZ may have misinterpreted some portion of the RFP or had
different perceptions of the discussions that took place in the one-on-one meetings. At
the most, different perceptions result from an ambiguity in the RFP documents, which
is a common issue with performance-based design-build requirements. UDOT has
implemented numerous processes to prevent this issue from occurring. UDOT accepts
that more specificity should have been provided regarding the limits of the full
reconstruction project versus transitions into those project limits.
UDOT agrees that ideally only one set of meeting minutes for each proposer should be
kept and agreed upon, which was initially implemented in the one-on-one discussions
to ensure consistency; however, they were abused by at least one proposer to try to
elevate them to contractual documents and to re-characterize the communications. This
led to extended disputes during the meetings over exact wording of the minutes rather
than substantive discussions regarding what information was exchanged. This
significantly impaired the usefulness of the discussion process. The next-best option
was deemed to be separate meeting minutes, which was then adopted in order to
prevent such abuse.
UDOT’s Response to Recommendation 2b: UDOT agrees with the recommendation

and asserts that it did provide instruction consistent with the ITP. As to the meeting
minutes, UDOT agrees in theory but notes that other considerations may affect the
practice. UDOT will include standardized protocol for the one-on-one meetings in the
Best Value Design-Build Selection Manual of Instruction.
c. Concerns Over Extended Warranty
The SRC might have been improperly influenced by a $49.3 million extended warranty
included in the Provo River proposal. The warranty was to provide extended
maintenance on the new structures, road surfaces, etc. built during the CORE project.
For purposes other than award selection, price allocations (or bids) were submitted as
part of the proposal. However, these amounts were not to be known or considered in
the evaluation and selection process since this was a best value design-build project
with a fixed price. But Provo River noted the amount of the warranty in their proposal;
therefore, the SRC knew the amount and, during deliberations to determine ratings and
scoring of the proposals, the SRC discussed the possibility of UDOT not exercising the
warranty, which would ultimately save UDOT $49.3 million. This, in essence, would
make the project no longer fixed price, but rather low bid.
We question the propriety of discussing the possible cost savings associated with not
exercising the warranty because the project was supposed to be evaluated based on the
5
goals and values listed in the Instructions to Proposers. Members of the SRC
represented that the warranty was evaluated and valued based only on its merits to the
project’s goals and values and that its potential cost savings did not influence the award
decision. Also, no decision was made as to whether to exercise the warranty or not
during the evaluation and selection process. However, it appears that UDOT decided to
not exercise this warranty early in the contract process because a change-order to
remove the warranty from the contract was made almost immediately after UDOT
issued a notice to Provo River to proceed with the contract, thus saving UDOT $49.3
million.
Therefore, associated cost savings might have improperly influenced the SRC in the
evaluation and selection process, and consequently, might have had some influence on
the decision to settle the bid protest with Flatiron Joint Venture since UDOT had these
unallocated resources.
Recommendation:
We recommend that the SRC limit its conversations to the project’s goals and
values as identified in the Instructions to Proposers while conducting deliberations
of adjectival ratings and scoring.
UDOT’S Response to Finding 2.c:
 Regarding the extended warranty, UDOT notes the selection documentation shows
that the SRC compared the extended warranty against the Project Goals and
values and assessed it as a “good value” in a low-significance category (which
UDOT upholds was the appropriate consideration).
 UDOT notes the extended warranty was unanticipated and that for a proposal to
identify a price associated with a value-added item was unusual. UDOT also notes
that despite the SRC’s access to this unanticipated information, it did not affect the
SRC’s recommendation. The audit states the cost saving might have influenced the
SRC. UDOT understands this, but states that it did not influence the decision-
making process.
 The audit is correct that the I-15 CORE project team moved soon after contract
execution to develop a change order to remove the extended warranty and recover
the associated costs. Prior experience indicates that it is difficult to capture the
full value of extended warranties. The timing of the change order was driven by
bonding costs, which are a direct pass-through to UDOT. Bonds are required
prior to starting work and are determined as a percentage of the contract amount.
By reducing the contract amount immediately, UDOT avoided having to reimburse
PRC almost $400,000 in unnecessary bonding costs.
6
UDOT’s Response to Recommendation 2c: UDOT agrees in principle with the

recommendation and believes the CORE evaluation process was consistent with that
principle. It will be expressly stated in the Best Value Design-Build Selection Manual
of Instruction; however, UDOT notes the conversation to which the audit refers was
regarding an unanticipated value-added element initiated by the proposer, and any
process established needs to allow for exploring the value and implications of all
proposal elements.
d. Sensitivity Analysis Not Part of Policies and Procedures
The SRC performed a sensitivity analysis which was not included in the Evaluation and
Selection Manual. After adjectival ratings were determined by the SRC, the SRC gave
numeric scores based on those ratings. Those scores were then weighted based on pre-
determined significance and the total weighted scores were used to determine who was
awarded the contract. The result was a one point difference between the top two
proposers. The sensitivity analysis was a process used to help the SRC determine the
effect and magnitude of scoring adjustments on the outcome of the award by
determining if it would take major or minor adjustments in the scores to get a different
result. Based on representations from members of the SRC, the scoring analysis was
not done to adjust any scores but instead to help them feel comfortable with the one
point difference in the scoring results. We found no documentation of the sensitivity
analysis and no evidence that scores were adjusted based on that analysis. However, if
this sensitivity analysis is performed, it should be included in the policies and
procedures to document the what, why, and how of the analysis to help ensure propriety
and compliance.
Recommendation:
We recommend that UDOT include the sensitivity analysis in its policies and
procedures if its use is continued.
UDOT’s Response to Finding 2.d:
The sensitivity analysis exercise was not intended to, nor did it, result in modifications
to the scoring.
UDOT’s Response to Recommendation 2d: UDOT will not use sensitivity analysis
during the evaluation of Design-Build proposals. Therefore, sensitivity analysis will not
be included in UDOT’s current policies, procedures, or manuals.
7
3. POSSIBLE CONTRIBUTORS TO CLOSE SCORES
Three possible factors that might have contributed to the proximity of the scoring results
are: a) The scope was too restricted; b) The evaluation criteria were too condensed; and c)
The adjectival ratings were not clearly defined to differentiate between the various ratings.
a. Scope Too Restricted
The scope (length of the project) appeared to be too confined in that the “Ultimate
Infrastructure Configuration” (the expected section of freeway to be completed) began
at the American Fork Main Street interchange and ended at the Provo Center Street
interchange. If the proposer was able to go beyond Provo Center Street, the
Instructions to Proposers appeared to provide a southern limit by using the words “as
far as Spanish Fork Main Street.” As indicated in Finding No. 2.b. above, UDOT
instructed the parties to stay within the “limits” of the Instructions to Proposers. Thus,
it appears there was a southern boundary. UDOT indicated to us that they were
surprised that all three proposers were able to meet the Ultimate Infrastructure
Configuration and that two were able to go beyond that to Spanish Fork. UDOT
explained that the two proposers were able to go beyond their expectations because of
the unexpected decline in the economy that occurred after the Instructions to Proposers
were finalized. Various individuals suggested, in hindsight, that the scope should have
been larger. The fact that two proposers were able to go beyond Provo Center Street to
Spanish Fork is an indication that the scope was too restricted. The idea of a best value
design-build project is to get the most value for a fixed price. In theory, the
competition between the proposers should motivate them to try to give more in order to
win the award. It is possible that the defined scope decreased the ability to get the best
value. Other indicators that the scope was too restricted are that one proposer was able
to propose HOV lanes, impressive interchanges, and an off-highway road
improvement, while another proposer was able to offer an extended maintenance
warranty of $49.3 million. One person interviewed suspected that the proposer added
the warranty because after calculating their price for meeting the Ultimate
Infrastructure Configuration they still had money left. If the scope had been expanded,
the evaluation process might have ended in clearer results and been less subject to
criticism because differentiations between proposals might have been clearer.
Additionally, an expanded scope might have allowed the proposers to offer more
project length, allowing UDOT the opportunity to evaluate the value of a lengthened
project over other values such as HOV lanes, impressive interchanges, etc.
8
b. Evaluation Criteria Too Condensed
Organizing the evaluation criteria into only three main categories (Project Definition,
Maintenance of Traffic, and Schedule), and then under each of those categories
combining together criteria based on relative significance or importance (High,
Medium, Low), may not have been broad enough to accurately differentiate the
strengths of the proposers. We reviewed the evaluation and selection process of two
other projects and found that the criteria for those projects were organized into twelve
different categories, such as technical solutions; lighting, traffic signals, signing, etc.;
pavement and geotechnical; roadway geometrics; etc. These expanded categories
might allow for a more detailed evaluation process where ratings and points can be
given for more specific criteria than that possible when all criteria is condensed and
combined. The expanded criteria categories would more easily identify differences
between proposers and allow a more obvious determination of the values the SRC
placed on those criteria, possibly resulting in more definite scoring differentiations.
c. Adjectival Ratings Not Clearly Defined
The adjectival ratings assigned to the specific criteria groups were not clearly defined
to easily differentiate between different ratings. UDOT defined the ratings in terms of
strengths and weaknesses. For example, an “exceptional” rating was primarily defined
as having significant strengths that significantly outweighed weaknesses, and a “very
good” rating was defined as having strengths that outweighed weaknesses. “Significant
strengths” were defined as having “a considerable positive influence on the Proposer’s
ability to advance the Project goals and values, and exceed requirements.” “Strengths”
were defined as benefits to the Project which are “expected to increase the Proposer’s
ability to advance the Project goals and values, and exceed requirements.” As can be
seen, the difference between the two ratings is not clear and is highly subjective. To
further complicate the issue, both definitions of these ratings included “Most
requirements are met or exceeded….” The differences between the two categories
seem to be minor differentiations in weaknesses. These definitions did not seem to
allow the SRC to adequately differentiate between proposals as is evident in that all
three proposals were given “exceptional” ratings in the Project Definition “High”
group. Documentation indicated that all three proposals met or exceeded project goals.
This issue is further complicated by the limitations in scope as addressed above. It
seems that an easy way to differentiate between the two ratings would have been to
define “exceptional” as exceeding requirements versus “very good” as meeting
requirements. If this differentiation was made, all three teams would not have received
exceptional ratings because only two teams exceeded the requirements.
UDOT should consider whether changes to these areas would be beneficial for future
projects to broaden scoring results and help alleviate protests caused by close scoring.
9
Recommendation:
We recommend that UDOT consider the effects of limited scope, condensed criteria,
and unclear adjectival rating definitions on the evaluation and selection process and
take necessary measures to help allow for more definite scoring results.
UDOT’s Response to Finding 3:
a. Scope too restricted.
 UDOT does not believe the scope was too restricted. Those assertions follow a
statement by the audit that links best-value on a design-build project to only one
evaluation element: scope. However, UDOT procured and contracted for three
elements (evaluation criteria): amount of scope, speed of delivery, and minimal
impacts to the public during construction. Scope constituted 60 percent of the
equation, with 40 percent allotted to the other two criteria.
o Each of the three categories requires considerable resources to deliver. The
criteria were established so the successful proposer would apply its
resources optimally to provide the greatest balance of the most scope,
shortest schedule, and least impacts to the public.
o The three criteria and relative weights were consistent with the Project
Goals and were established based on prior UDOT experience, public input,
and the specific characteristics of the proposed road.
o Scope was not evaluated based solely on how many lane miles were
proposed, but upon the level of service to be provided at the year 2030, or
the usefulness of the scope proposed. Minimal service level required was
“D,” with higher service levels recognized as adding value. It is important to
recognize that the most scope offered is not necessarily equal to the most
valued scope.
 UDOT does not believe that the scope was too restricted. Although two of the
proposers were able to deliver almost all of what was desired in scope, UDOT
believes that it ultimately estimated the scope just right. What was valued was useful
scope that would improve the movement of traffic throughout the corridor, which
included considerations such as increased regional mobility, the functionality and
types of interchanges on and adjacent to the corridor, and the future capacity of
cross-streets and arterials.
 To the south, the RFP stated that full functionality was valued as far as Spanish
Fork Main Street but did not specify the ending point for the transition. UDOT
stated it valued a smooth transition to a logical terminus, which was to be defined by
the proposer (this is the source of FSZ’s mistaken assumption regarding the
southern limit).
10
b. Evaluation criteria too condensed.
 UDOT’s experience indicates that while close proposals make the evaluation
process more difficult, (and more open to challenges), when bids are close the
project is typically well-defined in the ITP or plan set, the risk allocation is clear,
and other expectations are well communicated. A well-defined project combined
with stiff competition from capable proposers yields bids that are likely to be close
and at the same time provide the greatest value.
 Focusing on critical criteria amplifies the substantive differences rather than
compressing them. The evaluation and scoring for the I-15 CORE project focused on
the project elements that had the greatest value and the highest likelihood of
showing meaningful and significant differences.
 The narrow evaluation categories were derived from past lessons learned, and
UDOT intends to implement it similarly on future design-build projects (it has been
adopted by the industry and is being considered as a best practice nationwide).
 The audit suggestion of increasing the categories, especially by minor
considerations, would generally compress the numbers and could skew them away
from material considerations for the mundane ones. Extraneous and lesser criteria
were intentionally left out in order to focus on the true differentiators between the
proposals. This change was a direct response to prior UDOT best-value selections
in which values assigned to items such as lighting, signals, and signing flattened and
compressed scores between proposers and provided limited opportunity for
proposing teams to differentiate themselves from the competition. Such requirements
typically cause the proposal teams to simply restate the project requirements and
then commit to meet those requirements.
c. Adjectival ratings not clearly defined.
 UDOT concurs that some additional language in the Evaluation and Selection
Manual would be helpful in describing the assignment of adjectival ratings. UDOT
notes, however, the existing language was carefully considered and is consistent
with national best practices, which address issues such as balancing the advantages
and disadvantages of specificity, flexibility, and defensibility. Enough flexibility
needs to be built into the process to properly evaluate the innovative solutions of
proposers. Language that is too restrictive may limit the evaluation process and
prevent award to the team with the best solutions and value to the public.
UDOT’s Response to Recommendation 3: UDOT’s Best-Value Design-Build Selection

Manual of Instruction addressed the process to develop appropriate scope, criteria, and
evaluation definitions to design a selection process to best achieve the Project Goals.
11
4. EVALUATION OVERSIGHT TEAM ROLES AND RESPONSIBILITIES NOT

ADEQUATELY DEFINED
The Evaluation and Selection Manual (the Manual) was inadequate in providing instructions
to the Evaluation Oversight Team to help meet expectations of their intended role. The
Manual only stated that the Evaluation Oversight Team was responsible for providing “an
independent observation of the evaluation and selection process” in order to help ensure
compliance with the Evaluation and Selection Manual and the Instructions to Proposers.
The Manual did not specify whether the entire team or an individual was to observe the
evaluation and selection process in its entirety or if only specific procedures had to be
observed as determined necessary by individual team members during the process. In
addition, the Manual did not specify the written documentation to be generated by this team.
Members of the Evaluation Oversight Team provided written documentation indicating that
they observed the selection and that UDOT complied with the Evaluation and Selection
Manual and the Instructions to Proposers. However, in our review of the process, we found
that the evaluation and selection process was not observed in its entirety by the complete
team. The Evaluation Oversight Team consisted of an assistant attorney general for UDOT,
UDOT’s Engineering Services Director, and a federal highway administrator. The Assistant
Attorney General did not observe the technical evaluation procedures nor did she attend all
meetings of the SRC where deliberations and scoring were done. She indicated she did not
observe the technical evaluations because she felt she did not have the expertise necessary
to do so; however, the observation responsibilities did not require technical expertise. She
did not attend all meetings of the SRC because of scheduling conflicts. We feel her
attendance at these meetings, especially those where deliberating and scoring were done,
was important because these were the meetings with the most subjectivity applied and were
most at risk for noncompliance and/or public scrutiny. The oversight responsibilities would
require observation of the entire process to ensure compliance with the Evaluation and
Selection Manual and Instructions to Proposers. Therefore, it could be misleading for
UDOT to indicate that the evaluation and selection process was observed by three
individuals, when really only parts were observed by three individuals, and we question
whether the observation was adequately performed to meet the intended purpose of UDOT.
Recommendation:
We recommend that UDOT include adequate instruction regarding the role of the
Evaluation Oversight Teams in their manuals and policies.
12
 Regarding the audit statement “it could be misleading for UDOT to indicate that the
evaluation and selection process was observed by three individuals, when really only
parts were observed by three individuals,” no effort or intent was made to “mislead”
others to believe the oversight was all-inclusive. UDOT never stated that oversight
officials attended every hour of every meeting that took place in the selection process.
 Instead of constant monitoring, UDOT believed a sampling method was sufficient to
ensure the process was being administered in a fair manner. Throughout the evaluation
process, the independent observers attended meetings without prior notice and for
undeclared durations.
 Nationally, it is more typical to have only a single observer and process expert.
Historically, UDOT has found this to be sufficient and cost-effective. However, due to
the size and nature of the project and in an effort to raise the standard, UDOT
increased that number on the I-15 CORE project to three observers and three process
experts.
 UDOT concurs that additional guidance and instruction could be provided for future
oversight team members.
UDOT’s Response to Recommendation 4: UDOT agrees with this recommendation. The

Best Value Design-Build Selection Manual of Instruction provides for a process witness at
both the evaluation and selection. The roles and responsibilities of the process witness are
more fully defined.
5. GOVERNMENT OFFICIALS ROLES AND RESPONSIBILITIES NOT

ADEQUATELY DEFINED
The Evaluation and Selection Manual was very general and vague in describing the
responsibilities of the Government Officials, which consisted of a legislative fiscal analyst,
a legislative auditor, and two transportation commissioners. The Manual states simply “The
Governmental Officials will observe and advise the Selection Official during the final stage
of the procurement process.” (italics added). However, all four of the Government Officials
indicated that their responsibilities were to evaluate the process, i.e. determine if UDOT
followed the process, determine if there were any problems with the process, evaluate the
fairness of the process, observe any biases, evaluate the way the proposals had been
reviewed, and determine if UDOT’s review of the proposals would justify the final decision.
In other words, they were essentially asked to validate the process. Also, the Selection
Official indicated that the Government Officials were to give a broad, high-level review of
the process to determine that the process made sense. The Manual also states that “once an
acceptable recommendation has been delivered, the SRC will give a presentation to the
Selection Official and Government Officials.” Asking the Government Officials their
13
opinion of the process and if they agreed with the SRC’s conclusion after the SRC had
already determined their recommendation of the winning proposer seems to be inherently
flawed since there could be a high risk of bias in the presentation because the presenters
could skew their presentation in favor of their selection.
Since the Manual is too vague for anyone to know what the role of the Government
Officials is intended to be and nothing was issued in writing from the Government Officials,
there is a high risk that the Government Officials’ role could be easily misunderstood and
give a false sense of comfort by giving the appearance that the Government Officials were
accomplishing more or something completely different than what they did; therefore, for
future projects the Manual should state the responsibilities of the Government Officials
clearly and precisely.
Recommendation:
We recommend that UDOT include adequate instruction regarding the responsibilities

of Government Officials in their manuals and policies.
 UDOT concurs with the audit that the roles of the government officials were “not
adequately defined” in the Evaluation Manual and notes the process intended was not
well described in the manual.
 UDOT appreciated the attendance of the governmental officials and their feedback as
the process neared its conclusion. We agree that with limited exposure to the selection
and evaluation process, limited instructions, and a largely undefined role, it would
have been unfair to ask them to provide a complete endorsement of the final selection.
Nonetheless, the process was beneficial as it allowed the governmental officials to see
and understand the complex and comprehensive procurement process and the intense
effort that was undertaken to ensure the public received the best value for this immense
public infrastructure project.
UDOT’s Response to Recommendation 5: UDOT agrees with this recommendation.

Because of the uniqueness of the I-15 CORE project, government officials were included in
the selection process to provide an additional level of input into the process. In the future,
UDOT does not intend to use government officials in the selection process for projects.
Consequently, the Best Value Design-Build Selection Manual of Instruction does not refer
to the term “Government Official.” Instead, this manual defines the membership of the
selection committee and the selection officer. If the Department undertakes another project
with the characteristics of the I-15 project and decides that including government officials
in the process would add value, the Department will provide better role definition.
14
6. UNNECESSARY PRELIMINARY MEETING
The SRC presented their results in both a preliminary meeting and the meeting with the
Government Officials; however, the preliminary meeting excluded the Government
Officials. As a result, it appears that the preliminary meeting may have been conducted in
order for the Selection Official to pre-approve the information to be presented to the
Government Officials, including their recommendation of the winning proposer. Although
the preliminary meeting was not as detailed as the meeting that included the Government
Officials, the meeting with the Government Officials does not seem to accomplish anything
except to give the appearance that they were an independent group overseeing the process,
again causing the Government Officials’ role to be misunderstood. Also, the meeting to
review the preliminary findings was included in a narrative portion of the Manual, but it was
not included in the flowchart of the selection process in the Manual. UDOT should
eliminate the unnecessary preliminary meeting and have only one meeting to present the
results to the Selection Official and the Government Officials for future projects.
Recommendation:
We recommend that UDOT eliminate the unnecessary preliminary meeting and have
only one meeting to present the results to the Selection Official and the Government
Officials for future projects.
The preliminary meeting was to ensure proper planning and preparation for the subsequent,
complete presentation to the SO and public officials. If used in the future, it will be
described in the Evaluation Manual.
UDOT’s Response to Recommendation 6: UDOT agrees with this recommendation.
7. WEAKNESSES IN THE NEW BEST VALUE DESIGN-BUILD PROCESS
Subsequent to the CORE project, UDOT completed an instruction manual for Best Value
Design-Build projects to give “direction on how selection processes occur” on these
projects. This new manual should be approved and incorporated into UDOT’s formal
policies and procedures to help ensure standard procedures are followed on all projects. In
reviewing UDOT’s new manual, we noted the following problems:
a. The roles and responsibilities of the Selection Official are not included in the new
manual.
15
b. The roles and responsibilities of the Process Witnesses (called the Evaluation Oversight
Team in the CORE project) are not adequate as included in the new manual. The
processes, meetings, and/or deliberations that the Process Witnesses should observe are
not included. The diagram shows the process in general for which each Process
Witness is responsible; however, the written instructions should specify their individual
responsibilities as well as their combined responsibilities. The reports or other written
documentation to be completed by the Process Witnesses are not specified. The
observations of the Process Witnesses need to be adequately documented to evidence
this control in the evaluation and selection process.
c. The terms used in the new manual are not clearly defined or consistently used. The
manual uses the rating terms of technical, categorical, and adjectival; however, it is
unclear whether these represent different ratings or are interchangeable. For example,
the manual says that the Selection Committee should understand the Evaluation
Committee’s technical rating, but then says they should approve the categorical rating.
The terms should be consistently used in the manual or adequately defined to avoid
confusion or misunderstanding.
d. The new manual does not include instructions and procedures for handling and
resolving differences of opinions between committees, or for documenting the
resolutions. As in the example above, the Selection Committee must understand and
approve the ratings of the Evaluation Committee. This could mean the two committees
negotiate until they agree or that one committee trumps the opinion of another.
Another example is that the Selection Committee takes their determination to the
Selection Official for “concurrence.” The procedure to be followed when
disagreements occur should be included in the new manual to help ensure that
disagreements are properly handled, resolved, and adequately documented, as
considered necessary.
e. Standard definitions and requirements that should not vary between projects are not
included in the new manual. For instance, common definitions for adjectival ratings
and applicable numeric scoring ranges within those ratings could be defined so that
they are used consistently between projects.
Incorporating the process for best value design-build projects, as well as other types of
construction projects, into UDOT’s Policies and Procedures Manual will help to ensure that
the procurement of contracts is proper and consistent.
Recommendation:
We recommend that UDOT incorporate the Best Value Design-Build Manual into its
Policies and Procedures Manual and revise it to include:
16
a. The roles and responsibilities of the Selection Official.
b. Clearly defined roles and responsibilities of the Process Witnesses.
c. Clearly defined and consistently used terms.
d. The resolution processes to be followed when differences of opinions occur

between committees and the documentation of such resolutions.
e. Standard definitions and requirements between projects.
We also recommend that UDOT establish set procedures for the procurement of other
types of construction projects that are not best value design-build projects and
incorporate them into their policies and procedures.
 The procurement of the I-15 CORE project, with only one other similar example to
learn from in the entire country, pioneered a new standard in the industry. UDOT
selected the fixed-price best-value approach to leverage ideal bidding conditions and
stretch the limits of the size and quality the project could reach. UDOT believes the
achieved value of twice as much project completed two years sooner with fewer traffic
delays than every estimate had shown, validates the process. UDOT, as is its practice,
learns from every project procurement and has already begun to implement changes
based both on the process and on input received from the audit.
 Consistent with UDOT practices, information derived from the I-15 CORE project
procurement was added to existing lessons learned and will be incorporated into the
current draft procedures manual for application in future procurements. The audit
comments will also be addressed.
UDOT’s Response to Recommendations for Finding 7: UDOT concurs with the

recommendations and will incorporate the recommendations into the Best Value Design-
Build Selection Manual of Instruction.
8. INADEQUATE SECURITY AND PROTECTION OVER BID INFORMATION
UDOT did not adequately secure and protect the bid information on their Project
Development Business System (PDBS) for the CORE project. The proposers were required
to submit bid amounts (in the form of price allocations) which were entered into PDBS for
purposes other than award selection. Although bid amounts were received, they were not to
be considered in the evaluation and selection process and/or the eventual award of the
17
contract for the CORE project because it was not supposed to be awarded based on low bid
(See Finding No. 2.c. which reported a concern affecting low bid.)
Based on representations from UDOT employees, CORE project bid amounts were
supposed to be blocked or “blinded” so as not to be accessible to anyone through PDBS;
however, there was a programming error and the bid amounts were not blocked and
properly secured. An employee indicated that the CORE project was the first project to use
this blocked feature, and UDOT was evidently unaware that the programming did not work
as intended. Consequently, employees with access to PDBS were also able to access and
view the bid results. Based on information from informants, we determined that the Civil
Rights Manager accessed the bid amounts and improperly disclosed the bid rankings to an
employee at Wadsworth Brothers Construction, a contractor who is part of Provo River
Constructors, the proposer who won the award for the CORE project. Although the bid
information was misinterpreted and improper conclusions were reached, this breach of
confidentiality compromised the integrity of the evaluation and selection process.
Recommendation:
We recommend that UDOT secure and protect bid information by ensuring that bid
information is blocked when necessary. We also recommend that UDOT take
appropriate disciplinary action against the Civil Rights Manager for improperly
disclosing confidential information.
UDOT’s Response to Recommendations for Finding 8: UDOT concurs with the

recommendations. UDOT has already instituted changes to the security of the bid system
and took appropriate disciplinary action with the Civil Rights Manager.
SETTLEMENT:
9. INCOMPLETE AND INADEQUATE DOCUMENTATION TO SUPPORT

SETTLEMENT AMOUNT
UDOT did not obtain supporting documentation for the entire $13 million of costs incurred
by Flatiron/Skanska/Zachry Joint Venture (Flatiron Joint Venture), which was the basis for
determining the bid protest settlement amount. UDOT received a bid protest from Flatiron
Joint Venture as a result of awarding the contract for the CORE project to Provo River
Constructors. While neither Flatiron Joint Venture nor UDOT “admitted liability or
wrongdoing,” they agreed to settle for $13 million to avoid the costs of litigation. The
settlement amount was based on the representation that Flatiron Joint Venture incurred costs
in excess of $13 million in Flatiron Joint Venture’s pursuit of the CORE project proposal.
Although UDOT obtained a written certification of costs exceeding $13 million from
18
Flatiron Joint Venture, and UDOT represented to us that they had documentation from
Flatiron Joint Venture in their possession supporting those costs, we found that the
documentation received from Flatiron Joint Venture was incomplete since it supported only
$8.6 million of the $13 million of represented costs. The documentation did not include the
costs of Skanska and Zachry nor did it include source documentation, such as invoices,
employee time records, etc. Instead, the documentation consisted merely of Flatiron
Constructors’ portion of internal bid costs and spreadsheets representing third party bid
costs “incurred by the FSZ Joint Venture to pay the design firms and other outside
professionals and vendors.” UDOT should have verified that the documentation received
from Flatiron Joint Venture completely and adequately supported the representation of $13
million in costs prior to the payout of the settlement. Failure to adequately review
supporting documentation could result in incomplete and inadequate supporting
documentation and improper settlements.
Recommendation:
We recommend that UDOT obtain and review adequate supporting documentation

prior to paying settlements.
UDOT, while believing it would ultimately win any legal challenge, was concerned about
the additional costs that would accrue due to any delay to the project. UDOT explored three
options:
1. Fight the protest.

2. Re-procure the project.
3. Negotiate a settlement.
Because finite resources were a primary concern and the need for the project was
immediate, UDOT used expected costs and delays to the project as the two determining
factors for selecting the next course of action. UDOT also considered the past history of
legal challenges to large projects, particularly the Legacy Parkway. After an injunction to
halt that project was issued by the courts, eventual delay costs were in the hundreds of
millions of dollars — a prospect that UDOT felt was unacceptable for the I-15 CORE
project.
The first and second options — fighting the protest or re-procuring the project — would
incur substantial additional costs. Reviews of protest procedures showed delay periods were
unpredictable; with the first stage alone taking up to three months and the entire process
taking 18 months or longer to be fully resolved. A protest delay would have accumulated the
following estimated costs:
19
1. Escalation costs: The signed contract with PRC permitted them to collect
construction escalation costs that keep the bid intact in the event of a delay. Those
costs accumulate at a rate of 3 percent per year after the first 6 months, leading to a
$16.5 million cost increase for a one-year delay and a $33 million cost increase for
an 18-month delay.
2. PRC suspension costs (overhead and carrying): PRC could claim further
reimbursement of delay costs that could have reached $6 million per month.
3. Carrying costs: Project costs incurred by UDOT to keep certain elements of the
project moving forward (such as right-of-way acquisition, utility agreements, office
space rental, etc.) during a delay would have been between $.5 million and $1
million per month.
4. Litigation costs: A protracted legal battle would have incurred additional costs
upwards of $1 million.
Re-procuring the project would have added additional costs and delayed the project by
more than two years. A risk analysis was conducted, which indicated that costs to re-
procure would include the following:
1. Paying PRC damages for proposal costs, design costs, overhead, and
demobilization.
2. Additional design costs to prepare for a new bidding process.
3. Possible damages to FSZ and litigation costs.
4. Re-procurement costs of $20 million to $30 million.
5. Construction inflation costs.
In light of those high risks for substantial additional costs and long delays, UDOT
leadership selected the last option of an immediate settlement as a sound business decision
and most cost-effective way to move the project forward.
UDOT entered into negotiations with FSZ and proposed to pay FSZ’s certified costs to
prepare their bid proposal; a portion of which was covered by the original stipend that was
to be paid to each unsuccessful bidder.
Because of the certification of the costs, UDOT could reasonably rely upon the certification
because the certification is a guarantee that those costs are accurate, even if an
independent third party audited the records.
Certifying costs, instead of providing source documentation, is common construction

industry practice. This certification still allows UDOT the ability to audit all source
documentation, and the costs are not certified unless the costs can be independently
audited. If the documented costs are less than what was certified, UDOT can and would be
reimbursed.
20
Subsequent to the signed settlement agreement, UDOT obtained documentation from

Skanska and Zachry for the estimated costs for FSZ’s proposal for the I-15 CORE project.
The documented bid costs for Skanska and Zachry were over $5.9 million. FSZ, including
each of the joint venture partners, certified a cost in excess of $13 million. The
documentation provided later by the joint venture partners as a result of the audit
documented bid costs in excess of $14.5 million. Therefore, UDOT’s reliance upon certified
costs was justified and reasonable for the settlement.
UDOT’s Response to Recommendation 9: UDOT agrees that adequate documentation

should be in-hand prior to paying settlements. Documentation provided as a result of the
audit shows that the actual costs to prepare the FSZ proposal was greater than the amount
agreed to as part of the settlement.
10. LACK OF POLICIES AND PROCEDURES OVER BID PROTESTS
UDOT does not have written policies and procedures for handling and resolving bid
protests. The written policies and procedures should include such things as procedures and
responsibilities of employees, the requirements of the Procurement Code, and applicable
authorizations as required by the State Settlement Act as amended by HB 34 substitute
during the 2011 State Legislature. While certain procedures related to bid protests are
documented in specific projects’ Instructions to Proposers, they are not adequate and are not
formalized written policies and procedures. Written policies and procedures are necessary
to help ensure that protests are consistently handled and resolved as allowed by Utah Code.
The lack of written policies and procedures could cause bid protests to be handled
inconsistently or in violation of State law.
Recommendation:
We recommend that UDOT implement written policies and procedures over bid
protests.
Part 8 of the Utah Procurement Code provides the time deadlines for filing the protests, the
individual who decides the protest, the time period for rendering a decision, relief that is
granted when a protest is sustained and notification of the right to judicial or administrative
review.
UDOT has followed Part 8 of the Utah Procurement Code and applicable statutes in
resolving all the protests. Only the Executive Director has decided the protests as required
by the Utah Procurement Code. In this case, UDOT followed existing law.
UDOT’s Response to Recommendation 10: UDOT agrees with the recommendation.
21
CONFLICTS OF INTEREST:
11. INADEQUATE CONFLICT OF INTEREST/CONFIDENTIALITY POLICIES AND

PROCEDURES
UDOT does not have adequate policies and procedures related to employee conflict of
interest and confidentiality. UDOT has not effectively defined potential conflicts of interest
and has not adequately emphasized the importance of avoiding conflicts of interest and
maintaining confidentiality. We noted the following problems with conflict of interest and
confidentiality policies and procedures:
a. UDOT’s current conflict of interest and confidentiality policies are not comprehensive.
The current conflict of interest policy identifies possible conflicts of interest as outside
employment and exchanges, such as gifts and meals over $50. However, it does not
identify other possible conflicts, such as family or close personal relationships with
parties associated with UDOT, ownership interests, etc. In addition, the current
confidentiality policy is too brief. We noted one project-specific conflict of
interest/confidentiality form that did not include as much information as in the policy,
but another project-specific form (used on the CORE project) that included much more
detail than the written policy. UDOT should expand the wording of the written
confidentiality policy so that the policy is complete.
b. UDOT does not require employees to complete a conflict of interest/confidentiality

statement on a periodic basis. UDOT appears to require these statements to be
completed on a project basis. Therefore, employees not assigned to a specific project
are overlooked and do not complete the required statements. We noted the following
situations:
1) The former Director of Research and Bridge Operations, whose job position was
changed to Right-of-Way Engineer in 2010, worked for 19 years without
completing a conflict of interest/confidentiality statement. As a result, she had not
disclosed her private engineering and consulting company or her personal
relationship with a contractor who regularly conducts business with UDOT and
submitted a proposal for CORE. In August 2010, she was required to sign a
statement in response to a legislative audit recommendation; however, she still did
not disclose her conflicts. In addition, she did not maintain confidentiality in that
she disclosed confidential contractor information to a former UDOT employee
working in private industry.
2) The Civil Rights Manager did not complete a conflict of interest/confidentiality

statement, and she improperly disclosed CORE bid information to a contractor
who had submitted a proposal for the CORE project, as reported in Finding No. 8.
22
UDOT should evaluate which employees should complete a conflict of interest/

confidentiality statement and how often. At a minimum, a conflict of
interest/confidentiality statement should be completed annually or when changes in
employee circumstances or job assignments occur. In addition, UDOT should consider
taking disciplinary action against employees who do not properly disclose all conflicts or
who fail to maintain confidentiality.
c. The conflict of interest/confidentiality forms reviewed were inadequate. We noted

three different forms used by UDOT (and there could be more), and all three had
different verbiage. The forms we reviewed included a “Conflict of Interest
Declaration” form that was used as a general form and did not cover confidentiality, a
project-specific “Confidentiality and Disclosure Agreement” form that was used on the
CORE project, and a “Conflict of Interest & Confidentiality Statement” form that was
used for an employee associated with a bridge pool. The following components are
inconsistent or lacking in the forms and should be included (however, this list is not all-
inclusive):
1) Disclosure of conflicts of interest and agreement to maintain confidentiality on the

same form.
2) Disclosure of all outside employment.
3) Disclosure of substantial ownership interests.
4) Disclosure of family or other close personal relationships with parties associated

with UDOT.
5) Disclosure of all potential conflicts of interest. This section should require listing
potential conflicts or stating that no conflicts exist.
6) Management’s assessment of the potential conflicts of interest and their resolution

of any identified conflicts.
To ensure adequate information is included in conflict of interest/confidentiality forms,

UDOT should consider standardizing the forms.
The lack of adequate conflict of interest/confidentiality policies and procedures could result
in conflicts of interest or breaches of confidentiality occurring due to inadequate
communication, confusion, and/or misunderstanding by employees, or due to a perceived
lack of emphasis on the importance of avoiding conflicts and maintaining confidentiality.
Recommendation:
We recommend that UDOT:
23
a. Implement comprehensive policies and procedures regarding conflict of interest/

confidentiality.
b. Require employees to complete a conflict of interest/confidentiality statement at
least annually or more often if changes in employee circumstances or job
assignments occur.
c. Standardize the conflict of interest/confidentiality statements to include adequate
information.
UDOT’s Response to Recommendations for Finding 11: UDOT concurs and will revise the
Rules of Conduct and Code of Ethics Policy and the Conflict of Interest Declaration to
address the issues in the finding. UDOT is in the process of developing a training course
dealing with conduct and ethics for employees. Employees will be required to complete a
conflict of interest statement on an annual basis. UDOT will address the confidentiality
issue when it revises the Rules of Conduct and Code of Ethics Policy and the Conflict of
Interest Declaration. The project specific confidentiality and conflict of interest declaration
has recently been updated to address the issues raised in the finding.
12. LACK OF CONTRACT DECISION DOCUMENTATION AND POTENTIAL

CONFLICT OF INTEREST
Policies and procedures related to the Bridge Operation Division do not require adequate
documentation for final contract decisions related to emergency bridge repair. Because of
concerns that a potential conflict of interest existed between the Director of Research and
Bridge Operations and a particular contractor with whom she was personally involved, we
reviewed a sample of emergency bridge repair projects. For one project, UDOT employees
indicated that the contract decision was made by the Director to award the contract to this
particular contractor without going through the established process. However, due to a lack
of documentation of the contract decision on this contract, we were not able to determine
with certainty whether the process was followed in awarding the contract to this particular
contractor. Other similar contracts also lacked documentation of the contract decision; thus,
we could not distinguish whether this contract was handled differently than other similar
emergency contracts and whether a possible conflict of interest occurred.
The current emergency bridge repair policy requires two employees of the Bridge Operation
Division to evaluate the pool of bridge repair contractors, reach a consensus on which
contractor will be contacted first, and then obtain approval from the Director. However, the
policy does not require employees to document their contracting decisions. The policy
should require that the considerations and justification for final contract decisions be
documented in order to avoid even the appearance of a conflict of interest.
24
Recommendation:
We recommend that UDOT strengthen their emergency bridge repair policy to require
adequate documentation of considerations and justification for contract decisions.
UDOT’s Response to Recommendation 12: UDOT is currently revising the emergency

contracting policy. The revision will result in a selection process that includes an
abbreviated competitive process for all emergency contracts. The emergency contract will
only include work to stabilize the particular emergency. After the emergency, the existing
procurement rules will be followed.
13. INAPPROPRIATE HANDLING OF CONFLICT OF INTEREST
UDOT allowed an employee with a conflict of interest to be involved in the evaluation and
selection process as a member of one of the Technical Evaluation Teams. The spouse of the
Technical Evaluation Team member was employed by one of the contractors submitting a
proposal on the CORE project. The team member disclosed the conflict on his
Confidentiality and Disclosure Agreement, and the project director dismissed the conflict as
not being a problem because representations were made that the spouse was not
participating in the proposal process for the contractor. However, we consider this conflict
to be a serious situation that could have affected the outcome of the evaluation and selection
process in favor of the company for which the UDOT employee’s spouse worked.
Therefore, UDOT should have taken a more conservative approach and not allowed the
employee to be a member of a Technical Evaluation Team. Although this conflict of
interest did not cause a problem because the contractor was not awarded the contract, the
integrity of the entire evaluation and selection process could have been compromised had
the contractor won the award.
Recommendation:
We recommend that UDOT take conservative measures to prevent either actual or

perceived conflicts of interest to help ensure the integrity of the evaluation and
selection process.
The Consultant Services Manual of Instruction will be amended to include a conflict of

interest review process. UDOT will also review the UDOT Procurement Division processes
for any necessary revisions.
UDOT’s Response to Recommendation 13: UDOT agrees with the recommendation and has
implemented measures to prevent and/or reduce actual and perceived conflicts of interest.
25
Attachment 1

UDOT Audit

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

UDOT Audit

Uploaded by

Copyright:

Available Formats

UTAH DEPARTMENT OF TRANSPORTATION

I-15 UTAH COUNTY CORRIDOR EXPANSION

Findings and Recommendations

Report No. 11-DOT-A

Keeping Utah AUSTON G. JOHNSON, CPA

MANAGEMENT LETTER NO. 11-DOT-A

April 11, 2011

Gary R. Herbert, Governor

The Transportation Commission, Audit Committee,

I-15 CORE PROJECT EVALUATION AND SELECTION PROCESS

1. Evaluated the technical elements of the proposal. Rated whether

a. Blinding as reported in Finding No. 1.

b. Scoring as reported in Finding Nos. 2 and 3. Because of subjectivity and lack of

c. Roles, responsibilities, and reporting as reported in Finding Nos. 4 through 6.

 A weakness in PDBS resulted in a breach of confidentiality and information was

b) The CORE Portal

Auston G. Johnson, CPA

cc: Carlos Braceras, UDOT Deputy Director

FINDINGS AND RECOMMENDATIONS:

EVALUATION AND SELECTION PROCESS:

AUDIT RESPONSE LETTER Attachment 1

EVALUATION AND SELECTION PROCESS:

1. EVALUATION PROCESS NOT BLINDED

We recommend that UDOT include blinding in evaluation and selection procedures,

UDOT’S Response to Finding 1:

UDOT’S Response to Recommendation 1: UDOT agrees with the recommendation as a

2. ISSUES WITH SCORING BY THE SELECTION RECOMMENDATION

Refer to the flowchart on page ii for a summary of the scoring process.

a. Lack of Proper Documentation for the SRC's Conclusions and Recommendations

We recommend that UDOT provide adequate documentation to support

UDOT’S Response to Finding 2a:

UDOT’s Response to Recommendation 2a: UDOT believes the documentation was

b. Risk of Inconsistent/Misleading Information Given to Proposers

meeting minutes kept by UDOT, Flatiron/Skanska/Zachry Joint Venture (Flatiron Joint

We recommend that UDOT provide instruction and/or answers given in one-on-

UDOT’S Response to Finding 2.b:

UDOT’s Response to Recommendation 2b: UDOT agrees with the recommendation

c. Concerns Over Extended Warranty

UDOT’S Response to Finding 2.c:

UDOT’s Response to Recommendation 2c: UDOT agrees in principle with the

d. Sensitivity Analysis Not Part of Policies and Procedures

UDOT’s Response to Finding 2.d:

3. POSSIBLE CONTRIBUTORS TO CLOSE SCORES

a. Scope Too Restricted

b. Evaluation Criteria Too Condensed

c. Adjectival Ratings Not Clearly Defined

UDOT’s Response to Finding 3:

a. Scope too restricted.

b. Evaluation criteria too condensed.

c. Adjectival ratings not clearly defined.

UDOT’s Response to Recommendation 3: UDOT’s Best-Value Design-Build Selection

4. EVALUATION OVERSIGHT TEAM ROLES AND RESPONSIBILITIES NOT

UDOT’s Response to Finding 4:

UDOT’s Response to Recommendation 4: UDOT agrees with this recommendation. The

5. GOVERNMENT OFFICIALS ROLES AND RESPONSIBILITIES NOT

We recommend that UDOT include adequate instruction regarding the responsibilities

UDOT’s Response to Finding 5:

UDOT’s Response to Recommendation 5: UDOT agrees with this recommendation.

6. UNNECESSARY PRELIMINARY MEETING

UDOT’s Response to Finding 6:

UDOT’s Response to Recommendation 6: UDOT agrees with this recommendation.

7. WEAKNESSES IN THE NEW BEST VALUE DESIGN-BUILD PROCESS

a. The roles and responsibilities of the Selection Official.