User Guide

Performance Report – Part A and B

Manual – Consultant for Engineering Projects
September 2013
Copyright

http://creativecommons.org/licenses/by/3.0/au/

© State of Queensland (Department of Transport and Main Roads) 2015

Feedback: Please send your feedback regarding this document to: tmr.techdocs@tmr.qld.gov.au

Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013
Contents

1 General ............................................................................................................................................2
2 Performance Reports ....................................................................................................................2
2.1 Part A – Pre Construction ............................................................................................................... 2
2.2 Part B – Post Construction .............................................................................................................. 2
3 Process ...........................................................................................................................................2
4 Initiation ..........................................................................................................................................3
4.1 Initiation of Contract (C7561.1) ....................................................................................................... 3
5 Delivery ...........................................................................................................................................3
5.1 Milestone Performance Evaluation (C7561.2) ................................................................................ 3
5.2 Finalisation ...................................................................................................................................... 4
5.2.1 Finalisation of service (C7561.3) ....................................................................................4
5.3 Review ............................................................................................................................................ 4
5.3.1 Post Construction (C7563) .............................................................................................4
5.3.2 Assessment of Design ....................................................................................................4
5.3.3 Cooperation ....................................................................................................................4
5.4 Assessment .................................................................................................................................... 5
5.4.1 Performance Assessment ..............................................................................................6
5.4.2 Interpretation of Results .................................................................................................6
5.4.3 Key Assessment Characteristics ....................................................................................6

Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 i
1 General
Performance Report – Part A & B is used in assessing the performance of consultants on engineering
projects. The format is structured around simplifying and streamlining individual reporting requirements,
whilst also providing departmental staff with a greater emphasis in managing the delivery of engineering
projects.

This User Guide represents changes incorporated from industry and regional staff on improving the
previously used assessment methodology for engineering designers. Keeping with the spirit of Manual –
Consultants for Engineering Projects, this methodology highlights further relational and contractual benefits
to the Principal and Consultant.

Benefits to the department include:

• knowing where consultants are engaged

• being assured projects are delivered as required in the offer documents

• enabling regions to invite consultants in a manner that will balance workloads according to
their current capacity.

These benefits align with improvements highlighted in the reports “Guide to leading Practice for Dispute
Avoidance and Resolution” 1 and “Dispute Avoidance and Resolution (DAR)” 2 , where improved delivery
through better relationships, management, client leadership and dispute mitigation were cited.

Assessments undertaken will also provide the department with a clearer understanding of impacts
associated with the quality of design documentation and briefs.

2 Performance Reports
2.1 Part A – Pre Construction

Part A consists of three forms:

• Initiation of Contract (C7561.1) – to be completed at the commencement of the Consultancy.

• Milestone Performance Evaluation (C7561.2) – assessments are to be undertaken at agreed

milestones/hold points, which align with milestones/hold points in the Consultants schedule of work
submitted prior to commencement of services.

• Finalisation of Service (C7561.3) – a summary of the completed consultant contract.

2.2 Part B – Post Construction

Post Construction (C7563) is to be undertaken at the completion of the construction contract. The
information collated from the form is to be forwarded to the Prequalification Committee.

As with the Part A reports, it is envisaged that the Principal (or representative), Superintendent and
Consultant complete the report together so that any deliberation on results or comments can be made and
discussed jointly. This method highlights further a relational environment of improvement and learning.

3 Process
It is essential that all forms are completed and forwarded to the Prequalification Committee for reporting
purposes. The details contained provide information relevant to four stages within the contract – Initiation,
Delivery, Finalisation and Review.

1 “Guide to leading Practice for Dispute Avoidance and Resolution”, CRC-CI, 2009
2 “DAR: Dispute Avoidance and Resolution”, TMR 2009
Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 2
The Process Flow Chart (Figure 1) provides the process to be followed in completing all required stages of
consultant engagement.

All performance reports are to be forwarded electronically to consultantprequal@tmr.qld.gov.au

It is required that the subject line in the email contains the contract number and the type of performance
report attached.

4 Initiation
4.1 Initiation of Contract (C7561.1)

The Initiation of Contract form is to be completed as a stand alone document at the commencement of the
consultancy. The purpose of completing this form at an early stage is to provide contract information from
the Regions to the Prequalification Committee. This addresses the notion of the department not knowing
consultant engagement and capacity across the state.

5 Delivery
5.1 Milestone Performance Evaluation (C7561.2)

The Milestone Performance Evaluation form provides the ability for consultants to actively engage in the
performance assessment alongside the Principal. This enables both parties to discuss an agreed score for
each of the four elements of assessment:

• Technical Skills of Key Members

• Delivering the Service

• Relationship Management

• Local Knowledge and Interfaces

• Quality of Deliverables

It is anticipated that the milestones will take place during agreed times as detailed in the project schedule.
Minimum assessment requirements are indicated for varying contract sizes in Table 1 – Minimum
performance assessment requirements. The Principal and the Consultant will be required to provide
individual and agreed assessments to ensure fair and equitable responses are achieved.

Table 1 – Minimum performance assessment requirements

Small
• Complete Finalisation form
(C7561.3)
• Milestone Performance
Evaluation (C7561.2) form
not required
Contract Size
Medium
• Complete Milestone
Performance Evaluation form
(C7561.2) at milestones/hold
points
Large
• Complete Milestone
Performance Evaluation
form (C7561.2) monthly

Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 3
5.2 Finalisation

5.2.1 Finalisation of service (C7561.3)

The Finalisation form is to provide regions, consultants and the Prequalification Committee with information
that can assist in the engagement of future projects and delivery requirements for engineering consultants.
The results for undertaking this assessment may impact on the Consultant’s prequalification, where
performance can reflect unfavourably on their prequalification level or, alternatively, assist in upgrading of
prequalification levels.

Transport and Main Roads benefits from understanding prevailing consultant challenges which assists in
inviting consultants to offer on future projects. This system will also contribute information for use in
decision making and policy development for consultant engagement.

5.3 Review

5.3.1 Post Construction (C7563)

The Post Construction form provides the Principal and Consultant with knowledge relating to the impacts of
the design service provided. Delivery is assessed by the Contract Administrator/Superintendent against the
following engineering outputs:

• Assessment of the Design

• Cooperation

The Post Construction form also allows for comments by the Preconstruction Project Manager on the
Design.

The Post Construction forms are required to be completed within four (4) weeks of the completion of
construction works. It is essential that this information be supplied to the Prequalification Committee as it
can have implications to the Consultant’s prequalification.

5.3.2 Assessment of Design

The assessment of the design provides indication of the level of competency in the design. The design is
assessed against design errors/omissions, constructability, requests for review of the design and rework
requested by the construction contractor.

5.3.3 Cooperation

The level of liaison and cooperation between the Consultant and Contract Administrator is measured in this
section of post construction. This provides regions and the Prequalification Committee with relational
information regarding the Consultant’s ability to assist in the construction of their design following the
completion of their contract.

Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 4
Figure 1 – Process Flow Chart

5.4 Assessment

Performance Scores are defined in Table 2. These values are to be recorded in the Milestone Performance
Evaluation and Finalisation forms for the Principal, Consultant and/or other parties (if represented). Agreed
scores will require deliberation by the Principal and Consultant.

Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 5
5.4.1 Performance Assessment
The Performance Assessment Score is calculated as the sum of the five weighted Agreed scores
obtained for each criterion. Each of the criteria is weighted according to the weightings used in the
assessment of offers 3. The default weightings for each criterion are:

• Technical Skills of Key Members – 40%

• Delivering the Service – 30%

• Relationship Management – 10%

• Local Knowledge and Interfaces – 10%

• Quality of Deliverables – 10%

The formula to use in calculating the Performance Assessment Score is:

SPA = 0.4ATS + 0.3ADS + 0.1ARM + 0.1ALKI + 0.1AQD

NP%
Where:

SPA = Performance Assessment Score

ATS = Agreed Technical Skills of Key Members

ADS = Agreed Delivering the Service

ARM = Agreed Relationship Management

ALKI = Agreed Local Knowledge and Interfaces

AQD = Agreed Quality of Deliverables

NP% = Non Price % (VBS only)

In the instance that an agreed score cannot be negotiated, the lower of the two scores will be used only
when there is sufficient evidence to support the decision. The Performance Assessment Score, SPA, can
only be reported to two significant figures.

5.4.2 Interpretation of Results

A guide to issues that result from the delivery of services is attached in Table 3 – Assessment Guide.
The Guide provides a list of suggested actions (Considerations) for individual scores that can occur in
each Performance Assessment criteria. This Guide will assist project managers in making decisions to
positively impact on the delivery of works.
It is at the Project Manager’s discretion to apply what they feel is the most appropriate solution to any
issues within their project.

5.4.3 Key Assessment Characteristics

Each assessment criteria is measured against specific characteristics. Project managers can highlight,
prior to commencement of the contract, characteristics which are wholly relevant from each list. Location,
complexity and traffic are some considerations that can influence which characteristics can be used in the
assessment process. A list is supplied in Table 4 – Key Assessment Characteristics.

Additional characteristics that are not represented in the master list will require prior approval from the
Prequalification Committee. This occurrence should initially be noted at the time of issuing an offer, where
the consultant would have been assessed against the characteristic.

3 Clause 7.2 – Offer Assessment Non-financial Criteria Manual – Conditions for Engineering Projects, TMR
Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 6
Attachments
Table 2 – Performance Scores

Definition Score
Unsatisfactory performance. Does not meet expectations of most characteristics 1
Lower than expected performance. Does not meet expectations of a number of 2
characteristics.
Adequate performance. Meets expectations. Acceptable / Adequate. 3
Good performance. Exceeds expectations in a number of characteristics. 4
Excellent performance. Significantly exceeds expectations in regard to most 5
characteristics.

Table 3 – Assessment Guide

Score Consideration per Criteria

1 Review suitability of consultant for contract

Technical 2 Plan to be measured in upcoming review

Skills of Key 3 No additional considerations required
Members
(TS) 4 Notes on improved staffing skills to be recorded
5 Notes on exceeded staffing skills to be provided to Prequalification
Committee
1 Review suitability of consultant for contract
2 Have consultant develop an action plan to manage issues and monitor in
Delivering forthcoming review
the Service 3 No additional considerations required
(DS)
4 Notes of improved delivery to be recorded
5 Note any innovation and/or efficient practices and provide to Prequalification
Committee
1 Review suitability of consultant for contract
2 Agree to appropriate action to improve relationship and period to be achieved
Relationship by
Management 3 No additional considerations required
(RM)
4 Note reasons for better than required relationship management

5 Note the benefits to the business and the contract through strong
relationships to Prequalification Committee.
1 Review suitability of consultant for contract
Local 2 Agree to appropriate measurement in upcoming review of process
Knowledge
and 3 No additional considerations required
Interfaces 4 Notes of improved Local Knowledge and Interfaces to be recorded
(LKI)
5 Note any benefits or efficient practices to be provided to Prequalification
Committee
1 Review suitability of consultant for contract
2 Detail improvement and course of action and measure success in
Quality of forthcoming reviews
Deliverables 3 No additional considerations required
(QD)
4 Note type and benefit of better than required deliverables
5 Note exceptional work produced and quality achieved and advise
Prequalification Committee

Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 7
Table 4 – Key Assessment Characteristics
How well did the Key Team Members apply their technical skills to suit the requirements of this
project in relation to:
Road Design
- Geometric Design, including
channelised intersections and
interchanges
- Geotechnical & Pavement Design e.g.
use of local materials
- Hydraulics e.g. characteristics of local
flooding conditions, the design of
floodways
- Landscaping e.g. species suitable for
local conditions and circumstances
- Public Utility Plant
Technical Skills of Team Members
- Type Cross Sections, including
moisture control practices/techniques,
e.g. stability of batter slopes, pavement
protection
- Traffic Analysis
- Cultural Heritage management
- Native Title management
- Local Requirement
- Risk management and assessment
Understanding TMR design and
construction standards and processes
- Preconstruction Processes Manual
- Project Cost Estimating Manual
- Road Planning and Design Manual
- Drafting and Design Presentation
Standards Manual
- The Road Drainage Manual
- Manual of Uniform Traffic Control Devices
- Standard Drawings Manual
- Standard Specifications Manual
- Main Roads Project Delivery System
- Public Consultation Manual
- Environmental Management Manuals
How well was the consultant’s application of project management in consideration of the following:
- Fully understanding the problem
- Fully understanding and achieving the
corporate objective (Strategic Fit)
- Achieving the required function (operational) - Delivery the consultant services to the
Delivering the Service
outcomes
- Have a comprehensive understanding of
project management methodology (including - Offer clearly and succinctly address the
the use of the 9 PMBok elements)
- Effectively apply the methodology to the
project
- Implement the project management
methodology to realise a smooth, effective
and fully integrated delivery process that
delivered the required outcomes and outputs
- Demonstrated an appropriate risk
identification and management process
Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013
Delivering project outputs appropriate for
their intended use
- Application of design theory, including the
integration of work package components to
the project
- Appropriate outputs:
- Drawings - design, resumptions,
limited access, construction tables
- Supplementary specifications
- Schedules
- Estimates
- Electronic models
- Tender Documents
- Any other deliverables
Satisfying local requirements
- Local construction expertise/techniques:
- Environmental protection, e.g. Erosion
and sedimentation, flora and fauna,
noise, etc.
- Traffic sequencing during
construction/provision for traffic
- Availability and performance of local
materials
- Application of local design requirements:
- Design practices, e.g. when using
expansive black soils
- Supplementary specifications, e.g. for
use of non-standard materials
- Standard design drawings, e.g. specific
local design requirements
- Follow the agreed program (e.g. Gantt chart)
- Deliver the consultant services on time
- Deliver the consultant services to budget
required quality
- Provide value for money
selection criteria
- Provide appropriate project costing, including
unit hourly rates for the various key personnel
nominated in the Offer Documents
- Meet the requirements of the consultant
service to be performed in the office(s)
designated in the Offer Documents
- Achieve their overall performance
- Achieve their health and safety awareness
8
Performance Report, Part A and B, Consultant for Engineering Projects

How well did the consultant commit to working in a relational manner using appropriate relationship
management behaviours including:
Relationship Management

- Working in a partnering approach to manage - Providing timely notification of proposed

the Consultant Services
- Working together with the Principal on the
basis of the Principal playing an informed
leadership role
- Working in a relational way where the
Principal was part of the decision making
process on a progressive basis
- Setting up and attending regular progress
meetings together with relevant Key Team
Members, as appropriate
- Progressively identifying and resolving
variations at the earliest opportunity at
progress meetings
How well did the consultant demonstrate its knowledge and understanding of local requirements?
Local Knowledge &
project scope variations
- Providing timely notification of significant
differences in project cost estimates and the
RIP amount
- Doing business in a positive way, e.g. without
‘creative variations’
(as a Prequalified Consultant the nature of the
work is understood and therefore a negotiated
and agreed brief should include an ‘in principal’
approach not to seek variations for minor
discrepancies in the documents)

How well did the consultant demonstrate its approach to undertaking and managing the project
team interface?
Interfaces

- Local conditions - Regional delivery capability

- Local supply chains and materials - Availability of local personnel
- Local community requirements - Application of local design requirements
- Road user requirements - Face to face project reviews
- Communication capability

How well did the Consultant demonstrate its commitment to quality, including how this was put into
Deliverables
Quality of

practice in relation to:

- General reputation for work quality - Documentation Quality
- Design Quality - Estimating (Work Scheduling and Costing)
- Engineering Drawing Quality

Manual, Consultants for Engineering Projects, Transport and Main Roads, September 2013 1