Risk-Based Commissioning &

Qualification Benchmarking
Educational Track #1

Agenda
Part 1

(8:30 10:00)

ISPE Baseline Guide Volume 5: Commissioning &

Qualification vs. ASTM E2500 Mr. Robert Chew
Overview of C&Q approaches Abbott, Lilly, Pfizer

Break Vendor areas open

Part 2 (10:30 Noon)

Panel Q&A

Wrap-up

(10:00 10:30)

ISPE Baseline Guide Volume 5:

Commissioning and Qualification,
vs. ASTM E2500
Robert E. Chew, PE
President, Commissioning Agents, Inc.

My Background in Subject
Official trainer for ISPE Baseline
Commissioning and Qualification
Guide
Member of team that wrote ASTM
E2500

Key Points to Compare

Terminology
Qualification practices
Focusing the effort
Roles and responsibilities
Managing changes during the project

Terminology
C&Q Baseline guide
Commissioning
Qualification

ASTM E2500
Verification

Qualification Practices
C&Q Baseline Guide
To meet the demands of the regulators

ASTM E2500
Not mentioned

Focusing the Effort

C&Q Baseline Guide
Impact assessments
Direct impact systems and critical
components

V-model

ASTM E2500
Risk assessment -> critical aspects

Roles and Responsibilities

C&Q Baseline Guide
Quality pre and post approves protocols; quality control
role

ASTM E2500
Quality involved in process requirements, risk
assessments, critical aspects, and acceptance and
release
Subject matter experts used throughout

Managing Change
C&Q Baseline Guide
QA pre-approved change control
implemented at start of IQ/OQ

ASTM E2500
QA pre-approved change control
implemented at start of manufacture of
product for distribution

Common Elements

Good engineering practices

Planning
Requirements definition
Engineering design reviews
Project change management
Documented inspections and testing

Risk-Based Commissioning &

Qualification Approach
Abbott Laboratories
Global Pharmaceutical Operations

Steve Connelly

Overview
Where Abbott Is Today
C&Q Approach
Commissioning & Qualification Activities
How We Manage Cost
Lessons Learned

Where Abbott is Today

Risk Based Maturity Model*

*Adapted from 2007 ISPE Tampa Conference Presentation

Commissioning & Qualification Approach

C&Q Approach:
Application of ISPE Baseline Guide Volume 5:
Commissioning & Qualification (last 2 years)
Little to no implementation of ASTM E2500-07*
Level 3 (Risk-based Maturity Model)
Implementation:
Corporate policies support Science & Quality Risk
Management and includes Good Engineering Practice
(GEP) reference
Training ( RBA, GEP, ECM. Commissioning, Risk Assessments)
Tools and Templates
Discretion of the Validation Review Board (VRB)
*ASTM E2500-07: A Standard Guide for the Specification, Design, & Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and
Equipment

START
User
Requirements

Direct
Impact
Systems

VCR Submitted
w/Assessment
Report and
Rationale

Functional Design

Evaluate Risk/
Impact
Assessment

Does system
directly impact
product quality?

Yes

No

Science &
Quality Risk
Management
Qualification
Process
Flow

Design
Qualification

Does system
support a direct
impact system?
No
Impact
Systems

Commissioning
Plan

Indirect
Impact
Systems
Yes

VCR Submitted
w/Assessment
Report and
Rationale

No
VCR Submitted
w/Assessment
Report and
Rationale

Commissioning
Plan

Pre-Qualification
Engineering tests /
FAT as applicable
Engineering tests /
FAT as applicable

Process
Complete
VCR Closed

Yes

Installation
Qualification
No

System have
operational
functions?

Yes

Operational
Qualification

Support of
CPPs tested in
PV?

No

Performance
Qualification

Good Engineering Practices

Formal acknowledgement of GEP in corporate policy
Divisional GEP guideline consists of minimum requirements

Additional divisional guidelines are in development for lifecycle management

Calibration

Maintenance
Engineering change management
Training to aid site support (implementation)

Requirements Based On Impact Category

Three methods:
ISPE-Style System* Impact Assessment Questionnaire
Risk Assessment with any standard method (FMEA, HAZOP, etc.)
Technical Evaluation by SMEs
Documented and VRB approved
Determining impact to identifying critical elements

Quality Risk Control

Risk Assessment (i.e. FMEA, etc) used to enable:
Qualification focus on critical functions and/or components
Leverage limited commissioning testing (FAT/SAT) and
Eliminate some redundant activities (i.e. alarms, etc)
*System = equipment, facility, utility

Commissioning
When supporting Qualification, Commissioning must
include or address:
Engineering Change Management (ECM)
After point commissioning data intended to support
Qualification
Testing
Methods described in sufficient detail for repeatability
Documentation confirms critical elements met criteria
Acceptance criteria documented prior to execution
Criteria based on User Requirements
Documentation of observed results
Commissioning & Qualification (C&Q) Plans typically
contain run rules for project

Qualification Equipment

User requirements
Product and Process User Requirement (PURS)
General User Requirements (GURS)

Commissioning (in support of qualification)

Test it once (or one set of multiple test repetitions, where
appropriate); document it correctly
Primarily installation activities, operational testing
leveraged in some instances (VRB discretion);compile,
summarize test results for VRB approval

Qualification
When approved, commissioning reports complete the
IQ/OQ phase of Qualification with no additional protocol
testing

Qualification Software (APC/HMI/DCS)

Categorization (based on product quality impact)

Managing control issues for critical elements

Supplier Assessments for direct impact systems only

VRB discretion with technical advisory from Subject

Matter Expert

Holistic Approach
Integrated Qualification document (IOQ for
equipment/controls; as applicable)

APC specific qualification requirements

Traceability matrix = Design Qualifications (DQ)
Commissioning = Qualification except critical items

Qualification Facilities

Indirect Impact Facilities - Commission

Design and construction conforming to GEP
Abbott Engineering Standards

Direct Impact Facilities Commission/Qualify

Meet quality specifications (i.e. Aseptic facility, etc)
Design and construction conforming to GEP
Abbott Engineering Standards

Qualification HVAC

Categorization

Interdependent with facility quality attributes

For example: Continuous monitoring for critical
environmental aspects; therefore facility and HVAC
can be Indirect Impact (HVAC controls would then be
qualified)
For example: Temperature, RH, and room pressure
differentials typically critical elements which are
monitored and recorded continuously, alarming when
parameters exceed specifications; VRB discretion as
to qualify

Qualification Utilities

Impact Categories - Typically critical or non-critical

Utility systems typically not big source of Validation

Change Request (VCR) sources

Only regulatory requirement concern is typically Potable

Water feed to purification

Purpose of pretreatment usually to preserve life of

purification equipment rather than direct impact of water
quality

How We Manage Cost

$ In 2007, reviewed C&Q costs for several capital projects
to:
Establish a baseline
Compare costs for the traditional, validate everything C&Q
approach versus the risk-based C&Q approach
$ Cost comparisons across projects difficult because
Inconsistent reporting and project scope varies widely

$ Drive cost reporting model for capital projects needed to:

Address scope differences, track additional parameters:
Number of Qualification Protocols per project
Number of Qualified Systems
Total Number of Systems

Three parameters documented in C&Q Plan

How We Manage Cost

Commissioning & Qualification Cost Tracking Details
Monthly Capital Project Status
Reports are used to track
costs and progress
There is a wide variation in the
tracking systems used by the
engineers for the project costs
Most project engineers /
project managers use
spreadsheets of their own
design to track costs

How We Manage Cost

Opportunities Identified for Consistent Reporting
Engineering/Commissioning/Qualification costs categorized below
Activities not linear, categorization to clarify gray areas of cost
reporting

ENGINEERING
FEP 1-3
User Requirements
Functional Specification (Draft)
Preliminary Design
Detailed Design
Construction
Loop Tuning
Equipment Installation/Alignment

COMMISSIONING
Risk Assessment
Failure Mode Effect Analysis
I/O checkout
Calibration
Functional Specification (Revisions)
Factory Acceptance Testing
Startup
Trouble Shooting
Sequence Testing
As-Builts (Non Qualified Systems)

QUALIFICATION (CERTIFICATION)
Validation Plan
Design Qualification
Equipment Installation Qualification
Equipment Operational Qualification
Computer Hardware Installation Qualification
Computer Software Installation Qualification
Controller Modification Qualification
HVAC Installation/Operational Qualification
Utility Installation Qualification
Process Qualification
Validated Drawings
Samples/Testing
Cleaning/Passivation
Inspections

Engineering Work Orders / Contracts

Defines appropriate cost bucket for consistency across

projects

Lessons Learned
Risk-Based C&Q is an iterative process (one

tiny

step at a time)

Corporate & divisional policies in place, tools developed,

although success for various sites, concern about corporate &
regulatory audits
Sites want to maintain autonomy with guidance; common
elements, not same approach
(i.e. all engineers comply however variations due to specialty)

Application:
Site readiness tools - identifying opportunities for closing gaps

Lessons Learned
Listen, Speak, Deliver. Communication is key.
Quality may wonder:
What if engineering doesnt do its job?;ensure compliance
Engineering may wonder:
Quality = police & spell checkers; right the first tyme
Application:
Regular/frequent communication with VRB & project team

Lessons Learned
Sustainable Paradigm Shift
Sites: Implementation, accountability & ownership a must for
sustainability
Resources: People, time & money, site retaining technical
knowledge
Application:
Training & support implementation as applicable

Thank you!
Questions will be addressed during
the Panel Q&A session (after break)

Eli Lilly & Company

Commissioning and Qualification
Approach

Eli Lilly & Company Approach to

Commissioning & Qualification
C&Q Delivery Strategy
C&Q Metrics

C&Q Lessons Learned

Improved C&Q Delivery Strategy

Recent GFD C&Q Delivery Strategy

~ Baseline Guide approach (02 07)

Developed procedures governing C&Q activities

Fully commissioned all assets ~ GEP
Reserved qualification for critical aspects of DI systems only
Leveraged commissioning documentation to support
qualification commissioning subject to GDPs
Involved Quality at C&Q strategy level
In addition,
Simulation has been used to commission automation aspects
CSV testing has been integrated within C&Q test documents

Fundamental C&Q Delivery Strategy

~ for Global Facilities Delivery (GFD) Capital Projects

Profile

Approve
CQMP

System
Classification
DQ/DR
Component
Classification

Planning
Design

Construction
QA

Requirements

Initial Cals

Procurement

Loops

Construction/
Installation

Simulation/
Offline Testing

Commissioning
Package

Project

Commissioning

IQ OQ PQ

Qualification

Owner
IV

FT
SAT
FAT Startup
Approve Systemlevel Plans
RV

Validation
Operations/
Maintenance

C&Q Delivery Strategy ~ Results

This strategy has resulted in:

Earlier resolution of issues

Opportunity to understand systems
Reduced Qualification time
Reduced Qualification costs
Fewer Qualification comments/discrepancies

C&Q Metrics

Schedule Metrics
Cost Metrics

Discrepancy Metrics

Typical C&Q Schedule Metrics ~ Durations

10 months from facility mechanical completion to OQ completion

Total Commissioning duration: 24 mos. (172 days/system)

Total IQ/OQ duration: 6 months
FT duration/system: 42 days
IQ duration/system: 11 days
OQ duration/system: 17 days
Average # of days from MC to OQ completion: 227 days
Note: Metrics for new powder filling facility consisting of approx. 40 systems

C&Q Cost Metrics ~ Historical view

C&Q Cost
as % TIC
40%

25%

20%
10%

10%
5%

5%

No C&Q prog
1998- 2001

Part. C&Q prog

2001-2003

Full C&Q prog

2002- pres.

3% GOAL

C&Q Budget Metrics

~ capital projects completed in 07
Total Installed Cost
$319MM
$186MM
$180MM
$145MM
$14.5MM
$11.6MM
$7.2MM

C&Q Cost (% TIC)

6.0%
1.5% (comm. only)
5.5%
5.7% (incl. non-CQ activities)
4.3%
4.3%
2.2%

Average:

4.7% (C&Q projects only)

C&Q delivery strategy has resulted in a step change in C&Q savings!!

Lilly C&Q Work Breakdown Structure

Lilly budgets/tracks C&Q expenses via 4 primary
WBS categories:

Planning & Design

Commissioning
Qualification
Administration

New C&Q Cost Data spec recently approved to

provide guidance

Planning & Design - Subcategory

C&Q plans and strategy

Initial schedule development
Estimating
System classification
Component classification
System C&Q plans
Vendor assessments
User requirements
Design review, spec review, drawing review reviews/tag list
Engineering requirements
Design Qualification
Traceability Matrices

Commissioning - Subcategory

FAT development
FAT execution
Receipt Verification development
Receipt Verification execution
Installation Verification development
Installation Verification execution
SAT development
SAT execution
Construction turnover packages

Start-up protocol development

Start-up protocol execution
Functional testing development
Functional testing execution
Commissioning packages
C&Q plan summary reports
FAT travel
Supporting documents

Qualification - Subcategory

IQ development
IQ execution
OQ development
OQ execution
PQ development
PQ execution

Administration - Subcategory

Training
Project management
C&Q resource travel
Administrative assistance
Audits
CSV support

Typical C&Q Budget Metrics

~ Cost Breakdown (%)

Procedures 2%
OQ 2%
IQ 2%
CCs 2%

Training
1%

Impact Assessment
SAT
RI 1%
FAT
1%
1%
1%

Doc Revs 1%
Scheduling 0%
Doc mgmt 12%

DQ/EDR 3%
Project mgmt 11%
URs 3%
CSV Support 3%

FT 11%

Construction QA 4%
IV 7%
Planning 5%

SU 9%

Note: Only includes cost data for C&Q contractor

Travel 10%
RV 10%

C&Q Discrepancy Metrics ~ Philosophy

C&Q delivery strategy based on risk mitigation approach
DR/DQ reduced risk during Construction/Build, Commissioning,
Qualification
Construction quality assurance activities reduced Commissioning risks
Simulation reduced risk during Commissioning (Startup, SAT, FT)
Commissioning (RV, IV, FAT, SAT, FT) reduced risk during
subsequent Commissioning activities and Qualification
As risk decreased, the number of discrepancies also decreased resulted in cleaner qualification documentation and better schedule
adherence during qualification

Typical C&Q Discrepancy Metrics

# of Discrepancies

700
600
500
400
300
200
100
0
RV

SIM FAT

IV

SU

SAT

IL

IC

FT

CP

IQ

OQ

C&Q Activities

1230 discrepancies resulting from Construction QA activities

2800 discrepancies during commissioning
130 discrepancies during qualification

PQ

CQP

US

Pr
ot Aut
oc
om
ol
De atio
ve n/S
lo
pm oftw
M
ar
en
iss
e
t
Er
in
g/
r
M
In
ec ors
co
I
rre nco han
ic
ct
r
Do rec al
cu t D
a
m
en ta
Tr
ac tatio
kin
n
g
To
Ta ol
M
gg
od
Pr
e
l N ing
ot
oc
De
u
ol
cis mb
E
er
i
M
iss xec on T
ut
in
oo
io
g
M nE l
od
rr
Un el N ors
u
av
a i mb
la
bl er
e
Pr
da
oc
Sy
ed
Vo ta
st
em
ur
id
a
l
Ch ed
M
/P
at
er lan ang
ial
D
e
of evi s
Co ati
on
n
M stru s
isc
c
el tion
la
ne
ou
De s
sig
n

# of Discrepancies

Typical C&Q Discrepancy Metrics ~ Categories

900

800

700

600

500

400

300

200

100

Accepted

Corrective Action

C&Q Lessons Learned

1. Properly define User Requirements

Testable, verifiable, applicable

Not too prescriptive, no design details, no guesses

Include C&Q engineers in review/approval process

2. Clearly define data and documentation requirements

3. Change management

Dont implement formal change control too soon

Dont confuse change control with version control

Ensure commissioning change process is well understood

Ensure construction changes are communicated to team

Ensure automation changes are reflected in design docs

Current GFD C&Q Delivery Strategy

~ Enhanced Baseline Guide approach
Conducted several six sigma initiatives to improve C&Q processes
Developed additional procedures and best practices pertaining to C&Q

GDP SOP
Change Management SOP
Transfer of Care, Custody and Control SOP
Discrepancy Management Best Practice

Developed new specifications governing the following

Vendor data and documentation

FAT/SAT
C&Q cost data

Defined new project information management role

Implementing new requirements definition process based on systems
engineering principles
Implementing formal Construction QA program

(last 2 years)

Current GFD C&Q Delivery Strategy

~ Enhanced Baseline Guide approach

(cont.)

Developed Automation Lifecycle Management engineering standard to

accommodate integration of CSV and C&Q resulting in:

Elimination of duplicate requirements, design reviews, testing

Elimination of all or nothing approach to software testing
Allowance for risk-based delivery strategies Leverage offline or simulation testing
to meet requirements

Assessing risk in areas beyond product quality to streamline C&Q delivery

Vendor risk
Technology risk
Automation strategy

Leveraging vendor testing to support commissioning vs. just using as

contractual gate/risk mitigation
The resulting C&Q delivery strategy will hopefully be realized as another
step change in C&Q savings! Potential C&Q costs as low as 3% of TIC

Questions?

Q&A

Risk Based Commissioning & Qualification Benchmarking

Presented by:
Nick Andreopoulos
Senior Manager/Team leader
Pfizer Global Engineering
Peapack, NJ

Chris Beltz
Senior Manager/Team Leader
Pfizer Quality Operations
Kalamazoo, MI

Agenda
Basis of C&Q Practices
C&Q Program Management
Approach
C&Q Cost Management
Key Lessons

Current C&Q Practices

Primarily based on C&Q Baseline Guide 5
Certain sites have streamlined C&Q process
C&Q Lite version for small projects and minor equipment
changes
Leveraging of Commissioning tests into IOQ
Applying some ASTM concepts Focusing IOQ on risk to
product quality/patient safety
Functional Level Impact Assessment - Replacing
Component Level Impact Assessment

Currently developing and piloting full ASTM E2500

based Verification approach at targeted sites

C&Q Program Management

C&Q supported jointly by Quality,
Engineering and Automation/IT
Quality requirements defined in Corporate
Quality Standards
C&Q Approach defined in Global Practices
Sites develop their own procedures and
validation master plans based on the global
documents

C&Q Cost Management

C&Q costs that are associated with
equipment qualification are included
in Project Capital budgets. Process
Validation and Cleaning Validation
are not included in project budgets
C&Q cost managed by Project team

Key C&Q Lessons Learned

Upfront involvement by key stakeholders in the
Assessment steps to define the appropriate scope
for qualification
IOQ scope should be focused on critical
aspects/functions. All other testing is GEP
Testing documents should be based on SME
execution and leveraged, where possible, from
vendors. Level of documentation should be
commensurate with risk to product quality/patient
safety.
Roles/Responsibilities need to be clearly defined
at the outset of projects

Risk-Based Commissioning &

Qualification Benchmarking
Educational Track #1
Part 2

Agenda
Part 1

(8:30 10:00)

ISPE Baseline Guide Volume 5: Commissioning &

Qualification vs. ASTM E2500 Mr. Robert Chew
Overview of C&Q approaches Abbott, Lilly, Pfizer

Break Vendor areas open

Part 2 (10:30 Noon)

Panel Q&A

Wrap-up

(10:00 10:30)

Thank You!!

Robert Chew, Commissioning Agents

Matt Warhover, Commissioning Agents
Steve Connelly, Abbott Laboratories
Dave Dolgin, Abbott Laboratories
Scott Hamm, Eli Lilly
Rick Gunyon, Eli Lilly
Nick Andreopoulos, Pfizer
Chris Beltz, Pfizer
Great Lakes Chapter Board and Directors
Ron Dunn, GLC Manager