Cost & Complexity Trade-offs in

Prognostics
Dr. Ir. M. Sabri, MT

Condition Base Maintenance

(CBM)

Condition-Based Maintenance
The Opportunity

Condition Based
Maintenance (CBM)
promises to deliver
improved maintainability
and operational availability
of naval systems while
reducing life-cycle costs
The Challenge
Prognostics is the Achilles heel of CBM systems - predicting the
time to failure of critical machines requires new and innovative
methodologies that will effectively integrate diagnostic results with
maintenance scheduling practices

Prognostics

Objective
Determine time window over which maintenance must be
performed without compromising the systems operational
integrity

The CBM Architecture

PEDS Software System Architecture
Hardware
Hardware
Engine
Chiller
Sensors
Sensors
DAQ
DAQ

6. Prognostics
Interface

1.
1.DDL
DDL

Virtual
Sensor
(WNN)
failure
dimension

2.
2.Data
Data
Preprocessing
Preprocessing

3.3.Mode
Mode
Estimator
Estimator/ /
Usage
UsagePattern
Pattern
Identification
Identification

Event
EventDispatch
Dispatch

Central DB

DWNN

CPNN

Database
Database
Management
Management

5. Diagnostics
5.5.Feature
Feature
4.
Feature
4. Feature
Extraction
Extraction
Extraction
Extraction

Classifier
Classifier
(Fuzzy)
(Fuzzy)

Classifier
Classifier
(WNN)
(WNN)

The CBM Architecture (continued)

Diagnostic Models:
Fuzzy Logic Based
Wavelet Neural Network Model
Rough Set Theory based NN Model

Prognostic Models:
Dynamic Wavelet Neural Network Model
Confidence Prediction Neural Network Model

Physical Models of Failure Mechanisms

System
(FMTV, PLS,
etc.)

Sensors

DAQ/CPU

Interface

Army Vehicle
Systems/
Components

Preliminary Diagnostics

Object Oriented Hybrid

System Models

Intelligent Selection
Layer

Diagnostic Algorithms

Decision Support
Layer

Prognostic Algorithms

Online

Static and Dynamic

Case Library

Offline

Interface
Layer

Multiagent System
Intelligent Agent

Software Repository

Hardware

Model-Based CBM Architecture

Designer

Statistics
Optimization
Performance

Assessment
Module

Prescription
Maintenance
Plan

Maintainer

Community
of Agents
Multiagent
System

Centralized Control & KB

Architectures
Sensors

UUT
Events
Preprocessing

Knowledge
Base
Diagnostic
Algorithm

Sensors

UUT

Data-mining
Events
Sensors

UUT
Events

Feature
Extraction

Control

Prognostic
Algorithm

Diagnosis

Prognosis

A Generic Central Control and Knowledge Base Framework

Distributed Control & KB

Architectures
s

s
ent
Lo

ca

lK

n
Co

tro

Lo

l
ca

stic

Pro

gn

m
rith

s
s

Alg
stic

th m
ori

Knowledge
Fusion

Diagnostic Algorithms Diagnosis

Central
Control

Prognostic Algorithms
Prognosis

Central
Knowledge
Base

Local Control

Events

l KB

Local KB

L o ca

L o ca

Distributed Control and Knowledge

Base Framework
l Co

ntrol

L o ca

UUT

Sens

ors

L o ca

ts

UUT

Local Prognostic Algorithms

Sensors

Local Diagnostic Algorithms

Lo

l
ca

al

no

Even

Ev

sor

Sen

UU

c
Lo

g
Dia

o
Alg

l Dia

l Pro

gnos

gnos

tic A

tic A

lgorit

lgorit

hms

hms

Case-Based Reasoning &

Learning
CBR - an episodic memory of past experiences
CBR - initial cases by examples
CBR Methodology:
Indexing (generate indices for classification and categorization)
Retrieval (retrieve the best past cases from the memory)
Adaptation (modify old solution to conform to new situation)
Testing (did the proposed solution work)
Learning (explain failed & store successful solutions)

Case Library
Failure Mode i
Case #

1
2
3

S1

Symptoms
S2 Sm

Tests

Prescription

On-board/Off-board Diagnostics
Operational
Performance
Monitoring

Embedded Diagnostics

Operator

Abnormal
Performance
Detection

Periodic
Health
Check
Maintainer

Pre-Diagnostic
Session

Diagnostic Session

Post-Diagnostic
Session

Case-Based Reasoning Architecture

Diagnostics
Manager

Case Based
Diagnostics Reasoner

Knowledge Fusion
Module

Platform Data
Platform Family
Case Library

Platform Historical
Records Database

Current Diagnostics
Session Database

At Platform Diagnostics Session

Topology (Legacy)
1939

DCA

CAN
Embedded
Diagnostics
Interface

Etc..

J-1708

1553

Support Area
Database

*
Updates (brief case model)

Subsystem

*
?
Embedded
Diagnostics
Processor

Pressure Sensor

Fuel

Thermocouple

Starter

Current Shunt

Battery

MSD

Embedded
Diagnostics
Interface

Diagnostics
Database

Controls
Etc..

SPORT

Portable
Diagnostics &
Maintenance
Aid

Platform
Sensor

Diagnostics
Manager

Etc..

Interactive
Electronic
Technical Manual
(GUI)
Displays

Legend: (Unless Otherwise

Annotated)

Case Based
Diagnostics Reasoner

Uses

Uses
Feeds

Is

MIMOSA
Platform Family
Case Library

Knowledge Fusion
Module

Uses

* 1 or more
? 0 or more

Uses

Diagnostic
Test

Selects

Interfaces

Has

Army Central
Database

Updates

Feeds

Embedded
Diagnostics
Data Collector

Uses

Performance Measures
(How to Compare
and

Measures
Precision for Prognosis

a measure of the narrowness of an interval in which the remaining life

falls

Reliability

how the system responds to individual component failures

Extensibility or Scalability

how the system can be extended if new components are added

Robustness

how the system tolerates uncertainty

Reuse or Portability

how easy or hard it is to use this system in another problem domain

Accuracy

how an agent improves true positives and true negatives as a result of

learning, self-organization, and active diagnosis

Entropy

a measure of how the system learns and organizes over time.

Decreasing entropy signifies increasing order in a multi-agent system,
resulting in more accurate and timely diagnoses

Network Activity

how much network related activity results if the framework is

implemented for distributed systems

Implementation Issues
Embedded Distributed Diagnostic Platform (EDDP)

Hardware:
Modular I/O (e.g. NIs FieldPoint System, or MAX-IO).
Embedded PC (e.g. MPC - Matchbox PC of TIQIT or MAXPC of Strategic-Test).
Network (e.g. Ethernet, PROFIBUS, CAN).

Software:
Windows CE, Linux, QNX, VxWorks, or OsX operating
systems.
Embedded databases (like Polyhedra).
RAD tools (like eMbedded Visual Studio of Microsoft).

A Possible Agent Node

An Operator Interface
(LCD Display)

A Small PC
(MPC, MAX-PC)

Network (Ethernet, CAN, Profibus)

Distributed I/O System

(FieldPoint)

Sensors

Sensors

Sensors

CBM Performance Assessment

Objective:
To assess the technical and economic feasibility of various
prognostic algorithms

Technical Measures:
Accuracy, Speed, Complexity, Scalability

Overall Performance Measure:

w1Accuracy + w2Complexity + w3Speed +
(wi - weighting factors)

Performance

Assessment Matrix:

PM1

PM2

PM3

Algorithm #1

Algorithm #2

Algorithm #3

CBM Performance Assessment

Target Measure:
PM yr ( n f ) y p ( n f ) yr ( ns ) y p ( ns )

Behavior Measure:

Output y(n)
Real yr(n)
Predicted yp(n)

nf

PM w(i ) yr (i ) y p (i )
i ns

tpf

Mean and Variance Measures:

1
E{t pf }
N

t pf (i )
i 1

Discrete time n

1
V {t pf }
N

2
[
t
(
i
)

E
{
t
}]
pf
pf
i 1

Complexity/Cost-benefit Analysis
Complexity Measure

t p td
computation time
complexity E
E

t
time to failure
pf

Cost/Benefit Analysis

frequency of maintenance
downtime for maintenance
dollar cost
etc.

Overall Performance
Overall Performance = w 1accuracy + w 2complexity + w 3cost + .

Cost/Benefit Analysis
Establish Baseline Condition - estimate cost of
breakdown or time-based preventive maintenance from
maintenance logs
A good percentage of Breakdown Maintenance costs
may be counted as CBM benefits
If preventive maintenance is practiced, estimate how
many of these maintenance events may be avoided.
The cost of such avoided maintenance events is counted
as benefit to CBM.

Cost/Benefit Analysis (contd)

Intangible benefits - Assign severity index to impact of
BM on system operations
Estimate the projected cost of CBM, i.e. $ cost of
instrumentation, computing, etc.
Aggregate life-cycle costs and benefits from the
information detailed above

CINCLANTFLT Study
Question: What is the value of prognostics?

Summary of findings:
(1) Notional Development and Implementation for Predictive CBM Based on
CINFCLANTFLT I&D Maintenance Cost Savings
(2) Assumptions

CINCLANTFLT Annual $2.6B [FY96$] I&D Maintenance Cost

Fully Integrated CBM yields 30% reduction
Full Realization Occurs in 2017, S&T sunk cost included
Full Implementation Costs 1% of Asset Acquisition Cost
IT 21 or Equivalent in place Prior to CBM Technology

(3) Financial Factors

Inflation rate:
Investment Hurdle Rate:
Technology Maintenance Cost:

(4) Financial Metrics:

NPV
IRR

4%
10%
10% Installed Cost

15 year

20 year

$337M
22%

$1,306M
30%

Concluding Remarks
CBM/PHM are relatively new technologies - sufficient
historical data are not available
CBM benefits currently based on avoided costs
Cost of on-board embedded diagnostics primarily
associated with computing requirements
Advances in prognostic technologies (embedded
diagnostics, distributed architectures, etc.) and lower
hardware costs (sensors, computing, interfacing, etc.)
promise to bring CBM system costs within 1-2% of a
typical Army platform cost

The Dynamic Case-Based

Reasoning Architecture
Sensory data
Feature interpretation
(static, dynamic, composite)

Case indexing
AS path

Analytical Models and algorithms

Indexing path selection

Indexing rules

PD path
Phase matching

Case retrieval

Case similarity

evaluator

calculation

Case memory
inactive
active

Propagation evaluator
Case adaptation

Model-based reasoner
New case constructor

Test/evaluation

Remembrance calculation
Failure driven learning

Model base

Structure of Static and Dynamic

Case Library

tfailure

Tests

Case #j
Time to Failure
TF1
TF2
TF3

Dynamic Case Library

Failure Mode i
Conditions
time
y1 y 2 y m
t1
t2
t3

Case Library

Prescription

Tests

Static Case Library

Failure Mode i
Symptoms
Case #
S 1 S 2 Sm
1
2
3

Prediction-to-Failure Times

output

Mean:
1
E{t pf }
N

t
i 1

pf

(i )

Standard Deviation:

detect

failure

predict
Times

1
S{t df } [
N

(t
i 1

(i ) E{t pf }) ]
2

pf

Accuracy

real
predicted

upper bound

real

lower bound

tp

tf
time

DC

predicted
R

accuracy DC ( y real ) exp( R )

An Example
A defective bearing with a crack causes the machine to
vibrate abnormally
Vibrations can be caught with accelerometers which
translate mechanical movement into electrical signals
Bearing crack faults may be prognosed by examining and
predicting their vibration signals

An Experimental Setup

Bearing Vibration Data

Figure 1 Original signals: normal & defective

Figure 2 Spectra: good & defective

0.4

0.2

0.2

0.15

0
0.1
-0.2
0.05

-0.4
-0.6

100

200

300

400

500

600

700

800

900

1000

10

-2

-4

0
100 Signals
200 300from
400a good
500 and
600 a 700
800 bearing
900 1000
Vibration
defective

Vibration Signals

20

20

40

60

80

100

40
60 vibration
80
100
PSDs
of the
signals

120

140

120

140

Power Spectrum Densities

Growth of bearing crack fault

Small variations are added

5
4
PSD

PSD

4
3

3
2
2
1

20

40
60
Time Window

80

100

20

40
60
Time Window

80

100

4
PSD

PSD

Prediction

0
80

85

90
95
Time Window

Prediction by AR

100

105

0
80

85

90
Time Window

95

Prediction by WNN

100

Performance
Similarity
Error

Output
Error

Total
Error

100.00

Time
Dynamic
Error
1.0

0.1

1.0

N/A

0.20

0.20

0.20

0.20

0.20

1.0

0.4275

0.5200

0.4074

0.3448

0.3200

2.0197

0.1855

0.5500

0.2684

0.2857

0.3200

1.6096

Performance
Measures

TTF
Error Rate

Dynamic
Error

Scaling
Factor
Weighting
Coefficients
AR
Performance
WNN
Performance

1.0

Table: Performances of the AR predictor and the WNN predictor

Overall Performance Error:

---- 2.0197 for the AR predictor
---- 1.6096 for the WNN predictor
Thus, the WNN outperforms the AR in this case