Approximations

Lecture 6
6

L6.2

Overview
Introduction and Overview
Response Surface Models (RSM)
Radial Basis Functions (RBF)
Approximation Wizard
Workshop

Introduction to Isight

L6.3

Background
Approximation Models: mathematical models which approximate the
relationship between a set of inputs (independent variables) and
outputs (responses)

Developed initially by experimentalists

In use in analysis since 1964 1964

Also known as behavior models

Introduction to Isight

L6.4

Why Use Approximation Models?

Reduce evaluation time for an exact analysis

Provide smoothing effect on the design space

Provide insights into I/O dependencies

Reduce chances of getting trapped in local optima

Use in conjunction with design exploration techniques including:

DOE
Optimization
Monte Carlo

Introduction to Isight

L6.5

Approximation Model Implementation

Types:
Response Surface Model (RSM)

Radial Basis Function Approximation (RBF)

Features:
Drag and Drop in Workflow

Wizard-based Setup
Error Analysis
Visualization
Multiple approximations can be defined per
component

Introduction to Isight

All features are accessible via the

Approximations submenu in the
component title bar

L6.6

Response Surface Model

Approximates the components output(s) with simple algebraic functions, usually

polynomials of low order.

Most common approximation model type
RSM may be 1st, 2nd, 3rd, or 4th order:
1234
F(x) = a0 + S bixi + S ciixi2+ S cijxixj (i<j) + S dixi3 + S eixi4
No mixed polynomials (interactions) for 3rd and 4th order terms
34
Data points not guaranteed to lay on response surface

Introduction to Isight

L6.7

Response Surface Model

Initialize model
With random design points
By running a DOE

Using previously collected data (e.g., a DOE data set, lab data, etc.)

Initialization data
Linear initialization requires (N+1) points
(N+1)

Quadratic initialization requires (N+1)(N+2)/2 points

(N+1)(N+2)/2
Cubic initialization requires [(N+1)(N+2)/2] + N points
[(N+1)(N+2)/2] + N
Quartic initialization requires [(N+1)(N+2)/2] + 2N points
[(N+1)(N+2)/2] + 2N

Introduction to Isight

L6.8

Term Selection in RSM

Use term selection to

Remove terms with low significance

Improve prediction reliability of the model

Reduce data required to initialize model

Select best model with limited number of design points

Introduction to Isight

L6.9

RSM Term Selection Algorithms

Given a set of k predictor variables select a subset p (p<k)

variables while minimizing Residual Sum of Squares (RSS)
p
n
kp
2

RSS (Yi b j X i , j )
i 1

j 1

4 term selection alternatives:4

1. Sequential replacement

Start with constant and add polynomial terms one at a time

Check to see if existing terms can be replaced/dropped

Not guaranteed to find the best model

Introduction to Isight

L6.10

RSM Term Selection Algorithms

2.

Efroymsons stepwise regression Efroymsons

Variation of Forward Selection
Add one best term to minimize RSS

Add next best term to minimize RSS

Check if any included terms can be dropped
guaranteed to find the best model
3.

Two-at-a-time replacement
Better chance to find best model
More expensive

4.

Exhaustive search
Best model is selected
Very expensive

Introduction to Isight

L6.11

Term Selection in RSM

Method

Computational
Cost

Quality

Sequential

Low

Low

Stepwise

High

Average

High

Very Good

Very high

Best

Two-at-a-time

Exhaustive

Introduction to Isight

L6.12

R2 Analysis: Response Surface Functions

R2
R2 analysis is a measure of how well the model polynomial
approximates the actual function
R2
R2 = 1 polynomial values and response function values are identical
(at all design points)
R2=1
Valid only if the number of design points is greater (preferably 2-3
times) than the number of coefficients
23

Introduction to Isight

L6.13

Radial Basis Functions (RBF)

Type of neural network

Response surface goes through all data points
Isight implementation Isight
RBF uses a variable power spline:
|| x xj || cj
where

|| x xj || is the Euclidean distance

cj is a shape function, 0.2 < c < 3.0
Optimize the value of c to minimize sum of the errors
C

Introduction to Isight

L6.14

Radial Basis Functions (RBF)

Requires 2n+1 data points to initialize

2n+1
Initialize model
With random design points
By running a DOE
Using previously collected data (e.g., a DOE data set, lab data, etc.)

Introduction to Isight

L6.15

Approximations Implementation in Isight

Isight

Can replace any component (process or activity) with an Approximation model that estimates that
components outputs

Introduction to Isight

L6.16

Approximation Drops in Workflow

An approximation
applied to a subflow by
dropping on the process
component

Introduction to Isight

L6.17

Approximation Drops in Workflow

Introduction to Isight

An approximation applied to a
single simulation by dropping
on the activity component

L6.18

Approximation Drops in Workflow

The Approximation component used directly in

the workflow ( if the data is pre-existing)
The data can be sample points or coefficient data
that represents the approximation

Introduction to Isight

L6.19

Wizard Allows Easy Problem Setup

Method Selection

Selection of Inputs/Outputs

Sampling via a DOE matrix,

Data File or Random Sampling

Introduction to Isight

L6.20

Verifying Approximation Accuracy

Additional sample points allocated for error analysis

4 error measures: 4

Average: differences between actual (workflow execution)

and predicted (approx. model execution) are averaged and
then normalized by the range of allowed values

Maximum: normalized maximum difference between

actual and predicted values

Root mean square: squared differences between actual

and predicted values normalized as above

R-squared: Coefficient of determination, between 0 and 1

where 1 means no error

011

Introduction to Isight

L6.21

Verifying Approximation Accuracy

Acceptance Levels for Average, Maximum

and Root Mean Square error types

Lower values are better

Default upper bound value is 0.2 for

Average and Root Mean Square, 0.3
for Maximum

02
03

Acceptance Levels for R-squared

Higher values are desired

Default lower bound value is 0.9

09

Violations are shaded in red

Introduction to Isight

L6.22

Verifying Approximation Accuracy

Response fit
Plot of actual vs. predicted
response values

Diagonal line is the ideal fit:
predicted = actual

Blue horizontal line is mean

response value, based on error
analysis sample points

Responses with Acceptance

level violations are shaded in red

Introduction to Isight

L6.23

Verifying Approximation Accuracy

Residuals
Plot of difference between
actual and predicted
response values for all error
sample points

Blue horizontal line is ideal

fit: predicted = actual

Residuals should be random

no upward or downward
trends, discernable pattern

Introduction to Isight

L6.24

Verifying Approximation Accuracy

Residual Frequencies

Plot of residuals for all
error sample points

Range is from zero to
maximum residual value
0

Introduction to Isight

L6.25

Verifying Approximation Accuracy

Total Error
Plots of total error and the
standard deviation of error
for each response

Displays spread of the error

over the sample range for
each response

Introduction to Isight

L6.26

Visualization and Design Space Surfing

2D and 3D plots 23
Slider bars
Pan, zoom and rotate

Introduction to Isight

L6.27

Design Search using Visual Design Driver

Set Design Variables and bounds

Set Design Constraints
Set Objective
Search for Optimum on Approximation model directly

Introduction to Isight