INTRODUCTION

TO OPTIMUM
DESIGN
THIRD EDITION
JASBIR S. ARORA
The University of Iowa
College of Engineering
Iowa City, Iowa

AMSTERDAM BOSTON HEIDELBERG LONDON

NEW YORK OXFORD PARIS SAN DIEGO
SAN FRANCISCO SINGAPORE SYDNEY TOKYO
Academic Press is an imprint of Elsevier

Academic Press is an imprint of Elsevier

225 Wyman Street, Waltham, MA 02451, USA
The Boulevard, Langford Lane, Kidlington, Oxford, OX5 1GB, UK
2012 Elsevier Inc. All rights reserved
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or any information storage and retrieval system, without
permission in writing from the publisher. Details on how to seek permission, further information about the
Publishers permissions policies and our arrangements with organizations such as the Copyright Clearance
Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher
(other than as may be noted herein).
Notices
Knowledge and best practice in this field are constantly changing. As new research and experience broaden our
understanding, changes in research methods, professional practices, or medical treatment may become necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and
using any information, methods, compounds, or experiments described herein. In using such information or
methods they should be mindful of their own safety and the safety of others, including parties for whom they
have a professional responsibility.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any
liability for any injury and/or damage to persons or property as a matter of products liability, negligence or
otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material
herein.
MATLABs is a trademark of TheMathWorks, Inc., and is used with permission. TheMathWorks does not
warrant the accuracy of the text or exercises in this book. This books use or discussion of the MATLABs
software or related products does not constitute endorsement or sponsorship by TheMathWorks of a particular
pedagogical approach or particular use of the MATLABs software.
MATLABs and Handle Graphicss are registered trademarks of TheMathWorks, Inc.
Library of Congress Cataloging-in-Publication Data
Arora, Jasbir S.
Introduction to optimum design / Jasbir Arora. 3rd ed.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-12-381375-6 (hardback)
1. Engineering designMathematical models. I. Title.
TA174.A76 2011
620.0042015118 dc23
2011026976
British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library
For information on all Academic Press publications
visit our Web site at www.elsevierdirect.com
Printed in the United States
11 12 13 14 15 10 9 8 7 6 5 4 3 2 1

To
Ruhee
Rita
and
in memory of my parents
Balwant Kaur
Wazir Singh

This page intentionally left blank

Contents
Preface to Third Edition xiii
Acknowledgments xv
Key Symbols and Abbreviations xvi

2.4 Sawmill Operation 28

2.5 Design of a Two-Bar Bracket 30
2.6 Design of a Cabinet 37
2.6.1 Formulation 1 for Cabinet Design 37
2.6.2 Formulation 2 for Cabinet Design 38
2.6.3 Formulation 3 for Cabinet Design 39
2.7 Minimum-Weight Tubular Column Design 40
2.7.1 Formulation 1 for Column Design 41
2.7.2 Formulation 2 for Column Design 41
2.8 Minimum-Cost Cylindrical Tank Design 42
2.9 Design of Coil Springs 43
2.10 Minimum-Weight Design of a Symmetric
Three-Bar Truss 46
2.11 A General Mathematical Model for Optimum
Design 50
2.11.1 Standard Design Optimization
Model 50
2.11.2 Maximization Problem Treatment 51
2.11.3 Treatment of "Greater Than Type"
Constraints 51
2.11.4 Application to Different Engineering
Fields 52
2.11.5 Important Observations about the
Standard Model 52
2.11.6 Feasible Set 53
2.11.7 Active/Inactive/Violated
Constraints 53
2.11.8 Discrete and Integer Design
Variables 54
2.11.9 Types of Optimization Problems 55
Exercises for Chapter 2 56

I
THE BASIC CONCEPTS
1 Introduction to Design Optimization 1
1.1 The Design Process 2
1.2 Engineering Design versus Engineering
Analysis 4
1.3 Conventional versus Optimum Design
Process 4
1.4 Optimum Design versus Optimal Control 6
1.5 Basic Terminology and Notation 6
1.5.1 Points and Sets 6
1.5.2 Notation for Constraints 8
1.5.3 Superscripts/Subscripts and Summation
Notation 9
1.5.4 Norm/Length of a Vector 10
1.5.5 Functions 11
1.5.6 Derivatives of Functions 12
1.5.7 U.S. British versus SI Units 13

2 Optimum Design Problem

Formulation 17
2.1 The Problem Formulation Process 18
2.1.1 Step 1: Project/Problem Description 18
2.1.2 Step 2: Data and Information
Collection 19
2.1.3 Step 3: Definition of Design Variables 20
2.1.4 Step 4: Optimization Criterion 21
2.1.5 Step 5: Formulation of Constraints 22
2.2 Design of a Can 25
2.3 Insulated Spherical Tank Design 26

3 Graphical Optimization and Basic

Concepts 65
3.1 Graphical Solution Process 65
3.1.1 Profit Maximization Problem 65
3.1.2 Step-by-Step Graphical Solution
Procedure 67

vi

CONTENTS

3.2 Use of Mathematica for Graphical

Optimization 71
3.2.1 Plotting Functions 72
3.2.2 Identification and Shading of Infeasible
Region for an Inequality 73
3.2.3 Identification of Feasible Region 73
3.2.4 Plotting of Objective Function
Contours 74
3.2.5 Identification of Optimum Solution 74
3.3 Use of MATLAB for Graphical Optimization 75
3.3.1 Plotting of Function Contours 75
3.3.2 Editing of Graph 77
3.4 Design Problem with Multiple Solutions 77
3.5 Problem with Unbounded Solution 79
3.6 Infeasible Problem 79
3.7 Graphical Solution for the Minimum-Weight
Tubular Column 80
3.8 Graphical Solution for a Beam Design
Problem 82
Exercises for Chapter 3 83

4 Optimum Design Concepts: Optimality

Conditions 95
4.1 Definitions of Global and Local Minima 96
4.1.1 Minimum 97
4.1.2 Existence of a Minimum 102
4.2 Review of Some Basic Calculus Concepts 103
4.2.1 Gradient Vector: Partial Derivatives
of a Function 103
4.2.2 Hessian Matrix: Second-Order Partial
Derivatives 105
4.2.3 Taylors Expansion 106
4.2.4 Quadratic Forms and Definite
Matrices 109
4.3 Concept of Necessary and Sufficient
Conditions 115
4.4 Optimality Conditions: Unconstrained
Problem 116
4.4.1 Concepts Related to Optimality
Conditions 116
4.4.2 Optimality Conditions for Functions
of a Single Variable 117
4.4.3 Optimality Conditions for Functions
of Several Variables 122
4.5 Necessary Conditions: Equality-Constrained
Problem 130
4.5.1 Lagrange Multipliers 131

4.5.2 Lagrange Multiplier Theorem 135

4.6 Necessary Conditions for a General Constrained
Problem 137
4.6.1 The Role of Inequalities 137
4.6.2 Karush-Kuhn-Tucker Necessary
Conditions 139
4.6.3 Summary of the KKT Solution
Approach 152
4.7 Postoptimality Analysis: The Physical Meaning
of Lagrange Multipliers 153
4.7.1 Effect of Changing Constraint
Limits 153
4.7.2 Effect of Cost Function Scaling on the
Lagrange Multipliers 156
4.7.3 Effect of Scaling a Constraint on Its
Lagrange Multiplier 158
4.7.4 Generalization of Constraint Variation
Sensitivity Result 159
4.8 Global Optimality 159
4.8.1 Convex Sets 160
4.8.2 Convex Functions 162
4.8.3 Convex Programming Problem 164
4.8.4 Transformation of a Constraint 168
4.8.5 Sufficient Conditions for Convex
Programming Problems 169
4.9 Engineering Design Examples 171
4.9.1 Design of a Wall Bracket 171
4.9.2 Design of a Rectangular
Beam 174
Exercises for Chapter 4 178

5 More on Optimum Design Concepts:

Optimality Conditions 189
5.1 Alternate Form of KKT Necessary
Conditions 189
5.2 Irregular Points 192
5.3 Second-Order Conditions for Constrained
Optimization 194
5.4 Second-Order Conditions for Rectangular
Beam Design Problem 199
5.5 Duality in Nonlinear Programming 201
5.5.1 Local Duality: Equality Constraints
Case 201
5.5.2 Local Duality: The Inequality Constraints
Case 206
Exercises for Chapter 5 208

vii

CONTENTS

II
NUMERICAL METHODS FOR
CONTINUOUS VARIABLE
OPTIMIZATION
6 Optimum Design with Excel
Solver 213
6.1 Introduction to Numerical Methods for
Optimum Design 213
6.1.1 Classification of Search Methods 214
6.1.2 What to Do If the Solution Process
Fails 215
6.1.3 Simple Scaling of Variables 217
6.2 Excel Solver: An Introduction 218
6.2.1 Excel Solver 218
6.2.2 Roots of a Nonlinear Equation 219
6.2.3 Roots of a Set of Nonlinear
Equations 222
6.3 Excel Solver for Unconstrained Optimization
Problems 224
6.4 Excel Solver for Linear Programming
Problems 225
6.5 Excel Solver for Nonlinear Programming:
Optimum Design of Springs 227
6.6 Optimum Design of Plate Girders Using Excel
Solver 231
6.7 Optimum Design of Tension Members 238
6.8 Optimum Design of Compression Members 243
6.8.1 Formulation of the Problem 243
6.8.2 Formulation of the Problem for Inelastic
Buckling 247
6.8.3 Formulation of the Problem for Elastic
Buckling 249
6.9 Optimum Design of Members for Flexure 250
6.10 Optimum Design of Telecommunication
Poles 263
Exercises for Chapter 6 271

7 Optimum Design with MATLAB 275

7.1 Introduction to the Optimization Toolbox 275
7.1.1 Variables and Expressions 275
7.1.2 Scalar, Array, and Matrix Operations 276
7.1.3 Optimization Toolbox 276

7.2 Unconstrained Optimum Design Problems 278

7.3 Constrained Optimum Design Problems 281
7.4 Optimum Design Examples with MATLAB 284
7.4.1 Location of Maximum Shear Stress for
Two Spherical Bodies in Contact 284
7.4.2 Column Design for Minimum Mass 286
7.4.3 Flywheel Design for Minimum Mass 290
Exercises for Chapter 7 294

8 Linear Programming Methods

for Optimum Design 299
8.1 Linear Functions 300
8.2 Definition of a Standard Linear Programming
Problem 300
8.2.1 Standard LP Definition 300
8.2.2 Transcription to Standard LP 302
8.3 Basic Concepts Related to Linear Programming
Problems 305
8.3.1 Basic Concepts 305
8.3.2 LP Terminology 310
8.3.3 Optimum Solution to LP Problems 313
8.4 Calculation of Basic Solutions 314
8.4.1 The Tableau 314
8.4.2 The Pivot Step 316
8.4.3 Basic Solutions to Ax 5 b 317
8.5 The Simplex Method 321
8.5.1 The Simplex 321
8.5.2 Basic Steps in the Simplex
Method 321
8.5.3 Basic Theorems of Linear
Programming 326
8.6 The Two-Phase Simplex MethodArtificial
Variables 334
8.6.1 Artificial Variables 334
8.6.2 Artificial Cost Function 336
8.6.3 Definition of the Phase I Problem 336
8.6.4 Phase I Algorithm 337
8.6.5 Phase II Algorithm 339
8.6.6 Degenerate Basic Feasible Solution 345
8.7 Postoptimality Analysis 348
8.7.1 Changes in Constraint Limits 348
8.7.2 Ranging Right-Side Parameters 354
8.7.3 Ranging Cost Coefficients 359
8.7.4 Changes in the Coefficient
Matrix 361
Exercises for Chapter 8 363

viii

CONTENTS

9 More on Linear Programming Methods

for Optimum Design 377
9.1 Derivation of the Simplex Method 377
9.1.1 General Solution to Ax 5 b 377
9.1.2 Selection of a Nonbasic Variable that
Should Become Basic 379
9.1.3 Selection of a Basic Variable that Should
Become Nonbasic 381
9.1.4 Artificial Cost Function 382
9.1.5 The Pivot Step 384
9.1.6 Simplex Algorithm 384
9.2 An Alternate Simplex Method 385
9.3 Duality in Linear Programming 387
9.3.1 Standard Primal LP Problem 387
9.3.2 Dual LP Problem 388
9.3.3 Treatment of Equality Constraints 389
9.3.4 Alternate Treatment of Equality
Constraints 391
9.3.5 Determination of the Primal Solution
from the Dual Solution 392
9.3.6 Use of the Dual Tableau to Recover
the Primal Solution 395
9.3.7 Dual Variables as Lagrange
Multipliers 398
9.4 KKT Conditions for the LP Problem 400
9.4.1 KKT Optimality Conditions 400
9.4.2 Solution to the KKT Conditions 400
9.5 Quadratic Programming Problems 402
9.5.1 Definition of a QP Problem 402
9.5.2 KKT Necessary Conditions for the QP
Problem 403
9.5.3 Transformation of KKT Conditions 404
9.5.4 The Simplex Method for Solving QP
Problem 405
Exercises for Chapter 9 409

10 Numerical Methods for Unconstrained

Optimum Design 411
10.1 Gradient-Based and Direct Search
Methods 411
10.2 General Concepts: Gradient-Based
Methods 412
10.2.1 General Concepts 413
10.2.2 A General Iterative Algorithm 413
10.3 Descent Direction and Convergence of
Algorithms 415
10.3.1 Descent Direction and Descent
Step 415

10.3.2 Convergence of Algorithms 417

10.3.3 Rate of Convergence 417
10.4 Step Size Determination: Basic Ideas 418
10.4.1 Definition of the Step Size
Determination Subproblem 418
10.4.2 Analytical Method to Compute Step
Size 419
10.5 Numerical Methods to Compute Step Size 421
10.5.1 General Concepts 421
10.5.2 Equal-Interval Search 423
10.5.3 Alternate Equal-Interval Search 425
10.5.4 Golden Section Search 425
10.6 Search Direction Determination: The
Steepest-Descent Method 431
10.7 Search Direction Determination: The
Conjugate Gradient Method 434
10.8 Other Conjugate Gradient Methods 437
Exercises for Chapter 10 438

11 More on Numerical Methods for

Unconstrained Optimum Design 443
11.1 More on Step Size Determination 444
11.1.1 Polynomial Interpolation 444
11.1.2 Inexact Line Search: Armijos
Rule 448
11.1.3 Inexact Line Search: Wolfe
Conditions 449
11.1.4 Inexact Line Search: Goldstein Test 450
11.2 More on the Steepest-Descent Method 451
11.2.1 Properties of the Gradient Vector 451
11.2.2 Orthogonality of Steepest-Descent
Directions 454
11.3 Scaling of Design Variables 456
11.4 Search Direction Determination: Newtons
Method 459
11.4.1 Classical Newtons Method 460
11.4.2 Modified Newtons Method 461
11.4.3 Marquardt Modification 465
11.5 Search Direction Determination: Quasi-Newton
Methods 466
11.5.1 Inverse Hessian Updating: The DFP
Method 467
11.5.2 Direct Hessian Updating: The BFGS
Method 470
11.6 Engineering Applications of Unconstrained
Methods 472
11.6.1 Data Interpolation 472
11.6.2 Minimization of Total Potential
Energy 473

ix

CONTENTS

11.6.3 Solutions of Nonlinear Equations 475

11.7 Solutions to Constrained Problems Using
Unconstrained Optimization Methods 477
11.7.1 Sequential Unconstrained Minimization
Techniques 478
11.7.2 Augmented Lagrangian (Multiplier)
Methods 479
11.8 Rate of Convergence of Algorithms 481
11.8.1 Definitions 481
11.8.2 Steepest-Descent Method 482
11.8.3 Newtons Method 483
11.8.4 Conjugate Gradient Method 484
11.8.5 Quasi-Newton Methods 484
11.9 Direct Search Methods 485
11.9.1 Univariate Search 485
11.9.2 Hooke-Jeeves Method 486
Exercises for Chapter 11 487

12 Numerical Methods for Constrained

Optimum Design 491
12.1 Basic Concepts Related to Numerical
Methods 492
12.1.1 Basic Concepts Related to Algorithms
for Constrained Problems 492
12.1.2 Constraint Status at a Design
Point 495
12.1.3 Constraint Normalization 496
12.1.4 The Descent Function 498
12.1.5 Convergence of an Algorithm 498
12.2 Linearization of the Constrained Problem 499
12.3 The Sequential Linear Programming
Algorithm 506
12.3.1 Move Limits in SLP 506
12.3.2 An SLP Algorithm 508
12.3.3 The SLP Algorithm: Some
Observations 512
12.4 Sequential Quadratic Programming 513
12.5 Search Direction Calculation: The QP
Subproblem 514
12.5.1 Definition of the QP Subproblem 514
12.5.2 Solving of the QP Subproblem 518
12.6 The Step Size Calculation Subproblem 520
12.6.1 The Descent Function 520
12.6.2 Step Size Calculation: Line
Search 522
12.7 The Constrained Steepest-Descent
Method 525
12.7.1 The CSD Algorithm 526

12.7.2 The CSD Algorithm: Some

Observations 527
Exercises for Chapter 12 527

13 More on Numerical Methods

for Constrained Optimum
Design 533
13.1 Potential Constraint Strategy 534
13.2 Inexact Step Size Calculation 537
13.2.1 Basic Concept 537
13.2.2 Descent Condition 538
13.2.3 CSD Algorithm with Inexact Step
Size 542
13.3 Bound-Constrained Optimization 549
13.3.1 Optimality Conditions 549
13.3.2 Projection Methods 550
13.3.3 Step Size Calculation 552
13.4 Sequential Quadratic Programming: SQP
Methods 553
13.4.1 Derivation of the Quadratic
Programming Subproblem 554
13.4.2 Quasi-Newton Hessian
Approximation 557
13.4.3 SQP Algorithm 558
13.4.4 Observations on SQP Methods 561
13.4.5 Descent Functions 563
13.5 Other Numerical Optimization Methods 564
13.5.1 Method of Feasible Directions 564
13.5.2 Gradient Projection Method 566
13.5.3 Generalized Reduced Gradient
Method 567
13.6 Solution to the Quadratic Programming
Subproblem 569
13.6.1 Solving the KKT Necessary
Conditions 570
13.6.2 Direct Solution to the QP
Subproblem 571
Exercises for Chapter 13 572

14 Practical Applications
of Optimization 575
14.1 Formulation of Practical Design Optimization
Problems 576
14.1.1 General Guidelines 576
14.1.2 Example of a Practical Design
Optimization Problem 577

CONTENTS

14.2 Gradient Evaluation of Implicit Functions 582

14.3 Issues in Practical Design Optimization 587
14.3.1 Selection of an Algorithm 587
14.3.2 Attributes of a Good Optimization
Algorithm 588
14.4 Use of General-Purpose Software 589
14.4.1 Software Selection 589
14.4.2 Integration of an Application into
General-Purpose Software 589
14.5 Optimum Design of Two-Member Frame with
Out-of-Plane Loads 590
14.6 Optimum Design of a Three-Bar Structure for
Multiple Performance Requirements 592
14.6.1 Symmetric Three-Bar Structure 592
14.6.2 Asymmetric Three-Bar Structure 594
14.6.3 Comparison of Solutions 598
14.7 Optimal Control of Systems by Nonlinear
Programming 598
14.7.1 A Prototype Optimal Control
Problem 598
14.7.2 Minimization of Error in State
Variable 602
14.7.3 Minimum Control Effort Problem 608
14.7.4 Minimum Time Control Problem 609
14.7.5 Comparison of Three Formulations
for the Optimal Control of System
Motion 611
14.8 Alternative Formulations for Structural
Optimization Problems 612
14.9 Alternative Formulations for Time-Dependent
Problems 613
Exercises for Chapter 14 615

15.1.3 Overview of Solution Concepts 622

15.2 Branch-and-Bound Methods 623
15.2.1 Basic BBM 623
15.2.2 BBM with Local Minimization 625
15.2.3 BBM for General MV-OPT 627
15.3 Integer Programming 628
15.4 Sequential Linearization Methods 629
15.5 Simulated Annealing 630
15.6 Dynamic Rounding-Off Method 632
15.7 Neighborhood Search Method 633
15.8 Methods for Linked Discrete Variables 633
15.9 Selection of a Method 635
15.10 Adaptive Numerical Method for Discrete
Variable Optimization 636
15.10.1 Continuous Variable
Optimization 636
15.10.2 Discrete Variable Optimization 637
Exercises for Chapter 15 639

III

17.1 Problem Definition 658

17.2 Terminology and Basic Concepts 660
17.2.1 Criterion Space and Design Space 660
17.2.2 Solution Concepts 662
17.2.3 Preferences and Utility Functions 665
17.2.4 Vector Methods and Scalarization
Methods 666
17.2.5 Generation of Pareto Optimal Set 666
17.2.6 Normalization of Objective
Functions 667
17.2.7 Optimization Engine 667
17.3 Multi-objective Genetic Algorithms 667
17.4 Weighted Sum Method 671
17.5 Weighted Min-Max Method 672
17.6 Weighted Global Criterion Method 673
17.7 Lexicographic Method 674

ADVANCED AND MODERN

TOPICS ON OPTIMUM
DESIGN
15 Discrete Variable Optimum Design
Concepts and Methods 619
15.1 Basic Concepts and Definitions 620
15.1.1 Definition of Mixed Variable Optimum
Design Problem: MV-OPT 620
15.1.2 Classification of Mixed Variable
Optimum Design Problems 621

16 Genetic Algorithms for Optimum

Design 643
16.1 Basic Concepts and Definitions 644
16.2 Fundamentals of Genetic Algorithms 646
16.3 Genetic Algorithm for Sequencing-Type
Problems 651
16.4 Applications 653
Exercises for Chapter 16 653

17 Multi-objective Optimum Design

Concepts and Methods 657

xi

CONTENTS

17.8 Bounded Objective Function Method

17.9 Goal Programming 676
17.10 Selection of Methods 677
Exercises for Chapter 17 678

675

18 Global Optimization Concepts

and Methods 681
18.1 Basic Concepts of Solution Methods 682
18.1.1 Basic Solution Concepts 682
18.1.2 Overview of Methods 683
18.2 Overview of Deterministic Methods 684
18.2.1 Covering Methods 684
18.2.2 Zooming Method 685
18.2.3 Methods of Generalized Descent 686
18.2.4 Tunneling Method 688
18.3 Overview of Stochastic Methods 689
18.3.1 Pure Random Search Method 690
18.3.2 Multistart Method 691
18.3.3 Clustering Methods 691
18.3.4 Controlled Random Search: NelderMead Method 694
18.3.5 Acceptance-Rejection Methods 697
18.3.6 Stochastic Integration 698
18.4 Two Local-Global Stochastic Methods 699
18.4.1 Conceptual Local-Global
Algorithm 699
18.4.2 Domain Elimination Method 700
18.4.3 Stochastic Zooming Method 702
18.4.4 Operations Analysis of Methods 702
18.5 Numerical Performance of Methods 705
18.5.1 Summary of Features of Methods 705
18.5.2 Performance of Some Methods with
Unconstrained Problems 706
18.5.3 Performance of Stochastic Zooming and
Domain Elimination Methods 707
18.5.4 Global Optimization of Structural
Design Problems 708
Exercises for Chapter 18 710

19 Nature-Inspired Search
Methods 713
19.1 Differential Evolution Algorithm 714
19.1.1 Generation of an Initial
Population 715
19.1.2 Generation of a Donor Design 716

19.1.3 Crossover Operation to Generate

the Trial Design 716
19.1.4 Acceptance/Rejection of the Trial
Design 717
19.1.5 DE Algorithm 717
19.2 Ant Colony Optimization 718
19.2.1 Ant Behavior 718
19.2.2 ACO Algorithm for the Traveling
Salesman Problem 721
19.2.3 ACO Algorithm for Design
Optimization 724
19.3 Particle Swarm Optimization 727
19.3.1 Swarm Behavior and Terminology 727
19.3.2 Particle Swarm Optimization
Algorithm 728
Exercises for Chapter 19 729

20 Additional Topics on Optimum

Design 731
20.1 Meta-Models for Design Optimization 731
20.1.1 Meta-Model 731
20.1.2 Response Surface Method 733
20.1.3 Normalization of Variables 737
20.2 Design of Experiments for Response Surface
Generation 741
20.3 Discrete Design with Orthogonal Arrays 749
20.4 Robust Design Approach 754
20.4.1 Robust Optimization 754
20.4.2 The Taguchi Method 761
20.5 Reliability-Based Design OptimizationDesign
under Uncertainty 767
20.5.1 Review of Background Material for
RBDO 768
20.5.2 Calculation of the Reliability
Index 774
20.5.3 Formulation of Reliability-Based Design
Optimization 784

Appendix A: Vector and Matrix

Algebra 785
A.1 Definition of Matrices 785
A.2 Types of Matrices and Their Operations
A.2.1 Null Matrix 787
A.2.2 Vector 787
A.2.3 Addition of Matrices 787

787

xii
A.2.4 Multiplication of Matrices 788
A.2.5 Transpose of a Matrix 790
A.2.6 Elementary Row Column
Operations 790
A.2.7 Equivalence of Matrices 790
A.2.8 Scalar ProductDot Product of
Vectors 790
A.2.9 Square Matrices 791
A.2.10 Partitioning of Matrices 791
A.3 Solving n Linear Equations in n
Unknowns 792
A.3.1 Linear Systems 792
A.3.2 Determinants 793
A.3.3 Gaussian Elimination Procedure 796
A.3.4 Inverse of a Matrix: Gauss-Jordan
Elimination 800
A.4 Solution to m Linear Equations in n
Unknowns 803
A.4.1 Rank of a Matrix 803
A.4.2 General Solution of m 3 n Linear
Equations 804
A.5 Concepts Related to a Set of Vectors 810

CONTENTS

A.5.1 Linear Independence of a Set of

Vectors 810
A.5.2 Vector Spaces 814
A.6 Eigenvalues and Eigenvectors 816
A.7 Norm and Condition Number of a Matrix 818
A.7.1 Norm of Vectors and Matrices 818
A.7.2 Condition Number of a Matrix 819
Exercises for Appendix A 819

Appendix B: Sample Computer

Programs 823
B.1
B.2
B.3
B.4

Equal Interval Search 823

Golden Section Search 826
Steepest-Descent Method 829
Modified Newtons Method 829

Bibliography

841

Answers to Selected Exercises

Index 861

851

Preface to Third Edition

The philosophy of this third edition of
Introduction to Optimum Design is to provide
readers with an organized approach to
engineering design optimization that is
both rigorous and simple, that illustrates
basic concepts and procedures with simple
examples, and that demonstrates the applicability of these concepts and procedures to
engineering design problems. The key step
in the optimum design process is the formulation of a design problem as an optimization problem, which is emphasized and
illustrated with examples. In addition, insights into, and interpretations of, optimality conditions are discussed and illustrated.
Two main objectives were set for the
third edition: (1) to enhance the presentation of the books content and (2) to include
advanced topics so that the book will be
suitable for higher-level courses on design
optimization. The first objective is achieved
by making the material more concise, organizing it with more second-, third-, and
fourth-level headings, and using illustrations in example problems that have more
details. The second objective is achieved by
including several new topics suitable for
both alternate basic and advanced courses.
New topics include duality in nonlinear
programming, optimality conditions for the
Simplex method, the rate of convergence of
iterative algorithms, solution methods for
quadratic programming problems, direct
search methods, nature-inspired search
methods, response surface methods, design
of experiments, robust design optimization,
and reliability-based design optimization.

This edition can be broadly divided into

three parts. Part I, Chapters 1 through 5, presents the basic concepts related to optimum
design and optimality conditions. Part II,
Chapters 6 through 14, treats numerical
methods for continuous variable optimization problems and their applications. Finally,
Part III, Chapters 15 through 20, offers
advanced and modern topics on optimum
design, including methods that do not
require derivatives of the problem functions.
Introduction to Optimum Design, Third
Edition, can be used to construct several
types of courses depending on the instructors preference and learning objectives for
students. Three course types are suggested,
although several variations are possible.

Undergraduate/First-Year Graduate
Course
Topics for an undergraduate and/or firstyear graduate course include
Formulation of optimization problems
(Chapters 1 and 2)
Optimization concepts using the
graphical method (Chapter 3)
Optimality conditions for unconstrained
and constrained problems (Chapter 4)
Use of Excel and MATLABs illustrating
optimum design of practical problems
(Chapters 6 and 7)
Linear programming (Chapter 8)
Numerical methods for unconstrained
and constrained problems (Chapters 10
and 12)

xiii

xiv

PREFACE TO THIRD EDITION

The use of Excel and MATLAB is to be

introduced mid-semester so that students
have a chance to formulate and solve more
challenging project-type problems by semesters end. Note that advanced project-type
exercises and sections with advanced material are marked with an asterisk (*) next to
section headings, which means that they
may be omitted for this course.

First Graduate-Level Course

Topics for a first graduate-level course
include
Theory and numerical methods for
unconstrained optimization (Chapters 1
through 4 and 10 and 11)
Theory and numerical methods for
constrained optimization (Chapters 4, 5,
12, and 13)
Linear and quadratic programming
(Chapters 8 and 9)
The pace of material coverage should be
faster for this course type. Students can
code some of the algorithms into computer
programs and solve practical problems.

Second Graduate-Level Course

This course presents advanced topics on
optimum design:
Duality theory in nonlinear
programming, rate of convergence of
iterative algorithms, derivation of
numerical methods, and direct search
methods (Chapters 1 through 14)
Methods for discrete variable problems
(Chapter 15)
Nature-inspired search methods
(Chapters 16 and 19)
Multi-objective optimization
(Chapter 17)
Global optimization (Chapter 18)
Response surface methods, robust
design, and reliability-based design
optimization (Chapter 20)
During this course, students write computer programs to implement some of the
numerical methods and to solve practical
problems.

Acknowledgments
I would like to give special thanks to my
colleague, Professor Karim Abdel-Malek,
Director of the Center for Computer-Aided
Design at The University of Iowa, for his
enthuastic support for this project and for
getting me involved with the very exciting
research taking place in the area of digital
human modeling under the Virtual Soldier
Research Program.
I would also like to acknowledge the contributions of the following colleagues:
Professor Tae Hee Lee provided me with a
first draft of the material for Chapter 7;
Dr. Tim Marler provided me with a first
draft of the material for Chapter 17; Professor G. J. Park provided me with a first draft
of the material for Chapter 20; and Drs.
Marcelo A. da Silva and Qian Wang provided me with a first draft of some of the
material for Chapter 6. Their contributions
were invaluable in the polishing of these
chapters. In addition, Dr. Tim Marler, Dr.
Yujiang Xiang, Dr. Rajan Bhatt, Dr. Hyun
Joon Chung, and John Nicholson provided
me with valuable input for improving the
presentation of material in some chapters. I
would also like to acknowledge the help of
Jun Choi, Hyun-Jung Kwon, and John
Nicholson with parts of the books solutions
manual.
I am grateful to numerous colleagues and
friends around the globe for their fruitful
associations with me and for discussions on

the subject of optimum design. I appreciate

my colleagues at The University of Iowa
who used the previous editions of the book
to teach an undergraduate course on optimum design: Professors Karim AbdelMalek, Asghar Bhatti, Kyung Choi, Vijay
Goel, Ray Han, Harry Kane, George Lance,
and Emad Tanbour. Their input and suggestions greatly helped me improve the presentation of material in the first 12 chapters of
this edition. I would also like to acknowledge all of my former graduate students
whose thesis work on various topics of optimization contributed to the broadening of
my horizons on the subject.
I would like to thank Bob Canfield,
Hamid Torab, Jingang Yi, and others for
reviewing various parts the third edition.
Their suggestions helped me greatly in its
fine-tuning. I would also like to thank Steve
Merken and Marilyn Rash at Elsevier for
their superb handling of the manuscript
and production of the book. I also thank
Melanie Laverman for help with the editing
of some of the books chapters.
I am grateful to the Department of Civil
and Environmental Engineering, Center for
Computer-Aided Design, College of Engineering, and The University of Iowa for
providing me with time, resources, and
support for this very satisfying endeavor.
Finally, I would like to thank my family
and friends for their love and support.

xv