Intro
Preface to the Second Edition
Preface to the First Edition
Contents
About the Authors
Abbreviations
1 The Optimization Problem
1.1 Introduction
1.2 The Basic Optimization Problem
1.3 General Structure of Optimization Algorithms
1.4 Constraints
1.5 The Feasible Region
1.6 Branches of Mathematical Programming
1.6.1 Linear Programming
1.6.2 Integer Programming
1.6.3 Quadratic Programming
1.6.4 Nonlinear Programming
1.6.5 Dynamic Programming
2 Basic Principles
2.1 Introduction
2.2 Gradient Information
2.3 The Taylor Series

2.4 Types of Extrema
2.5 Necessary and Sufficient Conditions For Local Minima and Maxima
2.5.1 First-Order Necessary Conditions
2.5.2 Second-Order Necessary Conditions
2.6 Classification of Stationary Points
2.7 Convex and Concave Functions
2.8 Optimization of Convex Functions
3 General Properties of Algorithms
3.1 Introduction
3.2 An Algorithm as a Point-to-Point Mapping
3.3 An Algorithm as a Point-to-Set Mapping
3.4 Closed Algorithms
3.5 Descent Functions
3.6 Global Convergence
3.7 Rates of Convergence
4 One-Dimensional Optimization

4.1 Introduction
4.2 Dichotomous Search
4.3 Fibonacci Search
4.4 Golden-Section Search
4.5 Quadratic Interpolation Method
4.5.1 Two-Point Interpolation
4.6 Cubic Interpolation
4.7 Algorithm of Davies, Swann, and Campey
4.8 Inexact Line Searches
5 Basic Multidimensional Gradient Methods
5.1 Introduction
5.2 Steepest-Descent Method
5.2.1 Ascent and Descent Directions
5.2.2 Basic Method
5.2.3 Orthogonality of Directions
5.2.4 Step-Size Estimation for Steepest-Descent Method
5.2.5 Step-Size Estimation Using the Barzilai-Borwein Two-Point Formulas

5.2.6 Convergence
5.2.7 Scaling
5.3 Newton Method
5.3.1 Modification of the Hessian
5.3.2 Computation of the Hessian
5.3.3 Newton Decrement
5.3.4 Backtracking Line Search
5.3.5 Independence of Linear Changes in Variables
5.4 Gauss-Newton Method
6 Conjugate-Direction Methods
6.1 Introduction
6.2 Conjugate Directions
6.3 Basic Conjugate-Directions Method
6.4 Conjugate-Gradient Method
6.5 Minimization of Nonquadratic Functions
6.6 Fletcher-Reeves Method
6.7 Powell's Method
6.8 Partan Method
6.9 Solution of Systems of Linear Equations

7 Quasi-Newton Methods
7.1 Introduction
7.2 The Basic Quasi-Newton Approach
7.3 Generation of Matrix Sk
7.4 Rank-One Method
7.5 Davidon-Fletcher-Powell Method
7.5.1 Alternative Form of DFP Formula
7.6 Broyden-Fletcher-Goldfarb-Shanno Method
7.7 Hoshino Method
7.8 The Broyden Family
7.8.1 Fletcher Switch Method
7.9 The Huang Family
7.10 Practical Quasi-Newton Algorithm
8 Minimax Methods
8.1 Introduction
8.2 Problem Formulation
8.3 Minimax Algorithms
8.4 Improved Minimax Algorithms
9 Applications of Unconstrained Optimization
9.1 Introduction