Practical Optimization Methods

1 Optimization Problem Formulation.- 1.1 Optimization Problem Formulation.- 1.2 The Standard Form of an Optimization Problem.- 1.3 Solution of Optimization Problems.- 1.4 Time Value of Money.- 1.5 Concluding Remarks.- 1.6 Problems.- 2 Graphical Optimization.- 2.1 Procedure for Graphical Optimization.- 2.2 GraphicalSolution function.- 2.3 Graphical Optimization Examples.- 2.4 Problems.- 3 Mathematical Preliminaries.- 3.1 Vectors and Matrices.- 3.2 Approximation Using the Taylor Series.- 3.3 Solution of Nonlinear Equations.- 3.4 Quadratic Forms.- 3.5 Convex Functions and Convex Optimization Problems.- 3.6 Problems.- 4 Optimality Conditions.- 4.1 Optimality Conditions for Unconstrained Problems.- 4.2 The Additive Property of Constraints.- 4.3 Karush-Kuhn-Tucker (KT) Conditions.- 4.4 Geometric Interpretation of KT Conditions.- 4.5 Sensitivity Analysis.- 4.6 Optimality Conditions for Convex Problems.- 4.7 Second-Order Sufficient Conditions.- 4.8 Lagrangian Duality.- 4.9 Problems.- 5 Unconstrained Problems.- 5.1 Descent direction.- 5.2 Line Search Techniques—Step Length Calculations.- 5.3 Unconstrained Minimization Techniques.- 5.4 Concluding Remarks.- 5.5 Problems.- 6 Linear Programming.- 6.1 The Standard LP Problem.- 6.2 Solving a Linear System of Equations.- 6.3 Basic Solutions of an LP Problem.- 6.4 The Simplex Method.- 6.5 Unusual Situations Arising During the Simplex Solution.- 6.6 Post-Optimality Analysis.- 6.7 The Revised Simplex Method.- 6.8 Sensitivity Analysis Using the Revised Simplex Method.- 6.9 Concluding Remarks.- 6.10 Problems.- 7 Interior Point Methods.- 7.1 Optimality Conditions for Standard LP.- 7.2 The Primal Affine Scaling Method.- 7.3 The Primal-Dual Interior Point Method.- 7.4 Concluding Remarks.- 7.5 Appendix—Null and Range Spaces.- 7.6 Problems.- 8 Quadratic Programming.- 8.1 KT Conditions for Standard QP.- 8.2 The Primal Affine Scaling Method for Convex QP.- 8.3 The Primal-Dual Method for Convex QP.- 8.4 Active Set Method.- 8.5 Active Set Method for the Dual QP Problem.- 8.6 Appendix—Derivation of the Descent Direction Formula for the PAS Method.- 8.7 Problems.- 9 Constrained Nonlinear Problems.- 9.1 Normalization.- 9.2 Penalty Methods.- 9.3 Linearization of a Nonlinear Problem.- 9.4 Sequential Linear Programming—SLP.- 9.5 Basic Sequential Quadratic Programming—SQP.- 9.6 Refined SQP Methods.- 9.7 Problems.- A.1 Basic Manipulations in Mathematica.- A.2 Lists and Matrices.- A.3 Solving Equations.- A.7 Online Help.