Constrained Optimization and Lagrange Multiplier Methods

Dimitri P. Bertsekas

This reference textbook, first published in 1982 by Academic Press, is a comprehensive treatment of some of the most widely used constrained optimization methods, including the augmented Lagrangian/multiplier and sequential quadratic programming methods.

Review :

"This is an excellent reference book. The author has done a great job in at least three directions. First, he expertly, systematically and with ever-present authority guides the reader through complicated areas of numerical optimization. This is achieved by carefully explaining and illustrating (by figures, if necessary) the underlying principles and theory. Second, he provides extensive guidance on the merits of various types of methods. This is extremely useful to practitioners. Finally, this is truly a state of the art book on numerical optimization."
S. Zlobec, McGill University, in SIAM Review

The book may be downloaded from here or can be purchased from the publishing company, Athena Scientific. (ISBN 1-886529--04-3, 400 pages, softcover)

Constrained Optimization and Lagrange Multiplier Methods

  1. Introduction

    1. General Remarks
    2. Notation and Mathematical Background
    3. Unconstrained Optimization
    4. Constrained Minimization
    5. Algorithms for Minimization Subject to Simple Constraints
    6. Notes and Sources

  2. The Method of Multipliers for Equality Constrained Problems

    1. The Quadratic Penalty Function Method
    2. The Original Method of Multipliers
    3. Duality Framework for the Method of Multipliers
    4. Multiplier Methods with Partial Elimination of Constraints
    5. Asymptotically Exact Minimization in the Method of Multipliers
    6. Primal-Dual Methods Not Utilizing a Penalty Function
    7. Label Correcting Methods
    8. Notes and Sources

  3. The Method of Multipliers for Inequality Constrained and Nondifferentiable Optimization Problems

    1. One-Sided Inequality Constraints
    2. Two-Sided Inequality Constraints
    3. Approximation Procedures for Nondifferentiable and Ill-Conditioned Optimization Problems
    4. Notes and Sources

  4. Exact Penalty Methods and Lagrangian Methods

    1. Nondifferentiable Exact Penalty Functions
    2. Linearization Algorithms Based on Nondifferentiable Exact Penalty Functions
    3. Differentiable Exact Penalty Functions
    4. Lagrangian Methods - Local Convergence
    5. Lagrangian Methods - Global Convergence
    6. Notes and Sources

  5. Nonquadratic Penalty Functions - Convex Programming

    1. Classes of Penalty Functions and Corresponding Methods of Multipliers
    2. Convex Programming and Duality
    3. Convergence Analysis of Multiplier Methods
    4. Rate of Convergence Analysis
    5. Conditions for Penalty Methods to be Exact
    6. Large Scale Integer Programming Problems and the Exponential Method of Multipliers
    7. Notes and Sources

  6. References

  7. Index