Why is quadratic penalty method used for equaltiy-constrained optimization?

Question

When one equality-constrained optimization is formulated, the method of Lagrange multiplier will be the choice for me. In Chapter 17 from the book Numerical Optimization, quadratic penalty method can be used for such case. However, it doesn't mention when one should select quadratic penalty method over method of Lagrange multiplier. I hope to know the advantage and disadvantage of the two methods and how one select the method for one optimization problem.

Wolfgang Bangerth · Answer

The quadratic penalty is just easy to implement if you already have a solver for unconstrained problems. It converts the problem with constraints into an unconstrained one. It doesn't get any simpler. The penalty formulation also doesn't care about details such as whether the constraints are differentiable or not.
On the other hand, implementing a formulation that includes Lagrange multipliers substantially complicates the software. It also requires differentiability of the constraint functions and, furthermore, that these derivatives are computable in practice. It does, however, have the advantage, that at least for linear constraints, you can guarantee that all iterates satisfy the constraints exactly.
In other words, it's all about the various trade-offs involved.

Why is quadratic penalty method used for equaltiy-constrained optimization?

One Answer

Add your own answers!

Ask a Question