Skip to content

Lesson 43: Constrained Optimisation: Graphical Solution and Lagrange Multipliers

43.1 Learning Outcomes

By the end of this hour a student will be able to:

  1. Solve a two-variable constrained problem graphically using the tangency condition.
  2. Form the Lagrangian and derive the first-order conditions for equality constraints.
  3. Interpret \(\lambda\) as the shadow price — the rate of change of \(f^{*}\) with the RHS.
  4. Apply the bordered Hessian test to classify a constrained stationary point.
  5. Extend the method to several equality constraints.

43.2 The Problem and the Geometry

\[\begin{aligned} \text{minimise (or maximise)}\quad & f(\mathbf{x})\\ \text{subject to}\quad & h_j(\mathbf{x})=0,\quad j=1,\dots,p\quad (p<n) \end{aligned}\]

Lesson 37's condition \(\nabla f=\mathbf{0}\) no longer applies: the constrained optimum is generally at a point where the gradient is not zero, because moving downhill would leave the feasible set.

The tangency condition

     x2
      |        contours of f
      |     _________
      |    /  ______ \        h(x)=0
      |   /  /  __  \ \      /
      |  |  |  ( * ) |  |---/------
      |  |  |   \__/ |  |  /  <- tangency: the constraint curve
      |   \  \______/  / /       just touches a contour
      |____\__________/_/________ x1

At a constrained optimum the constraint curve is tangent to a contour of \(f\). If it crossed a contour, one could slide along the constraint to a lower contour, so the point would not be optimal.

Tangency means the two curves have parallel normals:

\[\boxed{\ \nabla f(\mathbf{x}^{*})=-\lambda\,\nabla h(\mathbf{x}^{*})\ }\tag{1.1}\]

for some scalar \(\lambda\). The sign and the symbol are conventions; the content is parallelism.

43.3 The Method of Lagrange Multipliers

Definition (Lagrangian). For one constraint, $\(L(\mathbf{x},\lambda)=f(\mathbf{x})+\lambda\,h(\mathbf{x})\)$ and in general $\(L(\mathbf{x},\boldsymbol\lambda)=f(\mathbf{x})+\sum_{j=1}^{p}\lambda_j h_j(\mathbf{x}).\)$

First-order necessary conditions. At a constrained optimum, $\(\frac{\partial L}{\partial x_i}=0\ (i=1,\dots,n),\qquad \frac{\partial L}{\partial \lambda_j}=h_j(\mathbf{x})=0\ (j=1,\dots,p).\)$

The second group simply restates the constraints, so the Lagrangian converts a constrained problem in \(n\) variables into an unconstrained one in \(n+p\) variables. That single idea is the whole method.

A caution — the constraint qualification. The conditions require the constraint gradients \(\nabla h_j(\mathbf{x}^{*})\) to be linearly independent. Where they are not (a cusp, or duplicated constraints), (1.1) can fail even at a genuine optimum. This regularity assumption reappears, more prominently, in Lesson 44.

The meaning of λ

Theorem (sensitivity). If the constraint is \(h(\mathbf{x})=b\) and \(f^{*}(b)\) is the optimal value, then $\(\boxed{\ \frac{df^{*}}{db}=-\lambda\ }\tag{2.1}\)$ (with the sign convention \(L=f+\lambda(h-b)\)).

\(|\lambda|\) is therefore the shadow price of the constraint — the improvement in the objective per unit of relaxation. This is precisely the dual variable of Lesson 7, now in a non-linear setting, and it is usually the number a decision-maker actually wants.

43.4 Second-Order Conditions: the Bordered Hessian

The first-order conditions locate candidates; they do not say whether a candidate is a minimum, a maximum or neither. The test uses the bordered Hessian:

\[\bar H=\begin{pmatrix} 0 & \dfrac{\partial h}{\partial x_1} & \dfrac{\partial h}{\partial x_2}\\[6pt] \dfrac{\partial h}{\partial x_1} & \dfrac{\partial^{2}L}{\partial x_1^{2}} & \dfrac{\partial^{2}L}{\partial x_1\partial x_2}\\[6pt] \dfrac{\partial h}{\partial x_2} & \dfrac{\partial^{2}L}{\partial x_2\partial x_1} & \dfrac{\partial^{2}L}{\partial x_2^{2}} \end{pmatrix}\]

Test for \(n=2\), \(p=1\). $\(\det\bar H<0\ \Rightarrow\ \text{constrained local MINIMUM},\)$ $\(\det\bar H>0\ \Rightarrow\ \text{constrained local MAXIMUM}.\)$

Note the reversed signs compared with the ordinary Hessian test of Lesson 37 — a standard source of error. The reason is that the test is really about the sign of \(\mathbf{d}^{T}\nabla^{2}L\,\mathbf{d}\) restricted to directions \(\mathbf{d}\) tangent to the constraint, and the border encodes that restriction with a sign flip.

Shortcut. If \(f\) is convex and the constraints are linear, the feasible set is convex and any stationary point of \(L\) is a global minimum. No bordered Hessian is needed. This covers most problems met in practice, including all of §4 below except Problem 2.

43.5 Solved Problems

Problem 1 — The basic method

Minimise \(f=x_1^{2}+x_2^{2}\) subject to \(x_1+x_2=10\).

Solution.

Step 1 — the Lagrangian.

\[L=x_1^{2}+x_2^{2}+\lambda(x_1+x_2-10)\]

Step 2 — first-order conditions.

\[\frac{\partial L}{\partial x_1}=2x_1+\lambda=0,\qquad \frac{\partial L}{\partial x_2}=2x_2+\lambda=0,\qquad \frac{\partial L}{\partial \lambda}=x_1+x_2-10=0\]

Step 3 — solve. The first two give \(x_1=x_2=-\lambda/2\). Substituting into the third: \(-\lambda=10\), so \(\lambda=-10\) and

\[\boxed{x_1^{*}=x_2^{*}=5,\qquad f^{*}=25+25=50,\qquad \lambda=-10}\]

Step 4 — verify with the bordered Hessian. \(\nabla h=(1,1)\) and \(\nabla^{2}L=\begin{pmatrix}2&0\\0&2\end{pmatrix}\):

\[\det\bar H=\det\begin{pmatrix}0&1&1\\1&2&0\\1&0&2\end{pmatrix} =0(4)-1(2-0)+1(0-2)=-4<0\ \Rightarrow\ \textbf{minimum}\ \checkmark\]

Step 5 — interpret \(\lambda\). By (2.1), \(\dfrac{df^{*}}{db}=-\lambda=10\). Check directly: with \(x_1+x_2=b\) the optimum is \(x_1=x_2=b/2\) and \(f^{*}=b^{2}/2\), so \(df^{*}/db=b=10\) at \(b=10\) ✓.

Geometric reading. The contours of \(f\) are circles centred at the origin; the constraint is a straight line. The smallest circle touching the line does so at the foot of the perpendicular from the origin, which is \((5,5)\). The Lagrange conditions are the algebra of that tangency.

Problem 2 — Maximisation with three variables

Maximise \(f=x_1x_2x_3\) subject to \(x_1+x_2+x_3=12\), with \(x_i>0\).

Solution.

\[L=x_1x_2x_3+\lambda(x_1+x_2+x_3-12)\]
\[\frac{\partial L}{\partial x_1}=x_2x_3+\lambda=0,\quad \frac{\partial L}{\partial x_2}=x_1x_3+\lambda=0,\quad \frac{\partial L}{\partial x_3}=x_1x_2+\lambda=0\]

Subtracting the first two: \(x_3(x_2-x_1)=0\), and since \(x_3>0\), \(x_1=x_2\). Similarly \(x_2=x_3\). The constraint then gives \(3x_1=12\):

\[\boxed{x_1^{*}=x_2^{*}=x_3^{*}=4,\qquad f^{*}=64,\qquad \lambda=-16}\]

Interpretation. This is the classical result that a product is maximised when the factors are equal — the AM–GM inequality, recovered by calculus. It answers the practical question "what rectangular box of given total edge length has the largest volume?": a cube.

The multiplier \(\lambda=-16\) says that one extra unit of total length would raise the maximum volume by \(16\). Check: with total \(13\) the optimum is \((13/3)^{3}=81.37\), an increase of \(17.37\) — close to \(16\), the difference being second-order curvature. \(\lambda\) is a derivative, so it is exact only for infinitesimal changes.

Problem 3 — Sensitivity in action

Minimise \(f=4x_1^{2}+5x_2^{2}\) subject to \(2x_1+3x_2=6\). Then predict the optimum for a right-hand side of \(7\) and check.

Solution.

\[L=4x_1^{2}+5x_2^{2}+\lambda(2x_1+3x_2-6)\]
\[8x_1+2\lambda=0\ \Rightarrow\ x_1=-\frac{\lambda}{4};\qquad 10x_2+3\lambda=0\ \Rightarrow\ x_2=-\frac{3\lambda}{10}\]

Substituting into the constraint:

\[2\left(-\frac{\lambda}{4}\right)+3\left(-\frac{3\lambda}{10}\right) =-\frac{\lambda}{2}-\frac{9\lambda}{10}=-\frac{14\lambda}{10}=6 \ \Rightarrow\ \lambda=-\frac{30}{7}=-4.28571\]
\[\boxed{x_1^{*}=\frac{15}{14}=1.07143,\quad x_2^{*}=\frac{9}{7}=1.28571,\quad f^{*}=\frac{90}{7}=12.85714}\]

Check the objective. \(4(1.07143)^{2}+5(1.28571)^{2}=4.59184+8.26531=12.85714\) ✓

Prediction for \(b=7\). \(\dfrac{df^{*}}{db}=-\lambda=\dfrac{30}{7}=4.28571\), so

\[f^{*}(7)\approx12.85714+4.28571=17.14286\]

Exact answer. Solving again with \(b=7\) (or noting \(f^{*}(b)=\tfrac{5}{14}b^{2}\)):

\[f^{*}(7)=\frac{5}{14}(49)=17.50000\]

Interpretation. The linear prediction is \(17.143\) against the true \(17.500\) — an error of \(2\,\%\) for a \(17\,\%\) change in the right-hand side. \(\lambda\) gives the tangent, and the tangent under-predicts for a convex \(f^{*}\). Exactly as with the 100 % rule of Lesson 7, a shadow price is reliable for small changes and must be recomputed for large ones. Here \(f^{*}\) is quadratic in \(b\), so the curvature shows up immediately.

Problem 4 — Two constraints

Minimise \(f=x_1^{2}+x_2^{2}+x_3^{2}\) subject to \(x_1+x_2+x_3=9\) and \(x_1-x_2=1\).

Solution. Two constraints require two multipliers:

\[L=x_1^{2}+x_2^{2}+x_3^{2}+\lambda_1(x_1+x_2+x_3-9)+\lambda_2(x_1-x_2-1)\]
\[2x_1+\lambda_1+\lambda_2=0,\qquad 2x_2+\lambda_1-\lambda_2=0,\qquad 2x_3+\lambda_1=0\]

From these, \(x_3=-\lambda_1/2\) and \(x_1+x_2=-\lambda_1\) (adding the first two), so \(x_1+x_2=2x_3\). With \(x_1+x_2+x_3=9\): \(3x_3=9\), giving \(x_3=3\), \(x_1+x_2=6\). Combined with \(x_1-x_2=1\):

\[\boxed{x_1^{*}=3.5,\quad x_2^{*}=2.5,\quad x_3^{*}=3,\qquad f^{*}=12.25+6.25+9=27.5}\]

Back-substituting: \(\lambda_1=-2x_3=-6\); then \(2(3.5)-6+\lambda_2=0\) gives \(\lambda_2=-1\).

Interpretation of the two multipliers.

  • \(-\lambda_1=6\): relaxing the total from \(9\) to \(10\) would raise \(f^{*}\) by about \(6\) — costly, since \(f\) is being minimised and a larger total forces larger squares.
  • \(-\lambda_2=1\): relaxing \(x_1-x_2=1\) to \(=2\) would raise \(f^{*}\) by about \(1\).

The second constraint is six times less binding than the first. With several constraints, the multipliers immediately rank them by importance, which is usually the most valuable output of the calculation — the same insight as the shadow-price analysis of Lesson 7.

Check the constraint qualification. \(\nabla h_1=(1,1,1)\) and \(\nabla h_2=(1,-1,0)\) are linearly independent ✓, so the conditions are valid.

Problem 5 — An applied problem: least-cost design

A closed cylindrical can must hold \(V=1000\) cm³. Material costs \(₹2\) per cm² for the curved surface and \(₹3\) per cm² for each circular end. Minimise the cost.

Solution.

Step 1 — formulate. With radius \(r\) and height \(h\):

\[\text{minimise } C=2(2\pi rh)+3(2\pi r^{2})=4\pi rh+6\pi r^{2}\]
\[\text{subject to } \pi r^{2}h=1000\]

Step 2 — Lagrangian.

\[L=4\pi rh+6\pi r^{2}+\lambda(\pi r^{2}h-1000)\]
\[\frac{\partial L}{\partial r}=4\pi h+12\pi r+2\lambda\pi rh=0 \tag{i}\]
\[\frac{\partial L}{\partial h}=4\pi r+\lambda\pi r^{2}=0 \tag{ii}\]

Step 3 — solve. From (ii), dividing by \(\pi r\ne0\): \(4+\lambda r=0\), so \(\lambda=-\dfrac{4}{r}\).

Substituting into (i) and dividing by \(\pi\): \(4h+12r+2\left(-\dfrac4r\right)rh=0\), i.e.

\[4h+12r-8h=0\ \Rightarrow\ 12r=4h\ \Rightarrow\ \boxed{h=3r}\]

Step 4 — apply the volume constraint. \(\pi r^{2}(3r)=1000\), so

\[r^{3}=\frac{1000}{3\pi}=106.1033\ \Rightarrow\ r^{*}=4.7342\text{ cm},\qquad h^{*}=14.2025\text{ cm}\]
\[C^{*}=4\pi(4.7342)(14.2025)+6\pi(4.7342)^{2}=844.92+422.46=\mathbf{₹1267.38}\]
\[\lambda=-\frac{4}{4.7342}=-0.84492\]

Step 5 — interpret.

  • The shape rule \(h=3r\) is independent of the volume. Whatever the required capacity, the optimal can has height three times its radius. Notice too that \(h/r=3=2\times(3/2)\): the ratio is exactly twice the ratio of end cost to side cost. With equal costs the rule becomes \(h=2r\) — the familiar result that the cheapest can is as tall as it is wide.
  • \(-\lambda=0.8449\) says that requiring one extra cm³ of capacity costs about \(₹0.84\). For a \(10\,\%\) larger can (\(100\) cm³ more) the linear estimate is \(₹84.49\); the exact recomputation gives \(r=4.8870\), \(h=14.6609\), \(C=₹1350.53\), an increase of \(₹83.14\). The tangent over-predicts by \(1.6\,\%\) because \(C^{*}\propto V^{2/3}\) is concave in \(V\).
  • The cost splits \(₹844.92\) on the side and \(₹422.46\) on the ends: exactly \(2:1\). That is not a coincidence — it follows from \(h=3r\) and is the constrained analogue of the "equal costs at the optimum" property of the EOQ in Lesson 31.

43.6 Practice Problems

  1. Minimise \(f=x_1^{2}+2x_2^{2}\) subject to \(x_1+x_2=6\). (Answer: \(\lambda=-8\), \(x^{*}=(4,2)\), \(f^{*}=24\))
  2. Maximise \(f=x_1x_2\) subject to \(2x_1+3x_2=12\). (Answer: \(x^{*}=(3,2)\), \(f^{*}=6\), \(\lambda=-1\))
  3. Find the point on the plane \(x_1+2x_2+3x_3=14\) closest to the origin. (Answer: minimise \(x_1^2+x_2^2+x_3^2\); \(x^{*}=(1,2,3)\), distance \(\sqrt{14}=3.742\))
  4. In Problem 3 of §4, use \(\lambda\) to predict \(f^{*}\) when \(b=6.5\), and compare with the exact value. (Answer: predicted \(12.857+0.5(4.286)=15.000\); exact \(\tfrac{5}{14}(42.25)=15.089\) — a \(0.6\,\%\) error)
  5. A rectangular box without a lid has volume \(32\) m³. Minimise its surface area. (Answer: \(L=W=4\), \(H=2\); area \(=48\) m²)

43.7 Summary

  • At a constrained optimum the constraint curve is tangent to a contour of \(f\), i.e. \(\nabla f=-\lambda\nabla h\).
  • The Lagrangian \(L=f+\sum_j\lambda_jh_j\) converts a constrained problem in \(n\) variables into an unconstrained one in \(n+p\); setting \(\partial L/\partial\lambda_j=0\) just restates the constraints.
  • \(\boxed{df^{*}/db=-\lambda}\): \(|\lambda|\) is the shadow price, the non-linear counterpart of the LP dual variable. It is a tangent, so it is reliable only for small changes.
  • With several constraints, the multipliers rank the constraints by how binding they are — usually the most useful output.
  • The bordered Hessian classifies the candidate, with signs reversed relative to Lesson 37: \(\det\bar H<0\) is a minimum for \(n=2\), \(p=1\).
  • Shortcut: convex \(f\) with linear constraints \(\Rightarrow\) any stationary point of \(L\) is a global minimum, and no second-order test is needed.
  • The constraint qualification — linearly independent constraint gradients — is required for the conditions to be valid.
  • Equality constraints only. Inequalities need Lesson 44.

Next lesson: L44 — the Kuhn–Tucker (KKT) conditions for inequality constraints.

Practise now: unit4-nlp-search-lab.html → Lagrange Lab: drag a constraint line across a contour map and watch the tangency point and the value of \(\lambda\) update live.