5.2 Maxima and minima

Optimization is the method of choosing an input or decision to make a quantity as large as possible (maximize) or as small as possible (minimize). The quantity being improved is the objective: height, profit, cost, distance, or error. This is especially useful in AI. Training a model often means choosing its adjustable parameters to minimize a loss that measures prediction error.

Why do optimization and differentiation appear together? The derivative tells us how the objective changes when we adjust an input slightly. At a smooth point with positive slope, moving right raises the value and moving left lowers it; with negative slope, the directions reverse. Such a point cannot be a highest or lowest point inside the allowed range. An interior smooth optimum must therefore have slope zero. Differentiation finds these candidates; derivative signs, the graph's shape, and the allowed boundaries tell us which candidate actually wins. This is why calculus is central to optimization over continuous choices.

This is intermediate calculus. Review Module 5.1 and OpenStax's maxima-and-minima section. The algebra of quadratics from Module 2.3 provides a useful check. A derivative is powerful for smooth continuous objectives, but a sharp corner, a boundary, or a discrete choice may need direct comparison too.

A basic idea: where could the direction change?

Suppose we want to maximize $f(x)=-(x-3)^2+10$ for $0\le x\le5$. The graph rises before $x=3$ and falls after it. Its derivative is $f'(x)=-2(x-3)$: positive to the left, zero at $3$, and negative to the right. Moving right helps before $3$ and hurts after $3$, so $x=3$ is a peak. Its value is $10$, higher than either endpoint: $f(0)=1$ and $f(5)=6$. It is the greatest allowed value.

A quick tour of candidate points

A critical point is an allowed interior input where the derivative is zero or does not exist. These are candidates for a local high or low point. A local maximum beats nearby values; an absolute maximum beats every allowed value. The same distinction applies to minima.

For a continuous function on a closed interval $[a,b]$, check:

1. Interior points where $f'(x)=0$.
2. Interior points where $f'(x)$ is undefined but $f(x)$ exists.
3. Both endpoints $a$ and $b$.

Evaluate $f$ at every candidate; the greatest and smallest values are the absolute extrema. Derivative signs help classify an interior point: $+\to-$ means a local maximum, and $-\to+$ means a local minimum. No sign change means a flat tangent without a turn, as for $x^3$ at zero. At a corner, such as $|x-2|$ at $x=2$, the derivative may fail to exist exactly where the minimum occurs.

In AI, a model can have many adjustable parameters rather than one $x$. The gradient collects the derivative with respect to each parameter; a small move opposite that direction tends to lower a differentiable loss. This is the many-variable version of following the slope downhill. The full gradient-based training method is beyond this one-variable introduction.

If the domain is open or unbounded, a maximum or minimum may not be attained. For instance, $f(x)=x$ on $0\le x<1$ approaches $1$ but never equals it. Module 5.4 applies this boundary check to constrained problems.

Application 1: the highest point of a flight model

A simplified height model is $h(t)=-t^2+6t+1$ metres for $0\le t\le5$ seconds. What is the greatest height on this interval?

1. Find interior candidates. $h'(t)=-2t+6=0$ at $t=3$.
2. Compare values. $h(0)=1$, $h(3)=10$, and $h(5)=6$ metres.
3. Choose the absolute maximum. The highest height is $10$ metres at $3$ seconds. The derivative changes from positive to negative there, agreeing with the comparison.

The interval endpoints matter even though the peak happens inside it.

Application 2: a sharp minimum

An error is $E(x)=|x-2|$ for $0\le x\le4$. Where is it smallest?

1. Read the distance. $E(x)$ is the distance from $x$ to $2$, so it cannot be negative.
2. Check the corner and endpoints. At $x=2$, $E=0$; at $x=0$ and $4$, $E=2$.
3. Conclude. The absolute minimum is $0$ at $x=2$. The graph has a corner there, so a method that searches only for $E'(x)=0$ would miss it.