$$

Given a convex function $f$, its convex conjugate $f^{\ast }(a)$ gives the (negative) intercept of the tangent to $f$ of slope $a$:

$$f^{\ast }(a)=\underset{t}{sup}(at-f(t))$$

#figure

This gives an alternate way to describe $f$, not as a series of points $(t,f(t))$ drawing its curve, but as series of lines $\{at-f^{\ast }(a)|t\in \mathbb{R}\}$ hugging its curve.

Duality

Put another way, $f^{\ast }$ is minimized under the constraint that

$$\begin{array}{}\text{(}\ast \text{)}& f(t)+f^{\ast }(a)\ge at,\end{array}$$

which shows by symmetry that $f$ is also $f^{\ast }$’s convex conjugate: indeed, there is also no way to make $f$ smaller without lowering one of its tangents, which would increase $f^{\ast }(a)$, so $f$ is also minimized under constraint $(\ast )$.

Derivatives

For a moment, suppose that $f$ is derivable and strictly convex, which means $f^{\prime }(t)$ exists and is smoothly increasing.

Consider the tangent of slope $a$, and suppose it touches $f$ at $t$ (which means $\frac{\mathrm{d}f}{\mathrm{d}t}(t)=a$). Then intuitively, if we slightly increase the slope $a$, gliding the tangent further along $f$, the intercept will decrease at a rate proportional to $t$ since it is a distance $t$ away, meaning that $\frac{\mathrm{d}{f}^{\ast }}{\mathrm{d}a}(a)=t$. That is, the derivatives of $f$ and $f^{\ast }$ are inverses of each other, and in particular this shows that $f^{\ast }$ is also convex.

#figure

Slightly more formally, since $f$ is smooth, we have $f(t^{\prime })\approx a{t}^{\prime }-f^{\ast }(a)$ for $t^{\prime }$ close to $t$, so for a small slope increase $ϵ$, if we take the point $t^{\prime }$ such that $\frac{\mathrm{d}f}{\mathrm{d}t}(t^{\prime })=a+ϵ$, we have

$$\begin{array}{rl}{f}^{\ast }(a+ϵ)& =(a+ϵ)t^{\prime }-f(t^{\prime })\\ & \approx f^{\ast }(a)+ϵt^{\prime }\\ & \approx f^{\ast }(a)+ϵt,\end{array}$$

which shows that $\frac{\mathrm{d}{f}^{\ast }}{\mathrm{d}a}(a)=t$ as desired.