Fenchel–Young inequality

Remarks

Since $f$ is finite somewhere, $f^*$ never takes the value $-\infty$, so the left side is a well-defined element of $(-\infty, +\infty]$. The proof is one line: $f^*(y) \ge y(x) - f(x)$ by the definition of $f^*$ as a supremum over $x$. Equality means that $x$ attains this supremum, which is the subgradient inequality rearranged. The set of subgradients is the subdifferential $\partial f(x)$.

Example: for $p > 1$ and $q = p/(p - 1)$, the function $f(x) = \abs{x}^p / p$ on $\R$ has $f^*(y) = \abs{y}^q / q$. The inequality becomes Young's inequality $xy \le \abs{x}^p/p + \abs{y}^q/q$, with equality exactly when $y = \sgn(x) \abs{x}^{p-1}$. For $p = 2$ it is $xy \le \tfrac12 x^2 + \tfrac12 y^2$, which is $(x - y)^2 \ge 0$.

Going deeper

For a student

The inequality gives weak duality in convex optimization: the value of a dual problem never exceeds the value of the primal problem. Its equality case links the conjugate with the subdifferential: $y \in \partial f(x)$ if and only if $f(x) + f^*(y) = y(x)$. For proper convex lower semicontinuous $f$, the Fenchel–Moreau theorem then gives the symmetric statement $x \in \partial f^*(y)$, so $\partial f^*$ is the inverse of $\partial f$. It is the source of many classical inequalities obtained by choosing $f$.