A concave function is one in which, for any two points we choose, the value of the function in between those points is larger than the value of the line segment connecting the points. Mathematically, a function $f$ is concave if, for any $\x_1, \x_2$, in the domain of $f$,
\[f(w\x_1 + (1-w)\x_2) \ge wf(\x_1) + (1-w)f(\x_2)\]for all $w \in (0,1)$. The function $f$ is strictly concave if
\[f(w\x_1 + (1-w)\x_2) > wf(\x_1) + (1-w)f(\x_2).\]It is worth mentioning that the domain of $f$ must be a convex set in order for $f$ to be a concave function. Otherwise, the idea of concavity is undefined.
Properties
- A strictly concave function has at most one unique maximum.
- If a strictly concave function has a local maximum, then that point is the global maximum.
- A strictly concave function that attains its maximum is unimodal. (Concavity alone is not enough: a constant function is concave but has no unique mode, and a strictly concave function such as $\log x$ need not have a maximum at all.)
See here for proofs (concavity proofs are the same as convexity proofs, with the inequalities reversed).
Second-order conditions
For a twice differentiable function, concavity can be checked using its Hessian matrix: $f$ is concave if and only if its Hessian is negative semidefinite everywhere, and a negative definite Hessian is sufficient for strict concavity.
Theorem: Let $S \subset \real^d$ be an open convex set and let $f: S \to \real$ be twice differentiable. Then
- $f$ is concave if and only if $\nabla^2 f(\x) \gle \zero$ for all $\x \in S$
- If $\nabla^2 f(\x) \glt \zero$ for all $\x \in S$, then $f$ is strictly concave
Proof: Suppose $\nabla^2 f \gle \zero$ on $S$. For any $\x, \y \in S$, there exists $\tilde{\x}$ on the line segment connecting them (and therefore in $S$) such that
\[\begin{alignat*}{2} f(\y) &= f(\x) + \nabla f(\x) \tp (\y - \x) + \tfrac{1}{2} (\y - \x) \tp \nabla^2 f(\tilde{\x}) (\y - \x) &\hspace{4em}& \href{taylor-expansion.html}{\text{Taylor (Lagrange)}} \\ &\le f(\x) + \nabla f(\x) \tp (\y - \x) && \nabla^2 f(\tilde{\x}) \gle \zero \end{alignat*}\]Now let $\z = w\x_1 + (1-w)\x_2$. Applying this inequality with $\x = \z$ and $\y = \x_1, \x_2$, and taking the weighted average,
\[wf(\x_1) + (1-w)f(\x_2) \le f(\z) + \nabla f(\z) \tp \{w\x_1 + (1-w)\x_2 - \z\} = f(\z),\]so $f$ is concave. If $\nabla^2 f \glt \zero$, the inequality is strict whenever $\y \ne \x$, so $f$ is strictly concave.
Conversely, suppose that for some $\x \in S$, $\nabla^2 f(\x)$ is not negative semidefinite: there exists $\v$ with $\v \tp \nabla^2 f(\x) \v > 0$. Since $S$ is open, $\x \pm t\v \in S$ for small $t$, and averaging the two expansions,
\[\begin{alignat*}{2} \tfrac{1}{2}\{f(\x + t\v) + f(\x - t\v)\} - f(\x) &= \tfrac{1}{2} t^2 \v \tp \nabla^2 f(\x) \v + o(t^2) &\hspace{4em}& \href{taylor-expansion.html}{\text{Taylor (basic)}} \\ &> 0 && \text{for small enough } t, \end{alignat*}\]which contradicts concavity, since $\x$ is the midpoint of $\x + t\v$ and $\x - t\v$.
The converse of part 2 is not true: $f(x) = -x^4$ is strictly concave, but $f^{\prime\prime}(0) = 0$.
Log-concave
A function $f:\real^d \to \real$ is said to be log-concave if $\log f$ is concave.
By the above theorem, a twice differentiable likelihood is log-concave if and only if the information matrix is positive semidefinite for all $\bt$; if the information is positive definite for all $\bt$, the log-likelihood is strictly concave.
Many common parametric models are log-concave, including everything in the exponential family (with respect to their natural parameters).