Non-probabilistic version
Theorem (Jensen’s inequality): For $\a,\x \in \real^d$ with $a_i > 0$ for all $i$, if $g$ is a convex function, then
\[g\left( \frac{\sum_i a_i x_i}{\sum_i a_i} \right) \leq \frac{\sum\nolimits_i a_i g(x_i)}{\sum_i a_i}\]Proof: The Cauchy-Schwarz Master Class (2004), Steele JM. Cambridge University Press.
The inequality is reversed if $g$ is concave.
Probabilistic version
Theorem: If $g$ is a convex function defined over an interval $I$, and $X$ is a random variable with $\Pr(X \in I) = 1$ and finite expectation, then
\[g(\Ex(X)) \leq \Ex(g(X)).\]If $g$ is strictly convex, the inequality is strict unless $X$ is a constant with probability 1.
Proof: Since $\Ex(X)$ is finite and $\Pr(X \in I) = 1$, we have $\Ex(X) \in I$. Now, consider the point $\lbrace\Ex(X), g(\Ex(X))\rbrace$. Since $g$ is convex, there exists a line $L(x)$ tangent to this point such that $L(x) \leq g(x)$ for all $x \in I$. Then
\[\begin{align*} \Ex\{g(X)\} &\geq \Ex\{L(X)\} \\ &= L(\Ex(X)) \\ &= g(\Ex(X)), \end{align*}\]with the inequality becoming strict if convexity is strict, unless $X$ is a constant with probability 1.
The inequality is reversed if $g$ is concave.