The following are the classical conditions to ensure that the likelihood is “regular”, meaning that at least asymptotically, the likelihood resembles that of a normal distribution. Note: saying “the classical conditions” is perhaps misleading, as there is some flexibility with respect to how these conditions are stated. Some additional commentary on this point is given below.

Core conditions

(A) IID: $X_1, \ldots, X_n$ are iid with density $p(x \vert \bts)$.

(B) Interior point: There exists an open set $\bTs \subset \bT \subset \real^d$ that contains $\bts$.

(C) Smoothness: For all $x$, $\ell(\bt \mid x)$ is continuously differentiable with respect to $\bt$ up to third order on $\bTs$, and satisfies the following conditions:

• (i) Derivatives up to second order can be passed under the integral sign in $\int dP(x \vert \bt)$.
• (ii) The Fisher information $\fI(\bts)$ is positive definite.
• (iii) The third derivatives $\nabla^3 \ell(\bt \vert x)$ are bounded by $M(x)$ on $\bTs$: $\sup_{\bt \in \bTs} \abs{\nabla^3 \ell(\bt \mid x)_{jkm}} \le M(x)$ for all $j, k, m$, with $\Ex M(X) < \infty$

Note #1: Condition (C) describes what happens for a single observation. What happens in a random sample of $n$ observations is governed by condition (A).

Note #2: Note that C(ii) applies to the Fisher information, while C(iii) applies to the derivative of the observed information. This is important! The observed information might randomly fail to be positive definite, but we don’t need to worry about that (asymptotically). Meanwhile, we need a bound on the observed derivatives, which can include $x$. This means that our bound $M(x)$ must be allowed to be random. Note that $\Ex M(X)$, however, is not random; also note that $\Ex M(x)$ is taken with respect to $p(x \mid \bts)$.

Note #3: Although not explicitly stated, the above conditions also ensure that the score and observed information are continuous functions of $\bt$: all differentiable functions are continuous, so by requiring the third derivative to exist, we require that the first and second derivatives are continuous. In fact, because the bound $M(x)$ in C(iii) does not depend on $\bt$, the conditions ensure something stronger: the observed information is (asymptotically) uniformly continuous over $\bTs$, so that $\tfrac{1}{n}\oI_n(\bth) \inP \fI(\bts)$ for any consistent estimator $\bth$. See information continuity for details, including why the analogous statement for the Fisher information is not guaranteed by these conditions.

Concavity

The above conditions apply only locally (within a neighborhood of $\bts$) and thus do not guarantee anything about the MLE, only about a local maximum near $\bts$. To guarantee consistency and asymptotic normality of the MLE, the following global condition is needed.

(D) Concavity: $\bT$ is a convex set and $\ell(\bt \mid x)$ is concave in $\bt$ for all $x$.

In other words, the likelihood is log-concave. When $\ell(\bt \mid x)$ is twice differentiable, this is equivalent to the information $\oI(\bt \mid x)$ being positive semidefinite for all $x$ and all $\bt \in \bT$, which is usually the easiest way to check it. A common, slightly stronger version of this condition requires the information to be positive definite; this makes the log-likelihood strictly concave, and therefore the MLE unique, but it isn’t needed for consistency or asymptotic normality.

Alternative statements

The above conditions are one way to state the conditions required for asymptotic normality of the MLE, but there is some flexibility. For example, the only reason condition C(i) is necessary is to ensure that $\Ex \u(\bts) = \zero$ and $\Ex \oI(\bts) = \fI(\bts)$. Thus, some authors will just state these conditions regarding the expected value of score and information directly rather than state necessary conditions for them.

Similarly, the purpose of C(iii) is to establish a uniform bound on the observed information over $\bTs$ – the core idea of the condition is that we have a single bound $M(x)$ that works for all values of $\bt \in \bTs$. Bounding the third derivative is one way of accomplishing that. It is possible to relax this condition and require a uniform bound only on the second derivatives (plus some other conditions), although the resulting proofs of consistency and asymptotic normality become more complicated. For this reason, the simpler regularity conditions presented above are more common in practice.

Finally, the IID assumption merely presents a basic, standard case in which likelihood theory holds. Certainly, likelihood estimates are asymptotically normal in all manner of non-IID settings (multiple groups, regression, etc.) as well – likelihood theory would not be very useful if this were not true. If one understands how the theory works in the IID case, it is typically relatively straightforward to extend theoretical results to other cases.