Skip to main content
5 of 8
simplified language
user avatar
user avatar

$\newcommand\th\theta\newcommand\R{\mathbb R}$Indeed, the densities $f_\th:=d\mu_\th/d\nu$ are defined only $\nu$-a.e., and hence it makes no sense to define a maximum likelihood estimate (MLE) as a maximizer of $f_\th(x)$ in $\th$ for arbitrary versions of the densities.

Proofs usually assume that versions $f_\th$ of the densities can be chosen to make the likelihood $L_x(\th):=f_\th(x)$ regular enough in $\th$.

  • Proofs that MLE is consistent standardly assume that the likelihood $L_x(\th)$ is continuous in $\th$.

  • Proofs of the central limit theorems for the MLE standardly assume that the likelihood $L_x(\th)$ is twice differentiable in $\th$.

  • Proofs may only discuss a family $(f_\th)$ (regular in $\th$ to some extent), ignoring the family $(d\mu_\th)=(f_\th\,d\nu)$, and avoiding the ambiguities of measures entirely.

There are some examples where $L_x(\th)$ can not be made continuous in $\th$. E.g., for real $\th>0$, let $\mu_\th$ be the joint distribution of $n$ iid random variables each uniformly distributed over $[0,\th]$, whose derivatives with respect to Lebesgue measure over $\R^n$ give the densities $f_\th$.

  • If we define $f_\th(x)=\th^{-n}\,1(\max_i x_i\le\th)$, and so the MLE is $\max_i x_i$.
  • if we define $f_\th(x)=\th^{-n}\,1(\max_i x_i<\th)$, then $L_x(\th)=f_\th(x)$ does not attain a maximum in $\th$ for any $x$, so that now an MLE will not exist, strictly speaking.

A generalized definition of an MLE is given in Section 2 of this paper. With that definition, even for $f_\th(x)=\th^{-n}\,1(\max_i x_i<\th)$, $\max_i x_i$ will be the generalized MLE.

Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229