3
$\begingroup$

I have a question about whether it is possible to use De Finetti's representation theorem for maximum likelihood estimation.

De Finnetti's theorem states that for any exchangable infinite sequence of random variables $(X_1,...,X_n,...)$, we can represent the probability distribution $f(X_1,...,X_n)$ as a mixture $$f(x_1,...,x_n) = \int \prod_{i=1}^n g(x_i | H) dP(H).$$

where the variables $x_1,...,x_n$ are conditionally independent given $H$ and $P(H)$ is a probability distribution over $H$.

I would like to theoretically show that $P(H)$ can be estimated if we observe a matrix $(x_{ij})_{i,j=1}^n$ where each row $(x_{i1},...,x_{in})$ is drawn independently from the distribution $f(x_1,...,x_n)$.

The idea would be to do a two step estimation:

  • For each i, estimate $H_i =_{def} argmax_H \prod_{i=1}^n g(x_i | H)$
  • Once we observe $H_1,H_2,H_3,...$ use these to estimate $P^* = argmax_P \prod_{i=1}^n Prob(H_i | P).$

It seems that for this idea to work, each $H_i$ would have to live in a compact set (otherwise, maximum likelihood estimation in the first step does not necessarily work). But since $H$ is drawn from $P(H)$ and $P$ does not necessarily have a compact support, this won't work.

Is there a way to parametrize $P$ to ensure it has compact support?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.