I have a question about whether it is possible to use De Finetti's representation theorem for maximum likelihood estimation.

De Finnetti's theorem states that for any exchangable infinite sequence of random variables $(X_1,...,X_n,...)$, we can represent the probability distribution $f(X_1,...,X_n)$ as a mixture $$f(x_1,...,x_n) = \int \prod_{i=1}^n g(x_i | H) dP(H).$$

where the variables $x_1,...,x_n$ are conditionally independent given $H$ and $P(H)$ is a probability distribution over $H$.

I would like to theoretically show that $P(H)$ can be estimated if we observe a matrix $(x_{ij})_{i,j=1}^n$ where each row $(x_{i1},...,x_{in})$ is drawn independently from the distribution $f(x_1,...,x_n)$.

The idea would be to do a two step estimation:

  • For each i, estimate $H_i =_{def} argmax_H \prod_{i=1}^n g(x_i | H)$
  • Once we observe $H_1,H_2,H_3,...$ use these to estimate $P^* = argmax_P \prod_{i=1}^n Prob(H_i | P).$

It seems that for this idea to work, each $H_i$ would have to live in a compact set (otherwise, maximum likelihood estimation in the first step does not necessarily work). But since $H$ is drawn from $P(H)$ and $P$ does not necessarily have a compact support, this won't work.

Is there a way to parametrize $P$ to ensure it has compact support?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.