I've posted this already in stats.stackexchange. I'm not sure what the rules are for cross-posting but mathoverflow seems to be more active.

Suppose we have data $x_i, i=1,2,3,...n$ that are dependent and identically distributed with marginal $f(\cdot|\alpha)$. If we model this with the likelihood

$ L = c(F(x_1|\alpha),F(x_2|\alpha),...F(x_n|\alpha)|\theta)\prod_{i=1}^n f(x_i|\alpha) $

and the dependence parameter $\theta$ is known, can we apply some variant of the Expectation Maximization algorithm to estimate $\alpha$ using an iterative procedure with relatively simple steps?

For instance, I considered a simple problem with exponential marginals and Gaussian copula (with known correlation), and did something procedural (and hokey). I introduced the unknown independent samples $\tilde{x}_i, i=1,2,...,n$ which you would compute by knowing the correct value of $\alpha$, mapping the $x_i$ to correlated Gaussians $y_i = \Phi^{-1}(F(x_i|\alpha))$ and then "undoing" the correlations $z = B^{-1}y$ and mapping the $z$ forward again to produce the independent $\tilde{x_i}=F^{-1}(\Phi(z_i)|\alpha)$. Here $C=BB^T$ is the correlation matrix. Here $\Phi$ is the (0,1)-normal cdf. If you turn this into an iterative procedure, using $x$ as the initial guess for the independent data, it seems to produce a series that (at least in my trials) converged. However, the whole thing is doubtful since it depends entirely on what you choose for $B$ (only defined up to a unitary matrix). I think it's the unitary invariance of the Gaussian hitting you when you try to basically do an inversion.

Is there an obvious way to turn this kind of problem into a sane iterative procedure using simple steps like the EM? I feel like I'm missing a simple trick.

  • 1
    $\begingroup$ 2 hours is definitely not long enough to wait for an answer. These communities are international, so you should at least wait a day, if not two, before cross-posting if it is warranted. The reason for not cross-posting is that you are not just asking for yourself, but to also be a future reference/resource for others asking the same question. If questions and answers get spread over multiple sites it's hard to track them down. For everyone else, here is the question on stats.sx stats.stackexchange.com/questions/35145 $\endgroup$ – David Roberts Aug 27 '12 at 3:14
  • $\begingroup$ Ideally we could close this question and if you get no answers at stats.sx in a few days, you could ask for this to be reopened. $\endgroup$ – David Roberts Aug 27 '12 at 3:15

The maximization by parts algorithm for maximum likelihood estimation is capable of exactly handling that situation

Peter X.-K Song, Yanqin Fan & John D Kalbfleisch (2005), Maximization by Parts in Likelihood Inference, Journal of the American Statistical Association.

One of the examples in that paper is slightly more general than the copula setting in your question (it does not assume $\theta$ is known, so I would expect the implementation in your case to be simpler than that example).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.