MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've posted this already in stats.stackexchange. I'm not sure what the rules are for cross-posting but mathoverflow seems to be more active.

Suppose we have data $x_i, i=1,2,3,...n$ that are dependent and identically distributed with marginal $f(\cdot|\alpha)$. If we model this with the likelihood

$ L = c(F(x_1|\alpha),F(x_2|\alpha),...F(x_n|\alpha)|\theta)\prod_{i=1}^n f(x_i|\alpha) $

and the dependence parameter $\theta$ is known, can we apply some variant of the Expectation Maximization algorithm to estimate $\alpha$ using an iterative procedure with relatively simple steps?

For instance, I considered a simple problem with exponential marginals and Gaussian copula (with known correlation), and did something procedural (and hokey). I introduced the unknown independent samples $\tilde{x}_i, i=1,2,...,n$ which you would compute by knowing the correct value of $\alpha$, mapping the $x_i$ to correlated Gaussians $y_i = \Phi^{-1}(F(x_i|\alpha))$ and then "undoing" the correlations $z = B^{-1}y$ and mapping the $z$ forward again to produce the independent $\tilde{x_i}=F^{-1}(\Phi(z_i)|\alpha)$. Here $C=BB^T$ is the correlation matrix. Here $\Phi$ is the (0,1)-normal cdf. If you turn this into an iterative procedure, using $x$ as the initial guess for the independent data, it seems to produce a series that (at least in my trials) converged. However, the whole thing is doubtful since it depends entirely on what you choose for $B$ (only defined up to a unitary matrix). I think it's the unitary invariance of the Gaussian hitting you when you try to basically do an inversion.

Is there an obvious way to turn this kind of problem into a sane iterative procedure using simple steps like the EM? I feel like I'm missing a simple trick.

share|cite|improve this question
2 hours is definitely not long enough to wait for an answer. These communities are international, so you should at least wait a day, if not two, before cross-posting if it is warranted. The reason for not cross-posting is that you are not just asking for yourself, but to also be a future reference/resource for others asking the same question. If questions and answers get spread over multiple sites it's hard to track them down. For everyone else, here is the question on – David Roberts Aug 27 '12 at 3:14
Ideally we could close this question and if you get no answers at in a few days, you could ask for this to be reopened. – David Roberts Aug 27 '12 at 3:15
up vote 1 down vote accepted

The maximization by parts algorithm for maximum likelihood estimation is capable of exactly handling that situation

Peter X.-K Song, Yanqin Fan & John D Kalbfleisch (2005), Maximization by Parts in Likelihood Inference, Journal of the American Statistical Association.

One of the examples in that paper is slightly more general than the copula setting in your question (it does not assume $\theta$ is known, so I would expect the implementation in your case to be simpler than that example).

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.