MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose you have two simple Ar[1] series of the form $y_n=y_{n-1}+e_n$ and $x_n=x_{n-1}+m_n$, where $e_n$ and $m_n$ are normal white noise processes with no auto-correlation and $Corr(e_n,m_n)=p$. Then suppose we have possibly non-overlapping data for Y and X (IE, observation 10 exists for Y but not for X), and to avoid data generating process issues, assume that the distribution of missing data is random.

Is there any way to estimate p?

As a follow-up question, is there a way to easily generalize to a situation where $y_n$ and $x_n$ are observed with known normally distributed measurement error?

share|cite|improve this question
This is definitely no answer for Your question, but there is procedure called imputation, so maybe this would be some help for You: – kakaz Feb 22 '10 at 10:19
you should now ask your question here – robin girard Aug 9 '10 at 15:18

The paper "Application of Two-Directional Time Series Models to Replace Missing Data" offers two methods (not necessarily under your precise model), one that minimizes "the average error associated with the missing value" (the other I can't understand from the abstract).

Edit. I have changed my (old) answer to community wiki. Would someone please vote this up so that the bot that reposts those questions for which there are no upvoted answers stops recycling this one? Thanks.

share|cite|improve this answer
Thanks to whomever! – Joseph O'Rourke Sep 6 '10 at 18:29

I have one big series with about 280 data points with only a couple missing observations. Then I have a couple other series I want to correlate with it that are much more sparse, with something like 30%-60% missing data. It basically corresponds to correlating a national trend with trends in individual states.

Your idea sounds pretty straightforward, but I'm wondering how it can generalize if my observations are subject to sampling error.

A crude approach would be to separately run every series through a kalman filter and then interpolate the missing points with a brownian bridge, then estimate p with a gibbs sampler like you suggested. But I feel like if there is a correlation $p$, then I need to jointly filter the two series together in order to get an accurate estimate.

share|cite|improve this answer

have you tried some kind of data augmentation approach, since this is very easy to simulate the missing data (missing data ~ Brownian bridge): you can easily study the posterior distribution of $p$ through MCMC simulations (Gibbs sampler in this case), or if you are just interested in the maximum likelihood estimate, the EM algorithm seems to work. Btw, how large your data are ?

A nice article about data augmentation.

share|cite|improve this answer

yes, the usual approach is to use Kalman filter + EM algorithm. But be aware that there are known instability issues: basically, the Kalman filter works quite well if you know that your model is right (eg:for satellite tracking, Newton's laws are correct, and you add some noisy observations -> Kalman works quite well). Nevertheless, when you try to model time series (financial, economic, etc..) with a simple model+ unknown model parameters + noisy observations, Kalman filter gives poor results. I remember having tried such an approach with financial times series (GDP, FX rates, etc...), and we have had a hard time trying to stabilize the algorithm. You can have a try with MCMC methods, but that may be quite slow.

share|cite|improve this answer

You are right. An idependent filtering of the two signals will introduce errors because it is not contrained to fix the correlation to p. One possible approach is to perform a unified maximum likelihood estimation of both the missing samples and the correlation p. This can be done as follows: Assuming that the processes m_n and e_n have the same variance, hence we may write:

m_n = p * e_n + q * f_n, p^2 + q^2 = 1,

where f_n is a normal white noise uncorrelated to e_n and has the same variance as e_n.

The log likelihood function is proportional to:

sum_n((x_n - sum_i=1 to n(e_n))^2) + sum_n((y_n - sum_i=1 to n(p * e_n + q * f_n))^2) + lambda (p^2 + q^2 -1)

where lambda is a Lagrange multiplier, and the outer sums are of course over the known samples only.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.