Consider a Wiener process with zero drift, infintesimal variance $\sigma^2$, and an unknown starting value $\nu$. That is, \begin{align} Y_t \sim \mathcal{N}(\nu, t\sigma^2). \end{align}

Now, suppose that we don't observe the $Y_i$ directly, but rather have a corresponding Gaussian likelihood for each $Y_i$: $X_i$ having mean $\mu_i$ and precision $\lambda_i$.

Question 1: If we know $\sigma$ and have $X_{i\le t}$, what is the posterior on $Y_t$?

Intuitively, I think we modify each $X_{t-i}$ by adding $i\sigma^2$ to its variance, and then the combined likelihood on $Y_t$ is Gaussian with mean and precision: \begin{align} \mu^\star &= \frac{\sum_{i\le t}\mu_i\lambda_i}{\lambda^\star} \\\\ \lambda^\star &= \sum_{i\le t}\lambda_i. \end{align}

Question 2: If we don't know $\sigma$, but have $X_{i\le t}$, what is the posterior on $Y_t$ and $\sigma$?

It seems that we would want to estimate $\sigma$ by setting a gamma prior on $\sigma^{-2}$. (That's what we would do in the special case that each of the $X_i$ has zero variance.) I'm having trouble proceeding from here. (Doesn't this make the combined likelihood on $Y_t$ student's t-distributed?)

  • 1
    $\begingroup$ en.wikipedia.org/wiki/Kalman_filter $\endgroup$ – Steve Huntsman Feb 9 '12 at 23:53
  • $\begingroup$ @Steve: It's a good comparison, thanks. Usually, Kalman filters hold the process noise constant. The wikipedia page references “Autocovariance Least-Squares (ALS)” as a way of estimating it. I looked into those papers to try to make sense of how to do that in the simple one-dimensional case that I have. I only spent a bit of time looking at it, but it seemed like they are trying many hypothetical covariances out (or lagged copies of the input?) and then choosing the one that has the least “validation error”, which is similar to what I'm doing in my experiments. $\endgroup$ – Neil Feb 13 '12 at 12:37
  • $\begingroup$ Put a prior on $\sigma$, such as a relatively uninformative inverse gamma distribution, then integrate the posterior over the distribution of $\sigma$ $\endgroup$ – Arthur B Sep 10 '14 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.