1
$\begingroup$

I have some data generated using MCMC methods and in particular Gibbs sampling. I computed the autocorrelation but I'm unsure how to determine how many samples to skip.

I'd like to determine that quantity from the autocorrelation function that I computed and not just use a "reasonable" value. Any documents to help me? Direct help is also welcome.

Improve this question
$\endgroup$
2
  • $\begingroup$ You might find the book by Levin, Peres and Wilmer helpful. $\endgroup$
    – Steve Huntsman
    Mar 22, 2012 at 9:36
  • $\begingroup$ If you're interested in integrating over a function of your random variable, the ergodic theorem tells you that you can use the samples from your chain, even if they are dependent. Otherwise, there's no good answer to your question, correlation is only loosely related to independence. If you really really care about roughly independent samples, then as a rule of thumb the number of step you skip should be on the order of 1/log(corr) $\endgroup$
    – Arthur B
    Mar 23, 2012 at 19:58

2 Answers 2

Reset to default
1
$\begingroup$

To expand a bit on Arthur B.'s comment that you can use samples even if they are dependent, consider this quote from Andrew Gelman and Kenneth Shirley:

The purpose of thinning (i.e. setting n to some integer greater than 1) is computational, not statistical. If we have a model with 2000 parameters and we are running three chains with a million iterations each, we do not want to be carrying around 6 billion numbers in our simulation. The key is to realize that, if we really needed a million iterations, they must be so highly autocorrelated that little is gained by saving them all. In practice, we find it is generally more than enough to save 1000 iterations in total, and so we thin accordingly. But ultimately this will depend on the size of the model and computational constraints.

The full article is full of great practical recommendations for using MCMC for inference.

Improve this answer
$\endgroup$
2
$\begingroup$

I assume you refer to the practice of "thinning"; e.g. only using every $k$th sample.

This practice is often recommended in textbooks in order to deal with autocorrelation however it appears that using the entire chain (no thinning or skipping of samples) is nearly always better. The only reason to use this practice is because of memory management.

I have seen a few papers discussing why we should not thin, a short one is:

On thinning of chains in MCMC (2011), Link & Eaton http://onlinelibrary.wiley.com/doi/10.1111/j.2041-210X.2011.00131.x/full

(I hope I have not double posted this, it is my first time on MathOverflow.)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.