Rate of convergence of Bayesian posterior

Question

Suppose a data generating process (DGP) is parameterized by some unknown parameter $\theta_0$, say $P_{\theta_0}$, and we want to estimate the value of $\theta_0$ using Bayesian method. Let $\pi(\theta)$ be the prior over the possible values of $\theta$, $\Theta$.

I understand that for almost all priors, if the data observed are iid, then the Bayesian posterior will eventually concentrate on $\theta_0$, as the number of observations ($n$) tends to infinity. (Correct me if I'm wrong.)

My question: Are there any results that predict differential rates of convergence of the posterior distribution based on different priors?

For example, suppose $\Theta=\{\theta_0,\theta_1\}$ and the DGP is $P_{\theta_0}$. Consider two priors on $\Theta$: $$\pi_1(\theta_0)=0.2,\qquad \pi_1(\theta_1)=0.8$$ and $$\pi_2(\theta_0)=\pi_2(\theta_1)=0.5.$$ Are there any results that says the posterior based on $\pi_2$ converges faster than that based on $\pi_1$? (or the other way around?)

Intuitively I would expect the posterior based on $\pi_2$ to have faster convergence, as it is "closer" to $\theta_0$. But in my experience, intuition is hardly reliable when it comes to probability theory.

Note: I've asked this quesiton on Math.SE but received no answer. I thought I would try my luck here.

Answer 1

One can measure the rate of convergence of the posterior distribution with density $p_n$ to the Dirac probability distribution at $\theta_0$ by how large the ratios $p_n(\theta_0)/p_n(\theta)$ are for $\theta\ne\theta_0$, where $\theta_0$ is the "true" value of the parameter. One has $$ \frac{p_n(\theta_0)}{p_n(\theta)}= \frac{\pi(\theta_0)}{\pi(\theta)}\,\frac{L_n(\theta_0)}{L_n(\theta)}, $$ where $\pi$ is the prior density and $L_n(\theta):=\prod_1^n f_\theta(X_i)$ is the likelihood value based on the first $n$ of the iid observations $X_1,X_2,\dots$ with common density $f_{\theta_0}$. It is clear from the above display that, for the same "data" $X_1,X_2,\dots$, the greater is the prior ratio $\pi(\theta_0)/\pi(\theta)$ the greater is the posterior "contrast" ratio $p_n(\theta_0)/p_n(\theta)$. In particular, in the extreme case when (say) $\pi(\theta)$ is $1$ if $\theta=\theta_0$ and is $0$ otherwise, the prior ratio is $\infty$ for any $\theta\ne\theta_0$, and hence so is the "contrast" ratio $p_n(\theta_0)/p_n(\theta)$.

One may want to measure the posterior contrast by one number, say by averaging the posterior contrast over the data space and over the parameter space, to get something like $$\int\mu(d\theta)\,\mathsf{E}\ln\frac{p_n(\theta_0)}{p_n(\theta)}= \int\mu(d\theta)\,\ln\frac{\pi(\theta_0)}{\pi(\theta)} +n\int\mu(d\theta)\,\mathsf{E}\ln\frac{f_{\theta_0}(X_1)}{f_{\theta}(X_1)},$$ where $\mu$ is a measure on the parameter space.
So, the greater is the average prior contrast $\int\mu(d\theta)\,\ln\frac{\pi(\theta_0)}{\pi(\theta)}$ the greater is the average posterior contrast $\int\mu(d\theta)\,\mathsf{E}\ln\frac{p_n(\theta_0)}{p_n(\theta)}$.

In particular, your prior $\pi_2$ should result, by this logic, in a greater posterior contrast than $\pi_1$.

Addendum: The average prior contrast and hence the average posterior contrast may behave somewhat surprisingly, though. E.g., suppose that the parameter space is $\Theta=\mathbb{R}$, $\theta_0=0$, $\pi$ is the normal density with mean $\theta_1$ and variance $\tau^2$, $f_\theta$ is the normal density with mean $\theta$ and variance $\sigma^2$, and $\mu$ is the normal distribution with mean $0$ and variance $\gamma^2$. Then for the average prior and posterior contrasts one has $$\int\mu(d\theta)\,\ln\frac{\pi(\theta_0)}{\pi(\theta)}=\frac{\gamma^2}{2\tau^2}\quad\text{and}\quad \int\mu(d\theta)\,\mathsf{E}\ln\frac{p_n(\theta_0)}{p_n(\theta)}= \frac{\gamma^2}2\Big(\frac1{\tau^2}+\frac n{\sigma^2}\Big),$$ respectively. As could be expected, both these average contrasts are the greater the smaller is the variance $\tau^2$ of the prior distribution. However, neither of these contrasts depends on the prior mean $\theta_1$ ! This will be so for any distribution $\mu$ symmetric about $0$ (or, more generally, about $\theta_0$) -- because, for the normal prior $\pi$ as in this addendum, the symmetrized prior contrast $$\ln\frac{\pi(0)}{\pi(\theta)}+\ln\frac{\pi(0)}{\pi(-\theta)} =\frac{\theta^2-2\theta \theta _1}{2 \tau ^2} +\frac{\theta^2+2\theta \theta _1}{2 \tau ^2} =\frac{\theta^2}{\tau^2}$$ does not depend on $\theta_1$.