Concentration rates for the posterior distribution

Question

Sanov's theorem and Dvoretzky–Kiefer–Wolfowitz's inequality tell us how fast the empirical distribution concentrates around the true underlying probabilty distribution.

What is known about the concentration of the posterior distribution ? Do such inequalities exist for a mode, a median, the mean or a sample of the posterior ?

Formally, let $\left\lbrace \mathbb{P}_{\theta} \ : \ \theta \in \Theta \right\rbrace$ be a set of probability distributions on $\mathbb{R}$ and $\mu_0$ a prior on $\Theta$.

Let $\mu_n$ be the posterior distribution, i.e. the regular conditional distribution of $\theta$ given $X_1,...,X_n$ when the distribution of $(\theta,X_1,...,X_n)$ is $\mathbb{P}=\mu_0 \otimes \mathbb{P}_{\theta}^{\otimes n}$. $$\mu_n(A,x_1,...,x_n) = \mathbb{P}(\theta \in A \ | \ X_1 = x_1,...,X_n = x_n)$$ If the model is true, this is indeed the distribution of $\theta$ when one has observed n i.i.d samples $(X_1,...,X_n)$ with values $(x_1,...,x_n)$. Even if the model is false, given $n$ values $(x_1,...,x_n)$, $A \mapsto \mu_n(A,x_1,...,x_n)$ still defines a probability distribution on $\Theta$.

Now if I'm a frequentist, I do not agree that $\theta$ is a random variable, I believe it is a parameter with a true (but unknown) value $\theta_{\text{true}}$ and the data $(X_1,...X_n)$ are then i.i.d with true underlying (but unknown) distribution $\mathbb{P}_{\theta_{\text{true}}}^{\otimes n}$. I then a get a random probability distribution $A \mapsto \mu_n(A,X_1,...,X_n)$ on $\Theta$ for which I can take a mode, a median (if $\Theta$ is a subset of $\mathbb{R})$), the mean (also if $\Theta$ is a subset of $\mathbb{R})$) or even a sample (which are all random quantities because they depend on the data).

I'm looking for finite-time upper bounds on the quantities $\mathbb{P}_{\theta_{\text{true}}}^{\otimes n}\left( \bullet \notin A \right)$ where $\bullet$ stands for the mean, a median, a mode or a sample of the posterior $\mu_n$ and $A$ is a measurable subset of $\Theta$ containing $\theta_{\text{true}}$.

I don't know much about bayesian statistics and would be happy with results in any setting ($\Theta$ can be as simple as you want, like a standard exponential family, or even the family of Bernoulli distributions, and $A$ can be any specific neighbourhood of $\theta_{\text{true}}$).

Answer 1

Well, in this case, the posterior distribution on $\Theta$, assuming $p_\theta(x)$ is a nice density of the $x_i$s, and assuming $\Theta\subset R$ and $\mu_0$ having density $p_0$ wrt Lebesgue, is $$\frac{p_0(\theta) e^{\sum_{i=1}^n g_\theta(x_i)}}{\int_\Theta p_0(\theta) e^{\sum_{i=1}^n g_\theta(x_i)} d\theta},$$ where $g_\theta(x)=-\log p_\theta(x)$. For $n$ large $$\sum_{i=1}^n g_\theta(x_i)=n \langle L_n, g_\theta(\cdot)\rangle= n \langle p_t,g_\theta(\cdot)\rangle+ \sqrt{n} G_\theta$$ where the error term $G_\theta$ converges to a Gaussian process and $p_t$ is your true distribution. If $\theta\to \langle p_t,g_\theta\rangle$ has a unique minimizer $\hat\theta\in \Theta$, the posterior measure thus will concentrate around $\hat\theta$. In particular, you can (from the estimates on convergence of $L_n$, which are exponential) estimate how fast the convergence toward $\hat \theta$ occur. Is that what you had in mind?

I am sure all this is written in standard references in the statistics literature, however I am no sure where.