Skip to main content
2 of 3
removed capitals, romanized text math
YCor
  • 63.9k
  • 5
  • 187
  • 286

Convergence of Bayesian posterior

Let $\Theta=\Delta [0,1]$ denote the set of all probability distributions on the unit interval. Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density nor necessarily has full support.

The decision-maker observes a sequence $X^T=(X_1,...,X_T)$, where $X_t \in \{0,1\}$ and assumes that all realizations $X_t$ are such that $X_t \sim \mathrm{Bernoulli}(\theta_0)$, where $\theta_0 \in \Theta$ is unknown.

(It is irrelevant for the question whether the data is indeed iid or not.)

Question: Is there $A \subset [0,1]$ such that $A$ has at most two points such that for all open neighborhood $U$ of $A$ $\lim_{T\to\infty}\mu|X^T(U)=1$?

My conjecture: The posterior accumulates on the set $A$ that minimizes the KL divergence with respect to the empirical mean. That is, because the data is assumed to be iid by the decision-maker, then some version of Berk (1966) would apply and $A=\arg \min_{\theta \in \text{supp} \mu} D(\theta||\theta_0)$, where $D$ denotes the Kullback-Leibler divergence.

Is this line of reasoning correct?

Are there any references available with respect to this case (limit behavior of Bayesian posteriors with potentially misspecified prior in a discrete setting)?