Let $\Theta=\Delta [0,1]$ denote the set of all probability distributions on the unit interval. Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density nor necessarily has full support.

The decision-maker observes a sequence $X^T=(X_1,...,X_T)$, where $X_t \in \{0,1\}$ and assumes that all realizations $X_t$ are such that $X_t \sim \mathrm{Bernoulli}(\theta_0)$, where $\theta_0 \in \Theta$ is unknown.

(It is irrelevant for the question whether the data is indeed iid or not.)

Question: Is there $A \subset [0,1]$ such that $A$ has at most two points such that for all open neighborhood $U$ of $A$ $\lim_{T\to\infty}\mu|X^T(U)=1$?

My conjecture: The posterior accumulates on the set $A$ that minimizes the KL divergence with respect to the empirical mean. That is, because the data is assumed to be iid by the decision-maker, then some version of Berk (1966) would apply and $A=\arg \min_{\theta \in \text{supp} \mu} D(\theta||\theta_0)$, where $D$ denotes the Kullback-Leibler divergence.

Is this line of reasoning correct?

Are there any references available with respect to this case (limit behavior of Bayesian posteriors with potentially misspecified prior in a discrete setting)?

Let $\Delta [0,1]$ denote the set of all probability distributions on the unit interval. Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density nor necessarily has full support.

The decision-maker observes a sequence $X^T=(X_1,...,X_T)$, where $X_t \in \{0,1\}$ and assumes that all realizations $X_t$ are such that $X_t \sim \mathrm{Bernoulli}(\theta_0)$, where $\theta_0 \in [0,1]$ is unknown.

(It is irrelevant for the question whether the data is indeed iid or not.)

Question: Is there $A \subset [0,1]$ such that $A$ has at most two points such that for all open neighborhood $U$ of $A$ $\lim_{T\to\infty}\mu|X^T(U)=1$?

My conjecture: The posterior accumulates on the set $A$ that minimizes the KL divergence with respect to the empirical mean. That is, because the data is assumed to be iid by the decision-maker, then some version of Berk (1966) would apply and $A=\arg \min_{\theta \in \text{supp} \mu} D(\theta||\theta_0)$, where $D$ denotes the Kullback-Leibler divergence.

Is this line of reasoning correct?

Are there any references available with respect to this case (limit behavior of Bayesian posteriors with potentially misspecified prior in a discrete setting)?

Edit:

The argument follows trivially from the definition of the Bayes posterior.

Let $p_t$ denote the empirical mean of the sequence $X^t$ and $H(p,q)=-\sum_i p_i \ln q_i$. Then,

$\mu|X^t(A)=\frac{\int_A \theta^{t p_t}(1-\theta)^{t(1-p_t)}\mu(d\theta)}{\int_\Theta \theta^{t p_t}(1-\theta)^{t(1-p_t)}\mu(d\theta)}=\frac{\int_A e^{-t H(p_t,\theta)}\mu(d\theta)}{\int_\Theta e^{-t H(p_t,\theta)}\mu(d\theta)}$.

It follows that the posterior will accumulate only on the points in the support that minimizes the KL divergence with respect to the empirical mean. There are at most two such points.

New Question:

Fix a $(X_t)_{t}$ such that $\frac{||X_t||_1}{t}=:\theta_t \to \theta_0$ and suppose that $\arg \min_{\theta \in \text{supp}\mu}D(\theta||\theta_0)=\{\theta_H,\theta_L\}$, with $\theta_H>\theta_0>\theta_L$.

(1) We know that $\mu|X^t(\{\theta_H,\theta_L\})\to 1$.
Is it the case that the posterior converges, i.e. $\mu|X^t \to \mu_\infty$ where $\mu_\infty(\{\theta_H\})=1-\mu_\infty(\{\theta_L\})=\alpha$, for some $\alpha \in [0,1]$?

(2) If so, can we say something about $\alpha$? E.g. $\alpha =\mu\left(\{\theta \in [0,1]:\theta>\theta_0\}\right)$?

(3) Let $\Delta_t(\epsilon):=\ln\frac{\mu|X^t(B(\epsilon,\theta_H)))}{\mu|X^t(B(\epsilon,\theta_L)))}-\ln\frac{\mu|X^{t-1}(B(\epsilon,\theta_H)))}{\mu|X^{t-1}(B(\epsilon,\theta_L)))}$, where $B(\epsilon,\theta)$ denotes the open ball of radius $\epsilon$ around $\theta$.
If instead we have that $X_t \sim Bernoulli(\theta_0)$, is it the case that for $\epsilon>0$ small, $\Delta_t(\epsilon)\to Y$, where $Y=\theta_H/\theta_L$ with probability $\theta_0$ and $Y=(1-\theta_H)/(1-\theta_L)$ with probability $1-\theta_0$?
In other words, will the change in log odds of $B(\epsilon,\theta_H)$ against $B(\epsilon,\theta_L)$ be approximately distributed as that of the log-odds of the prior with only $\theta_H$ and $\theta_L$ in the support?

Let $\Theta=\Delta [0,1]$ denote the set of all probability distributions on the unit interval. Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density nor necessarily has full support.

The decision-maker observes a sequence $X^T=(X_1,...,X_T)$, where $X_t \in \{0,1\}$ and assumes that all realizations $X_t$ are such that $X_t \sim Bernoulli(\theta_0)$, where $\theta_0 \in \Theta$ is unknown.

(It is irrelevant for the question whether the data is indeed iid or not.)

Question: Is there $A \subset [0,1]$ such that $A$ has at most two points such that for all open neighborhood $U$ of $A$ $\lim_{T\to\infty}\mu|X^T(U)=1$?

My conjecture: The posterior accumulates on the set $A$ that minimizes the KL divergence with respect to the empirical mean. That is, because the data is assumed to be iid by the decision-maker, then some version of Berk (1966) would apply and $A=\arg \min_{\theta \in \text{supp} \mu} D(\theta||\theta_0)$, where $D$ denotes the Kullback-Leibler divergence.

Is this line of reasoning correct?

Are there any references available with respect to this case (limit behavior of Bayesian posteriors with potentially misspecified prior in a discrete setting)?

Let $\Theta=\Delta [0,1]$ denote the set of all probability distributions on the unit interval. Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density nor necessarily has full support.

The decision-maker observes a sequence $X^T=(X_1,...,X_T)$, where $X_t \in \{0,1\}$ and assumes that all realizations $X_t$ are such that $X_t \sim \mathrm{Bernoulli}(\theta_0)$, where $\theta_0 \in \Theta$ is unknown.

(It is irrelevant for the question whether the data is indeed iid or not.)

Question: Is there $A \subset [0,1]$ such that $A$ has at most two points such that for all open neighborhood $U$ of $A$ $\lim_{T\to\infty}\mu|X^T(U)=1$?

My conjecture: The posterior accumulates on the set $A$ that minimizes the KL divergence with respect to the empirical mean. That is, because the data is assumed to be iid by the decision-maker, then some version of Berk (1966) would apply and $A=\arg \min_{\theta \in \text{supp} \mu} D(\theta||\theta_0)$, where $D$ denotes the Kullback-Leibler divergence.

Is this line of reasoning correct?

Are there any references available with respect to this case (limit behavior of Bayesian posteriors with potentially misspecified prior in a discrete setting)?