In two statistics papers (linked inline below) I have come across two definitions of certain probability measures. I conjecture that for particular choices of the construction that they are equivalent but am struggling to show so in the most general case.

I'm studying generalizations of conditioning for updating subjective probability distributions. My goal is to adapt these methods for practical statistical analysis; the previous literature is primarily scholarly/academic. My measure theory not being what it should, I'm running into a technical difficulty and I think the audience here can help.

I will begin by describing the case of a finite partition before moving on to the case that is bothering me, the uncountable setting.

THE GIBBS POSTERIOR (see Jiang and Tanner [via Wayback Machine] tech report)
One definitions I'm dealing with is that of a Gibbs posterior, which has density w.r.t. Lebesgue measure: $$\omega(d\beta) = \exp{(-R_Y(\beta))}\pi(d\beta) / \int_{\beta \in \Omega} \exp{(-R_Y(\beta))}\pi(d\beta),$$ where $\beta \in \Omega \subseteq \mathbb{R}^k$ and $R$ is a function from $\Omega \rightarrow \mathbb{R}$. The subscript $Y$ attached to $R$ is just a reminder that $R$ is usually defined in terms of some fixed set of observed data $Y = (y_1, y_2, \cdots, y_n)$. A typical example would be $R_Y(\beta) = \sum_{j=1}^n (y_j - \beta)^2$, the mean square error. You'll notice that if $R$ is the negative log-likelihood of $Y$, this expression gives the usual Bayesian posterior distribution. However, we may use this for general loss functions to get a so-called empirical risk based posterior. This expression provides a recipe for moving from $\pi(\beta)$ to $\pi^{\star}(\beta) = \omega(d\beta \mid Y)$, provided we specify the form of $R_Y$.

JEFFREY UPDATING (the easy case)
Another alternative to Bayes rule is something called Jeffrey updating. In the easy case, Jeffrey updating calls on us to find a "sufficient partition" of the parameter space. So if our parameter $\beta$ lives on the real line, we may partition it into finitely many disjoint subsets $E = (E_1, E_2, \cdots, E_n)$. The partition is judged to be sufficient in that within any of these subsets our prior beliefs/distribution $\pi(\beta)$ doesn't change. All that will change is the relative values of the subsets. Specifically for any set $A$ we have $$P^{\star}(A) = \sum_{j=1}^n P(A \mid E_j)P^{\star}(E_j).$$

If $\pi(\beta)$ is a Normal distribution then $\pi^{\star}(\beta)$ would look something like this. The prior is in red, the posterior in black and the partition is defined by 20 observed data values (I won't describe how $P^{\star}(E_i)$ was determined.)

alt text (source)

This approach is motivated by the idea of "conditioning on partial information". If you don't know for sure what partition the true parameter falls in, but you are handed some information about those relative probabilities (the $P^{\star}(E_j)$), this is how you would update. This should be equivalent to the Gibbs posterior in the case that $R$ is defined to be a simple/step function ranging over $\beta$. The finite jumps in $R$ corresponds to the partitions and $P^{\star}(E_j) \propto \exp(-R(\beta \in E_j))$.

JEFFREY UPDATING (the harder case)
Expression 6.1 from Diaconis and S.L. Zabell extends Jeffrey updating to the case where instead of a finite partition you have a presumed sufficient sub-$\sigma$-algebra. I'm reproducing the relevant section here.

We start with a probability space $(\Omega, \mathcal{A}, P)$. We want to move from $P$ to $P^{\star}$, a new probability measure on $\mathcal{A}$. We assume that we have a sub-$\sigma$-algebra $\mathcal{A}_0 \subseteq \mathcal{A}$. Let $C$ be an $\mathcal{A}_0$-measurable set with $P(C) = 0$ and $\bar{P} \ll \bar{P}^{\star}$ on $\Omega - C$, where the bar denotes restriction to $\mathcal{A}_0$. Then we define $$ P^{*}(A) = \int_{\Omega-C} P(A \mid \mathcal{A}_0) P^{\star}(d \omega) + P^{*}(A \cap C).$$


Are there definitions of $\mathcal{A}_0$ and $R(\beta)$ that would make the Jeffrey updated posterior in the continuous case equivalent to the Gibbs posterior?

If I do this naively, by letting $R$ be a continuous loss function (such as mean square error above) and let $\mathcal{A}_0$ be the $\sigma$-algebra generated by $R$ I think I get back all of the Borel sets, or all of $\mathcal{A}$. (A terminological question: does a sub-$\sigma$-algebra have to be strictly contained within the larger $\sigma$-algebra or is any $\sigma$-algebra a trivial sub-algebra of itself?) As a constructive matter I find that the clean correspondence in the simple function case loses its charm when one rather considers $\mathcal{A}_0$.

If one cannot directly specify $R(\beta)$ and $\mathcal{A}_0$ to make the two measures equivalent, can one define a (unique?) limiting equivalence by specifying a sequence of simple functions $R^i$ with limit $R$ and a sequence of finite partitions $E^i$ which generate $\mathcal{A}$ in the limit such that for every pair $(R^i, E^i)$ the Jeffrey and Gibbs posterior measures coincide?

And finally...

To help my intuition with the uncountable case, can anyone come up with a plausible candidate $\mathcal{A}_0$ in the simple case that $\Omega = \mathbb{R}$ and $\beta$ has density wrt the Lebesgue measure (perhaps by analogy to the discrete case I described)?

I hope some of you find this interesting enough!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.