1
$\begingroup$

Setup

Assume $p_Y \in \Delta^n$ is a probability vector obtained by $p_Y=L_{Y|X}p_X$, where $L_{Y|X} \in \mathbb{R}^{n \times m}$ is an arbitrary likelihood (i.e, a column stochastic matrix) and $p_X \in \Delta^m$ another probability vector.

Further consider the Bayesian posterior matrix $P_{X|Y}(u) \in \mathbb{R}^{m \times n}$ for some prior $u\in \Delta^m$, given by

$$P_{X|Y}(u) = diag(u) L_{Y|X}^T diag(L_{Y|X}u)^{-1}.$$

Entry $ij$ of this matrix contains the Bayesian posterior probability $p(x=j|y=i)$.

I'm interested in the following fixed point iteration: Starting with an initial $u_0$, repeatedly compute the posterior with prior $u_k$, averaged with respect to $p_Y$, that is

$$u_{k+1} = P_{X|Y}(u_k)p_Y = P_{X|Y}(u_k)L_{Y|X}p_X.$$

Problem

What statements can be made about the Shannon entropy of the distributions $u_k$ when the iteration is initialized with the uniform distribution $u_0 = (\frac{1}{m}, \dots, \frac{1}{m})^T$?

In particular, does it hold that $H(u_k) \geq H(p_X)$ for any $k$?

I've had some success for $u_1$ via majorization (since the matrix $P_{X|Y}(u_0)L_{Y|X}$ is doubly stochastic), but not further. What tools could be used to try to prove this or similar statements, or find a counterexample?

$\endgroup$

0

You must log in to answer this question.