Skip to main content
2 of 3
added 568 characters in body
Upax
  • 127
  • 4

About the monotonicity of the exponential entropy

This question was previously posted on MSE at About the monotonicity of the exponential entropy.

In the paper The Unifying Frameworks of Information Measures the conditional exponential entropy (see equation 29) is defined as:

$\mathcal{E}_{\alpha}(X|Y) = E_y\left(\int_{\mathbb{R}} f^{\alpha}(x|y)\,dx\right)^{\frac{1}{1-\alpha}}$

while the exponential entropy (see equation 9) is:

$\mathcal{E}_{\alpha}(X) = \left(\int_{\mathbb{R}} f^{\alpha}(x)\,dx\right)^{\frac{1}{1-\alpha}}$

$f(x)$ is the density of the random variable $X$, which is a non-negative real-valued Borel measurable function on $\mathbb{R}$ (i.e. is absolutely continuous).

I would like to prove the monotonicity of exponential entropy with order $\alpha$ by showing that the following expression holds for the continuous random variables X, Y, Z:

$\mathcal{E}_{\alpha}(X|Y Z) \leq \mathcal{E}_{\alpha}(X|Z) $.

Why this question

The condition of monotonicity is crucial and fairly obvious: the amount of information required to determine a particular choice cannot decrease as the number of available alternatives increases. The traditional framework for formalizing uncertainty theories can be extended by using a monotonic measure. In fact, entropy generalizations can be achieved within the framework of classical set theory by replacing the additivity requirement of probability measures with the weaker monotonicity.

My attempt to prove the above inequality

I am not sure how to proceed and would like to get your opinion. This is my attempt. Using Jensen's inequality, for $0 < \alpha < 1$, since $t^{\frac{1}{1-\alpha}}$ is convex for $t > 0$, we have:

$\mathcal{E}_{\alpha}(X|Z) = E_{z} \left(\int_{\mathbb{R}} f^{\alpha}(x|z)\,dx\right)^{\frac{1}{1-\alpha}}$ $\geq \left(E_{yz}\left(\int_{\mathbb{R}} f^{\alpha}(x|y,z)\,dx\right)\right)^{\frac{1}{1-\alpha}} \geq E_{z} \left(E_{y}\left(\int_{\mathbb{R}} f^{\alpha}(x|y,z)\,dx\right)\right)^{\frac{1}{1-\alpha}} = \mathcal{E}_{\alpha}(X|Y Z)$

For $\alpha > 1$, since $t^{\frac{1}{1-\alpha}}$ is concave and decreasing for $t > 0$, we similarly obtain:

$\mathcal{E}_{\alpha}(X|Z) = E_{z} \left(\int_{\mathbb{R}} f^{\alpha}(x|z)\,dx\right)^{\frac{1}{1-\alpha}}$ $\geq \left(E_{yz}\left(\int_{\mathbb{R}} f^{\alpha}(x|y,z)\,dx\right)\right)^{\frac{1}{1-\alpha}} \geq E_{z} \left(E_{y}\left(\int_{\mathbb{R}} f^{\alpha}(x|y,z)\,dx\right)\right)^{\frac{1}{1-\alpha}} = \mathcal{E}_{\alpha}(X|Y Z)$

Therefore, $\mathcal{E}_{\alpha}(X|Y Z) \leq \mathcal{E}_{\alpha}(X|Z)$ holds for all $\alpha > 0$ and $\alpha \neq 1$. Is this proof correct?

Upax
  • 127
  • 4