I encounter the following problem in my research and I hope you would kindly help me with it. Here is the description:

Let $M$ be an $m \times n$ stochastic matrix with each row summing up to $1$. An important feature of $M$ is that the rows $\{\mathbf{m}_i^{T}\}_{i=1,\dots, n}$ of $M$, which are probability vectors of dimension $n$, are all distinct. Now let $\mathbf{x}$ be a probability vector of dimension $m$, and let function $f$ be defined as: $$f(a) = \min_{h(\mathbf{x})=a}h\left(\mathbf{x}^{T}M\right) - \sum_{i=1}^{m}x_ih(\mathbf{m}_i),$$ where $h$ is the entropy that is $$h(\mathbf{x}) = -\sum_{i=1}^{m}x_i\log x_i$$where $\mathbf{x}=(x_1,\dots, x_m)$.

We note that, denoting $\mathbf{x}^{T}M$ by $\mathbf{y}$, we have in fact $$f(a) = \min_{h(\mathbf{x}) = a}I(\mathbf{x};\mathbf{y}).$$

Here are the properties I want to know about $f$:

  1. Is $f$ continuous and differentiable?
  2. By concavity of entropy, we have $f \geq 0$. However, if we have $\mathbf{m}_i = \mathbf{m}_j$, we do have $f(a)=0$ while $h(a)\neq 0$. For example let $\mathbf{m}_1 = \mathbf{m}_2$, then $\mathbf{x} = \left(.2, .8, 0,\dots, 0\right)$, or any probability vector with all entries $x_3,\dots, x_m=0$, makes $f(h(\mathbf{x}))=0$. But what happens if all $\mathbf{m}_i$s are distinct? To be more specific, if $f$ is differentiable, what is $f'(0)$? Or do we have something like $$f(a)\geq c \cdot h(a),$$ where $c>0$ is a constant?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.