User ppyang - MathOverflow

Derivative of the regularized upper incomplete gamma function

2013-02-22T08:22:40Z

Hello everyone!

I have a question about the derivative of the regularized upper incomplete gamma function. Considering the gamma function and the incomplete gamma function \begin{eqnarray} \Gamma(x)&=&\int_0^\infty{t^{x-1}e^{-t}\mathrm{d}t},\\ \Gamma(x,\lambda)&=&\int_\lambda^\infty{t^{x-1}e^{-t}\mathrm{d}t},\end{eqnarray} where the quotient of $\Gamma(x,\lambda)$ divided by $\Gamma(x)$ is usually denoted as regularized upper incomplete gamma function \begin{equation} Q(x,\lambda)=\frac{\Gamma(x,\lambda)}{\Gamma(x)}, \end{equation} there seems to be a precise estimation for the derivative $\frac{\partial Q(x,\lambda)}{\partial x}$, but I cannot prove it.

With numerical experiment, I found that given $\lambda$ is not small, $\frac{\partial Q(x,\lambda)}{\partial s}$ is almost proportional to \begin{equation} \frac{\lambda^xe^{-\lambda}}{x^xe^{-x}}=\exp\left(-\left(x\log x-x\log\lambda-x+\lambda\right)\right). \end{equation} Denoting \begin{equation} I(\lambda)=\int_0^\infty{\frac{\lambda^xe^{-\lambda}}{x^xe^{-x}}\mathrm{d}x}, \end{equation} the partial derivative $\frac{\partial Q(x,\lambda)}{\partial x}$ seems to have the following approximation \begin{equation} \frac{\partial Q(x,\lambda)}{\partial x} \approx\frac{\lambda^xe^{-\lambda}}{x^xe^{-x}}/I(\lambda). \end{equation}

I give the comparison of these two formulas from the numerical experiments with Matlab in following figures, where the blue solid line and green dotted line represent $\frac{\partial Q(x,\lambda)}{\partial x}$ and $\frac{\lambda^xe^{-\lambda}}{x^xe^{-x}}/I(\lambda)$ respectively.

From these figures, it's obvious that as the increase of $\lambda$, the two lines approach to be equal. However, I cannot prove such a relationship due to the complexity of the derivative $\frac{\partial Q(x,\lambda)}{\partial x}$.

Could you help me with some comments about this question? Any suggestions will be helpful and thank you very much!

Are there a group of mappings from (n-1)-dim space to an (n-1)-sphere guaranteeing the orthogonality of images?

2013-02-18T17:26:56Z

Hello, everyone.

As we know that in an $n$-dimensional Euclidean space $\mathbb{R}^n$, there exists a continuous bijective mapping from a subset $V^{n-1}\subseteq\mathbb{R}^{n-1}$ to a unit $(n-1)$-sphere $S^{n-1}=\{\mathbf{x}\in\mathbb{R}^n:\|\mathbf{x}\|=1\}$ since the $(n-1)$-sphere is indeed an $(n-1)$-dimensional manifold.

The most straightforward example is obtained from the spherical coordinate transformation by setting $r=1$, which bijectively maps a $(n-1)$-dimenional cuboid $V^{n-1}=[0,\pi]\times\ldots\times[0,\pi]\times[0,2\pi)$ to $S^{n-1}$. (Refer to http://en.wikipedia.org/wiki/N-sphere#Spherical_coordinates)

Then I wonder if there exist $n$ such mappings $\{\mathbf{p}_i:V^{n-1}\mapsto S^{n-1}\subset\mathbb{R}^n\}_{i=1}^n$ so that the $n$ images $\mathbf{p}_1(\theta),\ldots,\mathbf{p}_n(\theta)$ corresponding to any same inverseimage $\theta\in V^{n-1}$ are orthogonal to each other.

Consider the case where $n=2$. In $\mathbb{R}^2$, we can map a line segment $[0,2\pi)\subseteq\mathbb{R}^1$ to a unit circle $S^1=\{\mathbf{x}\in\mathbb{R}^2:\|\mathbf{x}\|=1\}$ by either of the following two maps:

\begin{eqnarray} \mathbf{p}_1(\theta)&=&\left[\cos(\theta),\sin(\theta)\right]^\top;\\ \mathbf{p}_2(\theta)&=&\left[-\sin(\theta),\cos(\theta)\right]^\top; \end{eqnarray} where $\mathbf{p}_1,\mathbf{p}_2:[0,2\pi)\mapsto S^1$ are bijections. What is more important, it is easy to verify that $\mathbf{p}_1^\top(\theta)\mathbf{p}_2(\theta)=0,\forall\theta\in[0,2\pi)$, and thus we obtain such a group of orthogonal mappings in $\mathbb{R}^2$.

Then, what about when $n>2$? Although we can map $V^{n-1}$ to $S^{n-1}$ using the spherical coordinate transformation as mentioned above, I cannot construct $n$ such mappings with orthogonality yet. Does there exist such a group of mappings?

Any suggestion will be welcome and thank you very much!

Comment

I asked another question yesterday about if it is possible to obtain a group of orthogonal vectors also orthogonal to a given one by orthogonal transformation. If there was such a transformation, this question would be solved as follows:

We can define $\mathbf{p}_1$ as the spherical coordinate transformation and then generate $\mathbf{p}_i(i\geq 2)$ by these orthogonal transformations. The orthogonality of $\mathbf{p}_i$'s is guaranteed by the definition of these transformations. Then, since $\mathbf{p}_i(i\geq 2)$ is obtained by an orthogonal transformation on $\mathbf{p}_1$, if $\mathbf{p}_1(\theta)$ is a bijection to $S^{n-1}$, so does $\mathbf{p}_i(i\geq 2)$.

Unfortunately, there does not exist such a orthogonal transformation in general. Then, can the question I asked today be solved in another way? Thank you very much!

Is it possible to obtain the vectors orthogonal to a given one by orthogonal transformations?

2013-02-17T14:34:47Z

Hello, everyone!

Supposing that there is a unit vector in $n$-dimensional real space $\mathbf{x}_1\in\mathbb{R}^n$, I want to get a group of $n-1$ vectors to form an orthogonal basis with $\mathbf{x}_1$. One way to achieve this goal is to firstly randomly generate $n-1$ linear independent vectors and then orthogonalize them use the Gram-Schmidt process to obtain the orthogonality.

Then, I wonder if it is possible to achieve this goal by a group of orthogonal transformations, i.e., if there are $n-1$ orthogonal matrices $\{A_i\}_{i=2}^n$ so that $\mathbf{x}_i^\top\mathbf{x}_j=0,\forall i\neq j$ where $\mathbf{x}_i=A_i\mathbf{x}_1,i=2,\ldots,n$.

Is there any result about this question, please? Any suggestion will be welcome. Thank you very much!

Comment

In the special case where $n=2$, it is possible by $A_2=\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}$. It is straightforward to check that the orghotonality of $A_2$ and $(A_2\mathbf{x}_1)^\top\mathbf{x}_1=0,\forall\mathbf{x}_1$. It is easy to comprehend because $A_2$ is indeed a ration of 90 degree in a 2-dimensional plane.

However, it is not so simple in the case of $n=3$. First, to obatin the orthogonality by a transformation $A_2$, i.e., $\mathbf{x}^\top A_2\mathbf{x}=0,\forall\mathbf{x}\in\mathbb{R}^n$, the following equation need to be satisfied. \begin{align} &a_{11}=a_{22}=a_{33}=0\\ &a_{12}+a_{21}=a_{13}+a_{31}=a_{23}+a_{32}=0 \end{align} where $a_{ij}$ is the $i,j$-th element of $A_2$.

With the conditions above, it is straightforward to check that $|A_2|=0$ and thus $A_2$ cannot be an orthogonal matrix, which means there does not exist an orthogonal transformation to obtain an orthogonal vector to a given one.

In the case of $n=4$, it is possible to find an orthogonal transformation to obatin an orthogonal vector to a given one. But I did not find how to find 3 such transformations to form an orthogonal basis including the given vector.

How to solve this optimization with the orthogonal constraint?

2012-12-12T09:37:28Z

Problem

Supposing that $A$ is a symmetric real matrix and $\{\mathbf{w}_i\}_{i=1}^n$ is any orthogonal basis on $\mathbb{R}^n$ such that $W^\top W=WW^\top=\mathbf{I}_n$ where $W=\left[\mathbf{w}_1\;\mathbf{w}_2\;\ldots\;\mathbf{w}_n\right]$, what is the maximum of $\sum_{i=1}^n{\left(\mathbf{w}_i^\top A\mathbf{w}_i\right)^2}$?

To be concise, we can reformulate it as the following optimization problem: \begin{equation} \begin{split} \max_W&S(W)=\sum_{i=1}^n{\left(\mathbf{w}_i^\top A\mathbf{w}_i\right)^2}\\ \mathrm{s.t.}\,&W^\top W=WW^\top=\mathbf{I}_n, \end{split} \end{equation} where $W=\left[\mathbf{w}_1\;\mathbf{w}_2\;\ldots\;\mathbf{w}_n\right]$.

With help of MATLAB, I found that the maximum of $S(W)$ is always obtained when $W=\bar{W}$ such that $\{\mathbf{w}_i\}_{i=1}^n$ are the eigenvectors of $A$ and the optimal value is $S(\bar{W})=\sum_{i=1}^n{\lambda_i^2}=\|A\|_\mathrm{F}^2$. I had ever given a proof but recently I found it was wrong. However, I think the conclusion should be correct. The constraint that $W$ is orthogonal makes the problem difficult.

Since it is obvious that the assumed optimal value $S(W)=\|A\|_\mathrm{F}^2$ is always obtained by choosing $W$ to be the eigenvectors of $A$, if we can prove that $S(W)\leq\|A\|_\mathrm{F}^2$ for $\forall W$, the problem is solved. However, this seems to be also difficult.

Comments

Actually, the problem may be furthermore simplified as follows.

Assume the spectral decomposition of $A$ is $A=V\Lambda V^\top$ and let $\tilde{W}=V^\top W$ (or $\tilde{\mathbf{w}}_i=V^\top\mathbf{w}_i$), then we have $S(W)=\sum_{i=1}^n{\left(\tilde{\mathbf{w}}_i^\top\Lambda\tilde{\mathbf{w}}_i\right)^2}$ where $\Lambda$ is a diagonal matrix contaning the eigenvalues of $A$.

Since $V$ is orthogonal, we can solve for $\tilde{W}$ instead of $W$. Then the problem turns to be \begin{equation} \begin{split} \max_\tilde{W}&S(\tilde{W}) =\sum_{i=1}^n{\left(\tilde{\mathbf{w}}_i^\top \Lambda\tilde{\mathbf{w}}_i\right)^2} =\sum_{i=1}^n{\left(\sum_j^n{\lambda_j\tilde{w}_{ij}^2}\right)^2}\\ \mathrm{s.t.}\,&\tilde{W}^\top\tilde{W}=\tilde{W}\tilde{W}^\top=\mathbf{I}_n, \end{split} \end{equation} where $\Lambda$ is diagonal and $\tilde{\mathbf{w}}_i=\left[\tilde{w}_{i1}\;\tilde{w}_{i2}\;\ldots\;\tilde{w}_{in}\right]^\top$.

However, this problem seems to be also difficult to solve. Please help me to give some suggestions about this problem.

Thank you very much!

Answer by ppyang for How to solve this optimization with the orthogonal constraint?

2012-12-13T13:41:08Z

I am sorry to trouble you all and I found the answer to this question finally which is very simple.

Due to the orthogonality of $\tilde{W}$, it is straightforward that $\sum_{i=1}^n{\tilde{w}_{ij}^2}=1,\forall j$ and $\sum_{j=1}^n{\tilde{w}_{ij}^2}=1,\forall i$.

Then for $\forall i$, from the convexity of $f(x)=x^2$, we have \begin{equation*} \left(\sum_{j=1}^n{\tilde{w}_{ij}^2\lambda_j}\right)^2\leq\sum_{j=1}^n{\tilde{w}_{ij}^2\lambda_j^2}, \end{equation*} and thus \begin{equation*} S(\tilde{W}) =\sum_{i=1}^n{\left(\sum_{j=1}^n{\lambda_j\tilde{w}_{ij}^2}\right)^2} \leq\sum_{i=1}^n{\sum_{j=1}^n{\tilde{w}_{ij}^2\lambda_j^2}} =\sum_{j=1}^n{\sum_{i=1}^n{\tilde{w}_{ij}^2\lambda_j^2}} =\sum_{j=1}^n{\lambda_j^2} =\|A\|_\mathrm{F}^2 \end{equation*}

Since $\|A\|_\mathrm{F}^2$ provides an upper bound for $S(\tilde{W})$ and this value is always obtainable by choosing $\left\{\mathrm{w}_i\right\}_{i=1}^n$ as the eigenvectors of $A$, this is the maximum value.

What is the geometry of the intersection of some cones defined by generalized inequalities?

2012-02-22T04:25:55Z

Hello, considering that for real numbers, the intersection of intervals defined by simple inequalities has a quite simple form as $$ \bigcap_i\{x|x\leq a_i\}=\{x|x\leq\min_i{a_i}\} $$

However, what is the case if the variables are chosen as Hermitian matrices, and the interval defined by inequality is replaced with the convex cone defined by the generalized inequality?

All variables in following are assumed to be Hermitian matrices.

To be specific, define the generalize inequality $X\preceq A_i$ to denote that $X-A_i$ is negative semi-definite, then $\{X|X\preceq A_i\}$ defines a convex cone in the Hermitian matrix space.

Is there any result about the intersection of these cones? To say, can the following set be simplified? $$ \bigcap_i\{X|X\preceq A_i\} $$

When does there exist such an $A$ to satisfy $\{X|X\preceq A\}=\bigcap_i\{X|X\preceq A_i\}$?

Or how to describe the geometry of the intersection of such cones?

Any suggestion or comment on this question will be appreciated and thanks very much for your help!

==================================================================================

Acknowledgement and more questions about @Suvrit's comment:

Thanks to @Suvrit for your suggestion! Your comment provides a good way to think about this problem. However, I thought about your method but the problem seems to be more complicated than expected.

Take an example for illustration. Denote $\mathcal{C}(A)=\{X|X\preceq A\}$, then if I want to solve \begin{eqnarray} \min_X&&f(X)\\ \mathrm{s.t.}&&X\in\mathcal{C}(A_1)\cap\mathcal{C}(A_2)\cap\mathcal{C}(A_3) \end{eqnarray} by first solve $\min_{X_1\in\mathcal{C}(A_1)\cap\mathcal{C}(A_2)} f(X_1)$ and then $\min_{X_2\in\mathcal{C}(X_1)\cap\mathcal{C}(A_3)}f(X_2)$, the solution in deed satisfies the constraints due to $$\mathcal{C}(X_1)\cap\mathcal{C}(A_3)\subseteq\mathcal{C}(A_1)\cap\mathcal{C}(A_2)\cap\mathcal{C}(A_3).$$ However, these two sets are not identical, and thus the optimal solution in $\mathcal{C}(X_1)\cap\mathcal{C}(A_3)$ is not guaranteed to be also optimal in $\mathcal{C}(A_1)\cap\mathcal{C}(A_2)\cap\mathcal{C}(A_3)$.

I think the difficulty of this problem results from the complex structure of the intersections of cones $\bigcap_i\mathcal{C}(A_i)$. Do you have some more suggestions about this problem?

Thank you very much for your help!

How to calculate the inverse of the sum of an identity and a Kronecker product efficiently?

2012-06-11T13:50:21Z

I have a matrix $K$ which is the sum of a identity and a Kronecker product of two symmetric matrices as following and I want to calculate the inverse of it $K^{-1}$. \begin{eqnarray} K=\mathbf{I}_{mn}+XX^\top\otimes YY^\top \end{eqnarray} where $X$ and $Y$ are $m\times p$ and $n\times q$ matrices respectively and $m>p,n>q$.

Since the size of Kronecker product of two matrices is the product of their sizes, the matrix to be inverted is very large. Can this calculation be simplified?

Using Woodbury matrix identity, the matrix to be inverted can be reduced from $mn\times mn$ to $pq\times pq$ as \begin{eqnarray} \left(\mathbf{I}+XX^\top\otimes YY^\top\right)^{-1} &=&\left(\mathbf{I}+(X\otimes Y)(X\otimes Y)^\top\right)^{-1}\\ &=&\mathbf{I}-(X\otimes Y)\left(\mathbf{I}+(X\otimes Y)^\top(X\otimes Y)\right)^{-1}(X\otimes Y)^\top \end{eqnarray} However, in my application, $p$ and $q$ are not much less than $m$ and $n$. Thus inverting a $pq\times pq$ matrix is still a consuming work and I want to find a more simple calculation of $K^{-1}$.

Consider the following problem. If there is not the identity matrix $\mathbf{I}_{m\times n}$, we can invert $XX^\top$ and $YY^\top$ respectively and then calculate the Kronecker product as $$ (XX^\top\otimes YY^\top)^{-1}=(XX^\top)^{-1}\otimes(YY^\top)^{-1} $$ using the property of Kronecker product. However, if the identity item exists, this property cannot be utilized.

Then my question is that if there is any way to decompose the calculation of $K^{-1}$ into inverses of some smaller matrices whose size is linear with $m$ and $n$ but not their products? Or is it possible to extract the Kronecker product $\otimes$ out of the inverse?

If you have any suggestion or idea, please let me know. Thank you very much for your help!

Is it possible to extend this inequality about Euclidean distance &Frobenius norm to more general Bregman divergence such as relative entropy & von Neumann divergence?

2012-04-04T17:40:44Z

Motivation- A Special Case

Supposing $A,B\in\mathbb{S}^{m\times m}$ are symmetric positive semi-definite (SPD) matrices and $\mathbf{x}\in\mathbb{R}^m$ is a unit vector where $\|\mathbf{x}\|=1$, we found that the squared Euclidean distance of two quadratic forms $\left(\mathbf{x}^\top A\mathbf{x}-\mathbf{x}^\top B\mathbf{x}\right)^2$ is bounded by the squared Frobenius norm of difference of the two matrices $\|A-B\|_F^2$.

Denoting the spectral decomposition of $A-B$ as $A-B=W\Phi W^\top$ where $\Phi=\mathrm{diag}\left(\phi_1,\phi_2,\ldots,\phi_m\right)$ is a diagonal matrix of eigenvalues, we have \begin{eqnarray} &&\left(\mathbf{x}^\top A\mathbf{x}-\mathbf{x}^\top B\mathbf{x}\right)^2 =\left(\mathbf{x}^\top(A-B)\mathbf{x}\right)^2 =\left(\mathbf{x}^\top W\Phi W^\top\mathbf{x}\right)^2\\ =&&\left(\sum_i{x_{W,i}^2\phi_i}\right)^2 \leq\max_i{\phi_i^2}\leq\sum_i{\phi_i^2} =\|A-B\|_F^2 \end{eqnarray} where $W^\top\mathbf{x}=\mathbf{x}_W=\left[x_{W,1}\;x_{W,2}\;\ldots\;x_{W,m}\right]^\top$ and $\mathbf{x}_W^\top\mathbf{x}_W=\sum_i{x_{W,i}^2}=1$.

Therefore, for $\forall \mathbf{x}\in\mathbb{R}^m\;\mathrm{s.t.}\;\|\mathbf{x}\|=1$, we have $\left(\mathbf{x}^\top A\mathbf{x}-\mathbf{x}^\top B\mathbf{x}\right)^2\leq\|A-B\|_F^2$.

Question- Could this be generalized?

However, the squared Euclidean distance is a special case of Bregman divergence $$ D_\varphi(\mathbf{x},\mathbf{y})=\varphi(\mathbf{x})-\varphi(\mathbf{y})-\nabla\varphi(\mathbf{y})^\top(\mathbf{x}-\mathbf{y}) $$ where $\varphi$ is the convex seed function.

On the other hand, the squared Frobenius norm of difference of two matrices is a special case of Bregman matrix divergence $$ D_\phi(A,B)=\phi(A)-\phi(B)-\mathrm{tr}\left((\nabla\phi(B))^\top(A-B)\right) $$ where $\phi(A)=(\varphi\circ\lambda)(A)$ is a compound matrix function in which $\lambda$ is the function that lists the eigenvalues of $A$ and $\varphi$ is the convex seed function.

In the example above, the seed function is $\varphi(\mathbf{x})=\mathbf{x}^\top\mathbf{x}$ and we can rewrite the inequality as $$ D_\varphi\left(\mathbf{x}^\top A\mathbf{x},\mathbf{x}^\top B\mathbf{x}\right) \leq D_\phi(A,B) $$ where $\|\mathbf{x}\|=1$ and $\phi=\varphi\circ\lambda$. The function $\lambda$ lists the eigenvalues of the matrix argument.

With the property of Bregman matrix divergence, the inequality can also be written as \begin{eqnarray} D_\varphi\left(\mathbf{x}^\top\mathbf{V}\Lambda\mathbf{V}^\top\mathbf{x}, \mathbf{x}^\top\mathbf{U}\Theta\mathbf{U}^\top\mathbf{x}\right) &=&D_\varphi\left(\sum_i(\mathbf{v}_i^\top\mathbf{x})^2\lambda_i,\sum_j(\mathbf{u}_j^\top\mathbf{x})^2\theta_j\right)\\ &\leq&\sum_i\sum_j{(\mathbf{v}_i^\top\mathbf{u}_j)^2D_\varphi(\lambda_i,\theta_j)} \end{eqnarray} where $A=V\Lambda V^\top,B=U\Theta B^\top$ are spectral decompositions and $\mathbf{v}_i,\mathbf{u}_j$ are columns of $V,U$ respectively.

My Question is: can this inequality be extended to general Bregman divergence and Bregman matrix divergence with different seed functions chosen?

Or under what condition such an inequality exists?

For example, if $\varphi(\mathbf{x})=\sum_i{x_i\log x_i-x_i}$,

then $D_\varphi$ is the relative entropy (KL-divergence) $$\mathrm{KL}(\mathbf{x},\mathbf{y})=\sum_i\left(x_i(\log x_i-\log y_i)-x_i+y_i\right),$$

and $D_\phi$ is the von Neumann divergence $$D_{vn}(A,B)=\mathrm{tr}\left(A\log A-A\log B-A+B\right).$$

In this case, does the following inequality holds for $\forall\mathbf{x}\in\mathbb{R}^m$ satisfying $\|\mathbf{x}\|=1$? $$ \mathrm{KL}\left(\mathbf{x}^\top A\mathbf{x},\mathbf{x}^\top B\mathbf{x}\right) \leq D_{vN}(A,B) $$

I did many experiments about this inequality about relative entropy and von Neumann divergence with random generalized SPD matrices using Matlab and it always holds. However, does it really hold?

Could anyone please give me some help for this question or recommend some relevant papers? Any suggestion will be appreciated. Thank you very much!

Answer by ppyang for Is it possible to extend this inequality about Euclidean distance &Frobenius norm to more general Bregman divergence such as relative entropy & von Neumann divergence?

2012-04-17T17:11:05Z

I found a proof of this problem for the case of $\varphi(x)=\sum_i{x_i\log x_i-x_i}$. If you find there is anything mistake in the proof, please let me know. Thank you!

The case of $\varphi(x)=\sum_i{x_i\log x_i-x_i}$ can be proved with the method similar to Lindblad, Completely positive maps and entropy inequalities, 1975 and Lindblad, Expectations and entropy inequalities for finite quantum systems, 1974. The inequality can be strengthened as $$ \sum_i{\mathrm{KL}\left(\mathbf{x}_i^\top A\mathbf{x}_i,\mathbf{x}_i^\top B\mathbf{x}_i\right)}\leq D_{vN}\left(A,B\right) $$

Actually, a very similar result has been proposed in some papers about quantum information theory, such as the two papers referred above. The referred result is that for any trace preserving map $\Phi$, given by $\Phi(A)=\sum_{i=1}^n{V_iAV_i^\top}$ and $\sum_{i=1}^n{V_i^\top V_i}=1$, we have that $\mathrm{tr}\left(\Phi(A),\Phi(B)\right)\leq D_\phi(A,B)$, where $A,B$ are both density operators which are Hermitian positive definite matrices satisfying $\mathrm{tr}A=\mathrm{tr}B=1$ and $\varphi(x)=x\log x$.

We found that if the trace constraints $\mathrm{tr}A=\mathrm{tr}B=1$ are dropped and $\varphi(x)=x\log x$ is replaced with $\varphi(x)=x\log x-x$, the inequality still holds.

The proof is outlined as following:

The von Neumann divergence has the following additivity property with Kronecker product: $$D_{vN}(A\otimes P,B\otimes P)=D_{vN}(A,B)\cdot\mathrm{tr}P$$
Using the joint convexity and the additivity, we can prove that the von Neumann divergence has the monotonicity with partial trace as \begin{equation*} \begin{split} D_{vN}(\mathrm{tr}_2(A),\mathrm{tr}_2(B)) =&D_{vN}\left(\mathrm{tr}_2(A)\otimes\frac{\mathbf{I}_2}{m}, \mathrm{tr}_2(B)\otimes\frac{\mathbf{I}_2}{m}\right) /\mathrm{tr}\left(\frac{\mathbf{I}_2}{m}\right)\\ =&D_{vN}\left(\sum_{j=1}^N{p_jW_jAW_j^+},\sum_{j=1}^N{p_jW_jBW_j^+}\right)\\ \leq&\sum_{j=1}^{N}{p_jD_{vN}\left(W_jAW_j^+,W_jBW_j^+\right)}\\ =&D_{vN}(A,B)\end{split} \end{equation*}
For any trace preserving map $\Phi$, given by $\Phi(A)=\sum_{i=1}^n{V_iAV_i^\top}$ and $\sum_{i=1}^n{V_i^\top V_i}=1$, it can be represented as a unitary operation+partial tracing. Therefore, we have that \begin{equation*} \begin{split} D_{vN}\left(\Phi(A),\Phi(B)\right) =&D_{vN}\left(\mathrm{tr}_2(W(A\otimes\mathbf{s}\mathbf{s}^\top)W^\top), \mathrm{tr}_2(W(B\otimes\mathbf{s}\mathbf{s}^\top)W^\top)\right)\\ \leq&D_{vN}\left(W(A\otimes\mathbf{s}\mathbf{s}^\top)W^\top, W(B\otimes\mathbf{s}\mathbf{s}^\top)W^\top\right)\\ =&D_{vN}\left(A,B\right) \end{split} \end{equation*}
Then the sum of relative entropy of the quadratic forms could be represented as matrix divergence and bounded. \begin{equation*} \begin{split} \sum_i{\mathrm{KL}\left(\mathbf{x}_i^\top A\mathbf{x}_i,\mathbf{x}_i^\top B\mathbf{x}_i\right)} =&\sum_{i,j}{(\mathbf{x}_i^\top\mathbf{x}_j)^2 \mathrm{KL}(\mathbf{x}_i^\top A\mathbf{x}_i,\mathbf{x}_j^\top B\mathbf{x}_j)}\\ =&D_{vN}(\sum_i{X_iAX_i^\top},\sum_i{X_iBX_i^\top})\\ \leq&D_{vN}\left(A,B\right) \end{split} \end{equation*} where $X_i=\mathbf{x}_i\mathbf{x}_i^\top$.

If there is any mistake in the proof, please let me know. Any other suggestions are also welcomed. Thank you very much!

How to calculate the maximum of the relative entropy of two quadratics?

2012-04-07T19:33:37Z

Hello, everyone!

Suppose that there are two Hermitian matrices $A,B$ and a unit vector $\mathbf{x}$, then how to calculate the relative entropy of the quadratic forms determined by them $\mathbf{x}^\top A\mathbf{x}$ and $\mathbf{x}^\top B\mathbf{x}$?

It can be equivalent to the following problem: \begin{eqnarray} \max_{\mathbf{x}}&&\mathrm{KL}\left(\mathbf{x}^\top A\mathbf{x},\mathbf{x}^\top B\mathbf{x}\right)\\ \mathrm{s.t.}&&\mathbf{x}^\top\mathbf{x}=1 \end{eqnarray} where $\mathrm{KL}\left(p,q\right)=p\log p-p\log q-p+q$.

Does there exist an analytic solution for this optimization problem? Please give me some help. Any suggestion will be appreciated!

Thank you very much!

Does this inequality of negative relative entropy and quantum relative entropy hold?

2012-04-05T17:18:01Z

Hello, everyone!

Question

I have a question about the relationship between general relative entropy and general quantum relative entropy: Given a unit vector $|i\rangle$ and two Hermitian matrices $A,B$, does the following inequality holds? $$H\left(\langle i|A|i\rangle\parallel\langle i|B|i\rangle\right)\leq S(A\parallel B)$$ where $H(\cdot\parallel\cdot)$ and $S(\cdot\parallel\cdot)$ are general relative entropy and general quantum relative entropy respectively defined as following.

Denote the general negative relative entropy as $$H(p\parallel q)=\sum_i\left(p_i\log\frac{p_i}{q_i}-p_i+q_i\right),$$ and the general quantum relative entropy (von Neumann divergence) as $$S(A\parallel B)=\mathrm{tr}\left(A(\log A-\log B)-A+B\right),$$ where $A,B$ are both positive semi-definite matrices which are not necessarily to be density matrices.

I have repeated the experiments about this inequality with more than 100,000 random generalized SPD matrices using Matlab, and it always holds. However, does it really hold in theory?

Motivation

What motivates this conjecture is the similar inequality of squared Euclidean distance and squared Frobenius norm. Specifically, given a unit vector $|i\rangle$ and two Hermitian matrices $A,B$, the following inequality holds with simple matrix calculations. $$\left\|\langle i|A|i\rangle-\langle i|B|i\rangle\right\|^2\leq\|A-B\|_\mathrm{F}^2.$$ In context of Bregman divergence, squared Euclidean distance and squared Frobenius norm are Bregman divergence and Bregman matrix divergence with the same seed function $\varphi(\mathbf{x})=\sum_ix_i^2$, while general relative entropy and general quantum relative entropy with the same seed function $\varphi(\mathbf{x})=\sum_i\left(x_i\log x_i-x_i\right)$.

Could anyone please give me some help for this question or recommend some relevant papers? Any suggestion will be appreciated.

Thank you very much!

Is there relationship between $f\left({\sum_i(\mathbf{v}_i^\top\mathbf{x})^2\lambda_i},\sum_j{(\mathbf{u}_j^\top\mathbf{x})^2\theta_j}\right)$ and $\sum_i\sum_j{(\mathbf{v}_i^\top\mathbf{u}_j)^2f(\lambda_i,\theta_j)}$ if $f$ is jointly convex?

2012-04-02T16:25:56Z

Hello, everyone!

As we know that by Jensen's inequality, for jointly convex function $f$ and $\sum_ix_i^2=1$, we have $$f(\sum_i{x_i^2\lambda_i},\sum_i{x_i^2\theta_i)}\leq\sum_i{x_i^2f(\lambda_i,\theta_i)}\leq\max_if(\lambda_i,\theta_i)\leq\sum_if(\lambda_i,\theta_i),$$ and we get a bound of $f(\sum_i{x_i^2\lambda_i},\sum_i{x_i^2\theta_i)}$ independent of $\{x_i\}$.

However, I wonder if this inequality can be extended to the case where the probability distribution $\{x_i\}$ on the two variables of $f$ are not identical but just constrained.

To be more specifically, suppose that $f:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}$ is jointly convex with both its arguments, $V=[\mathbf{v}_1\;\mathbf{v}_2\;\ldots\;\mathbf{v}_m]$ and $U=[\mathbf{u}_1\;\mathbf{u}_2\;\ldots\;\mathbf{u}_m]$ are orthogonal matrices and thus $\{\mathbf{v}_i\},\{\mathbf{u}_j\}$ consist an orthonormal basis in $\mathbb{R}^m$ respectively.

Then for any $\mathbf{x}\in\mathbb{R}^m$ satisfying $\|\mathbf{x}\|=1$, I wonder if there is a relationship between $L_1$ and $L_2$ shown in the following two formulas. \begin{eqnarray} L_1&=&f\left({\sum_i(\mathbf{v}_i^\top\mathbf{x})^2\lambda_i},\sum_j{(\mathbf{u}_j^\top\mathbf{x})^2\theta_j}\right)\\ L_2&=&\sum_i\sum_j{(\mathbf{v}_i^\top\mathbf{u}_j)^2f(\lambda_i,\theta_j)} \end{eqnarray}

In language of matrix, $L_1$ can also be formulated as $f\left(\mathbf{x}^\top V\Lambda V^\top\mathbf{x},\mathbf{x}^\top U\Theta U^\top\mathbf{x}\right)$.

Considering that $\sum_i(\mathbf{v}_i^\top\mathbf{x})^2=\sum_j(\mathbf{v}^\top\mathbf{x})^2=1$, my question is that does there exist an inequality about $L_1$ and $\gamma L_2$ where $\gamma$ is any constant independent of $\mathbf{x}$?

How should I consider about this problem? Or are there any papers about this topic for reference?

Could anyone be so kind to help me about this question? Any suggestion will be appreciated! Thank you very much!

Remark:

I tried to simply apply the Jensen's inequality to $L_1$ and get the result $$L_1\leq\sum_i\sum_j{(\mathbf{v}_i^\top\mathbf{x})^2(\mathbf{u}_j^\top\mathbf{x})^2f(\lambda_i,\theta_j)}.$$ Does there exists any relationship between this formula and $L_2$?

Any suggestion will be appreciated! Thank you very much!

Given a jointly convex function $f$, what is the bound of $f\left(\sum_ip_i^2x_i,\sum_jq_j^2y_j\right)$if $\mathbf{p},\mathbf{q}$ are constrained in a manifold?

2012-03-24T15:11:05Z

Suppose there is a jointly convex function $f:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}$, $\mathbf{x},\mathbf{y}\in\mathbb{R}^m$ and $\mathbf{p}=[p_1\;\ldots\;p_m]^\top,\mathbf{q}=[q_1\;\ldots\;q_m]^\top$ satisfying $\|p\|^2=\sum_ip_i^2=\|q\|^2=\sum_jq_j^2=1$, I want to find the bound of $f\left({\mathbf{p}^2}^\top\mathbf{x},{\mathbf{q}^2}^\top\mathbf{y}\right)$ by decomposing it into weighted sum of $f(x_i,y_j)$ if $\mathbf{p},\mathbf{q}$ are constrained in a manifold.

The case without constraints of $\mathbf{p}$ and $\mathbf{q}$

Denote the element-wise squared vector of $\mathbf{p},\mathbf{q}$ as $$\mathbf{p}^2=[p_1^2\;\ldots\;p_m^2]^\top,\mathbf{q}^2=[q_1^2\;\ldots\;q_m^2]^\top,$$ then by Jensen's inequality for jointly convex function, we have \begin{eqnarray} f\left({\mathbf{p}^2}^\top\mathbf{x},{\mathbf{q}^2}^\top\mathbf{y}\right) &=&f\left(\sum_ip_i^2x_i,\sum_jq_j^2y_j\right)\\ &=&f\left(\sum_i\left(p_i^2\sum_jq_j^2x_i\right),\sum_j\left(q_j^2\sum_ip_i^2y_j\right)\right)\\ &=&f\left(\sum_i\sum_jp_i^2q_j^2x_i,\sum_i\sum_jp_i^2q_j^2y_j\right)\\ &\leq&\sum_i\sum_j{p_i^2q_j^2f(x_i,y_j)}\\ &=&{\mathbf{p}^2}^\top F\mathbf{q}^2 \end{eqnarray} where $F_{ij}=f(x_i,y_j)$. This provides a general bound for $f\left({\mathbf{p}^2}^\top\mathbf{x},{\mathbf{q}^2}^\top\mathbf{y}\right)$.

Suppose $C=[c_{ij}]_{m\times m}$ is an orthogonal matrix and the element squared matrix of $C$ is $$C^2=[c_{ij}^2]_{m\times m},$$ then $C^2$ is a doubly stochastic matrix where $C^2\mathbf{1}=\mathbf{1}$and ${C^2}^\top\mathbf{1}=\mathbf{1}$.

The case if with constraint ${\mathbf{q}^2}={C^2}^\top\mathbf{p}^2$

If an additional constraint $\mathbf{q}^2={C^2}^\top\mathbf{p}^2$ is applied so that $\mathbf{p}$ and $\mathbf{q}$ are constrained on a manifold, we can substitute the constraint into the inequality above, we get the bound \begin{eqnarray} f\left({\mathbf{p}^2}^\top\mathbf{x},{\mathbf{q}^2}^\top\mathbf{y}\right) \leq{\mathbf{p}^2}^\top F{C^2}^\top\mathbf{p}^2 \end{eqnarray}

On the other hand, applying Jensen's inequality directly on the original formular, we can obtain another bound as \begin{eqnarray} f\left({\mathbf{p}^2}^\top\mathbf{x},{\mathbf{q}^2}^\top\mathbf{y}\right) &=&f\left(\sum_ip_i^2x_i,\sum_j\sum_ic_{ij}^2p_i^2y_j\right)\\ &=&f\left(\sum_ip_i^2x_i,\sum_i\left(p_i^2\sum_jc_{ij}^2y_j\right)\right)\\ &\leq&\sum_ip_i^2f\left(x_i,\sum_jc_{ij}^2y_j\right)\\ &\leq&\sum_i\sum_j{p_i^2c_{ij}^2f(x_i,y_j)}\\ &=&{\mathbf{p}^2}^\top(F\circ C^2)\mathbf{1} \end{eqnarray} where $\circ$ is the Hadamard (element-wise) product.

It is notable that this bound is simpler than the one obtained by substitution and has $\mathbf{p}$ is homogeneous in both sides of the inequality.

What is the case if with constraint $\mathbf{q}=C^\top\mathbf{p}$?

My question is that if the constraint is replaced with $\mathbf{q}=C^\top\mathbf{p}$, could we obtain the bound of $f\left({\mathbf{p}^2}^\top\mathbf{x},{\mathbf{q}^2}^\top\mathbf{y}\right)$ with Jensen's equality? If such a bound exists, could it have the simple form where $\mathbf{p}$ similar to ${\mathbf{p}^2}^\top(F\circ C^2)\mathbf{1}$?

Could anyone be so kind to give me some help on this problem? Any suggestion will be appreciated! Thank you very much!

Given $\mathbf{x}_i^\top A\mathbf{x}_i$ for a SPD matrix $A$ and orthonormal bases $\mathbf{x}_i$, what is the bound of its eigenvalues?

2012-03-01T08:38:26Z

Assume that $A_{d\times d}$ is a symmetric positive semi-definite matrix, and $\{\mathbf{x}_1,\ldots,\mathbf{x}_d\}$ composes a group of orthogonal bases of $\mathbb{R}^d$ where $\mathbf{x}_i\bot\mathbf{x}_j,\forall i\neq j$ and $\|\mathbf{x}_i\|=1$.

Then, my question is that given $$ \mathbf{x}_i^\top A\mathbf{x}_i=a_i,\forall i $$ how to estimate the bound of the eigenvalues of $A$?

Using eigenvalue decomposition $A=U\Lambda U^\top$ and let $\tilde{\mathbf{x}}_i=U^\top\mathbf{x}_i$, this question can be simplified as to estimate the bound of the elements of the diagonal matrix $\Lambda$ given $$ \tilde{\mathbf{x}}_i^\top\Lambda\tilde{\mathbf{x}}_i=a_i,\forall i. $$ Since $U$ is orthogonal, we also have $\tilde{\mathbf{x}}_i\bot\tilde{\mathbf{x}}_j,\forall i\neq j$ and $\|\tilde{\mathbf{x}}_i\|=1$.

Denote $\Lambda=\mathrm{diag}[\lambda_1\;\ldots\;\lambda_d]^\top$ and $\tilde{\mathbf{x}}_i=[\tilde{x}_{i1}\;\ldots\;\tilde{x}_{id}]$, the equations above can be represented as $$ \sum_j\lambda_j\tilde{x}_{ij}^2=a_i,\forall i $$

However, this question seems to be not so simple yet.

In the case of $d=2$, I found the following method to solve it.

Denote $\tilde{\mathbf{x}}_1=[\cos\theta\;\sin\theta]^\top$ and $\tilde{\mathbf{x}}_2=[-\sin\theta\;\cos\theta]^\top$, we have \begin{eqnarray} \lambda_1\cos^2\theta+\lambda_2\sin^2\theta&=&a_1\\ \lambda_1\sin^2\theta+\lambda_2\cos^2\theta&=&a_2 \end{eqnarray} Solve the equation system and we get \begin{eqnarray} \lambda_1&=&\frac{1}{\Delta}(a_1\cos^2\theta-a_2\sin^2\theta) =\frac{a_1-a_2\tan^2\theta}{1-\tan^2\theta}=a_2+\frac{a_1-a_2}{1-\tan^2\theta}\\ \lambda_2&=&\frac{1}{\Delta}(-a_1\sin^2\theta+a_2\cos^2\theta) =\frac{-a_1\tan^2\theta+a_2}{1-\tan^2\theta}=a_1-\frac{a_1-a_2}{1-\tan^2\theta} \end{eqnarray} where $\Delta=\cos^4\theta-\sin^4\theta=\cos^2\theta-\sin^2\theta$.

Without loss of generality, assume that $a_1\geq a_2$ and $\lambda_1>\lambda_2$, we can deduce that $1-\tan^2\theta>0$, then from $\lambda_i\geq 0$, we have $$\tan^2\theta\leq\min\{\frac{a_2}{a_1},\frac{a_1}{a_2}\}=\frac{a_2}{a_1}$$ Therefore it is obvious that when $\tan^2\theta=\frac{a_2}{a_1}$, $\lambda_1$ and $\lambda_2$ take its maximum and minimum respectively as \begin{eqnarray} \lambda_1&=&a_1+a_2\\ \lambda_2&=&0 \end{eqnarray} and we get the bound of the eigenvalues of $A$.

However, how to solve the problem if $d\geq3$?

Any suggestion will be appreciated! Thank you very much!

What is the minimum of the Frobenius norm in the intersection of positive semidefinite cones?

2012-02-21T17:56:31Z

For scalar variables $x$, we have a simple solution for the following problem. \begin{eqnarray} \min_x&&\alpha(x-a)^2+\beta(x-b)^2 \\ \mathrm{s.t. }&&x\leq a\\ &&x\leq b \end{eqnarray} where $\alpha,\beta>0$.

The optimal solution $x=\min(a,b)$ is straightforward and independent of $\alpha,\beta$.

However, in the case of real symmetrical matrix variables, the problem seems to be much more complex because the relationship $\leq$ in constraints has to be replaced with $\preceq$ which is a more complex relationship defined as $$ A\preceq B\Leftrightarrow A-B\preceq0 $$ where $A\preceq0$ means $A$ is negative semi-definite.

Then the problem above is reformulated as following with matrix variables. Assume that all matrices in this problem are real symmetrical. \begin{eqnarray} \min_X&&\alpha\|X-A\|_F^2+\beta\|X-B\|_F^2\\ \mathrm{s.t.}&&X\preceq A\\ &&X\preceq B \end{eqnarray} where $\alpha,\beta\geq0$.

If $A$ and $B$ can be diagonalized by the same orthogonal matrix $U$, the problem reduces to a the problems of eigenvalues since $$ \alpha\|X-A\|_F^2+\beta\|X-B\|_F^2=\alpha\|\tilde{X}-\Lambda\|_F^2+\beta\|\tilde{X}-\Theta\|_F^2 $$ and \begin{eqnarray} X\preceq A\Leftrightarrow\tilde{X}\preceq\Lambda\\ Y\preceq B\Leftrightarrow\tilde{X}\preceq\Theta \end{eqnarray} where $\tilde{X}=U^\top XU$, $\Lambda=U^\top AU$, $\Theta=U^\top BU$. Since $\Lambda$ and $\Theta$ are diagonal matrices, the problem can decompose to sum of some scalar variables and solved independently.

However, how to solve it if $A$ and $B$ have different eigenvectors? Is the solution independent of $\alpha$ and $\beta$ yet?

Could any one be so kind to help me with the question or give some suggestions? Thank you very much!

How to calculate the integral of the exponential of negative unnormalized relative entropy?

2012-02-04T16:32:55Z

I want to calculate the integral of the exponential of negative unnormalized relative entropy shown as follows. This integral seems to be finite but I don't know how to calculate it. $$ \int_0^{+\infty}{e^{-(x\log x-x\log y-x+y)}dx} $$ where $y$ is a known as a positive constant.

Could anyone be so kind to help me with some suggestions? Thank you very much!

How to deal with the vector norm item as a denominator in this expectation?

2011-12-13T07:24:50Z

Hello, everyone. I want to calculate the expectation shown in the following formula, where $X$ follows a standard $d$-dimensional multi-variable normal distribution as $X\sim\mathbb{N}(\mathbf{0},\mathbb{I}_{d\times d})$ $$ E_X\left[\frac{X^\top\mathbf{A}XX^\top\mathbf{B}X}{\|X\|^2}\right] $$ where $\|X\|^2=X^\top X$, and $\mathbf{A},\mathbf{B}$ are both real symmetric matrices.

The expectation of the numerator is straightforward and there was result for it. However, the denominator seems to make the problem more difficult. Are there any simple approaches to deal with this problem? Thank you very much!

How to calculate this expectation with logarithm?

2011-11-21T17:10:10Z

If $\mathbf{x}\sim\mathbf{N}(\mathbf{0},\mathbf{I})$, and assume that $A$ is a symmetric positive definite matrix, how can I calculate the following two expectations where there is a logarithm in it? $$ \mathrm{E}_\mathbf{x}\left[\log(\mathbf{x}^\top A\mathbf{x})\right] $$ and

$$ \mathrm{E}_\mathbf{x}\left[(\mathbf{x}^\top A\mathbf{x})\log(\mathbf{x}^\top A\mathbf{x})\right] $$

Using the decomposition $A=U\Lambda U^\top$ and $\mathbf{y}=U^\top\mathbf{x}$, the first expectation can be reduced to $$ \mathrm{E}_\mathbf{y}\left[\log(\mathbf{y}^\top \Lambda\mathbf{y})\right] $$

$$ =\mathrm{E}_\mathbf{y}\left[\log\left(\sum_i{\lambda_i\mathbf{y}_i^2}\right)\right] $$ where $y$ has the same distribution as $x$ because $U$ is a orthogonal matrix.

However, since the random variables $y_i$ are in the logarithm function, I cannot decompose the expectation further and such a expectation seems to be difficult to calculate.

Could you please help me with this problem or give some suggestions about it? Thank you very much!

How to calculate this expectation where the random variable is restricted on a sphere?

2011-11-20T13:35:33Z

Hello! I have a question about how to calculate the expectation of a quadratic form as follows, where $X$ is a random variable that uniformly distributed on the unit sphere: $$ E_X[(\mathbf{x}^\top A\mathbf{x})^2] =\int_{\mathbf{x}\in S}{p(\mathbf{x})(\mathbf{x}^\top A\mathbf{x})^2dS(\mathbf{x})} =\int_{\mathbf{x}\in S}{\frac{1}{4\pi}(\mathbf{x}^\top A\mathbf{x})^2dS(\mathbf{x})} $$ where $S={\mathbf{x}\in\mathcal{R}^N|\mathbf{x}^\top\mathbf{x}=1}$.

If $\mathbf{x}$ is Gaussian, there are some conclusions about the expectation of the quadratic forms. However, I find it difficult to deal with when the variable is distributed on a sphere. When the dimension is 2 or 3, this problem can be solved by representing it with polar coordinate. However, when the dimension is high, such a representation will be rather redundant, how can I calculate this integral then? Please give some help for this problem if you have any idea. Thank you very much!

Comment by ppyang

2013-02-25T00:35:27Z

@Suvrit Sorry to have made confusion by using different variables in the definition of $Q$. Thank you for your attention, but I cannot understand your last sentense clearly. Can you explain how to use this formula to solve my question in more detail? Thank you very much!

Comment by ppyang

2013-02-23T12:30:59Z

@Suvrit Thank you for your attention! Do you mean there is a formula for $\frac{\partial Q(x,λ)}{\partial x}$ in wikipedia? I am sorry that I did not find such a formula but only the formula $\frac{\partial\Gamma(x,λ)}{\partial x}=\ln\lambda\Gamma(x,\lambda)+\lambda T(3,x,\lambda)$ instead. If you found the formula for the derivative of $Q(x,\lambda)$ w.r.t. $x$, could you please show me the url of the wikipedia? Thank you very much!

Comment by ppyang

2013-02-18T09:32:33Z

It's helpful of your answer. However there are some points in your answer that I cannot understand clearly since I do not major in mathematics. Could you please recommend some references or papers to explain your answer in more detail, please? Thank you very much!

Comment by ppyang

2013-02-18T09:17:01Z

I think both of your answers are very helpful. However, since I cannot accept both of your answers in the system and the answer given by Charles Rezk is in more detail, I choose to set his answer as the accepted answer. Anyway, it's very kind of you and thank you very much!

Comment by ppyang

2013-02-18T08:12:28Z

It's very kind of you for your help! Could you tell me the title of Adams's paper mentioned in your answer, please? I would like to understand how the result is obtained. Thank you!

Comment by ppyang

2012-12-13T13:43:28Z

@Robert Bryant, I am sorry to trouble you all with this simple question and I have solved it. However, I have to say that this is indeed not my homework or exam problem. I will think more by myself before posting the question next time. Thank you very much!

Comment by ppyang

2012-11-29T02:22:45Z

I am reading the papers you suggested and they indeed provide an interesting standpoint for this question. Thank you very much!

Comment by ppyang

2012-06-17T11:38:42Z

You provide an excellent idea that reduce the most complex calculation of an $mn\times mn$ matrix into a $m\times m$ matrix and a $n\times n$ matrix, where the inverse is taken on only a diagonal. I think it solve my problem. Thank you very much!

Comment by ppyang

2012-06-12T17:34:29Z

@Felix Goldberg: I am still interested in whether there exists some way to simplify this calculation but I have no idea yet.

Comment by ppyang

2012-06-12T02:12:02Z

@Federico Poloni, thank you for your suggestion. What is more, the purpose that I took that example is to show that I want to simplify the calculation utilizing the property of Kronecker product, however, there is some mistake in it. Thank you!

Comment by ppyang

2012-06-12T02:07:20Z

@Federico Poloni, yes, you are right! $(XX^\top)^{-1}$ really does not exist. Thank you!

Comment by ppyang

2012-04-17T17:39:45Z

I found a proof of this problem and the inequality does holds. A related problem has been studied several years ago. I have presented the outline of the proof in another question as the following url: mathoverflow.net/questions/93149/… If there is any mistake in the proof, please let me know. Thank you!

Comment by ppyang

2012-04-17T14:45:48Z

I have found the proof of the case of $\varphi(x)=\sum_i{x_i\log x_i-x_i}$ and the inequality $\mathrm{KL}\left(\mathbf{x}^\top A\mathbf{x},\mathbf{x}^\top B\mathbf{x}\right)\leq D_{vN}(A,B)$ indeed holds. I will present the proof soon.

Comment by ppyang

2012-04-03T10:01:34Z

Thank you for your counter example!

Comment by ppyang

2012-04-03T02:29:37Z

@Marc, OK! I will remember this next time. Thanks!