Let us define a distance metric between two joint probability math functions $p(x,y)$ and $q(x,y)$ as in the following

\begin{align} \sum_{y}\sqrt{\sum_{x}p(x)\left(p(y|x)-q(y|x)\right)^2}. \end{align}

By Jensen inequality, we have

\begin{align*} \sum_{y}\sqrt{\sum_{x}p(x)\left(p(y|x)-q(y|x)\right)^2}&\geq\sum_{x,y}\lvert p(x,y)-q(x,y)\rvert\\ &=\lVert p(x,y)-q(x,y)\rVert_1 \end{align*}

where $\lVert\cdot\rVert_1$ is the total variation distance.

What is the name of this distance? Is there any other relationship between this distance and any other stochastic distances?

  • $\begingroup$ are you assuming $q(x,y) = q(y|x)p(x)$? $\endgroup$
    – Memming
    Sep 11, 2017 at 23:43
  • $\begingroup$ some sort of nested Csiszár's f-divergence? $\endgroup$
    – Memming
    Sep 11, 2017 at 23:50
  • $\begingroup$ Is this symmetric? $\endgroup$ Sep 12, 2017 at 1:42
  • $\begingroup$ It is symmetric for $p(x)=q(x)$. $\endgroup$
    – Math_Y
    Sep 12, 2017 at 3:29

1 Answer 1


\begin{align*} \sum_{y}\sqrt{\sum_{x}p(x)\left(p(y|x)-p(y)\right)^2} &=\sum_{y}p(y)\sqrt{\sum_{x}p(x)\left(\frac{p(y|x)}{p(y)}-1\right)^2}\\ &=\sum_{y}p(y)\sqrt{\sum_{x}p(x)\left(\frac{p(x|y)}{p(x)}-1\right)^2}\\ &=\sum_{y}p(y)\sqrt{\sum_{x}\frac{p^2(x|y)}{p(x)}-2p(x|y)+p(x)}\\ &=\sum_{y}p(y)\sqrt{\sum_{x}\frac{p^2(x|y)}{p(x)}-1}\\ &=\mathbb{E}_Y\sqrt{\chi^2\left(p(x|y)\parallel p(x)\right)} \end{align*} where $\chi^2(\cdot||\cdot)$ is chi-square distance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.