Given two Gaussian random variables A and B with (mean, standard deviation) of (a,s) and (b,m) respectively, is there a scalar w in [0,1] that indicates how close A and B are?
$\begingroup$
$\endgroup$
2
-
3$\begingroup$ Instinctively, the answer depends on a closeness metric. If I care about |x-y|, where x and y are draws, the resulting metric will be very different from if my metric is $(x-y)^2$ $\endgroup$– Cort AmmonCommented Apr 29, 2022 at 20:30
-
1$\begingroup$ There are lots and lots of different statistical distances and divergences: metrics (or nonsymmetric generalizations thereof) on spaces of probability measures. The field of optimal transport is largely focused on studying such distances. So in short: yes. But to get a more useful answer, you should maybe clarify why you want this and what you would use it for. $\endgroup$– Nate EldredgeCommented Apr 30, 2022 at 19:47
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
As the measure of the closeness of two distributions $p_A$ and $p_B$ You could use the Bhattacharyya coefficient $$w=\int \sqrt{p_A(x)p_B(x)}\,dx\in[0,1],$$ which for two Gaussian distributions (means $a,b$; variances $s^2$, $m^2$) is given by $w=e^{-d}$ with $$d=\frac{1}{4} \ln \left [ \frac 1 4 \left( \frac{s^2}{m^2}+\frac{m^2}{s^2}+2\right ) \right ] +\frac{1}{4} \frac{(a-b)^{2}}{s^2+m^2}. $$
answered Apr 29, 2022 at 20:39
-
$\begingroup$ For help with interpretation see for instance stats.stackexchange.com/questions/296361/… $\endgroup$ Commented Apr 30, 2022 at 1:08
-
1$\begingroup$ Alternate interpretation: Taking the square root converts both distributions into unit vectors in $L^2$, and then the inner product gives the cosine of the angle between these vectors. $\endgroup$ Commented Apr 30, 2022 at 19:58
$\begingroup$
$\endgroup$
This depends really on the use you will have of that distance ... For many (statistical) purposes an asymmetric divergence is more natural than a symmetric distance. So look into the Kullback-Leibler divergence, for univariate normals see https://stats.stackexchange.com/a/7449/11887, for the multivariate case see https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence#Multivariate_normal_distributions.
For help with interpretations see https://stats.stackexchange.com/questions/188903/intuition-on-the-kullback-leibler-kl-divergence/189758#189758
answered Apr 30, 2022 at 1:16
$\begingroup$
$\endgroup$
The Kolmogorov distance is a possible answer. I derived it in this article of my former blog (at the end of the article).
answered Apr 30, 2022 at 6:52