Search Results

Let $\mu$ be a Borel probability measure on $\mathbb{R}^n$. For a linear subspace $E\subset \mathbb{R}^n$, let $\mu_E$ denote the marginal of $\mu$ on $E$. The usual orthogonal complement of $E$ is de …

Since relative entropy behaves locally like a squared distance, we might expect the squared Fisher-Rao metric to be comparable to the symmetrized KL divergence. This is indeed the case. Let $d_F$ den …

If you only have the hypothesis of sub-Gaussianity, this is the best you can do. Work in dimension $n=1$ for simplicity, let $X\sim N(0,1)$, and let $A$ be independent of $X$. If $AX$ is to be sub-G …

Let $\mathcal{G}_n = \{ N(\mu,\Sigma) ; \mu \in \mathbb{R}^n, \Sigma > 0\}$ be the collection of Gaussian distributions on $\mathbb{R}^n$ with full support. If $f : \mathbb{R}^n \to \mathbb{R}^k$ is m …

This is impossible if $f$ is injective, without further assumptions such as bijective, differentiable, etc. Let $Q_1,Q_2$ be probability measures on a measurable space $(\Omega, \mathcal{F})$, and as …

You are missing dimensional constants, but you have the right idea. Namely, it is true that $$ E|X-\hat{X}(Y)|^2 \geq \frac{m}{2\pi e} e^{2 h(X|Y)/m }. $$ Like in the 1-dimensional case, this is a co …

The characterization is given in terms of a so-called auxiliary random variable. It is as explicit of an answer as you'll get, unless you consider very special cases (like jointly Gaussian $X,Y$, or …

I'm a bit hesitant to resurrect an old post, but as a result of some of the things I worked on recently, I'd like to share a new answer to the question in the title. Hopefully some will find it inte …