Skip to main content
MathOverflow

Return to Question

added 18 characters in body
Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

A random variable $X$ is said to be sub-gaussian with mean $\mu$ and pseudo-variance $\sigma^2$ iff $$\mathbb \log(E[\exp(t(X-\mu))]) \le \frac{t^2}{2\sigma^2},\;\forall t \in \mathbb R. $$

It's a standard computation that the KL divergence between two random variables with means $\mu_1$ and $\mu_2$ and same variance $\sigma^2 > 0$ is $\dfrac{(\mu_1 - \mu_2)^2}{2\sigma^2}$.

Question

What is good upper-bound for the KL divergence between two sub-gaussian variables with means $\mu_1$ and $\mu_2$ and same pseudo-variance $\sigma^2 > 0$, and full support ?

A random variable $X$ is said to be sub-gaussian with mean $\mu$ and pseudo-variance $\sigma^2$ iff $$\mathbb \log(E[\exp(t(X-\mu))]) \le \frac{t^2}{2\sigma^2},\;\forall t \in \mathbb R. $$

It's a standard computation that the KL divergence between two random variables with means $\mu_1$ and $\mu_2$ and same variance $\sigma^2 > 0$ is $\dfrac{(\mu_1 - \mu_2)^2}{2\sigma^2}$.

Question

What is good upper-bound for the KL divergence between two sub-gaussian variables with means $\mu_1$ and $\mu_2$ and same pseudo-variance $\sigma^2 > 0$ ?

A random variable $X$ is said to be sub-gaussian with mean $\mu$ and pseudo-variance $\sigma^2$ iff $$\mathbb \log(E[\exp(t(X-\mu))]) \le \frac{t^2}{2\sigma^2},\;\forall t \in \mathbb R. $$

It's a standard computation that the KL divergence between two random variables with means $\mu_1$ and $\mu_2$ and same variance $\sigma^2 > 0$ is $\dfrac{(\mu_1 - \mu_2)^2}{2\sigma^2}$.

Question

What is good upper-bound for the KL divergence between two sub-gaussian variables with means $\mu_1$ and $\mu_2$ and same pseudo-variance $\sigma^2 > 0$, and full support ?

Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

Upper-bound KL divergence between sub-gaussian variables with same variance

A random variable $X$ is said to be sub-gaussian with mean $\mu$ and pseudo-variance $\sigma^2$ iff $$\mathbb \log(E[\exp(t(X-\mu))]) \le \frac{t^2}{2\sigma^2},\;\forall t \in \mathbb R. $$

It's a standard computation that the KL divergence between two random variables with means $\mu_1$ and $\mu_2$ and same variance $\sigma^2 > 0$ is $\dfrac{(\mu_1 - \mu_2)^2}{2\sigma^2}$.

Question

What is good upper-bound for the KL divergence between two sub-gaussian variables with means $\mu_1$ and $\mu_2$ and same pseudo-variance $\sigma^2 > 0$ ?