All Questions
68 questions
23
votes
7
answers
5k
views
What makes Gaussian distributions special?
I'm looking for as many different arguments or derivations as possible that support the informal claim that Gaussian/Normal distributions are "the most fundamental" among all distributions.
...
16
votes
6
answers
3k
views
A normal distribution inequality
Let $n(x) := \frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}$, and $N(x) := \int_{-\infty}^x n(t)dt$. I have plotted the curves of the both sides of the following inequality. The graph shows that the ...
- 2,239
13
votes
1
answer
10k
views
KL divergence and mixture of Gaussians
Do we have an exact formula to compute the KL divergence between 2 mixtures of Gaussians (i.e convex combinations of a finite number of Gaussian distributions)?
If not exactly known, are there good ...
- 2,246
8
votes
0
answers
422
views
Non-affine smooth transformation of Gaussian is Gaussian
Suppose $Z\sim N(0,1)$ (standard Gaussian) and $f: \mathbb{R} \to \mathbb{R}$ is a differentiable function such that $f(Z)\sim N(0,1)$. My question is whether there exists any such $f$ other than $f(x)...
- 399
6
votes
1
answer
237
views
Ordering preference for two zero mean Gaussian outcomes
Let $X\sim \mathcal{N}(0,1)$ be a standard Gaussian random variable. If we let $f_a(x)\triangleq\mathbb{E}[\max\{aX,x\}]$ for $a,x >0$, how to prove that $$f_a(f_b(1))<f_b(f_a(1))~~\text{for }0&...
- 171
6
votes
1
answer
264
views
Which orthant probabilities are the largest? (For a multivariate normal distribution)
I have a $k$-dimensional multivariate normal distribution $X∼N(0,\Sigma)$ with covariance matrix $\Sigma$. $\Sigma$ has two distinct eigenvalues, say $\lambda_1 > \lambda_2$, with orthogonal ...
5
votes
3
answers
665
views
The relative error of approximating a binomial
Are there any good approximations for a binomial CDF that work well in terms of the relative error, as opposed to absolute? For the usual normal approximation, the absolute error is very well-studied ...
- 4,049
5
votes
1
answer
942
views
Moments of maximum of independent Gaussian random variables
Let $X = (X_1, \ldots, X_d) \in \mathbb{R}^d$ be a mean-zero Gaussian random vector with identity covariance matrix. Are there upper bounds for
$$E \left(\|X\|_{\infty}^k \right)$$ for $k=1, \ldots, ...
- 519
5
votes
1
answer
392
views
comparing Gaussian to order statistic of Gaussian
I would like to compute the probability of
$$\mathbb{P}[Y > \max(X_i)], Y\sim N(0, 1), X_i \sim N(0, \sigma_i)$$
All the random variables have zero mean, but the variances are different.
My ...
- 151
5
votes
1
answer
225
views
Anti-concentration of Gaussian when conditioning on event
Let $v$ be a given vector with $\|v\|_{\Sigma^{-1}} \leq 1$, where $\Sigma$ is a positive semi-definite matrix and $\|v\|_{\Sigma^{-1}} = \sqrt{v^\top\Sigma v}$. Meanwhile, let $u$ be a random vector ...
- 1,127
5
votes
2
answers
174
views
Integrability of Gaussian sums
Let $(X_1, \ldots, X_n)$ be a Gaussian vector, and $Z = \sum_{i=1}^n |X_i|$.
Since the map $x \mapsto e^{x^2}$, is convex, for any $t>0$
$$
e^{tZ^2} \, = \, e^{t \big(\sum_{i=1}^n |X_i| \big)^2}...
- 51
5
votes
1
answer
1k
views
Explicit constant for Carbery–Wright inequality
The Carbery–Wright inequality is a seminal result about the anti-concentration of polynomials of Gaussian random variables.
See e.g. Meka, Nguyen, and Vu - Anti-concentration for polynomials of ...
- 111
5
votes
0
answers
204
views
anti-concentration of multi-linear polynomials in Gaussian variables
A Gaussian variable $X_i\sim {\cal N}(0,1)$ is anti-concentrated in the following sense: for any $\epsilon>0$ we have:
$$
\mathbf{P}( |X_i| \leq \epsilon ) = O(\epsilon).
$$
Hence if we consider a ...
- 445
4
votes
1
answer
681
views
Tail bound for product of normal distribution
Let $U, V$ be two standard normal random variables with covariance $cov(U,V) = \beta \in [0,1)$. Let $W = UV$ be the product of two RV's, and $W_1, W_2, \ldots, W_n$ be n i.i.d copies of $W$, what's ...
- 515
4
votes
1
answer
771
views
Maximal component of a multivariate Gaussian distribution
Suppose you have a general random Gaussian vector $\mathbf{X}\sim\mathcal{N}\left(\boldsymbol{\mu},\boldsymbol{\Sigma}\right)$. I'm looking for the simple way to calculate the distribution of the ...
- 968
4
votes
1
answer
349
views
Variance of maximum of mixture of gaussians
Let $\{X_i\}$ be an iid collection of standard normal $(N(0,1))$ random variables . Let $X = (X_1,\ldots,X_n)$, and consider a function of the form $f(X) = \max(A\cdot X)$, where $A$ is some symmetric,...
- 941
4
votes
2
answers
2k
views
Distribution of a product of two discrete i.i.d. variables
The problem is to estimate the distribution of product of two $\textit{discretized Gaussian}$ random variables with zero means. The discretized Gaussian means that the p.m.f. looks like
$D_s(x)=\...
4
votes
0
answers
2k
views
Show that $\mathbb{P}[ a V\le Z| V+Z]=\mathbb{P}[aV \ge Z| V+Z] \text{ a.s.} $ iff $V=\frac{1}{\sqrt{a}}Z'$ where $Z'$ is standard normal
Consider a pair of independent random variables $(V,Z)$ where $Z$ is standard normal. Now suppose that the following equality holds: for a given $a>0$
\begin{align}
\mathbb{P}[ a V\le Z| V+Z]=\...
- 671
3
votes
3
answers
2k
views
How close are two Gaussian random variables?
Given two Gaussian random variables A and B with (mean, standard deviation) of (a,s) and (b,m) respectively, is there a scalar w in [0,1] that indicates how close A and B are?
- 159
3
votes
1
answer
1k
views
Computation complexity of calculating the cdf of an n-th dimensional gaussian random vector
Suppose you have a general $n$-th dimensional random Gaussian vector with probability distribution function $\mathcal{N}\left(\mathbf{x}|\boldsymbol{\mu},\boldsymbol{\Sigma}\right)$.
What is the ...
- 968
3
votes
1
answer
694
views
Asymptotic bound on the total variation distance between a standard multivariate normal and a simple mixture
Let $P = N(\vec{0}, I^d)$ be a standard multivariate Gaussian distribution in $d$ dimensions. Let $Q$ be distributed the same as $P$, except that samples from $Q$ have one of their coordinates, chosen ...
- 167
3
votes
1
answer
113
views
maximum likelihood estimation of X is better than that of f(X)?
Consider a binary variable $C$ with $\Pr(C=0)=\Pr(C=1)=0.5$. Consider a random vector $X \in \mathbb{R}^d$, characterized by $C$, with PDF $p_m(x)$, $m\in\{0,1\}$. Define a maximum likelihood (ML) ...
- 482
3
votes
1
answer
1k
views
Normal approximation to the pointwise/Hadamard/Schur product of two multivariate Gaussian/normal random variables
Let $X \sim \mathcal{N}\left( {{\mu _x},\sigma _x^2} \right)$ and $Y \sim \mathcal{N}\left( {{\mu _y},\sigma _y^2} \right)$ be two univariate and independent Gaussian/normal random variables and let $...
- 1,026
3
votes
1
answer
460
views
Derive concentration bound for the derivative
It that true to conclude that if a random $f(z)$ is a sub-Gaussian random variable for a constant value of z, its derivative $f'(z)|_{z=k}$ with respect to variable $z$ is also sub-Gaussian?
In ...
3
votes
0
answers
131
views
Matrix-Gaussian distributions
The point of this question is to ask for references on matrix-variate Gaussian distributions. But I will explain what I mean by a matrix-variate Gaussian with an example (the notion I have in mind is ...
- 193
3
votes
0
answers
354
views
Moments of normalized multivariate Gaussians (and Wick's/Isserlis theorems)
Suppose $x = \begin{bmatrix}x_1 \\ x_2\end{bmatrix}$ is distributed according to the real two-dimensional Gaussian with mean-$0$ and covariance matrix $\Sigma$. I am interested in a closed form for ...
- 193
3
votes
0
answers
75
views
Covariance of censored/clipped Gaussians
I am interested in the covariance of two clipped (or censored) Gaussian variables.
More precisely, let $g_1 \sim N(0,\sigma_1^2)$ and $g_2 \sim N(0,\sigma_2^2)$ be two (dependent) Gaussians with $\...
- 53
2
votes
1
answer
188
views
Probability distribution of vectors obtained from Gram-Schmidt process on i.i.d. Gaussian vectors
Given $N$ vectors in $K$ dimensions that are independently and identically distributed according to a Gaussian distribution with mean $0$ and standard deviation equal to an identity matrix, what is ...
- 29
2
votes
1
answer
213
views
Gaussian expectation restricted to a convex polytope
Let $X$ be a Gaussian vector in $\mathbb{R}^n$ with $\mathbb{E}[X]=0$ and $\mathbb{E}[X X^\intercal]=I_n$. Let $\mathbf{S}$ be a convex polytope in $\mathbb{R}^n$ defined as the intersection of $m$ $(...
- 21
2
votes
1
answer
872
views
Estimating the average of two gaussians' mean
Assume that $X\sim \mathcal N(\sigma_1,\mu_1)$ and $Y\sim \mathcal N(\sigma_2,\mu_2)$.
I want to estimate $\frac{\mu_1+\mu_2}{2}$ after observing $X,Y$.
In my setting, $\sigma_1,\sigma_2$ are known ...
- 618
2
votes
3
answers
166
views
On the probability of the multivariate normal with fixed pairwise correlations being coordinate-wise non-negative
This problem itself, admittedly, is not a research problem; but rather an intermediate step I've encountered in my research.
Let $(X_i:1\le i\le N)$ be a multivariate normal random vector where i) ...
- 1,096
2
votes
1
answer
759
views
History of the name "subexponential distribution" in probability
In probability theory, the term subexponential distribution has historically been used for a distribution whose CDF $F(x)$ satisfies the relation
$$
n(1-F(x)) \sim 1 - F^{*n}(x)
$$ for any $n \ge 1$ ...
- 1,124
2
votes
1
answer
281
views
Hermite polynomial and Gaussian random variable
The following formula is well known: $E[H_k(X,E[X])H_q(Y,E[Y])]=\delta_{kq}E[XY]^k$ for a joint Gaussian r.v. $(X, Y),$ $H_k$ are Hermite polynomiale.
Is there a generalization for this to a joint ...
- 573
2
votes
1
answer
1k
views
measure of a degenerate Gaussian distribution
I want to do computations with a degenerate Gaussian measure, but I do not know how to represent it
in a close form.
After starting with a Gaussian random variable and restricting it to a condition, I ...
- 23
2
votes
1
answer
64
views
Maximum Number of modes of $V=U+Z$ where $Z$ standard normal and $|U|\le a$
Let $f_V$ be a pdf of random variable $V$ where
\begin{align}
V=U+Z
\end{align}
and where $U$ and $Z$ are independent and $Z$ is Gaussian. Moreover, suppose that $|U| \le A$.
Can we find the upper ...
- 671
2
votes
1
answer
124
views
Limiting behavior of $k^{th}$ order statistics of n non-i.i.d chi square random variables
This is related to one of my previous questions here.
Let $(Z_1, Z_2, \ldots, Z_n)\sim N(0, \Omega)$, where $\Omega = (1-\mu) I_{n\times n} + \mu \boldsymbol{1}_n\boldsymbol{1}_n^\top $. Here $\...
- 399
2
votes
1
answer
119
views
Deriving the distribution of standardized variables with empirical mean and standard deviation
I'm working with a set of independent and identically distributed random variables $\{ x_i \}_{i=1}^N$, where each $x_i$ follows a Gaussian distribution $P_X(x) = \mathcal{N}(x; \mu, \sigma^2)$. This ...
- 305
2
votes
0
answers
247
views
Moments of a Normal-Wishart distribution
Do known expressions exist for the moments of a gaussian-wishart (aka normal wishart) distribution?
$$NW(\mu,K\mid\mu_0,\lambda_0, v, W) =
\frac{|\lambda_0K|^{1/2}}{(2\pi)^{d/2}}e^{-0.5([\mu - \mu_0]...
- 121
1
vote
1
answer
798
views
Which distributions of $X$ and $Y$ yield a Gaussian $Z=XY$?
Let $Z=XY$ where $X$, $Y$ are random variables with support of non-trivial measure. For what distributions of $X$ and $Y$ can $Z$ be guaranteed to be Gaussian?
- 409
1
vote
1
answer
208
views
Extreme confusion with the exact meaning of Gaussian measure with "translation-invariant" covariance
In physics literature, the covariance of a Gaussian measure $\mu$ on a function space is denoted as $C(x,y)$. Moreover, they say that if the covariance is translation-invariant, then actually $C(x,y)=\...
- 3,477
1
vote
1
answer
2k
views
Convolution of two Gaussian mixture model
Suppose I have two independent random variables $X$, $Y$, each modeled by the Gaussian mixture model (GMM). That is,
$$
f(x)=\sum _{k=1}^K \pi _k \mathcal{N}\left(x|\mu _k,\sigma _k\right)
$$
$$
g(y)=\...
- 41
1
vote
1
answer
82
views
Expectation value of multilinear forms over independent Gaussian vectors
Let $A$ be a symmetric multilinear form on $\left(\mathbb{R}^d\right)^{\otimes n}\times \left(\mathbb{R}^d\right)^{\otimes n}$ and consider the random variable:
\begin{align*}
X=A(g_1,\ldots,g_n,g_1,\...
- 111
1
vote
3
answers
293
views
Lipschitz continuous maps from $\mathbb R^n$ to $\mathbb R^n$ that preserve Gaussian measure?
The only ones I can think of are linear maps like rotations and permutations. Is there a more general characterization?
- 41
1
vote
1
answer
613
views
Integral of the product of a gaussian pdf and cdf
I am trying to solve the integral of a gaussian cumulative distribution function and a gaussian probability function. On this site I have seen solutions of similar, less general integrals (e.g. ...
- 11
1
vote
1
answer
101
views
Estimating the average of two gaussians' mean with minimal squared error
This is a follow-up to my previous question.
Assume that $X\sim \mathcal N(\mu_1,\sigma_1^2)$ and $Y\sim \mathcal N(\mu_2,\sigma_2^2)$.
I want to estimate $\frac{\mu_1+\mu_2}{2}$ after observing $X,Y$....
- 618
1
vote
1
answer
169
views
Probability involving dependent random variables constructed from i.i.d. Gaussians
This is a problem I need to address for a certain computation in my research.
Let $Y_1,\dots,Y_n$ be a sequence of i.i.d. standard normal variables; and let $I\subset[0,+\infty)$ be an interval. In my ...
- 1,096
1
vote
1
answer
417
views
Obtaining the error term of binomial distribution's entropy from the differential entropy of a Gaussian distribution
It is known that the first order error term in the Shannon entropy formula for a binomial distribution is $1/n$ (for example, see the Wikipedia page Binomial distribution), where in the limit $n \to \...
user325283
1
vote
0
answers
176
views
Gaussian order statistics
Setup. Let $\alpha\in(0,1)$ fixed; and $\tau\in[0,1]$ (think of it very close to one).
Suppose $X_1,\dots,X_n$ are i.i.d. standard normal.
Let $Y_1,\dots,Y_n$ be another sequence of standard normals ...
- 11
1
vote
0
answers
121
views
Relation satisfied by a Gaussian random variable
I want to prove the following relation for $X\sim \mathcal{N}(0,1)$, $x\in \mathbb{R}$ and $f(x)=\mathbb{E}[\max(X,x)]$:
$$f(\frac{f(x+1)+f(x-1)}{2})\leq \frac{f(f(x)-1)+f(f(x)+1)}{2}$$
It seems that ...
- 171
1
vote
1
answer
66
views
Comparing noisy truncated RV with noisy regular RV
For some reason, I'm having difficulties proving something that is intuitively simple.
Assuming I have two a random variable, $x$ and $x^{truncated}$, where $x^{truncated}$ is the truncated version of ...
- 183