All Questions
83 questions
18
votes
3
answers
3k
views
Entropy and total variation distance
Let $X$, $Y$ be discrete random variables taking values within the same set of $N$ elements. Let the total variation distance $|P-Q|$ (which is half the $L_1$ distance between the distributions of $P$ ...
- 20.2k
16
votes
6
answers
3k
views
A normal distribution inequality
Let $n(x) := \frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}$, and $N(x) := \int_{-\infty}^x n(t)dt$. I have plotted the curves of the both sides of the following inequality. The graph shows that the ...
- 2,239
10
votes
3
answers
803
views
Discrete entropy of the integer part of a random variable
Let $X$ be a real valued random variable. Of course, the integer part $\lfloor X \rfloor$ of $X$ is a discrete random variable taking values in $\mathbb{Z}$. We can therefore define its discrete ...
- 2,306
8
votes
2
answers
4k
views
Lower bounds on Kullback-Leibler divergence
This was originally a question on Cross Validated.
Are there any (nontrivial) lower bounds on the Kullback-Leibler divergence $KL(f\Vert g)$ between two measures / densities?
Informally, I am ...
- 710
8
votes
2
answers
2k
views
Median and mean of the sample mean of i.i.d. log-normal
Let $y:=\frac1n\sum_{i=1}^n x_i$, where $\{x_i\}_{i=1}^n$ is a set of i.i.d. random variables, and every $x_i$ has a lognormal distribution $x_i \sim\text{Lognormal}(\mu,\sigma^2)$. Let $\text{Med}[y]$...
- 2,239
7
votes
3
answers
3k
views
expected value of squared infinity norm of vector of iid gaussians
Given a random vector
\begin{equation}
x=(x_1, \ldots, x_n)
\end{equation}
with independent and identically distributed entries $x_i \sim \mathcal{N}(0,\sigma^2)$, I would like to find a lower ...
- 237
7
votes
1
answer
556
views
A variation on the Borel–Cantelli lemma theme
Let $X,X_0,X_1,\dots$ be nonnegative independent identically distributed (i.i.d.) random variables. Let
\begin{equation*}
E:=\bigcap_{n\ge0}B_n,
\end{equation*}
where
\begin{equation*}
B_n:=\...
- 128k
7
votes
2
answers
392
views
On a von Bahr–Esseen-type inequality for pairwise independent zero-mean random variables
For $p\in(1,2)$, let $C_p$ be the smallest constant factor $C$ in the von Bahr–Esseen-type inequality
\begin{equation}\label{eq:pair}\tag{1}
E\Bigl\lvert\sum_{j=1}^n X_j\Bigr\rvert^p\le C\sum_{j=1}...
- 128k
6
votes
2
answers
2k
views
Is there a universal bound for this ratio of expectations?
Let $X$ and $Y$ be two zero-mean independent and identically distributed random variables. Is there a bound for the following ratio,
$$\frac{\mathbb{E}[|X+Y|]}{\mathbb{E}[|X|+|Y|]}=\frac{\mathbb{E}[|...
- 287
6
votes
4
answers
1k
views
Improvement of Chernoff bound in Binomial case
We know from Chernoff bound
$P\bigg(X \leq (\frac{1}{2}-\epsilon)N\bigg)\leq e^{-2\epsilon^2 N}$ where
$X$ follows Binomial($N, \frac{1}{2}$).
If I take $N=1000, \epsilon=0.01$, the upper bound is ...
- 191
6
votes
2
answers
735
views
Negative probabilities - what are two ordinary pgfs that correspond to the gf of a half-coin?
In Half of a Coin: Negative Probabilities, author considers pgf of a fair coin represented by random variable, $X = 1_H$:
$$G_X(z) = E[z^X] = \sum_{x=0,1} z^xP(X=x) = (z^0)(1/2) + (z^1)(1/2) = \frac{...
- 247
6
votes
1
answer
1k
views
Variance of the norm of a random variable under finite-moment assumptions
There is the following exercise in Vershynin's book on High-Dimensional Probability.
Exercise 3.1.6:
Let $X = (X_1, \dots, X_n) \in \mathbb{R}^n$ be a random vector with independent coordinates $X_i$ ...
- 163
5
votes
2
answers
185
views
Density near at $0$ for the integral of the positive part of the Brownian motion
This question was asked recently on MO and then deleted by the owner, user Aalon. I think the question deserves to be answered, which is what I will try to do here. Aalon was reading this paper, where ...
- 128k
5
votes
1
answer
516
views
Bounding the variance of a truncated Gaussian random variable
Suppose $X_1, X_2, X_3 \sim N(0, 1)$ are three independent standard normal random variables. I am interested in showing that:
$$\text{Var}[X_2\mid X_2 \geq X_1 - a, X_1 \leq X_3 + b] < 1,$$
where ...
- 269
5
votes
1
answer
512
views
Concentration inequality for Hilbert space valued random variables
I have read in a paper about the following result:
Let $V$ be a separable Hilbert space and $(\Omega,A_{\Omega},P)$ a probability space. Suppose that $Y_1,Y_2,...$ is a sequence of independent $V$-...
- 115
5
votes
1
answer
150
views
Kullback–Leibler chains
The following question was asked and then deleted by the post author:
Let $P$ and $Q$ be two probability distributions defined over the same space, with $KL(P \parallel Q) < \infty$. For $\epsilon ...
- 128k
5
votes
1
answer
225
views
Anti-concentration of Gaussian when conditioning on event
Let $v$ be a given vector with $\|v\|_{\Sigma^{-1}} \leq 1$, where $\Sigma$ is a positive semi-definite matrix and $\|v\|_{\Sigma^{-1}} = \sqrt{v^\top\Sigma v}$. Meanwhile, let $u$ be a random vector ...
- 1,127
5
votes
0
answers
205
views
Strange inequality relating Binomial pmf and cdf
I'm encountering a strange inequality I need to prove, relating the Binomial pmf and cdf.
Suppose we have $n$ coin flips, and fix an arbitrary $k \le n/2$ heads. Suppose further that we have some ...
- 101
4
votes
3
answers
300
views
Reconstructing probability distribution with high probability
Sample $m$ times from unknown probability distribution $p=(p_1,p_2,\cdots,p_n)$, we can construct a probability distribution $q=(q_1.q_2,\cdots,q_n)$.
How large $m$ should be to achieve that the ...
- 1,503
4
votes
3
answers
161
views
Find distribution that minimises a function of its moments
Imagine a probability density function $f(x)$, defined for positive $x$, and let's note its $n$th non-centred moment $x_{n}$. The mean $x_{1}$ is fixed (and positive).
How can I find $f(x)$ that ...
- 223
4
votes
0
answers
131
views
Log of a truncated binomial
Let $X$ follow a binomial distribution with $n$ trials and success probability $p$, and let $0\leq k\leq n$. Are there any natural approximations or bounds for the ratio $$\frac{\boldsymbol{E}\log\...
- 4,049
4
votes
0
answers
96
views
Is this conjecture about the binomial and beta distributions true?
Let $X$ follow a binomial distribution with parameters $n$ and $p$, and also fix $k$ such that $1<k<n$. Define
$$a = \mathbb{E}(X-k)^+$$
and
$$b = \mathbb{E}\log\binom{X}{(X-k)^+}$$
where the ...
4
votes
0
answers
146
views
An inequality for three iid random variables with a log-concave density
It was previously shown that
$$H\ge cG,\tag{1}$$
where $c:=1/14334$,
$$G:=E|X-Y|,\quad H:=E|X-Y|-\tfrac12\,E|X+Y-2Z|,$$
and $X,Y,Z$ are independent random variables with the same log-concave density.
...
- 128k
4
votes
0
answers
205
views
Dimension reduction for low-order moments of Rademacher-weighted sums of vectors
Let $x_1,\dots,x_n$ be vectors in a Euclidean space $H$. Let $\varepsilon_1,\dots,\varepsilon_n$ be independent Rademacher random variables (r.v.'s), so that $P(\varepsilon_i=\pm1)=1/2$ for all $i$.
...
- 128k
4
votes
0
answers
1k
views
Total variation and Hellinger distance inequality between truncated Gaussians
We know that the total variation distance, $d_{TV}(P,Q) = \frac{1}{2}\left|\left|P-Q\right|\right|_1$, between any two distributions $P$ and $Q$ is lower bounded by their squared Hellinger distance, $...
- 41
3
votes
1
answer
346
views
Simple anticoncentration bound for binomially distributed variable
The following question, which arose during my research, seems deceivingly simple to me, but I could not find any elegant and formal argument.
For a binomially distributed variable $X \sim \text{Bin} \...
- 85
3
votes
2
answers
189
views
Is the covariance of squares always bounded from below by two times the covariance?
I came across the following inequality in one of my calculations ($X,Y$ are centered random variables):
$$\operatorname{E}(X^2Y^2)-\operatorname{E}(X^2)\operatorname{E}(Y^2) \geq 2 \operatorname{E}(...
- 617
3
votes
1
answer
209
views
Log concavity of the maximum of dependent Gaussians
Let $Z_1,\dots,Z_n$ be dependent Gaussian random variables. Is it true that $X=\max\{Z_1,\dots,Z_n\}$ has a log-concave distribution function? This is true for the independent case, but is it true in ...
- 2,288
3
votes
1
answer
3k
views
Is there a tight lower bound for the expectation of the product of two positive valued random variables?
Let $X,Y$ be two (dependent) random variables with $\mathbb{P}(X\ge 0)=\mathbb{P}(Y\ge 0)=1$.
I want to find a tight lower bound of $\mathbb{E}(XY)$ when $X,Y$ are non-negative, almost surely.
...
3
votes
2
answers
1k
views
Expected value of a truncated binomial
Let $X\sim B(n,p)$ be a binomial random variable and fix $0<k<n$. Are there any well-known bounds for $\mathbb{E} (X-k)^+$, where $(X-k)^+ =\max\{0,X-k\}$? I am particularly interested in ...
- 4,049
3
votes
1
answer
142
views
How does the integral of pseudo Gaussian kernel on $(0,\infty)$ depend on its variance?
Let $a, b: \mathbb R_+ \to [0,1]$ be continuous functions. Let $k: \mathbb R_+\times\mathbb R \to [1,2]$ be $1-$Lipschitz. Set, for $0<s<t$ and $y>0$,
$$A(s,t,y):=\int_s^t\frac{k(u,y)}{1+a(u)}...
user478492
3
votes
1
answer
431
views
Inequality on the Hellinger distance between Poisson and mixture of Poisson
Let $H$ denote the Hellinger distance; i.e., for two discrete distributions $p,q$ (identified with their pmf) over $\mathbb{N}$,
$$
H(p,q)^2 = \frac{1}{2}\sum_{n=0}^\infty \left(\sqrt{p(n)}-\sqrt{q(n)}...
- 1,372
3
votes
1
answer
364
views
Can anyone give a reference to the proof of this concentration inequality?
The following concentration inequality for the supremum of a Gaussian process indexed by a separable metric space appears here: http://math.iisc.ac.in/~manju/GP/6-Concentration%20and%20comparison%...
- 123
3
votes
0
answers
169
views
Probabilistic behavior of greedy point selection in the plane
Let $\mathcal{X} = X_1,\dots,X_n$ be a collection of independent, uniform samples in the unit square. Let $\mathcal{S}=\{X_1\}$, and consider the following process: for $i=2,\dots,n$, let $x^*$ be ...
- 4,049
3
votes
0
answers
104
views
Minimizing/Maximizing the tail of the convex combinations of Chi Squared i.i.d random variables
Consider $N$ i.i.d random variables, $X_{1}, X_{2}, \ldots, X_{N}$ , that are chi-squared of degree $K \geq 2$. Also consider the following 3 vectors:
\begin{eqnarray*}
\bar{a} &=& (\frac{1}{...
- 51
2
votes
1
answer
138
views
Comparison between $\|X\|_2$ and $\|X\|_{2,1}$
For any real random variable $X$, define
$$\|X\|_{2,1}=\int_0^\infty \sqrt{\Pr(|X|>t)}dt.$$
This quantity (it is not a norm) appears in various problems, e.g. the multiplier central limit theorem (...
- 197
2
votes
1
answer
222
views
Does the inequality $\mu F(\mu)^2\geq \int_{\mu}^{b}F(x)\cdot(1-F(x))\,\mathrm dx$ hold for compactly supported continuous random variables?
This question was originally asked on the Mathematics StackExchange by User smcc
Consider a continuous random variable $V$ with cumulative distribution function $F$ and density function $f$. Suppose ...
- 1,326
2
votes
1
answer
78
views
Existence of stationary stochastic processes with very high correlation
A question was recently asked by a new user, SomeoneHAHA, and then deleted by the user, after receiving an answer. I think the question and the answer (QA) to it may be of interest to some users. ...
- 128k
2
votes
1
answer
136
views
Does higher volatility of SDE imply lower probability of staying positive?
Given two SDEs $X^1$, $X^2$ :
$$X^i_t=1+t+\int_0^t\sigma_i(s)dW_s,\quad \forall t\ge 0,$$
where $\sigma_i:\mathbb R_+\to [1/2,1]$ are non-decreasing s.t. $\sigma_1(t)\le \sigma_2(t)$ for all $t\ge 0$....
- 1,334
2
votes
1
answer
198
views
Bounds for the beta CDF
This question is closely related to a previous question that I asked here:
An inequality involving the beta distribution
Let $a,b$ be strictly positive integers, and let $F_{a,b}(x)$ denote the CDF ...
- 4,049
2
votes
1
answer
675
views
Moment generating function of random unit vector
Let $X$ be uniformly distributed on the unit sphere $S^{n-1}$. Is there any result concerning the calculation or bound (particularly lower bound) of
$$\mathbb{E}[\exp(X^Tv)]$$
for any $v$?
- 1,495
2
votes
1
answer
280
views
Complicated bound after using Stirling's approximation
I have this inequality $$\frac{1}{a}\exp\bigl\{-\frac{4}{h^2}\bigr\} \geq \frac{1}{f}$$ where $$ a \leq \Bigl(\pi^{d/2}\Gamma(\frac{1}{2}d+1)^{-1} + 1\Bigr) \left(\frac{h^{d+1}}{2} \Gamma \left(\frac{...
- 225
2
votes
2
answers
161
views
Determine the affine envelope of a random process's MGF
Suppose that a stationary random process $S(t)$ can be characterized as the figure below, which for most of the time is a straight line $S(t)=c\cdot t$, but occasionally would "stall" for a ...
- 265
2
votes
1
answer
90
views
A probability inequality: $p+(1-p)E[v|v\geq a] \geq E[v|v \geq p+(1-p)a]$
There is a random variable $v \sim F(\cdot)$ with support $[0,1]$. For a parameter $p \in (0,1)$ and $a \in (0,1)$. Define $A$ and $B$ as the following:
$$A=p+(1-p)E[v|v\geq a], B=E[v|v \geq p+(1-p)a]$...
- 121
2
votes
1
answer
599
views
Cantelli's inequality: the original source
Does anyone know where and when Cantelli's inequality was originally published? Strangely enough, I have not been able to find this information online.
- 128k
2
votes
1
answer
271
views
How to compute bounding coefficients for McDiarmid's inequality?
I am trying to understand the proof in Sec. A2 of Gretton et al.. To make the question self-contained, I summarize below the key ingredients. At the end of the post, I state my question.
Given a ...
2
votes
1
answer
105
views
Inequality for Gaussian measures
Let $\mu$ denote a centered Gaussian measure on $\mathbb{R}^k$, $K=(-\infty, a] \times \mathbb{R}^{k-1}$ ($a\ge 0$) and $L=\mathbb{R}\times C$ where $C$ is a convex set in $\mathbb{R}^{k-1}$, ...
- 197
2
votes
1
answer
248
views
Ratio of expectation involving random unit vectors
Let $u=(u_1,...,u_n), v=(v_1,...,v_n)$ be two random vectors independently and uniformly distributed on the unit sphere in $\mathbb{R}^n$. Define two other random variables $X=\sum_{i=1}^nu_i^2v_i^2$, ...
- 1,495
2
votes
0
answers
55
views
stochastic process and integral
Let $(X_n(t))_{t\in [1,+\infty], n\geqslant 1}$ be a sequence of nonnegative random variables and $(\mathcal{F}_s)$ a filtration ($\mathcal{F}_s \subset \mathcal{F}_r$ for $s\leqslant r$). We assume ...
- 29
2
votes
0
answers
69
views
A distribution $\pi \propto \exp(-f)$ satisfies log-Sobolev inequality, does $\exp(-af)$ also satisfy LSI?
Assume a distribution $\pi \propto e^{-f}$ satisfies log-Sobolev inequality (LSI)
$$\forall \rho \in P(\mathbb{R}^n), \quad KL(\rho\| \pi) \le \frac{1}{2\lambda} I(\rho \| \pi)$$
with LSI constant $\...
- 21