All Questions
Tagged with pr.probability st.statistics
1,135 questions
1
vote
0
answers
129
views
Inverse Wishart
Assume that $X\sim \operatorname{IW}_p(n,\Sigma)$ has an inverse Wishart distribution, which probability density function is
$$f(X\mid n,\Sigma)=C(n,\Sigma)|X|^{-\frac{n+p+1}{2}} \exp\Big(-\frac{1}{2}...
4
votes
1
answer
206
views
Inner product of sorted Gaussian vector
Suppose $X_1,\ldots,X_n$ are i.i.d. standard normal. I'm wondering how to analyze the following quantity:
$$\left|\frac{X_{(1)}X_{(n)}+X_{(2)}X_{(n-1)}+\cdots+X_{(n)}X_{(1)}}{n}\right|$$
where $X_{(1)}...
CommunityBot
- 1
2
votes
1
answer
676
views
Distribution of ratio between complex Gaussian and Chi-square R.V.s
What would be the distribution (p.d.f.) of the following ratio?
$$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
1
vote
1
answer
186
views
Expected norm of linear maps
I want to compute the expected norm of a vector-matrix multiplication. I have a vector $x \in \mathbb{R}^n$ with norm one and a matrix $M \in \mathbb{R}^{n \times n}$, whose entries are iid taken from ...
- 128k
0
votes
0
answers
103
views
Expectation of maximal Wasserstein distance between empirical distribution and a pdf
Let $P$ be a continuous probability distribution on $R^d$, $X$ the random variable $\sim P$, and $
\hat{X}$ be n i.i.d samples drawn according to $P$. We have another variable $\mu \in S^{d-1}$.
Do ...
- 109
0
votes
0
answers
57
views
Absolute continuity of probability measures determined by dependence structure
We are on $\mathbb{R}^d$ with Borel $\sigma$-algebra. Let $\mu_1, ..., \mu_d$ be probability measures on $\mathbb{R}$ and $\Pi(\mu_1, \mu_2, ..., \mu_d)$ be the set of probability measures on $\mathbb{...
- 1,095
7
votes
1
answer
231
views
Relation between the two possible KL divergences of two distributions
Given that I know $$D\left(P\parallel Q\right)<\alpha,$$ can I say anything about $D\left(Q\parallel P\right)$ in terms of an upper bound on it?
Also, given this upper bound on $D\left(P\parallel ...
- 128k
17
votes
4
answers
2k
views
Good introduction to statistics from a algebraic point of view?
There are already lots of questions on this subject like
Is there an introduction to probability theory from a structuralist/categorical perspective?
Is there a combinatorial/topological treatment ...
- 96
0
votes
1
answer
239
views
Anti-concentration: upper bound for $P(\sup_{a \in \mathbb S_{n-1}}\sum_{i=1}^na_i^2Z_i^2 \ge \epsilon)$
Let $\mathbb S_{n-1}$ be the unit sphere in $\mathbb R^n$ and $z_1,\ldots,z_n$ be a i.i.d sample from $\mathcal N(0, 1)$.
Question
Given $\epsilon > 0$ (may be assumed to be very small), what is ...
- 6,853
0
votes
0
answers
268
views
Does the linear combination of the quantile $\alpha F^{-1}(\tau)+\beta G^{-1}(\tau)$ still a quantile
$F(x)$ and $G(y)$ are distribution functions.
Define the $\tau$th quantile for cdf $F(x)$, $G(y)$ as
$$\xi_\tau\equiv F^{-1}(\tau)=\inf\{x:F(x)\ge \tau\}$$
and
$$\eta_\tau\equiv G^{-1}(\tau)=\inf\{y:...
- 141
1
vote
1
answer
468
views
Sharp tail bounds for the maximum of an iid sample of a random variable supported on $[0, 1]$
Let $X_1,\ldots,X_n$ be an iid sample from a distribution supported on $[0, 1]$.
Question
What are some sharp concentration inequalities (i.e tail bounds) empirical statistic defined by $Z_n := \max(...
- 128k
-2
votes
1
answer
92
views
Existence or impossibility of Gaussian factory
Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.
5
votes
1
answer
2k
views
Earth mover/Wasserstein distance between a pdf and an empirical distribution
This question is inspired by this much older question:
Convergence of an empirical distribution w.r.t. the Hellinger distance
Let $P$ be a continuous probability distribution on a compact subset of $...
- 3,574
0
votes
1
answer
144
views
A problem related to the comparison of two integer-valued random variables
Consider an urn containing red, blue and green balls (the situation is the same illustrated in this post).
Let $X$ be the non-negative, integer-valued random variable defined as the number of trials (...
- 128k
5
votes
2
answers
1k
views
Practical bounds for the Wasserstein distance in 2 dimensions
Let $X_1,\dots,X_n$ be a set of independent samples of a distribution $\mu$ on the unit square, let $\hat\mu_n$ be the empirical distribution on the points $X_1,\dots,X_n$, and let $W_1(\mu,\hat\mu_n)$...
- 6,853
4
votes
3
answers
205
views
Expected distance of nearest matching pair in the game of pairs
Recently I was playing several rounds of the game of pairs with my children. I was surprised that almost every time, one matching pair was adjacent (either next to each other in a row, or vertically). ...
- 842
1
vote
1
answer
338
views
Expected values of two non-negative, integer-valued random variables related to an urn problem
Consider an urn containing $c$ distinguishable balls, $\alpha$ of which are red, $\beta$ of which are blue, and $\gamma$ of which are green, and $\alpha+\beta+\gamma=c$. We assume $\alpha,\beta,\gamma&...
- 145
6
votes
1
answer
1k
views
Why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?
I'm reading 《Algebraic geometry and statistical learning theory》.My problem is why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?...
- 128k
2
votes
2
answers
2k
views
Dependence between direction and magnitude of multivariate normal random vector
Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then
\begin{align}
f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^...
6
votes
1
answer
1k
views
Largest eigenvalues of a (random) correlation matrix?
I am recently studying on eigenvalues of a (random) correltion matrix. For a $N\times N$ correlation matrix (with a given meaning of randomness), its (1st, 2nd, etc.) eigenvalues have some ...
- 96.4k
4
votes
0
answers
241
views
What happens in the martingale CLT if I norm by the conditional variance instead?
TLDR: I'm a statistician (bear with me!) trying to use the martingale CLT but I only can estimate the conditional variance instead of the unconditional one. Can I do anything to get a CLT with norming ...
- 91
1
vote
1
answer
118
views
What is the order of the left tail of a mixture of non-central chi-square?
Let $\mu\sim N(0,1)$, $Z\sim N(\mu,1)$. Then $Z$ can be viewed as a mixture of Gaussians. It can also be viewed as a Gaussian but there is a prior for the mean.
Let $X\sim\exp(\lambda)$ where the ...
CommunityBot
- 1
4
votes
0
answers
1k
views
Concentration of sum of independent random variables
Let $X_1, ..., X_n$ be i.i.d. sub-Gaussian random variables with mean $0$ and variance $1$. That is, we have $\Pr[|X_i| > t] \leq \exp(1-t^2/K^2)$ for all $t>0$ and a parameter $K$.
Then we can ...
- 3,993
2
votes
1
answer
412
views
References for Hellinger distance/affinity involving mixture distributions
For two continuous probability distributions $F,G$ and their densities, $f,g$, the (squared) Hellinger distance/affinity is given by $d^2_H(F,G)=1-\int_{\mathbb{R}} \sqrt{fg}~dx$. Suppose that $f,g$ ...
- 128k
2
votes
0
answers
130
views
Open problems in Monte Carlo Simulation [closed]
I want to know some open problems in Monte Carlo Simulation, which is being studied or in a stalemate.
Could you please give me some advice? Thanks a alot
4
votes
2
answers
415
views
Effect of perturbing the atoms of a measure on the Wasserstein distance
Let $(X,d)$ be a metric space, $x_1,\ldots,x_N\in X$ and $x_1',\ldots,x_N'\in X$ be atoms, and $G=\sum_{i=1}^Np_i\delta_{x_i}$, $G'=\sum_{i=1}^Np_i'\delta_{x_i}$, and $G''=\sum_{i=1}^Np_i'\delta_{x_i'}...
- 576
1
vote
2
answers
462
views
lower bound the probability of at least L collisions
Lets say we get a list $M$ containing $|M|=\sqrt{L\cdot N}$ randomly and independtly drawn elements from a set of size $N$. And lets denote the $i$-th element of the list $M$ by $M[i]$.
If we now ask ...
- 128k
10
votes
2
answers
455
views
Largest deviations for uniform order statistics
Let $n >0$.
Let $X_1,\ldots,X_n$ be i.i.d. uniform random variable on $[0,1].$ Denote by $X^{(1)}\leq X^{(2)} \leq \cdots \leq X^{(n)}$ their order statistics, and write $\Delta^{(i)} = \vert X^{(...
4
votes
1
answer
275
views
A metric stronger than total variation
Let $P,Q$ be two distributions on a finite set $X$. Consider the following metric*
$$ d(P,Q) = \frac12\max_{\emptyset\neq A\subseteq X} \|P(\cdot\mid A)-Q(\cdot\mid A)\|_1. $$
Obviously, the total ...
3
votes
1
answer
171
views
Why does the assumption $|U_t| \le \frac1{p_{\min}}$ work in this paper?
I am reading a 2009 paper right now "Importance Weighted Active Learning" and on page 5, there is a theorem that uses the inequality $|U_t| \leq \frac{1}{p_{\min}}$. I am not sure how the paper found ...
0
votes
1
answer
731
views
Is the normal product distribution sub-gaussian?
Consider the normal product distribution, which is the distribution of the product of two or more independent normal variables. Particulary, focus in the case where the multiplied normal variables are ...
- 29.8k
1
vote
1
answer
514
views
Showing non-attainment of supremum
This is just an extension of my previous question Tightness of probabilty distributions
Let $\mathcal{P}(\mathbb{N})$ be the set of all PMF's on $\mathbb{N}=\{1,2,\dots \}$. Let $E$ be a convex ...
- 60.6k
3
votes
2
answers
100
views
Left tail of convex combinations of $\chi_1^2$
Suppose $a_1,...,a_n\geq0, \sum_{i=1}^na_i=1$ and $Z_1,...,Z_n$ are i.i.d. standard normal, what is a sharp upper bound of the following probability as $\delta\to0$ and what is the order?
$$\mathbb{P}(...
- 128k
2
votes
1
answer
258
views
A modest generalization of the law of large numbers
Suppose I collect $2n$ independent samples of a probability density function $f$, which are separated into pairs $\{X_i^1, X_i^2\}$ for $1\leq i\leq n$. Suppose I now consider the set of all $2^n$ ...
- 23.2k
3
votes
1
answer
473
views
Expected value of the maximum of the periodogram
Let us suppose that $X_1,\ldots,X_n$ with $n\ge1$ are iid random variables such that $\operatorname EX_1=0$ and $\operatorname E|X_1|^s<\infty$ with some $s>2$ and define the DFT of $X_1,\ldots,...
- 61.9k
4
votes
0
answers
157
views
Geometric meaning of the chi-square "measure of association"
In Statistics, there's a standard test of independence of two random variables taking values in finite sets $I,J$. It relies on the computation of $\chi$-square statistics,
$$
\chi^2:=\sum_{(i,j)\in ...
- 8,992
3
votes
1
answer
193
views
about an interesting moment generating function
Let $X_1,\ldots,X_n$ be iid Rademacher variables, i.e., $P(X_1=1)=P(X_1=-1)=1/2$. CLT says that $Y_n\equiv \sqrt{n}\bar{X}$ converges in distribution to $N(0,1)$ as $n\to\infty$. So $Y_n^2$ is ...
- 128k
13
votes
1
answer
10k
views
KL divergence and mixture of Gaussians
Do we have an exact formula to compute the KL divergence between 2 mixtures of Gaussians (i.e convex combinations of a finite number of Gaussian distributions)?
If not exactly known, are there good ...
- 188k
2
votes
2
answers
104
views
Draw samples from distribitions in the neighborhood of a fixed distribution
Disclaimer
Sorry in advance for vagueness. I'm still trying to get my ideas right on this one.
Setup
So, let $P$ be a distribution on a Euclidean space $X$ with an $\ell_p$ metric, and let $P_\...
- 1,095
20
votes
1
answer
4k
views
Using Fisher Information to bound KL divergence
Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)?
KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...
CommunityBot
- 1
3
votes
1
answer
4k
views
Expectation of random matrix inverse
Given a $K\times M$ matrix $X$, where $M\gg K$, comprising independent complex Gaussian random variables, each one with mean
$$E[X_{k,m}]=B_{k,m}$$
and variance
$$Var[X_{k,m}]=\Sigma_{k,m}$$
define ...
- 67
4
votes
0
answers
1k
views
Hoeffding's inequality for random vectors
Let $x_1, \ldots, x_n$ be $n$ i.i.d. samples of a bounded random variable $X \in [a, b]$. We know from the Hoeffding's inequality that :
$$\mathbb{P} \left( \left| \frac{1}{n} \sum_{i=1}^n x_n - \...
- 113
10
votes
2
answers
674
views
Expectation of minimum of correlated Gaussian
What is the order of the following expectation with respect to $n$?:
$$\mathbb{E}(\min_{1\leq i\leq n}|z_i|^2)$$
where
$$(z_1,...,z_n)^T\sim N(0,I+11^T), 1=(1,1,...,1)^T$$
I know that when $z_i$ are ...
- 1,495
32
votes
3
answers
12k
views
What is the Katz-Sarnak philosophy?
It has been recently mentioned by a speaker (his talk is completely not relevant to random matrix theory/RMT though) that modern statistics, especially random matrices theory, will help solving some ...
- 8,071
4
votes
1
answer
125
views
How to find the optimal convergence rate?
I have already asked that Question on Cross Validated:
Link
Suppose there is some data $X_{1},X_{2},\ldots,X_{n}$. We further suppose that there is some parameter $\theta$, for which we want to do ...
- 128k
4
votes
0
answers
139
views
Probability of having complete conversion in a box of three different object with interaction rules
Say there a 3 types of Objects A,B,C which randomly interact in pairs to form new objects following the below rules:
$$A + B = AB$$
$$B + C = BC$$
$$C + A = CA$$
$$AB + C = ABC$$
$$BC + A = ABC$$
$$CA ...
- 43.7k
6
votes
2
answers
191
views
Order statistics of bounded variable : L2 concentration?
Let n >0
Let $X_1,...,X_n$ be i.i.d. random variable with a density (say $f(x)$) on [a,b]. Denote by $X_{(1)}\leq X_{(2)} \leq \ldots \leq X_{(n)}$ their order statistics.
I'm interested in ...
- 245
2
votes
1
answer
628
views
Fastest convergence of sum of uniform independent distributions to a Gaussian
The sum of uniform i.i.d. random variables follows the Irwin-Hall distribution. Through observation it seems that the convergence is faster in comparison to the sum of uniform independent but not ...
- 128k
2
votes
1
answer
137
views
Local distribution of sample covariance matrix when the number of observations/realisations is less than the matrix dimension
Given a true covariance matrix $M$ of dimension $p \times p$, we generate $n$ gaussian random vectors $X_1,..X_n \sim N(0,M)$. We then get a sample covariance matrix $M_s$ based on these $n$ ...
- 188k
4
votes
1
answer
1k
views
Bound for a conditional expectation
Let $a_i, i=1, \ldots, n$ be real numbers. Let $\epsilon_i, \, i=1, \ldots, n$ be a random variables that take values $\pm 1$ with equal probability and $r_i, i=1, \ldots, n$ be random variables ...
- 34.7k