All Questions
Tagged with pr.probability st.statistics
102 questions
25
votes
1
answer
4k
views
What kind of random matrices have rapidly decaying singular values?
I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the $...
- 118k
2
votes
1
answer
676
views
Distribution of ratio between complex Gaussian and Chi-square R.V.s
What would be the distribution (p.d.f.) of the following ratio?
$$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
17
votes
4
answers
2k
views
Good introduction to statistics from a algebraic point of view?
There are already lots of questions on this subject like
Is there an introduction to probability theory from a structuralist/categorical perspective?
Is there a combinatorial/topological treatment ...
- 283
6
votes
1
answer
274
views
Spectrum asymptotics for a product of $k$ random matrices?
How does the spectrum of a product of $k$ random matrices behave around 0?
In particular, I'm wondering if the CDF of squared singular values behaves as $x^{\frac{1}{k+1}}$ around 0. The result for $k=...
- 2,242
4
votes
1
answer
626
views
How to get the lower bound of the following $\tau$?
Let $A=\{a_{ij}\}_{1\le i,j\le n}$ be an $n$ by $n$ normalized Gaussian random matrix with $E[a_{ij}]=0$ and $E[a_{ij}^2]=1/n$. Ordering its eigenvalues by $\lambda_1\le \lambda_2\le \cdots \lambda_n$ ...
- 288
2
votes
2
answers
351
views
Weak convergence for discrete-time processes using characteristic functions
I am looking for a good reference about the analogues of the Bochner Theorem and the Lévy Continuity Theorem
for probability measures on $\mathbb{R}^{\mathbb{N}}$ with the product topology.
...
40
votes
5
answers
5k
views
"Entropy" proof of Brunn-Minkowski Inequality?
I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality.
The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then
$$ m(...
- 22.8k
18
votes
4
answers
4k
views
Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables
I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables.
Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
- 591
16
votes
1
answer
2k
views
Normal approximation of tail probability in binomial distribution
My problem: From the Berry--Esseen theorem I know, that $$\sup_{x\in\mathbb R}|P(B_n \le x)-\Phi(x)|=O\left(\frac 1{\sqrt n}\right),$$ where $B_n$ has the standardized binomial distribution and $\Phi$ ...
- 497
14
votes
8
answers
3k
views
Relevant mathematics to the recent coronavirus outbreak
I would like to ask about (old* and new) reliable mathematical literature relevant to various mathematical aspects of the recent coronavirus outbreak: In particular, standard statistical/mathematical ...
12
votes
1
answer
10k
views
Square root of normal distribution
Let $X$ and $Y$ be independent random variates with the same probability distribution, $P(x)$. Assuming that the product $Z=XY$ is a random variate with normal distribution, say $$f_Z(x) = \frac{1}{\...
- 377
9
votes
0
answers
2k
views
Has the Lie group preserving a probability distribution been used in Bayesian statistics?
For a (possibly signed) nondegenerate probability measure $\pi$ on $\{1,\dots,n\}$ define
$$\langle \pi \rangle := \{R \in \operatorname{STO}(n): \pi R = \pi \}.$$
Here $\operatorname{STO}(n)$ denotes ...
- 15.4k
8
votes
2
answers
2k
views
Moment matching: construction of a mixture of Gaussian distribution with lower moments identical to Gaussian
This is a question related to the statistical model behind independent component analysis (ICA).
We assume that $Z \sim N(0,1)$. Our goal is to construct a random variable $X$ that follows a ...
- 1,127
4
votes
1
answer
5k
views
Asymptotic behavior of max of chi-squared distribution
Suppose $X_{\max}$ is the maximum in a sequence $X_1,X_2,\ldots,X_n$ where each $X_i\sim\chi^2_k$ is an i.i.d. chi-squared random variable with $k$ degrees of freedom.
Since chi squared distribution ...
- 907
4
votes
1
answer
258
views
When is $\prod_{i=0}^\infty (I-x_i x_i^T)=0$ for isotropic Gaussian $x_i$?
Suppose $x_i$ is sampled IID from isotropic zero-centered Gaussian random variable in $d$ dimensions with covariance $\Sigma=c*I$. When is the following true with probability 1?
$$\prod_{i=0}^\infty (...
- 2,242
4
votes
2
answers
1k
views
Expectation of the trace of inverse of a Gaussian random matrix
Given a $N×M$ random complex gaussian matrix $X$ and $N×K$ random complex gaussian matrix $Y$ I'm interested in approximating the expectation expressed as:
\begin{align}
E[trace({(aX{X^H} + I)^{ - ...
- 377
4
votes
1
answer
485
views
Expected norms of Wishart matrices
Suppose $x_i \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu,\Sigma)$. What can we say about dependence on $b$ of Frobenius/spectral norm quantities below?
$$f(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^...
- 2,242
4
votes
1
answer
812
views
On the largest and smallest spacings for the uniform distribution
Let $Z_1,\dots,Z_n$ be iid random variables (r.v.'s) each uniformly distributed on $[0,1]$. Let $Z_{n:1}\le\cdots\le Z_{n:n}$ be the corresponding order statistics. For $i=1,\dots,n-1$, let $G_i:=Z_{n:...
- 128k
2
votes
1
answer
649
views
distribution on the inverse Wishart matrix eigenvalues summation
Let $\lambda_1>\lambda_2>....>\lambda_N$ be the ordered eigenvalues of Wishart matrix my objective is to find if it is possible the distribution of:
\begin{align}
s = \sum\limits_{i = 1}^...
- 377
1
vote
1
answer
146
views
Upper bound on difference of correlated ratios
Suppose $(x_n, y_n)$ are i.i.d. samples (that is, $x_n$ and $y_n$ are not independent, but $(x_n, y_n)$ is i.i.d. with regards to $(x_m, y_m)$ if $n\ne m$) from a joint distribution, with $0 < x_n, ...
- 15
1
vote
2
answers
234
views
Find $\inf_{P_{X_1,X_2}}P_{X_1,X_2}(\|X_1-X_2\| > 2\alpha)$ , where $\alpha > 0$ and inf is over couplings
Let $\mathcal X$ be a seperable Banach space with norm $\|\cdot\|$, and let $X_1$ and $X_2$ be random vectors on $\mathcal X$ with finite means.
Question. Given $\alpha > 0$, what is value of, ...
- 6,853
39
votes
3
answers
4k
views
Manifold of probability measures: connections between two types of metrics
The space of probability measures could be viewed as an infinite-dimensional manifold, equipped with two possible types of metrics — (1) Wasserstein and (2) Fisher-Rao. Metric (1) is connected with ...
- 1,127
24
votes
2
answers
1k
views
Drawing natural numbers without replacement.
Suppose we start with an initial probability distribution on $\mathbb{N}$ that gives positive probability to each $n$. Let's call this random variable $X_1$ so we have $P(X_1=n)=p_{1,n}>0$ for all $...
- 551
23
votes
7
answers
5k
views
What makes Gaussian distributions special?
I'm looking for as many different arguments or derivations as possible that support the informal claim that Gaussian/Normal distributions are "the most fundamental" among all distributions.
...
21
votes
3
answers
5k
views
James-Stein phenomenon: What does it mean that a James-Stein estimator beats least squares estimator?
Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing.
It is claimed that if one wants to estimate the mean $\Theta$ of
Gaussian ...
- 24.9k
18
votes
1
answer
1k
views
How fast can extreme eigenvalues of the average of random matrices converge to their expectation?
Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
- 181
17
votes
1
answer
10k
views
Conjugate prior of the Dirichlet distribution?
What is the conjugate prior distribution of the Dirichlet distribution?
Edit: Since I asked this question many years ago, I've written a Python library for working with exponential families. Maximum ...
- 598
15
votes
2
answers
5k
views
What areas of algebra could be interesting to probability theorists?
I would like to find some topic of algebra (beyond linear algebra; algebraic number theory is fine) that would be interesting both to a student that wants to specialize in probability theory and to me ...
- 16.9k
14
votes
1
answer
3k
views
How is the "conformal prediction" conformal?
The question is clarified by Prof.V.Vovk. See his answer below for discussion.
Recently, early works of Gammerman, Vanpnik and Vovk[4] are rediscovered by Wasserman et.al[1] and proposed it as a ...
- 8,071
14
votes
1
answer
1k
views
Berry Esseen type result for probability density functions
Let $X_1, X_2, \cdots$ be i.i.d. random variables with $E(X_1) = 0, E(X_1^2) = \sigma^2 >0, E(|X_1|^3) = \rho < \infty$.
Let $Y_n = \frac{1}{n} \sum_{i=1}^n X_i$ and let us note $F_n$ (resp. $\...
13
votes
1
answer
10k
views
KL divergence and mixture of Gaussians
Do we have an exact formula to compute the KL divergence between 2 mixtures of Gaussians (i.e convex combinations of a finite number of Gaussian distributions)?
If not exactly known, are there good ...
- 2,246
12
votes
4
answers
4k
views
Mixtures of Gaussian distributions dense in distributions?
It seems that a mixture of Gaussians can approach any probability distribution, as the number of mixture components approaches infinity. Is this true? And if so, is it precise and correct to say ...
- 141
11
votes
2
answers
819
views
Estimate rate of real correct/wrong from 4 answers quiz.
I recently read that one in ten students think the first man on the moon was Buzz Lightyear, a Toy story cartoon. I'm not here to discuss the data in itself, rather, this reading got me into a problem ...
- 219
11
votes
1
answer
1k
views
What are some of the surprising results of finite sample statistical estimation?
I'm trying to familiarize myself with the latest results in finite sample statistics. It seems to me that these results can be classified into two categories:
Unsurprising results confirm that the ...
- 395
11
votes
3
answers
3k
views
Distance between distributions and distance of moments
Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{$\ast$}$$ where $X$ is a random variable of ...
- 623
10
votes
2
answers
590
views
"Fractional sampling" from a probability distribution
My question concerns an operation on probability distributions which has arisen in some applied research. It is well-defined mathematically (at least in a limited context), but I don't know how to ...
- 8,491
10
votes
2
answers
2k
views
Convergence of an empirical distribution w.r.t. the Hellinger distance
Let $P$ be a probability distribution on a finite set $\mathcal{X}$ and let $X_1, X_2, \ldots, X_n$ be drawn i.i.d. according to $P$. Define the empirical distribution:
$\hat{P_n}(x) = \frac{1}{n} \...
- 165
10
votes
2
answers
847
views
Minimum separation among $m$ random points on an $n$-dimensional unit sphere
Consider $m$ points $v_1, \ldots, v_m \in R^{n}$, which are uniformly distributed on the $n$-dimensional unit sphere $S^{n-1} = \{v:\|v\|_2 = 1\}$. Let the minimum separation be
$$
\rho = \min_{i,j\in{...
- 1,127
10
votes
2
answers
1k
views
Random Voronoi Diagrams
I'm interested in what research has already been done with regards to the statistics of random voronoi diagrams. I have had a look on google scholar and results are a little inconclusive. I'm ...
- 903
9
votes
2
answers
878
views
Is there a combinatorial/topological treatment of statistical independence?
Is there any reference which studies sets of random variables as independence systems, a type of combinatorial object (see below)?
Motivation:
In particular, since independence systems are abstract ...
- 2,680
8
votes
3
answers
2k
views
randomness in nature [closed]
What is the explanation of the apparent randomness of high-level phenomena in nature?
For example the distribution of females vs. males in a population (I am referring to randomness in terms of the ...
- 307
8
votes
2
answers
1k
views
Does Multiplicative Version of Azuma's Inequality Hold?
It is known that there are multiplicative version concentration inequalities for
sums of independent random variables. For example, the following
multiplicative version Chernoff bound.
Chernoff bound:...
- 175
7
votes
1
answer
569
views
Singular Fisher information matrix and existence of unbiased estimators
I'm doing some research into the Cramer-Rao bound for time of arrival localization and have come across a rather strange result: the FIM is singular, but there exists an unbiased estimator. My ...
- 75
7
votes
2
answers
649
views
What's the standard name for sets of a given size with maximal probability (or a given probability and minimal size)?
The definition I'm going to give isn't quite the concept I really want, but it's a good approximation. I don't want to make the definition too technical and specific because if there's a standard name ...
- 5,992
6
votes
2
answers
2k
views
Is the Binomial Expectation of Convex Function Convex in p?
Suppose $X$ has a binomial distribution with success probability $p$ and $n$ trials and let $h(\cdot)$ be a positive convex real-valued function.
Is the function $g(p)=\mathbb{E}[h(X)\ |\ p]$ convex ...
- 103
6
votes
0
answers
295
views
Dimension-free sample complexity for estimating Gaussian covariance
(also asked on math.se, with no answers)
Suppose I have $m$ samples drawn from a Gaussian in $\mathbb{R}^n$, and need sample covariance $\Sigma_m$ to be $\epsilon$-close to true covariance $\Sigma$:
$$...
- 2,242
6
votes
1
answer
375
views
Deviation bound for the maximum of the norm of Wiener process
Let $W(t)$ be an $n$-dimensional Wiener process. Denote by $\chi_n^2$ a chi-squared random variable with $n$ degrees of freedom. I have recently found the following inequality given without proof:
$$
{...
- 61
6
votes
1
answer
1k
views
How to check if a symmetric random variables is the difference of two iid symmetric random variables
I have the continuous symmetric random variable $X$ in $\mathbb{R}$. If I know its distribution function $F(x)$ what are the conditions on $F(x)$ so that $X=Y_1 - Y_2$ where $Y_i$ are also iid ...
5
votes
1
answer
2k
views
Mathematics research relating to machine learning
What branch/branches of math are most relevant in enhancing machine learning (mostly in terms of practical use as opposed to theoretical/possible use)? Specifically, I want to know about math research ...
- 173
4
votes
2
answers
1k
views
expected values over binomial distributions
In some works of economics/risk analysis etc., I have seen situations where people take the expected value of a function (such as a utility function/cost function) over a binomial distribution:
$$F(n)...
- 7,320