Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
149
questions
20
votes
8
answers
12k
views
Lower bound for sum of binomial coefficients?
Hi! I'm new here. It would be awesome if someone knows a good answer.
Is there a good lower bound for the tail of sums of binomial coefficients? I'm particularly interested in the simplest case $\...
25
votes
7
answers
9k
views
Uniformly Sampling from Convex Polytopes
How to choose a point uniformly from a convex polytope $P \subset [0,1]^n$ defined by some inequalities, $Ax < b$? (Here $A$ is an $m \times n$ matrix, $x \in \mathbb{R}^n$, and $b \in \mathbb{R}^...
24
votes
1
answer
3k
views
What kind of random matrices have rapidly decaying singular values?
I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the $...
2
votes
1
answer
600
views
Distribution of ratio between complex Gaussian and Chi-square R.V.s
What would be the distribution (p.d.f.) of the following ratio?
$$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
48
votes
2
answers
13k
views
Research situation in the field of Information Geometry
I am now doing an article survey on the field of information geometry started by S.Amari and Barndorff-Nielson. I want to know some research situation in this field.
I have read (4) and parts of (3). ...
17
votes
4
answers
2k
views
Good introduction to statistics from a algebraic point of view?
There are already lots of questions on this subject like
Is there an introduction to probability theory from a structuralist/categorical perspective?
Is there a combinatorial/topological treatment ...
4
votes
1
answer
610
views
How to get the lower bound of the following $\tau$?
Let $A=\{a_{ij}\}_{1\le i,j\le n}$ be an $n$ by $n$ normalized Gaussian random matrix with $E[a_{ij}]=0$ and $E[a_{ij}^2]=1/n$. Ordering its eigenvalues by $\lambda_1\le \lambda_2\le \cdots \lambda_n$ ...
2
votes
2
answers
306
views
Weak convergence for discrete-time processes using characteristic functions
I am looking for a good reference about the analogues of the Bochner Theorem and the Lévy Continuity Theorem
for probability measures on $\mathbb{R}^{\mathbb{N}}$ with the product topology.
...
136
votes
15
answers
35k
views
Statistics for mathematicians
I'm looking for an overview of statistics suitable for the mathematically mature reader: someone familiar with measure theoretic probability at say Billingsley level, but almost completely ignorant of ...
41
votes
5
answers
5k
views
"Entropy" proof of Brunn-Minkowski Inequality?
I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality.
The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then
$$ m(...
18
votes
4
answers
3k
views
Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables
I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables.
Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
16
votes
1
answer
2k
views
Normal approximation of tail probability in binomial distribution
My problem: From the Berry--Esseen theorem I know, that $$\sup_{x\in\mathbb R}|P(B_n \le x)-\Phi(x)|=O\left(\frac 1{\sqrt n}\right),$$ where $B_n$ has the standardized binomial distribution and $\Phi$ ...
14
votes
8
answers
3k
views
Relevant mathematics to the recent coronavirus outbreak
I would like to ask about (old* and new) reliable mathematical literature relevant to various mathematical aspects of the recent coronavirus outbreak: In particular, standard statistical/mathematical ...
12
votes
1
answer
9k
views
Square root of normal distribution
Let $X$ and $Y$ be independent random variates with the same probability distribution, $P(x)$. Assuming that the product $Z=XY$ is a random variate with normal distribution, say $$f_Z(x) = \frac{1}{\...
9
votes
0
answers
1k
views
Has the Lie group preserving a probability distribution been used in Bayesian statistics?
For a (possibly signed) nondegenerate probability measure $\pi$ on $\{1,\dots,n\}$ define
$$\langle \pi \rangle := \{R \in \operatorname{STO}(n): \pi R = \pi \}.$$
Here $\operatorname{STO}(n)$ denotes ...
8
votes
2
answers
1k
views
Moment matching: construction of a mixture of Gaussian distribution with lower moments identical to Gaussian
This is a question related to the statistical model behind independent component analysis (ICA).
We assume that $Z \sim N(0,1)$. Our goal is to construct a random variable $X$ that follows a ...
8
votes
1
answer
576
views
Estimate population size based on repeated observation
I take the bus to work every day. Every bus has a serial number, but unlike in the German Tank Problem, I don't know if they are numbered uniformly $1...n$.
Suppose the first $k$ buses are all ...
7
votes
1
answer
411
views
A counterexample for the Mean Ergodic Theorem in $L_\infty$
The so-called Mean Ergodic Theorem goes back to von Neumann for Hilbert spaces. Later on, versions of this result in reflexive Banach spaces have also appeared (see, e.g., the book by Krengel, Ergodic ...
4
votes
1
answer
242
views
When is $\prod_{i=0}^\infty (I-x_i x_i^T)=0$ for isotropic Gaussian $x_i$?
Suppose $x_i$ is sampled IID from isotropic zero-centered Gaussian random variable in $d$ dimensions with covariance $\Sigma=c*I$. When is the following true with probability 1?
$$\prod_{i=0}^\infty (...
4
votes
1
answer
396
views
Expected norms of Wishart matrices
Suppose $x_i \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu,\Sigma)$. What can we say about dependence on $b$ of Frobenius/spectral norm quantities below?
$$f(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^...
4
votes
1
answer
687
views
On the largest and smallest spacings for the uniform distribution
Let $Z_1,\dots,Z_n$ be iid random variables (r.v.'s) each uniformly distributed on $[0,1]$. Let $Z_{n:1}\le\cdots\le Z_{n:n}$ be the corresponding order statistics. For $i=1,\dots,n-1$, let $G_i:=Z_{n:...
4
votes
1
answer
4k
views
Asymptotic behavior of max of chi-squared distribution
Suppose $X_{\max}$ is the maximum in a sequence $X_1,X_2,\ldots,X_n$ where each $X_i\sim\chi^2_k$ is an i.i.d. chi-squared random variable with $k$ degrees of freedom.
Since chi squared distribution ...
3
votes
2
answers
933
views
Expectation of the trace of inverse of a Gaussian random matrix
Given a $N×M$ random complex gaussian matrix $X$ and $N×K$ random complex gaussian matrix $Y$ I'm interested in approximating the expectation expressed as:
\begin{align}
E[trace({(aX{X^H} + I)^{ - ...
2
votes
2
answers
143
views
Expectation of $\left| \frac{\textbf{x}^{H} \textbf{y} }{\| \textbf{x} \|^2} \right|^2$, where $\textbf{x}$ and $\textbf{y}$ are complex Gaussians?
Given that following two random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{x}^{2}\textbf{I}_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{y}^{2}\textbf{I}_{M})$ ...
2
votes
1
answer
567
views
distribution on the inverse Wishart matrix eigenvalues summation
Let $\lambda_1>\lambda_2>....>\lambda_N$ be the ordered eigenvalues of Wishart matrix my objective is to find if it is possible the distribution of:
\begin{align}
s = \sum\limits_{i = 1}^...
1
vote
1
answer
352
views
How fast does this Gaussian random walk move away from the origin?
Suppose $z_i$ are IID zero-centered $d$-dimensional Gaussian random variables with unit-trace covariance $\Sigma$ and $g(z_i)$ is the sum of its components.
Consider the following random walk:
$$x_s=\...
1
vote
2
answers
71
views
Expectation of $\left| \frac{(\textbf{x}+\textbf{y})^{H} \textbf{x} }{\| \textbf{x} + \textbf{y} \|^2} \right|^2$, with complex Gaussians?
Given that following two random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{x}^{2}\textbf{I}_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{y}^{2}\textbf{I}_{M})$ ...
1
vote
2
answers
232
views
Find $\inf_{P_{X_1,X_2}}P_{X_1,X_2}(\|X_1-X_2\| > 2\alpha)$ , where $\alpha > 0$ and inf is over couplings
Let $\mathcal X$ be a seperable Banach space with norm $\|\cdot\|$, and let $X_1$ and $X_2$ be random vectors on $\mathcal X$ with finite means.
Question. Given $\alpha > 0$, what is value of, ...
1
vote
1
answer
131
views
Upper bound on difference of correlated ratios
Suppose $(x_n, y_n)$ are i.i.d. samples (that is, $x_n$ and $y_n$ are not independent, but $(x_n, y_n)$ is i.i.d. with regards to $(x_m, y_m)$ if $n\ne m$) from a joint distribution, with $0 < x_n, ...
56
votes
4
answers
13k
views
Group theory in machine learning
I'm a Machine Learning researcher who would like to research applications of group theory in ML.
There is a term "Partially Observed Groups" in machine learning theory which has been ...
38
votes
3
answers
4k
views
Manifold of probability measures: connections between two types of metrics
The space of probability measures could be viewed as an infinite-dimensional manifold, equipped with two possible types of metrics — (1) Wasserstein and (2) Fisher-Rao. Metric (1) is connected with ...
24
votes
2
answers
1k
views
Drawing natural numbers without replacement.
Suppose we start with an initial probability distribution on $\mathbb{N}$ that gives positive probability to each $n$. Let's call this random variable $X_1$ so we have $P(X_1=n)=p_{1,n}>0$ for all $...
22
votes
7
answers
5k
views
What makes Gaussian distributions special?
I'm looking for as many different arguments or derivations as possible that support the informal claim that Gaussian/Normal distributions are "the most fundamental" among all distributions.
...
21
votes
3
answers
5k
views
James-Stein phenomenon: What does it mean that a James-Stein estimator beats least squares estimator?
Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing.
It is claimed that if one wants to estimate the mean $\Theta$ of
Gaussian ...
18
votes
1
answer
1k
views
How fast can extreme eigenvalues of the average of random matrices converge to their expectation?
Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
17
votes
1
answer
10k
views
Conjugate prior of the Dirichlet distribution?
What is the conjugate prior distribution of the Dirichlet distribution?
Edit: Since I asked this question many years ago, I've written a Python library for working with exponential families. Maximum ...
16
votes
7
answers
3k
views
Correlation and Causation. When can we believe correlation (reasonably, at least) imply causation
We always hear, when reading on correlation, that "correlation does not imply causation."
Still, I have never seen any source that tries to answer the question of when can we reasonably conclude a ...
15
votes
2
answers
5k
views
What areas of algebra could be interesting to probability theorists?
I would like to find some topic of algebra (beyond linear algebra; algebraic number theory is fine) that would be interesting both to a student that wants to specialize in probability theory and to me ...
14
votes
1
answer
3k
views
How is the "conformal prediction" conformal?
The question is clarified by Prof.V.Vovk. See his answer below for discussion.
Recently, early works of Gammerman, Vanpnik and Vovk[4] are rediscovered by Wasserman et.al[1] and proposed it as a ...
12
votes
1
answer
9k
views
KL divergence and mixture of Gaussians
Do we have an exact formula to compute the KL divergence between 2 mixtures of Gaussians (i.e convex combinations of a finite number of Gaussian distributions)?
If not exactly known, are there good ...
11
votes
2
answers
810
views
Estimate rate of real correct/wrong from 4 answers quiz.
I recently read that one in ten students think the first man on the moon was Buzz Lightyear, a Toy story cartoon. I'm not here to discuss the data in itself, rather, this reading got me into a problem ...
11
votes
1
answer
1k
views
What are some of the surprising results of finite sample statistical estimation?
I'm trying to familiarize myself with the latest results in finite sample statistics. It seems to me that these results can be classified into two categories:
Unsurprising results confirm that the ...
11
votes
4
answers
4k
views
Mixtures of Gaussian distributions dense in distributions?
It seems that a mixture of Gaussians can approach any probability distribution, as the number of mixture components approaches infinity. Is this true? And if so, is it precise and correct to say ...
11
votes
3
answers
25k
views
Distance metric between two sample distributions (histograms)
Context: I want to compare the sample probability distributions (PDFs) of two datasets (generated from a dynamical system). These datasets depend on a set of parameters, and I want a concise way to ...
10
votes
2
answers
803
views
Minimum separation among $m$ random points on an $n$-dimensional unit sphere
Consider $m$ points $v_1, \ldots, v_m \in R^{n}$, which are uniformly distributed on the $n$-dimensional unit sphere $S^{n-1} = \{v:\|v\|_2 = 1\}$. Let the minimum separation be
$$
\rho = \min_{i,j\in{...
10
votes
3
answers
2k
views
Distance between distributions and distance of moments
Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{$\ast$}$$ where $X$ is a random variable of ...
10
votes
2
answers
2k
views
Convergence of an empirical distribution w.r.t. the Hellinger distance
Let $P$ be a probability distribution on a finite set $\mathcal{X}$ and let $X_1, X_2, \ldots, X_n$ be drawn i.i.d. according to $P$. Define the empirical distribution:
$\hat{P_n}(x) = \frac{1}{n} \...
10
votes
2
answers
1k
views
Random Voronoi Diagrams
I'm interested in what research has already been done with regards to the statistics of random voronoi diagrams. I have had a look on google scholar and results are a little inconclusive. I'm ...
9
votes
4
answers
4k
views
Averages of Euler-phi function and similar
What are the odds two numbers are relatively prime? This is known to be $\frac{6}{\pi^2}$. The proof involves calculating averages of the Euler phi function.
\[ \phi(1) + \phi(2) + \dotsb + \phi(n) \...
9
votes
2
answers
850
views
Is there a combinatorial/topological treatment of statistical independence?
Is there any reference which studies sets of random variables as independence systems, a type of combinatorial object (see below)?
Motivation:
In particular, since independence systems are abstract ...