Questions tagged [st.statistics]

Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.

Filter by
Sorted by
Tagged with
20 votes
8 answers
12k views

Lower bound for sum of binomial coefficients?

Hi! I'm new here. It would be awesome if someone knows a good answer. Is there a good lower bound for the tail of sums of binomial coefficients? I'm particularly interested in the simplest case $\...
user13006's user avatar
  • 233
25 votes
7 answers
9k views

Uniformly Sampling from Convex Polytopes

How to choose a point uniformly from a convex polytope $P \subset [0,1]^n$ defined by some inequalities, $Ax < b$? (Here $A$ is an $m \times n$ matrix, $x \in \mathbb{R}^n$, and $b \in \mathbb{R}^...
john mangual's user avatar
  • 22.6k
24 votes
1 answer
3k views

What kind of random matrices have rapidly decaying singular values?

I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the $...
Qiaochu Yuan's user avatar
2 votes
1 answer
600 views

Distribution of ratio between complex Gaussian and Chi-square R.V.s

What would be the distribution (p.d.f.) of the following ratio? $$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$ where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
Felipe Augusto de Figueiredo's user avatar
48 votes
2 answers
13k views

Research situation in the field of Information Geometry

I am now doing an article survey on the field of information geometry started by S.Amari and Barndorff-Nielson. I want to know some research situation in this field. I have read (4) and parts of (3). ...
Henry.L's user avatar
  • 7,951
17 votes
4 answers
2k views

Good introduction to statistics from a algebraic point of view?

There are already lots of questions on this subject like Is there an introduction to probability theory from a structuralist/categorical perspective? Is there a combinatorial/topological treatment ...
doofin's user avatar
  • 283
4 votes
1 answer
610 views

How to get the lower bound of the following $\tau$?

Let $A=\{a_{ij}\}_{1\le i,j\le n}$ be an $n$ by $n$ normalized Gaussian random matrix with $E[a_{ij}]=0$ and $E[a_{ij}^2]=1/n$. Ordering its eigenvalues by $\lambda_1\le \lambda_2\le \cdots \lambda_n$ ...
Hermi's user avatar
  • 274
2 votes
2 answers
306 views

Weak convergence for discrete-time processes using characteristic functions

I am looking for a good reference about the analogues of the Bochner Theorem and the Lévy Continuity Theorem for probability measures on $\mathbb{R}^{\mathbb{N}}$ with the product topology. ...
Abdelmalek Abdesselam's user avatar
136 votes
15 answers
35k views

Statistics for mathematicians

I'm looking for an overview of statistics suitable for the mathematically mature reader: someone familiar with measure theoretic probability at say Billingsley level, but almost completely ignorant of ...
41 votes
5 answers
5k views

"Entropy" proof of Brunn-Minkowski Inequality?

I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality. The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then $$ m(...
john mangual's user avatar
  • 22.6k
18 votes
4 answers
3k views

Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables

I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables. Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
Adrien's user avatar
  • 591
16 votes
1 answer
2k views

Normal approximation of tail probability in binomial distribution

My problem: From the Berry--Esseen theorem I know, that $$\sup_{x\in\mathbb R}|P(B_n \le x)-\Phi(x)|=O\left(\frac 1{\sqrt n}\right),$$ where $B_n$ has the standardized binomial distribution and $\Phi$ ...
Stephan Kulla's user avatar
14 votes
8 answers
3k views

Relevant mathematics to the recent coronavirus outbreak

I would like to ask about (old* and new) reliable mathematical literature relevant to various mathematical aspects of the recent coronavirus outbreak: In particular, standard statistical/mathematical ...
12 votes
1 answer
9k views

Square root of normal distribution

Let $X$ and $Y$ be independent random variates with the same probability distribution, $P(x)$. Assuming that the product $Z=XY$ is a random variate with normal distribution, say $$f_Z(x) = \frac{1}{\...
FreeQuark's user avatar
  • 377
9 votes
0 answers
1k views

Has the Lie group preserving a probability distribution been used in Bayesian statistics?

For a (possibly signed) nondegenerate probability measure $\pi$ on $\{1,\dots,n\}$ define $$\langle \pi \rangle := \{R \in \operatorname{STO}(n): \pi R = \pi \}.$$ Here $\operatorname{STO}(n)$ denotes ...
Steve Huntsman's user avatar
8 votes
2 answers
1k views

Moment matching: construction of a mixture of Gaussian distribution with lower moments identical to Gaussian

This is a question related to the statistical model behind independent component analysis (ICA). We assume that $Z \sim N(0,1)$. Our goal is to construct a random variable $X$ that follows a ...
Minkov's user avatar
  • 1,117
8 votes
1 answer
576 views

Estimate population size based on repeated observation

I take the bus to work every day. Every bus has a serial number, but unlike in the German Tank Problem, I don't know if they are numbered uniformly $1...n$. Suppose the first $k$ buses are all ...
Grandpa's user avatar
  • 183
7 votes
1 answer
411 views

A counterexample for the Mean Ergodic Theorem in $L_\infty$

The so-called Mean Ergodic Theorem goes back to von Neumann for Hilbert spaces. Later on, versions of this result in reflexive Banach spaces have also appeared (see, e.g., the book by Krengel, Ergodic ...
Antonio J. Urena's user avatar
4 votes
1 answer
242 views

When is $\prod_{i=0}^\infty (I-x_i x_i^T)=0$ for isotropic Gaussian $x_i$?

Suppose $x_i$ is sampled IID from isotropic zero-centered Gaussian random variable in $d$ dimensions with covariance $\Sigma=c*I$. When is the following true with probability 1? $$\prod_{i=0}^\infty (...
Yaroslav Bulatov's user avatar
4 votes
1 answer
396 views

Expected norms of Wishart matrices

Suppose $x_i \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu,\Sigma)$. What can we say about dependence on $b$ of Frobenius/spectral norm quantities below? $$f(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^...
Yaroslav Bulatov's user avatar
4 votes
1 answer
687 views

On the largest and smallest spacings for the uniform distribution

Let $Z_1,\dots,Z_n$ be iid random variables (r.v.'s) each uniformly distributed on $[0,1]$. Let $Z_{n:1}\le\cdots\le Z_{n:n}$ be the corresponding order statistics. For $i=1,\dots,n-1$, let $G_i:=Z_{n:...
Iosif Pinelis's user avatar
4 votes
1 answer
4k views

Asymptotic behavior of max of chi-squared distribution

Suppose $X_{\max}$ is the maximum in a sequence $X_1,X_2,\ldots,X_n$ where each $X_i\sim\chi^2_k$ is an i.i.d. chi-squared random variable with $k$ degrees of freedom. Since chi squared distribution ...
Bullmoose's user avatar
  • 897
3 votes
2 answers
933 views

Expectation of the trace of inverse of a Gaussian random matrix

Given a $N×M$ random complex gaussian matrix $X$ and $N×K$ random complex gaussian matrix $Y$ I'm interested in approximating the expectation expressed as: \begin{align} E[trace({(aX{X^H} + I)^{ - ...
hichem hb's user avatar
  • 367
2 votes
2 answers
143 views

Expectation of $\left| \frac{\textbf{x}^{H} \textbf{y} }{\| \textbf{x} \|^2} \right|^2$, where $\textbf{x}$ and $\textbf{y}$ are complex Gaussians?

Given that following two random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{x}^{2}\textbf{I}_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{y}^{2}\textbf{I}_{M})$ ...
Felipe Augusto de Figueiredo's user avatar
2 votes
1 answer
567 views

distribution on the inverse Wishart matrix eigenvalues summation

Let $\lambda_1>\lambda_2>....>\lambda_N$ be the ordered eigenvalues of Wishart matrix my objective is to find if it is possible the distribution of: \begin{align} s = \sum\limits_{i = 1}^...
hichem hb's user avatar
  • 367
1 vote
1 answer
352 views

How fast does this Gaussian random walk move away from the origin?

Suppose $z_i$ are IID zero-centered $d$-dimensional Gaussian random variables with unit-trace covariance $\Sigma$ and $g(z_i)$ is the sum of its components. Consider the following random walk: $$x_s=\...
Yaroslav Bulatov's user avatar
1 vote
2 answers
71 views

Expectation of $\left| \frac{(\textbf{x}+\textbf{y})^{H} \textbf{x} }{\| \textbf{x} + \textbf{y} \|^2} \right|^2$, with complex Gaussians?

Given that following two random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{x}^{2}\textbf{I}_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{y}^{2}\textbf{I}_{M})$ ...
Felipe Augusto de Figueiredo's user avatar
1 vote
2 answers
232 views

Find $\inf_{P_{X_1,X_2}}P_{X_1,X_2}(\|X_1-X_2\| > 2\alpha)$ , where $\alpha > 0$ and inf is over couplings

Let $\mathcal X$ be a seperable Banach space with norm $\|\cdot\|$, and let $X_1$ and $X_2$ be random vectors on $\mathcal X$ with finite means. Question. Given $\alpha > 0$, what is value of, ...
dohmatob's user avatar
  • 6,716
1 vote
1 answer
131 views

Upper bound on difference of correlated ratios

Suppose $(x_n, y_n)$ are i.i.d. samples (that is, $x_n$ and $y_n$ are not independent, but $(x_n, y_n)$ is i.i.d. with regards to $(x_m, y_m)$ if $n\ne m$) from a joint distribution, with $0 < x_n, ...
ArBo's user avatar
  • 15
56 votes
4 answers
13k views

Group theory in machine learning

I'm a Machine Learning researcher who would like to research applications of group theory in ML. There is a term "Partially Observed Groups" in machine learning theory which has been ...
drosophyllum's user avatar
38 votes
3 answers
4k views

Manifold of probability measures: connections between two types of metrics

The space of probability measures could be viewed as an infinite-dimensional manifold, equipped with two possible types of metrics — (1) Wasserstein and (2) Fisher-Rao. Metric (1) is connected with ...
Minkov's user avatar
  • 1,117
24 votes
2 answers
1k views

Drawing natural numbers without replacement.

Suppose we start with an initial probability distribution on $\mathbb{N}$ that gives positive probability to each $n$. Let's call this random variable $X_1$ so we have $P(X_1=n)=p_{1,n}>0$ for all $...
HMPanzo's user avatar
  • 551
22 votes
7 answers
5k views

What makes Gaussian distributions special?

I'm looking for as many different arguments or derivations as possible that support the informal claim that Gaussian/Normal distributions are "the most fundamental" among all distributions. ...
21 votes
3 answers
5k views

James-Stein phenomenon: What does it mean that a James-Stein estimator beats least squares estimator?

Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing. It is claimed that if one wants to estimate the mean $\Theta$ of Gaussian ...
Alexander Chervov's user avatar
18 votes
1 answer
1k views

How fast can extreme eigenvalues of the average of random matrices converge to their expectation?

Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
sbahmani's user avatar
  • 181
17 votes
1 answer
10k views

Conjugate prior of the Dirichlet distribution?

What is the conjugate prior distribution of the Dirichlet distribution? Edit: Since I asked this question many years ago, I've written a Python library for working with exponential families. Maximum ...
Neil's user avatar
  • 598
16 votes
7 answers
3k views

Correlation and Causation. When can we believe correlation (reasonably, at least) imply causation

We always hear, when reading on correlation, that "correlation does not imply causation." Still, I have never seen any source that tries to answer the question of when can we reasonably conclude a ...
Herb's user avatar
  • 351
15 votes
2 answers
5k views

What areas of algebra could be interesting to probability theorists?

I would like to find some topic of algebra (beyond linear algebra; algebraic number theory is fine) that would be interesting both to a student that wants to specialize in probability theory and to me ...
Mikhail Bondarko's user avatar
14 votes
1 answer
3k views

How is the "conformal prediction" conformal?

The question is clarified by Prof.V.Vovk. See his answer below for discussion. Recently, early works of Gammerman, Vanpnik and Vovk[4] are rediscovered by Wasserman et.al[1] and proposed it as a ...
Henry.L's user avatar
  • 7,951
12 votes
1 answer
9k views

KL divergence and mixture of Gaussians

Do we have an exact formula to compute the KL divergence between 2 mixtures of Gaussians (i.e convex combinations of a finite number of Gaussian distributions)? If not exactly known, are there good ...
gradstudent's user avatar
  • 2,136
11 votes
2 answers
810 views

Estimate rate of real correct/wrong from 4 answers quiz.

I recently read that one in ten students think the first man on the moon was Buzz Lightyear, a Toy story cartoon. I'm not here to discuss the data in itself, rather, this reading got me into a problem ...
Stefano Borini's user avatar
11 votes
1 answer
1k views

What are some of the surprising results of finite sample statistical estimation?

I'm trying to familiarize myself with the latest results in finite sample statistics. It seems to me that these results can be classified into two categories: Unsurprising results confirm that the ...
Mike Izbicki's user avatar
11 votes
4 answers
4k views

Mixtures of Gaussian distributions dense in distributions?

It seems that a mixture of Gaussians can approach any probability distribution, as the number of mixture components approaches infinity. Is this true? And if so, is it precise and correct to say ...
Azure's user avatar
  • 131
11 votes
3 answers
25k views

Distance metric between two sample distributions (histograms)

Context: I want to compare the sample probability distributions (PDFs) of two datasets (generated from a dynamical system). These datasets depend on a set of parameters, and I want a concise way to ...
Marc Kjerland's user avatar
10 votes
2 answers
803 views

Minimum separation among $m$ random points on an $n$-dimensional unit sphere

Consider $m$ points $v_1, \ldots, v_m \in R^{n}$, which are uniformly distributed on the $n$-dimensional unit sphere $S^{n-1} = \{v:\|v\|_2 = 1\}$. Let the minimum separation be $$ \rho = \min_{i,j\in{...
Minkov's user avatar
  • 1,117
10 votes
3 answers
2k views

Distance between distributions and distance of moments

Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{$\ast$}$$ where $X$ is a random variable of ...
Julian's user avatar
  • 613
10 votes
2 answers
2k views

Convergence of an empirical distribution w.r.t. the Hellinger distance

Let $P$ be a probability distribution on a finite set $\mathcal{X}$ and let $X_1, X_2, \ldots, X_n$ be drawn i.i.d. according to $P$. Define the empirical distribution: $\hat{P_n}(x) = \frac{1}{n} \...
Anand Sarwate's user avatar
10 votes
2 answers
1k views

Random Voronoi Diagrams

I'm interested in what research has already been done with regards to the statistics of random voronoi diagrams. I have had a look on google scholar and results are a little inconclusive. I'm ...
Pavan Sangha's user avatar
9 votes
4 answers
4k views

Averages of Euler-phi function and similar

What are the odds two numbers are relatively prime? This is known to be $\frac{6}{\pi^2}$. The proof involves calculating averages of the Euler phi function. \[ \phi(1) + \phi(2) + \dotsb + \phi(n) \...
john mangual's user avatar
  • 22.6k
9 votes
2 answers
850 views

Is there a combinatorial/topological treatment of statistical independence?

Is there any reference which studies sets of random variables as independence systems, a type of combinatorial object (see below)? Motivation: In particular, since independence systems are abstract ...
Chill2Macht's user avatar
  • 2,622