Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [st.statistics]

Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.

16
votes
7answers
8k views

Lower bound for sum of binomial coefficients?

Hi! I'm new here. It would be awesome if someone knows a good answer. Is there a good lower bound for the tail of sums of binomial coefficients? I'm particularly interested in the simplest case $\...
13
votes
7answers
4k views

Uniformly Sampling from Convex Polytopes

How to choose a point uniformly from a convex polytope $P \subset [0,1]^n$ defined by some inequalities, $Ax < b$? (Here $A$ is an $m \times n$ matrix, $x \in \mathbb{R}^n$, and $b \in \mathbb{R}^...
21
votes
1answer
2k views

What kind of random matrices have rapidly decaying singular values?

I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the $...
12
votes
4answers
695 views

Good introduction to statistics from a algebraic point of view?

There are already lots of questions on this subject like Is there an introduction to probability theory from a structuralist/categorical perspective? Is there a combinatorial/topological treatment ...
32
votes
4answers
3k views

“Entropy” proof of Brunn-Minkowski Inequality?

I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality. The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then $$ m(...
27
votes
0answers
9k views

Research situation in the field of Information Geometry

I am now doing an article survey on the field of information geometry started by S.Amari and Barndorff-Nielson. I want to know some research situation in this field. I have read (4) and parts of (3). ...
9
votes
0answers
1k views

Has the Lie group preserving a probability distribution been used in Bayesian statistics?

For a (possibly signed) nondegenerate probability measure $\pi$ on $\{1,\dots,n\}$ define $$\langle \pi \rangle := \{R \in \operatorname{STO}(n): \pi R = \pi \}.$$ Here $\operatorname{STO}(n)$ denotes ...
6
votes
1answer
3k views

Square root of normal distribution

Let $X$ and $Y$ be independent random variates with the same probability distribution, $P(x)$. Assuming that the product $Z=XY$ is a random variate with normal distribution, say $$f_Z(x) = \frac{1}{\...
13
votes
1answer
878 views

Normal approximation of tail probability in binomial distribution

My problem: From the Berry--Esseen theorem I know, that $$\sup_{x\in\mathbb R}|P(B_n \le x)-\Phi(x)|=O\left(\frac 1{\sqrt n}\right),$$ where $B_n$ has the standardized binomial distribution and $\Phi$ ...
3
votes
1answer
2k views

Asymptotic behavior of max of chi-squared distribution

Suppose $X_{\max}$ is the maximum in a sequence $X_1,X_2,\ldots,X_n$ where each $X_i\sim\chi^2_k$ is an i.i.d. chi-squared random variable with $k$ degrees of freedom. Since chi squared distribution ...
6
votes
2answers
571 views

Moment matching: construction of a mixture of Gaussian distribution with lower moments identical to Gaussian

This is a question related to the statistical model behind independent component analysis (ICA). We assume that $Z \sim N(0,1)$. Our goal is to construct a random variable $X$ that follows a ...
100
votes
14answers
22k views

Statistics for mathematicians

I'm looking for an overview of statistics suitable for the mathematically mature reader: someone familiar with measure theoretic probability at say Billingsley level, but almost completely ignorant of ...
21
votes
3answers
2k views

Persistent homology of Gaussian Fields in Euclidean space

If you generate points in $\mathbb R^n$ via a process that respects a Gaussian normal distribution, then compute the persistent homology / barcodes, to my eye something fairly regular seems to be ...
11
votes
1answer
1k views

What areas of algebra could be interesting to probability theorists?

I would like to find some topic of algebra (beyond linear algebra; algebraic number theory is fine) that would be interesting both to a student that wants to specialize in probability theory and to me ...
22
votes
2answers
1k views

Drawing natural numbers without replacement.

Suppose we start with an initial probability distribution on $\mathbb{N}$ that gives positive probability to each $n$. Let's call this random variable $X_1$ so we have $P(X_1=n)=p_{1,n}>0$ for all $...
14
votes
7answers
3k views

Correlation and Causation. When can we believe correlation (reasonably, at least) imply causation

We always hear, when reading on correlation, that "correlation does not imply causation." Still, I have never seen any source that tries to answer the question of when can we reasonably conclude a ...
8
votes
3answers
21k views

Distance metric between two sample distributions (histograms)

Context: I want to compare the sample probability distributions (PDFs) of two datasets (generated from a dynamical system). These datasets depend on a set of parameters, and I want a concise way to ...
14
votes
3answers
2k views

James-Stein phenomenon: What does it mean that a James-Stein estimator beats least squares estimator?

Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing. It is claimed that if one wants to estimate the mean $\Theta$ of Gaussian ...
10
votes
1answer
508 views

What are some of the surprising results of finite sample statistical estimation?

I'm trying to familiarize myself with the latest results in finite sample statistics. It seems to me that these results can be classified into two categories: Unsurprising results confirm that the ...
17
votes
1answer
666 views

How fast can extreme eigenvalues of the average of random matrices converge to their expectation?

Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
10
votes
1answer
6k views

Conjugate prior of the Dirichlet distribution?

What is the conjugate prior distribution of the Dirichlet distribution?
7
votes
3answers
2k views

randomness in nature [closed]

What is the explanation of the apparent randomness of high-level phenomena in nature? For example the distribution of females vs. males in a population (I am referring to randomness in terms of the ...
10
votes
4answers
2k views

Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables

I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables. Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
6
votes
1answer
307 views

Is there a combinatorial/topological treatment of statistical independence?

Is there any reference which studies sets of random variables as independence systems, a type of combinatorial object (see below)? Motivation: In particular, since independence systems are abstract ...
7
votes
2answers
1k views

Convergence of an empirical distribution w.r.t. the Hellinger distance

Let $P$ be a probability distribution on a finite set $\mathcal{X}$ and let $X_1, X_2, \ldots, X_n$ be drawn i.i.d. according to $P$. Define the empirical distribution: $\hat{P_n}(x) = \frac{1}{n} \...
4
votes
3answers
767 views

Incremental entropy computation

After a quick internet search I found no method for incremental entropy computation. Question 1 Let $\{x_i\}_{i=1}^n$ and $\{x_i\}_{i=1+n}^{n+m}$ be two samples and let $S_i^j:=\sum_{k=i}^j x_k$. ...
8
votes
2answers
960 views

Random Voronoi Diagrams

I'm interested in what research has already been done with regards to the statistics of random voronoi diagrams. I have had a look on google scholar and results are a little inconclusive. I'm ...
8
votes
2answers
620 views

Distance between distributions and distance of moments

Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{$\ast$}$$ where $X$ is a random variable of ...
7
votes
2answers
587 views

What's the standard name for sets of a given size with maximal probability (or a given probability and minimal size)?

The definition I'm going to give isn't quite the concept I really want, but it's a good approximation. I don't want to make the definition too technical and specific because if there's a standard name ...
6
votes
1answer
483 views

Does Multiplicative Version of Azuma's Inequality Hold?

It is known that there are multiplicative version concentration inequalities for sums of independent random variables. For example, the following multiplicative version Chernoff bound. Chernoff bound:...
3
votes
0answers
403 views

Maximization of a total variation distance subject to another total variation distance in Markov chain

Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...
1
vote
2answers
162 views

Weak convergence for discrete-time processes using characteristic functions

I am looking for a good reference about the analogues of the Bochner Theorem and the Lévy Continuity Theorem for probability measures on $\mathbb{R}^{\mathbb{N}}$ with the product topology. ...
1
vote
1answer
1k views

Unbiased estimate of the variance of an *unnormalised* weighted mean

I have a follow-up question to this one: unbiased estimate of the variance of a weighted mean Specifically, how do I generalise the result given here (and on Wikipedia) for the unbiased sample ...
11
votes
2answers
754 views

Estimate rate of real correct/wrong from 4 answers quiz.

I recently read that one in ten students think the first man on the moon was Buzz Lightyear, a Toy story cartoon. I'm not here to discuss the data in itself, rather, this reading got me into a problem ...
7
votes
4answers
2k views

averages of Euler-phi function and similar

What are the odds two numbers are relatively prime? This is known to be $\frac{6}{\pi^2}$. The proof involves calculating averages of the Euler phi function. \[ \phi(1) + \phi(2) + \dots + \phi(n) \...
6
votes
2answers
1k views

Is the Binomial Expectation of Convex Function Convex in p?

Suppose $X$ has a binomial distribution with success probability $p$ and $n$ trials and let $h(\cdot)$ be a positive convex real-valued function. Is the function $g(p)=\mathbb{E}[h(X)\ |\ p]$ convex ...
6
votes
1answer
445 views

Estimate population size based on repeated observation

I take the bus to work every day. Every bus has a serial number, but unlike in the German Tank Problem, I don't know if they are numbered uniformly $1...n$. Suppose the first $k$ buses are all ...
6
votes
1answer
386 views

On the distribution of roots modulo primes of an integral polynomial

For motivation and related questions, see below. Rough sketch of the question. View $\bigsqcup_{p \text{ prime}} (\mathbb{Z}/p\mathbb{Z})$ as a ‘subset’ of the unit circle, via $a\pmod{p} \mapsto e^{...
5
votes
1answer
142 views

Rate-Distortion theory: What is the distribution of distortion on an optimal Gaussian encoder?

If we wish to encode a gaussian source, $X\sim\mathcal{N}(0,\sigma^2)$ at rate $R$, then decode it to create an estimate $\hat{X}$, rate-distortion theory tells us that the lowest mean-squared-error ...
4
votes
0answers
164 views

Distributions over permutation groups $\mathcal{S}_n$

Partly inspired by recent developments in enumeration of pattern avoiding permutations, which is known to be connected with Brownian excursions [Hoffman&Rizzolo]. The exciting milestone is the ...
3
votes
1answer
70 views

Optimal linear measurement operator

Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...
3
votes
1answer
2k views

Order statistics of independent NOT identically distributed random variables

I want to find the p.d.f of the n-th order statistics from a set of independent, but NOT identically distributed random variables $X_1, \dots, X_n$ (the p.d.f. of the $X_i$'s is at hand)
2
votes
1answer
206 views

Distribution of the Gram matrix

Let $\mathbf{X}$ be an $m\times k$ random matrix ($m>k$) of rank $k$, having the density function $f_\mathbf{X}(X)$. What is the distribution of $\mathbf{Y}=\mathbf{XX}^T$? Basically my question is ...
2
votes
2answers
72 views

Why the autoregressive process to generate random time series?

in order to test some forecasting methods, I desire to generate random time series. I'm about to use the AR(1) model: $X_k=\alpha X_{k-1}+ \epsilon_k$ With eventually: $\alpha>1$ How can I ...
2
votes
1answer
369 views

Mean -> Frechet mean, Standard deviation ->?

Given a finite set $A$ of points of a metric space $(X, d)$, I would like to find its mean. A Frechet mean seems appropriate here: $\arg \min_{x \in X} \sum_{a \in A} d(x, a)^2$. I also would like ...
1
vote
1answer
957 views

Finding number of independent variables in “statistical” dataset

I would like to know if there are any standard techniques (that I don't know about) to solve the following problem. Suppose we have $n$ variables, $\mathbf{q} = (a_1, a_2, \ldots, a_n)$, but not all ...
1
vote
1answer
270 views

Rademacher complexity of a Lipschitz class: Are the boundedness constraints necessary?

Consider the following function class: $F={f:R^d\rightarrow [a,b], f(x)=\sigma(w^Tx)}$ where $\sigma(.)$ is Lipschitz, and $w\in R^d$ is a parameter vector. The problem I'm working on is a machine ...
1
vote
1answer
141 views

Asymptotic distribution of $\mathbb E_{\hat{P}_n}[Z]^T\operatorname{Cov}_{\hat{P}_n}[Z]^{-1}\mathbb E_{\hat{P}_n}[Z]$

Under very general conditions on the random $p$-dimensional vector $Z$, what can be said about the asymptotic distribution of the (random) scalar quantity $R_n := \mathbb E_{\hat{P}_n}[Z]^T\...
0
votes
1answer
96 views

Expected values of two non-negative, integer-valued random variables related to an urn problem

Consider an urn containing $c$ distinguishable balls, $\alpha$ of which are red, $\beta$ of which are blue, and $\gamma$ of which are green, and $\alpha+\beta+\gamma=c$. We assume $\alpha,\beta,\gamma&...
0
votes
0answers
77 views

How to calculate the exact probability $p$ at which maxima occurs in the curves in an infinite system?

I'm writing with respect to this paper: Khatun, Dutta, and Tarafdar - "Islands in Sea" and "Lakes in Mainland" phases and related transitions simulated on a square lattice Here's a link to the PDF ...