Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,851
questions
136
votes
15
answers
35k
views
Statistics for mathematicians
I'm looking for an overview of statistics suitable for the mathematically mature reader: someone familiar with measure theoretic probability at say Billingsley level, but almost completely ignorant of ...
91
votes
8
answers
16k
views
Is there a natural random process that is rigorously known to produce Zipf's law?
Zipf's law is the empirical observation that in many real-life populations of $n$ objects, the $k^\text{th}$ largest object has size proportional to $1/k$, at least for $k$ significantly smaller than $...
- 109k
81
votes
6
answers
11k
views
Is data science mathematically interesting?
I have seen a plethora of job advertisements in the last few years on mathjobs.org for academic positions in data science. Now I understand why economic pressures would cause this to happen, but from ...
56
votes
4
answers
13k
views
Group theory in machine learning
I'm a Machine Learning researcher who would like to research applications of group theory in ML.
There is a term "Partially Observed Groups" in machine learning theory which has been ...
- 577
48
votes
2
answers
13k
views
Research situation in the field of Information Geometry
I am now doing an article survey on the field of information geometry started by S.Amari and Barndorff-Nielson. I want to know some research situation in this field.
I have read (4) and parts of (3). ...
- 7,961
46
votes
13
answers
23k
views
Why is it so cool to square numbers (in terms of finding the standard deviation)?
When we want to find the standard deviation of $\{1,2,2,3,5\}$ we do
$$\sigma = \sqrt{ {1 \over 5-1} \left( (1-2.6)^2 + (2-2.6)^2 + (2-2.6)^2 + (3-2.6)^2 + (5 - 2.6)^2 \right) } \approx 1.52$$.
Why ...
- 643
44
votes
4
answers
14k
views
Applications of algebraic geometry to machine learning
I am interested in applications of algebraic geometry to machine learning. I have found some papers and books, mainly by Bernd Sturmfels on algebraic statistics and machine learning. However, all this ...
- 443
41
votes
5
answers
5k
views
"Entropy" proof of Brunn-Minkowski Inequality?
I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality.
The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then
$$ m(...
- 22.6k
38
votes
3
answers
4k
views
Manifold of probability measures: connections between two types of metrics
The space of probability measures could be viewed as an infinite-dimensional manifold, equipped with two possible types of metrics — (1) Wasserstein and (2) Fisher-Rao. Metric (1) is connected with ...
- 1,117
37
votes
5
answers
7k
views
Inference using Topological Data Analysis: Is it worth it for a regular statistician to learn TDA?
After having read Gunnar Carlsson's Topology and Data I feel enthusiastic to use some topological data analysis (TDA) methods in my current research, mostly in social sciences. We often handle huge ...
- 729
36
votes
3
answers
3k
views
On Mathematical Analysis of MathSciNet & MathOverflow
This question has two original motivations: mathematical and social.
The mathematical motivation is mainly based on what I have seen about Zipf's law here and there. The Zipf's law simply states ...
32
votes
4
answers
7k
views
Bayesian statistics for pure mathematicians
Could someone please recommend reading on Bayesian statistics presented from a pure mathematical point of view? That is, works that start assuming a good knowledge of measure theoretic probability. ...
31
votes
3
answers
12k
views
What is the Katz-Sarnak philosophy?
It has been recently mentioned by a speaker (his talk is completely not relevant to random matrix theory/RMT though) that modern statistics, especially random matrices theory, will help solving some ...
- 7,961
31
votes
5
answers
2k
views
You pass X people and Y people pass you: how relatively fast are you?
This question occurs to me every time I go jogging. I suspect every runner probabilist in the world must have thought of it (though I'm no probabilist), but I could not specifically find it online. I ...
- 2,721
27
votes
3
answers
2k
views
Why do statistical randomness tests seem so ad hoc?
Wikipedia describes Kendall and Smith's 1938 statistical randomness tests like this:
The frequency test, was very basic: checking to make sure that there were roughly the same number of 0s, 1s, ...
- 595
25
votes
7
answers
9k
views
Uniformly Sampling from Convex Polytopes
How to choose a point uniformly from a convex polytope $P \subset [0,1]^n$ defined by some inequalities, $Ax < b$? (Here $A$ is an $m \times n$ matrix, $x \in \mathbb{R}^n$, and $b \in \mathbb{R}^...
- 22.6k
25
votes
3
answers
10k
views
L1 distance between gaussian measures
L1 distance between gaussian measures: Definition
Let $P_1$ and $P_0$ be two gaussian measures on $\mathbb{R}^p$ with respective "mean,Variance" $m_1,C_1$ and $m_0,C_0$ (I assume matrices have full ...
- 823
24
votes
1
answer
3k
views
What kind of random matrices have rapidly decaying singular values?
I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the $...
- 115k
24
votes
2
answers
1k
views
Drawing natural numbers without replacement.
Suppose we start with an initial probability distribution on $\mathbb{N}$ that gives positive probability to each $n$. Let's call this random variable $X_1$ so we have $P(X_1=n)=p_{1,n}>0$ for all $...
- 551
22
votes
7
answers
5k
views
What makes Gaussian distributions special?
I'm looking for as many different arguments or derivations as possible that support the informal claim that Gaussian/Normal distributions are "the most fundamental" among all distributions.
...
22
votes
3
answers
6k
views
What is quantum Brownian motion?
It seems that the current state of quantum Brownian motion is ill-defined. The best survey I can find is this one by László Erdös, but the closest the quantum Brownian motion comes to appearing is in ...
- 8,392
22
votes
0
answers
1k
views
Random Distance Matrices
My question is motivated by the following recent paper:
http://arxiv.org/abs/1110.6333
Assume you have a metric space $(X,d)$ equipped with a Borel probability measure $\mu$. We can further assume ...
- 3,616
21
votes
5
answers
3k
views
Easier reference for material like Diaconis's "Group representations in probability and statistics"
I'm teaching a class on the representation theory of finite groups at the advanced undergrad level. One of the things I'd like to talk about, or possibly have a student do any independent project on ...
- 44k
21
votes
3
answers
5k
views
James-Stein phenomenon: What does it mean that a James-Stein estimator beats least squares estimator?
Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing.
It is claimed that if one wants to estimate the mean $\Theta$ of
Gaussian ...
21
votes
1
answer
3k
views
Intuitive Proof of Cramer's Decomposition Theorem
Cramer's decomposition theorem states that if $X$ and $Y$ are independent real random variables and $X+Y$ has normal distribution, then both $X$ and $Y$ are normally distributed. I've seen a few ...
- 4,902
20
votes
8
answers
13k
views
Lower bound for sum of binomial coefficients?
Hi! I'm new here. It would be awesome if someone knows a good answer.
Is there a good lower bound for the tail of sums of binomial coefficients? I'm particularly interested in the simplest case $\...
- 233
20
votes
1
answer
4k
views
Using Fisher Information to bound KL divergence
Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)?
KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...
- 2,380
19
votes
4
answers
3k
views
Are gaussians with different moments far in total variation distance?
If two Gaussians disagree on one moment, it seems like this should imply that they have a large variation distance--equivalently, if two Gaussians are close in variation distance it's hard for their ...
- 777
19
votes
0
answers
3k
views
What does a product of many Gaussian matrices converge to?
Let $A$ be a product of $n$ $d\times d$ matrices with IID standard Gaussian entries and consider the value of $g(x)=x f(x)$ where $f(x)$ is the density of squared singular values of $A/\|A\|$.
Is ...
- 2,380
18
votes
3
answers
966
views
Deceptively simple inequality involving expectations of products of functions of just one variable
For a proof to go through in a paper I am writing, I need to prove the following deceptively simple inequality:
$$(*)\qquad E(X^a) E(X^{a+1}\log X) > E(X^{a+1})E(X^a\log X) $$
where $X>e$ has ...
- 353
18
votes
9
answers
24k
views
Why isn't likelihood a probability density function?
I've been trying to get my head around why a likelihood isn't a probability density function. My understanding says that for an event $X$ and a model parameter $m$:
$P(X\mid m)$ is a probability ...
- 283
18
votes
4
answers
3k
views
Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables
I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables.
Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
- 591
18
votes
2
answers
4k
views
When is the function of a median closer to the median of the function than the mean of the function is to the function of the mean?
Background
notation: RV= random variable, $\mu=$ mean $m=$ median
Jensen's Inequality considers the relationship between the mean of a function of an RV and the function of the mean of an RV.
If $f(...
- 225
18
votes
1
answer
1k
views
Applications of the Giry monad in probability and statistics
In another thread, I asked about the $M$ endofunctor on the category $\operatorname{Meas}$ of measurable spaces, which sends a space $X$ to its space of measures $M(X)$.
Will Sawin described the ...
18
votes
1
answer
1k
views
How fast can extreme eigenvalues of the average of random matrices converge to their expectation?
Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
- 181
18
votes
1
answer
2k
views
Gini Coefficient and Renyi Entropy
Gini coefficient (aka Gini Index) is a quantity used in economics to describe income inequality. It is 0 for uniformly distributed income, and approaches 1 when all income is in hands of one ...
- 1,592
17
votes
2
answers
2k
views
Calculating the "Most Helpful" review
How would you calculate the order of a list of reviews sorted by "Most Helpful" to "Least Helpful"?
Here's an example inspired by product reviews on Amazon:
Say a product has 8 total reviews and ...
17
votes
4
answers
2k
views
Good introduction to statistics from a algebraic point of view?
There are already lots of questions on this subject like
Is there an introduction to probability theory from a structuralist/categorical perspective?
Is there a combinatorial/topological treatment ...
- 283
17
votes
1
answer
10k
views
Conjugate prior of the Dirichlet distribution?
What is the conjugate prior distribution of the Dirichlet distribution?
Edit: Since I asked this question many years ago, I've written a Python library for working with exponential families. Maximum ...
- 598
17
votes
2
answers
1k
views
The Chow & Robbins game ≈ 0.79295350640: improvements could come from simple statistics, or from a continuous version of the game
This question seeks help with improving a numerical estimate of the value of the Chow and Robbins game. Much about this game is unknown, such as whether its value is rational, but there are two routes ...
- 199
17
votes
1
answer
3k
views
Distribution of maximum of random walk conditioned to stay positive
I have an $n$ step random walk which starts at zero $X_0 = 0 = S_0$ where the steps $X_i$ are independent uniform random variates in $[-1,1]$, but the walk is conditioned on the hypothesis that it ...
- 707
16
votes
7
answers
3k
views
Correlation and Causation. When can we believe correlation (reasonably, at least) imply causation
We always hear, when reading on correlation, that "correlation does not imply causation."
Still, I have never seen any source that tries to answer the question of when can we reasonably conclude a ...
- 361
16
votes
5
answers
4k
views
Is a fair lottery possible?
I'm trying to come up with a scheme for a lottery where each individual has roughly the same chance of becoming the winner, regardless of the number of tickets one holds. So no individual should have ...
- 169
16
votes
1
answer
2k
views
Normal approximation of tail probability in binomial distribution
My problem: From the Berry--Esseen theorem I know, that $$\sup_{x\in\mathbb R}|P(B_n \le x)-\Phi(x)|=O\left(\frac 1{\sqrt n}\right),$$ where $B_n$ has the standardized binomial distribution and $\Phi$ ...
- 497
16
votes
2
answers
637
views
How to sample uniformly from singular matrices
I would like to uniformly sample from all singular $n$ by $n$ Bernoulli matrices (that is each entry is $1$ or $0$ with probability $1/2$). I could of course just sample from all $n$ by $n$ Bernoulli ...
- 283
15
votes
5
answers
1k
views
Longest increasing subsequence as measure of randomness
Although I am by no means an applied mathematician, I like to occasionally explain applications of the math I teach to real world problems. Right now I am teaching some students about longest ...
- 22.9k
15
votes
2
answers
5k
views
What areas of algebra could be interesting to probability theorists?
I would like to find some topic of algebra (beyond linear algebra; algebraic number theory is fine) that would be interesting both to a student that wants to specialize in probability theory and to me ...
- 16.5k
15
votes
3
answers
2k
views
entropy and flatness of densities
I was reading C.R Rao's Linear Statistical inference. Rao presents the entropy of a continuous distribution (expectation of -log density) as a measure of closeness to the uniform distribution, and ...
- 529
15
votes
2
answers
3k
views
Bounding sum of multinomial coefficients by highest entropy one
When does the following hold?
$$\sum_{(i_1,\ldots,i_k)\in E}
\frac{n!}{i_1! \ldots i_k!}
\le \exp(n H^*)$$
where $H^*=\max_{(i_1,\ldots,i_k)\in E} -(\frac{i_1}{n}\log \frac{i_1}{n}+\ldots +\frac{...
15
votes
1
answer
1k
views
Table with the most seated customers in Chinese restaurant process
Suppose we have some initial configuration of people seated at some tables. We start taking new customers and seat them following Chinese restaurant process. Is there some known work on finding the ...
- 151