All Questions
Tagged with pr.probability st.statistics
1,134 questions
91
votes
8
answers
16k
views
Is there a natural random process that is rigorously known to produce Zipf's law?
Zipf's law is the empirical observation that in many real-life populations of $n$ objects, the $k^\text{th}$ largest object has size proportional to $1/k$, at least for $k$ significantly smaller than $...
- 114k
49
votes
13
answers
24k
views
Why is it so cool to square numbers (in terms of finding the standard deviation)?
When we want to find the standard deviation of $\{1,2,2,3,5\}$ we do
$$\sigma = \sqrt{ {1 \over 5-1} \left( (1-2.6)^2 + (2-2.6)^2 + (2-2.6)^2 + (3-2.6)^2 + (5 - 2.6)^2 \right) } \approx 1.52$$.
Why ...
- 673
40
votes
5
answers
5k
views
"Entropy" proof of Brunn-Minkowski Inequality?
I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality.
The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then
$$ m(...
- 22.8k
39
votes
3
answers
4k
views
Manifold of probability measures: connections between two types of metrics
The space of probability measures could be viewed as an infinite-dimensional manifold, equipped with two possible types of metrics — (1) Wasserstein and (2) Fisher-Rao. Metric (1) is connected with ...
- 1,127
37
votes
3
answers
3k
views
On Mathematical Analysis of MathSciNet & MathOverflow
This question has two original motivations: mathematical and social.
The mathematical motivation is mainly based on what I have seen about Zipf's law here and there. The Zipf's law simply states ...
32
votes
3
answers
12k
views
What is the Katz-Sarnak philosophy?
It has been recently mentioned by a speaker (his talk is completely not relevant to random matrix theory/RMT though) that modern statistics, especially random matrices theory, will help solving some ...
- 8,071
32
votes
4
answers
7k
views
Bayesian statistics for pure mathematicians
Could someone please recommend reading on Bayesian statistics presented from a pure mathematical point of view? That is, works that start assuming a good knowledge of measure theoretic probability. ...
32
votes
5
answers
2k
views
You pass X people and Y people pass you: how relatively fast are you?
This question occurs to me every time I go jogging. I suspect every runner probabilist in the world must have thought of it (though I'm no probabilist), but I could not specifically find it online. I ...
- 2,791
26
votes
3
answers
11k
views
L1 distance between gaussian measures
L1 distance between gaussian measures: Definition
Let $P_1$ and $P_0$ be two gaussian measures on $\mathbb{R}^p$ with respective "mean,Variance" $m_1,C_1$ and $m_0,C_0$ (I assume matrices have full ...
- 833
25
votes
1
answer
4k
views
What kind of random matrices have rapidly decaying singular values?
I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the $...
- 118k
24
votes
2
answers
1k
views
Drawing natural numbers without replacement.
Suppose we start with an initial probability distribution on $\mathbb{N}$ that gives positive probability to each $n$. Let's call this random variable $X_1$ so we have $P(X_1=n)=p_{1,n}>0$ for all $...
- 551
23
votes
7
answers
5k
views
What makes Gaussian distributions special?
I'm looking for as many different arguments or derivations as possible that support the informal claim that Gaussian/Normal distributions are "the most fundamental" among all distributions.
...
22
votes
3
answers
6k
views
What is quantum Brownian motion?
It seems that the current state of quantum Brownian motion is ill-defined. The best survey I can find is this one by László Erdös, but the closest the quantum Brownian motion comes to appearing is in ...
- 8,512
22
votes
1
answer
1k
views
Random distance matrices
My question is motivated by the following recent paper:
Gadgil, Siddhartha; Krishnapur, Manjunath, Lipschitz correspondence between metric measure spaces and random distance matrices, Int. Math. Res. ...
- 3,626
21
votes
3
answers
5k
views
James-Stein phenomenon: What does it mean that a James-Stein estimator beats least squares estimator?
Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing.
It is claimed that if one wants to estimate the mean $\Theta$ of
Gaussian ...
- 24.9k
21
votes
1
answer
3k
views
Intuitive Proof of Cramer's Decomposition Theorem
Cramer's decomposition theorem states that if $X$ and $Y$ are independent real random variables and $X+Y$ has normal distribution, then both $X$ and $Y$ are normally distributed. I've seen a few ...
- 4,952
20
votes
1
answer
4k
views
Using Fisher Information to bound KL divergence
Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)?
KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...
- 2,442
19
votes
4
answers
4k
views
Are gaussians with different moments far in total variation distance?
If two Gaussians disagree on one moment, it seems like this should imply that they have a large variation distance--equivalently, if two Gaussians are close in variation distance it's hard for their ...
- 777
19
votes
0
answers
3k
views
What does a product of many Gaussian matrices converge to?
Let $A$ be a product of $n$ $d\times d$ matrices with IID standard Gaussian entries and consider the value of $g(x)=x f(x)$ where $f(x)$ is the density of squared singular values of $A/\|A\|$.
Is ...
- 2,442
18
votes
9
answers
25k
views
Why isn't likelihood a probability density function?
I've been trying to get my head around why a likelihood isn't a probability density function. My understanding says that for an event $X$ and a model parameter $m$:
$P(X\mid m)$ is a probability ...
- 283
18
votes
4
answers
4k
views
Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables
I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables.
Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
- 591
18
votes
2
answers
4k
views
When is the function of a median closer to the median of the function than the mean of the function is to the function of the mean?
Background
notation: RV= random variable, $\mu=$ mean $m=$ median
Jensen's Inequality considers the relationship between the mean of a function of an RV and the function of the mean of an RV.
If $f(...
- 225
18
votes
1
answer
1k
views
Applications of the Giry monad in probability and statistics
In another thread, I asked about the $M$ endofunctor on the category $\operatorname{Meas}$ of measurable spaces, which sends a space $X$ to its space of measures $M(X)$.
Will Sawin described the ...
18
votes
1
answer
1k
views
How fast can extreme eigenvalues of the average of random matrices converge to their expectation?
Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
- 181
18
votes
1
answer
3k
views
Distribution of maximum of random walk conditioned to stay positive
I have an $n$ step random walk which starts at zero $X_0 = 0 = S_0$ where the steps $X_i$ are independent uniform random variates in $[-1,1]$, but the walk is conditioned on the hypothesis that it ...
- 747
18
votes
1
answer
2k
views
Gini Coefficient and Renyi Entropy
Gini coefficient (aka Gini Index) is a quantity used in economics to describe income inequality. It is 0 for uniformly distributed income, and approaches 1 when all income is in hands of one ...
- 1,612
17
votes
4
answers
2k
views
Good introduction to statistics from a algebraic point of view?
There are already lots of questions on this subject like
Is there an introduction to probability theory from a structuralist/categorical perspective?
Is there a combinatorial/topological treatment ...
- 283
17
votes
1
answer
10k
views
Conjugate prior of the Dirichlet distribution?
What is the conjugate prior distribution of the Dirichlet distribution?
Edit: Since I asked this question many years ago, I've written a Python library for working with exponential families. Maximum ...
- 598
16
votes
5
answers
4k
views
Is a fair lottery possible?
I'm trying to come up with a scheme for a lottery where each individual has roughly the same chance of becoming the winner, regardless of the number of tickets one holds. So no individual should have ...
- 169
16
votes
1
answer
397
views
Examples of problems in statistics accessible only using information geometry
I am just curious if there are some examples of problems in statistics that are indeed accessible using information geometry while proofs completely avoiding geometry are unknown. In other words, ...
- 269
16
votes
1
answer
2k
views
Normal approximation of tail probability in binomial distribution
My problem: From the Berry--Esseen theorem I know, that $$\sup_{x\in\mathbb R}|P(B_n \le x)-\Phi(x)|=O\left(\frac 1{\sqrt n}\right),$$ where $B_n$ has the standardized binomial distribution and $\Phi$ ...
- 497
16
votes
2
answers
646
views
How to sample uniformly from singular matrices
I would like to uniformly sample from all singular $n$ by $n$ Bernoulli matrices (that is each entry is $1$ or $0$ with probability $1/2$). I could of course just sample from all $n$ by $n$ Bernoulli ...
- 283
15
votes
2
answers
5k
views
What areas of algebra could be interesting to probability theorists?
I would like to find some topic of algebra (beyond linear algebra; algebraic number theory is fine) that would be interesting both to a student that wants to specialize in probability theory and to me ...
- 16.9k
15
votes
3
answers
2k
views
entropy and flatness of densities
I was reading C.R Rao's Linear Statistical inference. Rao presents the entropy of a continuous distribution (expectation of -log density) as a measure of closeness to the uniform distribution, and ...
- 549
15
votes
2
answers
3k
views
Bounding sum of multinomial coefficients by highest entropy one
When does the following hold?
$$\sum_{(i_1,\ldots,i_k)\in E}
\frac{n!}{i_1! \ldots i_k!}
\le \exp(n H^*)$$
where $H^*=\max_{(i_1,\ldots,i_k)\in E} -(\frac{i_1}{n}\log \frac{i_1}{n}+\ldots +\frac{...
15
votes
1
answer
1k
views
Table with the most seated customers in Chinese restaurant process
Suppose we have some initial configuration of people seated at some tables. We start taking new customers and seat them following Chinese restaurant process. Is there some known work on finding the ...
- 151
14
votes
8
answers
3k
views
Relevant mathematics to the recent coronavirus outbreak
I would like to ask about (old* and new) reliable mathematical literature relevant to various mathematical aspects of the recent coronavirus outbreak: In particular, standard statistical/mathematical ...
14
votes
4
answers
2k
views
How long for a simple random walk to exceed $\sqrt{T}$?
Let $R_n$ be a simple random walk with $R_0 = 0$, and let $T$ be the smallest index such that $k\sqrt{T} < |R_T|$ for some positive $k$.
What is an expression for the probability distribution of $...
- 2,302
14
votes
1
answer
3k
views
How is the "conformal prediction" conformal?
The question is clarified by Prof.V.Vovk. See his answer below for discussion.
Recently, early works of Gammerman, Vanpnik and Vovk[4] are rediscovered by Wasserman et.al[1] and proposed it as a ...
- 8,071
14
votes
1
answer
1k
views
Berry Esseen type result for probability density functions
Let $X_1, X_2, \cdots$ be i.i.d. random variables with $E(X_1) = 0, E(X_1^2) = \sigma^2 >0, E(|X_1|^3) = \rho < \infty$.
Let $Y_n = \frac{1}{n} \sum_{i=1}^n X_i$ and let us note $F_n$ (resp. $\...
13
votes
5
answers
71k
views
How do I convert a uniform value in [0,1) to a standard normal (Gaussian) distribution value?
I have uniform value in [0,1). I'd like to transform it into a standard normal distribution value, in a deterministic fashion.
What I'm confused about with the Box-Muller transform is that it takes ...
- 287
13
votes
7
answers
1k
views
Probabilistic (and other mathematical) methods of physics without the physics?
Many of the methods of physics are vastly more general than their use in that discipline. For example, information theory overlaps with a lot of statistical mechanics, and the latter actually ...
13
votes
1
answer
6k
views
What's the maximum entropy probability distribution given bounds [a,b] and mean?
What is the continuous probability distribution that maximizes entropy, given only the bounds of the random variable [a,b] and the mean mu of the probability distribution?
For example:
if a=0, b=1, ...
- 233
13
votes
1
answer
3k
views
What is the maximum-entropy distribution given mean, variance, skewness, and kurtosis?
$X\in \mathbb{R}$. Which distribution $P(X)$ has the highest possible entropy given its expected value, variance, skewness, and kurtosis? Is it an exponential family distribution of the form $P(X) \...
- 233
13
votes
1
answer
10k
views
KL divergence and mixture of Gaussians
Do we have an exact formula to compute the KL divergence between 2 mixtures of Gaussians (i.e convex combinations of a finite number of Gaussian distributions)?
If not exactly known, are there good ...
- 2,246
12
votes
4
answers
2k
views
Throwing a fair die until most recent roll is smaller than previous one
I roll a fair die with $n>1$ sides until the most recent roll is smaller than the previous one. Let $E_n$ be the expected number of rolls. Do we have $\lim_{n\to\infty} E_n < \infty$? If not, ...
- 48.9k
12
votes
5
answers
1k
views
What is hidden in Hidden Markov Models? [closed]
Why the word "hidden" present in hidden markov model? What exactly is hidden.
Whatever is hidden in HMM isn't it hidden in normal Markov Models?
- 139
12
votes
4
answers
4k
views
Mixtures of Gaussian distributions dense in distributions?
It seems that a mixture of Gaussians can approach any probability distribution, as the number of mixture components approaches infinity. Is this true? And if so, is it precise and correct to say ...
- 141
12
votes
3
answers
8k
views
Constructing Bernoulli random variables with prescribed correlation
For which $n \times n$ correlation matrix $C$ can one construct Bernoulli random variables $(B_1, \ldots, B_n)$ with correlation $C$ ?
Following the approach described in this MO thread, one can ...
12
votes
1
answer
10k
views
Square root of normal distribution
Let $X$ and $Y$ be independent random variates with the same probability distribution, $P(x)$. Assuming that the product $Z=XY$ is a random variate with normal distribution, say $$f_Z(x) = \frac{1}{\...
- 377