All Questions
Tagged with pr.probability st.statistics
1,135 questions
12
votes
4
answers
4k
views
Mixtures of Gaussian distributions dense in distributions?
It seems that a mixture of Gaussians can approach any probability distribution, as the number of mixture components approaches infinity. Is this true? And if so, is it precise and correct to say ...
- 2,772
11
votes
0
answers
225
views
Functional Weak Convergence of Maximum Likelihood Estimator
Let $\hat{\theta}_n$ be the Maximum Likelihood Estimator of parameter $\theta$, where $n$ is the sample size. It is well-known that under sufficient regularity conditions, we have the asymptotic ...
- 87
2
votes
1
answer
192
views
Upper confidence bound for Poisson process rate parameter
Admittedly, this is an elementary question for mathoverflow. However, I've had no real bites on math and stats.stackexchange so I'm cross-posting.
I am interested in computing an upper confidence ...
- 128k
3
votes
0
answers
95
views
Empirically random, quickly multiplicable matrices
I have encountered a need for fast computation of a transformation $Ax$ where $A\in \mathbb{C}^{K\times N},\ K\sim 10^7,\ N\sim 10^3$ is designed, and $x\in \mathbb{C}^N$ has iid $\mathcal{CN}(0,1)$ ...
5
votes
1
answer
222
views
Switching oriented paths in a graph
Consider an oriented graph (e.g. a finite part of the standard grid with some random orientations).
Each minute the following operation takes place: we choose uniformly randomly an ordered pair $(A,B)...
CommunityBot
- 1
1
vote
2
answers
237
views
Fair partitioning of a set - Weighted sums of Bernoullis
For $n$ an integer, let $a_n$ be the number of ways in which one may partition the set $\{1, \ldots, 2n \}$ in two parts with:
the same number of elements: $n$
and the same sum: $2n(2n+1)/4$.
...
- 14.2k
2
votes
0
answers
244
views
An inequality regarding centered Bernoulli random variables
Let $\left\{V_1,\ldots,V_n\right\}$ be a set of (possibly dependent) identically distributed Bernoulli random variables, with
$$ p = \mathbb{P}\left(V_i=1\right) = 1-\mathbb{P}\left(V_i=0\right),\quad ...
- 159
2
votes
0
answers
80
views
Bridging between Rosethal Inequalities and log convex tails
Let $X_1,\ldots,X_n$ be independent with $\mathbf{E}[X_i] = 0$ and $\mathbf{E}[|X_i|^t] < \infty$ for some $t \ge 2$. Write $\|X\|_p = (E|X|^p)^{1/p}$.
Then we have the classical "Rosenthal-type ...
10
votes
1
answer
484
views
Stochastic Covering Number of a Convex Set
Consider a convex set, say $S = [0,1]^d$. Let $X_1, X_2,\ldots,X_n, \ldots$ be i.i.d. random variables that are uniformly distributed on $S$. Denote the Euclidean ball centered at $x \in \mathbb{R}^d$ ...
- 724
1
vote
1
answer
251
views
Expand the pdf of Wishart distribution into power series via orthogonal polynomials
In the univariate case ($\chi^2$ distribution), I know we can expand the pdf into power series of the variance $\sigma^2$ with Laguerre polynomials. Indeed, since the Laguerre polynomials are related ...
3
votes
1
answer
167
views
Maximal correlation and independence
Let $X$ and $Y$ be random variables. Then the maximal correlation $\rho_m(X;Y)$ is defined as
$$ \rho_m (X;Y) := \max_{(f(X),g(Y))\in S} \mathbb{E} [f(X)g(Y)] $$
where $S$ is the collection of pairs ...
- 128k
6
votes
1
answer
203
views
Good upper-bound for $\mathbb E[|X-np|^r]$ where $X \sim \text{Binomial}(n,p)$ and $r \ge 1$
Disclaimer. Question moved from SE.
Setup
Let $X \sim \text{Binomial}(p, n)$, and $r \ge 1$.
Question
What is a good upper-bound for $\mathbb E[|X-np|^r]$ ?
Solution for small $r$
If $r=2$, then ...
- 128k
5
votes
0
answers
711
views
Concentration inequality for max component of a multivariate Gaussian in the general case
I am looking to bound the variance of the maximum component of a vector distributed multivariate Gaussian in the general case where the Gaussian distribution has arbitrary mean and full covariance ...
- 283
7
votes
0
answers
179
views
Can one "smooth over" k-wise independence to get actual independence?
I came across the following toy problem and was curious if there was a simple solution or counterexample. Suppose you have a distribution $p$ on $m$ random variables $X_1, \ldots, X_m$, each with ...
1
vote
2
answers
250
views
Finite-sample deviation bound of empirical distribution from true distribution
Let $P=(p_1,\ldots,p_k) \in \Delta_k$ be distribution supported on set of size $k$ and let $\hat{P}_n$ be an empirical version of $P$ based on an iid sample of size $n$.
Question
What's a good non-...
- 4,529
1
vote
0
answers
448
views
Gaussian Integrals over Spheres
I'm after a reference for an integral. In particular, I am looking a way to approximate or calculate the following:
$$ \int \limits_{\| \theta \|_2 = 1} e^{(-(\theta - \mu)^T \Sigma (\theta - \mu))} ...
- 11
4
votes
4
answers
4k
views
"Square root" of Beta(a,b) distribution
Under what conditions on a and b is there a distribution $f_{a,b}$ such that the product $XY$ of two independent realizations $X$ and $Y$ from $f_{a,b}$ has a Beta(a,b) distribution?
A standard ...
- 6,367
4
votes
1
answer
839
views
A balls into bins problem with combinatorial constraints
We are given $m$ balls and $n$ bins, with $m \ge n$. Each bin can contain at most $c$ balls (we assume that $c$ is an even integer). In a sequential fashion, at each time step, one ball is placed into ...
- 1,677
2
votes
0
answers
57
views
Sample $k$ of $n$ numbers (with replacement), what is the probability for a certain from the $n$ numbers to be the median of the $k$ numbers [closed]
Suppose we have ordered numbers $a_1 < a_2 < \dots < a_n$. Now we sample $k$ of them with equal probability and with replacement and compute the median of these and call it $m$. (Let us ...
- 29
4
votes
1
answer
142
views
Linear combination of coordinates of random unit vector
Let $v\in \mathbb{R}^n$ be uniformly distributed on the unit sphere. Let $\lambda_1,...,\lambda_n$ be given real numbers. What is the distribution of
$$X=\sum_{i=1}^n\lambda_iv_i^2\;?$$
Does it happen ...
- 128k
2
votes
1
answer
206
views
Density of random matrix only depends on its spectrum
Suppose a random positive definite matrix $A\in\mathbb{R}^{n\times n}$ has density function (with respect to the lebesgue measure on $\mathbb{R}^{n(n+1)/2}$) $f(A)=g(\lambda_1(A),...,\lambda_n(A))$ ...
- 128k
2
votes
1
answer
110
views
Is independence preserved if a random entry in an independent sequence is replaced by a constant?
Let $\xi=(\xi_1,\ldots,\xi_n)$ be a sequence of independent random variables.
Let us pick an index $\nu\in \{1,\ldots,n\}$, and replace the entry $\xi_\nu$ by a constant $c$. The rest of the $\xi_i$ ...
- 128k
2
votes
1
answer
235
views
Kolmogoroff condition for truncated random variables
Question summary. Does the Kolmogoroff condition $\sum_{n=1}^\infty\frac{\mathbb V Y_n}{n^2} < \infty$ hold for truncated random variables $Y_n := X_n \cdot 1_{\{X_n \le n\}}$ (see below for a more ...
- 128k
1
vote
1
answer
41
views
Probability that maximal elements has the same position in samples from correlated random variables
Let $x$ and $y$ be two correlated random variable (say, standard normal) with correlation coefficient $\rho>0$. Let $X= \{x_1, x_2, ..., x_L\}$ and $Y= \{y_1, y_2, .. y_L\}$ be samples of size $L$ ...
- 128k
8
votes
2
answers
976
views
Approximation of Wasserstein distance between $p_\theta$ and $p_{\theta + d\theta}$
Given a parametric family of distributions $\{p_\theta\mid\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid
$$\operatorname{KL}(p_\theta\...
- 4,729
1
vote
0
answers
123
views
Sanov-type finite-sample bound on $KL(P\|\hat{P}_n)$
Let $P$ be a distribution on an alphabet of size $k$ and let $\hat{P}_n$ be an empirical version of $P$ via $n$ i.i.d samples $a_1,\ldots,a_n \sim P$, i.e $\hat{P}_n := (1/n)\sum_{i=1}^n\delta_{a_i}$.
...
- 6,853
1
vote
1
answer
313
views
Bounds on difference between "logsumexp" and variance?
Let $Z$ be a random variable with finite moment-generating function $M_Z(\theta):=E[e^{\frac{1}{\theta}Z}]<\infty$ for all $\theta > 0$, and for $\delta \in (0,1]$, define
$C_Z^\delta := \inf_{\...
- 128k
2
votes
1
answer
790
views
Is there some generalization of the "Maximum Coverage Problem" for information in random variables?
Say I have a set $X=x_1,x_2,\ldots,x_n$ of random variables, and would like to find a size $k\leq|X|$ subset that contains as much information as possible. This is complicated because the variables ...
- 155
1
vote
1
answer
92
views
boundig variation from median [closed]
Given a scalar random variable $X$, suppose that there are positive constants $c_{1}$ and $c_{2}$ such that
$$\forall t\geq 0 : \,\,\,\,\,\,\ \mathbb P\{|X-\mathbb EX|\geq t\}\leq c_{1}e^{-c_{2}t^{2}}...
- 128k
6
votes
1
answer
2k
views
Kullback Leibler "variance": does that divergence have a name?
If you consider two probability distributions $p$ and $q$, one way to measure the distance between the two is the Kullback-Leibler divergence:
$$KL(p,q)=\int p \log (p/q) = E_p(\log p/q)$$
and this ...
- 1
1
vote
1
answer
88
views
Independence of r.v.'s following a distribution that is the ratio between complex Gaussian and Chi-square r.v.'s
Given the following two R.V.s
$$z_1 = \frac{x_1}{|x_1|^2 + |x_2|^2 + \cdots + |x_M|^2}$$
and
$$z_2 = \frac{x_2}{|x_1|^2 + |x_2|^2 + \cdots + |x_M|^2}$$
where $x_i \sim \mathcal{CN}(0,a), \forall i$...
- 63.9k
2
votes
0
answers
107
views
Expectation of a Random Matrix that Contains Wishart Form
I am interested in calculating the expectation of the following random matrix:
$$A=WX(X^TWX)^{-1},$$
where $W \sim W_p(n,I)$ is a $p\times p$ random Wishart matrix, and $X$ is a fixed $p\times m$ ...
- 43
2
votes
1
answer
254
views
Asymptotic rate for the expected value of the square root of sample average
I have iid random variables $X_1, \dots, X_n$ with $X_i \geq 0$, $E[X_i]=1$ and $V[X_i] = \sigma^2$.
Let $S_n = \frac{\sum_{i=1}^n X_i}{n}$.
I'd like to say that $E[\sqrt{S_n}] = 1-O(1/n)$.
My first ...
- 128k
3
votes
1
answer
498
views
Strictly Proper Scoring Rules and f-Divergences
Let $S$ be a scoring rule for probability functions. Define
$EXP_{S}(Q|P) = \sum \limits_{w} P(w)S(Q, w)$.
Say that $S$ is striclty proper if and only if $P$ always minimises $EXP_{S}(Q|P)$ as a ...
- 4,529
2
votes
1
answer
214
views
Bounding integral arising from expectation of a random variable satisfying Bernstein's inequality
Let $X$ be a random variable s.t. for $v, b > 0$ and $C \geq 1$:
$$ P(X \geq t) \leq C\exp\left(-\frac{t^2}{2(v^2 + bt)} \right) $$
I am trying to show that $\mathbb{E}X \leq 2v(\sqrt{\pi} + \...
- 128k
3
votes
1
answer
826
views
concentration inequality for a weighted sum of independent but not identical binary variables
Let $\alpha\in[0,1]$ be a fixed constant, and let
$w,x\in[0,1]^n$ be two vectors such that $\sum_i w_i x_i=\alpha$.
Define $Y = \sum_i w_i X_i$, where $X_i \sim \operatorname{Bernoulli}(x_i)$, so it ...
- 3,993
3
votes
0
answers
150
views
Central Limit Theorem for simultaneous sums
Take a sample $X_1 \ldots X_n$ of $n$ independent observations $X_j \in \mathbb{R}$ with zero mean and finite variances $\sigma_j^2$. For $i = 1, 2, \ldots$, define the sums $$S^n_i = \frac{\pm X_1 \...
CommunityBot
- 1
1
vote
0
answers
56
views
About a class of expectations
Consider being given a $n-$dimensional random vector with a distribution ${\cal D}$, vectors $a \in \mathbb{R}^k$, $\{ b_i \in \mathbb{R}^n \}_{i=1}^k$ and non-linear Lipschitz functions, $f_1,f_2 : \...
- 2,246
1
vote
0
answers
151
views
Clarification about the ϵ -net argument
I have been reading the paper Do GANs learn the distribution? Some theory and empirics.
In Corollary D.1, they reference the paper Generalization and Equilibrium in Generative Adversarial Nets which ...
- 19
7
votes
4
answers
4k
views
Estimating the probability that one Poisson RV is larger than another
Let $X$ and $Y$ be Poisson random variables with means $\lambda$ and $1$, respectively. The difference of $X$ and $Y$ is a Skellam random variable, with probability density function
$$\mathbb P(X - Y ...
- 2,821
2
votes
1
answer
848
views
Concentration inequality for quadratic form of Gaussian variables with non-idempotent matrix
Given $y \sim N(0,\sigma^2 I)$, and $M$ that is a symmetric matrix (not necessarily idempotent)
what is the distribution of ${y^T M y}$?
is there a high probability bound on $|{y^T M y}|$?
Most ...
- 128k
3
votes
1
answer
196
views
Uniform Convergence for Vectors
$\textbf{Problem statement:}$
Let $\mathcal H:\mathcal X \rightarrow \{0,1\}$ be a class of Boolean functions for $\mathcal X \subset \mathbb R^n$, and let the VC Dimension of $\mathcal H$ be $VC_{...
- 63.9k
11
votes
1
answer
413
views
First Table of Random Numbers
What was the first table of random numbers of any sort?
The best I can do is Tippett and Pearson's Random Sampling Numbers of 1927.
Can anybody identify an earlier table?
Thanks for any insight....
CommunityBot
- 1
10
votes
3
answers
2k
views
Random Walks on high dimensional spaces
I've read on a paper that, in the two dimensional case, if you start from the origin and take steps of length one in arbitrary directions (uniformely on the unit sphere $S^1$, not left-right-up-down), ...
3
votes
2
answers
430
views
Multivariate normal concentration
If $X\sim N(0,\Sigma)$ for some $d$-dimensional normal distribution, then $X = \Sigma^{1/2} Z$ where $Z\sim (0,I)$. How to compute the following quantity?
$$
\operatorname{var} (X^T X)
=
\...
- 593
1
vote
1
answer
170
views
Minimization Proof of Conditioning on Gaussian is Gaussian
It is well known that $E[X|X+Y]$ is Gaussian if both $X$ and $Y$ are, and the result can be derived using standard density arguments. However, how can one prove it by only resulting to optimization ...
- 11
8
votes
2
answers
1k
views
Does Multiplicative Version of Azuma's Inequality Hold?
It is known that there are multiplicative version concentration inequalities for
sums of independent random variables. For example, the following
multiplicative version Chernoff bound.
Chernoff bound:...
- 128k
1
vote
1
answer
137
views
Approximate $\log \mathbb E_P[\exp(th(x)]$ for a function $h$ which is lipschitz and has finite moments of order 1 and 2 w.r.t $P$
Let $P$ be a probability measure on a space $\mathcal X$ and $h: \mathcal X \rightarrow \mathbb R$ is measurable function with finite moments of order 1 and 2. I'm interested in approximating the ...
- 128k
1
vote
0
answers
84
views
Central limit theorem is expected, but variance is apparently sublinear? So?
In Perplexing Problems in Probability, the following statement about first passage percolation (FPP) on $\mathbb{Z}^{d}$ is made. See e.g. this paper of Benjamini, Kalai and Schramm, where they quote ...
- 640
-3
votes
2
answers
450
views
Expected values of two random variables related to a simple urn problem
In an urn there are $u$ balls, $b$ of which are black.
If we perform $n$ trials of one ball at a time with replacement, the probability of the event $E$ to get $n$ times a black ball is $P(E)=\left(\...
- 1,226