All Questions
Tagged with pr.probability st.statistics
1,135 questions
1
vote
1
answer
217
views
How to normalize an Inverse Wishart random matrix?
Background:
Let $d\in \mathbb{N}$.
Define the space of (real symmetric) positive definite matrices of size $d\times d$ as follows:
\begin{align}
\mathcal{S}_{++}^d := \big\{\mathbb{M}\in \mathbb{R}^{d\...
- 219
6
votes
1
answer
527
views
Can information be extracted more precisely using more random trials?
Write $n$ iid draws of $(x,y)$ as $(x^n, y^n)$. Fix $R\in (0,H(x))$. What is the min of $n^{-1}H(y^n|f(x^n))$ over maps $f$ with range $\lbrace 1,\dots,\exp nR\}$, taking $n\to \infty$?
4
votes
1
answer
114
views
Consistent empirical estimation of Radon transform of a multivariate density function
Let $P$ be a "nice" distribution on $\mathbb R^m$ (e.g., multivariate Gaussian, etc.), with density $p$. Let $H := \{x \in \mathbb R^m \mid x^\top w = b\}$ be a hyperplane in $\mathbb R^m$ ...
- 6,853
0
votes
2
answers
341
views
Conditions for existence of a distribution with full support
Consider a $6\times 1$ continuous random vector
$$
\eta\equiv (\eta_1,\eta_2,..., \eta_6)
$$
satisfying the following property:
$$
\underbrace{\begin{pmatrix}
\eta_1\\
\eta_2\\
\eta_3
\end{pmatrix}}_{\...
- 108
2
votes
1
answer
165
views
Is a random $(r+1,r)$-biregular bipartite graph $r$-edge connected w.h.p?
A uniformly random $r$-regular bipartite graph on $n$ vertices is known to be $r$-edge connected. That is, with high probability as $n$ grows large, the minimum size of a cut in a random $r$-regular ...
- 153
0
votes
0
answers
195
views
Upper-bound for bracketing number in terms of VC-dimension
Let $P$ be a probability distribution on a measurable space $\mathcal X$ (e.g;, some euclidean $\mathbb R^m$) and let $F$ be a class of funciton $f:\mathcal X \to \mathbb R$. Given, $f_1,f_2 \in F$, ...
- 6,853
2
votes
2
answers
690
views
Concentration and anti-concentration of gap between largest and second largest value in Gaussian iid sample
Let $n \ge 3$ be an integer and let $X=(X_1,\ldots,X_n)$ be random vector with iid coordinates from $N(0,1)$. For $1 \le k \le n$, let $X_{(k)}$ be the value of the $k$th largest coordinate of $X$.
...
- 6,853
1
vote
1
answer
143
views
Permute a sparse random matrix to resemble a diagonal matrix as much as possible
Say we generate an $N \times N$ sparse random matrix $W$, where each element $W_{ij}$ was independently chosen to be $1$ with probability $p=\frac{a}{N}$, and $0$ with probability $1-p$. We are ...
- 433
0
votes
0
answers
97
views
Verification of a certain computation of VC dimension
Disclaimer: I'm not very familiar with the concept of VC dimensions and how to manipulate such objects. I'd be grateful if expects on the subject (learning theory, probability), could kindly proof ...
- 6,853
1
vote
0
answers
351
views
Tail bounds for random Gaussian chaos?
Let $g = (g_1, \dots, g_d)$ be a sequence of independent standard Normal random variables, and suppose $\Sigma$ is a $d \times d$ (deterministic), real, symmetric, positive definite matrix. The Hanson-...
- 380
0
votes
0
answers
29
views
k-means errors for a block Gaussian vector
Consider a standard centered Gaussian vector $(X_1,...,X_n)$ with an approximate block structure, i.e. there is $q$ and a partition of $\{1,...,n\}$ in $q$ classes such that if $i,j$ are in the same ...
- 1,352
0
votes
1
answer
133
views
How to demonstrate a correlation inequality? [closed]
If there are 3 vectors X, Y, Z of the same length, for any $x_i \in X,y_i \in Y,z_i \in Z$, we have $0<x_i<1,0<y_i<1,0<z_i<1$.
The correlation between Z, Y is greater than between X, ...
- 11
1
vote
1
answer
475
views
Convergence of quadratic form $y^T Q y$ where $y$ is a random iid sequence of length $n$ and $Q$ is an $n \times n$ random matrix independent of $y$
For each positive integer, let $Q_n=(q_{i,j})_{i,j \in [n]}$ be a random $n \times n$ psd matrix. In the limit $n \to \infty$, suppose the eigenvalues of this sequence of matrices are uniformly ...
- 6,853
3
votes
1
answer
318
views
Divergence-free Gaussian vector field with given mean magnitude and correlation function
My general question is how to construct an isotropic random vector field $\vec f: \mathbb{R}^3 \to \mathbb{R}^3$ with a given mean magnitude $\mathbb{E}[\|\vec f(\vec x)\|]=\mu$ and with vector ...
- 147
13
votes
5
answers
71k
views
How do I convert a uniform value in [0,1) to a standard normal (Gaussian) distribution value?
I have uniform value in [0,1). I'd like to transform it into a standard normal distribution value, in a deterministic fashion.
What I'm confused about with the Box-Muller transform is that it takes ...
- 287
1
vote
1
answer
221
views
Large deviation for empirical median
I found this exercise while reading some notes on Large Deviation Principle. This exercise is at the end of the very first chapter, including Cramer's Theorem and essentially nothing more (no Sanov ...
- 445
1
vote
1
answer
202
views
A problem related to stochastic ordering
Let $\boldsymbol{X} = (X_1,X_2)^{\rm T}\sim \mathcal{N}_2(\boldsymbol{\mu}, \mathrm{\Sigma})$, where
\begin{eqnarray*}
\boldsymbol{\mu} = (\mu_1, \mu_2)^{\rm T}& = &(\sqrt{\xi_1\xi_2/(\xi_1+\...
- 211
1
vote
0
answers
489
views
Can we generalize the concept of "characters" in group theory via methods from statistics and probability theory?
$\DeclareMathOperator\Cov{Cov}$Motivation: If $G$ is a finite group and $\phi=X+iY: G\to \mathbb{T}$ is a character of $G$, then $\Cov(X,Y)=0$ where $X$, $Y$ are considered as two real random ...
- 366
2
votes
1
answer
256
views
About a mixture
Consider the following mixture model for a univariate density function
$$
(1) \quad f(x)=\int_{(m, \sigma^2)\in D} g(x; m, \sigma^2) \mu(d(m, \sigma^2))
$$
where $D$ is a compact subset of $\mathbb{R}\...
- 108
1
vote
1
answer
157
views
Moments of rescaled Bernoulli random matrix
Suppose $X \in \{0,1\}^{n \times m}$ is a matrix generated according to the following generative process:
$$Z_{ij} \sim \text{Bernoulli}(p) \implies X_{ij} = \frac{Z_{ij}}{\sum_{k=1}^m Z_{ik}}.$$
Is ...
- 269
2
votes
2
answers
633
views
An alternative proof of Bayesian Cramer-Rao
My question is:
Are there an alternative proof of Cramer-Rao lower bound that does not use
Cauchy-Swartz inequality?
Let me outline the classical proof and explain why I am interested in this ...
- 671
3
votes
0
answers
93
views
Explaning why the spectrum of a setting simple structure random matrix is always spiked ($d-1$ eigenvalues close to zero, and $1$ away from zero)
For concreteness, let $m=500$, $d=600$, $N=1000$. Let $W$ be and $d \times m$ matrix with unit-norm rows and let $u$ be a uni-norm vector of length $m$. Given a binary vector $b$ of length $m$, length ...
- 6,853
2
votes
0
answers
61
views
Approximate logarithmic bound on expected maximum via central limit theorem
If $Z_i$ are standard normal, possibly dependent, one can show that
$$E\left[\max_{i=1,...,M} Z_i^2\right]\leq 3\ln M + 1.$$
I'm looking for a similar (asymptotic) bound for asymptotically normal ...
- 203
1
vote
0
answers
24
views
Minimax statistical estimation of proximal transform $\mbox{prox}_g(\theta_0)$, from linear model data $y_i := x_i^\top \theta_0 + \epsilon_i$
tl;dr: My question pertains the subject of minimax estimation theory (mathematical statistics), in the context of linear regression.
Given a vector $\theta_0 \in \mathbb R^d$, consider the linear ...
- 6,853
2
votes
1
answer
187
views
Compute the limit of trace of inverse of square of rank-1 perturbation of Wishart matrix
Let $a \ge 0$, $b,c>0$ be fixed constants, and let $X$ be an $m \times d$ random matrix with entries drawn iid from $N(0,1/d)$. Consider the random psd matrix $S := a 1_m 1_m^\top + b XX^\top + c ...
- 6,853
1
vote
1
answer
613
views
Integral of the product of a gaussian pdf and cdf
I am trying to solve the integral of a gaussian cumulative distribution function and a gaussian probability function. On this site I have seen solutions of similar, less general integrals (e.g. ...
- 11
10
votes
3
answers
2k
views
Random Walks on high dimensional spaces
I've read on a paper that, in the two dimensional case, if you start from the origin and take steps of length one in arbitrary directions (uniformely on the unit sphere $S^1$, not left-right-up-down), ...
- 899
7
votes
2
answers
524
views
Projections in infinite dimensional statistical manifolds
I'm struggling to understand the geometry of projection for infinite dimensional statistical manifolds. In finite dimensions, a strictly convex smooth function $F$ defines a Bregman divergence. From ...
- 31
2
votes
0
answers
87
views
The covariance of certain random variable
We define two random variables $X_n,Y_n $ on the sample space $\{1,2,3,\cdots,n\}$ with counting measure. We denote by $C_n$ the covariance of theses two random variables: $C_n=Cov(X_n,Y_n)$.
...
- 366
0
votes
1
answer
3k
views
In linear regression, we have 0 training error if data dimension is high, but are there similar results for other supervised learning problems?
I tried posting this question on Cross Validated (the stack exchange for statistics) but didn't get an answer, so posting here:
Let's consider a supervised learning problem where $\{(x_1,y_1) \dots (...
- 1,467
5
votes
1
answer
150
views
Kullback–Leibler chains
The following question was asked and then deleted by the post author:
Let $P$ and $Q$ be two probability distributions defined over the same space, with $KL(P \parallel Q) < \infty$. For $\epsilon ...
- 128k
4
votes
2
answers
175
views
Almost independence of $x^\top a$ and $x^\top b$ for $x$ uniform on the sphere in $\mathbb R^d$ and $a,b \in \mathbb R^d$ with $a^\top b = 0$
Let $d$ be a large positive integer. Let $x$ be uniformly distributed on the unit-sphere in $\mathbb R^d$ and let $a$ and $b$ be perpendicular vectors in $\mathbb R^d$, i.e such that $a^\top b=0$. Let ...
- 6,853
3
votes
3
answers
483
views
$H(p) \le H(q) + KL(p, q)$?
Let $H(p) = \sum_i p_i\log\frac{1}{p_i}$ be the entropy of $p$
and $KL(p, q) = \sum_i p_i\log\frac{p_i}{q_i}$ be the KL divergence between $p$ and $q$. Does it hold that $H(p) \le H(q) + KL(p, q)$?
...
- 143
3
votes
1
answer
514
views
Complete statistical manifolds
Here, by a statistical manifold I mean a $d$-dimensional Riemannian manifold whose points are probability measures on $\mathbb{R}^n$. What are some well-studied/interesting examples of statistical ...
2
votes
1
answer
1k
views
Finding the expectation $\mathrm{E} (1/ X)$ for a negative binomial random variable $X$
Suppose a random variable $X$ is distributed as $\operatorname{NB}(\mu, \theta)$, and its mass is as follows
$$ \mathrm{P}(X = y) = \binom{y + \theta - 1}{y} \left(\frac{\mu}{\mu + \theta}\right)^{y}\...
- 331
1
vote
1
answer
104
views
Limiting value of $\dfrac{1_n^\top B^{-1} A B^{-1} 1_n}{d}$, where $A=WW^\top + a I_n$, $B = WW^\top + b I_n$, and $W \sim N(0,\Sigma_d)$
Let $n$ and $d$ be positive integers with
$$
n,d \to \infty, \quad n/d \to \rho \in (0,\infty).
$$
Let $\Sigma_d$ be a psd matrix such that
$\mbox{trace}(\Sigma_d) = 1$.
$\|\Sigma_d\|_{op} = \mathcal ...
- 6,853
1
vote
3
answers
270
views
Practical pseudorandom generators
It is known that existence of pseudorandom generators (PRGs) is equivalent to the existence of one-way functions. In turn, the latter is an open problem.
I am curious if someone developed kind of &...
- 791
1
vote
1
answer
106
views
What is the maximum possible coefficient of variation for data taking values within a specified range?
I have a question that seems very basic, and yet I have not managed to find an answer after probably several hours of Google-searching.
Fix $0<a<b<\infty$, and let $\mathcal{P}_{[a,b]}$ be ...
- 708
0
votes
1
answer
209
views
Factorisation of Gaussian random matrix into random Hermitian and correction factor
By the Bartlett decomposition, one has that for $k \leq n$ and $\mathbf{\Gamma}_{n\times k} \in \mathbb{R}^{n\times k}$ a standard Gaussian matrix with independent entries
$$\mathbf{\Gamma}_{n\times k}...
user274127
1
vote
1
answer
226
views
Orthogonal transformation of multivariate Bernoulli-Gaussian distribution
Actually, I have asked this question in https://math.stackexchange.com/questions/4330127/orthogonal-transformation-of-multivariate-bernoulli-gaussian-distribution, but I think mathoverflow might be ...
- 21
1
vote
0
answers
910
views
Is Steven J. Miller's "research" on election fraud sound? And was he paid for it? [closed]
I recently encountered the following piece regarding alleged massive voter fraud in Pennsylvania:
https://justthenews.com/sites/default/files/2020-11/...
0
votes
1
answer
83
views
The distribution of number of reverse order pairs in a randomly permuted array
There is an array $a_1,\dotsc,a_n$ whose elements are pairwise distinct. We define a reverse order pair to be an ordered pair $(a_i,a_j)$ such that $i < j$ and $a_i > a_j$. Consider the total ...
4
votes
0
answers
144
views
Exponential families closed under affine transformations
Let $(\Omega,\Sigma,\mu)$ be a probability space and let $\mathcal{M}$ be an exponential family of probability distributions for $\mu$ of the following form: There are $\varphi_1,\dots,\varphi_n:\...
- 5,405
3
votes
2
answers
488
views
Question about a new pseudo-random number generator
While investigating non-periodic RNG's (random number generators) for irrational numbers, I came up with a version that actually produces pseudo-random words consisting of $N$ bits, where $N$ is ...
- 3,098
2
votes
2
answers
322
views
Integral of product of Hermite polynomials w.r.t marginal distribution of first two-coordinate of random vector on unit-sphere
This question is related to: https://math.stackexchange.com/q/4270522/168758
Let $H_n(x) \in \mathbb R[x]$ be the probabilist's $n$th Hermite polynomial. This an $n$th degree polynomial given by the ...
- 6,853
16
votes
5
answers
4k
views
Is a fair lottery possible?
I'm trying to come up with a scheme for a lottery where each individual has roughly the same chance of becoming the winner, regardless of the number of tickets one holds. So no individual should have ...
- 169
-1
votes
1
answer
297
views
The distribution of the sum of values from a normal and a truncated normal distribution
Using R to extract truncated normal distribution samples and normal distribution samples separately, when they are combined, the image drawn by the hist function is very similar to a normal ...
21
votes
3
answers
5k
views
James-Stein phenomenon: What does it mean that a James-Stein estimator beats least squares estimator?
Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing.
It is claimed that if one wants to estimate the mean $\Theta$ of
Gaussian ...
- 24.9k
0
votes
0
answers
36
views
How to recalculate the weights for an event that happens multiple times and requires all outcumes to be unique?
I think it's easiest to explain with an example.
I have a weighted probability list
A : 0.15
B : 0.15
C : 0.15
D : 0.1
E : 0.1
F : 0.1
G : 0.1
H : 0.075
I : 0.075
...
1
vote
1
answer
276
views
Upper bound for $P(X \geq x)$, where $X \sim \operatorname{Pois}(\lambda)$
I posted the following question in a comment on CDF of a log-concave discrete random variable. Since it is not related to my main question, I thought of reposting it as separate post.
Question:
Let $X ...
- 135