All Questions
Tagged with pr.probability st.statistics
1,134 questions
5
votes
1
answer
346
views
Bounding the sensitivity of a posterior mean to changes in a single data point
There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...
CommunityBot
- 1
19
votes
0
answers
3k
views
+200
What does a product of many Gaussian matrices converge to?
Let $A$ be a product of $n$ $d\times d$ matrices with IID standard Gaussian entries and consider the value of $g(x)=x f(x)$ where $f(x)$ is the density of squared singular values of $A/\|A\|$.
Is ...
- 2,242
1
vote
1
answer
160
views
Given iid $w_1,\dotsc,w_N \sim N(0,1/d)$ iid, find a simple matrix $A$ s.t $\|aa^T-A\|_\text{op}\to0$, where $a_i := E_{G \sim N(0,1)}[f(\|w_i\| G)]$
Let $d$ and $N$ be two large comparable integers, for example assume
$$
N,d \to \infty, \quad d/N \to \gamma \in (0,\infty).
$$
Let $w_1,\dotsc,w_N$ be iid from $N(0,(1/d)I_d)$ and let $f:\mathbb R \...
- 12.9k
1
vote
0
answers
39
views
Can conditional distributions with respect to a sufficient sub-$\sigma$-algebra be represented by a single Markov kernel?
Let $(\Omega, \mathcal{F})$ be a measurable space, and let $\mathcal{P}$ be a collection of probability measures on this space. A sub-$\sigma$-algebra $\mathcal{G} \subset \mathcal{F}$ is said to be ...
- 111
14
votes
1
answer
1k
views
Berry Esseen type result for probability density functions
Let $X_1, X_2, \cdots$ be i.i.d. random variables with $E(X_1) = 0, E(X_1^2) = \sigma^2 >0, E(|X_1|^3) = \rho < \infty$.
Let $Y_n = \frac{1}{n} \sum_{i=1}^n X_i$ and let us note $F_n$ (resp. $\...
- 103
9
votes
2
answers
429
views
Hermite–Fourier expansion for the median
Let $n$ be an odd positive integer. Let $M : \mathbb{R}^n \to \mathbb{R}$ be the median function: $M(x_1,\dots,x_n)$ is the median of $x_1,\dots,x_n$. What can be said about the Hermite–Fourier ...
- 128k
49
votes
13
answers
24k
views
Why is it so cool to square numbers (in terms of finding the standard deviation)?
When we want to find the standard deviation of $\{1,2,2,3,5\}$ we do
$$\sigma = \sqrt{ {1 \over 5-1} \left( (1-2.6)^2 + (2-2.6)^2 + (2-2.6)^2 + (3-2.6)^2 + (5 - 2.6)^2 \right) } \approx 1.52$$.
Why ...
- 30.1k
0
votes
0
answers
343
views
Can KL divergence go to 0, but $E[\log(p/q)^2]$ diverge in certain cases?
Let $p(x)$ be a fixed distribution over a discrete space.
Let $A, C > 0$ be constants.
Let $\epsilon > 0$. Can we find an example of a distribution
$q_{\epsilon}$ such that $\mathrm{KL}(p||q_{\...
- 3,065
2
votes
1
answer
415
views
High-probability lower bound for norm of least squares solution when both design matrix $X$ and response vector $y$ are random (and independent)
Let $n,d \to \infty$ with $n/d \to \gamma \in (0,\infty)$. Let $X$ be a random $n \times d$ matrix independent rows uniformly distributed on the the unit-sphere in $\mathbb R^d$ and let $y$ be a ...
CommunityBot
- 1
3
votes
0
answers
92
views
Tighter Freedman's inequality for a special martingale difference sequence
Let $X_{1}, \ldots, X_{T} \in \{0, 1\}$ be a sequence of Boolean random variables with
$$
\mathbb{E}[X_{t} | X_{1}, \dots, X_{t - 1}] = p_{t}.
$$
Consider the sequence $Y_{t} := X_{t} - p_{t}$ (which ...
- 63.9k
0
votes
1
answer
552
views
Hypothesis testing for not identically distributed random variables conditioned on the outcome of a subset
I encountered the following problem (I give more details of the problem at the end of the post) and I am trying to figure out the best way of performing a null hypothesis testing. I looked for similar ...
CommunityBot
- 1
8
votes
2
answers
547
views
Concentration inequality for minimal eigenvalue of sample covariance
I was reading an article of matrix completion and met the following lemma
The concentration inequality for $\sigma_{\max}$ part is a standard result. However, I didn't find any results like the $\...
CommunityBot
- 1
4
votes
1
answer
1k
views
Quantile convergence
Let $X^1,\dots,X^n$ be a sample of (not necessarily iid) random variables and denote
$$F^n(x)=\frac{1}{n}\sum_{i=1}^n \mathbf 1_{X^i\leq x}$$
the empirical distribution function. Suppose that we know ...
CommunityBot
- 1
4
votes
2
answers
519
views
Cramér–Rao type bound for absolute estimation error
Let $\{X_1, X_2, \dotsc, X_n\}$ be independent and identically distributed (i.i.d.) random variables sampled from a common distribution with density $f_{\theta}(x)$, where $\theta$ is an unknown ...
- 12.9k
2
votes
1
answer
208
views
Proving an exponential sum inequality for symmetric Hamming distance sequences in binary vectors
Background: Let $X = \{0,1\}^k$ represent the set of all binary vectors of length $k$. For two binary vectors $x, y \in X$, the Hamming distance $d_H(x, y)$ is defined as the number of positions where ...
- 349
4
votes
1
answer
287
views
Local maxima of the sum of Gaussian functions in *multiple dimensions* are always strict local maxima - prove/disprove/prove conditionally?
This is a follow up of the question in one dimension, that asked to show that the all the maxima of the sum of Gaussian
$$f_n(x):= \sum_{i=1}^{n}e^{-(x-x_i)^2}, x_1 < x_2 < \dots < x_n$$
are ...
CommunityBot
- 1
1
vote
1
answer
415
views
Approximate the singular values of a certain random dot-product kernel matrix (in the sense of El Karoui, Cheng-Singer, etc.)
Let $g:\mathbb R \to \mathbb R $ be a continuous function which is
"sufficiently smooth" (e.g $\mathcal C^3$) around $0$, and
"sufficiently integrable" (e.g integrable w.r.t $N(0,...
CommunityBot
- 1
1
vote
1
answer
241
views
Expectation of top-K selection of squared Gaussian random variables
Let us have
$$
Z = [z_1, z_2, \dots, z_n],
$$ where $z_i \sim N(0, \sigma^2)$ and are iid. Additionally, consider
$$
X_k := \{ x \in \{0, 1\}^n : e^T x = k \}
$$ If $Y = \max_{X \in X_k} |Z^T X|^2,$ ...
CommunityBot
- 1
0
votes
1
answer
108
views
RMT for modified Wishard matrix $Y'Y$ (where $i$th row of $Y$ is zero if $|x_i^\top u| \le \theta$; else it equals $x_i$)
Let $n$ and $d$ be positive integers tending to infinity such that $d/n \to \phi \in (0,\infty)$. Let $X$ be an $n \times d$ random matrix with iid rows $x_1,\ldots,x_n$ from $N(0, \Sigma)$, where $\...
CommunityBot
- 1
3
votes
1
answer
607
views
Show that $\sup_{\|g\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n g(Z_i)\right|\rightarrow_{\text{a.s.}}0.$ when $\delta_n\rightarrow 0$?
UPDATE: The result below can be understood as an almost sure stochastic equicontinuity condition. I don't know of any result establishing primitives of almost sure stochastic equicontinuity. If you ...
CommunityBot
- 1
16
votes
1
answer
397
views
Examples of problems in statistics accessible only using information geometry
I am just curious if there are some examples of problems in statistics that are indeed accessible using information geometry while proofs completely avoiding geometry are unknown. In other words, ...
- 6,001
4
votes
1
answer
332
views
Asymptotic limit of trace of random matrix $(aI_m + WW^\top)^{-1}$, where $W$ has iid rows from $N(0,\Sigma)$
Let $m$ and $d$ be positive integers with $m,d \to \infty$ such that $m/d \to \rho \in (0,\infty)$. Let $W$ be a random $m \times d$ matrix with iid rows $w_1,\ldots,w_m \sim N(0,\Sigma)$ for a ...
CommunityBot
- 1
22
votes
1
answer
1k
views
Random distance matrices
My question is motivated by the following recent paper:
Gadgil, Siddhartha; Krishnapur, Manjunath, Lipschitz correspondence between metric measure spaces and random distance matrices, Int. Math. Res. ...
- 6,367
0
votes
0
answers
44
views
Large Deviation Principle for an adaptive sampling rule for Multi Armed Bandits
Consider the following adaptive strategy for sampling from a Multi Armed Bandit with $K$ arms:
Split the $T$ rounds into $N (\in \mathbb{N})$ disjoint intervals. Each interval is indexed by $i=1,2,\...
- 9
1
vote
0
answers
48
views
Quantile maximization of the difference of random constrained quadratic optimization problems
I am interested in understanding the family of parametrized random variables defined by the pushforward map
$$
\lambda_x : \varepsilon \mapsto \underset{z_1 \in \mathbb{R}^n :\, h^T z_1 = 0, \; z_1 \...
- 11
0
votes
1
answer
100
views
Expressing a multivariate normal distribution as a mixture of uniform distributions?
Context: Given a scalar normal distribution $X\sim \mathrm{N}(\mu, \sigma^2)$, it is possible to express $X$ as a mixture of uniform distributions over intervals (compound probability distributions), ...
- 63.9k
2
votes
0
answers
77
views
Inequalities concerning cummulative distributions of binomials
For random variable $Z$, let $F_Z$ denote its cdf, i.e., $F_Z(t)=\mathbb{P}(Z\leq t)$. Let $X$ be a binomial distribution with parameters $(n,p)$ and $Y$ a binomial distribution with parameters $(m,p)$...
- 121
1
vote
0
answers
61
views
Bound on $\int_0^1\sqrt{\log N_{[]}(\varepsilon,\mathcal{F},d)} \, d\varepsilon$ over the class of half-spaces $\mathcal{F}$ on $\mathbb{R}^d$?
For a class of functions $\mathcal{F}$ and a pair $f,g\in\mathcal{F}$ with $f\leq g$, the interval $[f,g]=\{h:f(x)\leq h(x)\leq g(x),\forall x\in\mathbb{R}^d\}$ is called a bracket for $\mathcal{F}$. ...
3
votes
1
answer
379
views
Concentration inequality for norm of solution to nonlinear least-squares problem
Define the piecewise-linear function $\psi(t):=\max(t,0)$ for all $t \in \mathbb R$.
Let $d,n,k \to \infty$ at the same rate (i.e $n \asymp k \asymp d$).
Let $y_1,\ldots,y_n \in \{-1,1\}$ uniformly ...
CommunityBot
- 1
8
votes
3
answers
8k
views
Upper bound total variation by Wasserstein distance for continuous distance
I am reading the survey of the relationships between metrics of distributions (see https://arxiv.org/pdf/math/0209021.pdf for the paper).
The general results show that for general distributions, we ...
- 211
1
vote
1
answer
56
views
How to study the convergence of the sample mode for arbitrary probability spaces
(This is not the problem I actually care about, but an analogy with similar issues to the problem I'm actually considering.)
Consider a probability space with i.i.d. random variables $X_i$ producing ...
- 128k
0
votes
1
answer
101
views
Realizations of alternative configurations
Consider a discrete distribution $P(\mathbf{X},Y)$ with $\mathbf X = \{ X_1, \dotsc, X_N \}$. I use the shorthand notation $p(\mathbf{x}, y)$ for $P(\mathbf{X}=\mathbf{x}, Y=y)$. Consider $P_\text{ind}...
CommunityBot
- 1
2
votes
1
answer
245
views
Sum of arrival times of Chinese Restaurant Process (CRP)
Suppose that a random sample $X_1, X_2, \ldots$ is drawn from a continuous spectrum of colors, or species, following a Chinese Restaurant Process distribution with parameter $|\alpha|$ (or ...
CommunityBot
- 1
6
votes
2
answers
607
views
Whence “uniform distribution”?
The “Earliest Uses” site suggests that the expression “uniform distribution” first appeared in Uspensky (1937), and “uniformly distributed” in Sakamoto (1943). Is that true?
- 37.7k
1
vote
1
answer
153
views
Minimax estimation rate of sparse vector $w_\star$, w.r.t to mixed norm $\|\hat w_n-w_\star\| := \|\hat w_n - w_\star\|_2 + \|\hat w_n-w_\star\|_q$
Let $n,d,s$ be positive integers with $s \le d$, and let $B_0(d,s)$ be the set of all (real) $d$-dimensional vectors with at most $s$ nonzero components. Given an $n \times d$ matrix $X$ with rows $...
CommunityBot
- 1
0
votes
1
answer
113
views
Inequality on conditional variance of a vector
I have a random vector $X$ and an event $\mathcal{E}$ such that $\mathbb{P}(\mathcal{E}) = p$. I am trying to show the following inequality :
\begin{equation}
p\mathbb{E}[\|X - \mathbb E [X \vert \...
CommunityBot
- 1
1
vote
1
answer
69
views
Expected value of MGIG distribution
I'm currently dealing with a Gibbs sampler of the multivariate generalized inverse Gaussian distribution (MGIG). In order to check the correctness of the sampler, I'd like to know the expected value ...
CommunityBot
- 1
4
votes
1
answer
223
views
Existence of disintegrations for improper priors on locally-compact groups
In wide generality, the disintegration theorem says that Radon probability measures admit disintegrations. I'm trying to understand the case when we weaken this to infinite measures, specifically ...
CommunityBot
- 1
1
vote
0
answers
44
views
Constrained random sampling from partitioned sets with quotas
Let $D$ be a finite set, $\mathcal{P} = \{D_{i,j}\}_{(i,j) \in I \times J}$ a partition of $D$, $N: J \to \mathbb{N}$ a quota function, and $k \in \mathbb{N}^+$. A subset $F \subseteq D$ is considered ...
- 63.9k
1
vote
1
answer
245
views
expectation of the function of Wishart matrix eigenvalues
For Given a $N×M$ random complex gaussian matrix $X$ where $M=XX^H$, let $\lambda_1>\lambda_2>\cdots>\lambda_N$ be the ordered eigenvalues of $M$ my objective is to get an estimation of
$$
f =...
1
vote
0
answers
80
views
Moments from characteristic function for matrices
When $x$ is a random variable with the smooth characteristic function $\phi_x(t) = \mathbb{E}e^{itx}$, we can easily compute the moments as $\mathbb{E}[x^k] = i^{-n}\phi_x^{(n)}(0)$. There is no magic ...
CommunityBot
- 1
2
votes
0
answers
56
views
Sum of independent Wisharts
Suppose random vectors $y_1,y_2,\ldots,y_m$ are independent and the distribution of each $y_i$ is a $d$-dimensional complex Gaussian with mean $0$ and covariance $\Gamma_i$, that is $y_i \sim \mathcal{...
3
votes
0
answers
353
views
Moments of normalized multivariate Gaussians (and Wick's/Isserlis theorems)
Suppose $x = \begin{bmatrix}x_1 \\ x_2\end{bmatrix}$ is distributed according to the real two-dimensional Gaussian with mean-$0$ and covariance matrix $\Sigma$. I am interested in a closed form for ...
- 193
3
votes
0
answers
131
views
Matrix-Gaussian distributions
The point of this question is to ask for references on matrix-variate Gaussian distributions. But I will explain what I mean by a matrix-variate Gaussian with an example (the notion I have in mind is ...
- 193
5
votes
1
answer
392
views
comparing Gaussian to order statistic of Gaussian
I would like to compute the probability of
$$\mathbb{P}[Y > \max(X_i)], Y\sim N(0, 1), X_i \sim N(0, \sigma_i)$$
All the random variables have zero mean, but the variances are different.
My ...
-2
votes
1
answer
43
views
$E(\mathbf{y}|\mathbf{x}+\mathbf{z})=g(\mathbf{x})$ almost surely, if $\mathbf{z}\perp \!\!\! \perp \{\mathbf{y},\mathbf{x}\}$ jointly?
Let $\mathbf{y},\mathbf{x}$ and $\mathbf{z}$ be real-valued random vectors with possibly different dimensions.
If $\mathbf{z} \perp\!\!\!\perp \{\mathbf{y},\mathbf{x}\}$ (i.e., $\mathbf{z}$ is ...
- 1,724
2
votes
1
answer
177
views
Optimization over Poisson-binomial distributions
I am studying the problem of how an expected utility maximizer should optimally form a portfolio of uncorrelated Bernoullis.
Fix an increasing sequence of $n$ numbers in $(0,1)$, $0<p_1<\dots<...
- 266
1
vote
1
answer
81
views
Inference for the normal distribution with known variance from multiple clusters
Here's the question:
We have: $q \sim N\left(q_p, \frac{1}{\tau}\right), q_i \sim N\left(q, \frac{1}{\zeta}\right), t_n \sim N\left(0, \frac{1}{\eta}\right)$. Let $$ r_n=\sum_{i=1}^{\theta k_{n}} \...
- 25.8k
0
votes
0
answers
85
views
When is a family of distributions "closed" with respect to minimal sufficient statistics?
As in the title, I am interested in understanding how to express the idea that a parametric family of distribution is "closed" with respect to minimal sufficient statistics. Before giving ...
3
votes
0
answers
80
views
Seeking strong bounds on KL-divergence and martingales for a hypothesis-testing inequality
Let's say we have a finite set $\mathcal{O}$ of observations, and let $\mathcal{C}(\Delta\mathcal{O})$ denote the space of closed convex sets of probability distributions.
We have two hypotheses which ...
- 353