All Questions
Tagged with pr.probability st.statistics
317 questions with no upvoted or accepted answers
20
votes
0
answers
3k
views
+200
What does a product of many Gaussian matrices converge to?
Let $A$ be a product of $n$ $d\times d$ matrices with IID standard Gaussian entries and consider the value of $g(x)=x f(x)$ where $f(x)$ is the density of squared singular values of $A/\|A\|$.
Is ...
- 2,252
11
votes
0
answers
225
views
Functional Weak Convergence of Maximum Likelihood Estimator
Let $\hat{\theta}_n$ be the Maximum Likelihood Estimator of parameter $\theta$, where $n$ is the sample size. It is well-known that under sufficient regularity conditions, we have the asymptotic ...
- 87
11
votes
0
answers
536
views
Bounding the probability that a random variable is maximal
Question: Suppose we have $N$ independent random variables $X_1$, $\ldots$, $X_N$ with finite means $\mu_1 \leq \ldots \leq \mu_N$ and variances $\sigma_1^2$, $\ldots$, $\sigma_N^2$.
I am looking ...
- 119
10
votes
0
answers
391
views
Question from an economist: solving a model of traders' behavior with expectations about the future values of the variable they are currently optimizing
Motivation
I am an economist writing a paper for an academic finance journal. My paper is about the behavior of currency traders, who choose the price at which they will sell currency today, based on ...
- 101
9
votes
0
answers
2k
views
Has the Lie group preserving a probability distribution been used in Bayesian statistics?
For a (possibly signed) nondegenerate probability measure $\pi$ on $\{1,\dots,n\}$ define
$$\langle \pi \rangle := \{R \in \operatorname{STO}(n): \pi R = \pi \}.$$
Here $\operatorname{STO}(n)$ denotes ...
- 15.4k
8
votes
2
answers
547
views
Concentration inequality for minimal eigenvalue of sample covariance
I was reading an article of matrix completion and met the following lemma
The concentration inequality for $\sigma_{\max}$ part is a standard result. However, I didn't find any results like the $\...
- 323
8
votes
0
answers
4k
views
Taylor approximation of a function of a random variable
Suppose we have a random variable $X$ and a smooth function $g$. We want to calculate the expectation value $\mathbb{E}(g(X))$. To be able to write down at least an approximate solution, we perform a ...
- 91
7
votes
0
answers
179
views
Can one "smooth over" k-wise independence to get actual independence?
I came across the following toy problem and was curious if there was a simple solution or counterexample. Suppose you have a distribution $p$ on $m$ random variables $X_1, \ldots, X_m$, each with ...
- 91
6
votes
0
answers
137
views
Why wavelet methods are not popular anymore in nonparametric statistics?
Back in my master years, I took a nonparametric statistics class. In this class, a few nonparametric methods were presented, but I remember spending a lot of times on methods based on wavelet ...
- 73
6
votes
0
answers
295
views
Dimension-free sample complexity for estimating Gaussian covariance
(also asked on math.se, with no answers)
Suppose I have $m$ samples drawn from a Gaussian in $\mathbb{R}^n$, and need sample covariance $\Sigma_m$ to be $\epsilon$-close to true covariance $\Sigma$:
$$...
- 2,252
6
votes
0
answers
388
views
Closedness of a set of measures, where conditional marginals are in closed $\varepsilon$-ball w.r.t. Wasserstein distance
Let $(E,d)$ be a bounded polish space (separable, complete metric space satisfying $\sup_{x,y\in E} d(x,y) < \infty$). By $\mathcal{P}(E)$ we denote the space of Borel probability measures on $E$ ...
- 1,095
6
votes
0
answers
554
views
a variation on Hanson-Wright inequality
The classic Hanson-Wright inequality states that for a Gaussian random vector $\mathbf{x}\in\mathbb{R}^n$ distributed as $\mathcal{N}(\mathbf{0},\mathbf{I})$ and $\mathbf{A}\in\mathbb{R}^{n\times n}$ ...
- 859
6
votes
0
answers
578
views
Maximal Correlation versus Correlation Coefficient When one RV is Gaussian
Let a pair of random variables $(X,Y)$ be continuous random variables (i.e., they both have density with respect to Lebesgue measure) with joint distribution $P_{XY}$. The maximal correlation $\rho_m(...
- 1,109
6
votes
0
answers
189
views
Pettis Integrability and Laws of Large Numbers
Let $(\Omega, \mathcal F, \mathbb P)$ be a probability space, and let $V$ be a topological vector space with a dual space that separates points. Let $v_n : \Omega \to V$ be a sequence of Pettis ...
- 8,512
6
votes
0
answers
262
views
Given that a conditional measure is Gaussian, how bad can the original measure be?
Let $X$ and $Y$ be Banach spaces, and let $\varphi : X \to Y$ be a continuous linear map. Suppose that $\mathbb P$ is a probability measure on $X$ which satisfies the continuous disintegration ...
- 8,512
5
votes
1
answer
353
views
Bounding the sensitivity of a posterior mean to changes in a single data point
There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...
- 1,068
5
votes
0
answers
711
views
Concentration inequality for max component of a multivariate Gaussian in the general case
I am looking to bound the variance of the maximum component of a vector distributed multivariate Gaussian in the general case where the Gaussian distribution has arbitrary mean and full covariance ...
- 283
5
votes
0
answers
485
views
Hierarchical Random Walk (also known as Hierarchical Hidden Markov Model)
Let us consider the following hierarchical (recursive) random walk model, which is also known as the hierarchical hidden Markov model in computer science (https://en.wikipedia.org/wiki/...
- 1,127
5
votes
0
answers
136
views
What's the variance in the Six Degrees model?
Recall the six degrees of Kevin Bacon game. You can even play the game at The Oracle of Bacon, and their search works via Breadth First Search.
I interpret the punchline as saying that if I start ...
- 30.3k
5
votes
0
answers
327
views
Inverse moment of the number of inversions of a permutation
Let $\pi$ be a permutation of $\{1,2,...,n\}$. A pair of elements ($\pi_i$,$\pi_j$) is called an inversion if $i$ $>$ $j$ and $\pi_i$ $<$ $\pi_j$. The total number of inversions in $\pi$ is ...
- 51
5
votes
0
answers
154
views
Positive estimator
Suppose that one knows how to generate (independent) random samples $X_1, X_2, \ldots$ distributed as the random varable $X$ with $\mathbb{E}[X]=\mu \in \mathbb{R}$. It is then easy to construct an ...
- 2,133
5
votes
0
answers
1k
views
Multidimensional Berry–Esseen for probability density functions
This is a follow up to this recent question: Berry Esseen type result for probability density functions
There exists a multidimensional version of the usual Berry–Esseen theorem (for cumulative ...
5
votes
0
answers
506
views
Missing mass estimate
Let $S$ be a finite set with probability distribution $P$. Define the random variable $m_i$ to be the "missing mass" after seeing $i$ iid samples from $S$ under $P$. That is, $m_i$ is the total mass ...
- 6,541
5
votes
1
answer
363
views
Inverse marginal property of a collection of $\sigma$-algebras
In my paper "On the inverse best approximation property of systems of subspaces of a Hilbert space"
I introduced the Inverse marginal property (IMP) for a collection of $\sigma$-algebras.
Let $(\...
- 935
4
votes
0
answers
156
views
Known relations between mutual information and covering number?
This is a question about statistical learning theory. Consider a hypothesis class $\mathcal{F}$, parameterized by real vectors $w \in \mathbb{R}^p$. Suppose I have a data distribution $D \sim \mu$ and ...
- 141
4
votes
0
answers
143
views
Projection of log-concave distribution on unit sphere surface
Let $\mathbf X : \Omega \to \mathbb R^d$ be a random vector following a zero mean, identity covariance log-concave distribution.
Is there any known upper bound for the probability density function of $...
- 141
4
votes
0
answers
309
views
When is $\prod_{i=0}^\infty (I-x_i x_i^T)=0$ for zero-centered Gaussian $x_i$?
Suppose $x_i\in \mathbb{R}^d$ is sampled IID from $\mathcal{N}(0,H)$. Let $A_i=(I-x_i x_i^T)$ and assume $d$ is large. What are necessary conditions for the following to converge with probability 1?
$...
- 2,252
4
votes
1
answer
223
views
Existence of disintegrations for improper priors on locally-compact groups
In wide generality, the disintegration theorem says that Radon probability measures admit disintegrations. I'm trying to understand the case when we weaken this to infinite measures, specifically ...
- 8,512
4
votes
1
answer
287
views
Local maxima of the sum of Gaussian functions in *multiple dimensions* are always strict local maxima - prove/disprove/prove conditionally?
This is a follow up of the question in one dimension, that asked to show that the all the maxima of the sum of Gaussian
$$f_n(x):= \sum_{i=1}^{n}e^{-(x-x_i)^2}, x_1 < x_2 < \dots < x_n$$
are ...
- 1,467
4
votes
0
answers
164
views
Convergence rates for kernel empirical risk minimization, i.e empirical risk minimization (ERM) with kernel density estimation (KDE)
Let $\Theta$ be an open subset of some $\mathbb R^m$ and let $P$ be a probability distribution on $\mathbb R^d$ with density $f$ in a Sobolev space $W_p^s(\mathbb R^d)$, i.e all derivatives of $f$ ...
- 6,853
4
votes
0
answers
144
views
Exponential families closed under affine transformations
Let $(\Omega,\Sigma,\mu)$ be a probability space and let $\mathcal{M}$ be an exponential family of probability distributions for $\mu$ of the following form: There are $\varphi_1,\dots,\varphi_n:\...
- 5,405
4
votes
0
answers
656
views
Eigenvalues of Matérn covariance function
Recall that Matérn covariance function $C_\nu(d)$ is defined as
$$
C_\nu(d)=\sigma^2\frac{2^{1-\nu}}{\Gamma(\nu)}\left(\sqrt{2\nu}\frac{d}{\rho}\right)^\nu K_\nu\left(\sqrt{2\nu}\frac{d}{\rho}\right), ...
- 397
4
votes
1
answer
332
views
Asymptotic limit of trace of random matrix $(aI_m + WW^\top)^{-1}$, where $W$ has iid rows from $N(0,\Sigma)$
Let $m$ and $d$ be positive integers with $m,d \to \infty$ such that $m/d \to \rho \in (0,\infty)$. Let $W$ be a random $m \times d$ matrix with iid rows $w_1,\ldots,w_m \sim N(0,\Sigma)$ for a ...
- 6,853
4
votes
0
answers
75
views
Marginalization of Wishart distribution
Consider the following Wishart distribution
$$
f({\bf W}) = \frac{ |{\bf W}|^{(n-p-1)/2} \exp\big[-\frac{1}{2}\text{tr}({\bf V}^{-1}{\bf W} ) \big] }{2^{np/2} |{\bf V}| \Gamma_p(\frac{n}{2})} \tag{1}
$...
4
votes
0
answers
638
views
Comparison of concentrations of different $L^p$-norms of (sub) Gaussian distributions
It's well-known that the Euclidean $2$-norm of subgaussian random vectors concentrates in high dimensions, e.g. when $X \sim \mathcal{N}(0,I_n),$ (or in general $X$ is subgaussian with independent co-...
- 1,467
4
votes
0
answers
240
views
What happens in the martingale CLT if I norm by the conditional variance instead?
TLDR: I'm a statistician (bear with me!) trying to use the martingale CLT but I only can estimate the conditional variance instead of the unconditional one. Can I do anything to get a CLT with norming ...
- 91
4
votes
0
answers
156
views
Geometric meaning of the chi-square "measure of association"
In Statistics, there's a standard test of independence of two random variables taking values in finite sets $I,J$. It relies on the computation of $\chi$-square statistics,
$$
\chi^2:=\sum_{(i,j)\in ...
- 8,992
4
votes
0
answers
1k
views
Hoeffding's inequality for random vectors
Let $x_1, \ldots, x_n$ be $n$ i.i.d. samples of a bounded random variable $X \in [a, b]$. We know from the Hoeffding's inequality that :
$$\mathbb{P} \left( \left| \frac{1}{n} \sum_{i=1}^n x_n - \...
- 113
4
votes
0
answers
139
views
Probability of having complete conversion in a box of three different object with interaction rules
Say there a 3 types of Objects A,B,C which randomly interact in pairs to form new objects following the below rules:
$$A + B = AB$$
$$B + C = BC$$
$$C + A = CA$$
$$AB + C = ABC$$
$$BC + A = ABC$$
$$CA ...
4
votes
0
answers
823
views
Total Variation distance of polynomials of Bernoulli R.V.s
Let $X_i, Y_i$ be i.i.d Bernoulli $0/1$ random variables with
$\mathbb{E}[X_i] = p$ and $\mathbb{E}[Y_i] = q$.
Let
\begin{align*}
X &= X_1 X_2 + Χ_2 Χ_3 + \ldots +X_{n-2} X_{n-1}+ X_{n-1} X_n\\...
- 175
4
votes
0
answers
91
views
What is the entropy of binomial decay?
Let's play a game. I start with $N$ indistinguishable tokens, and I wait $T$ turns. Every turn, each token has probability $p$ of disappearing. I want an analytic formula for the entropy of this ...
- 141
4
votes
0
answers
188
views
Distributions over permutation groups $\mathcal{S}_n$
Partly inspired by recent developments in enumeration of pattern avoiding permutations, which is known to be connected with Brownian excursions [Hoffman&Rizzolo]. The exciting milestone is the ...
- 8,071
4
votes
0
answers
141
views
Is there an example that both Berry-Essen bound and DKW bound are attained?
The Berry-Essen bound stated that
$$\sup _{{x\in {\mathbb R}}}\left|\widehat{F_{n}(x)}-\Phi (x)\right|\leq C_{0}\cdot \psi _{0}$$
where $\psi _{0}(n)={\Big (}{\textstyle \sum \limits _{{i=1}}^{n}\...
- 8,071
4
votes
0
answers
147
views
The asymptotic behavior of the ratio between the largest two of $n$ i.i.d. chi-square random variables
My question is about the asymptotic behavior of the ratio between the largest and second largest values of $n$ independent chi-square random variables.
Let $X_1, \ldots, X_n$ be $n$ independent and ...
- 1,127
4
votes
0
answers
73
views
Regularity Conditions for L1 convergence of maximum likelihood estimators
Let $X_1,\ldots, X_n$ be i.i.d. observations from a family of pdf or pmf $\{f_{\theta}: \theta \in \Theta \}$. We know that there are sufficient regularity conditions on the family $\{f_{\theta}: \...
- 41
4
votes
0
answers
76
views
How well does an estimator perform on another dataset?
Suppose $X \sim N(0, \Sigma)$ is a $d$-dimensional Gaussian random vector. And we have $2n$ $i.i.d$ sample $X_1, \ldots, X_{n}, \ldots, X_{2n}$.
Let $\hat{\Sigma}_1 = \frac{1}{n}\sum_{i=1}^nX_i X_i^\...
- 515
4
votes
0
answers
573
views
An inequality involving conditional variance and its connection to information theory
Given absolutely continuous random variables $(X, Y)$ with joint distribution $P_{XY}$, we construct $Z:=\sqrt{\gamma} Y+N_\mathsf{G}$ where $N_\mathsf{G}\sim N(0, 1)$ and is independent of $(X,Y)$ ...
- 1,109
4
votes
0
answers
213
views
Optimization problem involving Multivariate Normal
I use $\phi(t)$ to describe the standard normal distribution density and $\Phi(t)$ as the normal distribution CDF and would like to prove that for all
$n\geq3$, the function:
$$h(\mu_{1},\ldots,\...
- 41
4
votes
1
answer
1k
views
Quantile convergence
Let $X^1,\dots,X^n$ be a sample of (not necessarily iid) random variables and denote
$$F^n(x)=\frac{1}{n}\sum_{i=1}^n \mathbf 1_{X^i\leq x}$$
the empirical distribution function. Suppose that we know ...
- 293
4
votes
0
answers
988
views
Probability distribution function for singular value sum of Gaussian random matrix
Let $\mathbf{X}$ be an $N \times N$ random matrix with IID Gaussian entries. They can be standard normal, but $N$ is not large: that is $N$ $<$ 6, typically. Call its singular value decomposition (...
