All Questions
Tagged with pr.probability st.statistics
1,135 questions
3
votes
3
answers
203
views
$\mathbf{y}=f(\mathbf{x},\mathbf{z})=g(\mathbf{x})$ if $\mathbf{z}\perp \!\!\! \perp \{\mathbf{y},\mathbf{x}\}$ jointly?
Let $\mathbf{y},\mathbf{x},\mathbf{z}$ are real-valued random vectors with possibly different dimensions. Assume $\mathbf{y}=f(\mathbf{x},\mathbf{z})$ for some function $f$.
If $\mathbf{z} \perp\!\!\!\...
- 193
0
votes
1
answer
101
views
Realizations of alternative configurations
Consider a discrete distribution $P(\mathbf{X},Y)$ with $\mathbf X = \{ X_1, \dotsc, X_N \}$. I use the shorthand notation $p(\mathbf{x}, y)$ for $P(\mathbf{X}=\mathbf{x}, Y=y)$. Consider $P_\text{ind}...
- 189
0
votes
1
answer
61
views
What can we say about the order of convergence of a critical point of Gaussian mixture density to its limit when the parameter $h$ goes to $0?$
Density of Gaussian mixture with $n$ components is given by:
$$f(x):=C \sum_{i=1}^{n}e^{-\frac{1}{2}||\frac{x-x_i}{h}||^2}, x_i \in \mathbb{R}^d, h > 0$$
where $C$ is a normalization constant ...
- 1,467
3
votes
1
answer
243
views
Independent input feature z can be removed: if y=f(x+z,z), then y=g(x)?
Let $y\in \mathbb{R}$ and $\mathbf{x},\mathbf{z}\in\mathbb{R}^p$ be random variable and random vectors. Assume $y=f(\mathbf{x}+\mathbf{z},\mathbf{z})$ for some function $f$.
Is the following statement ...
- 193
3
votes
1
answer
607
views
Show that $\sup_{\|g\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n g(Z_i)\right|\rightarrow_{\text{a.s.}}0.$ when $\delta_n\rightarrow 0$?
UPDATE: The result below can be understood as an almost sure stochastic equicontinuity condition. I don't know of any result establishing primitives of almost sure stochastic equicontinuity. If you ...
- 59
0
votes
0
answers
49
views
Gaussian white noise model in application
I am interested in applications (to data) of non-parametric statistics, and my question concerned the Gaussian white noise model defined by,
$$
X_{t_1, \ldots, t_d}=f\left(t_1, \ldots, t_d\right) d ...
- 73
3
votes
0
answers
93
views
Asymptotic approximation of Fisher information matrix for small Gaussian perturbation
Let
$$
X=Y/a+b+\epsilon Z,
$$
where $Y\sim\operatorname{Poisson}(\lambda)$ and $Z\sim\mathcal N(0,1)$ are independent. Also define $\theta=(\lambda,a,b,\epsilon)$. The Fisher information matrix
$$
...
32
votes
3
answers
12k
views
What is the Katz-Sarnak philosophy?
It has been recently mentioned by a speaker (his talk is completely not relevant to random matrix theory/RMT though) that modern statistics, especially random matrices theory, will help solving some ...
- 8,071
3
votes
1
answer
257
views
Triangle equality for cosine similarity in high dimensions
I'm trying to understand whether I can use the following equality in my application -- for $u,v,w \in \mathbb{R}^d$:
$$\cos(u,w)\approx \cos(u,v)\cos(v,w)$$
Where $\cos(x,y)$ gives cosine of the angle ...
- 2,252
0
votes
0
answers
60
views
Norms of Wigner matrices under power law decay
Suppose $\Sigma=\operatorname{diag}(h)$ where $h=(1^{-p},2^{-p},3^{-p},\ldots,d^{-p})$ and $p> 1$
$X$ is a matrix with $b$ rows sampled independently from $\operatorname{Normal}(0,\Sigma)$
Suppose $...
- 2,252
3
votes
2
answers
169
views
On finding an upper bound on the error of a sparse approximation
I posted this question on math.stackexchange earlier, but didn't see any response. So, I am posting it here, in case someone else has an answer.
Original question: https://math.stackexchange.com/...
- 65
2
votes
1
answer
1k
views
Bootstrapping and the central limit theorem
I have been looking into bootstrapping lately and although I believe to have understood the basic process somewhat, I am fuzzy on the mathematical details. I will begin with my understanding of what ...
- 149
0
votes
1
answer
161
views
Analogues of Kac-Bernstein characterisation theorem for non-normal distributions
Let $X,Y$ be two independent random variables.
The Kac-Bernstein theorem states that if $X+Y,X-Y$ are also independent, then $X,Y$ are Normal.
Are there analogues of this theorem for non-normal, ...
- 1,459
4
votes
2
answers
1k
views
Expectation of the trace of inverse of a Gaussian random matrix
Given a $N×M$ random complex gaussian matrix $X$ and $N×K$ random complex gaussian matrix $Y$ I'm interested in approximating the expectation expressed as:
\begin{align}
E[trace({(aX{X^H} + I)^{ - ...
- 377
7
votes
1
answer
347
views
Expectation for game choosing uniformly number in $[0,1]$ until it decreases
We are playing a game where we keep on choosing a number from the uniform distribution U(0,1). The game goes on until we have the current number less than the previously picked number, i.e. the game ...
0
votes
0
answers
78
views
Kernel density estimation is sub-gaussian
Let $X_1, ..., X_n$ be i.i.d. samples drawn from a pdf $f(x)$ on the real line. The kernel density estimator is defined as follows,
$$\hat{f_n}(x) = \frac{1}{nh}\sum_1^n K(\frac{x-X_k}{h})$$
where $K:\...
- 81
0
votes
1
answer
552
views
Hypothesis testing for not identically distributed random variables conditioned on the outcome of a subset
I encountered the following problem (I give more details of the problem at the end of the post) and I am trying to figure out the best way of performing a null hypothesis testing. I looked for similar ...
- 9
3
votes
1
answer
210
views
Probabilistic Taylor theorem for concave functions
This paper proves a probabilistic version of Taylor's theorem
\begin{equation*}
\mathbb{E}g(X) = \sum_{k=0}^{n-1} \frac{g^{(k)}(0)}{k!} \mathbb{E}X^k + \frac{\mathbb{E}X^n}{n!} \mathbb{E} g^{(n)}(X_{(...
- 45
3
votes
1
answer
206
views
Asymptotic results for smallest gap of Gaussian random matrix
For a symmetric Gaussian random matrix $G=\{G\}_{1\le i,j \le n}$ with iid $E[G_{ij}]=0$ and $E[G_{ij}^2]=1/n$ (it is normalized), ordering its eigenvalues $\lambda_1\le \lambda_2\le\cdots \lambda_n$.
...
- 288
2
votes
0
answers
87
views
A complex problem involving densities (likelihood functions) and optimization
Consider the following autoregressive process with normal errors:
\begin{equation}\label{7YlUV4i8nuO}\tag{I}
y_t = \phi y_{t-1}+ u_t, \quad u_t \overset{iid}{\sim} N(0,\sigma^2)
\end{equation}
We ...
- 13
1
vote
1
answer
87
views
Is the main part of certain exponential family sub-Gaussian?
$X$ is in the form of exponential family i.e.
$$\mathbb{P_\theta}x = h(x)e^{\langle \theta,T(x)\rangle-\phi(\theta)}$$
where $\theta\in \mathbb{R}^d$. If $\nabla\phi(\theta)$ is L-Lipschitz i.e.
$$\...
- 81
2
votes
3
answers
1k
views
How can I prove Chebyshev's sum inequality with probabilistic methods?
I would like to prove Chebyshev's sum inequality, which states that:
If $a_1\geq a_2\geq \cdots \geq a_n$ and $b_1\geq b_2\geq \cdots \geq b_n$, then
$$
\frac{1}{n}\sum_{k=1}^n a_kb_k\geq \left(\frac{...
- 39
1
vote
1
answer
93
views
An inequality relating $\ell_1$ distance of input and output of a Markov krnel
Let $K$ be a Markov kernel from $\mathcal{X}$ to $\mathcal{Y}$, i.e., $K(\cdot|x)$ is a probability measure on $\mathcal{Y}$ for all $x\in \mathcal{X}$.
Let $\mu$ and $\nu$ be two probability measures ...
- 1,109
3
votes
2
answers
348
views
General version of $d$-separation
I find the $d$-separation criterion (see, e.g., Theorem 2 here; note however the preceding definition, which basically means we are treating discrete random variables) a really useful sufficient ...
- 1,095
3
votes
1
answer
534
views
What is a tensor product of random variables?
I am trying to understand the the following paper https://arxiv.org/pdf/1810.10971.pdf, in particular Example 2:
If $ Y \sim N(0,1)$, the standard normal on $\mathbb{R}$, then
$ \begin{align*} \Big( \...
- 53
1
vote
1
answer
245
views
expectation of the function of Wishart matrix eigenvalues
For Given a $N×M$ random complex gaussian matrix $X$ where $M=XX^H$, let $\lambda_1>\lambda_2>\cdots>\lambda_N$ be the ordered eigenvalues of $M$ my objective is to get an estimation of
$$
f =...
- 377
0
votes
1
answer
59
views
Given positive $\epsilon$ and $c$, find a density $\phi$ such that $t\phi(\epsilon/t) \ge c \|\phi'\|_\infty$ for all positive $t$
A nice density (on $\mathbb R$) is function $\phi:\mathbb R \to \mathbb R$ such that
(1) $\phi(x) \ge 0$ for all $x \in \mathbb R$,
(2) $\int_{-\infty}^\infty \phi(x) \mathrm{d}x = 1$,
(3) $\phi$ is ...
- 6,853
8
votes
2
answers
550
views
Concentration inequality for minimal eigenvalue of sample covariance
I was reading an article of matrix completion and met the following lemma
The concentration inequality for $\sigma_{\max}$ part is a standard result. However, I didn't find any results like the $\...
- 323
2
votes
1
answer
256
views
Does taking minimum preserve density monotonicity?
Suppose $X$ and $Y$ are continuous random variables with a joint density function $f_{X,Y}$. Both $X$ and $Y$ are supported on $(0,1)$ and have continuous (can be assumed differentiable) and non-...
- 31
1
vote
0
answers
100
views
$L_1$ convergence rates for multivariate kernel density estimation
Let $X$ be a random variable on $\mathbb R^d$ with probability density function $f$, and let $X_1,\ldots,X_n$ of $X$ be $n$ iid copies of $X$. Given a bandwidth parameter $h=h_n > 0$ and a kernel $...
- 6,853
1
vote
1
answer
200
views
Rademacher complexity for a family of bounded, nondecreasing functions?
Let $\{\phi_k\}_{k=1}^K$ be a family of functions mapping from an interval $[a, b]$ to $[-1, 1]$.
That is, $\phi_k \colon[ a,b] \to [-1, 1]$ are nondecreasing maps on some finite interval $[a, b] \...
- 380
5
votes
1
answer
397
views
comparing Gaussian to order statistic of Gaussian
I would like to compute the probability of
$$\mathbb{P}[Y > \max(X_i)], Y\sim N(0, 1), X_i \sim N(0, \sigma_i)$$
All the random variables have zero mean, but the variances are different.
My ...
- 151
0
votes
1
answer
218
views
Is the unconditional variance of a RV an upper bound for the variance of any conditional expectation of the RV?
Let $X$ and $Y$ be continuous random variables with finite first and second moments. Then, is it true that $Var[X]\geq Var[E(X|Y)]$?
0
votes
1
answer
198
views
Spectral norm of matrices of bounded random variables
Assume $A\in \mathbb{R}^{n\times n}$ with each entry being i.i.d. bounded r.v. in $[a,b]$, is $\Vert A\Vert_2$ is sub-Gaussian?
Intuitively, since $\{A_{ij}\}_{i,j=1,...,n}$ is bounded, then
$$\Vert A ...
- 81
2
votes
1
answer
110
views
Lower bound on likelihood of binary outcomes
I am wondering about the following: does there exist a stochastic process $(X_n)_{n \ge 1}$ with values in $\{0,1\}$ on a probability space $(\Omega, \mathcal F, \mathbb P)$ such that for all $n \ge 1$...
- 341
3
votes
1
answer
615
views
An inequality relating the Kullback-Leibler divergence of two discrete distributions with constant reference distribution
Suppose that $D_{KL}(p_1\parallel q)<1$ and $D_{KL}(p_2\parallel q)<1$. I'm trying to show that either $D_{KL}(p_1\parallel p_2)$ or $D_{KL}(p_2\parallel p_1)$ will have an upper bound close to ...
0
votes
1
answer
116
views
What's the cumulative probability of these particular bags of liquorice allsorts?
After eating a bag of liquorice allsorts in one sitting, as one does, I noticed that it had contained an unusual amount of brown ones (which, you will agree, are an abomination that should never have ...
- 123
7
votes
1
answer
1k
views
reverse KL-divergence: Bregman or not?
I am having a little trouble getting my head around the two "directions" of the Kullback-Leibler divergence:
Definition (Kullback-Leibler divergence) For discrete probability distributions $...
- 101
1
vote
0
answers
423
views
Conditions for equivalence of RKHS norm and $L^2(P)$ norm
Let $K$ be a psd kernel on an abstract space $X$ and let $H_K$ be the induced Reproducing Kernel Hilbert Space (RKHS). Let $P$ be a probability measure on $X$ such that $H_K \subseteq L^2(P_X)$ and ...
- 6,853
5
votes
1
answer
357
views
Bounding the sensitivity of a posterior mean to changes in a single data point
There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...
- 1,068
1
vote
1
answer
141
views
How to get the estimator?
They introduce a new correlation. For $\pi\in \Pi(\mu,\nu)$ the set of coupling of two probability measures $\mu$ and $\nu$ on a Polish space $(X,d)$. The author introduces a plugin estimator.
...
- 288
15
votes
2
answers
5k
views
What areas of algebra could be interesting to probability theorists?
I would like to find some topic of algebra (beyond linear algebra; algebraic number theory is fine) that would be interesting both to a student that wants to specialize in probability theory and to me ...
- 16.9k
11
votes
3
answers
3k
views
Distance between distributions and distance of moments
Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{$\ast$}$$ where $X$ is a random variable of ...
- 623
1
vote
2
answers
317
views
Central limit theorem of random vectors when the dimension is increasing
This is a question about central limit theorems when the dimension is increasing. Suppose now I have a random vector $X_N = (X_{N1}, \cdots, X_{Np})\in\mathbb{R}^p$. For all $c_p\in\mathbb{R}^p$ with $...
1
vote
1
answer
169
views
How to prove that is a consistent estimator?
Let $\hat{\pi}^N$ be an AW-consistent estimator of $\pi$ (i.e., $\hat{\pi}^N$ is a strongly consistent estimator of $\pi$ under adapted (or called nested) Wasserstein distance $AW(\pi, \hat{\pi}^N)\to ...
- 288
3
votes
1
answer
111
views
Distribution/moments of transformed normally distributed random vector
Let $\varepsilon \sim N\left ( 0,I_{k} \right )$, consider the following function of $\varepsilon$,
$y=\left ( A+B\varepsilon \varepsilon {}'B{}' \right )^{^{\frac{1}{2}}}\varepsilon $,
where $A$ is a ...
- 31
1
vote
1
answer
223
views
Bound error in approximating $E_x [H(f(x))]$ with random $(1/n) \sum_{i=1}^n \Phi(f(x_i)/h)$ where $H$ is Heaviside function and $\Phi$ is normal CDF
Let $f:\mathbb R^d \to \mathbb R$ be a "sufficiently smooth" function. For simplicity, we may consider $f$ to be an affine function, i.e $f(x) \equiv b-x^\top w$, for some $(w,b) \in \mathbb ...
- 6,853
2
votes
1
answer
304
views
An approximation problem w.r.t marginal distribution of coordinates of uniform random vector on high-dimensional unit-sphere
Let $X=(X_1,\ldots,X_d)$ be uniformly distributed on the sphere of radius $\sqrt{d}$ in $\mathbb R^d$. Fix a "sufficiently integrable" function $h:\mathbb R \to \mathbb R$, and define ...
- 6,853
3
votes
1
answer
848
views
Concentration inequality for the sample covariance matrix
I'd like to know if there is a concentration inequality for the sample covariance matrix that don't assume the knowledge of the true mean.
Background.
Given a probability distribution $\mu$ on $\...
- 903
4
votes
1
answer
276
views
About non-reversible Metropolis Hastings Markov chain
I am reading a paper about constructing a non-reversible Metropolis Hastings Markov chain from a reversible one as described at a high level in paragraph $3$ of page $1$.
But I don't understand how, ...
- 201