All Questions
Tagged with pr.probability st.statistics
1,134 questions
2
votes
0
answers
124
views
Rough path expected signature vs cumulant-generating function / characteristic function
What is the point of using rough path expected signature to characterize the law of а stochastic process when the cumulant generating function is known ($\log\mathbb{E}[e^{i\theta X(t)}]$)?
Since an ...
- 53
3
votes
1
answer
257
views
Triangle equality for cosine similarity in high dimensions
I'm trying to understand whether I can use the following equality in my application -- for $u,v,w \in \mathbb{R}^d$:
$$\cos(u,w)\approx \cos(u,v)\cos(v,w)$$
Where $\cos(x,y)$ gives cosine of the angle ...
- 2,242
1
vote
0
answers
59
views
How to calculate the unifrom entropy or VC dimension of the following class of functions?
When dealing with U process I meet with such a uniform entropy to calculate.
For any $\eta>0$, function class $\mathcal{F}$ containing functions $f=\left(f_{i, j}\right)_{1 \leq i \neq j \leq n}: \...
- 11
3
votes
1
answer
532
views
What is a tensor product of random variables?
I am trying to understand the the following paper https://arxiv.org/pdf/1810.10971.pdf, in particular Example 2:
If $ Y \sim N(0,1)$, the standard normal on $\mathbb{R}$, then
$ \begin{align*} \Big( \...
- 53
4
votes
1
answer
485
views
Expected norms of Wishart matrices
Suppose $x_i \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu,\Sigma)$. What can we say about dependence on $b$ of Frobenius/spectral norm quantities below?
$$f(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^...
- 2,242
4
votes
1
answer
489
views
CLT convergence rate for sum of uniforms (in TV distance)
Suppose $X_1, \cdots, X_n \sim_{\mathrm{iid}} U([-1,1])$, where $U([-1, 1])$ denotes the continuous uniform distribution over the interval $[-1, 1]$ (so $E[X_i] = 0$ and $\text{Var}[X_i]= 1/3$). Let $...
- 43
1
vote
0
answers
83
views
Properties of max of many linear combinations of a multivariate normal vector and/or sum of top $k$ elements of a multivariate normal vector
Thank you in advance for your help!
I am interested in studying the following probability:
$$P\big[\max_{H \subset X,|H|=k} \sum_{i \in H} \mathbf{a}_i^T \mathbf{w} \ge 0 \big],$$
where $\mathbf{a}_i$ ...
- 11
4
votes
2
answers
349
views
Does the average of correlated Gaussian random variables with mean zero and different variances converge in probability to their mean?
Let $X_i\sim N(0,\sigma_i^2)$ and $\operatorname{Corr}(X_i,X_j)>0$. Is it possible to show that $$\frac{1}{N} \sum_{i=1}^N X_i \overset{p}\rightarrow E[X_i]=0.$$ Do you have a reference to a law of ...
3
votes
3
answers
203
views
$\mathbf{y}=f(\mathbf{x},\mathbf{z})=g(\mathbf{x})$ if $\mathbf{z}\perp \!\!\! \perp \{\mathbf{y},\mathbf{x}\}$ jointly?
Let $\mathbf{y},\mathbf{x},\mathbf{z}$ are real-valued random vectors with possibly different dimensions. Assume $\mathbf{y}=f(\mathbf{x},\mathbf{z})$ for some function $f$.
If $\mathbf{z} \perp\!\!\!\...
- 193
0
votes
0
answers
29
views
k-means errors for a block Gaussian vector
Consider a standard centered Gaussian vector $(X_1,...,X_n)$ with an approximate block structure, i.e. there is $q$ and a partition of $\{1,...,n\}$ in $q$ classes such that if $i,j$ are in the same ...
- 1,352
3
votes
1
answer
243
views
Independent input feature z can be removed: if y=f(x+z,z), then y=g(x)?
Let $y\in \mathbb{R}$ and $\mathbf{x},\mathbf{z}\in\mathbb{R}^p$ be random variable and random vectors. Assume $y=f(\mathbf{x}+\mathbf{z},\mathbf{z})$ for some function $f$.
Is the following statement ...
- 193
5
votes
2
answers
2k
views
Relationship between KL, chi-squared, and Hellinger
There are many well-known relationships between the KL divergence, chi-squared ($\chi^2$) divergence, and the Hellinger metric. In the paper "Assouad, Fano, and Le Cam" by Bin Yu, the author ...
- 63
1
vote
1
answer
153
views
Minimax estimation rate of sparse vector $w_\star$, w.r.t to mixed norm $\|\hat w_n-w_\star\| := \|\hat w_n - w_\star\|_2 + \|\hat w_n-w_\star\|_q$
Let $n,d,s$ be positive integers with $s \le d$, and let $B_0(d,s)$ be the set of all (real) $d$-dimensional vectors with at most $s$ nonzero components. Given an $n \times d$ matrix $X$ with rows $...
- 6,853
2
votes
0
answers
138
views
Optimal Monte Carlo Trace Estimator
For a psd real symmetric $d\times d$ matrix $A$ and a function $f: \mathbb{R}^d \to \mathbb{R}$, with $f(x) := x^T A x$ we have that with $p(x) = \mathcal{N}(0_d, I_d)$ (i.e. standard multivariate ...
6
votes
2
answers
344
views
Entropy & difference between max and min values of probability mass
Let $X$ be a random variable with probability mass function $p(x) = \mathbb{P}[X = x]$.
I know entropy $H(X)$ of $X$ measures the uncertainty of $X$ and
a large value of $H(X)$ means $p(x)$ is nearly ...
- 163
3
votes
1
answer
614
views
An inequality relating the Kullback-Leibler divergence of two discrete distributions with constant reference distribution
Suppose that $D_{KL}(p_1\parallel q)<1$ and $D_{KL}(p_2\parallel q)<1$. I'm trying to show that either $D_{KL}(p_1\parallel p_2)$ or $D_{KL}(p_2\parallel p_1)$ will have an upper bound close to ...
0
votes
1
answer
116
views
What's the cumulative probability of these particular bags of liquorice allsorts?
After eating a bag of liquorice allsorts in one sitting, as one does, I noticed that it had contained an unusual amount of brown ones (which, you will agree, are an abomination that should never have ...
- 123
0
votes
0
answers
103
views
Does such probability distribution exist?
I'm hunting for a probability distribution with the following properties:
The support is $(0,\infty)$.
Denote by $F(x)$ the CDF of this distribution.
If $X_1, X_2,...$ are independent random ...
- 9
1
vote
1
answer
251
views
Using Hoeffding inequality for risk / loss function
I've got a question to the Hoeffding Inequality which states, that for data points $X_1, \dots, X_n \in X$, which are i.i.d. according to a probability measure $P$ on $X$, we find an upper bound for:
$...
- 13
1
vote
1
answer
169
views
How to prove that is a consistent estimator?
Let $\hat{\pi}^N$ be an AW-consistent estimator of $\pi$ (i.e., $\hat{\pi}^N$ is a strongly consistent estimator of $\pi$ under adapted (or called nested) Wasserstein distance $AW(\pi, \hat{\pi}^N)\to ...
- 288
1
vote
1
answer
141
views
How to get the estimator?
They introduce a new correlation. For $\pi\in \Pi(\mu,\nu)$ the set of coupling of two probability measures $\mu$ and $\nu$ on a Polish space $(X,d)$. The author introduces a plugin estimator.
...
- 288
0
votes
1
answer
108
views
On the invertibility of $Z^\top Z$, where $Z$ is a Random matrix with concentrated weakly correlated entries
Let $d$, $n$, and $m$ be large positive integers. Let $X=(x_1,\ldots,x_n) \in \mathbb R^{n \times d}$ be a random matrix iid rows from some distribition $P$ on $\mathbb R^d$ which admits a density. ...
- 6,853
2
votes
1
answer
936
views
Upper-bound for spectral norm of the covariance matrix of a certain Gaussian vector with correlated entries
Let $n$ and $m$ be large positive integers. Let $x=(x_1,\ldots,x_n)$ be a vector of independent random variables from $N(0,1)$. It is clear that the covariance matrix of $x$ is $I_n$, the identity ...
- 6,853
3
votes
0
answers
92
views
What dynamical properties should we expect from systems satisfying statistical ones?
Some results on probability theory can be generalized to more abstract ones in ergodic theory, for example:
the strong law of large numbers can be seen as a particular case of Birkhoff's ergodic ...
1
vote
0
answers
417
views
Conditions for equivalence of RKHS norm and $L^2(P)$ norm
Let $K$ be a psd kernel on an abstract space $X$ and let $H_K$ be the induced Reproducing Kernel Hilbert Space (RKHS). Let $P$ be a probability measure on $X$ such that $H_K \subseteq L^2(P_X)$ and ...
- 6,853
3
votes
1
answer
111
views
Distribution/moments of transformed normally distributed random vector
Let $\varepsilon \sim N\left ( 0,I_{k} \right )$, consider the following function of $\varepsilon$,
$y=\left ( A+B\varepsilon \varepsilon {}'B{}' \right )^{^{\frac{1}{2}}}\varepsilon $,
where $A$ is a ...
- 31
4
votes
1
answer
164
views
Limiting value of expectation of trace of exponential of Wishart matrix
Let $X$ be an $n \times d$ random matrix with iid entries from $N(0, 1/d)$. Let $S:=X^\top X/n$, a $d \times d$ Wishart matrix and let $T = e^{S} := \sum_{k=0}^\infty \dfrac{S^k}{k!}$ be its ...
- 6,853
2
votes
1
answer
154
views
Random probability following a log concave distribution of order p
In the article "Concentration of the information in data with
Log-concave distributions" of Bobkov and Madiman, it is written that if $X$ is a positive random variable following a log ...
0
votes
1
answer
83
views
Functional relationship between two quantities
Let $\mu \in \mathbb R^n$ and let $\Sigma$ be a positive-definite matrix of order $n \ge 2$. Fix $t \ge 0$ and define $\alpha(\mu,\Sigma,t) > 0$ by
$$
\alpha(\mu,\Sigma,t) := \sup_{\|w\| = 1}\frac{...
- 6,853
2
votes
1
answer
84
views
Estimation of Lévy measure of ID distribution
Suppose that the positive random variable $X$ is infinitely divisible and supported on $\mathbb R_+$. Due to Lévy-Khintchine, its moment generating function then writes :
$$M(t) = \mathbb E\left(e^{tX}...
- 686
3
votes
2
answers
169
views
On finding an upper bound on the error of a sparse approximation
I posted this question on math.stackexchange earlier, but didn't see any response. So, I am posting it here, in case someone else has an answer.
Original question: https://math.stackexchange.com/...
- 65
1
vote
1
answer
259
views
Non-independent Sub-gaussian variables and concentration
Let $g \in R^{d}$ have $iid$ Gaussian components. Let $a \in R^{d}$, and let $b \in R^{d}$. be arbitrary vectors.
Consider the random variable $Y_{g,g}:= \frac{1}{n}\langle g,a \rangle \langle g, b \...
- 143
2
votes
0
answers
61
views
Approximate logarithmic bound on expected maximum via central limit theorem
If $Z_i$ are standard normal, possibly dependent, one can show that
$$E\left[\max_{i=1,...,M} Z_i^2\right]\leq 3\ln M + 1.$$
I'm looking for a similar (asymptotic) bound for asymptotically normal ...
- 203
0
votes
0
answers
36
views
How to recalculate the weights for an event that happens multiple times and requires all outcumes to be unique?
I think it's easiest to explain with an example.
I have a weighted probability list
A : 0.15
B : 0.15
C : 0.15
D : 0.1
E : 0.1
F : 0.1
G : 0.1
H : 0.075
I : 0.075
...
3
votes
1
answer
271
views
For a random sequence $X_0, X_1, X_2, \ldots$ and $F_n$ the empirical CDF, does $F_n(X_0)$ converge to a uniform random variable?
Let $X_0, X_1, X_2, \ldots$ be a sequence of i.i.d. real-valued random variables on some probability space $(\Omega, \mathcal{F}, \mathbb{P})$ with continuous CDF $F(x)$ and define a sequence of ...
- 277
0
votes
0
answers
97
views
Verification of a certain computation of VC dimension
Disclaimer: I'm not very familiar with the concept of VC dimensions and how to manipulate such objects. I'd be grateful if expects on the subject (learning theory, probability), could kindly proof ...
- 6,853
1
vote
1
answer
216
views
Rademacher complexity of function class $(x,y) \mapsto 1[|yf(x)-\alpha| \ge \beta]$ in terms of $\alpha$, $\beta$, and Rademacher complexity of $F$
Let $X$ be a measurable space and let $P$ be a probability distribution on $X \times \{\pm 1\}$. Let $F$ be a function class on $X$, i.e., a collection of (measurable) functions from $X$ to $\mathbb R$...
- 6,853
0
votes
0
answers
195
views
Upper-bound for bracketing number in terms of VC-dimension
Let $P$ be a probability distribution on a measurable space $\mathcal X$ (e.g;, some euclidean $\mathbb R^m$) and let $F$ be a class of funciton $f:\mathcal X \to \mathbb R$. Given, $f_1,f_2 \in F$, ...
- 6,853
1
vote
1
answer
223
views
Bound error in approximating $E_x [H(f(x))]$ with random $(1/n) \sum_{i=1}^n \Phi(f(x_i)/h)$ where $H$ is Heaviside function and $\Phi$ is normal CDF
Let $f:\mathbb R^d \to \mathbb R$ be a "sufficiently smooth" function. For simplicity, we may consider $f$ to be an affine function, i.e $f(x) \equiv b-x^\top w$, for some $(w,b) \in \mathbb ...
- 6,853
1
vote
0
answers
100
views
$L_1$ convergence rates for multivariate kernel density estimation
Let $X$ be a random variable on $\mathbb R^d$ with probability density function $f$, and let $X_1,\ldots,X_n$ of $X$ be $n$ iid copies of $X$. Given a bandwidth parameter $h=h_n > 0$ and a kernel $...
- 6,853
4
votes
0
answers
164
views
Convergence rates for kernel empirical risk minimization, i.e empirical risk minimization (ERM) with kernel density estimation (KDE)
Let $\Theta$ be an open subset of some $\mathbb R^m$ and let $P$ be a probability distribution on $\mathbb R^d$ with density $f$ in a Sobolev space $W_p^s(\mathbb R^d)$, i.e all derivatives of $f$ ...
- 6,853
2
votes
0
answers
122
views
Consistent approximation of weighted Radon transform of smooth probability density, using kernel density estimation
Let $X$ be a random vector in $\mathbb R^d$, with "sufficiently smooth" probability density function on $\rho$. For unit-vectors $w$ and $u$ in $\mathbb R^d$, and a scalar $b \in \mathbb R$, ...
- 6,853
2
votes
1
answer
256
views
Does taking minimum preserve density monotonicity?
Suppose $X$ and $Y$ are continuous random variables with a joint density function $f_{X,Y}$. Both $X$ and $Y$ are supported on $(0,1)$ and have continuous (can be assumed differentiable) and non-...
- 31
2
votes
1
answer
88
views
Lower bound on the error of proportion estimation
Let $X \sim \operatorname{Bin}(n,p)$. Suppose we estimate $p$ by $\hat{p}=\frac{X}{n}$. By Hoeffding’s inequality
it holds for all $\delta \in (0,1)$ with probability at least $1-\delta$ that, $$\...
- 133
-1
votes
1
answer
297
views
The distribution of the sum of values from a normal and a truncated normal distribution
Using R to extract truncated normal distribution samples and normal distribution samples separately, when they are combined, the image drawn by the hist function is very similar to a normal ...
4
votes
1
answer
276
views
About non-reversible Metropolis Hastings Markov chain
I am reading a paper about constructing a non-reversible Metropolis Hastings Markov chain from a reversible one as described at a high level in paragraph $3$ of page $1$.
But I don't understand how, ...
- 201
4
votes
1
answer
114
views
Consistent empirical estimation of Radon transform of a multivariate density function
Let $P$ be a "nice" distribution on $\mathbb R^m$ (e.g., multivariate Gaussian, etc.), with density $p$. Let $H := \{x \in \mathbb R^m \mid x^\top w = b\}$ be a hyperplane in $\mathbb R^m$ ...
- 6,853
1
vote
2
answers
316
views
Central limit theorem of random vectors when the dimension is increasing
This is a question about central limit theorems when the dimension is increasing. Suppose now I have a random vector $X_N = (X_{N1}, \cdots, X_{Np})\in\mathbb{R}^p$. For all $c_p\in\mathbb{R}^p$ with $...
1
vote
1
answer
386
views
Expected value of a function of normal random variable
Suppose $X\sim \mathcal{N}(0,\sigma^2)$, find the expectation $\mathbb{E}\left[\frac{1}{(1+X^2)^a}\right]$ where $a$ is a fixed positive real number.
Is there an explicit formula for the above ...
- 161
1
vote
1
answer
143
views
Permute a sparse random matrix to resemble a diagonal matrix as much as possible
Say we generate an $N \times N$ sparse random matrix $W$, where each element $W_{ij}$ was independently chosen to be $1$ with probability $p=\frac{a}{N}$, and $0$ with probability $1-p$. We are ...
- 433