Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,848
questions
10
votes
3
answers
2k
views
Random Walks on high dimensional spaces
I've read on a paper that, in the two dimensional case, if you start from the origin and take steps of length one in arbitrary directions (uniformely on the unit sphere $S^1$, not left-right-up-down), ...
3
votes
2
answers
407
views
Multivariate normal concentration
If $X\sim N(0,\Sigma)$ for some $d$-dimensional normal distribution, then $X = \Sigma^{1/2} Z$ where $Z\sim (0,I)$. How to compute the following quantity?
$$
\operatorname{var} (X^T X)
=
\...
0
votes
1
answer
117
views
CDF of a RV that is the ratio between a complex Gaussian and a Chi-squared RVs
Given the following p.d.f., which is the p.d.f. of the real and imaginary parts of a random variable that is the ratio between a complex Gaussian and a Chi-squared RVs:
\begin{equation*}
f_U(u)=\exp\...
1
vote
1
answer
138
views
Minimization Proof of Conditioning on Gaussian is Gaussian
It is well known that $E[X|X+Y]$ is Gaussian if both $X$ and $Y$ are, and the result can be derived using standard density arguments. However, how can one prove it by only resulting to optimization ...
2
votes
2
answers
143
views
Expectation of $\left| \frac{\textbf{x}^{H} \textbf{y} }{\| \textbf{x} \|^2} \right|^2$, where $\textbf{x}$ and $\textbf{y}$ are complex Gaussians?
Given that following two random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{x}^{2}\textbf{I}_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{y}^{2}\textbf{I}_{M})$ ...
8
votes
2
answers
1k
views
Does Multiplicative Version of Azuma's Inequality Hold?
It is known that there are multiplicative version concentration inequalities for
sums of independent random variables. For example, the following
multiplicative version Chernoff bound.
Chernoff bound:...
2
votes
1
answer
97
views
p.d.f. of $\left| \frac{\textbf{x}^{H} \textbf{y} }{\| \textbf{x} \|^2} \right|^2$, where $\textbf{x}$ and $\textbf{y}$ are complex Gaussians?
Given that the random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{x}^{2}\textbf{I}_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{y}^{2}\textbf{I}_{M})$ are ...
10
votes
3
answers
400
views
disconnected or poorly connected graphs in sport ratings systems
I've briefly read about rating systems that provide rankings to players based only on their performance wrt other players, in the context of chess. (for example, elo). When there is a lot of ...
1
vote
1
answer
112
views
Approximate $\log \mathbb E_P[\exp(th(x)]$ for a function $h$ which is lipschitz and has finite moments of order 1 and 2 w.r.t $P$
Let $P$ be a probability measure on a space $\mathcal X$ and $h: \mathcal X \rightarrow \mathbb R$ is measurable function with finite moments of order 1 and 2. I'm interested in approximating the ...
1
vote
0
answers
81
views
Central limit theorem is expected, but variance is apparently sublinear? So?
In Perplexing Problems in Probability, the following statement about first passage percolation (FPP) on $\mathbb{Z}^{d}$ is made. See e.g. this paper of Benjamini, Kalai and Schramm, where they quote ...
1
vote
2
answers
269
views
Closed expression for $\mathbb{E} \left\lbrace \Re \frac{(\textbf{x} + \textbf{y})^{H}\textbf{x}}{\| \textbf{x} + \textbf{y}\|^{2}} \right\rbrace$?
Given the random variables $\textbf{x} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{x}^{2}\textbf{I}_{M})$ and $\textbf{y} \sim \mathcal{CN}(\textbf{0}_{M},\sigma_{y}^{2}\textbf{I}_{M})$ are independent, ...
-3
votes
2
answers
439
views
Expected values of two random variables related to a simple urn problem
In an urn there are $u$ balls, $b$ of which are black.
If we perform $n$ trials of one ball at a time with replacement, the probability of the event $E$ to get $n$ times a black ball is $P(E)=\left(\...
1
vote
0
answers
128
views
Inverse Wishart
Assume that $X\sim \operatorname{IW}_p(n,\Sigma)$ has an inverse Wishart distribution, which probability density function is
$$f(X\mid n,\Sigma)=C(n,\Sigma)|X|^{-\frac{n+p+1}{2}} \exp\Big(-\frac{1}{2}...
4
votes
1
answer
175
views
Inner product of sorted Gaussian vector
Suppose $X_1,\ldots,X_n$ are i.i.d. standard normal. I'm wondering how to analyze the following quantity:
$$\left|\frac{X_{(1)}X_{(n)}+X_{(2)}X_{(n-1)}+\cdots+X_{(n)}X_{(1)}}{n}\right|$$
where $X_{(1)}...
2
votes
1
answer
600
views
Distribution of ratio between complex Gaussian and Chi-square R.V.s
What would be the distribution (p.d.f.) of the following ratio?
$$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
1
vote
1
answer
125
views
Expected norm of linear maps
I want to compute the expected norm of a vector-matrix multiplication. I have a vector $x \in \mathbb{R}^n$ with norm one and a matrix $M \in \mathbb{R}^{n \times n}$, whose entries are iid taken from ...
4
votes
1
answer
284
views
How sensitive are probability distributions to noise?
I'm trying to prove a result but I'm stuck at the very end of it: I'm having troubles understanding how noise propagates when considering a probability distribution. In other words, if I inject some ...
0
votes
0
answers
103
views
Expectation of maximal Wasserstein distance between empirical distribution and a pdf
Let $P$ be a continuous probability distribution on $R^d$, $X$ the random variable $\sim P$, and $
\hat{X}$ be n i.i.d samples drawn according to $P$. We have another variable $\mu \in S^{d-1}$.
Do ...
0
votes
0
answers
56
views
Absolute continuity of probability measures determined by dependence structure
We are on $\mathbb{R}^d$ with Borel $\sigma$-algebra. Let $\mu_1, ..., \mu_d$ be probability measures on $\mathbb{R}$ and $\Pi(\mu_1, \mu_2, ..., \mu_d)$ be the set of probability measures on $\mathbb{...
2
votes
1
answer
1k
views
Mutual information between continuous and discrete variables from numerical data
I am looking for references/measures to estimate the mutual information between a continuous (C) and discrete (D) variable, given a real-world (i.e. finite sample) data set. C is uniformly distributed ...
3
votes
1
answer
388
views
How to simulate random paths of a non-homogeneous continuous-time Markov process with discrete state space for a given infinitesimal generator matrix?
Let $X=(X_{t},\,t \in T)$ be a non-homogeneous, continuous time Markov process with a finite state space $S=\{1,...,K\}$.
Let $\alpha_{i,j}(t)$ be the hazard rates of some $\varGamma$-distributed ...
2
votes
1
answer
212
views
Are Linear Maps resistant to Noise?
Let's assume I have a $m \times m$ matrix $M$ with Frobenius norm $1$ and a unit vector $x \in S^{m-1}$. I also have a second $m \times m$ matrix $M^*$ which is obtained from the first one plus some ...
7
votes
1
answer
213
views
Relation between the two possible KL divergences of two distributions
Given that I know $$D\left(P\parallel Q\right)<\alpha,$$ can I say anything about $D\left(Q\parallel P\right)$ in terms of an upper bound on it?
Also, given this upper bound on $D\left(P\parallel ...
1
vote
2
answers
187
views
Approximate the variance of multiple normal distributions with the same standard deviation
Given a number of normal distributions $N(\mu_1, \sigma^2), N(\mu_2, \sigma^2), ..., N(\mu_n, \sigma^2)$ with fixed variance $\sigma^2$, but not necessary equal means. My question is how to ...
1
vote
0
answers
148
views
Hypergraph partitioning and bipartite graph partitioning
Are hypergraph partitioning, and bipartite graph partitioning related, or equivalent, given that hypergraphs can be represented as bipartite graphs?
In the first case, we want to partition the set of ...
17
votes
4
answers
2k
views
Good introduction to statistics from a algebraic point of view?
There are already lots of questions on this subject like
Is there an introduction to probability theory from a structuralist/categorical perspective?
Is there a combinatorial/topological treatment ...
0
votes
1
answer
214
views
Anti-concentration: upper bound for $P(\sup_{a \in \mathbb S_{n-1}}\sum_{i=1}^na_i^2Z_i^2 \ge \epsilon)$
Let $\mathbb S_{n-1}$ be the unit sphere in $\mathbb R^n$ and $z_1,\ldots,z_n$ be a i.i.d sample from $\mathcal N(0, 1)$.
Question
Given $\epsilon > 0$ (may be assumed to be very small), what is ...
0
votes
0
answers
255
views
Does the linear combination of the quantile $\alpha F^{-1}(\tau)+\beta G^{-1}(\tau)$ still a quantile
$F(x)$ and $G(y)$ are distribution functions.
Define the $\tau$th quantile for cdf $F(x)$, $G(y)$ as
$$\xi_\tau\equiv F^{-1}(\tau)=\inf\{x:F(x)\ge \tau\}$$
and
$$\eta_\tau\equiv G^{-1}(\tau)=\inf\{y:...
1
vote
1
answer
132
views
Cumulative Order Statistics of Independent Non-Identical Distributions
I understand that the p.d.f of order statistics for Independent Non-Identical Distributions are given by the Bapat-Beg theorem as previously explained in another question. As explained in the article, ...
44
votes
4
answers
14k
views
Applications of algebraic geometry to machine learning
I am interested in applications of algebraic geometry to machine learning. I have found some papers and books, mainly by Bernd Sturmfels on algebraic statistics and machine learning. However, all this ...
1
vote
1
answer
427
views
Sharp tail bounds for the maximum of an iid sample of a random variable supported on $[0, 1]$
Let $X_1,\ldots,X_n$ be an iid sample from a distribution supported on $[0, 1]$.
Question
What are some sharp concentration inequalities (i.e tail bounds) empirical statistic defined by $Z_n := \max(...
-2
votes
1
answer
90
views
Existence or impossibility of Gaussian factory
Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.
-1
votes
2
answers
153
views
What is the likehood function in the noise free observation case
In the nonlinear Bayesian Tracking problem, if we consider the noise exists only in the state equation : x[k] = f(x[k-1],v[k-1]) where vk-1 here is an iid process noise sequence
And we suppose that ...
5
votes
1
answer
2k
views
Earth mover/Wasserstein distance between a pdf and an empirical distribution
This question is inspired by this much older question:
Convergence of an empirical distribution w.r.t. the Hellinger distance
Let $P$ be a continuous probability distribution on a compact subset of $...
0
votes
1
answer
142
views
A problem related to the comparison of two integer-valued random variables
Consider an urn containing red, blue and green balls (the situation is the same illustrated in this post).
Let $X$ be the non-negative, integer-valued random variable defined as the number of trials (...
5
votes
2
answers
1k
views
Practical bounds for the Wasserstein distance in 2 dimensions
Let $X_1,\dots,X_n$ be a set of independent samples of a distribution $\mu$ on the unit square, let $\hat\mu_n$ be the empirical distribution on the points $X_1,\dots,X_n$, and let $W_1(\mu,\hat\mu_n)$...
4
votes
3
answers
198
views
Expected distance of nearest matching pair in the game of pairs
Recently I was playing several rounds of the game of pairs with my children. I was surprised that almost every time, one matching pair was adjacent (either next to each other in a row, or vertically). ...
1
vote
0
answers
32
views
Practical statistics for queueing networks
There is a theory for queueing networks where we postulate some nicely behaving base distributions of arrival processes and service processes and then calculate the behaviour of the system.
Now, in ...
2
votes
0
answers
72
views
Proof of a technical fact in the book of Schapire and Freund on boosting
Disclaimer: I asked this question on math.stackexchange.com two weeks ago but it has not been answered yet so I figured that I might as well try to also post it here.
I am currently looking at ...
1
vote
1
answer
300
views
Reconstructing Euclidian space from distance matrix
The setup. Let's say that we have a set of objects $O_i$ for which we have a dissimilarity measure $M(O_1,O_2)$. With this we can build a distance matrix $D_{ij}$.
Let's also assume that we have NO ...
1
vote
1
answer
326
views
Expected values of two non-negative, integer-valued random variables related to an urn problem
Consider an urn containing $c$ distinguishable balls, $\alpha$ of which are red, $\beta$ of which are blue, and $\gamma$ of which are green, and $\alpha+\beta+\gamma=c$. We assume $\alpha,\beta,\gamma&...
3
votes
1
answer
158
views
Non-asymptotic tail bounds for $D_{\text{Hellinger}}(P\|\hat{P}_N)$
Let P be a distribution on a finite set of size $k$ and let $\hat{P}_N=(N_1/N,\ldots,N_k/N)$ be the empirical distribution (frequencies) from a samples of size $N$. Consider the Hellinger distance ...
4
votes
1
answer
380
views
Minimize the variance of a Boltzmann distribution
N.B.: Sorry for cross-posting from https://stats.stackexchange.com/posts/347804/edit (I realized it was the wrong venue for the question, but couldn't find an easy way to transfer the question here).
...
2
votes
0
answers
60
views
Gaussian as a product of two independent random variables [duplicate]
Ideally what I am looking for two random variables, $X$ and $Y$ (if one is positive then that's even better) such that $Z=X\cdot Y\sim\mathcal{N}(0,1)$ where $X,Y$ are some distributions I can ...
6
votes
1
answer
1k
views
Why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?
I'm reading 《Algebraic geometry and statistical learning theory》.My problem is why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?...
2
votes
1
answer
153
views
upper bound on power of neyman-pearson hypothesis test
Let $H_0$ and $H_1$ be two distributions. The Neyman-Pearson lemma says that of all rejection regions $R$ with fixed probability $\alpha$ under $H_0$, the one with maximal probability under $H_1$ is ...
2
votes
2
answers
2k
views
Dependence between direction and magnitude of multivariate normal random vector
Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then
\begin{align}
f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^...
4
votes
2
answers
893
views
Are there known expressions for total variation distance between mean centered multivariate normal densities $N(0,\Sigma_{1})$ and $N(0,\Sigma_{2})$
Consider two mean centered multivariate normal densities $N(0,\Sigma_{1})$ and $N(0,\Sigma_{2})$. Are there known expressions (as opposed to bounds provided by the Pinsker inequality) for the total ...
1
vote
0
answers
135
views
Choice of residual function for least squares error minimization
Good morning,
I have the a set of data $(\sigma,D,\alpha_0)_i$, $i=1...n$ data.
I want to determine two parameters $K_{IC}$, $C_f$ in the basic equation given as
$K_{IC} = \sigma \sqrt{D} k_0(\...
17
votes
2
answers
1k
views
The Chow & Robbins game ≈ 0.79295350640: improvements could come from simple statistics, or from a continuous version of the game
This question seeks help with improving a numerical estimate of the value of the Chow and Robbins game. Much about this game is unknown, such as whether its value is rational, but there are two routes ...