Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,847
questions
1
vote
1
answer
117
views
Local maxima of the sum of Gaussian functions in *one dimension* are always strict local maxima - proof?
Motivated by this question asked earlier, I was wondering whether one can prove easily that the local maxima of the sum of Gaussians:
$$f_n(x):= \sum_{i=1}^{n}e^{-(x-x_i)^2}, \quad x_1 < x_2 < \...
0
votes
0
answers
97
views
Support function of the intersection of a hyper-ellipsoid and a Euclidean ball
Le $\lambda_1 \ge \lambda_2 \ge \ldots \ge \lambda_d$ be positive numbers. For any $x \in \mathbb R^d$ and $r \ge 0$, define $\gamma(x,r) := \sup_{z \in E(r)}x^\top z$, where
$$
E(r) := E \cap B_2^d(r)...
3
votes
2
answers
369
views
Precise asymptotics for moments of order statistics of normal distribution
Let $X_1, \cdots, X_n \sim N(0,1)$ be i.i.d. normal random variates. I am interested in understanding the first two moments of the quasi-range $X_{(n)}-X_{(n-1)}$ (i.e., the maximum value minus the ...
2
votes
0
answers
54
views
Probability bounds of some ranked version of Dirichlet distribution
Recently I have come across a distribution defined on the open ranked simplex $\nabla^{n-1}_+ = \{\vec x \in \mathbb{R}^n:\sum_{k=1}^n x_k =1, x_1 \geq x_2 \geq \cdots \geq x_n > 0\}$, whose ...
9
votes
1
answer
604
views
Popular mistakes in probability
$\DeclareMathOperator\Var{Var}\DeclareMathOperator\Bern{Bern}\DeclareMathOperator\Pois{Pois}$Question: What not-trivial mistakes do students often make when solving problems in probability theory, ...
-1
votes
1
answer
118
views
Under which conditions Mean Square Continuity implies Sample Continuity for Gaussian Processes?
First, let us give the setting.
Let $(\Omega, \Sigma, \mathbf{P})$ be a probability space, let $T$ be some interval of time, and let $X: T \times \Omega \rightarrow S$ be a stochastic process.
By Mean ...
0
votes
1
answer
131
views
Is the unconditional variance of a RV an upper bound for the variance of any conditional expectation of the RV?
Let $X$ and $Y$ be continuous random variables with finite first and second moments. Then, is it true that $Var[X]\geq Var[E(X|Y)]$?
1
vote
1
answer
85
views
An inequality relating $\ell_1$ distance of input and output of a Markov krnel
Let $K$ be a Markov kernel from $\mathcal{X}$ to $\mathcal{Y}$, i.e., $K(\cdot|x)$ is a probability measure on $\mathcal{Y}$ for all $x\in \mathcal{X}$.
Let $\mu$ and $\nu$ be two probability measures ...
2
votes
1
answer
312
views
Lower bound on sum of independent heavy-tailed random variables
I have a sum of $n$ i.i.d random variables $X_i$ such that $E[X_i] = 0$,$\mathrm{E}[|X_i|^{1 + \delta}]$ exists for some $0 < \delta < 1$ but $\mathrm{E}[|X_i|^{1 + \delta+ \epsilon}]$ does not ...
4
votes
1
answer
156
views
Existence of copula bound pointwise strictly smaller than the Fréchet-Hoeffding upper bound
Consider bivariate copulas $C_1$ and $C_2$ with $\max\{C_1(u,v), C_2(u,v)\}< M_2(u,v)$ for all $u,v \in(0,1)$, where $M_2(u,v) := \min\{u,v\}$ is the Fréchet-Hoeffding upper bound.
Is there a ...
2
votes
1
answer
119
views
Justification of the use of residual plot
$\DeclareMathOperator\Cov{Cov}$Backround of my Question
Let $Y$ be the response variable, $\mathbb{X}$ be the explanatory variables. The ultimate goal of prediction is finding a function $f^{*}$ that ...
0
votes
1
answer
57
views
Given positive $\epsilon$ and $c$, find a density $\phi$ such that $t\phi(\epsilon/t) \ge c \|\phi'\|_\infty$ for all positive $t$
A nice density (on $\mathbb R$) is function $\phi:\mathbb R \to \mathbb R$ such that
(1) $\phi(x) \ge 0$ for all $x \in \mathbb R$,
(2) $\int_{-\infty}^\infty \phi(x) \mathrm{d}x = 1$,
(3) $\phi$ is ...
6
votes
1
answer
232
views
Violating an order statistic inequality?
[Edit: for posterity, I'm adding two small comments to the code explaining how to fix it, in light of Iosef Pinelis' answer below. Look for "Should be:" to find the corrections.]
Suppose we ...
1
vote
0
answers
46
views
Sample complexity of estimating a doubly stochastic matrix
Let $P\in\mathbb{R}^{n\times n}$ be a doubly-stochastic matrix. That is:
$$P(x,y)\geq 0,\quad \sum_xP(x,y)=1,\quad \sum_yP(x,y)=1.$$
I would like to know if lower and upper bounds on the sample ...
2
votes
0
answers
85
views
Training an energy-based model (EBM) using MCMC
I'm reading this paper about training energy-based models (EBMs) and don't understand the parameters that we are training for? The part that is relevant to the question is in pages 1-4. Here is the ...
3
votes
2
answers
2k
views
Expected gradient vs. gradient of expectation
Suppose a function $f(x): \mathbb R^d \mapsto \mathbb R^D$, and its stochastic approximator, $g(x; W): \mathbb R^d \mapsto \mathbb R^D$. Here $W$ is some random variable. Then $g(x; W)$ is unbiased in ...
2
votes
1
answer
131
views
Normalized concentration inequality for empirical CDF (iid sum)
Consider the empirical and population CDF,
$$
F_n(t) = \frac{1}{n} \sum_{i=1}^n 1\{X_i \leq t\} \quad \mbox{and} \quad
F(t) = \mathbb{E} [F_n(t)],
$$
where above $X_1, \dots, X_n$ are iid, real-...
2
votes
1
answer
617
views
Bootstrapping and the central limit theorem
I have been looking into bootstrapping lately and although I believe to have understood the basic process somewhat, I am fuzzy on the mathematical details. I will begin with my understanding of what ...
1
vote
1
answer
171
views
Rademacher complexity for a family of bounded, nondecreasing functions?
Let $\{\phi_k\}_{k=1}^K$ be a family of functions mapping from an interval $[a, b]$ to $[-1, 1]$.
That is, $\phi_k \colon[ a,b] \to [-1, 1]$ are nondecreasing maps on some finite interval $[a, b] \...
-1
votes
1
answer
53
views
Strategy optimization based on biased data
This is a question from high-frequency trading (HFT). A market maker sends transaction requests to the exchange's server via a certain number of gateways. At these gateways the requests incur some ...
4
votes
1
answer
610
views
How to get the lower bound of the following $\tau$?
Let $A=\{a_{ij}\}_{1\le i,j\le n}$ be an $n$ by $n$ normalized Gaussian random matrix with $E[a_{ij}]=0$ and $E[a_{ij}^2]=1/n$. Ordering its eigenvalues by $\lambda_1\le \lambda_2\le \cdots \lambda_n$ ...
2
votes
2
answers
222
views
Minimal conditions on random vector $X \in R^n$ to ensure that $\lim_{t\to 0^+}\sup_{\|w\|_p = 1}\sup_{u \in \mathbb R}\mathbb P(|X'w-u| \le t)=0$
Let $X$ be a random variable on $\mathbb R^n$ and let $S_p^n := \{w \in \mathbb R^n \mid \|w\|_p = 1\}$ be the unit-sphere w.r.t to the $\ell_p$-norm in $\mathbb R^n$. We will be particularly ...
0
votes
1
answer
294
views
Deduce that a function is zero on interval $[0,M]$
I have been thinking about this for the last few days but I was not able to produce a definitive answer.
Take an integrable function $g$ that maps in $\mathbb{R}$ and with domain contained in $[0,M]$ (...
1
vote
1
answer
119
views
Tight upper-bounds for the Gaussian width of intersection of intersection of hyper-ellipsoid and unit-ball
Let $\Lambda$ be a positive-definite matrix of size $n$ and let $R \ge 0$, which may depend on $n$. Consider the set $S := \{x \in \mathbb R^n \mid \|x\|_2 \le R,\,\|x\|_{\Lambda^{-1}} \le 1\}$ where $...
1
vote
0
answers
28
views
Finding variance-minimizing weights [closed]
I'm trying to solve the following matrix calculus problem:
$\text{argmin}_{v \in R_+^K}(v'\Sigma v) \hspace{0.5pc} \text{subject to} \hspace{0.5pc} 1'v=1$
where $\Sigma$ is a well-behaved (symmetric, ...
1
vote
1
answer
183
views
Lower bound on ratio of extreme order statistics
This question relates to bounds on expectations of order statistics, elaborated upon in the Book by Arnold and Balakrishnan (1989). Let $X_1,\ldots,X_n$ be i.i.d. continuous random variables ...
3
votes
1
answer
173
views
Asymptotic results for smallest gap of Gaussian random matrix
For a symmetric Gaussian random matrix $G=\{G\}_{1\le i,j \le n}$ with iid $E[G_{ij}]=0$ and $E[G_{ij}^2]=1/n$ (it is normalized), ordering its eigenvalues $\lambda_1\le \lambda_2\le\cdots \lambda_n$.
...
4
votes
2
answers
269
views
Maximal entropy of integer partitions of $n$
Let $\operatorname{Part}(n)$ be the set of integer partitions of $n$.
A partition $p \in \operatorname{Part}(n)$ has $k$ summands and $d$ distinct summand $n_i$, with $d \leq k$ and $d$ frequencies $...
1
vote
0
answers
24
views
Minimax statistical estimation of proximal transform $\mbox{prox}_g(\theta_0)$, from linear model data $y_i := x_i^\top \theta_0 + \epsilon_i$
tl;dr: My question pertains the subject of minimax estimation theory (mathematical statistics), in the context of linear regression.
Given a vector $\theta_0 \in \mathbb R^d$, consider the linear ...
1
vote
1
answer
183
views
Gaussian width of intersection of cube and ball in high-dimensional euclidean space
Let $d$ be a large positive integer and fix $r \ge 0$. Set $S := B_2^n \cap [-r,r]^d$, where $B_2^d$ is the euclidean unit-ball in $\mathbb R^d$. Finally, let $\omega(S)$ be the Gaussian width of $S$, ...
4
votes
1
answer
166
views
Least squares problem with left and right unknowns
For $i=1,...,n$, let $b_i$ be a scalar and $A_i$ be an $k\times l$ matrix. Is there a closed form solution for the following problem assuming $n>k+l$?
$$\min_{x\in \mathbb{R}^k ,y\in \mathbb{R}^l} \...
1
vote
0
answers
74
views
Closed-form expression for combinatorial summation with a quadratic exponent?
In a current project, I have encountered sums of the form $$A_N(\theta_1,\theta_2) = \sum_{x=0}^{N}{N \choose x} \theta_1^x \theta_2^{x^2}$$ for $\theta_1$ and $\theta_2$ positive reals. My current ...
1
vote
1
answer
310
views
How to calculate this limit (if exist)?
I have just asked the calculation of the following summation see here $$S(a,b,m,n_1,n_2)=\sum_{k=0}^m a^k b^{m-k} {n_1\choose k} {n_2\choose m-k}, $$
which is motivated by the calculation of the ...
0
votes
1
answer
76
views
WLLN for bootstrap means of stationary ergodic processes?
Setup:$\quad$
Suppose that $(X_n)$ is a stationary ergodic process with $E|X_1|<\infty$.
Given $X^{(n)}=(X_1, \dots, X_n)$, select a standard Efron bootstrap subsample $(X_{n,1}^*, \dots, X_{n,m(n)}...
1
vote
0
answers
73
views
Concavity of expected size of a maximum matching (in a bipartite graph) w.r.t. edge probability
Given a n*n bipartite graph where each edge (between any two nodes on the opposite side) is formed i.i.d. with probability $p$, can we show a concavity result on the expected size of a maximum ...
3
votes
1
answer
540
views
Does $E[1/f]\overset{d}\to 1/E[f]$ for $\operatorname{Tr}H=1,\operatorname{Tr}H^2=0.5$?
Suppose $x$ is a Gaussian random variable in $d$ dimensions with $H=E[xx^T],\ \operatorname{Tr}(H)=1,\operatorname{Tr}(H^2)=0.5$. Take $m$ I.I.D. samples of $x$ and stack them as rows of $X$.
Is it ...
1
vote
1
answer
349
views
How fast does this Gaussian random walk move away from the origin?
Suppose $z_i$ are IID zero-centered $d$-dimensional Gaussian random variables with unit-trace covariance $\Sigma$ and $g(z_i)$ is the sum of its components.
Consider the following random walk:
$$x_s=\...
3
votes
1
answer
186
views
Probabilistic Taylor theorem for concave functions
This paper proves a probabilistic version of Taylor's theorem
\begin{equation*}
\mathbb{E}g(X) = \sum_{k=0}^{n-1} \frac{g^{(k)}(0)}{k!} \mathbb{E}X^k + \frac{\mathbb{E}X^n}{n!} \mathbb{E} g^{(n)}(X_{(...
3
votes
2
answers
223
views
$\min(\det(\mathbf{A}))$ for special matrix $\mathbf{A}$
(The construction of matrix $\mathbf{A}$ is not difficult to be understood. You can first jump to A Toy Example to take a glance. Any idea or suggestion would be appealing for me.)
The Original ...
2
votes
0
answers
114
views
Rough path expected signature vs cumulant-generating function / characteristic function
What is the point of using rough path expected signature to characterize the law of а stochastic process when the cumulant generating function is known ($\log\mathbb{E}[e^{i\theta X(t)}]$)?
Since an ...
0
votes
1
answer
113
views
Behavior of F distribution quantile as degree of freedom varies
I want to analyze the $1-\alpha$-quantile, $\alpha\in(0,1)$, of a $F_{n, m}$ distribution, keeping n fixed while increasing m. It seems that the quantile decreases monotonically, but I would like to ...
2
votes
1
answer
223
views
Triangle equality for cosine similarity in high dimensions
I'm trying to understand whether I can use the following equality in my application -- for $u,v,w \in \mathbb{R}^d$:
$$\cos(u,w)\approx \cos(u,v)\cos(v,w)$$
Where $\cos(x,y)$ gives cosine of the angle ...
1
vote
0
answers
56
views
How to calculate the unifrom entropy or VC dimension of the following class of functions?
When dealing with U process I meet with such a uniform entropy to calculate.
For any $\eta>0$, function class $\mathcal{F}$ containing functions $f=\left(f_{i, j}\right)_{1 \leq i \neq j \leq n}: \...
3
votes
1
answer
379
views
What is a tensor product of random variables?
I am trying to understand the the following paper https://arxiv.org/pdf/1810.10971.pdf, in particular Example 2:
If $ Y \sim N(0,1)$, the standard normal on $\mathbb{R}$, then
$ \begin{align*} \Big( \...
4
votes
1
answer
396
views
Expected norms of Wishart matrices
Suppose $x_i \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu,\Sigma)$. What can we say about dependence on $b$ of Frobenius/spectral norm quantities below?
$$f(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^...
4
votes
1
answer
405
views
CLT convergence rate for sum of uniforms (in TV distance)
Suppose $X_1, \cdots, X_n \sim_{\mathrm{iid}} U([-1,1])$, where $U([-1, 1])$ denotes the continuous uniform distribution over the interval $[-1, 1]$ (so $E[X_i] = 0$ and $\text{Var}[X_i]= 1/3$). Let $...
1
vote
0
answers
81
views
Properties of max of many linear combinations of a multivariate normal vector and/or sum of top $k$ elements of a multivariate normal vector
Thank you in advance for your help!
I am interested in studying the following probability:
$$P\big[\max_{H \subset X,|H|=k} \sum_{i \in H} \mathbf{a}_i^T \mathbf{w} \ge 0 \big],$$
where $\mathbf{a}_i$ ...
4
votes
2
answers
297
views
Does the average of correlated Gaussian random variables with mean zero and different variances converge in probability to their mean?
Let $X_i\sim N(0,\sigma_i^2)$ and $\operatorname{Corr}(X_i,X_j)>0$. Is it possible to show that $$\frac{1}{N} \sum_{i=1}^N X_i \overset{p}\rightarrow E[X_i]=0.$$ Do you have a reference to a law of ...
3
votes
3
answers
190
views
$\mathbf{y}=f(\mathbf{x},\mathbf{z})=g(\mathbf{x})$ if $\mathbf{z}\perp \!\!\! \perp \{\mathbf{y},\mathbf{x}\}$ jointly?
Let $\mathbf{y},\mathbf{x},\mathbf{z}$ are real-valued random vectors with possibly different dimensions. Assume $\mathbf{y}=f(\mathbf{x},\mathbf{z})$ for some function $f$.
If $\mathbf{z} \perp\!\!\!\...
0
votes
0
answers
28
views
k-means errors for a block Gaussian vector
Consider a standard centered Gaussian vector $(X_1,...,X_n)$ with an approximate block structure, i.e. there is $q$ and a partition of $\{1,...,n\}$ in $q$ classes such that if $i,j$ are in the same ...