Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,851
questions
0
votes
0
answers
163
views
Upper-bound for bracketing number in terms of VC-dimension
Let $P$ be a probability distribution on a measurable space $\mathcal X$ (e.g;, some euclidean $\mathbb R^m$) and let $F$ be a class of funciton $f:\mathcal X \to \mathbb R$. Given, $f_1,f_2 \in F$, ...
1
vote
1
answer
218
views
Bound error in approximating $E_x [H(f(x))]$ with random $(1/n) \sum_{i=1}^n \Phi(f(x_i)/h)$ where $H$ is Heaviside function and $\Phi$ is normal CDF
Let $f:\mathbb R^d \to \mathbb R$ be a "sufficiently smooth" function. For simplicity, we may consider $f$ to be an affine function, i.e $f(x) \equiv b-x^\top w$, for some $(w,b) \in \mathbb ...
2
votes
0
answers
182
views
Harish-Chandra–Itzykson–Zuber integral with two terms
We know
$$
\int \mathcal{D}U \exp(\mathrm{Tr}(AUBU^*))
$$
can be computed by Harish-Chandra–Itzykson–Zuber(HCIZ) integral. I am wondering whether it is possible to compute
$$
I=\int \mathcal{D}U \exp(\...
1
vote
0
answers
92
views
$L_1$ convergence rates for multivariate kernel density estimation
Let $X$ be a random variable on $\mathbb R^d$ with probability density function $f$, and let $X_1,\ldots,X_n$ of $X$ be $n$ iid copies of $X$. Given a bandwidth parameter $h=h_n > 0$ and a kernel $...
4
votes
0
answers
156
views
Convergence rates for kernel empirical risk minimization, i.e empirical risk minimization (ERM) with kernel density estimation (KDE)
Let $\Theta$ be an open subset of some $\mathbb R^m$ and let $P$ be a probability distribution on $\mathbb R^d$ with density $f$ in a Sobolev space $W_p^s(\mathbb R^d)$, i.e all derivatives of $f$ ...
5
votes
2
answers
313
views
Placing pins on a Galton board to approximate an arbitrary distribution
Inspired by this reddit post: https://old.reddit.com/r/math/comments/tv3cbg/how_do_you_unbell_curve_a_galtonplinko_board/
The Nth Galton Board, G(N), is a triangular lattice of pegs of height N-1.
...
2
votes
0
answers
116
views
Consistent approximation of weighted Radon transform of smooth probability density, using kernel density estimation
Let $X$ be a random vector in $\mathbb R^d$, with "sufficiently smooth" probability density function on $\rho$. For unit-vectors $w$ and $u$ in $\mathbb R^d$, and a scalar $b \in \mathbb R$, ...
2
votes
1
answer
255
views
Does taking minimum preserve density monotonicity?
Suppose $X$ and $Y$ are continuous random variables with a joint density function $f_{X,Y}$. Both $X$ and $Y$ are supported on $(0,1)$ and have continuous (can be assumed differentiable) and non-...
4
votes
2
answers
279
views
Bounds on the number of samples needed to learn a real valued function class
Let us see Theorem 6.8 in this book, https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf
It gives us a lowerbound (and also an ...
2
votes
1
answer
81
views
Lower bound on the error of proportion estimation
Let $X \sim \operatorname{Bin}(n,p)$. Suppose we estimate $p$ by $\hat{p}=\frac{X}{n}$. By Hoeffding’s inequality
it holds for all $\delta \in (0,1)$ with probability at least $1-\delta$ that, $$\...
-1
votes
1
answer
245
views
The distribution of the sum of values from a normal and a truncated normal distribution
Using R to extract truncated normal distribution samples and normal distribution samples separately, when they are combined, the image drawn by the hist function is very similar to a normal ...
4
votes
1
answer
259
views
About non-reversible Metropolis Hastings Markov chain
I am reading a paper about constructing a non-reversible Metropolis Hastings Markov chain from a reversible one as described at a high level in paragraph $3$ of page $1$.
But I don't understand how, ...
4
votes
1
answer
111
views
Consistent empirical estimation of Radon transform of a multivariate density function
Let $P$ be a "nice" distribution on $\mathbb R^m$ (e.g., multivariate Gaussian, etc.), with density $p$. Let $H := \{x \in \mathbb R^m \mid x^\top w = b\}$ be a hyperplane in $\mathbb R^m$ ...
2
votes
1
answer
180
views
Derive equation for regularized logistic regression with batch updates
I am trying to understand this paper by Chapelle and Li "An Empirical Evaluation of Thompson Sampling" (2011). In particular, I am failing to derive the equations in algorithm 3 (page 6). ...
0
votes
1
answer
140
views
Sparse linear regression model
Define a set $C_3(S):=\{\Delta\in R^d: \|\Delta_{S^c}\|_1\le 3\|\Delta_S\|_1\}$. Suppose we form a random design matrix $X\in R^{n\times d}$ with rows drawn iid from a $N(0,\Sigma)$ distribution and ...
1
vote
1
answer
177
views
Lasso of sparse linear regression model
Consider the sparse linear regression model $y=X\theta^*+w$, where $w\sim N(0, \sigma^2 I_{n\times n})$ and $\theta^*\in R^d$ is supported on a subset $S$. Suppose that the sample covariance matrix $\...
1
vote
2
answers
247
views
Central limit theorem of random vectors when the dimension is increasing
This is a question about central limit theorems when the dimension is increasing. Suppose now I have a random vector $X_N = (X_{N1}, \cdots, X_{Np})\in\mathbb{R}^p$. For all $c_p\in\mathbb{R}^p$ with $...
2
votes
1
answer
170
views
The mean of positive points on a unit $n$-sphere $S^n$
My question is similar to The mean of points on a unit n-sphere $S^n$.
I have a unit $n$-sphere $S^n$ and a set $P$ of points lying on its surface.
I use geodesic distance metric $d(p,q)=\arccos(pq^T)$...
1
vote
1
answer
366
views
Expected value of a function of normal random variable
Suppose $X\sim \mathcal{N}(0,\sigma^2)$, find the expectation $\mathbb{E}\left[\frac{1}{(1+X^2)^a}\right]$ where $a$ is a fixed positive real number.
Is there an explicit formula for the above ...
2
votes
1
answer
415
views
Weakly dependent central limit theorem
Say I have $N$ random variables $X_1,\cdots,X_i,\cdots,X_N$, with zero mean and finite variance. $X_i$ and $X_j$ are independent iif $|i-j|>m$, and positively correlated otherwise (say the ...
1
vote
1
answer
130
views
Permute a sparse random matrix to resemble a diagonal matrix as much as possible
Say we generate an $N \times N$ sparse random matrix $W$, where each element $W_{ij}$ was independently chosen to be $1$ with probability $p=\frac{a}{N}$, and $0$ with probability $1-p$. We are ...
3
votes
0
answers
92
views
Explaning why the spectrum of a setting simple structure random matrix is always spiked ($d-1$ eigenvalues close to zero, and $1$ away from zero)
For concreteness, let $m=500$, $d=600$, $N=1000$. Let $W$ be and $d \times m$ matrix with unit-norm rows and let $u$ be a uni-norm vector of length $m$. Given a binary vector $b$ of length $m$, length ...
2
votes
1
answer
154
views
Representer theorem for a loss / functional of the form $L(h) := \sum_{i=1}^n (|h(x_i)-y_i|+t\|h\|)^2$
Let $K:X \times X \to \mathbb R$ be a (positive-definite) kernel and let $H$ be the induced reproducing kernel Hilbert space (RKHS). Fix $(x_1,y_1),\ldots,(x_n,y_n) \in X \times \mathbb R$. For $t \ge ...
1
vote
1
answer
113
views
Bayesian inverse problems on non-separable Banach spaces
I am now studying Bayesian inverse problems. In the note of Dashti and Stuart https://arxiv.org/abs/1302.6989, they mentioned that "... when considering a non-separable Banach space $B$, it is ...
1
vote
0
answers
34
views
Correlating two matrices $A,B$ with stochastic dependency structure imposed by cross-validation
Consider a labelled data set
$$D = \{(x_1, y_1),...,(x_n, y_n)\} $$
on which we want to evaluate a machine learning algorithm using $k$-fold cross validation with $m$ different random seeds. This ...
5
votes
0
answers
187
views
Divergence for Bhattacharya Information matrix
The Fisher information matrix (in the scalar parameter case) can be obtained from the Kullback-Leibler divergence by
$$g(\theta) = -\frac{\partial}{\partial \theta}\frac{\partial}{\partial \theta'}D(...
4
votes
0
answers
118
views
Exponential families closed under affine transformations
Let $(\Omega,\Sigma,\mu)$ be a probability space and let $\mathcal{M}$ be an exponential family of probability distributions for $\mu$ of the following form: There are $\varphi_1,\dots,\varphi_n:\...
3
votes
2
answers
334
views
General version of $d$-separation
I find the $d$-separation criterion (see, e.g., Theorem 2 here; note however the preceding definition, which basically means we are treating discrete random variables) a really useful sufficient ...
1
vote
1
answer
79
views
What is the maximum possible coefficient of variation for data taking values within a specified range?
I have a question that seems very basic, and yet I have not managed to find an answer after probably several hours of Google-searching.
Fix $0<a<b<\infty$, and let $\mathcal{P}_{[a,b]}$ be ...
2
votes
1
answer
474
views
Expectation value of inverse covariance matrix when sampling from unit sphere
Let $X \sim \operatorname{Unif}S_{d-1}$, so $X\in\mathbb{R}^d$ and is distributed uniformly on the unit sphere.
Then let $X_1, \dots, X_n \sim X$ iid and define the matrix $\mathbf{X}\in\mathbb{R}^{n\...
1
vote
1
answer
483
views
Integral of the product of a gaussian pdf and cdf
I am trying to solve the integral of a gaussian cumulative distribution function and a gaussian probability function. On this site I have seen solutions of similar, less general integrals (e.g. ...
1
vote
1
answer
195
views
Large deviation for empirical median
I found this exercise while reading some notes on Large Deviation Principle. This exercise is at the end of the very first chapter, including Cramer's Theorem and essentially nothing more (no Sanov ...
1
vote
1
answer
152
views
Multiple integral and integral with respect to a function of variables
This is concerning Eq. (3.7) of C R Rao's 1945 paper (see p.81 of this article). Can someone help me in figuring out the second equality in Eq. (3.7)?
His claim is (since $\phi(x,\theta) = \Phi(T,\...
2
votes
1
answer
152
views
Is a random $(r+1,r)$-biregular bipartite graph $r$-edge connected w.h.p?
A uniformly random $r$-regular bipartite graph on $n$ vertices is known to be $r$-edge connected. That is, with high probability as $n$ grows large, the minimum size of a cut in a random $r$-regular ...
2
votes
0
answers
86
views
The covariance of certain random variable
We define two random variables $X_n,Y_n $ on the sample space $\{1,2,3,\cdots,n\}$ with counting measure. We denote by $C_n$ the covariance of theses two random variables: $C_n=Cov(X_n,Y_n)$.
...
2
votes
0
answers
190
views
When does Le Cam's method give tight lower bounds for distribution testing?
In the context of statistical estimation or distribution testing, Le Cam's method is a way to prove lower bounds on the minimax sample complexity ([1,2,3,4], further details below). My question is: ...
4
votes
5
answers
1k
views
Reference request: importance of Lipschitz continuity
I see that Lipschitz continuity is a common assumption used in optimisation, statistics, machine learning, etc.
Could you point me in the direction of some literature that discusses why Lipschitz ...
0
votes
0
answers
87
views
Spectrally-weighted Stieltjes transform of random matrix $Z=XX^\top$ in terms of Stieltjes transform of $Z$ and the weighting function
Let $n$ and $d$ positive integers going to infinity such that $d/n \to \gamma \in (0,\infty)$. Let $X$ be a random $n \times d$ iid rows from $N(0,\Sigma)$, where $\Sigma = diag(\lambda_1,\ldots,\...
1
vote
0
answers
243
views
Comparing two multivariate normal distribution
Let $\mathbf{Z}\sim N(\boldsymbol{\mu},\mathrm{\Sigma})$, where
\begin{equation}\label{Eq.Mean}
\boldsymbol{\mu}^{\rm T} = \delta[-\sqrt{\frac{x_1x_2}{x_1+x_2}},\frac{-1}2\sqrt{\frac{x_1x_3}{x_1+x_3}},...
1
vote
1
answer
190
views
Orthogonal transformation of multivariate Bernoulli-Gaussian distribution
Actually, I have asked this question in https://math.stackexchange.com/questions/4330127/orthogonal-transformation-of-multivariate-bernoulli-gaussian-distribution, but I think mathoverflow might be ...
0
votes
1
answer
177
views
Properties of a function $C_\ell(\ell)$ which checks an inequality in ideal case (decreasing assumption) and after estimating impact in general case
Suppose that
$X=2 \ell+1, Y=C_{\ell}$, both $X$ and $Y$ are function of $\ell$, $X$ is increasing and $Y$ is assuming to be decreasing.
But in reality, my data follow a $C_\ell$ increasing for a small ...
2
votes
1
answer
365
views
Extension of subcopulas to copulas
This question is about the extension of subcopulas to copulas, shown in Sklar, A. (1996), "Random variables, distribution functions, and copulas: A personal look backward and forward." ...
1
vote
1
answer
97
views
Limiting value of $\dfrac{1_n^\top B^{-1} A B^{-1} 1_n}{d}$, where $A=WW^\top + a I_n$, $B = WW^\top + b I_n$, and $W \sim N(0,\Sigma_d)$
Let $n$ and $d$ be positive integers with
$$
n,d \to \infty, \quad n/d \to \rho \in (0,\infty).
$$
Let $\Sigma_d$ be a psd matrix such that
$\mbox{trace}(\Sigma_d) = 1$.
$\|\Sigma_d\|_{op} = \mathcal ...
2
votes
1
answer
88
views
How to fit a set of parametrized data to a parametrized distribution?
I have a time series $d_i(a)$ which depends on the parameter $a$. On the other hand, I have a sequence of normal distributions $\mathcal{N}(0,Q_i(a))$, where the variance $Q_i$ depends on time and ...
2
votes
1
answer
135
views
Comparison between $\|X\|_2$ and $\|X\|_{2,1}$
For any real random variable $X$, define
$$\|X\|_{2,1}=\int_0^\infty \sqrt{\Pr(|X|>t)}dt.$$
This quantity (it is not a norm) appears in various problems, e.g. the multiplier central limit theorem (...
1
vote
0
answers
119
views
Using maximum entropy principle for joint probability estimation
Let $X_1, \dots, X_n, Y$ be random variables, each taking values in $\{0,1\}$. Assume that we are interested in estimating, for each $v=(v_1,\dots,v_n)\in \{0,1\}^n$, the probability
$$
p(v) = P[Y=1|...
1
vote
1
answer
121
views
Moments of rescaled Bernoulli random matrix
Suppose $X \in \{0,1\}^{n \times m}$ is a matrix generated according to the following generative process:
$$Z_{ij} \sim \text{Bernoulli}(p) \implies X_{ij} = \frac{Z_{ij}}{\sum_{k=1}^m Z_{ik}}.$$
Is ...
1
vote
2
answers
254
views
Distribution of interarrival times for a special class of stochastic point processes
I am interested in Poisson-binomial stationary point processes (here on the real line) defined as follows. Let
$t_k=k/\lambda$, with $k\in\mathbb{Z}$ and $\lambda>0$,
$F_s(x)$ be a symmetric, ...
4
votes
1
answer
301
views
What journal(s) do you recommend for submitting a paper on a topic that spans information theory and estimation theory?
I've written a paper that a) demonstrates an equivalence between conditional complexity $K$($Y$|$X$) in information theory and the random component of an effect size estimate $r_{xy}$, and then b) ...
0
votes
1
answer
125
views
How to demonstrate a correlation inequality? [closed]
If there are 3 vectors X, Y, Z of the same length, for any $x_i \in X,y_i \in Y,z_i \in Z$, we have $0<x_i<1,0<y_i<1,0<z_i<1$.
The correlation between Z, Y is greater than between X, ...