Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,847
questions
0
votes
0
answers
14
views
Quotient of estimators
Say $A$ is a set of a finite number of samples, and $\hat{\mu}_A$ and $\hat{\sigma}_A$ are unbiased estimators (computed over $A$) of $\mu$ and $\sigma$ which are some distinct population statistics. ...
-2
votes
0
answers
16
views
Population-level measurable function from samples
Consider a set of points $\{(x_i, y_i)\}_{i=1}^\infty$, where $x_i \sim X$ for some random variable $X$. Let’s say they are all in $[0, 1]$. We may assume $x_i$ are dense in $[0,1]$. Is it generally ...
1
vote
0
answers
31
views
Concentration result for self-normalized empirical process
In Theorem 1.1 of this paper by Bercu, Gassiat and Rio, a concentration result is derived for the 'self-normalized' empirical process. Specifically, suppose that $(X,X_n)_{n \ge 1}$ is a sequence of i....
1
vote
0
answers
54
views
Gain of a steady state Kálmán filter
It is well known that the state covariance of a steady-state Kálmán filter is the solution of a discrete Riccati equation.
$$P_\infty = F(P_\infty - P_\infty H^T(HP_\infty H^T+R)^{-1}HP_\infty)F^T + Q$...
0
votes
0
answers
27
views
Limiting value of trace of resolvent matrix involving two independent Wishart random matrices
Let $n_1$, $n_2$, and $d$ be positive integers tending to infinity such that
$$
d/n_k \to \phi_k \in (0,\infty).
$$
Let $X_1 \in \mathbb R^{n_1 \times d}$ and $X_2^{n_2 \times d}$ be independent ...
-1
votes
0
answers
31
views
What are the assumptions when dealing with the EM Algorithm in order to calculate $f_{\textbf{Y}|\textbf{X},\theta}(\textbf{Y}|\textbf{X},\theta)$?
Consider the EM Algorithm. In order to apply it, we are given the observed data $\textbf{X}$ (generated by some distribution depending on some parameters), which can be a vector, a matrix or a matrix ...
2
votes
1
answer
121
views
How can one construct a confidence interval for the mean of a uniformly distributed random variable using a sample of size 2
Two numbers will be randomly (and independently) selected from a uniform distribution on an interval with length $L$ and center $M.$
It is very easy to estimate $M$ (just take the average of the two ...
0
votes
0
answers
63
views
Inequality related with log-concave distributions
Fix any $n$-dimensional unit vector $\mathbf v$.
Let $\mathbf x$ be a random vector following the $n$-dimensional standard normal distribution. It has been shown (Analysis of Perceptron-Based Active ...
1
vote
1
answer
85
views
Concentration inequalities for heavy-tailed distributions
Suppose $X_1,...,X_N$ are $N$ i.i.d random variables with heavy tailed distributions. For example, $E[X_i^p]\leq 1$ for some $p\geq 1$. Are there some concentration inequalities to bound the tail
$$P(\...
1
vote
1
answer
108
views
A property of the distribution related to stochastic ordering
Let $X$ be a random variable with a symmetric support $S\subset[-M,M]$ for some $M>0$. (i.e., if x is a point of increase of CDF $F_X(\cdot)$, so is $-x$.)
Has the infimum value of $c$ such that
\...
1
vote
0
answers
57
views
Asymptotic stochastic ordering for weighted sum of i.i.d. random variables
Are you aware of any literature focusing on the conditions such that for two i.i.d. sequences of discrete r.v.'s $\{X_n\}$ and $\{Y_n\}$,
\begin{equation}
a_1X_1+a_2X_2+\ldots+a_nX_n\geq_1 a_1Y_1+...
-1
votes
0
answers
80
views
Stochastic dominance for (~)random harmonic series
$\DeclareMathOperator\Pr{Pr}$Consider the series $\sum_n^\prime a_nR_n$, where $a_n=\frac{(-1)^n}{n+c}$ for some constant $c\in(0,1)$ and $\{R_n\}$ denotes a sequence of i.i.d. Bernoulli random ...
0
votes
0
answers
65
views
Gibbs Priors form a Martingale
I am working on adapting variational inference to the recently developed Martingale posterior distributions. The first case, which reduces the VI framework to Gibbs priors, is proving hard to show as ...
1
vote
2
answers
225
views
Joint moments like $\tau(XYXYXY)$ in terms of individual moments of free variables $X,Y$
Terry Tao RMT book has the following formula for joint moment of freely independent random variables $X,Y$ in Section 2.5
$$\tau(XYXY)=\tau(X)^2\tau(Y^2)+\tau(X^2)\tau(Y)^2-\tau(X)^2\tau(Y)^2$$
...
0
votes
0
answers
70
views
Some new questions on Rademacher complexity
For $A\subset R^n$,$A=(a_1,a_2,\dots, a_n)$, $\sigma_i$ are Rademacher random variable.
Is $|\mathbb{E}_\sigma \inf_{a\in A}\sum_{i=1}^n\sigma_ia_i| \le |\mathbb{E}_\sigma \sup_{a\in A}\sum_{i=1}^n\...
1
vote
0
answers
109
views
Random partition of an interval – Dirichlet distributed?
Let $X_1, \ldots, X_N \sim \operatorname{Unif}[0,1]$ and consider the intervals between successive order statistics: $[0, X_{(1)}], [X_{(1)}, X_{(2)}], \ldots, [X_{(N)}, 1]$.
What is the distribution ...
1
vote
0
answers
58
views
Estimates on number of observations and iterations depending on number of states, when applying the Baum-Welch algorithm to HMMs
Are there papers estimating how many observations $\mathcal{O}$ and how many random initializations $\ell$ one needs, to get an arbitrary good agreement between the model $\lambda = (\pi,A,B)$ that ...
0
votes
0
answers
29
views
Multivariate delta method gradient calculation with mixed moments
Here's a slightly simplified version of my problem (using fewer dimensions than what I'm actually solving).
Take $X \in \mathbb{R}^2$. We already know that $\sqrt{n}(X - \mu) \overset{d}{\to} N_2(0,\...
0
votes
2
answers
61
views
$n$-wise extension of covariance / correlation
Suppose we have $n$ random variables $X_1, \ldots, X_n$ and we want some sort of characterization about how "statistically" related they are as a whole, with the motivation being that for $n=...
0
votes
1
answer
173
views
Expectation of top-K selection of squared Gaussian random variables
Let us have
$$
Z = [z_1, z_2, \dots, z_n],
$$ where $z_i \sim N(0, \sigma^2)$ and are iid. Additionally, consider
$$
X_k := \{ x \in \{0, 1\}^n : e^T x = k \}
$$ If $Y = \max_{X \in X_k} |Z^T X|^2,$ ...
1
vote
0
answers
59
views
A small lemma on cache resets (Bloom filters in particular)
Assume a fixed set of message $D$ and an associated distribution for selecting each message $d_i$ such that the total probability $\sum_{i \in D} d_i = 1$. We create a cache with $M$ bits and $k$ ...
1
vote
2
answers
199
views
Relationship between fixed points and inversions in permutations
Inversions in a permutation $Y$ are defined as pairs where $Y_a < Y_b$ but $a > b$, while fixed points in $Y$ are defined as elements where $Y_a = a$ (i.e., 1-cycles). Let $S_\alpha$ be the set ...
1
vote
0
answers
58
views
Dimension-free sample complexity for the inverse of Gaussian sample covariance?
Suppose I have $m$ samples drawn from a Gaussian in $\mathbb{R}^n$, and need the inverse of the sample covariance $\Sigma_m^{-1}$ to be $\varepsilon$-close to true inverse covariance $\Sigma^{-1}$ (in ...
2
votes
0
answers
176
views
Statistical invariants of Riemannian manifolds
$\DeclareMathOperator\diam{diam}\DeclareMathOperator\rad{rad}\DeclareMathOperator\iso{iso}\DeclareMathOperator\com{com}\DeclareMathOperator\con{con}$A cheap way of defining invariants of Riemannian ...
0
votes
0
answers
74
views
High probability bound on number of sparse solutions to Gaussian linear system
Suppose we have a random matrix $A \in \mathbb{R}^{m \times n}$ with all entries i.i.d. from the standard Normal distribution $\mathcal{N}(0, 1)$. Suppose $k$ divides $n$, and let $S \subseteq \mathbb{...
7
votes
2
answers
216
views
Evolution of the empirical mean of a list as we remove elements proportional to their value
Consider a list of $N$ integers $k_1,k_2,\dots k_N$, drawn independently from some distribution $P(k)$ with $k_i \geq 1$. We denote its mean with $\langle k\rangle=\sum_{k=1}kP(k)$. The first two ...
0
votes
2
answers
93
views
Points based partial ranking
I want to rank a population $P=\{P_1,\ldots,P_n\}$. I am given a set $R=\{R_1,\ldots,R_k\}$ of partial rankings. The partial rankings may have varying sizes (e.g. the first ranking ranks only 8 ...
3
votes
2
answers
271
views
Minimax optimal multiple hypothesis test
Let us consider the following two-player game
between Chooser and Guesser.
There is a finite set $\Omega$
and $k$ probability distributions
on $\Omega$, denoted by $
\mathcal{P}
=\{P_1,\ldots,P_k\}
$.
...
2
votes
0
answers
134
views
Sum of arrival times of Chinese Restaurant Process (CRP)
Suppose that a random sample $X_1, X_2, \ldots$ is drawn from a continuous spectrum of colors, or species, following a Chinese Restaurant Process distribution with parameter $|\alpha|$ (or ...
3
votes
1
answer
95
views
When does the optimal model exist in learning theory?
In the context of learning theory, we usually have: data $(x,y)\sim P(x,y)$, with $x\in\mathcal{X}\subseteq\mathbb{R}^d$ and $y\in\mathcal{Y}\subseteq\mathbb{R}^k$, a hypothesis class $\mathcal{F}\...
2
votes
1
answer
91
views
expectation of the product of Gaussian kernels and their input
I was wondering if anybody knows how to solve: $$\mathbb{E}{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}\left[ (\mathbf{x}{i} - \mathbf{z})(\mathbf{x}{j} - \mathbf{z})^\top \exp\left( - (\...
0
votes
0
answers
18
views
Analyzing point distributions in Voronoi tessellations from two probability sources $p$ and $\tilde{p}$
Let's suppose I have a probability distribution $p$ and another distribution $\tilde{p}$. Suppose I sample $K$ points from the distribution $p$ which will be my centroids for my Voronoi tessellation. ...
0
votes
0
answers
44
views
Approximate CDF of integral using the Berry-Esseen theorem
I'm trying to approximate CDF of the integral $$\frac{1}{T}\int_0^T e^{\sigma W_t+\left(r-\frac{\sigma^2}{2}\right)t}dt,$$
where $W_t$ is the Wiener process, i.e. $W_t\sim N(0,t)$.
For this I use ...
1
vote
1
answer
99
views
Analytical solution for a double integral involving logistic functions and Gaussian distributions
I am working on a mathematical problem involving the evaluation of a double integral, and I am seeking an analytical solution or techniques to solve it. The integral I'm dealing with is as follows:
$$...
7
votes
2
answers
386
views
Upper bound on VC-dimension of partitioned class
Fix $n,k\in \mathbb{N}_+$.
Let $\mathcal{H}$ be a set of functions from $\mathbb{R}^n$ to $\mathbb{R}$ with finite VC-dimension $d\in \mathbb{N}$. Let $\mathcal{H}_k$ denote the set of maps of the ...
3
votes
0
answers
129
views
Known relations between mutual information and covering number?
This is a question about statistical learning theory. Consider a hypothesis class $\mathcal{F}$, parameterized by real vectors $w \in \mathbb{R}^p$. Suppose I have a data distribution $D \sim \mu$ and ...
0
votes
0
answers
88
views
Weighted least squares regression: Iterative modeling of variance
In chemical analysis, the instrument's signal are plotted as a function of chemical concentration. In general, higher the concentration higher is the response and the relationship is linear. At ...
0
votes
1
answer
74
views
Existence and uniqueness of a posterior distribution
I am wondering about the existence and uniqueness of a posterior distribution.
While Bayes' theorem gives the form of the posterior, perhaps there are pathological cases (over some weird probability ...
0
votes
0
answers
43
views
Tight Chernoff Concentration for Bernoulli(p) RV
I remember seeing a research paper on tight concentration of Bernoulli(p) random variable in terms of $p$.
What I mean is that they used a stronger upper bound for the MGF than $E[e^{s(X-p)
}]\leq e^{...
2
votes
0
answers
64
views
Construct a Bregman divergence from Wasserstein distance
I was wondering whether one has studied the Bregman divergence arising from a squared Wasserstein distance.
More precisely, let $\Omega\subset \mathbb{R}^d$ be a compact set and $c\in \Omega\times \...
0
votes
1
answer
174
views
Concentration inequalities for random sampling without replacement
Let a population $C$ consist of $N$ values $c_1, c_2, \cdots, c_N$, with $c_i\in \{0,1\}$. Let $X_1, X_2, \cdots, X_n$ denote a random sample without replacement from $C$ and let $Y_1, Y_2, \cdots, ...
2
votes
0
answers
54
views
Continuous-time Wold decomposition
I'm looking for a reference for the Wold–Zasukhin decomposition in continuous time for stationary random processes on the real line.
I am aware of the classic result in the book from Rozanov, which ...
1
vote
0
answers
62
views
Approximation of continuous function by multilayer Relu neural network
For continuous/holder function $f$ defined on a compact set K, a fix $L$ and $m_1,m_2,\dots,m_L$, can we find a multilayer Relu fully connected network g with depth $L$ and each $i$-th layer has width ...
1
vote
0
answers
138
views
conjecture for general form of minimax estimator
I had previously posed an overly ambitious version of this conjecture here,
Form of minimax estimator,
which was quickly shot down by Václav Voráček (on twitter) and Iosif Pinelis (MO answer in the ...
0
votes
0
answers
29
views
Probability related to record index that cross zeros in $[0, 1, \cdots, N]^{\mathbb{N}}$
This question is inspired by this paper. For those who are interested in more details and applications about record index and record values, you can find them in the paper.
Let $X=\{0, 1, 2, \cdots, N\...
1
vote
1
answer
321
views
Form of minimax estimator
Let $\Delta$ be the set of all probability distributions over $\mathbb{N}=\{1,2,\ldots\}$ and fix some $\mathcal{P}\subseteq\Delta$.
Suppose additionally that $\Delta$ is endowed with some norm $||\...
2
votes
1
answer
162
views
Law of iterated logarithm for quadratic variation of Brownian motion
Let $(\Omega, \mathcal{F}, \mathbb{P})$ denote a probability space supporting a standard Brownian motion $B$. Let $\Pi=\{\pi_n : n \ge 0\}$ denote the sequence of dyadic uniform partitions of the ...
0
votes
0
answers
45
views
Relation between Kernel density estimation and Reproducing kernel Hilbert space?
The procedure of kernel density estimation using a kernel $K$ is very similar to the construction of an RKHS from the kernel $K(x,y) = K(x-y).$ However, this viewpoint is not mentioned every place I ...
6
votes
2
answers
234
views
Does "perpendicular phase incoherence" satisfy the triangle inequality?
I asked this question at https://math.stackexchange.com/q/4783968/222867, but even after a 200-point bounty, no solution was provided, only some thoughts regarding possible directions. So I'm now ...
1
vote
0
answers
55
views
Limiting value of expectation of trace of truncated Gram matrix
Let $n$ and $d$ be large positive integers such that $d/n = a \in (0,1)$, fixed. Let $x_1,\ldots,x_n$ be iid random vectors from $N(0,I_d)$. Fix $b \in (0,1]$ and a unit-vector $v \in \mathbb R^d$, ...