Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,847
questions
0
votes
0
answers
13
views
Quotient of estimators
Say $A$ is a set of a finite number of samples, and $\hat{\mu}_A$ and $\hat{\sigma}_A$ are unbiased estimators (computed over $A$) of $\mu$ and $\sigma$ which are some distinct population statistics. ...
-2
votes
0
answers
16
views
Population-level measurable function from samples
Consider a set of points $\{(x_i, y_i)\}_{i=1}^\infty$, where $x_i \sim X$ for some random variable $X$. Let’s say they are all in $[0, 1]$. We may assume $x_i$ are dense in $[0,1]$. Is it generally ...
2
votes
1
answer
121
views
How can one construct a confidence interval for the mean of a uniformly distributed random variable using a sample of size 2
Two numbers will be randomly (and independently) selected from a uniform distribution on an interval with length $L$ and center $M.$
It is very easy to estimate $M$ (just take the average of the two ...
1
vote
0
answers
31
views
Concentration result for self-normalized empirical process
In Theorem 1.1 of this paper by Bercu, Gassiat and Rio, a concentration result is derived for the 'self-normalized' empirical process. Specifically, suppose that $(X,X_n)_{n \ge 1}$ is a sequence of i....
2
votes
1
answer
253
views
A result about sub-exponential random variables
I am reading the proof of Theorem 1(a) in the paper that proposed the CLIME method for estimating precision matrix. I am puzzled by an inequality on Page 605 three lines above formula (29). I isolate ...
5
votes
1
answer
749
views
Mathematics research relating to machine learning
What branch/branches of math are most relevant in enhancing machine learning (mostly in terms of practical use as opposed to theoretical/possible use)? Specifically, I want to know about math research ...
1
vote
0
answers
54
views
Gain of a steady state Kálmán filter
It is well known that the state covariance of a steady-state Kálmán filter is the solution of a discrete Riccati equation.
$$P_\infty = F(P_\infty - P_\infty H^T(HP_\infty H^T+R)^{-1}HP_\infty)F^T + Q$...
0
votes
1
answer
438
views
Hypothesis testing for not identically distributed random variables conditioned on the outcome of a subset
I encountered the following problem (I give more details of the problem at the end of the post) and I am trying to figure out the best way of performing a null hypothesis testing. I looked for similar ...
0
votes
0
answers
63
views
Inequality related with log-concave distributions
Fix any $n$-dimensional unit vector $\mathbf v$.
Let $\mathbf x$ be a random vector following the $n$-dimensional standard normal distribution. It has been shown (Analysis of Perceptron-Based Active ...
0
votes
0
answers
27
views
Limiting value of trace of resolvent matrix involving two independent Wishart random matrices
Let $n_1$, $n_2$, and $d$ be positive integers tending to infinity such that
$$
d/n_k \to \phi_k \in (0,\infty).
$$
Let $X_1 \in \mathbb R^{n_1 \times d}$ and $X_2^{n_2 \times d}$ be independent ...
6
votes
5
answers
3k
views
Approximation to the ratio of a Gaussian CDF to PDF
Johnstone and Silverman (2005) claimed that for large x
$\frac{1-\Phi(x)}{\phi(x)} \approx \frac{1}{x}$
where $\Phi(x)$ and $\phi(x)$ are the CDF and PDF for a normal random variable.
I was able ...
-1
votes
0
answers
31
views
What are the assumptions when dealing with the EM Algorithm in order to calculate $f_{\textbf{Y}|\textbf{X},\theta}(\textbf{Y}|\textbf{X},\theta)$?
Consider the EM Algorithm. In order to apply it, we are given the observed data $\textbf{X}$ (generated by some distribution depending on some parameters), which can be a vector, a matrix or a matrix ...
8
votes
2
answers
408
views
Concentration inequality for minimal eigenvalue of sample covariance
I was reading an article of matrix completion and met the following lemma
The concentration inequality for $\sigma_{\max}$ part is a standard result. However, I didn't find any results like the $\...
1
vote
1
answer
85
views
Concentration inequalities for heavy-tailed distributions
Suppose $X_1,...,X_N$ are $N$ i.i.d random variables with heavy tailed distributions. For example, $E[X_i^p]\leq 1$ for some $p\geq 1$. Are there some concentration inequalities to bound the tail
$$P(\...
1
vote
1
answer
108
views
A property of the distribution related to stochastic ordering
Let $X$ be a random variable with a symmetric support $S\subset[-M,M]$ for some $M>0$. (i.e., if x is a point of increase of CDF $F_X(\cdot)$, so is $-x$.)
Has the infimum value of $c$ such that
\...
1
vote
0
answers
57
views
Asymptotic stochastic ordering for weighted sum of i.i.d. random variables
Are you aware of any literature focusing on the conditions such that for two i.i.d. sequences of discrete r.v.'s $\{X_n\}$ and $\{Y_n\}$,
\begin{equation}
a_1X_1+a_2X_2+\ldots+a_nX_n\geq_1 a_1Y_1+...
1
vote
0
answers
109
views
Random partition of an interval – Dirichlet distributed?
Let $X_1, \ldots, X_N \sim \operatorname{Unif}[0,1]$ and consider the intervals between successive order statistics: $[0, X_{(1)}], [X_{(1)}, X_{(2)}], \ldots, [X_{(N)}, 1]$.
What is the distribution ...
-1
votes
0
answers
80
views
Stochastic dominance for (~)random harmonic series
$\DeclareMathOperator\Pr{Pr}$Consider the series $\sum_n^\prime a_nR_n$, where $a_n=\frac{(-1)^n}{n+c}$ for some constant $c\in(0,1)$ and $\{R_n\}$ denotes a sequence of i.i.d. Bernoulli random ...
4
votes
1
answer
239
views
Local maxima of the sum of Gaussian functions in *multiple dimensions* are always strict local maxima - prove/disprove/prove conditionally?
This is a follow up of the question in one dimension, that asked to show that the all the maxima of the sum of Gaussian
$$f_n(x):= \sum_{i=1}^{n}e^{-(x-x_i)^2}, x_1 < x_2 < \dots < x_n$$
are ...
3
votes
1
answer
85
views
Perturbation results for statistical estimators
Suppose I have a continuous random variable whose distribution $f$ is some parametric form (normal, exponential, etc.) that is known to me. If I draw many independent samples $x_i$ from $f$, I can ...
3
votes
1
answer
253
views
Latent Dirichlet allocation and properties of digamma function
In the paper Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3(4–5), 993–1022. http://www.jmlr.org/papers/volume3/blei03a/blei03a....
1
vote
2
answers
225
views
Joint moments like $\tau(XYXYXY)$ in terms of individual moments of free variables $X,Y$
Terry Tao RMT book has the following formula for joint moment of freely independent random variables $X,Y$ in Section 2.5
$$\tau(XYXY)=\tau(X)^2\tau(Y^2)+\tau(X^2)\tau(Y)^2-\tau(X)^2\tau(Y)^2$$
...
0
votes
0
answers
65
views
Gibbs Priors form a Martingale
I am working on adapting variational inference to the recently developed Martingale posterior distributions. The first case, which reduces the VI framework to Gibbs priors, is proving hard to show as ...
1
vote
1
answer
367
views
Approximate the singular values of a certain random dot-product kernel matrix (in the sense of El Karoui, Cheng-Singer, etc.)
Let $g:\mathbb R \to \mathbb R $ be a continuous function which is
"sufficiently smooth" (e.g $\mathcal C^3$) around $0$, and
"sufficiently integrable" (e.g integrable w.r.t $N(0,...
0
votes
0
answers
70
views
Some new questions on Rademacher complexity
For $A\subset R^n$,$A=(a_1,a_2,\dots, a_n)$, $\sigma_i$ are Rademacher random variable.
Is $|\mathbb{E}_\sigma \inf_{a\in A}\sum_{i=1}^n\sigma_ia_i| \le |\mathbb{E}_\sigma \sup_{a\in A}\sum_{i=1}^n\...
1
vote
0
answers
58
views
Estimates on number of observations and iterations depending on number of states, when applying the Baum-Welch algorithm to HMMs
Are there papers estimating how many observations $\mathcal{O}$ and how many random initializations $\ell$ one needs, to get an arbitrary good agreement between the model $\lambda = (\pi,A,B)$ that ...
0
votes
1
answer
173
views
Expectation of top-K selection of squared Gaussian random variables
Let us have
$$
Z = [z_1, z_2, \dots, z_n],
$$ where $z_i \sim N(0, \sigma^2)$ and are iid. Additionally, consider
$$
X_k := \{ x \in \{0, 1\}^n : e^T x = k \}
$$ If $Y = \max_{X \in X_k} |Z^T X|^2,$ ...
3
votes
1
answer
551
views
Show that $\sup_{\|g\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n g(Z_i)\right|\rightarrow_{\text{a.s.}}0.$ when $\delta_n\rightarrow 0$?
UPDATE: The result below can be understood as an almost sure stochastic equicontinuity condition. I don't know of any result establishing primitives of almost sure stochastic equicontinuity. If you ...
0
votes
0
answers
29
views
Multivariate delta method gradient calculation with mixed moments
Here's a slightly simplified version of my problem (using fewer dimensions than what I'm actually solving).
Take $X \in \mathbb{R}^2$. We already know that $\sqrt{n}(X - \mu) \overset{d}{\to} N_2(0,\...
4
votes
1
answer
855
views
Collatz conjecture and stationarity of time series
The Collatz conjecture is known to all. Has this question been approached by methods related to statistics? I think of Collatz iterates as a time series, and the question of whether we always get the ...
0
votes
2
answers
61
views
$n$-wise extension of covariance / correlation
Suppose we have $n$ random variables $X_1, \ldots, X_n$ and we want some sort of characterization about how "statistically" related they are as a whole, with the motivation being that for $n=...
4
votes
1
answer
302
views
Asymptotic limit of trace of random matrix $(aI_m + WW^\top)^{-1}$, where $W$ has iid rows from $N(0,\Sigma)$
Let $m$ and $d$ be positive integers with $m,d \to \infty$ such that $m/d \to \rho \in (0,\infty)$. Let $W$ be a random $m \times d$ matrix with iid rows $w_1,\ldots,w_m \sim N(0,\Sigma)$ for a ...
1
vote
2
answers
199
views
Relationship between fixed points and inversions in permutations
Inversions in a permutation $Y$ are defined as pairs where $Y_a < Y_b$ but $a > b$, while fixed points in $Y$ are defined as elements where $Y_a = a$ (i.e., 1-cycles). Let $S_\alpha$ be the set ...
1
vote
0
answers
59
views
A small lemma on cache resets (Bloom filters in particular)
Assume a fixed set of message $D$ and an associated distribution for selecting each message $d_i$ such that the total probability $\sum_{i \in D} d_i = 1$. We create a cache with $M$ bits and $k$ ...
0
votes
2
answers
93
views
Points based partial ranking
I want to rank a population $P=\{P_1,\ldots,P_n\}$. I am given a set $R=\{R_1,\ldots,R_k\}$ of partial rankings. The partial rankings may have varying sizes (e.g. the first ranking ranks only 8 ...
3
votes
2
answers
922
views
Expectation of the trace of inverse of a Gaussian random matrix
Given a $N×M$ random complex gaussian matrix $X$ and $N×K$ random complex gaussian matrix $Y$ I'm interested in approximating the expectation expressed as:
\begin{align}
E[trace({(aX{X^H} + I)^{ - ...
1
vote
0
answers
58
views
Dimension-free sample complexity for the inverse of Gaussian sample covariance?
Suppose I have $m$ samples drawn from a Gaussian in $\mathbb{R}^n$, and need the inverse of the sample covariance $\Sigma_m^{-1}$ to be $\varepsilon$-close to true inverse covariance $\Sigma^{-1}$ (in ...
7
votes
2
answers
216
views
Evolution of the empirical mean of a list as we remove elements proportional to their value
Consider a list of $N$ integers $k_1,k_2,\dots k_N$, drawn independently from some distribution $P(k)$ with $k_i \geq 1$. We denote its mean with $\langle k\rangle=\sum_{k=1}kP(k)$. The first two ...
6
votes
4
answers
2k
views
Question about this ratio in Metropolis-Hastings MCMC algorithm
I have a stupid question about the Metropolis-Hastings sampling algorithm.
If I got this right, for every variable $X$ in turn, which currently has value $x_{old}$, you generate a new sample $x_{new}$....
2
votes
0
answers
176
views
Statistical invariants of Riemannian manifolds
$\DeclareMathOperator\diam{diam}\DeclareMathOperator\rad{rad}\DeclareMathOperator\iso{iso}\DeclareMathOperator\com{com}\DeclareMathOperator\con{con}$A cheap way of defining invariants of Riemannian ...
0
votes
0
answers
74
views
High probability bound on number of sparse solutions to Gaussian linear system
Suppose we have a random matrix $A \in \mathbb{R}^{m \times n}$ with all entries i.i.d. from the standard Normal distribution $\mathcal{N}(0, 1)$. Suppose $k$ divides $n$, and let $S \subseteq \mathbb{...
4
votes
1
answer
515
views
Detecting a shift in distribution where the distribution is clumpy
I'm responsible for a charity donation site. We're about the change the site design, and we want to know the best way of detecting if the distribution of donations changes after the design. The ...
3
votes
2
answers
271
views
Minimax optimal multiple hypothesis test
Let us consider the following two-player game
between Chooser and Guesser.
There is a finite set $\Omega$
and $k$ probability distributions
on $\Omega$, denoted by $
\mathcal{P}
=\{P_1,\ldots,P_k\}
$.
...
3
votes
1
answer
324
views
Concentration inequality for norm of solution to nonlinear least-squares problem
Define the piecewise-linear function $\psi(t):=\max(t,0)$ for all $t \in \mathbb R$.
Let $d,n,k \to \infty$ at the same rate (i.e $n \asymp k \asymp d$).
Let $y_1,\ldots,y_n \in \{-1,1\}$ uniformly ...
4
votes
1
answer
446
views
Gaussian process kernel parameter tuning
I am reading on gaussian processes and there are multiple resources that say how the parameters of the prior (kernel, mean) can be fitted based on data,specifically by choosing those that maximize the ...
4
votes
3
answers
1k
views
Probability theory and measuring the true strength of chessplayers
If you wanted to measure the strength of, say, a chess player, the best way would involve knowing the true value of each position: then you could compute the frequency $W$ with which the player finds ...
2
votes
1
answer
87
views
How to fit a set of parametrized data to a parametrized distribution?
I have a time series $d_i(a)$ which depends on the parameter $a$. On the other hand, I have a sequence of normal distributions $\mathcal{N}(0,Q_i(a))$, where the variance $Q_i$ depends on time and ...
2
votes
0
answers
134
views
Sum of arrival times of Chinese Restaurant Process (CRP)
Suppose that a random sample $X_1, X_2, \ldots$ is drawn from a continuous spectrum of colors, or species, following a Chinese Restaurant Process distribution with parameter $|\alpha|$ (or ...
91
votes
8
answers
16k
views
Is there a natural random process that is rigorously known to produce Zipf's law?
Zipf's law is the empirical observation that in many real-life populations of $n$ objects, the $k^\text{th}$ largest object has size proportional to $1/k$, at least for $k$ significantly smaller than $...
2
votes
2
answers
1k
views
How to derive the mean of inverse-Wishart distribution?
How to derive the mean of inverse-Wishart distribution in
Inverse-Wishart distribution?
I have no idea about it. Thanks for your help.