Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,851
questions
7
votes
0
answers
178
views
Can one "smooth over" k-wise independence to get actual independence?
I came across the following toy problem and was curious if there was a simple solution or counterexample. Suppose you have a distribution $p$ on $m$ random variables $X_1, \ldots, X_m$, each with ...
- 91
7
votes
0
answers
617
views
Convergence of Maximum Likelihood Estimator
I apologize for the basic question. If $\{p_\theta(x): \theta\in K\subseteq\mathbb{R}\}$ is a smooth family of distributions, then the MLE $\hat{\theta}_n,$ under suitable regularity conditions ...
- 1,269
6
votes
5
answers
3k
views
Approximation to the ratio of a Gaussian CDF to PDF
Johnstone and Silverman (2005) claimed that for large x
$\frac{1-\Phi(x)}{\phi(x)} \approx \frac{1}{x}$
where $\Phi(x)$ and $\phi(x)$ are the CDF and PDF for a normal random variable.
I was able ...
- 203
6
votes
2
answers
2k
views
Are Gaussian Processes more important than other stochastic processes?
I am doing a course at university and it deals with Gaussian Processes mainly. We use them for fitting data and prediction, machine learning, regression, classification. Is there any particular reason ...
- 87
6
votes
2
answers
526
views
Probability of winning game whereby $T+1$ heads in a row of a coin flip is required to win where $T$ is the number of cumulative tails flipped
I have a weird question which probably seems out of place here but it has proven more difficult than anticipated. I am going to describe the game without showing work toward a solution. Numerically, ...
user504691
6
votes
1
answer
795
views
reverse KL-divergence: Bregman or not?
I am having a little trouble getting my head around the two "directions" of the Kullback-Leibler divergence:
Definition (Kullback-Leibler divergence) For discrete probability distributions $...
- 91
6
votes
2
answers
724
views
Strange pattern in rounding errors?
This will look at first like a posting about trigonometry, then maybe about statistics, then finally about peculiarities of either
a certain random process; or
the pseudorandom number generator that ...
- 11.9k
6
votes
2
answers
1k
views
Algebraic Statistics textbook
Hey
A friend and I are thinking of having an algebraic statistics seminar next semester. Does anyone know of a good book to try learn it out of?
- 436
6
votes
2
answers
329
views
Is there a way to reconstruct the convolution $(f * g)(x)$ of $f$ with a Gaussian $g$ from sampled values, $(f*g)(a), a \in A$?
Suppose that $f: \mathbb{R} \to \mathbb{C}$ is a function which has support in $[-1,1]$. Let $g = g_\sigma$ be a centered Gaussian with variance $\sigma^2$. Is there a way to reconstruct the ...
- 367
6
votes
2
answers
2k
views
Is the Binomial Expectation of Convex Function Convex in p?
Suppose $X$ has a binomial distribution with success probability $p$ and $n$ trials and let $h(\cdot)$ be a positive convex real-valued function.
Is the function $g(p)=\mathbb{E}[h(X)\ |\ p]$ convex ...
- 103
6
votes
2
answers
235
views
Does "perpendicular phase incoherence" satisfy the triangle inequality?
I asked this question at https://math.stackexchange.com/q/4783968/222867, but even after a 200-point bounty, no solution was provided, only some thoughts regarding possible directions. So I'm now ...
- 678
6
votes
3
answers
1k
views
Bound on probabilities of the sum of uniform order statistics
Let $X_1,...,X_n$ be i.i.d. random variable with a uniform distribution on [0,1]. Denote by $X_{(1)}\leq X_{(2)} \leq \ldots \leq X_{(n)}$ their order statistics.
Given $k\geq 1$ and $u\in[0,k]$, I ...
- 607
6
votes
1
answer
645
views
On the distribution of roots modulo primes of an integral polynomial
For motivation and related questions, see below.
Rough sketch of the question.
View $\bigsqcup_{p \text{ prime}} (\mathbb{Z}/p\mathbb{Z})$ as a ‘subset’ of the unit circle, via $a\pmod{p} \mapsto e^{...
- 83
6
votes
2
answers
2k
views
Interesting thesis topic on statistical inference that is sufficiently mathematical
Hello , I am a student who's gonna start honours in mathematics . Currently , I am at the stage of finding a suitable honours thesis topic . I've chosen my supervisor , who's research interest is on ...
- 61
6
votes
1
answer
1k
views
Why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?
I'm reading 《Algebraic geometry and statistical learning theory》.My problem is why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?...
- 103
6
votes
4
answers
15k
views
Linear Regression Coefficients W/ X, Y swapped
Let's say I have a linear regression model of the form $ y = B_x x + I_x + \epsilon $, where $B_x$ is the beta coefficient of the $x$ term, $I_x$ is the intercept term and $\epsilon$ is additive, ...
- 159
6
votes
1
answer
235
views
Spectrum asymptotics for a product of $k$ random matrices?
How does the spectrum of a product of $k$ random matrices behave around 0?
In particular, I'm wondering if the CDF of squared singular values behaves as $x^{\frac{1}{k+1}}$ around 0. The result for $k=...
- 2,380
6
votes
1
answer
226
views
Positive semidefinite ordering for covariance matrices
Suppose that X and Z are matrices with the same number of rows. Let
$$ D = \left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1} - \left[\begin{array}{cc} (X' X)^{-1} & ...
- 1,058
6
votes
1
answer
1k
views
Kullback Leibler "variance": does that divergence have a name?
If you consider two probability distributions $p$ and $q$, one way to measure the distance between the two is the Kullback-Leibler divergence:
$$KL(p,q)=\int p \log (p/q) = E_p(\log p/q)$$
and this ...
6
votes
1
answer
954
views
An integral with respect to the Haar measure on a unitary group
Let $A,D\in \mathbb{C}^{n \times n}$ be diagonal matrices. I need to calculate
$$\int_{U(n)}\det{(A-HDH^\dagger)}\,\mathrm{d}H$$
where $dH$ is the unit invariant Haar measure on the group of unitary ...
- 141
6
votes
2
answers
3k
views
Is a function of complete statistics again complete?
suppose $T$ is a complete stats for a parameter $\theta$.
Is any function $f(T)$ again complete?
It sounds weird but the definition seems to confirm that $f(T)$ is indeed complete..
- 183
6
votes
4
answers
2k
views
Question about this ratio in Metropolis-Hastings MCMC algorithm
I have a stupid question about the Metropolis-Hastings sampling algorithm.
If I got this right, for every variable $X$ in turn, which currently has value $x_{old}$, you generate a new sample $x_{new}$....
- 63
6
votes
1
answer
232
views
Violating an order statistic inequality?
[Edit: for posterity, I'm adding two small comments to the code explaining how to fix it, in light of Iosef Pinelis' answer below. Look for "Should be:" to find the corrections.]
Suppose we ...
- 3,809
6
votes
1
answer
198
views
Good upper-bound for $\mathbb E[|X-np|^r]$ where $X \sim \text{Binomial}(n,p)$ and $r \ge 1$
Disclaimer. Question moved from SE.
Setup
Let $X \sim \text{Binomial}(p, n)$, and $r \ge 1$.
Question
What is a good upper-bound for $\mathbb E[|X-np|^r]$ ?
Solution for small $r$
If $r=2$, then ...
- 6,726
6
votes
1
answer
928
views
Largest eigenvalues of a (random) correlation matrix?
I am recently studying on eigenvalues of a (random) correltion matrix. For a $N\times N$ correlation matrix (with a given meaning of randomness), its (1st, 2nd, etc.) eigenvalues have some ...
- 358
6
votes
1
answer
3k
views
Does MCMC overcome the curse of dimensionality?
I want to compute an integral like this
$$\frac{\int_y g(y) e^{-\beta f(y)} \text{d} y } {\int_y e^{-\beta f(y)} \text{d} y}$$
where $f(y)$ is not necessarily convex and the dimension $d$ of $y$ is ...
- 73
6
votes
1
answer
2k
views
Minimizing KL divergence: the asymmetry, when will the solution be the same?
The KL divergence between two distribution $p$ and $q$ is defined as
$$
D( q \| p)\int q(x)\log \frac{q(x)}{p(x)} dx
$$
and is known to be asymmetry: $D(q\|p)\neq D(p\|q)$.
If we fix $p$ and try to ...
- 185
6
votes
1
answer
3k
views
Statistical distance between discrete and continuous distributions
Are there any statistical distance functions that are capable of comparing a continuous and a discrete distribution? From reading this list
http://en.wikipedia.org/wiki/Statistical_distance
the only ...
- 69
6
votes
2
answers
2k
views
Absolute moments of symmetrical distributions
Suppose $F~$ is a probability distribution symmetrical about 0, for which all moments exist. Let $\mu_i~$be the $i$-th moment (of course $\mu_i=0$ if $i~$ is odd).
We know there are some conditions ...
- 37.2k
6
votes
2
answers
174
views
Order statistics of bounded variable : L2 concentration?
Let n >0
Let $X_1,...,X_n$ be i.i.d. random variable with a density (say $f(x)$) on [a,b]. Denote by $X_{(1)}\leq X_{(2)} \leq \ldots \leq X_{(n)}$ their order statistics.
I'm interested in ...
- 245
6
votes
1
answer
170
views
Existence of distribution for certain order statistics
This is an open question: given a sequence of $n$ real numbers $x_1<x_2<\dots<x_n$, does there always exist a probability distribution, such that $\{x_i\}$ happens to be the $n$ expected ...
- 123
6
votes
1
answer
2k
views
Brownian motion and its maximum and its minimum
Let $W_u, 0\leq u \leq t$ be Brownian motion.
Let $m_t= min_{0\leq u\leq t} W_u$ and $M_t = max_{0 \leq u \leq t} W_u$.
The fact that $(M_t , W_t)$ is absolutely continuous with respect to Lebesgue ...
- 161
6
votes
1
answer
388
views
Hypothesis test beyond simple hypotheses (mathematical statistics)
In mathematical statistics, the following problem (simple hypothesis test) is considered: given a data sample, test the hypothesis $H_0$ stating that all sampled values are values of a random variable ...
- 71
6
votes
1
answer
522
views
Is there a mistake in Vapnik's "Basic Lemma"?
I have a concern about the "Basic Lemma" which Valdimir Vapnik states and proves in his 1998 book Statistical Learning Theory (ch. 14.3, pp. 574–76): It seems like a certain coefficient should have ...
- 303
6
votes
1
answer
451
views
Computing the correlation between two vectors without divulging them
Alice and Bob respectively know a vector of $N$ real numbers $u$ and $v$. They would both like to know $\rho = \langle u,v \rangle/N$ but Alice does not want Bob to gain anymore information about $u$ ...
- 1,892
6
votes
2
answers
270
views
Confidence intervals when the number of samples is random
I am interested in computing confidence intervals for the mean of a random variable $X$ given $\require{cancel}\xcancel{N \text{ i.i.d. samples}}$ an i.i.d. sample of $N$ copies of $X$, where $N$ is $\...
- 63
6
votes
1
answer
212
views
How close $k$-sums of a random set of numbers are on average?
Consider a set of random iid variables $x_1, \ldots x_n$ uniformly distributed on $[0, 1]$. For each $S \subset [n]$ with $1 \leq |S| = k < n$ take $\sigma_S = \sum_{i \in S}x_i$. Obviously $\...
- 5,380
6
votes
1
answer
2k
views
Rate of convergence of uniform order statistics to their expectations
This is a problem that I encountered in my research and have no clues to fully
resolve it. Basically, I need large (or moderate) deviation bounds on the
difference between an order statistic of ...
- 934
6
votes
3
answers
584
views
What is the order of the constant $K$ in the multidimensional Dvoretzky-Kiefer-Wolfowitz inequality($Ke^{-c z}$)?
Let $F_n$ be the empirical distribution obtained from an i.i.d. sample
of the distribution $F:R ^d \to [0, 1]$.
Kiefer (1961) shows that the convergence of the empirical distribution is like
$$
P\left(...
user24451
6
votes
1
answer
345
views
Deviation bound for the maximum of the norm of Wiener process
Let $W(t)$ be an $n$-dimensional Wiener process. Denote by $\chi_n^2$ a chi-squared random variable with $n$ degrees of freedom. I have recently found the following inequality given without proof:
$$
{...
- 61
6
votes
1
answer
836
views
Bochner's theorem, in stages
Bochner's theorem (for the real line version) asserts an infinite tower of inequalities, as a positivity condition. Taking each one, what do they mean, in an elementary fashion (at least at the start)?...
- 275
6
votes
1
answer
1k
views
Probability distributions: The maximum of a pair of iid draws, where the minimum is an order statistic of other minimums?
General question: What is the distribution for the maximum of 2 independent draws from cdf F(x), when we know that the minimum of those same two draws is the kth order statistic of the minimum of n ...
- 233
6
votes
2
answers
538
views
Extension of copulas
Let $(X,Y)$ be a random vector. Suppose that the marginal distribution functions of $X$ and $Y$ are known (say $F_1$ and $F_2$). Then the joint law of $(X,Y)$ is given by the following formula:
$$F_{...
- 921
6
votes
1
answer
807
views
Peakedness of multimodal distributions
In Probability theory, does there exist some measures of peaked-ness for multi-modal distributions. I guess kurtosis as such would not be a good measure of peaked-ness for multimodal distributions. ...
- 99
6
votes
1
answer
4k
views
Conditioning on one term of a sum of random variables
Let $\theta$ be normally distributed with mean $\bar \theta$ and variance $s^2$. Let $Z$ be normally distributed with mean $0$ and variance $\sigma^2$, and chosen independently of $\theta$. Define $...
- 8,392
6
votes
1
answer
525
views
Can information be extracted more precisely using more random trials?
Write $n$ iid draws of $(x,y)$ as $(x^n, y^n)$. Fix $R\in (0,H(x))$. What is the min of $n^{-1}H(y^n|f(x^n))$ over maps $f$ with range $\lbrace 1,\dots,\exp nR\}$, taking $n\to \infty$?
6
votes
1
answer
289
views
Is there a name for this quantity between two distributions?
Let $f$ be a probability density on a compact domain $D$, and say that $x_1,\dots,x_n$ are samples from $f$. If we wanted to compute the Wasserstein distance between $f$ and the empirical ...
- 3,929
6
votes
1
answer
1k
views
How to check if a symmetric random variables is the difference of two iid symmetric random variables
I have the continuous symmetric random variable $X$ in $\mathbb{R}$. If I know its distribution function $F(x)$ what are the conditions on $F(x)$ so that $X=Y_1 - Y_2$ where $Y_i$ are also iid ...
6
votes
2
answers
419
views
how to sample a conditioned diffusion
there are several reasons why we could be interested in sampling conditioned diffusions:
if we observed a diffusion at discrete time and want to do some kind of inference on the parameters of the ...
- 2,133
6
votes
0
answers
125
views
Why wavelet methods are not popular anymore in nonparametric statistics?
Back in my master years, I took a nonparametric statistics class. In this class, a few nonparametric methods were presented, but I remember spending a lot of times on methods based on wavelet ...
- 73