Applied, computational and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments, case studies.

learn more… | top users | synonyms

2
votes
2answers
87 views

estimating variance of dependent normal distributed data

Let $X_{ij}$ with $1\leq i<j\leq n$ (that are $X_{12},\dots, X_{1n},\dots,X_{(n-1)n}$) be ${n \choose 2}$ identically normal distributed $N(0,\sigma^2)$ such that $ \text{corr}(X_{ij},X_{rs})=\rho ...
0
votes
2answers
96 views

Determine joint distribution from projections

Let $X=(X_1,\dots,X_d)$ be a random vector, and a.s. $X \in [0,1]^d$. Suppose that for every $a \in \mathbb{R}^d$, we know the probability distribution of the random variable $Y_a = <a,X>$. My ...
2
votes
0answers
108 views

MLRP of random variables and order statistics

Suppose we have $N$ independent random variables $X_1, \cdots, X_N$ drawn from $f_1 > \cdots > f_N$ where $f_i > f_j$ indicates that $f_i$ and $f_j$ satisfy the monotone likelihood ratio ...
11
votes
0answers
256 views

What kind of random matrices have rapidly decaying singular values?

I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the ...
1
vote
1answer
70 views

ordinary least square and random projection

Let $X$ a $d \times T$ given matrix and $M$ a $n \times d$ random matrix (say i.i.d. centered coefficients). Define $Y=MX$ in $\mathbb{R}^n$ and $H=Y'(YY')^{-1}Y$ where $'$ denotes the transpose ...
5
votes
3answers
423 views

Deconvolution of sum of two random variables

Let $Z = X + c \cdot Y$ where $X$ and $Y$ are independent random variables drawn form the same distribution given by the pdf $g()$ and $0 < c < 1$ I have observations of $Z_i$'s and thus can ...
3
votes
1answer
181 views

How to perform Importance Sampling with Prior Information

Let us define a random variable $X$ with density function $p(x)$. We wish to calculate $\mathbb{E}[f(X)] = \int f(x)p(x)dx$. We can compute the expectation by Monte Carlo simulations as ...
0
votes
0answers
81 views

Fitting distribution to spatial data

I am studying a physical process generating data which projects nicely into two dimensions with non-negative values. Each process has a (projected) track of $x$-$y$ points -- see the image below. ...
2
votes
1answer
257 views

What are the Reasons for the Ambiguous Meaning of “Distribution” in Mathematics

The term "distribution" is commonly associated with statistics and, less commonly known, to generalized functions. Questions: what is known about the origin of the term in the two fields? are the ...
0
votes
1answer
231 views

Generating independent random variable from two correlated random variables

Suppose two random variables $X$ and $V$ are given. I am wondering what kind of condition we need to impose on joint distribution of $V$ and $X$ to make sure that there exists a random variable $Z$ ...
1
vote
0answers
172 views

Incoherence of the row/column span

Due to V.Chandrasekaran., et al‎ (p.11) : In general for any $k$-dimensional subspace of $A_{n×n}$ we have that: $$\sqrt{(k/n)} \leq incoherence(A)\leq 1$$ where the lower bound is achieved (for ...
1
vote
1answer
253 views

Sum of covariance matrix of products of dependent variables

Consider the sequences of random variables $\{X_i\}_{i=1}^n$ and $\{Y_i\}_{i=1}^n$, as well as the corresponding sequence of products, $\{X_i Y_i\}_{i=1}^n$. All $X_i$ share the same mean value, ...
4
votes
2answers
241 views

Estimate on gaussian distribution

Let X be an $\mathbb R^d$-valued random variable with distribution $N_d(0,\Sigma)$. I'm looking for a function $f$ such that $$P(|X_1|\leq M, |X_2|\leq M,\dots, |X_d|\leq M)\geq f(M),$$ and such that ...
15
votes
6answers
1k views

Is a fair lottery possible?

I'm trying to come up with a scheme for a lottery where each individual has roughly the same chance of becoming the winner, regardless of the number of tickets one holds. So no individual should have ...
14
votes
2answers
389 views

How to sample uniformly from singular matrices

I would like to uniformly sample from all singular $n$ by $n$ Bernoulli matrices (that is each entry is $1$ or $0$ with probability $1/2$). I could of course just sample from all $n$ by $n$ Bernoulli ...
2
votes
0answers
73 views

sufficient statistics and isometries

Let $(M,g)$ be an infinite dimensional statistical manifold with the Fisher information metric $g$. Is it true that any isometry on this manifold must correspond to a sufficient statistic?
4
votes
1answer
167 views

Consecutive Primes mod 3

Is anything known asymptotically about the binary "primes mod 3" sequence besides Dirichlet's result that 1 and 2 occur half of the time? For example, can you prove that it does not eventually cycle ...
3
votes
0answers
185 views

Maximization of a total variation distance subject to another total variation distance in Markov chain

Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...
8
votes
2answers
352 views

Inequality in information theory

I am reading the paper "chain independence and common information" (http://ttic.uchicago.edu/~yury/papers/independ.pdf). In this paper, an inequality is used several times (without proof) which looks ...
3
votes
0answers
126 views

Quantile convergence

Let $X^1,\dots,X^n$ be a sample of (not necessarily iid) random variables and denote $$F^n(x)=\frac{1}{n}\sum_{i=1}^n \mathbf 1_{X^i\leq x}$$ the empirical distribution function. Suppose that we know ...
4
votes
1answer
122 views

Weak ergodicity of nonhomogenous products of 0-1 matrices

Here is a question which probably has a negative answer, but I couldn't find any literature directly on it. Let $(A_n)$ be a sequence of rectangular 0-1 matrices (that is, the entries are restricted ...
0
votes
0answers
77 views

Monte carlo Method to estimate a proportion

I'd like to use Monte Carlo method to estimate a proportion and I'd like to be sure my idea is correct mathematically speaking. Let a pool full of red and blue balls. I'd like to estimate the ...
5
votes
1answer
108 views

Deviation bound for the maximum of the norm of Wiener process

Let $W(t)$ be an $n$-dimensional Wiener process. Denote by $\chi_n^2$ a chi-squared random variable with $n$ degrees of freedom. I have recently found the following inequality given without proof: $$ ...
5
votes
3answers
237 views

How do you call the problem of approximating a continuous distribution with a simple discrete distribution?

The following problem came up on the Mathematica forum as "Generating a list of integers that roughly satisfy a distribution": Given $n$, find $n$ integers (possibly with duplicates) whose ...
0
votes
1answer
121 views

Expected rank of players in a Bradley-Terry round-robin tournament

Let $[n]$=$\{1,\dots,n\}$ be a set of players in a round-robin tournament. Each player $i$ has an associated skill parameter, $\lambda_{i}$, and the probability that player $i$ defeats player $j$ is ...
1
vote
0answers
88 views

What is the range of a positive random variable after whitening?

Let ${\bf x}\in\mathbb R^N$ be a positive multivariate random variable, i.e. $$x_i\in [0,\infty).$$ What is the range after whitening, i.e. the range of ${\bf y} = \sqrt{C}^{-1}{\bf x}$ with the ...
0
votes
0answers
78 views

Eigen value distribution of autocorrelated Wishart matrix

Suppose the matrix W is constructed as $W=XX^T$ where $X_i(t) = \phi_i X_i(t-1) + a_i(t)$, and $a_i(t)$ ~ $N(0,1)$. I am interested in knowing the eigen value distribution of W. My google search on ...
1
vote
2answers
266 views

Proof of Von Neumann's debiasing algorithm

Assume you have a source of random binary information that has a bias but no correlation between consecutive bits. John von Neumann describes an algorithm to debias the random source and output a ...
1
vote
0answers
59 views

Stochastic process inference from partial observations

Consider a set $U$. My signal is a piece-wise constant "function" $Sig: t \mapsto s$, i.e. the signal at time $t$ equals to some subset $s \subset U$. One can see $Sig(t)$ as a stochastic process. ...
3
votes
1answer
145 views

Equivalent method for maximum likelihood estimation of covariance parameters

My goal is to estimate the parameters of a covariance matrix $\Omega$, by maximizing the following log-likelihood function: $$\log L(\vec\tau, \rho, \sigma \mid W, X) = -m\ln(\left | \Omega \right |) ...
2
votes
2answers
319 views

What is the maximum entropy distribution on the natural numbers?

On the reals $\mathbb{R}$, the maximum entropy distribution with a given mean and variance is the Gaussian distribution. Let $\mu, \sigma > 0$. What is the maximum entropy distribution on the ...
4
votes
3answers
388 views

Online estimation of covariance matrix

I am trying to dynamically estimate the (low-dimensional) covariance matrix ${\mathbb E}[{\bf x}_t{\bf x}_t^\top]$ of a stream of data points ${\bf x}_t\in{\mathbb R}^N$ online, without any memory. ...
1
vote
0answers
112 views

Uniform Law Of Iterated Logarithm for VC classes

Kenneth Alexander proved a uniform Law Of Iterated logarithm for Vapnik-Chervonenkis classes in the article Probability Inequalities for Empirical Processes and a Law of the Iterated Logarithm (Ann. ...
1
vote
0answers
61 views

Small ball probabilities for functions of correlated normals

Let $f : \mathbb{R}^k \rightarrow \mathbb{R}$ and let $X$ be distributed k-dimensional normal with mean $0$ (with "arbitrary" covariance matrix). I am looking for references with bounds of the form: ...
16
votes
1answer
741 views

Gini Coefficient and Renyi Entropy

Gini coefficient (aka Gini Index) is a quantity used in economics to describe income inequality. It is 0 for uniformly distributed income, and approaches 1 when all income is in hands of one ...
3
votes
1answer
140 views

Concentration rates for the posterior distribution

Sanov's theorem and Dvoretzky–Kiefer–Wolfowitz's inequality tell us how fast the empirical distribution concentrates around the true underlying probabilty distribution. What is known about the ...
4
votes
2answers
206 views

Bounding the tail of an average using the the tail of individual members

Let $X_1,X_2,\ldots,X_n$ be an i.i.d. sequence of $n$ positive random variables with mean $E[X_1]=\mu_X<\infty$ and the second moment $E[X_1^2]=\infty$. I am interested in upper-bounding ...
4
votes
1answer
153 views

Statistical models in terms of families of random variables

A statistical model is a function $P : \Theta \to \Delta(X)$, where $\Theta$ is a parameter space, and $\Delta(X)$ is the set of probability measures on a state space $X$. Suppose that $\Theta$ and ...
0
votes
1answer
251 views

Size of KL-divergence neighbourhoods

I am new here. I was reading another post here and this got me wondering what can be said about the size of the following kl divergence neighborhoods. Consider these two kl-divergence neighbourhood ...
-2
votes
1answer
118 views

Forms of multivariate CLT [closed]

I am looking for a good reference for differnt kinds of multivariate central limit theorems. I was wondering how far the i.i.d. condition of the standard multivariate clt can be relaxed, as in can the ...
4
votes
0answers
115 views

Pettis Integrability and Laws of Large Numbers

Let $(\Omega, \mathcal F, \mathbb P)$ be a probability space, and let $V$ be a topological vector space with a dual space that separates points. Let $v_n : \Omega \to V$ be a sequence of Pettis ...
0
votes
1answer
84 views

Independence of Eigenvalues of Wishart

This question regards a previous post, but it is not immediately obvious the two are related, so I ask it anyways: are the eigenvalues of a Wishart matrix $\mathbf{S}$ $=$ ...
5
votes
3answers
362 views

Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables

I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables. Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
4
votes
0answers
242 views

Probability distribution function for singular value sum of Gaussian random matrix

Let $\mathbf{X}$ be an $N \times N$ random matrix with IID Gaussian entries. They can be standard normal, but $N$ is not large: that is $N$ $<$ 6, typically. Call its singular value decomposition ...
2
votes
1answer
449 views

Order statistics of independent NOT identically distributed random variables

I want to find the p.d.f of the n-th order statistics from a set of independent, but NOT identically distributed random variables $X_1, \dots, X_n$ (the p.d.f. of the $X_i$'s is at hand)
2
votes
1answer
285 views

Is this a closed set?

Let $\Theta$ and $X$ be two (Hausdorff) topological spaces. Let $\mathbb P : \Theta \to \Delta(X)$ be a "statistical model", i.e., a continuous function from parameter space $\Theta$ to the space of ...
3
votes
1answer
119 views

Variance of central limit distribution for $P(x) \sim 1/x^{1+\alpha}$ for finite but large $N$?

Is it known what the next-to-leading order term is in the variance of the central limit distribution for the average of $N$ variables each of which is distributed according to $P(x) \sim ...
2
votes
1answer
354 views

Central limit theorem for $P(x)\sim 1/x^3$ distribution

I have a random variable $x \in (0,\infty)$ with distribution $P(x)$ falling off slowly $P(x) \sim 1/x^3$ for large $x$. So the expectation value $\bar{x}$ is finite but the second moment $\bar{x^2}$ ...
2
votes
1answer
233 views

Maximum of a sequence of $n$ positive random variables where variance is an increasing function of $n$

Suppose I have a sequence of $n$ i.i.d. random variables $X_1,X_2,\ldots,X_n$. Each $X_i$ is positive and has variance $\sigma(n)$ that is an increasing function of the number of variables in the ...
9
votes
2answers
277 views

“Fractional sampling” from a probability distribution

My question concerns an operation on probability distributions which has arisen in some applied research. It is well-defined mathematically (at least in a limited context), but I don't know how to ...