Applied, computational and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments, case studies.

learn more… | top users | synonyms

7
votes
1answer
254 views

How similar are discrete stable RVs to their continuous analogues?

The generalized central limit theorem of Gnedenko-Levy describes the asymptotic behavior of a sum of IIDRVs which may not have finite mean or variance. Only a small class of limit laws can be ...
-1
votes
0answers
34 views

Maximum chi-square distance between norm vectors [closed]

What is the maximum possible chi-square distance between two normalized vectors? The representation of chi-square distance is below. $d(x,y) = \sum_i \frac{(x_i-y_i)^2}{x_i+y_i}$
2
votes
2answers
91 views

Bounds for the fat tail after trimming the mean?

I am interested in the quantity $$f(X,t) = \int_t^\infty\negthinspace x\ p(x)\ dx,$$ where $p$ is a probability distribution for a positive variable $X$. 1) Does this quantity $f(X,t)$ have a name? ...
0
votes
1answer
236 views

Expected value with a kronecker product and Gaussian distributional assumption

What is the expected value, $ \mathbb{E}\left[ I \otimes \left( \operatorname{diag}(ZZ^T\mathbf{1}) - ZZ^T\right)\right]$ where $Z \sim N(0, \sigma^2I) $? The kronecker product is where the confusion ...
1
vote
0answers
46 views

Whether r.v. with p.g.f. $\exp [\sum\limits_{i = 1}^\infty {{q_i}({z^i}} - 1)]$ is overdispersion?

When discrete r.v. $X$ is not Poisson distributed and ${\rm{Var}}X,EX < \infty $, I want to know whether r.v. $X$ with p.g.f. $\exp [\sum\limits_{i = 1}^\infty {{q_i}({z^i}} - 1)],({q_i} \in ...
1
vote
2answers
729 views

Question on Maximum Likelihood Estimation

My question concerns using Maximum Likelihood to estimate unknown parameters. I will sincerely appreciate if anyone can help me find out if my approach is flawed. Assume we have a random vector $V$, ...
0
votes
0answers
30 views

Variance of the estimator of a geometric distribution [closed]

Suppose I have $N$ geometric random variables, $x_1, x_2, ..., x_N$, which the following probability mass function $ f(x) = (1 - p)^{x-1} p$ The maximum likelihood estimation of $p$ will be ...
1
vote
1answer
47 views

Can I test many p-values with KS or AD

(Sorry if this is a noob question. I'm a mathematician learning statistics.) I would like to know if it's sound (or advisable) to test many p-values against the continuous uniform distribution using ...
1
vote
2answers
109 views

Empirical estimator for total variation distance between two product distributions

Let $X = (X_1, X_2, \ldots , X_n)$ be an $n$-dimensional random variable, where each $X_i$ is a random variable on finite discrete set $S$. In addition, $X_i$ are independent of each other (but not ...
4
votes
0answers
120 views

Optimization problem involving Multivariate Normal

I use $\phi(t)$ to describe the standard normal distribution density and $\Phi(t)$ as the normal distribution CDF and would like to prove that for all $n\geq3$, the function: ...
0
votes
1answer
160 views

Size of KL-divergence neighbourhoods

I am new here. I was reading another post here and this got me wondering what can be said about the size of the following kl divergence neighborhoods. Consider these two kl-divergence neighbourhood ...
0
votes
2answers
76 views

What is the likehood function in the noise free observation case

In the nonlinear Bayesian Tracking problem, if we consider the noise exists only in the state equation : x[k] = f(x[k-1],v[k-1]) where vk-1 here is an iid process noise sequence And we suppose that ...
4
votes
3answers
409 views

Incremental entropy computation

After a quick internet search I found no method for incremental entropy computation. Question 1 Let $\{x_i\}_{i=1}^n$ and $\{x_i\}_{i=1+n}^{n+m}$ be two samples and let $S_i^j:=\sum_{k=i}^j x_k$. ...
0
votes
0answers
13 views

How to generalize uncertainty coefficient to set-valued classes?

This question is the reason I asked How to estimate the entropy of a distribution on a power set? Proficiency (AKA uncertainty coefficient) is an information-theoretic measure of predictor quality, ...
1
vote
3answers
89 views

How to estimate the entropy of a distribution on a power set?

Given a probability distribution $(X,p)$, its entropy is defined as $H=-\sum_{x\in X} p(x)\log p(x)$. Given a sample of observations $x_n,n=1..N$, one can estimate $p(x)=\frac{\#\{i:x_i=x\}}{N}$ and ...
4
votes
7answers
2k views

Uniformly Sampling from Convex Polytopes

How to choose a point uniformly from a convex polytope $P \subset [0,1]^n$ defined by some inequalities, Ax < b ? (Here A is an m-by-n matrix, x is n-by-1 and b is m-by-1.) I imagine that you ...
0
votes
1answer
46 views

Finiteness of “novel variance” from a kernel on a compact space [closed]

Let $c(i,i')$ be a kernel function on a reasonable index space $I$. Choose a dense sequence of points $\{i_1, i_2, \cdots \} \subseteq I$, and define the one-point kernel functions $k_n := c(\cdot, ...
-4
votes
1answer
70 views

Is it possible to determine if these random numbers are not really random? [closed]

I've been given a big ordered list of integer numbers. Looks like this : 10 -11 -3 -6 -10 -1 ..... ..... ..... Allegedly, these values are random from -12 to +12 However, there has been ...
5
votes
1answer
163 views

Measures which exhibit the “uncorrelated implies independent” property

Let $X$ be a topological linear space, and let $X^*$ be its dual space. Suppose that $X$ is complete and Hausdorff, and $X^*$ separates points. Let $Y$ be another such space, and let $f : X \to Y$ be ...
0
votes
1answer
69 views

Understanding the derivation of a ML-estimator (statistics)

I'm trying to understand the derivation of a ML-estimator and more specifically the rewriting of the covariance matrix $\Sigma$. In this rewriting, a lemma is used to show that: $$ \tag{1} ...
5
votes
2answers
425 views

Concentration of sum of pairwise squared Euclidean distances of random vectors

Let $X_1, \ldots, X_n $ be independent random vectors in $B(0, D) \subset R^d$ ($\ell_2$ ball of radius $D$ centered at the origin). I am trying to find the concentration of the following quantity ...
1
vote
0answers
45 views

forward algorithm Hidden Markov Model

I am studying the the forward-backward algorithm used in Hidden Markov Models. I understand that that you are trying to propagate through a sequence (and the available states) to find the most ...
13
votes
4answers
453 views

Are gaussians with different moments far in total variation distance?

If two Gaussians disagree on one moment, it seems like this should imply that they have a large variation distance--equivalently, if two Gaussians are close in variation distance it's hard for their ...
0
votes
1answer
55 views

Estimating the variance of error in empirical approximation to a distribution

Let $X_1,X_2,\ldots,X_n$ be i.i.d. random variables in $\mathbb{R}$ with common cumulative distribution function (CDF) $F(x)$. The empirical approximation to $F(x)$ is defined as follows: ...
1
vote
1answer
65 views

What is known about the distribution of the errors in empirical approximation of a CDF?

Let $X_1,X_2,\ldots,X_n$ be i.i.d. random variables in $\mathbb{R}$ with common cumulative distribution function (CDF) $F(x)$. The empirical approximation to $F(x)$ is defined as follows: ...
2
votes
2answers
63 views

estimating variance of dependent normal distributed data

Let $X_{ij}$ with $1\leq i<j\leq n$ (that are $X_{12},\dots, X_{1n},\dots,X_{(n-1)n}$) be ${n \choose 2}$ identically normal distributed $N(0,\sigma^2)$ such that $ \text{corr}(X_{ij},X_{rs})=\rho ...
1
vote
0answers
52 views

Question in Wainwright's paper about signed support recovery in lasso

Sharp thresholds for high dimensional and noisy sparsity recovery using $l_1$ constrained quadratic programming (Lasso) This paper is about support recovery guarantees of the Lasso. I have an issue ...
1
vote
1answer
50 views

ordinary least square and random projection

Let $X$ a $d \times T$ given matrix and $M$ a $n \times d$ random matrix (say i.i.d. centered coefficients). Define $Y=MX$ in $\mathbb{R}^n$ and $H=Y'(YY')^{-1}Y$ where $'$ denotes the transpose ...
14
votes
2answers
368 views

How to sample uniformly from singular matrices

I would like to uniformly sample from all singular $n$ by $n$ Bernoulli matrices (that is each entry is $1$ or $0$ with probability $1/2$). I could of course just sample from all $n$ by $n$ Bernoulli ...
4
votes
3answers
154 views

Online estimation of covariance matrix

I am trying to dynamically estimate the (low-dimensional) covariance matrix ${\mathbb E}[{\bf x}_t{\bf x}_t^\top]$ of a stream of data points ${\bf x}_t\in{\mathbb R}^N$ online, without any memory. ...
2
votes
1answer
51 views

Why does differencing create wide-sense stationary time series?

In time series analysis, a common assumption made is that the series is wide-sense stationary, ex. that it has time invariant mean and covariance. However, as this is often not the case in real life, ...
3
votes
2answers
84 views

expectation of log(x+a) when X follows a beta distribution

Is there a closed form expression for the expectation of $\log(x+a)$ (with $a>0$, the case $a=0$ is obvious) when X follows a beta distribution?
2
votes
0answers
48 views

Smallest distribution of points with genuinely different clusterings

An hierarchical clustering algorithm for (finite) sets of points in a given metric space is essentially determined by its linkage criterion, which defines the distance between arbitrary (finite) sets ...
0
votes
2answers
86 views

Determine joint distribution from projections

Let $X=(X_1,\dots,X_d)$ be a random vector, and a.s. $X \in [0,1]^d$. Suppose that for every $a \in \mathbb{R}^d$, we know the probability distribution of the random variable $Y_a = <a,X>$. My ...
2
votes
0answers
98 views

MLRP of random variables and order statistics

Suppose we have $N$ independent random variables $X_1, \cdots, X_N$ drawn from $f_1 > \cdots > f_N$ where $f_i > f_j$ indicates that $f_i$ and $f_j$ satisfy the monotone likelihood ratio ...
5
votes
3answers
312 views

Deconvolution of sum of two random variables

Let $Z = X + c \cdot Y$ where $X$ and $Y$ are independent random variables drawn form the same distribution given by the pdf $g()$ and $0 < c < 1$ I have observations of $Z_i$'s and thus can ...
9
votes
0answers
193 views

What kind of random matrices have rapidly decaying singular values?

I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the ...
12
votes
2answers
466 views

Covariance of INID order statistics [closed]

In the IID case, it is known that all order statistics are positively correlated.* Thus, we know that $$\text{Cov}(X_{(i)},X_{(j)}) \geq 0.$$ Is this known in the INID (independent, non-identically ...
1
vote
0answers
148 views

Incoherence of the row/column span

Due to V.Chandrasekaran., et al‎ (p.11) : In general for any $k$-dimensional subspace of $A_{n×n}$ we have that: $$\sqrt{(k/n)} \leq incoherence(A)\leq 1$$ where the lower bound is achieved (for ...
3
votes
1answer
147 views

How to perform Importance Sampling with Prior Information

Let us define a random variable $X$ with density function $p(x)$. We wish to calculate $\mathbb{E}[f(X)] = \int f(x)p(x)dx$. We can compute the expectation by Monte Carlo simulations as ...
1
vote
2answers
1k views

Inversion of Moment-generating functions (aka Laplace transform of prob dist)

I want to embark on a project about inverting a Moment-generating function of a probabilitiy distribution. That is given by \begin{equation} M_X(t) = \text{E} \exp(tX) \end{equation} Since I have ...
-1
votes
1answer
183 views

Generating independent random variable from two correlated random variables

Suppose two random variables $X$ and $V$ are given. I am wondering what kind of condition we need to impose on joint distribution of $V$ and $X$ to make sure that there exists a random variable $Z$ ...
2
votes
1answer
375 views

Multinomial transformation for matrices

Suppose we have a vector of probabilities $\mathbf{p}=(p_1,...,p_n)$, where $p_i>0$ for $i=1,...n$ and $\sum p_i=1$. Define new vector $\mathbf{r}=(r_1,...,r_{n-1})$ in a following way: ...
0
votes
0answers
70 views

Fitting distribution to spatial data

I am studying a physical process generating data which projects nicely into two dimensions with non-negative values. Each process has a (projected) track of $x$-$y$ points -- see the image below. ...
15
votes
1answer
631 views

Gini Coefficient and Renyi Entropy

Gini coefficient (aka Gini Index) is a quantity used in economics to describe income inequality. It is 0 for uniformly distributed income, and approaches 1 when all income is in hands of one ...
2
votes
1answer
239 views

What are the Reasons for the Ambiguous Meaning of “Distribution” in Mathematics

The term "distribution" is commonly associated with statistics and, less commonly known, to generalized functions. Questions: what is known about the origin of the term in the two fields? are the ...
1
vote
1answer
207 views

Sum of covariance matrix of products of dependent variables

Consider the sequences of random variables $\{X_i\}_{i=1}^n$ and $\{Y_i\}_{i=1}^n$, as well as the corresponding sequence of products, $\{X_i Y_i\}_{i=1}^n$. All $X_i$ share the same mean value, ...
1
vote
0answers
51 views

Stochastic process inference from partial observations

Consider a set $U$. My signal is a piece-wise constant "function" $Sig: t \mapsto s$, i.e. the signal at time $t$ equals to some subset $s \subset U$. One can see $Sig(t)$ as a stochastic process. ...
15
votes
6answers
985 views

Is a fair lottery possible?

I'm trying to come up with a scheme for a lottery where each individual has roughly the same chance of becoming the winner, regardless of the number of tickets one holds. So no individual should have ...
4
votes
2answers
222 views

Estimate on gaussian distribution

Let X be an $\mathbb R^d$-valued random variable with distribution $N_d(0,\Sigma)$. I'm looking for a function $f$ such that $$P(|X_1|\leq M, |X_2|\leq M,\dots, |X_d|\leq M)\geq f(M),$$ and such that ...