All Questions
Tagged with pr.probability st.statistics
102 questions
4
votes
3
answers
1k
views
Incremental entropy computation
After a quick internet search I found no method for incremental entropy computation.
Question 1
Let $\{x_i\}_{i=1}^n$ and $\{x_i\}_{i=1+n}^{n+m}$ be two samples and let $S_i^j:=\sum_{k=i}^j x_k$. ...
- 161
4
votes
2
answers
349
views
Does the average of correlated Gaussian random variables with mean zero and different variances converge in probability to their mean?
Let $X_i\sim N(0,\sigma_i^2)$ and $\operatorname{Corr}(X_i,X_j)>0$. Is it possible to show that $$\frac{1}{N} \sum_{i=1}^N X_i \overset{p}\rightarrow E[X_i]=0.$$ Do you have a reference to a law of ...
4
votes
2
answers
1k
views
expected values over binomial distributions
In some works of economics/risk analysis etc., I have seen situations where people take the expected value of a function (such as a utility function/cost function) over a binomial distribution:
$$F(n)...
- 7,320
4
votes
0
answers
309
views
When is $\prod_{i=0}^\infty (I-x_i x_i^T)=0$ for zero-centered Gaussian $x_i$?
Suppose $x_i\in \mathbb{R}^d$ is sampled IID from $\mathcal{N}(0,H)$. Let $A_i=(I-x_i x_i^T)$ and assume $d$ is large. What are necessary conditions for the following to converge with probability 1?
$...
- 2,252
4
votes
1
answer
168
views
Existence of copula bound pointwise strictly smaller than the Fréchet-Hoeffding upper bound
Consider bivariate copulas $C_1$ and $C_2$ with $\max\{C_1(u,v), C_2(u,v)\}< M_2(u,v)$ for all $u,v \in(0,1)$, where $M_2(u,v) := \min\{u,v\}$ is the Fréchet-Hoeffding upper bound.
Is there a ...
- 143
4
votes
1
answer
287
views
Local maxima of the sum of Gaussian functions in *multiple dimensions* are always strict local maxima - prove/disprove/prove conditionally?
This is a follow up of the question in one dimension, that asked to show that the all the maxima of the sum of Gaussian
$$f_n(x):= \sum_{i=1}^{n}e^{-(x-x_i)^2}, x_1 < x_2 < \dots < x_n$$
are ...
- 1,467
4
votes
1
answer
1k
views
Does variants of Bernstein and Freedman concentration inequalities exist with NO uniform bound on the range of RV or martingale differences
A classic formulation of the Bernstein inequality (from Wikipedia) is as follow:
Let $X_1, \ldots, X_n$ be independent zero-mean random variables. Suppose that $|X_i|\leq M$ almost surely, for all $i$...
- 53
4
votes
0
answers
188
views
Distributions over permutation groups $\mathcal{S}_n$
Partly inspired by recent developments in enumeration of pattern avoiding permutations, which is known to be connected with Brownian excursions [Hoffman&Rizzolo]. The exciting milestone is the ...
- 8,071
4
votes
0
answers
970
views
Expected operator norm of inverse Wishart matrix
Let $ W\sim W_p(n,I)$ be a white $p\times p$ Wishart matrix, and assume $n>p+1$, which ensures that $W$ is invertible almost surely. Let $\|W^{-1}\|_{\text{op}}$ be the operator norm (maximum ...
- 41
3
votes
1
answer
473
views
Expected value of the maximum of the periodogram
Let us suppose that $X_1,\ldots,X_n$ with $n\ge1$ are iid random variables such that $\operatorname EX_1=0$ and $\operatorname E|X_1|^s<\infty$ with some $s>2$ and define the DFT of $X_1,\ldots,...
- 423
3
votes
1
answer
484
views
What is known about the distribution of the errors in empirical approximation of a CDF?
Let $X_1,X_2,\ldots,X_n$ be i.i.d. random variables in $\mathbb{R}$ with common cumulative distribution function (CDF) $F(x)$. The empirical approximation to $F(x)$ is defined as follows:
$$\hat{F}...
- 907
3
votes
0
answers
494
views
Maximization of a total variation distance subject to another total variation distance in Markov chain
Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...
- 1,109
3
votes
3
answers
203
views
$\mathbf{y}=f(\mathbf{x},\mathbf{z})=g(\mathbf{x})$ if $\mathbf{z}\perp \!\!\! \perp \{\mathbf{y},\mathbf{x}\}$ jointly?
Let $\mathbf{y},\mathbf{x},\mathbf{z}$ are real-valued random vectors with possibly different dimensions. Assume $\mathbf{y}=f(\mathbf{x},\mathbf{z})$ for some function $f$.
If $\mathbf{z} \perp\!\!\!\...
- 193
3
votes
1
answer
243
views
Independent input feature z can be removed: if y=f(x+z,z), then y=g(x)?
Let $y\in \mathbb{R}$ and $\mathbf{x},\mathbf{z}\in\mathbb{R}^p$ be random variable and random vectors. Assume $y=f(\mathbf{x}+\mathbf{z},\mathbf{z})$ for some function $f$.
Is the following statement ...
- 193
3
votes
1
answer
87
views
Optimal linear measurement operator
Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...
- 6,541
3
votes
1
answer
206
views
Asymptotic results for smallest gap of Gaussian random matrix
For a symmetric Gaussian random matrix $G=\{G\}_{1\le i,j \le n}$ with iid $E[G_{ij}]=0$ and $E[G_{ij}^2]=1/n$ (it is normalized), ordering its eigenvalues $\lambda_1\le \lambda_2\le\cdots \lambda_n$.
...
- 288
3
votes
1
answer
607
views
Show that $\sup_{\|g\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n g(Z_i)\right|\rightarrow_{\text{a.s.}}0.$ when $\delta_n\rightarrow 0$?
UPDATE: The result below can be understood as an almost sure stochastic equicontinuity condition. I don't know of any result establishing primitives of almost sure stochastic equicontinuity. If you ...
- 59
3
votes
3
answers
244
views
Example of a (strictly) proper scoring rule on a general measurable space?
Most of the literature on scoring rules that I know of deals with discrete measurable spaces, but in this paper by Gneiting and Raferty a very general definition of a scoring rule is given. I don't ...
- 869
2
votes
1
answer
3k
views
Empirical estimator fot the total variation distance on a finite space
I have two probability measures $p$ and $p'$ on a finite set $X$ which I do not know precisely, but which I can sample from. I would like to estimate their total variation (omitting multiplier $2$):
$$...
- 1,655
2
votes
1
answer
936
views
Upper-bound for spectral norm of the covariance matrix of a certain Gaussian vector with correlated entries
Let $n$ and $m$ be large positive integers. Let $x=(x_1,\ldots,x_n)$ be a vector of independent random variables from $N(0,1)$. It is clear that the covariance matrix of $x$ is $I_n$, the identity ...
- 6,853
2
votes
2
answers
739
views
Multinomial transformation for matrices
Suppose we have a vector of probabilities $\mathbf{p}=(p_1,...,p_n)$, where $p_i>0$ for $i=1,...n$ and $\sum p_i=1$. Define new vector $\mathbf{r}=(r_1,...,r_{n-1})$ in a following way:
$r_i=\log(...
- 203
2
votes
2
answers
2k
views
Dependence between direction and magnitude of multivariate normal random vector
Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then
\begin{align}
f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^...
- 269
2
votes
1
answer
645
views
Estimating the variance of error in empirical approximation to a distribution
Let $X_1,X_2,\ldots,X_n$ be i.i.d. random variables in $\mathbb{R}$ with common cumulative distribution function (CDF) $F(x)$. The empirical approximation to $F(x)$ is defined as follows:
$$\hat{F}...
- 907
2
votes
1
answer
170
views
Law of large numbers for a continuum of Bernoullis
Suppose I have a family of $n$ independent Bernoulli random variables described by a vector of parameters $(p_i)_{i=1}^n$. As it is well known, the number of successes within this family is a random ...
2
votes
1
answer
378
views
Distribution of the Gram matrix
Let $\mathbf{X}$ be an $m\times k$ random matrix ($m>k$) of rank $k$, having the density function $f_\mathbf{X}(X)$. What is the distribution of $\mathbf{Y}=\mathbf{XX}^T$? Basically my question is ...
- 141
2
votes
2
answers
322
views
Integral of product of Hermite polynomials w.r.t marginal distribution of first two-coordinate of random vector on unit-sphere
This question is related to: https://math.stackexchange.com/q/4270522/168758
Let $H_n(x) \in \mathbb R[x]$ be the probabilist's $n$th Hermite polynomial. This an $n$th degree polynomial given by the ...
- 6,853
2
votes
2
answers
690
views
Concentration and anti-concentration of gap between largest and second largest value in Gaussian iid sample
Let $n \ge 3$ be an integer and let $X=(X_1,\ldots,X_n)$ be random vector with iid coordinates from $N(0,1)$. For $1 \le k \le n$, let $X_{(k)}$ be the value of the $k$th largest coordinate of $X$.
...
- 6,853
1
vote
1
answer
338
views
Expected values of two non-negative, integer-valued random variables related to an urn problem
Consider an urn containing $c$ distinguishable balls, $\alpha$ of which are red, $\beta$ of which are blue, and $\gamma$ of which are green, and $\alpha+\beta+\gamma=c$. We assume $\alpha,\beta,\gamma&...
- 145
1
vote
1
answer
341
views
Form of minimax estimator
Let $\Delta$ be the set of all probability distributions over $\mathbb{N}=\{1,2,\ldots\}$ and fix some $\mathcal{P}\subseteq\Delta$.
Suppose additionally that $\Delta$ is endowed with some norm $||\...
- 6,541
1
vote
2
answers
237
views
Fair partitioning of a set - Weighted sums of Bernoullis
For $n$ an integer, let $a_n$ be the number of ways in which one may partition the set $\{1, \ldots, 2n \}$ in two parts with:
the same number of elements: $n$
and the same sum: $2n(2n+1)/4$.
...
- 468
1
vote
1
answer
385
views
How fast does this Gaussian random walk move away from the origin?
Suppose $z_i$ are IID zero-centered $d$-dimensional Gaussian random variables with unit-trace covariance $\Sigma$ and $g(z_i)$ is the sum of its components.
Consider the following random walk:
$$x_s=\...
- 2,252
1
vote
1
answer
1k
views
Null hypothesis test for independent but not identically distributed samples
I'm trying to figure out the best statistical test to use for an edge case I've run into: trying to figure out the likelihood of the null hypothesis for a set of samples that each (potentially) come ...
- 111
1
vote
1
answer
269
views
Can I prove that a polynomial representing the 4th moment of a weighted-sum of random variables is a sos?
I am looking at the 4th central moment of a weighted-sum of correlated random variables, which takes the form
$$\mu_4 = \sum_{i,j,k,l=1}^n w_i w_j w_k w_l \mu_{ijkl}$$
where $\mu_{ijkl}$ are the ...
- 173
1
vote
1
answer
216
views
Rademacher complexity of function class $(x,y) \mapsto 1[|yf(x)-\alpha| \ge \beta]$ in terms of $\alpha$, $\beta$, and Rademacher complexity of $F$
Let $X$ be a measurable space and let $P$ be a probability distribution on $X \times \{\pm 1\}$. Let $F$ be a function class on $X$, i.e., a collection of (measurable) functions from $X$ to $\mathbb R$...
- 6,853
1
vote
1
answer
694
views
Rademacher complexity of a Lipschitz class: Are the boundedness constraints necessary?
Consider the following function class: $F={f:R^d\rightarrow [a,b], f(x)=\sigma(w^Tx)}$ where $\sigma(.)$ is Lipschitz, and $w\in R^d$ is a parameter vector. The problem I'm working on is a machine ...
- 517
1
vote
1
answer
151
views
Noncentral matrix beta distributions of type I and II
In Gupta & Nagar's book Matrix variate distributions, the noncentral Beta type I(B) distribution with parameters $a$, $b$ and noncentrality parameter $\Theta$ is defined by $U={(S_1+S_2)}^{-\...
- 2,560
1
vote
1
answer
202
views
A problem related to stochastic ordering
Let $\boldsymbol{X} = (X_1,X_2)^{\rm T}\sim \mathcal{N}_2(\boldsymbol{\mu}, \mathrm{\Sigma})$, where
\begin{eqnarray*}
\boldsymbol{\mu} = (\mu_1, \mu_2)^{\rm T}& = &(\sqrt{\xi_1\xi_2/(\xi_1+\...
- 211
1
vote
1
answer
88
views
Independence of r.v.'s following a distribution that is the ratio between complex Gaussian and Chi-square r.v.'s
Given the following two R.V.s
$$z_1 = \frac{x_1}{|x_1|^2 + |x_2|^2 + \cdots + |x_M|^2}$$
and
$$z_2 = \frac{x_2}{|x_1|^2 + |x_2|^2 + \cdots + |x_M|^2}$$
where $x_i \sim \mathcal{CN}(0,a), \forall i$...
1
vote
0
answers
463
views
How far away is the maximum of $n$ i.i.d. chi-squared random variables from the rest of the sequence as $n$ gets large?
Suppose that I have a sequence of $n$ i.i.d. chi-squared random variables with $k$ degrees of freedom $X_1, X_2, \ldots, X_n$, and denote $X_{\max}=\max(X_1, X_2, \ldots, X_n)$. Let $k$ be increasing ...
- 907
1
vote
0
answers
78
views
Canonical representation of the a probability distribution for Hammersley Clifford Theorem
I'm reading the following paper
http://www2.stat.duke.edu/~scs/Courses/Stat376/Papers/GibbsFieldEst/BesagJRSSB1974.pdf
On page 7 they give the result that
$$Q(\textbf{x}) = \sum_{1 \leq i \leq n} ...
- 903
1
vote
1
answer
415
views
Approximate the singular values of a certain random dot-product kernel matrix (in the sense of El Karoui, Cheng-Singer, etc.)
Let $g:\mathbb R \to \mathbb R $ be a continuous function which is
"sufficiently smooth" (e.g $\mathcal C^3$) around $0$, and
"sufficiently integrable" (e.g integrable w.r.t $N(0,...
- 6,853
0
votes
1
answer
266
views
CDF of a log-concave discrete random variable
In the continuous setting, it's known that if a density function is log-concave , then its CDF is also log-concave.
My questions:
What can we say about this in the discrete setting?. For ex: Is the ...
- 135
0
votes
1
answer
822
views
The distribution of the sum of inner products of two independent complex normal vectors
If I have $\mathbf x_n=[x_0, x_1,... ,x_K]^T$ and $\mathbf y_n=[y_0, y_2, ..., y_K]^T$, where $x,y\sim\mathcal C\mathcal N(\mathbf 0,\sigma^2\mathbf I)$.
What is the distribution of the following ...
- 105
0
votes
1
answer
165
views
About another potential characterization of normal numbers
Normal numbers, in a nutshell, are real numbers that have a "uniform" distribution of digits in standard numeration systems (binary, decimal, and so on.) You can find a formal definition and ...
- 3,098
0
votes
1
answer
496
views
Laplace transform inversion
I have a probability distribution that is defined through it's Laplace transform by :
$$L(t) = \mathbb E(e^{-tX}) = e^{1 - \frac{1+t}{t}\ln(1+t)}$$
Using R and the invLT package, i have a numerical ...
- 686
0
votes
0
answers
213
views
Behavior of the sum of the exponents of chi-squared random variables normalized by their maximum
Let $X_1,X_2,\ldots,X_n$ be a sequence of $n$ i.i.d. chi-squared random variables with $k$ degrees of freedom, and denote by $X_\max$ the maximum of this sequence. Furthermore, let $k=\omega(1)$ ...
- 907
0
votes
1
answer
107
views
Bounding $\|X_1/(X_1+X_2) - Y_1/(Y_1+Y_2)\|_p$ by the closeness of $X$ and $Y$
This question is inspired by the answer to this other question, but I have tried to make it self-contained and to zoom in on the counter-example from this answer.
Suppose $\{(X_n, Y_n)\}_{n=1}^2$ are ...
- 15
0
votes
2
answers
534
views
Wasserstein distance between $N(0,1/d)$ and the marginal distribution of $x_1$ when $x=(x_1,\ldots,x_d)$ is uniform on the unit-sphere in $R^d$
Let $x=(x_1,\ldots,x_d)$ be uniformly distributed on the unit-sphere in $\mathbb R^d$.
Question.
What is a good upper-bound for Wasserstein distance between $N(0,1/d)$ and the marginal distribution ...
- 6,853
0
votes
1
answer
208
views
Local behavior of the Vandermonde convolution
An interesting combinatorial identity is the Vandermonde convolution identity:
$$ \sum_k {n\choose k}{m\choose s-k} = {n+m \choose s},$$
which can be proved by considering the coefficients in $(x+1)^{...
- 5,230
0
votes
2
answers
341
views
Conditions for existence of a distribution with full support
Consider a $6\times 1$ continuous random vector
$$
\eta\equiv (\eta_1,\eta_2,..., \eta_6)
$$
satisfying the following property:
$$
\underbrace{\begin{pmatrix}
\eta_1\\
\eta_2\\
\eta_3
\end{pmatrix}}_{\...
- 108