All Questions
Tagged with pr.probability st.statistics
1,134 questions
1
vote
0
answers
39
views
Can conditional distributions with respect to a sufficient sub-$\sigma$-algebra be represented by a single Markov kernel?
Let $(\Omega, \mathcal{F})$ be a measurable space, and let $\mathcal{P}$ be a collection of probability measures on this space. A sub-$\sigma$-algebra $\mathcal{G} \subset \mathcal{F}$ is said to be ...
- 111
9
votes
2
answers
429
views
Hermite–Fourier expansion for the median
Let $n$ be an odd positive integer. Let $M : \mathbb{R}^n \to \mathbb{R}$ be the median function: $M(x_1,\dots,x_n)$ is the median of $x_1,\dots,x_n$. What can be said about the Hermite–Fourier ...
- 24.7k
3
votes
0
answers
92
views
Tighter Freedman's inequality for a special martingale difference sequence
Let $X_{1}, \ldots, X_{T} \in \{0, 1\}$ be a sequence of Boolean random variables with
$$
\mathbb{E}[X_{t} | X_{1}, \dots, X_{t - 1}] = p_{t}.
$$
Consider the sequence $Y_{t} := X_{t} - p_{t}$ (which ...
- 41
2
votes
1
answer
208
views
Proving an exponential sum inequality for symmetric Hamming distance sequences in binary vectors
Background: Let $X = \{0,1\}^k$ represent the set of all binary vectors of length $k$. For two binary vectors $x, y \in X$, the Hamming distance $d_H(x, y)$ is defined as the number of positions where ...
- 349
16
votes
1
answer
397
views
Examples of problems in statistics accessible only using information geometry
I am just curious if there are some examples of problems in statistics that are indeed accessible using information geometry while proofs completely avoiding geometry are unknown. In other words, ...
- 269
0
votes
0
answers
44
views
Large Deviation Principle for an adaptive sampling rule for Multi Armed Bandits
Consider the following adaptive strategy for sampling from a Multi Armed Bandit with $K$ arms:
Split the $T$ rounds into $N (\in \mathbb{N})$ disjoint intervals. Each interval is indexed by $i=1,2,\...
- 9
1
vote
0
answers
48
views
Quantile maximization of the difference of random constrained quadratic optimization problems
I am interested in understanding the family of parametrized random variables defined by the pushforward map
$$
\lambda_x : \varepsilon \mapsto \underset{z_1 \in \mathbb{R}^n :\, h^T z_1 = 0, \; z_1 \...
- 11
0
votes
1
answer
100
views
Expressing a multivariate normal distribution as a mixture of uniform distributions?
Context: Given a scalar normal distribution $X\sim \mathrm{N}(\mu, \sigma^2)$, it is possible to express $X$ as a mixture of uniform distributions over intervals (compound probability distributions), ...
2
votes
0
answers
77
views
Inequalities concerning cummulative distributions of binomials
For random variable $Z$, let $F_Z$ denote its cdf, i.e., $F_Z(t)=\mathbb{P}(Z\leq t)$. Let $X$ be a binomial distribution with parameters $(n,p)$ and $Y$ a binomial distribution with parameters $(m,p)$...
- 121
1
vote
0
answers
61
views
Bound on $\int_0^1\sqrt{\log N_{[]}(\varepsilon,\mathcal{F},d)} \, d\varepsilon$ over the class of half-spaces $\mathcal{F}$ on $\mathbb{R}^d$?
For a class of functions $\mathcal{F}$ and a pair $f,g\in\mathcal{F}$ with $f\leq g$, the interval $[f,g]=\{h:f(x)\leq h(x)\leq g(x),\forall x\in\mathbb{R}^d\}$ is called a bracket for $\mathcal{F}$. ...
- 141
1
vote
1
answer
56
views
How to study the convergence of the sample mode for arbitrary probability spaces
(This is not the problem I actually care about, but an analogy with similar issues to the problem I'm actually considering.)
Consider a probability space with i.i.d. random variables $X_i$ producing ...
- 277
6
votes
2
answers
607
views
Whence “uniform distribution”?
The “Earliest Uses” site suggests that the expression “uniform distribution” first appeared in Uspensky (1937), and “uniformly distributed” in Sakamoto (1943). Is that true?
- 31.5k
1
vote
0
answers
44
views
Constrained random sampling from partitioned sets with quotas
Let $D$ be a finite set, $\mathcal{P} = \{D_{i,j}\}_{(i,j) \in I \times J}$ a partition of $D$, $N: J \to \mathbb{N}$ a quota function, and $k \in \mathbb{N}^+$. A subset $F \subseteq D$ is considered ...
- 11
1
vote
0
answers
80
views
Moments from characteristic function for matrices
When $x$ is a random variable with the smooth characteristic function $\phi_x(t) = \mathbb{E}e^{itx}$, we can easily compute the moments as $\mathbb{E}[x^k] = i^{-n}\phi_x^{(n)}(0)$. There is no magic ...
- 193
2
votes
0
answers
56
views
Sum of independent Wisharts
Suppose random vectors $y_1,y_2,\ldots,y_m$ are independent and the distribution of each $y_i$ is a $d$-dimensional complex Gaussian with mean $0$ and covariance $\Gamma_i$, that is $y_i \sim \mathcal{...
- 193
3
votes
0
answers
131
views
Matrix-Gaussian distributions
The point of this question is to ask for references on matrix-variate Gaussian distributions. But I will explain what I mean by a matrix-variate Gaussian with an example (the notion I have in mind is ...
- 193
3
votes
0
answers
353
views
Moments of normalized multivariate Gaussians (and Wick's/Isserlis theorems)
Suppose $x = \begin{bmatrix}x_1 \\ x_2\end{bmatrix}$ is distributed according to the real two-dimensional Gaussian with mean-$0$ and covariance matrix $\Sigma$. I am interested in a closed form for ...
- 193
-2
votes
1
answer
43
views
$E(\mathbf{y}|\mathbf{x}+\mathbf{z})=g(\mathbf{x})$ almost surely, if $\mathbf{z}\perp \!\!\! \perp \{\mathbf{y},\mathbf{x}\}$ jointly?
Let $\mathbf{y},\mathbf{x}$ and $\mathbf{z}$ be real-valued random vectors with possibly different dimensions.
If $\mathbf{z} \perp\!\!\!\perp \{\mathbf{y},\mathbf{x}\}$ (i.e., $\mathbf{z}$ is ...
- 193
1
vote
1
answer
81
views
Inference for the normal distribution with known variance from multiple clusters
Here's the question:
We have: $q \sim N\left(q_p, \frac{1}{\tau}\right), q_i \sim N\left(q, \frac{1}{\zeta}\right), t_n \sim N\left(0, \frac{1}{\eta}\right)$. Let $$ r_n=\sum_{i=1}^{\theta k_{n}} \...
- 31
0
votes
0
answers
85
views
When is a family of distributions "closed" with respect to minimal sufficient statistics?
As in the title, I am interested in understanding how to express the idea that a parametric family of distribution is "closed" with respect to minimal sufficient statistics. Before giving ...
3
votes
0
answers
80
views
Seeking strong bounds on KL-divergence and martingales for a hypothesis-testing inequality
Let's say we have a finite set $\mathcal{O}$ of observations, and let $\mathcal{C}(\Delta\mathcal{O})$ denote the space of closed convex sets of probability distributions.
We have two hypotheses which ...
- 353
2
votes
0
answers
50
views
Weighted squared norm of multivariate truncated normal vector
Let $X \sim \mathcal{N}(0, \Sigma)$ be a multivariate normal vector with zero mean and inverse covariance matrix
$$
\Sigma^{-1} = \begin{pmatrix}
n & 1 & 1 & \cdots & 1 &...
1
vote
1
answer
84
views
Limiting value of Stieltjes transform of sum of independent Wishart matrices
Let $n_1$, $n_2$, and $d$ positive integers tending to infinity such that $d/n_k \to \phi_k \in (0,\infty)$ and $n_1/(n_1+n_2) \to p \in (0,1)$. Let $X_k$ be an $n_k \times d$ random matrix with iid ...
- 6,853
2
votes
1
answer
177
views
Optimization over Poisson-binomial distributions
I am studying the problem of how an expected utility maximizer should optimally form a portfolio of uncorrelated Bernoullis.
Fix an increasing sequence of $n$ numbers in $(0,1)$, $0<p_1<\dots<...
2
votes
1
answer
170
views
Law of large numbers for a continuum of Bernoullis
Suppose I have a family of $n$ independent Bernoulli random variables described by a vector of parameters $(p_i)_{i=1}^n$. As it is well known, the number of successes within this family is a random ...
0
votes
1
answer
159
views
Theories for "fuzzy" distributions
When calculating the probability density function for the quotients of adjacent values in an empirical time series, the image of the PDF looked like this:
It seems to resemble a lognormal ...
- 13.2k
0
votes
1
answer
108
views
RMT for modified Wishard matrix $Y'Y$ (where $i$th row of $Y$ is zero if $|x_i^\top u| \le \theta$; else it equals $x_i$)
Let $n$ and $d$ be positive integers tending to infinity such that $d/n \to \phi \in (0,\infty)$. Let $X$ be an $n \times d$ random matrix with iid rows $x_1,\ldots,x_n$ from $N(0, \Sigma)$, where $\...
- 6,853
0
votes
0
answers
89
views
Stein's Lemma for conditional expectation?
Let $X=(X_1,\ldots,X_d)$ be a standard normal random vector in $\mathbb R^d$, let $m:\mathbb R^d \to \mathbb R$ be a function, and let $E=E_m$ denote the expectation operator conditioned on $m(X) > ...
- 6,853
1
vote
1
answer
97
views
Bayes classifiers with cost of misclassification
A minimum ECM classifier disciminate the features $\underline{x}$ to belong to class $t$ ($\delta(\underline{x}) = t$) if $\forall j \ne t$:
$$\sum_{k\ne t} c(t|k) f_k(\underline{x})p_k \le \sum_{k\ne ...
- 31
0
votes
0
answers
37
views
Compatibility of 2-copulas
An $n$-copula is the joint distribution function of a distribution on $[0,1]^n$ with uniform marginals. A family of 2-copulas $(C_{i,j})_{i<j\leq n}$ is compatible if there exists an $n$-copula $\...
- 677
4
votes
1
answer
234
views
Maximum entropy probability distribution with fixed interval and variance?
What is the maximum entropy probability distribution if the support is a fixed interval (e.g. $[-1,1]$) with an already known variance?
If we know the support is a fixed interval, then the maximum ...
- 43
1
vote
0
answers
53
views
The limit ratio of two Markov Chain Probability
Suppose there are two given SDE in $\mathbb{R}^d$:
$$
\begin{align}
\left\{
\begin{aligned}
dX_t&=\begin{bmatrix}-\nabla V(X_t)+2\beta^{-1}v_F^\theta(X_t)\end{bmatrix}dt+\sqrt{2\beta^{-1}}dW_t,&...
- 303
2
votes
0
answers
84
views
Concentration result for self-normalized empirical process
In Theorem 1.1 of this paper by Bercu, Gassiat and Rio, a concentration result is derived for the 'self-normalized' empirical process. Specifically, suppose that $(X,X_n)_{n \ge 1}$ is a sequence of i....
- 206
0
votes
0
answers
42
views
Limiting value of trace of resolvent matrix involving two independent Wishart random matrices
Let $n_1$, $n_2$, and $d$ be positive integers tending to infinity such that
$$
d/n_k \to \phi_k \in (0,\infty).
$$
Let $X_1 \in \mathbb R^{n_1 \times d}$ and $X_2^{n_2 \times d}$ be independent ...
- 6,853
2
votes
1
answer
243
views
Concentration inequalities for heavy-tailed distributions
Suppose $X_1,...,X_N$ are $N$ i.i.d random variables with heavy tailed distributions. For example, $E[X_i^p]\leq 1$ for some $p\geq 1$. Are there some concentration inequalities to bound the tail
$$P(\...
- 31
1
vote
1
answer
115
views
A property of the distribution related to stochastic ordering
Let $X$ be a random variable with a symmetric support $S\subset[-M,M]$ for some $M>0$. (i.e., if x is a point of increase of CDF $F_X(\cdot)$, so is $-x$.)
Has the infimum value of $c$ such that
\...
- 19
0
votes
0
answers
73
views
Asymptotic stochastic ordering for weighted sum of i.i.d. random variables
Are you aware of any literature focusing on the conditions such that for two i.i.d. sequences of discrete r.v.'s $\{X_n\}$ and $\{Y_n\}$,
\begin{equation}
a_1X_1+a_2X_2+\ldots+a_nX_n\geq_1 a_1Y_1+...
- 19
1
vote
0
answers
68
views
Gibbs Priors form a Martingale
I am working on adapting variational inference to the recently developed Martingale posterior distributions. The first case, which reduces the VI framework to Gibbs priors, is proving hard to show as ...
- 29
1
vote
2
answers
306
views
Joint moments like $\tau(XYXYXY)$ in terms of individual moments of free variables $X,Y$
Terry Tao RMT book has the following formula for joint moment of freely independent random variables $X,Y$ in Section 2.5
$$\tau(XYXY)=\tau(X)^2\tau(Y^2)+\tau(X^2)\tau(Y)^2-\tau(X)^2\tau(Y)^2$$
...
- 2,242
0
votes
0
answers
91
views
Some new questions on Rademacher complexity
For $A\subset R^n$,$A=(a_1,a_2,\dots, a_n)$, $\sigma_i$ are Rademacher random variable.
Is $|\mathbb{E}_\sigma \inf_{a\in A}\sum_{i=1}^n\sigma_ia_i| \le |\mathbb{E}_\sigma \sup_{a\in A}\sum_{i=1}^n\...
- 185
2
votes
2
answers
195
views
Random partition of an interval – Dirichlet distributed?
Let $X_1, \ldots, X_N \sim \operatorname{Unif}[0,1]$ and consider the intervals between successive order statistics: $[0, X_{(1)}], [X_{(1)}, X_{(2)}], \ldots, [X_{(N)}, 1]$.
What is the distribution ...
- 57
1
vote
1
answer
241
views
Expectation of top-K selection of squared Gaussian random variables
Let us have
$$
Z = [z_1, z_2, \dots, z_n],
$$ where $z_i \sim N(0, \sigma^2)$ and are iid. Additionally, consider
$$
X_k := \{ x \in \{0, 1\}^n : e^T x = k \}
$$ If $Y = \max_{X \in X_k} |Z^T X|^2,$ ...
1
vote
0
answers
62
views
A small lemma on cache resets (Bloom filters in particular)
Assume a fixed set of message $D$ and an associated distribution for selecting each message $d_i$ such that the total probability $\sum_{i \in D} d_i = 1$. We create a cache with $M$ bits and $k$ ...
1
vote
0
answers
72
views
Dimension-free sample complexity for the inverse of Gaussian sample covariance?
Suppose I have $m$ samples drawn from a Gaussian in $\mathbb{R}^n$, and need the inverse of the sample covariance $\Sigma_m^{-1}$ to be $\varepsilon$-close to true inverse covariance $\Sigma^{-1}$ (in ...
- 517
0
votes
0
answers
82
views
High probability bound on number of sparse solutions to Gaussian linear system
Suppose we have a random matrix $A \in \mathbb{R}^{m \times n}$ with all entries i.i.d. from the standard Normal distribution $\mathcal{N}(0, 1)$. Suppose $k$ divides $n$, and let $S \subseteq \mathbb{...
- 43
7
votes
2
answers
235
views
Evolution of the empirical mean of a list as we remove elements proportional to their value
Consider a list of $N$ integers $k_1,k_2,\dots k_N$, drawn independently from some distribution $P(k)$ with $k_i \geq 1$. We denote its mean with $\langle k\rangle=\sum_{k=1}kP(k)$. The first two ...
- 274
3
votes
2
answers
358
views
Minimax optimal multiple hypothesis test
Let us consider the following two-player game
between Chooser and Guesser.
There is a finite set $\Omega$
and $k$ probability distributions
on $\Omega$, denoted by $
\mathcal{P}
=\{P_1,\ldots,P_k\}
$.
...
- 6,541
2
votes
1
answer
245
views
Sum of arrival times of Chinese Restaurant Process (CRP)
Suppose that a random sample $X_1, X_2, \ldots$ is drawn from a continuous spectrum of colors, or species, following a Chinese Restaurant Process distribution with parameter $|\alpha|$ (or ...
- 141
3
votes
1
answer
108
views
When does the optimal model exist in learning theory?
In the context of learning theory, we usually have: data $(x,y)\sim P(x,y)$, with $x\in\mathcal{X}\subseteq\mathbb{R}^d$ and $y\in\mathcal{Y}\subseteq\mathbb{R}^k$, a hypothesis class $\mathcal{F}\...
- 199
2
votes
1
answer
138
views
expectation of the product of Gaussian kernels and their input
I was wondering if anybody knows how to solve: $$\mathbb{E}{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}\left[ (\mathbf{x}{i} - \mathbf{z})(\mathbf{x}{j} - \mathbf{z})^\top \exp\left( - (\...
- 275