All Questions
Tagged with pr.probability st.statistics
1,135 questions
-1
votes
0
answers
19
views
What is the expected value of the set when N elements are chosen from the same probability distribution?
Suppose we have a parameter that follows some probability distribution $f(x)$. When simulating an $N$-body with that parameter as an attribute, how should values of the parameter be chosen?
Let each ...
- 1
1
vote
0
answers
41
views
Can conditional distributions with respect to a sufficient sub-$\sigma$-algebra be represented by a single Markov kernel?
Let $(\Omega, \mathcal{F})$ be a measurable space, and let $\mathcal{P}$ be a collection of probability measures on this space. A sub-$\sigma$-algebra $\mathcal{G} \subset \mathcal{F}$ is said to be ...
- 111
9
votes
2
answers
431
views
Hermite–Fourier expansion for the median
Let $n$ be an odd positive integer. Let $M : \mathbb{R}^n \to \mathbb{R}$ be the median function: $M(x_1,\dots,x_n)$ is the median of $x_1,\dots,x_n$. What can be said about the Hermite–Fourier ...
- 24.7k
16
votes
1
answer
397
views
Examples of problems in statistics accessible only using information geometry
I am just curious if there are some examples of problems in statistics that are indeed accessible using information geometry while proofs completely avoiding geometry are unknown. In other words, ...
- 269
3
votes
0
answers
92
views
Tighter Freedman's inequality for a special martingale difference sequence
Let $X_{1}, \ldots, X_{T} \in \{0, 1\}$ be a sequence of Boolean random variables with
$$
\mathbb{E}[X_{t} | X_{1}, \dots, X_{t - 1}] = p_{t}.
$$
Consider the sequence $Y_{t} := X_{t} - p_{t}$ (which ...
- 41
2
votes
1
answer
208
views
Proving an exponential sum inequality for symmetric Hamming distance sequences in binary vectors
Background: Let $X = \{0,1\}^k$ represent the set of all binary vectors of length $k$. For two binary vectors $x, y \in X$, the Hamming distance $d_H(x, y)$ is defined as the number of positions where ...
- 359
6
votes
2
answers
608
views
Whence “uniform distribution”?
The “Earliest Uses” site suggests that the expression “uniform distribution” first appeared in Uspensky (1937), and “uniformly distributed” in Sakamoto (1943). Is that true?
- 31.5k
0
votes
1
answer
100
views
Expressing a multivariate normal distribution as a mixture of uniform distributions?
Context: Given a scalar normal distribution $X\sim \mathrm{N}(\mu, \sigma^2)$, it is possible to express $X$ as a mixture of uniform distributions over intervals (compound probability distributions), ...
1
vote
0
answers
48
views
Quantile maximization of the difference of random constrained quadratic optimization problems
I am interested in understanding the family of parametrized random variables defined by the pushforward map
$$
\lambda_x : \varepsilon \mapsto \underset{z_1 \in \mathbb{R}^n :\, h^T z_1 = 0, \; z_1 \...
- 11
2
votes
0
answers
77
views
Inequalities concerning cummulative distributions of binomials
For random variable $Z$, let $F_Z$ denote its cdf, i.e., $F_Z(t)=\mathbb{P}(Z\leq t)$. Let $X$ be a binomial distribution with parameters $(n,p)$ and $Y$ a binomial distribution with parameters $(m,p)$...
- 121
1
vote
0
answers
61
views
Bound on $\int_0^1\sqrt{\log N_{[]}(\varepsilon,\mathcal{F},d)} \, d\varepsilon$ over the class of half-spaces $\mathcal{F}$ on $\mathbb{R}^d$?
For a class of functions $\mathcal{F}$ and a pair $f,g\in\mathcal{F}$ with $f\leq g$, the interval $[f,g]=\{h:f(x)\leq h(x)\leq g(x),\forall x\in\mathbb{R}^d\}$ is called a bracket for $\mathcal{F}$. ...
- 141
0
votes
0
answers
44
views
Large Deviation Principle for an adaptive sampling rule for Multi Armed Bandits
Consider the following adaptive strategy for sampling from a Multi Armed Bandit with $K$ arms:
Split the $T$ rounds into $N (\in \mathbb{N})$ disjoint intervals. Each interval is indexed by $i=1,2,\...
- 9
1
vote
1
answer
56
views
How to study the convergence of the sample mode for arbitrary probability spaces
(This is not the problem I actually care about, but an analogy with similar issues to the problem I'm actually considering.)
Consider a probability space with i.i.d. random variables $X_i$ producing ...
- 277
0
votes
1
answer
108
views
RMT for modified Wishard matrix $Y'Y$ (where $i$th row of $Y$ is zero if $|x_i^\top u| \le \theta$; else it equals $x_i$)
Let $n$ and $d$ be positive integers tending to infinity such that $d/n \to \phi \in (0,\infty)$. Let $X$ be an $n \times d$ random matrix with iid rows $x_1,\ldots,x_n$ from $N(0, \Sigma)$, where $\...
- 6,853
1
vote
0
answers
44
views
Constrained random sampling from partitioned sets with quotas
Let $D$ be a finite set, $\mathcal{P} = \{D_{i,j}\}_{(i,j) \in I \times J}$ a partition of $D$, $N: J \to \mathbb{N}$ a quota function, and $k \in \mathbb{N}^+$. A subset $F \subseteq D$ is considered ...
- 11
3
votes
0
answers
355
views
Moments of normalized multivariate Gaussians (and Wick's/Isserlis theorems)
Suppose $x = \begin{bmatrix}x_1 \\ x_2\end{bmatrix}$ is distributed according to the real two-dimensional Gaussian with mean-$0$ and covariance matrix $\Sigma$. I am interested in a closed form for ...
- 193
2
votes
1
answer
177
views
Optimization over Poisson-binomial distributions
I am studying the problem of how an expected utility maximizer should optimally form a portfolio of uncorrelated Bernoullis.
Fix an increasing sequence of $n$ numbers in $(0,1)$, $0<p_1<\dots<...
3
votes
0
answers
132
views
Matrix-Gaussian distributions
The point of this question is to ask for references on matrix-variate Gaussian distributions. But I will explain what I mean by a matrix-variate Gaussian with an example (the notion I have in mind is ...
- 193
0
votes
1
answer
159
views
Theories for "fuzzy" distributions
When calculating the probability density function for the quotients of adjacent values in an empirical time series, the image of the PDF looked like this:
It seems to resemble a lognormal ...
- 13.2k
2
votes
1
answer
170
views
Law of large numbers for a continuum of Bernoullis
Suppose I have a family of $n$ independent Bernoulli random variables described by a vector of parameters $(p_i)_{i=1}^n$. As it is well known, the number of successes within this family is a random ...
1
vote
1
answer
242
views
Expectation of top-K selection of squared Gaussian random variables
Let us have
$$
Z = [z_1, z_2, \dots, z_n],
$$ where $z_i \sim N(0, \sigma^2)$ and are iid. Additionally, consider
$$
X_k := \{ x \in \{0, 1\}^n : e^T x = k \}
$$ If $Y = \max_{X \in X_k} |Z^T X|^2,$ ...
1
vote
0
answers
80
views
Moments from characteristic function for matrices
When $x$ is a random variable with the smooth characteristic function $\phi_x(t) = \mathbb{E}e^{itx}$, we can easily compute the moments as $\mathbb{E}[x^k] = i^{-n}\phi_x^{(n)}(0)$. There is no magic ...
- 193
2
votes
0
answers
56
views
Sum of independent Wisharts
Suppose random vectors $y_1,y_2,\ldots,y_m$ are independent and the distribution of each $y_i$ is a $d$-dimensional complex Gaussian with mean $0$ and covariance $\Gamma_i$, that is $y_i \sim \mathcal{...
- 193
20
votes
0
answers
3k
views
+200
What does a product of many Gaussian matrices converge to?
Let $A$ be a product of $n$ $d\times d$ matrices with IID standard Gaussian entries and consider the value of $g(x)=x f(x)$ where $f(x)$ is the density of squared singular values of $A/\|A\|$.
Is ...
- 2,252
1
vote
1
answer
81
views
Inference for the normal distribution with known variance from multiple clusters
Here's the question:
We have: $q \sim N\left(q_p, \frac{1}{\tau}\right), q_i \sim N\left(q, \frac{1}{\zeta}\right), t_n \sim N\left(0, \frac{1}{\eta}\right)$. Let $$ r_n=\sum_{i=1}^{\theta k_{n}} \...
- 31
-2
votes
1
answer
43
views
$E(\mathbf{y}|\mathbf{x}+\mathbf{z})=g(\mathbf{x})$ almost surely, if $\mathbf{z}\perp \!\!\! \perp \{\mathbf{y},\mathbf{x}\}$ jointly?
Let $\mathbf{y},\mathbf{x}$ and $\mathbf{z}$ be real-valued random vectors with possibly different dimensions.
If $\mathbf{z} \perp\!\!\!\perp \{\mathbf{y},\mathbf{x}\}$ (i.e., $\mathbf{z}$ is ...
- 193
0
votes
0
answers
85
views
When is a family of distributions "closed" with respect to minimal sufficient statistics?
As in the title, I am interested in understanding how to express the idea that a parametric family of distribution is "closed" with respect to minimal sufficient statistics. Before giving ...
2
votes
1
answer
245
views
Sum of arrival times of Chinese Restaurant Process (CRP)
Suppose that a random sample $X_1, X_2, \ldots$ is drawn from a continuous spectrum of colors, or species, following a Chinese Restaurant Process distribution with parameter $|\alpha|$ (or ...
- 141
1
vote
1
answer
84
views
Limiting value of Stieltjes transform of sum of independent Wishart matrices
Let $n_1$, $n_2$, and $d$ positive integers tending to infinity such that $d/n_k \to \phi_k \in (0,\infty)$ and $n_1/(n_1+n_2) \to p \in (0,1)$. Let $X_k$ be an $n_k \times d$ random matrix with iid ...
- 6,853
2
votes
0
answers
50
views
Weighted squared norm of multivariate truncated normal vector
Let $X \sim \mathcal{N}(0, \Sigma)$ be a multivariate normal vector with zero mean and inverse covariance matrix
$$
\Sigma^{-1} = \begin{pmatrix}
n & 1 & 1 & \cdots & 1 &...
4
votes
1
answer
234
views
Maximum entropy probability distribution with fixed interval and variance?
What is the maximum entropy probability distribution if the support is a fixed interval (e.g. $[-1,1]$) with an already known variance?
If we know the support is a fixed interval, then the maximum ...
- 43
1
vote
2
answers
308
views
Joint moments like $\tau(XYXYXY)$ in terms of individual moments of free variables $X,Y$
Terry Tao RMT book has the following formula for joint moment of freely independent random variables $X,Y$ in Section 2.5
$$\tau(XYXY)=\tau(X)^2\tau(Y^2)+\tau(X^2)\tau(Y)^2-\tau(X)^2\tau(Y)^2$$
...
- 2,252
3
votes
0
answers
80
views
Seeking strong bounds on KL-divergence and martingales for a hypothesis-testing inequality
Let's say we have a finite set $\mathcal{O}$ of observations, and let $\mathcal{C}(\Delta\mathcal{O})$ denote the space of closed convex sets of probability distributions.
We have two hypotheses which ...
- 353
1
vote
1
answer
97
views
Bayes classifiers with cost of misclassification
A minimum ECM classifier disciminate the features $\underline{x}$ to belong to class $t$ ($\delta(\underline{x}) = t$) if $\forall j \ne t$:
$$\sum_{k\ne t} c(t|k) f_k(\underline{x})p_k \le \sum_{k\ne ...
- 31
7
votes
2
answers
235
views
Evolution of the empirical mean of a list as we remove elements proportional to their value
Consider a list of $N$ integers $k_1,k_2,\dots k_N$, drawn independently from some distribution $P(k)$ with $k_i \geq 1$. We denote its mean with $\langle k\rangle=\sum_{k=1}kP(k)$. The first two ...
- 274
2
votes
2
answers
196
views
Random partition of an interval – Dirichlet distributed?
Let $X_1, \ldots, X_N \sim \operatorname{Unif}[0,1]$ and consider the intervals between successive order statistics: $[0, X_{(1)}], [X_{(1)}, X_{(2)}], \ldots, [X_{(N)}, 1]$.
What is the distribution ...
- 57
3
votes
2
answers
358
views
Minimax optimal multiple hypothesis test
Let us consider the following two-player game
between Chooser and Guesser.
There is a finite set $\Omega$
and $k$ probability distributions
on $\Omega$, denoted by $
\mathcal{P}
=\{P_1,\ldots,P_k\}
$.
...
- 6,541
2
votes
1
answer
243
views
Concentration inequalities for heavy-tailed distributions
Suppose $X_1,...,X_N$ are $N$ i.i.d random variables with heavy tailed distributions. For example, $E[X_i^p]\leq 1$ for some $p\geq 1$. Are there some concentration inequalities to bound the tail
$$P(\...
- 31
1
vote
1
answer
115
views
A property of the distribution related to stochastic ordering
Let $X$ be a random variable with a symmetric support $S\subset[-M,M]$ for some $M>0$. (i.e., if x is a point of increase of CDF $F_X(\cdot)$, so is $-x$.)
Has the infimum value of $c$ such that
\...
- 19
4
votes
2
answers
2k
views
Why MLEs are asymptotically efficient whereas method of moment estimators are not?
Under appropriate regularity conditions it is well-known that Maximum Likelihood Estimation (MLE) produces asymptotically efficient estimators in the sense that their asymptotic covariance is given by ...
5
votes
1
answer
2k
views
Mathematics research relating to machine learning
What branch/branches of math are most relevant in enhancing machine learning (mostly in terms of practical use as opposed to theoretical/possible use)? Specifically, I want to know about math research ...
- 173
8
votes
3
answers
2k
views
Recommendation for learning mathematical statistics and probability
I can easily find my way reading a book on homological algebra or algebraic geometry, but I tried once reading a book on statistics and... I felt dumb really: I simply do not understand the ...
- 1,031
3
votes
1
answer
108
views
When does the optimal model exist in learning theory?
In the context of learning theory, we usually have: data $(x,y)\sim P(x,y)$, with $x\in\mathcal{X}\subseteq\mathbb{R}^d$ and $y\in\mathcal{Y}\subseteq\mathbb{R}^k$, a hypothesis class $\mathcal{F}\...
- 199
1
vote
0
answers
53
views
The limit ratio of two Markov Chain Probability
Suppose there are two given SDE in $\mathbb{R}^d$:
$$
\begin{align}
\left\{
\begin{aligned}
dX_t&=\begin{bmatrix}-\nabla V(X_t)+2\beta^{-1}v_F^\theta(X_t)\end{bmatrix}dt+\sqrt{2\beta^{-1}}dW_t,&...
- 303
2
votes
1
answer
138
views
expectation of the product of Gaussian kernels and their input
I was wondering if anybody knows how to solve: $$\mathbb{E}{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}\left[ (\mathbf{x}{i} - \mathbf{z})(\mathbf{x}{j} - \mathbf{z})^\top \exp\left( - (\...
- 275
0
votes
0
answers
91
views
Some new questions on Rademacher complexity
For $A\subset R^n$,$A=(a_1,a_2,\dots, a_n)$, $\sigma_i$ are Rademacher random variable.
Is $|\mathbb{E}_\sigma \inf_{a\in A}\sum_{i=1}^n\sigma_ia_i| \le |\mathbb{E}_\sigma \sup_{a\in A}\sum_{i=1}^n\...
- 185
2
votes
0
answers
84
views
Concentration result for self-normalized empirical process
In Theorem 1.1 of this paper by Bercu, Gassiat and Rio, a concentration result is derived for the 'self-normalized' empirical process. Specifically, suppose that $(X,X_n)_{n \ge 1}$ is a sequence of i....
- 206
1
vote
0
answers
68
views
Gibbs Priors form a Martingale
I am working on adapting variational inference to the recently developed Martingale posterior distributions. The first case, which reduces the VI framework to Gibbs priors, is proving hard to show as ...
- 29
0
votes
0
answers
73
views
Asymptotic stochastic ordering for weighted sum of i.i.d. random variables
Are you aware of any literature focusing on the conditions such that for two i.i.d. sequences of discrete r.v.'s $\{X_n\}$ and $\{Y_n\}$,
\begin{equation}
a_1X_1+a_2X_2+\ldots+a_nX_n\geq_1 a_1Y_1+...
- 19
2
votes
2
answers
823
views
Kolmogorov-Smirnov distance and expectation
Let $P$ and $Q$ be two probability measures over $R^n$, with CDF denoted by $F_P,F_Q$, respectively (that is, $F_P(x)=P(\{x'\in R^n:x'\leq x\})$, where $\leq$ is taken componentwise. The Kolmogorov-...
- 333