All Questions
Tagged with pr.probability st.statistics
1,134 questions
1
vote
1
answer
84
views
Limiting value of Stieltjes transform of sum of independent Wishart matrices
Let $n_1$, $n_2$, and $d$ positive integers tending to infinity such that $d/n_k \to \phi_k \in (0,\infty)$ and $n_1/(n_1+n_2) \to p \in (0,1)$. Let $X_k$ be an $n_k \times d$ random matrix with iid ...
- 6,853
2
votes
0
answers
50
views
Weighted squared norm of multivariate truncated normal vector
Let $X \sim \mathcal{N}(0, \Sigma)$ be a multivariate normal vector with zero mean and inverse covariance matrix
$$
\Sigma^{-1} = \begin{pmatrix}
n & 1 & 1 & \cdots & 1 &...
2
votes
1
answer
170
views
Law of large numbers for a continuum of Bernoullis
Suppose I have a family of $n$ independent Bernoulli random variables described by a vector of parameters $(p_i)_{i=1}^n$. As it is well known, the number of successes within this family is a random ...
- 63.9k
0
votes
1
answer
159
views
Theories for "fuzzy" distributions
When calculating the probability density function for the quotients of adjacent values in an empirical time series, the image of the PDF looked like this:
It seems to resemble a lognormal ...
- 128k
8
votes
2
answers
2k
views
Median and mean of the sample mean of i.i.d. log-normal
Let $y:=\frac1n\sum_{i=1}^n x_i$, where $\{x_i\}_{i=1}^n$ is a set of i.i.d. random variables, and every $x_i$ has a lognormal distribution $x_i \sim\text{Lognormal}(\mu,\sigma^2)$. Let $\text{Med}[y]$...
- 4,713
1
vote
1
answer
97
views
Bayes classifiers with cost of misclassification
A minimum ECM classifier disciminate the features $\underline{x}$ to belong to class $t$ ($\delta(\underline{x}) = t$) if $\forall j \ne t$:
$$\sum_{k\ne t} c(t|k) f_k(\underline{x})p_k \le \sum_{k\ne ...
- 31
2
votes
1
answer
331
views
Prove or disprove the linearity of expectiles
For expectation (mean), there are many useful properties such as Linearity of Expectation:
$\mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]$
$\mathbb{E}[\alpha X]=\alpha\mathbb{E}[X]$
(The two equations ...
0
votes
0
answers
89
views
Stein's Lemma for conditional expectation?
Let $X=(X_1,\ldots,X_d)$ be a standard normal random vector in $\mathbb R^d$, let $m:\mathbb R^d \to \mathbb R$ be a function, and let $E=E_m$ denote the expectation operator conditioned on $m(X) > ...
- 6,853
1
vote
1
answer
216
views
Rademacher complexity of function class $(x,y) \mapsto 1[|yf(x)-\alpha| \ge \beta]$ in terms of $\alpha$, $\beta$, and Rademacher complexity of $F$
Let $X$ be a measurable space and let $P$ be a probability distribution on $X \times \{\pm 1\}$. Let $F$ be a function class on $X$, i.e., a collection of (measurable) functions from $X$ to $\mathbb R$...
- 12.9k
0
votes
0
answers
37
views
Compatibility of 2-copulas
An $n$-copula is the joint distribution function of a distribution on $[0,1]^n$ with uniform marginals. A family of 2-copulas $(C_{i,j})_{i<j\leq n}$ is compatible if there exists an $n$-copula $\...
- 677
4
votes
1
answer
234
views
Maximum entropy probability distribution with fixed interval and variance?
What is the maximum entropy probability distribution if the support is a fixed interval (e.g. $[-1,1]$) with an already known variance?
If we know the support is a fixed interval, then the maximum ...
2
votes
2
answers
195
views
Random partition of an interval – Dirichlet distributed?
Let $X_1, \ldots, X_N \sim \operatorname{Unif}[0,1]$ and consider the intervals between successive order statistics: $[0, X_{(1)}], [X_{(1)}, X_{(2)}], \ldots, [X_{(N)}, 1]$.
What is the distribution ...
1
vote
0
answers
53
views
The limit ratio of two Markov Chain Probability
Suppose there are two given SDE in $\mathbb{R}^d$:
$$
\begin{align}
\left\{
\begin{aligned}
dX_t&=\begin{bmatrix}-\nabla V(X_t)+2\beta^{-1}v_F^\theta(X_t)\end{bmatrix}dt+\sqrt{2\beta^{-1}}dW_t,&...
- 303
3
votes
1
answer
1k
views
Convergence of empirical measures in Wasserstein distance
Let $X_1, X_2, \ldots$ be iid random variables with common distribution $\gamma$, the standard Gaussian distribution on $\mathbb {R}$, and let
$\mu_n = \frac 1n \sum_{i=1}^n \delta_{X_i}$, $n \geq 1$, ...
CommunityBot
- 1
2
votes
0
answers
84
views
Concentration result for self-normalized empirical process
In Theorem 1.1 of this paper by Bercu, Gassiat and Rio, a concentration result is derived for the 'self-normalized' empirical process. Specifically, suppose that $(X,X_n)_{n \ge 1}$ is a sequence of i....
- 206
5
votes
1
answer
2k
views
Mathematics research relating to machine learning
What branch/branches of math are most relevant in enhancing machine learning (mostly in terms of practical use as opposed to theoretical/possible use)? Specifically, I want to know about math research ...
- 30.3k
0
votes
0
answers
42
views
Limiting value of trace of resolvent matrix involving two independent Wishart random matrices
Let $n_1$, $n_2$, and $d$ be positive integers tending to infinity such that
$$
d/n_k \to \phi_k \in (0,\infty).
$$
Let $X_1 \in \mathbb R^{n_1 \times d}$ and $X_2^{n_2 \times d}$ be independent ...
- 6,853
2
votes
1
answer
243
views
Concentration inequalities for heavy-tailed distributions
Suppose $X_1,...,X_N$ are $N$ i.i.d random variables with heavy tailed distributions. For example, $E[X_i^p]\leq 1$ for some $p\geq 1$. Are there some concentration inequalities to bound the tail
$$P(\...
- 128k
1
vote
1
answer
115
views
A property of the distribution related to stochastic ordering
Let $X$ be a random variable with a symmetric support $S\subset[-M,M]$ for some $M>0$. (i.e., if x is a point of increase of CDF $F_X(\cdot)$, so is $-x$.)
Has the infimum value of $c$ such that
\...
- 128k
0
votes
0
answers
73
views
Asymptotic stochastic ordering for weighted sum of i.i.d. random variables
Are you aware of any literature focusing on the conditions such that for two i.i.d. sequences of discrete r.v.'s $\{X_n\}$ and $\{Y_n\}$,
\begin{equation}
a_1X_1+a_2X_2+\ldots+a_nX_n\geq_1 a_1Y_1+...
- 19
3
votes
1
answer
96
views
Perturbation results for statistical estimators
Suppose I have a continuous random variable whose distribution $f$ is some parametric form (normal, exponential, etc.) that is known to me. If I draw many independent samples $x_i$ from $f$, I can ...
- 27
1
vote
2
answers
306
views
Joint moments like $\tau(XYXYXY)$ in terms of individual moments of free variables $X,Y$
Terry Tao RMT book has the following formula for joint moment of freely independent random variables $X,Y$ in Section 2.5
$$\tau(XYXY)=\tau(X)^2\tau(Y^2)+\tau(X^2)\tau(Y)^2-\tau(X)^2\tau(Y)^2$$
...
1
vote
0
answers
68
views
Gibbs Priors form a Martingale
I am working on adapting variational inference to the recently developed Martingale posterior distributions. The first case, which reduces the VI framework to Gibbs priors, is proving hard to show as ...
0
votes
0
answers
91
views
Some new questions on Rademacher complexity
For $A\subset R^n$,$A=(a_1,a_2,\dots, a_n)$, $\sigma_i$ are Rademacher random variable.
Is $|\mathbb{E}_\sigma \inf_{a\in A}\sum_{i=1}^n\sigma_ia_i| \le |\mathbb{E}_\sigma \sup_{a\in A}\sum_{i=1}^n\...
- 3,726
1
vote
0
answers
62
views
A small lemma on cache resets (Bloom filters in particular)
Assume a fixed set of message $D$ and an associated distribution for selecting each message $d_i$ such that the total probability $\sum_{i \in D} d_i = 1$. We create a cache with $M$ bits and $k$ ...
4
votes
2
answers
1k
views
Expectation of the trace of inverse of a Gaussian random matrix
Given a $N×M$ random complex gaussian matrix $X$ and $N×K$ random complex gaussian matrix $Y$ I'm interested in approximating the expectation expressed as:
\begin{align}
E[trace({(aX{X^H} + I)^{ - ...
- 6,853
1
vote
0
answers
72
views
Dimension-free sample complexity for the inverse of Gaussian sample covariance?
Suppose I have $m$ samples drawn from a Gaussian in $\mathbb{R}^n$, and need the inverse of the sample covariance $\Sigma_m^{-1}$ to be $\varepsilon$-close to true inverse covariance $\Sigma^{-1}$ (in ...
7
votes
2
answers
235
views
Evolution of the empirical mean of a list as we remove elements proportional to their value
Consider a list of $N$ integers $k_1,k_2,\dots k_N$, drawn independently from some distribution $P(k)$ with $k_i \geq 1$. We denote its mean with $\langle k\rangle=\sum_{k=1}kP(k)$. The first two ...
- 10.3k
0
votes
0
answers
82
views
High probability bound on number of sparse solutions to Gaussian linear system
Suppose we have a random matrix $A \in \mathbb{R}^{m \times n}$ with all entries i.i.d. from the standard Normal distribution $\mathcal{N}(0, 1)$. Suppose $k$ divides $n$, and let $S \subseteq \mathbb{...
- 43
3
votes
2
answers
358
views
Minimax optimal multiple hypothesis test
Let us consider the following two-player game
between Chooser and Guesser.
There is a finite set $\Omega$
and $k$ probability distributions
on $\Omega$, denoted by $
\mathcal{P}
=\{P_1,\ldots,P_k\}
$.
...
- 165
4
votes
3
answers
1k
views
Probability theory and measuring the true strength of chessplayers
If you wanted to measure the strength of, say, a chess player, the best way would involve knowing the true value of each position: then you could compute the frequency $W$ with which the player finds ...
- 540
91
votes
8
answers
16k
views
Is there a natural random process that is rigorously known to produce Zipf's law?
Zipf's law is the empirical observation that in many real-life populations of $n$ objects, the $k^\text{th}$ largest object has size proportional to $1/k$, at least for $k$ significantly smaller than $...
3
votes
1
answer
108
views
When does the optimal model exist in learning theory?
In the context of learning theory, we usually have: data $(x,y)\sim P(x,y)$, with $x\in\mathcal{X}\subseteq\mathbb{R}^d$ and $y\in\mathcal{Y}\subseteq\mathbb{R}^k$, a hypothesis class $\mathcal{F}\...
- 12.7k
11
votes
2
answers
758
views
Notions of "independent" and "uncorrelated" for subsets of the natural numbers
In probability/statistics, there is a notion of two things being "independent", which would basically mean that any information we can get about one thing has no effect on our (probabilistic)...
- 14k
2
votes
2
answers
632
views
An alternative proof of Bayesian Cramer-Rao
My question is:
Are there an alternative proof of Cramer-Rao lower bound that does not use
Cauchy-Swartz inequality?
Let me outline the classical proof and explain why I am interested in this ...
2
votes
1
answer
138
views
expectation of the product of Gaussian kernels and their input
I was wondering if anybody knows how to solve: $$\mathbb{E}{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}\left[ (\mathbf{x}{i} - \mathbf{z})(\mathbf{x}{j} - \mathbf{z})^\top \exp\left( - (\...
- 128k
0
votes
0
answers
19
views
Analyzing point distributions in Voronoi tessellations from two probability sources $p$ and $\tilde{p}$
Let's suppose I have a probability distribution $p$ and another distribution $\tilde{p}$. Suppose I sample $K$ points from the distribution $p$ which will be my centroids for my Voronoi tessellation. ...
- 63.9k
4
votes
5
answers
2k
views
Martingales and Betting Strategies
Does anyone know of a good introduction to the theory of martingales and betting strategies from the point of view of statistics and/or probability theory? I'm looking for something basic, with lots ...
2
votes
1
answer
122
views
Analytical solution for a double integral involving logistic functions and Gaussian distributions
I am working on a mathematical problem involving the evaluation of a double integral, and I am seeking an analytical solution or techniques to solve it. The integral I'm dealing with is as follows:
$$...
0
votes
0
answers
50
views
Approximate CDF of integral using the Berry-Esseen theorem
I'm trying to approximate CDF of the integral $$\frac{1}{T}\int_0^T e^{\sigma W_t+\left(r-\frac{\sigma^2}{2}\right)t}dt,$$
where $W_t$ is the Wiener process, i.e. $W_t\sim N(0,t)$.
For this I use ...
- 49
4
votes
0
answers
156
views
Known relations between mutual information and covering number?
This is a question about statistical learning theory. Consider a hypothesis class $\mathcal{F}$, parameterized by real vectors $w \in \mathbb{R}^p$. Suppose I have a data distribution $D \sim \mu$ and ...
- 141
1
vote
0
answers
148
views
conjecture for general form of minimax estimator
I had previously posed an overly ambitious version of this conjecture here,
Form of minimax estimator,
which was quickly shot down by Václav Voráček (on twitter) and Iosif Pinelis (MO answer in the ...
- 6,541
2
votes
1
answer
294
views
An inequality in the optimality of Bayes' theorem
$\DeclareMathOperator\Ent{Ent}\newcommand{\prior}{\mathrm{prior}}\newcommand\Data{\mathrm{Data}}$I came across this paper on the optimality of Bayes' theorem
https://sinews.siam.org/Portals/Sinews2/...
0
votes
1
answer
116
views
Existence and uniqueness of a posterior distribution
I am wondering about the existence and uniqueness of a posterior distribution.
While Bayes' theorem gives the form of the posterior, perhaps there are pathological cases (over some weird probability ...
0
votes
1
answer
231
views
Concentration inequalities for random sampling without replacement
Let a population $C$ consist of $N$ values $c_1, c_2, \cdots, c_N$, with $c_i\in \{0,1\}$. Let $X_1, X_2, \cdots, X_n$ denote a random sample without replacement from $C$ and let $Y_1, Y_2, \cdots, ...
18
votes
4
answers
4k
views
Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables
I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables.
Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
- 63
1
vote
2
answers
221
views
Beating the $1/\sqrt n$ rate of uniform-convergence over a linear function class
Let $P$ be a probability distribution on $\mathbb R^d \times \mathbb R$, and let $(x_1,y_1), \ldots, (x_n,y_n)$ be an iid sample of size $n$ from $P$. Fix $\epsilon,t\gt 0$. For any unit-vector $w \in ...
11
votes
3
answers
3k
views
Distance between distributions and distance of moments
Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{$\ast$}$$ where $X$ is a random variable of ...
- 696
2
votes
0
answers
92
views
Construct a Bregman divergence from Wasserstein distance
I was wondering whether one has studied the Bregman divergence arising from a squared Wasserstein distance.
More precisely, let $\Omega\subset \mathbb{R}^d$ be a compact set and $c\in \Omega\times \...
- 503
8
votes
3
answers
2k
views
Sampling uniformly from a sphere
Let $B^{n} _p= ${$ (x_1, \dots, x_n) : |x_1|^p + \dots |x_n|^p = 1 $} be the unit ball in $\mathbb{R}^n$ in the $\ell^p$ norm.
If $X_1,\dots,X_n$ are iid $\exp(1)$ -distributed random variables, then ...
