All Questions
Tagged with pr.probability st.statistics
1,134 questions
0
votes
0
answers
19
views
Analyzing point distributions in Voronoi tessellations from two probability sources $p$ and $\tilde{p}$
Let's suppose I have a probability distribution $p$ and another distribution $\tilde{p}$. Suppose I sample $K$ points from the distribution $p$ which will be my centroids for my Voronoi tessellation. ...
- 101
0
votes
0
answers
50
views
Approximate CDF of integral using the Berry-Esseen theorem
I'm trying to approximate CDF of the integral $$\frac{1}{T}\int_0^T e^{\sigma W_t+\left(r-\frac{\sigma^2}{2}\right)t}dt,$$
where $W_t$ is the Wiener process, i.e. $W_t\sim N(0,t)$.
For this I use ...
- 49
2
votes
1
answer
122
views
Analytical solution for a double integral involving logistic functions and Gaussian distributions
I am working on a mathematical problem involving the evaluation of a double integral, and I am seeking an analytical solution or techniques to solve it. The integral I'm dealing with is as follows:
$$...
- 31
4
votes
0
answers
156
views
Known relations between mutual information and covering number?
This is a question about statistical learning theory. Consider a hypothesis class $\mathcal{F}$, parameterized by real vectors $w \in \mathbb{R}^p$. Suppose I have a data distribution $D \sim \mu$ and ...
- 141
0
votes
1
answer
116
views
Existence and uniqueness of a posterior distribution
I am wondering about the existence and uniqueness of a posterior distribution.
While Bayes' theorem gives the form of the posterior, perhaps there are pathological cases (over some weird probability ...
2
votes
0
answers
92
views
Construct a Bregman divergence from Wasserstein distance
I was wondering whether one has studied the Bregman divergence arising from a squared Wasserstein distance.
More precisely, let $\Omega\subset \mathbb{R}^d$ be a compact set and $c\in \Omega\times \...
- 503
0
votes
1
answer
231
views
Concentration inequalities for random sampling without replacement
Let a population $C$ consist of $N$ values $c_1, c_2, \cdots, c_N$, with $c_i\in \{0,1\}$. Let $X_1, X_2, \cdots, X_n$ denote a random sample without replacement from $C$ and let $Y_1, Y_2, \cdots, ...
- 105
1
vote
0
answers
148
views
conjecture for general form of minimax estimator
I had previously posed an overly ambitious version of this conjecture here,
Form of minimax estimator,
which was quickly shot down by Václav Voráček (on twitter) and Iosif Pinelis (MO answer in the ...
- 6,541
1
vote
1
answer
341
views
Form of minimax estimator
Let $\Delta$ be the set of all probability distributions over $\mathbb{N}=\{1,2,\ldots\}$ and fix some $\mathcal{P}\subseteq\Delta$.
Suppose additionally that $\Delta$ is endowed with some norm $||\...
- 6,541
2
votes
1
answer
187
views
Law of iterated logarithm for quadratic variation of Brownian motion
Let $(\Omega, \mathcal{F}, \mathbb{P})$ denote a probability space supporting a standard Brownian motion $B$. Let $\Pi=\{\pi_n : n \ge 0\}$ denote the sequence of dyadic uniform partitions of the ...
- 41
1
vote
0
answers
57
views
Limiting value of expectation of trace of truncated Gram matrix
Let $n$ and $d$ be large positive integers such that $d/n = a \in (0,1)$, fixed. Let $x_1,\ldots,x_n$ be iid random vectors from $N(0,I_d)$. Fix $b \in (0,1]$ and a unit-vector $v \in \mathbb R^d$, ...
- 6,853
0
votes
0
answers
55
views
Modeling player interactions in multi-dimensional rating systems
In traditional rating systems (such as Elo), a player's strength is represented by a single scalar value, which is assumed to be consistent across different opponents. However, in some games, the ...
- 1
1
vote
2
answers
221
views
Beating the $1/\sqrt n$ rate of uniform-convergence over a linear function class
Let $P$ be a probability distribution on $\mathbb R^d \times \mathbb R$, and let $(x_1,y_1), \ldots, (x_n,y_n)$ be an iid sample of size $n$ from $P$. Fix $\epsilon,t\gt 0$. For any unit-vector $w \in ...
- 6,853
0
votes
0
answers
140
views
Bounding the difference of weighted-mean estimations
I recently asked a question related to a proof I'm working on as well as a follow-up to that. The detailed counter-examples I received elucidated a lot, but it quickly transpired (i) that I was ...
- 15
0
votes
1
answer
107
views
Bounding $\|X_1/(X_1+X_2) - Y_1/(Y_1+Y_2)\|_p$ by the closeness of $X$ and $Y$
This question is inspired by the answer to this other question, but I have tried to make it self-contained and to zoom in on the counter-example from this answer.
Suppose $\{(X_n, Y_n)\}_{n=1}^2$ are ...
- 15
0
votes
1
answer
107
views
What's the lower bound for this quantity?
Suppose $p$ is a discrete distribution with $n$ values and the random variable $x$ satisfies $\mathbb{E}_p[x] = 0$ and $|x| < \infty$. Given $\alpha \in (0,1)$, does there exist a lower bound for ...
- 49
1
vote
1
answer
146
views
Upper bound on difference of correlated ratios
Suppose $(x_n, y_n)$ are i.i.d. samples (that is, $x_n$ and $y_n$ are not independent, but $(x_n, y_n)$ is i.i.d. with regards to $(x_m, y_m)$ if $n\ne m$) from a joint distribution, with $0 < x_n, ...
- 15
1
vote
1
answer
191
views
Concentration inequality for square roots
Given a sequence of (not-necessarily-iid) real-valued random variables $X_n$ that converge to $a\in\mathbb{R}$ in probability, suppose we have an exponential concentration inequality of the form
$$
P(|...
- 13
2
votes
2
answers
823
views
Kolmogorov-Smirnov distance and expectation
Let $P$ and $Q$ be two probability measures over $R^n$, with CDF denoted by $F_P,F_Q$, respectively (that is, $F_P(x)=P(\{x'\in R^n:x'\leq x\})$, where $\leq$ is taken componentwise. The Kolmogorov-...
- 333
1
vote
0
answers
67
views
Random matrix theory: accounting for mean
Assume a random matrix, denoted as $X$, which is an $n$ by $T$ matrix, $T\geq n$. While I understand the typical scenario where the random variables $X_{ij}$ are sampled from a $\mathcal{N}(0,\sigma_{...
0
votes
1
answer
165
views
Does this inequality hold for the cumulant generating function?
Suppose a random variable $X$ is zero-mean and the cumulant generating function is
$$
K\left( t \right) =\log \mathbb{E}[e^{tX}].
$$
Given any positive constant $\tau > 0$, does this inequality
$$
\...
- 49
3
votes
1
answer
165
views
Why is the logistic regression model good? (and its relation with maximizing entropy)
Suppose we're trying to train a classifier $\pi$ for $k$ classes that takes as input a feature vector $x\in\mathbb{R}^n$ and outputs a probability vector $\pi(x)\in\mathbb{R}^k$ such that $\sum_{v=1}^...
- 7,527
1
vote
0
answers
112
views
A high probability bound for a Rademacher process
Let $\{x_i(t)\}_{i=1}^n$ be i.i.d. Gaussian processes for $t \in [0, T]$ with
\begin{align*}
\mathbb{E}[x_i(t)] & = 0, \quad \forall i \in [1 : n], \ t \in [0, T] \\
\mathbb{E}[x_i(s) x_i(t)] &...
- 81
1
vote
0
answers
143
views
Estimator for the conditional expectation operator with convergence rate in operator norm
Let $X$ and $Z$ be two random variables defined on the same probability space, taking values in euclidian spaces $E_X$ and $E_Z$, with distributions $\pi$ and $\nu$, respectively.
Let $L^2(\pi)$ ...
- 111
0
votes
1
answer
113
views
Inequality on conditional variance of a vector
I have a random vector $X$ and an event $\mathcal{E}$ such that $\mathbb{P}(\mathcal{E}) = p$. I am trying to show the following inequality :
\begin{equation}
p\mathbb{E}[\|X - \mathbb E [X \vert \...
- 11
0
votes
0
answers
52
views
Classifier-specific lower bounds on the misclassification rate in binary classification
Consider a binary classification problem for $(X,Y)$, and let $\hat{f}$ be a proposed classifier. We wish to bound the misclassification rate $P(\hat{f}(X)\ne Y)$. There are many known lower bounds on ...
- 13
4
votes
2
answers
2k
views
Why MLEs are asymptotically efficient whereas method of moment estimators are not?
Under appropriate regularity conditions it is well-known that Maximum Likelihood Estimation (MLE) produces asymptotically efficient estimators in the sense that their asymptotic covariance is given by ...
2
votes
0
answers
115
views
Equivalence of score function expressions in SDE-based generative modeling
I am studying the paper "Score-Based Generative Modeling through Stochastic Differential Equations" (arXiv:2011.13456) by Yang et al. The authors use the following loss function (Equation 7 ...
- 43
1
vote
1
answer
129
views
Average of cosine similarity between $n$-vectors sampled from $d$-dim hypersphere
I'm having a tough time on this problem, suppose we have $v_i$ for $i=1\cdots n$ unit vectors sampled from a $d$-dimensional hypersfere. How can we evaluate this average over these vectors?
$$
\mathbb{...
- 113
20
votes
0
answers
3k
views
+200
What does a product of many Gaussian matrices converge to?
Let $A$ be a product of $n$ $d\times d$ matrices with IID standard Gaussian entries and consider the value of $g(x)=x f(x)$ where $f(x)$ is the density of squared singular values of $A/\|A\|$.
Is ...
- 2,252
1
vote
1
answer
69
views
Expected value of MGIG distribution
I'm currently dealing with a Gibbs sampler of the multivariate generalized inverse Gaussian distribution (MGIG). In order to check the correctness of the sampler, I'd like to know the expected value ...
- 2,560
3
votes
0
answers
107
views
Supremum process of a Cauchy RV
I've asked the same question on stats.stackexchange a week ago to no avail, so here we go again:
Suppose $X_i$ are $\mathrm{Cauchy}(0,~\gamma)$ IID RV's. Does an expression exist for the CDF of the ...
- 41
2
votes
0
answers
68
views
What is an efficient non-adaptive group testing scheme if the number of defectives, $d$, grows proportionally to the number of items, $n$?
Suppose that for some $p \in \left(0, 1\right)$ and some $n \in \mathbb{N}$, we have $n$ independent Bernoulli random variables, $X_{1}, X_{2}, \dots, X_{n}$, each with mean $p$. We shall call $X_{1}, ...
6
votes
1
answer
274
views
Spectrum asymptotics for a product of $k$ random matrices?
How does the spectrum of a product of $k$ random matrices behave around 0?
In particular, I'm wondering if the CDF of squared singular values behaves as $x^{\frac{1}{k+1}}$ around 0. The result for $k=...
- 2,252
8
votes
3
answers
2k
views
Recommendation for learning mathematical statistics and probability
I can easily find my way reading a book on homological algebra or algebraic geometry, but I tried once reading a book on statistics and... I felt dumb really: I simply do not understand the ...
- 1,031
0
votes
0
answers
49
views
Gaussian white noise model in application
I am interested in applications (to data) of non-parametric statistics, and my question concerned the Gaussian white noise model defined by,
$$
X_{t_1, \ldots, t_d}=f\left(t_1, \ldots, t_d\right) d ...
- 73
3
votes
2
answers
226
views
Recovering measure support from the sequence of I.I.D random variables
Assume we have a Borel probability measure $\mu$ in $\mathbb{R}^n$ and a sequence of $\mu$ distributed I.I.D. random variables $x_n$. Is there a limit formula for $supp(\mu)$, something like closure ...
- 728
0
votes
1
answer
940
views
Derivative of log-likelihood function for Gaussian distribution with parameterized variance
Suppose we have a parameter $\theta \in R^{n}$ that defines some noisy observation $z=\mu(\theta)+\eta, z\in R^{m}$ where the noise follows a Gaussian distribution whose covariance is a function of ...
- 75
5
votes
1
answer
516
views
Bounding the variance of a truncated Gaussian random variable
Suppose $X_1, X_2, X_3 \sim N(0, 1)$ are three independent standard normal random variables. I am interested in showing that:
$$\text{Var}[X_2\mid X_2 \geq X_1 - a, X_1 \leq X_3 + b] < 1,$$
where ...
- 269
2
votes
0
answers
96
views
Limiting value of $\dfrac{1}{m}\mathrm{tr}(FAF^\top (FBF^\top)^{-1})$, where $F$ has iide $N(0,1)$ entries and $A,B$ are deterministic
Let $F=F_{m,d}$ be a random $m \times d$ matrix with iid entries from $N(0,1)$. Let $A=A_d$ and $B=B_d$ be deterministic $d \times d$ positive-definite matrices. In case it helps, it may be assumed ...
- 6,853
3
votes
1
answer
355
views
Quantitative results (with formal proof) on the median approximation of Chi-squared distribution
It is well-known that Chi-squared distribution $X_n$ with degree-$n$ freedom has an approximate formula for its median as $n\left(1-\frac{2}{9n}\right)^3$. Or $(X_n/n)^{\frac{1}{3}}$ is approximately ...
- 457
1
vote
0
answers
39
views
Characteristic function of a Dirichlet Process
Suppose $P \sim \text{DP}(\alpha,G) $ where $G \sim N(0,1)$ is the base measure and $\alpha > 0$ is the concentration parameter. The stick breaking representation says that $P$ can be expressed as \...
- 11
6
votes
0
answers
137
views
Why wavelet methods are not popular anymore in nonparametric statistics?
Back in my master years, I took a nonparametric statistics class. In this class, a few nonparametric methods were presented, but I remember spending a lot of times on methods based on wavelet ...
- 73
1
vote
0
answers
68
views
Limiting value of expectation of $\operatorname{tr}(BR(z))$, where $R(z) := (X^\top X - z I_d)^{-1}$ and $X \sim N_{n,d}(0,A)$
Let $A=A(d)$, and $B=B(d)$ be (sequences of) deterministic positive-definite $d \times d$ matrices and let $X$ be an $n \times d$ random matrix with iid rows from $N(0,A)$. Let $R$ be the resolvent of ...
- 6,853
7
votes
1
answer
569
views
Singular Fisher information matrix and existence of unbiased estimators
I'm doing some research into the Cramer-Rao bound for time of arrival localization and have come across a rather strange result: the FIM is singular, but there exists an unbiased estimator. My ...
- 75
0
votes
1
answer
77
views
Estimation on rotationally-disturbed random vectors
During developing a new statistical estimator, I faced the following problem.
Let $\mathbf{x}_i$ be a sequence of i.i.d. $d$-dimensional random vectors with
\begin{align*}
\mathbf{x}_i = \mathbf{O}...
- 284
3
votes
1
answer
171
views
Variance lower bound for natural exponential family
Let $Q$ be a probability measure on $\mathbb{R}$. Let $$Q_h(dy) = e^{y \cdot h} Q(dy) / M(h) \quad \text{where} \quad M(h) = \int e^{y \cdot h} Q(dy)$$ defined for $h \in (-c,\infty)$ with some $c &...
- 143
4
votes
0
answers
143
views
Projection of log-concave distribution on unit sphere surface
Let $\mathbf X : \Omega \to \mathbb R^d$ be a random vector following a zero mean, identity covariance log-concave distribution.
Is there any known upper bound for the probability density function of $...
- 141
0
votes
1
answer
110
views
Positivity of linear combination of gaussian variables
Consider a collection of independent standard Gaussian variables $w_i$ for $i = 1, 2, \ldots, N$. Define its linear combination $f:=\sum_{i=1}^Na_iw_i+b_i$, where $a_i=pb_i$ ($p$ is a fixed parameter),...
- 49
1
vote
2
answers
331
views
Anti-concentration of gaussian variable
Let $X$ be $\mathcal{N}(\mu,\sigma^2)$ gaussian. Its expectation $\mu$ is positive. Can we derive a lower bound on
$$\mathbb{P}(X\geq\epsilon)\geq g(\epsilon,\mu,\sigma) \text{ where } \epsilon\leq\mu$...
- 405