All Questions
Tagged with pr.probability st.statistics
1,134 questions
1
vote
1
answer
341
views
Form of minimax estimator
Let $\Delta$ be the set of all probability distributions over $\mathbb{N}=\{1,2,\ldots\}$ and fix some $\mathcal{P}\subseteq\Delta$.
Suppose additionally that $\Delta$ is endowed with some norm $||\...
- 128k
4
votes
2
answers
2k
views
Why MLEs are asymptotically efficient whereas method of moment estimators are not?
Under appropriate regularity conditions it is well-known that Maximum Likelihood Estimation (MLE) produces asymptotically efficient estimators in the sense that their asymptotic covariance is given by ...
2
votes
1
answer
187
views
Law of iterated logarithm for quadratic variation of Brownian motion
Let $(\Omega, \mathcal{F}, \mathbb{P})$ denote a probability space supporting a standard Brownian motion $B$. Let $\Pi=\{\pi_n : n \ge 0\}$ denote the sequence of dyadic uniform partitions of the ...
- 3,808
1
vote
0
answers
57
views
Limiting value of expectation of trace of truncated Gram matrix
Let $n$ and $d$ be large positive integers such that $d/n = a \in (0,1)$, fixed. Let $x_1,\ldots,x_n$ be iid random vectors from $N(0,I_d)$. Fix $b \in (0,1]$ and a unit-vector $v \in \mathbb R^d$, ...
- 6,853
0
votes
0
answers
55
views
Modeling player interactions in multi-dimensional rating systems
In traditional rating systems (such as Elo), a player's strength is represented by a single scalar value, which is assumed to be consistent across different opponents. However, in some games, the ...
- 1
2
votes
1
answer
110
views
Lower bound on likelihood of binary outcomes
I am wondering about the following: does there exist a stochastic process $(X_n)_{n \ge 1}$ with values in $\{0,1\}$ on a probability space $(\Omega, \mathcal F, \mathbb P)$ such that for all $n \ge 1$...
CommunityBot
- 1
0
votes
0
answers
140
views
Bounding the difference of weighted-mean estimations
I recently asked a question related to a proof I'm working on as well as a follow-up to that. The detailed counter-examples I received elucidated a lot, but it quickly transpired (i) that I was ...
- 15
0
votes
1
answer
107
views
Bounding $\|X_1/(X_1+X_2) - Y_1/(Y_1+Y_2)\|_p$ by the closeness of $X$ and $Y$
This question is inspired by the answer to this other question, but I have tried to make it self-contained and to zoom in on the counter-example from this answer.
Suppose $\{(X_n, Y_n)\}_{n=1}^2$ are ...
- 128k
0
votes
1
answer
107
views
What's the lower bound for this quantity?
Suppose $p$ is a discrete distribution with $n$ values and the random variable $x$ satisfies $\mathbb{E}_p[x] = 0$ and $|x| < \infty$. Given $\alpha \in (0,1)$, does there exist a lower bound for ...
- 128k
1
vote
1
answer
146
views
Upper bound on difference of correlated ratios
Suppose $(x_n, y_n)$ are i.i.d. samples (that is, $x_n$ and $y_n$ are not independent, but $(x_n, y_n)$ is i.i.d. with regards to $(x_m, y_m)$ if $n\ne m$) from a joint distribution, with $0 < x_n, ...
- 15
6
votes
2
answers
2k
views
Interesting thesis topic on statistical inference that is sufficiently mathematical
Hello , I am a student who's gonna start honours in mathematics . Currently , I am at the stage of finding a suitable honours thesis topic . I've chosen my supervisor , who's research interest is on ...
- 2,821
1
vote
1
answer
191
views
Concentration inequality for square roots
Given a sequence of (not-necessarily-iid) real-valued random variables $X_n$ that converge to $a\in\mathbb{R}$ in probability, suppose we have an exponential concentration inequality of the form
$$
P(|...
- 128k
2
votes
2
answers
823
views
Kolmogorov-Smirnov distance and expectation
Let $P$ and $Q$ be two probability measures over $R^n$, with CDF denoted by $F_P,F_Q$, respectively (that is, $F_P(x)=P(\{x'\in R^n:x'\leq x\})$, where $\leq$ is taken componentwise. The Kolmogorov-...
- 128k
1
vote
0
answers
67
views
Random matrix theory: accounting for mean
Assume a random matrix, denoted as $X$, which is an $n$ by $T$ matrix, $T\geq n$. While I understand the typical scenario where the random variables $X_{ij}$ are sampled from a $\mathcal{N}(0,\sigma_{...
- 6,367
0
votes
1
answer
165
views
Does this inequality hold for the cumulant generating function?
Suppose a random variable $X$ is zero-mean and the cumulant generating function is
$$
K\left( t \right) =\log \mathbb{E}[e^{tX}].
$$
Given any positive constant $\tau > 0$, does this inequality
$$
\...
- 128k
3
votes
1
answer
165
views
Why is the logistic regression model good? (and its relation with maximizing entropy)
Suppose we're trying to train a classifier $\pi$ for $k$ classes that takes as input a feature vector $x\in\mathbb{R}^n$ and outputs a probability vector $\pi(x)\in\mathbb{R}^k$ such that $\sum_{v=1}^...
2
votes
2
answers
625
views
Measuring the independence between the components of a stochastic process
In a context of blind source separation (e.g. you want to extract the voice of a singer from a song), many approaches consist in maximizing the independence between the components of a certain ...
- 6,367
2
votes
1
answer
150
views
Normalized concentration inequality for empirical CDF (iid sum)
Consider the empirical and population CDF,
$$
F_n(t) = \frac{1}{n} \sum_{i=1}^n 1\{X_i \leq t\} \quad \mbox{and} \quad
F(t) = \mathbb{E} [F_n(t)],
$$
where above $X_1, \dots, X_n$ are iid, real-...
CommunityBot
- 1
1
vote
0
answers
100
views
$L_1$ convergence rates for multivariate kernel density estimation
Let $X$ be a random variable on $\mathbb R^d$ with probability density function $f$, and let $X_1,\ldots,X_n$ of $X$ be $n$ iid copies of $X$. Given a bandwidth parameter $h=h_n > 0$ and a kernel $...
1
vote
0
answers
144
views
Estimator for the conditional expectation operator with convergence rate in operator norm
Let $X$ and $Z$ be two random variables defined on the same probability space, taking values in euclidian spaces $E_X$ and $E_Z$, with distributions $\pi$ and $\nu$, respectively.
Let $L^2(\pi)$ ...
1
vote
0
answers
112
views
A high probability bound for a Rademacher process
Let $\{x_i(t)\}_{i=1}^n$ be i.i.d. Gaussian processes for $t \in [0, T]$ with
\begin{align*}
\mathbb{E}[x_i(t)] & = 0, \quad \forall i \in [1 : n], \ t \in [0, T] \\
\mathbb{E}[x_i(s) x_i(t)] &...
- 81
0
votes
0
answers
52
views
Classifier-specific lower bounds on the misclassification rate in binary classification
Consider a binary classification problem for $(X,Y)$, and let $\hat{f}$ be a proposed classifier. We wish to bound the misclassification rate $P(\hat{f}(X)\ne Y)$. There are many known lower bounds on ...
CommunityBot
- 1
2
votes
0
answers
115
views
Equivalence of score function expressions in SDE-based generative modeling
I am studying the paper "Score-Based Generative Modeling through Stochastic Differential Equations" (arXiv:2011.13456) by Yang et al. The authors use the following loss function (Equation 7 ...
- 63.9k
1
vote
1
answer
129
views
Average of cosine similarity between $n$-vectors sampled from $d$-dim hypersphere
I'm having a tough time on this problem, suppose we have $v_i$ for $i=1\cdots n$ unit vectors sampled from a $d$-dimensional hypersfere. How can we evaluate this average over these vectors?
$$
\mathbb{...
4
votes
1
answer
485
views
Expected norms of Wishart matrices
Suppose $x_i \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu,\Sigma)$. What can we say about dependence on $b$ of Frobenius/spectral norm quantities below?
$$f(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^...
CommunityBot
- 1
3
votes
0
answers
107
views
Supremum process of a Cauchy RV
I've asked the same question on stats.stackexchange a week ago to no avail, so here we go again:
Suppose $X_i$ are $\mathrm{Cauchy}(0,~\gamma)$ IID RV's. Does an expression exist for the CDF of the ...
- 41
2
votes
0
answers
68
views
What is an efficient non-adaptive group testing scheme if the number of defectives, $d$, grows proportionally to the number of items, $n$?
Suppose that for some $p \in \left(0, 1\right)$ and some $n \in \mathbb{N}$, we have $n$ independent Bernoulli random variables, $X_{1}, X_{2}, \dots, X_{n}$, each with mean $p$. We shall call $X_{1}, ...
3
votes
3
answers
2k
views
Is the min function ever an unbiased estimator for the mean?
Given $n$ i.i.d. variables $X_1$ to $X_n$ with an unknown probability distribution, the sample average is an unbiased estimator for the mean of the distribution. Is there some non-trivial probability ...
6
votes
1
answer
274
views
Spectrum asymptotics for a product of $k$ random matrices?
How does the spectrum of a product of $k$ random matrices behave around 0?
In particular, I'm wondering if the CDF of squared singular values behaves as $x^{\frac{1}{k+1}}$ around 0. The result for $k=...
- 188k
8
votes
3
answers
2k
views
Recommendation for learning mathematical statistics and probability
I can easily find my way reading a book on homological algebra or algebraic geometry, but I tried once reading a book on statistics and... I felt dumb really: I simply do not understand the ...
- 128k
0
votes
0
answers
49
views
Gaussian white noise model in application
I am interested in applications (to data) of non-parametric statistics, and my question concerned the Gaussian white noise model defined by,
$$
X_{t_1, \ldots, t_d}=f\left(t_1, \ldots, t_d\right) d ...
- 63.9k
7
votes
1
answer
569
views
Singular Fisher information matrix and existence of unbiased estimators
I'm doing some research into the Cramer-Rao bound for time of arrival localization and have come across a rather strange result: the FIM is singular, but there exists an unbiased estimator. My ...
- 128k
3
votes
2
answers
226
views
Recovering measure support from the sequence of I.I.D random variables
Assume we have a Borel probability measure $\mu$ in $\mathbb{R}^n$ and a sequence of $\mu$ distributed I.I.D. random variables $x_n$. Is there a limit formula for $supp(\mu)$, something like closure ...
- 2,101
18
votes
9
answers
25k
views
Why isn't likelihood a probability density function?
I've been trying to get my head around why a likelihood isn't a probability density function. My understanding says that for an event $X$ and a model parameter $m$:
$P(X\mid m)$ is a probability ...
0
votes
1
answer
940
views
Derivative of log-likelihood function for Gaussian distribution with parameterized variance
Suppose we have a parameter $\theta \in R^{n}$ that defines some noisy observation $z=\mu(\theta)+\eta, z\in R^{m}$ where the noise follows a Gaussian distribution whose covariance is a function of ...
- 128k
5
votes
1
answer
516
views
Bounding the variance of a truncated Gaussian random variable
Suppose $X_1, X_2, X_3 \sim N(0, 1)$ are three independent standard normal random variables. I am interested in showing that:
$$\text{Var}[X_2\mid X_2 \geq X_1 - a, X_1 \leq X_3 + b] < 1,$$
where ...
2
votes
0
answers
96
views
Limiting value of $\dfrac{1}{m}\mathrm{tr}(FAF^\top (FBF^\top)^{-1})$, where $F$ has iide $N(0,1)$ entries and $A,B$ are deterministic
Let $F=F_{m,d}$ be a random $m \times d$ matrix with iid entries from $N(0,1)$. Let $A=A_d$ and $B=B_d$ be deterministic $d \times d$ positive-definite matrices. In case it helps, it may be assumed ...
- 6,853
3
votes
1
answer
355
views
Quantitative results (with formal proof) on the median approximation of Chi-squared distribution
It is well-known that Chi-squared distribution $X_n$ with degree-$n$ freedom has an approximate formula for its median as $n\left(1-\frac{2}{9n}\right)^3$. Or $(X_n/n)^{\frac{1}{3}}$ is approximately ...
- 128k
1
vote
0
answers
39
views
Characteristic function of a Dirichlet Process
Suppose $P \sim \text{DP}(\alpha,G) $ where $G \sim N(0,1)$ is the base measure and $\alpha > 0$ is the concentration parameter. The stick breaking representation says that $P$ can be expressed as \...
- 11
6
votes
0
answers
137
views
Why wavelet methods are not popular anymore in nonparametric statistics?
Back in my master years, I took a nonparametric statistics class. In this class, a few nonparametric methods were presented, but I remember spending a lot of times on methods based on wavelet ...
- 73
1
vote
0
answers
68
views
Limiting value of expectation of $\operatorname{tr}(BR(z))$, where $R(z) := (X^\top X - z I_d)^{-1}$ and $X \sim N_{n,d}(0,A)$
Let $A=A(d)$, and $B=B(d)$ be (sequences of) deterministic positive-definite $d \times d$ matrices and let $X$ be an $n \times d$ random matrix with iid rows from $N(0,A)$. Let $R$ be the resolvent of ...
- 6,853
0
votes
1
answer
77
views
Estimation on rotationally-disturbed random vectors
During developing a new statistical estimator, I faced the following problem.
Let $\mathbf{x}_i$ be a sequence of i.i.d. $d$-dimensional random vectors with
\begin{align*}
\mathbf{x}_i = \mathbf{O}...
- 128k
3
votes
1
answer
171
views
Variance lower bound for natural exponential family
Let $Q$ be a probability measure on $\mathbb{R}$. Let $$Q_h(dy) = e^{y \cdot h} Q(dy) / M(h) \quad \text{where} \quad M(h) = \int e^{y \cdot h} Q(dy)$$ defined for $h \in (-c,\infty)$ with some $c &...
- 143
4
votes
0
answers
143
views
Projection of log-concave distribution on unit sphere surface
Let $\mathbf X : \Omega \to \mathbb R^d$ be a random vector following a zero mean, identity covariance log-concave distribution.
Is there any known upper bound for the probability density function of $...
- 141
0
votes
1
answer
110
views
Positivity of linear combination of gaussian variables
Consider a collection of independent standard Gaussian variables $w_i$ for $i = 1, 2, \ldots, N$. Define its linear combination $f:=\sum_{i=1}^Na_iw_i+b_i$, where $a_i=pb_i$ ($p$ is a fixed parameter),...
- 49
1
vote
2
answers
331
views
Anti-concentration of gaussian variable
Let $X$ be $\mathcal{N}(\mu,\sigma^2)$ gaussian. Its expectation $\mu$ is positive. Can we derive a lower bound on
$$\mathbb{P}(X\geq\epsilon)\geq g(\epsilon,\mu,\sigma) \text{ where } \epsilon\leq\mu$...
- 128k
6
votes
2
answers
774
views
Probability of winning game whereby $T+1$ heads in a row of a coin flip is required to win where $T$ is the number of cumulative tails flipped
I have a weird question which probably seems out of place here but it has proven more difficult than anticipated. I am going to describe the game without showing work toward a solution. Numerically, ...
- 1,986
9
votes
4
answers
3k
views
How can the Kalman filter be adapted to handle binary observations?
Imagine a coin with a time-varying probability of coming up heads. (For example, perhaps the probability follows a random walk that is constrained to live in $[0, 1]$. And say we have some ...
0
votes
1
answer
61
views
What can we say about the order of convergence of a critical point of Gaussian mixture density to its limit when the parameter $h$ goes to $0?$
Density of Gaussian mixture with $n$ components is given by:
$$f(x):=C \sum_{i=1}^{n}e^{-\frac{1}{2}||\frac{x-x_i}{h}||^2}, x_i \in \mathbb{R}^d, h > 0$$
where $C$ is a normalization constant ...
- 128k
17
votes
1
answer
10k
views
Conjugate prior of the Dirichlet distribution?
What is the conjugate prior distribution of the Dirichlet distribution?
Edit: Since I asked this question many years ago, I've written a Python library for working with exponential families. Maximum ...
- 598