All Questions
Tagged with pr.probability st.statistics
1,134 questions
6
votes
2
answers
774
views
Probability of winning game whereby $T+1$ heads in a row of a coin flip is required to win where $T$ is the number of cumulative tails flipped
I have a weird question which probably seems out of place here but it has proven more difficult than anticipated. I am going to describe the game without showing work toward a solution. Numerically, ...
user504691
0
votes
1
answer
61
views
What can we say about the order of convergence of a critical point of Gaussian mixture density to its limit when the parameter $h$ goes to $0?$
Density of Gaussian mixture with $n$ components is given by:
$$f(x):=C \sum_{i=1}^{n}e^{-\frac{1}{2}||\frac{x-x_i}{h}||^2}, x_i \in \mathbb{R}^d, h > 0$$
where $C$ is a normalization constant ...
- 1,467
2
votes
0
answers
87
views
A complex problem involving densities (likelihood functions) and optimization
Consider the following autoregressive process with normal errors:
\begin{equation}\label{7YlUV4i8nuO}\tag{I}
y_t = \phi y_{t-1}+ u_t, \quad u_t \overset{iid}{\sim} N(0,\sigma^2)
\end{equation}
We ...
- 13
5
votes
1
answer
2k
views
Mathematics research relating to machine learning
What branch/branches of math are most relevant in enhancing machine learning (mostly in terms of practical use as opposed to theoretical/possible use)? Specifically, I want to know about math research ...
- 173
1
vote
1
answer
56
views
Covariance inequality for left skewed distributions
Consider a left skewed random variable $X$ with mean $1$, median $>1$ and support on $[0,2)$. Suppose we have a class of functions $\mathbf{G}$ and each of it's members satisfy $G(x): [0,\infty) ...
- 45
1
vote
1
answer
86
views
Is the main part of certain exponential family sub-Gaussian?
$X$ is in the form of exponential family i.e.
$$\mathbb{P_\theta}x = h(x)e^{\langle \theta,T(x)\rangle-\phi(\theta)}$$
where $\theta\in \mathbb{R}^d$. If $\nabla\phi(\theta)$ is L-Lipschitz i.e.
$$\...
- 81
0
votes
1
answer
198
views
Spectral norm of matrices of bounded random variables
Assume $A\in \mathbb{R}^{n\times n}$ with each entry being i.i.d. bounded r.v. in $[a,b]$, is $\Vert A\Vert_2$ is sub-Gaussian?
Intuitively, since $\{A_{ij}\}_{i,j=1,...,n}$ is bounded, then
$$\Vert A ...
- 81
2
votes
2
answers
323
views
Continuity of Nash equilibrium for a family of games
The question may be too vague, but ultimately in search of various (counter)examples or theorems to exhibit the following:
Do continuous families $t\mapsto G_t$ of "games" (say each $G_t$ is ...
- 17.5k
4
votes
0
answers
309
views
When is $\prod_{i=0}^\infty (I-x_i x_i^T)=0$ for zero-centered Gaussian $x_i$?
Suppose $x_i\in \mathbb{R}^d$ is sampled IID from $\mathcal{N}(0,H)$. Let $A_i=(I-x_i x_i^T)$ and assume $d$ is large. What are necessary conditions for the following to converge with probability 1?
$...
- 2,442
2
votes
1
answer
462
views
Converse of the Herbst argument?
Background
For a real-valued random variable $X$, define its entropy by $H(X) = E[\phi(X)] - \phi(E[X])$, where $\phi(u) = u \log u$.
It can be shown that, if the entropy satisfies the bound
$$
H(e^{\...
- 163
2
votes
1
answer
415
views
Bounding Kullback-Leibler
Suppose we have a probability distribution $P$ on a finite set $S$. We draw $N$ i.i.d. samples according to $P$ and use these samples to define an empirical distribution $R$. We measure the Kullback-...
- 3,979
0
votes
2
answers
261
views
What mathematical formalism might be used to disprove natural selection, on the basis that there are too many independent genetic parameters? [closed]
I have nagging doubts that the random genetic mutation process of natural selection is sufficient to explain evolution, even when coupled with sexual selection (Darwin proposed that evolution is ...
- 41
4
votes
1
answer
258
views
When is $\prod_{i=0}^\infty (I-x_i x_i^T)=0$ for isotropic Gaussian $x_i$?
Suppose $x_i$ is sampled IID from isotropic zero-centered Gaussian random variable in $d$ dimensions with covariance $\Sigma=c*I$. When is the following true with probability 1?
$$\prod_{i=0}^\infty (...
- 2,442
0
votes
1
answer
161
views
Analogues of Kac-Bernstein characterisation theorem for non-normal distributions
Let $X,Y$ be two independent random variables.
The Kac-Bernstein theorem states that if $X+Y,X-Y$ are also independent, then $X,Y$ are Normal.
Are there analogues of this theorem for non-normal, ...
- 1,459
0
votes
0
answers
60
views
Norms of Wigner matrices under power law decay
Suppose $\Sigma=\operatorname{diag}(h)$ where $h=(1^{-p},2^{-p},3^{-p},\ldots,d^{-p})$ and $p> 1$
$X$ is a matrix with $b$ rows sampled independently from $\operatorname{Normal}(0,\Sigma)$
Suppose $...
- 2,442
1
vote
2
answers
2k
views
Upper bound about Gaussian tail bound
From the definition of sub-Gaussian distribution $X$ w.r.t. $\sigma$ i.e.
$$\mathbb{P}(|X-\mathbb{E}(X)|\geq t) \leq 2 \exp(-\frac{t^2}{2\sigma^2}).$$
It's natural that when $X \sim \mathcal{N}(\mu, \...
- 81
0
votes
0
answers
78
views
Kernel density estimation is sub-gaussian
Let $X_1, ..., X_n$ be i.i.d. samples drawn from a pdf $f(x)$ on the real line. The kernel density estimator is defined as follows,
$$\hat{f_n}(x) = \frac{1}{nh}\sum_1^n K(\frac{x-X_k}{h})$$
where $K:\...
- 81
1
vote
0
answers
124
views
Using projections to determine equidistribution
Suppose I have a collection of points on $\mathbb{S}^{n-1} \subset \mathbb{R}^n.$ I want to know that they are equidistributed (if you want to be more precise, you have a sequence of such collections, ...
- 96.4k
3
votes
0
answers
93
views
Asymptotic approximation of Fisher information matrix for small Gaussian perturbation
Let
$$
X=Y/a+b+\epsilon Z,
$$
where $Y\sim\operatorname{Poisson}(\lambda)$ and $Z\sim\mathcal N(0,1)$ are independent. Also define $\theta=(\lambda,a,b,\epsilon)$. The Fisher information matrix
$$
...
2
votes
1
answer
909
views
Can we use Bernstein's inequality without knowledge of variance?
I have a question about Bernstein’s inequality for bounded random variables.
Its statement is the following. Let $X_1, \ldots, X_N$ be independent, mean zero random variables with $|X_i| \leq K \ (i = ...
- 163
4
votes
1
answer
223
views
Existence of disintegrations for improper priors on locally-compact groups
In wide generality, the disintegration theorem says that Radon probability measures admit disintegrations. I'm trying to understand the case when we weaken this to infinite measures, specifically ...
- 8,512
4
votes
1
answer
287
views
Local maxima of the sum of Gaussian functions in *multiple dimensions* are always strict local maxima - prove/disprove/prove conditionally?
This is a follow up of the question in one dimension, that asked to show that the all the maxima of the sum of Gaussian
$$f_n(x):= \sum_{i=1}^{n}e^{-(x-x_i)^2}, x_1 < x_2 < \dots < x_n$$
are ...
- 1,467
3
votes
2
answers
505
views
Precise asymptotics for moments of order statistics of normal distribution
Let $X_1, \cdots, X_n \sim N(0,1)$ be i.i.d. normal random variates. I am interested in understanding the first two moments of the quasi-range $X_{(n)}-X_{(n-1)}$ (i.e., the maximum value minus the ...
- 151
2
votes
0
answers
62
views
Probability bounds of some ranked version of Dirichlet distribution
Recently I have come across a distribution defined on the open ranked simplex $\nabla^{n-1}_+ = \{\vec x \in \mathbb{R}^n:\sum_{k=1}^n x_k =1, x_1 \geq x_2 \geq \cdots \geq x_n > 0\}$, whose ...
- 21
9
votes
1
answer
723
views
Popular mistakes in probability
$\DeclareMathOperator\Var{Var}\DeclareMathOperator\Bern{Bern}\DeclareMathOperator\Pois{Pois}$Question: What not-trivial mistakes do students often make when solving problems in probability theory, ...
0
votes
1
answer
177
views
Under which conditions Mean Square Continuity implies Sample Continuity for Gaussian Processes?
First, let us give the setting.
Let $(\Omega, \Sigma, \mathbf{P})$ be a probability space, let $T$ be some interval of time, and let $X: T \times \Omega \rightarrow S$ be a stochastic process.
By Mean ...
- 141
0
votes
1
answer
218
views
Is the unconditional variance of a RV an upper bound for the variance of any conditional expectation of the RV?
Let $X$ and $Y$ be continuous random variables with finite first and second moments. Then, is it true that $Var[X]\geq Var[E(X|Y)]$?
1
vote
1
answer
93
views
An inequality relating $\ell_1$ distance of input and output of a Markov krnel
Let $K$ be a Markov kernel from $\mathcal{X}$ to $\mathcal{Y}$, i.e., $K(\cdot|x)$ is a probability measure on $\mathcal{Y}$ for all $x\in \mathcal{X}$.
Let $\mu$ and $\nu$ be two probability measures ...
- 1,109
2
votes
1
answer
351
views
Lower bound on sum of independent heavy-tailed random variables
I have a sum of $n$ i.i.d random variables $X_i$ such that $E[X_i] = 0$,$\mathrm{E}[|X_i|^{1 + \delta}]$ exists for some $0 < \delta < 1$ but $\mathrm{E}[|X_i|^{1 + \delta+ \epsilon}]$ does not ...
- 23
4
votes
1
answer
168
views
Existence of copula bound pointwise strictly smaller than the Fréchet-Hoeffding upper bound
Consider bivariate copulas $C_1$ and $C_2$ with $\max\{C_1(u,v), C_2(u,v)\}< M_2(u,v)$ for all $u,v \in(0,1)$, where $M_2(u,v) := \min\{u,v\}$ is the Fréchet-Hoeffding upper bound.
Is there a ...
- 143
2
votes
1
answer
122
views
Justification of the use of residual plot
$\DeclareMathOperator\Cov{Cov}$Backround of my Question
Let $Y$ be the response variable, $\mathbb{X}$ be the explanatory variables. The ultimate goal of prediction is finding a function $f^{*}$ that ...
- 23
0
votes
1
answer
59
views
Given positive $\epsilon$ and $c$, find a density $\phi$ such that $t\phi(\epsilon/t) \ge c \|\phi'\|_\infty$ for all positive $t$
A nice density (on $\mathbb R$) is function $\phi:\mathbb R \to \mathbb R$ such that
(1) $\phi(x) \ge 0$ for all $x \in \mathbb R$,
(2) $\int_{-\infty}^\infty \phi(x) \mathrm{d}x = 1$,
(3) $\phi$ is ...
- 6,853
6
votes
1
answer
248
views
Violating an order statistic inequality?
[Edit: for posterity, I'm adding two small comments to the code explaining how to fix it, in light of Iosef Pinelis' answer below. Look for "Should be:" to find the corrections.]
Suppose we ...
- 3,979
1
vote
0
answers
48
views
Sample complexity of estimating a doubly stochastic matrix
Let $P\in\mathbb{R}^{n\times n}$ be a doubly-stochastic matrix. That is:
$$P(x,y)\geq 0,\quad \sum_xP(x,y)=1,\quad \sum_yP(x,y)=1.$$
I would like to know if lower and upper bounds on the sample ...
- 111
2
votes
1
answer
150
views
Normalized concentration inequality for empirical CDF (iid sum)
Consider the empirical and population CDF,
$$
F_n(t) = \frac{1}{n} \sum_{i=1}^n 1\{X_i \leq t\} \quad \mbox{and} \quad
F(t) = \mathbb{E} [F_n(t)],
$$
where above $X_1, \dots, X_n$ are iid, real-...
- 410
2
votes
1
answer
1k
views
Bootstrapping and the central limit theorem
I have been looking into bootstrapping lately and although I believe to have understood the basic process somewhat, I am fuzzy on the mathematical details. I will begin with my understanding of what ...
- 149
1
vote
1
answer
199
views
Rademacher complexity for a family of bounded, nondecreasing functions?
Let $\{\phi_k\}_{k=1}^K$ be a family of functions mapping from an interval $[a, b]$ to $[-1, 1]$.
That is, $\phi_k \colon[ a,b] \to [-1, 1]$ are nondecreasing maps on some finite interval $[a, b] \...
- 410
4
votes
1
answer
626
views
How to get the lower bound of the following $\tau$?
Let $A=\{a_{ij}\}_{1\le i,j\le n}$ be an $n$ by $n$ normalized Gaussian random matrix with $E[a_{ij}]=0$ and $E[a_{ij}^2]=1/n$. Ordering its eigenvalues by $\lambda_1\le \lambda_2\le \cdots \lambda_n$ ...
- 288
2
votes
2
answers
228
views
Minimal conditions on random vector $X \in R^n$ to ensure that $\lim_{t\to 0^+}\sup_{\|w\|_p = 1}\sup_{u \in \mathbb R}\mathbb P(|X'w-u| \le t)=0$
Let $X$ be a random variable on $\mathbb R^n$ and let $S_p^n := \{w \in \mathbb R^n \mid \|w\|_p = 1\}$ be the unit-sphere w.r.t to the $\ell_p$-norm in $\mathbb R^n$. We will be particularly ...
- 6,853
0
votes
1
answer
327
views
Deduce that a function is zero on interval $[0,M]$
I have been thinking about this for the last few days but I was not able to produce a definitive answer.
Take an integrable function $g$ that maps in $\mathbb{R}$ and with domain contained in $[0,M]$ (...
- 141
1
vote
1
answer
176
views
Tight upper-bounds for the Gaussian width of intersection of intersection of hyper-ellipsoid and unit-ball
Let $\Lambda$ be a positive-definite matrix of size $n$ and let $R \ge 0$, which may depend on $n$. Consider the set $S := \{x \in \mathbb R^n \mid \|x\|_2 \le R,\,\|x\|_{\Lambda^{-1}} \le 1\}$ where $...
- 6,853
3
votes
1
answer
206
views
Asymptotic results for smallest gap of Gaussian random matrix
For a symmetric Gaussian random matrix $G=\{G\}_{1\le i,j \le n}$ with iid $E[G_{ij}]=0$ and $E[G_{ij}^2]=1/n$ (it is normalized), ordering its eigenvalues $\lambda_1\le \lambda_2\le\cdots \lambda_n$.
...
- 288
1
vote
0
answers
24
views
Minimax statistical estimation of proximal transform $\mbox{prox}_g(\theta_0)$, from linear model data $y_i := x_i^\top \theta_0 + \epsilon_i$
tl;dr: My question pertains the subject of minimax estimation theory (mathematical statistics), in the context of linear regression.
Given a vector $\theta_0 \in \mathbb R^d$, consider the linear ...
- 6,853
1
vote
1
answer
229
views
Gaussian width of intersection of cube and ball in high-dimensional euclidean space
Let $d$ be a large positive integer and fix $r \ge 0$. Set $S := B_2^n \cap [-r,r]^d$, where $B_2^d$ is the euclidean unit-ball in $\mathbb R^d$. Finally, let $\omega(S)$ be the Gaussian width of $S$, ...
- 6,853
1
vote
1
answer
318
views
How to calculate this limit (if exist)?
I have just asked the calculation of the following summation see here $$S(a,b,m,n_1,n_2)=\sum_{k=0}^m a^k b^{m-k} {n_1\choose k} {n_2\choose m-k}, $$
which is motivated by the calculation of the ...
- 57
0
votes
1
answer
82
views
WLLN for bootstrap means of stationary ergodic processes?
Setup:$\quad$
Suppose that $(X_n)$ is a stationary ergodic process with $E|X_1|<\infty$.
Given $X^{(n)}=(X_1, \dots, X_n)$, select a standard Efron bootstrap subsample $(X_{n,1}^*, \dots, X_{n,m(n)}...
- 127
1
vote
0
answers
82
views
Concavity of expected size of a maximum matching (in a bipartite graph) w.r.t. edge probability
Given a n*n bipartite graph where each edge (between any two nodes on the opposite side) is formed i.i.d. with probability $p$, can we show a concavity result on the expected size of a maximum ...
- 53
3
votes
1
answer
549
views
Does $E[1/f]\overset{d}\to 1/E[f]$ for $\operatorname{Tr}H=1,\operatorname{Tr}H^2=0.5$?
Suppose $x$ is a Gaussian random variable in $d$ dimensions with $H=E[xx^T],\ \operatorname{Tr}(H)=1,\operatorname{Tr}(H^2)=0.5$. Take $m$ I.I.D. samples of $x$ and stack them as rows of $X$.
Is it ...
- 2,442
1
vote
1
answer
385
views
How fast does this Gaussian random walk move away from the origin?
Suppose $z_i$ are IID zero-centered $d$-dimensional Gaussian random variables with unit-trace covariance $\Sigma$ and $g(z_i)$ is the sum of its components.
Consider the following random walk:
$$x_s=\...
- 2,442
3
votes
1
answer
210
views
Probabilistic Taylor theorem for concave functions
This paper proves a probabilistic version of Taylor's theorem
\begin{equation*}
\mathbb{E}g(X) = \sum_{k=0}^{n-1} \frac{g^{(k)}(0)}{k!} \mathbb{E}X^k + \frac{\mathbb{E}X^n}{n!} \mathbb{E} g^{(n)}(X_{(...
- 45