All Questions
Tagged with pr.probability st.statistics
1,134 questions
5
votes
0
answers
485
views
Hierarchical Random Walk (also known as Hierarchical Hidden Markov Model)
Let us consider the following hierarchical (recursive) random walk model, which is also known as the hierarchical hidden Markov model in computer science (https://en.wikipedia.org/wiki/...
- 1,127
1
vote
1
answer
711
views
Sequential sampling of Gaussian and von Mises-Fisher Random Variable
I don't find any article discussing this problem, so I dare to ask it.
Suppose we are dealing with a data $x_0 \in \mathbb{R}$ and a function $f:\mathbb{R} \to \mathbb{R}$. Say we repeatedly apply $f$...
- 44.4k
9
votes
2
answers
560
views
Integrating a simple exponential over the space of matrices that define a metric
I want to interpret an $n\times n$ matrix $D$ as a set of pairwise distances, and assume that $D$ obeys metric properties. Namely, $D_{ii} = 0$, $D_{ij} \geq 0$, $D_{ij} = D_{ji}$ and $D_{ij} \leq D_{...
- 44.4k
3
votes
0
answers
436
views
Rank of Hadamard product with random matrices
I do research in statistics and am not sure whether the following is considered research level or not in mathematics. If it isn't, I'm happy because that means the answer is probably known and I can ...
4
votes
1
answer
294
views
Finding high-dimensional correlation matrices that are both sparse and low-rank
Let $\boldsymbol{R}$ be the correlation matrix of $X_i,i=1,\dots,p$ with a large $p\gg q=\text{rank}(\boldsymbol{R})$. Is that reasonable to assume that $\boldsymbol{R}$ is both (approximately) sparse ...
CommunityBot
- 1
6
votes
1
answer
2k
views
Brownian motion and its maximum and its minimum
Let $W_u, 0\leq u \leq t$ be Brownian motion.
Let $m_t= min_{0\leq u\leq t} W_u$ and $M_t = max_{0 \leq u \leq t} W_u$.
The fact that $(M_t , W_t)$ is absolutely continuous with respect to Lebesgue ...
- 29.8k
1
vote
1
answer
146
views
Conformal prediction for the case of single tailed events
I'll start with a motivating example and only then proceed to the question.
Consider a list of total packages of milk that were purchased on 9 consecutive days on a given store,
$z_1,\ldots,z_9 = 1,...
2
votes
1
answer
269
views
Square integrable conditional expectations as projections
I see this page Ordinary least square and random projection, and I am thinking that how $L^2$ integrable random variables be regarded as projections over a defined filtration sequence $\mathcal{F_n}$ ?...
- 8,071
5
votes
3
answers
117
views
Looking for a certain kind of a distribution
Is there any probability distribution supported on a compact or a half-open interval (of $\mathbb{R}$) such that if a vector $\vec{x} \in \mathbb{R}^n$ is sampled by sampling its coordinates like that ...
- 4,529
2
votes
2
answers
351
views
Weak convergence for discrete-time processes using characteristic functions
I am looking for a good reference about the analogues of the Bochner Theorem and the Lévy Continuity Theorem
for probability measures on $\mathbb{R}^{\mathbb{N}}$ with the product topology.
...
3
votes
1
answer
253
views
Can we find an Stein operator characterizing a distribution without density function?
It is known that Stein operator characterizes a probability distribution and there are a lot of ways of find a Stein operator.
For example, if $Z$ is the standard normal distribution, with pdf(...
- 8,071
3
votes
1
answer
650
views
Poisson process with stochastic intensity correlated with a Brownian Motion
I am currently confused with the moment of non-homogeneous compound Poisson process and a Brownian Motion. I know that generally Poisson Process and Brownian Motion are independent if they are adapted ...
- 593
4
votes
0
answers
91
views
What is the entropy of binomial decay?
Let's play a game. I start with $N$ indistinguishable tokens, and I wait $T$ turns. Every turn, each token has probability $p$ of disappearing. I want an analytic formula for the entropy of this ...
- 141
15
votes
1
answer
1k
views
Table with the most seated customers in Chinese restaurant process
Suppose we have some initial configuration of people seated at some tables. We start taking new customers and seat them following Chinese restaurant process. Is there some known work on finding the ...
CommunityBot
- 1
7
votes
1
answer
624
views
Expectation involving maximum of Gaussian variables
Let $X\sim N(0, I_d)$ be a $d$-dimensional Gaussian random vector. Let $W_1, \ldots, W_k \in \mathbb{R}^d$ be $k$ fixed vectors in general positions. It is clear that $w_i^\top X, \ldots, w_k^\top X$ ...
3
votes
1
answer
150
views
Convex lower bound for probability that a random subset of [n] has cardinality at most k
For $n\in\mathbb{N}$, the probability that a random subset of $[n]=\{1,\cdots n\}$ has cardinality at most $k$ is $f_k(n)=2^{-n}\sum\limits_{i=0}^k{n\choose i}$. I'm looking for a lower bound $g_k(x)\...
- 30.1k
4
votes
0
answers
188
views
Distributions over permutation groups $\mathcal{S}_n$
Partly inspired by recent developments in enumeration of pattern avoiding permutations, which is known to be connected with Brownian excursions [Hoffman&Rizzolo]. The exciting milestone is the ...
- 8,071
3
votes
1
answer
157
views
Bound for expectation of function of 3 normal distributions
Let $X,Y,Z$ be three standard normal distribution. Let $\rho_{XY},\rho_{YZ},\rho_{XZ}$ be the correlation between those random variables.
Let $f()$ be a monotone, odd, bounded, and differentiable ...
- 2,600
4
votes
1
answer
225
views
Multivariate Zero-Bias Transform
The zero-bias transform for a univariate random variable $W$ is defined as a random variable $W^*$ satisfying
\begin{align}
\mathbb{E} [ W \cdot f(W )] = \mathbb{E} [ f' (W^*)]
\end{align}
for any ...
CommunityBot
- 1
2
votes
0
answers
149
views
Min Max Equality in Information Theory
Let $\mathcal{Y}$ and $\mathcal{X}$ be finite sets and let $Q_Y$ be a fixed probability mass function on $\mathcal{Y}$. Also, let $P_{X | Y}$ be some fixed conditional distribution on $\mathcal{X} \...
- 8,071
1
vote
1
answer
140
views
Reference request: Cover times, Mixing Times and DGFF applied in statistics?
I am trying to find if in active research in statistics, there is interest in mixing times, cover times of graphs, and/or the discrete Gaussian free field?
I haven't found anything so far for the ...
CommunityBot
- 1
10
votes
1
answer
210
views
Distribution of the maximum of the norm of k-averages of n i.i.d. d-dimensional random vectors
Suppose $X_1, ... X_n$ are i.i.d. random vectors in $d$-dimensional space (i.e., $R^d$) with continuous centrally symmetric density function $f(\cdot)$ (i.e., symmetric with respect to the origin). ...
- 8,071
1
vote
1
answer
122
views
Variance bound of a functional
$X_1,\ldots,X_n$ are i.i.d standard normal random variables.
$a_1,\ldots, a_n$ are constants with $a_i \in [\kappa_1, \kappa_2]$ for all $i$ and $\kappa_1>0$.
$\hat c_n$ is given as the solution ...
5
votes
1
answer
372
views
What are some of results in low dimensional statistics that do not hold in high dimensions?
This question is partially inspired by the following MO post: What are some of the surprising results of finite sample statistical estimation? and current heated research front of high dimensional ...
- 8,071
2
votes
0
answers
448
views
Relation between pseudo-dimension and Rademacher complexity
With techniques of Dudley's entropy bound and Haussler's upper bound one can show that there exists a constant $C$ such that any class of $\{0,1\}$ indicator functions with Vapnik-Chervonenkis ...
- 517
5
votes
1
answer
567
views
Donsker's Theorem for triangular arrays
I should mention that I already posed this question on Math Stack Exchange, but didn't receive much feedback.
Assume we have a sequence of smooth i.i.d. random variables $(X_i)_{i=1}^{\infty}$. Given ...
- 138
3
votes
0
answers
98
views
Asymptotic results on statistical graph models
This post is partly inspired by this post.
Reference request: results on the asymptotic distribution of singular values related to a random orthogonal matrix
While it is well-known that two basic ...
- 8,071
8
votes
1
answer
2k
views
Taylor expansion of cumulant generating function
For the characteristic function $\mathbf E e^{i t X}$ of a random variable $X$ with $n+1$ finite moments, there is the well known and easy to prove bound on the remainder of the Taylor series
$$\left\...
- 8,071
3
votes
1
answer
940
views
What is the mathematical characterization of sufficient statistics of a given $\sigma$-dominated probability model?
Given a probability model $\mathcal{P}=\{P_{\theta},\theta \in \Theta \}$ dominated by a $\sigma$-finite measure $\lambda$ (e.g. Lebesgue measure) on a locally compact space $\cal{X}$ along with $\...
CommunityBot
- 1
2
votes
1
answer
2k
views
Bounds on the eigenvalues of the covariance matrix of a sub-Gaussian vector
Suppose that $\boldsymbol{x}\in\mathbb{R}^n$ is subgaussian random vector of variance proxy $\sigma^2$, i.e.,
$$\forall \boldsymbol{\alpha}\in\mathbb{R}^n: \quad \quad \mathbb{E}\left[ \exp\right(\...
- 8,071
1
vote
0
answers
201
views
Rank of cross-covariance matrix
Let $\boldsymbol{X}=(X_1,\dots,X_p)^T$ and $\boldsymbol{Y}=(Y_1,\dots,Y_q)^T$ be two random vectors. Denote $r_x=\text{rank}(\text{Cov}(\boldsymbol{X})),r_y=\text{rank}(\text{Cov}(\boldsymbol{Y})), r_{...
- 19.6k
1
vote
0
answers
282
views
Strict monotonicity of conditional variances
Let $K \geq 2$ be a positive integer and $C$ be any $K \times K$ non-singular matrix (if necessary, can assume that all $K$ rows of $C$ are needed to span the coordinate row vector $e_1'$). For ...
- 3,968
2
votes
2
answers
1k
views
Lower bound on number of samples for an epsilon delta approximation matching the Chernoff bound
So we have two biased coins, one comes out head w.p. $1/2+\epsilon$ and the other w.p. $1/2-\epsilon$. How many times should we flip these two coins to be able to tell them apart w.p. at least $\delta$...
- 6,541
10
votes
4
answers
645
views
Expected value of Bernoulli quadratic forms
Let $\mathbf{Y}\in\mathbb{R}^{n\times n}$ be a symmetric matrix. Let $\mathbf{x}\in\mathbb{R}^n$ be random vectors with entries i.i.d. $\pm 1$ with equal probability. I'm interested in a lower bound ...
- 3,946
3
votes
1
answer
213
views
Is this generalization bound proof wrong?
This is an ICML02 paper by Garg, Har-Peled & Roth:
http://sarielhp.org/p/01/bounds/bounds.pdf
The equation after eq. (3) is the well-known symmetrization trick for $\sup_{h\in {\mathcal H}} |E(h)-...
- 3,968
2
votes
1
answer
302
views
Order of independent random variables
Let $(p_\pi)_{\pi\in S_3}$ be given nonnegative reals such that $\sum_{\pi \in S_3} p_\pi = 1$. What are necessary and sufficient conditions for there to exist independent random variables $X_1,X_2,...
- 8,071
6
votes
1
answer
2k
views
Minimizing KL divergence: the asymmetry, when will the solution be the same?
The KL divergence between two distribution $p$ and $q$ is defined as
$$
D( q \| p)\int q(x)\log \frac{q(x)}{p(x)} dx
$$
and is known to be asymmetry: $D(q\|p)\neq D(p\|q)$.
If we fix $p$ and try to ...
- 12.7k
18
votes
1
answer
1k
views
How fast can extreme eigenvalues of the average of random matrices converge to their expectation?
Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
- 8,071
25
votes
1
answer
4k
views
What kind of random matrices have rapidly decaying singular values?
I've been told that in machine learning it's common to compute the singular value decomposition of matrices in order to throw out all information in the matrix except that corresponding to, say, the $...
- 8,071
4
votes
0
answers
141
views
Is there an example that both Berry-Essen bound and DKW bound are attained?
The Berry-Essen bound stated that
$$\sup _{{x\in {\mathbb R}}}\left|\widehat{F_{n}(x)}-\Phi (x)\right|\leq C_{0}\cdot \psi _{0}$$
where $\psi _{0}(n)={\Big (}{\textstyle \sum \limits _{{i=1}}^{n}\...
- 8,071
4
votes
1
answer
421
views
Order statistic of Markov chain sample path and related probabilities
Consider a one dimensional sample path, denoted as $\{X(1), ..., X(t), ..., X(n)\}$, generated from a discrete time finite state (time homogeneous) Markov chain over states $\{1,...,m\}$, with ...
- 8,071
2
votes
1
answer
658
views
Extension of Talagrand contraction lemma (on empirical Rademacher complexity)
Is the following true?
Let $(x_1,...,x_N)$ be a set of points on the unit sphere $S^{d-1}$.
Let $\ell_x: [-1,1]\rightarrow [0,1]$ be a family of Lipschitz functions indexed by $x\in S^{d-1}$, with ...
- 517
1
vote
1
answer
86
views
Clarification on margin bound uniform w.r.t. the margin parameter
Theorem 4.5. in the book "Foundations of Machine Learning" by Mohri et al:
http://prlab.tudelft.nl/sites/default/files/Foundations_of_Machine_Learning.pdf
derives a generalization bound to hold ...
- 26
2
votes
1
answer
114
views
Does maximizing $D_u$ imply stochastic ordering?
Let $\mathscr P _0$ and $\mathscr P _1$ be two non-overlapping sets of probability distributions defined on $(\Omega,\mathcal{A})$. Consider the distance defined as $$D_u(P_0,P_1)=\int_\Omega \left(\...
- 8,071
6
votes
1
answer
2k
views
Rate of convergence of uniform order statistics to their expectations
This is a problem that I encountered in my research and have no clues to fully
resolve it. Basically, I need large (or moderate) deviation bounds on the
difference between an order statistic of ...
- 984
3
votes
2
answers
195
views
What is known about the PDFs for the $\ell^2$-norm of these multivariate distributions?
I'm looking for resources giving the PDFs for the $\ell^2$-norm of various spherically symmetric, continuous multivariate distributions.
For instance, the PDF for the $\ell^2$-norm of a multivariate ...
0
votes
1
answer
172
views
constrained optimization problem/proof
Im trying to maximize the probability of a particular outcome occurring subject to a constraint. In particular
$$\max \prod_{i \leq n} 1 - (1 - x_i)^{y_i} \;\;\; \text{ s.t. } \;\;\; i \in \mathbb{N}...
2
votes
1
answer
1k
views
Rademacher complexity of composition of functions
I am looking for a bound on the empirical Rademacher complexity of the following class:
$G=\left\{x \rightarrow \frac{h^T f(x)}{\|h\|_2 \cdot \|f(x)\|_2} : h\in R^d, f()=(f_1(),\ldots,f_d()), f_j \in ...
3
votes
1
answer
833
views
Sampling from a particular multivariate probability distribution
Given $3$ real variables $x_1, x_2, x_3 \equiv \bf{x}$, consider their probability density function (PDF)
\begin{equation}
P({\bf x}) = C \, p(x_1) \cdots p(x_3) \exp[f({\bf x})],
\end{equation}
where ...
CommunityBot
- 1
3
votes
0
answers
158
views
How are these two multi-armed bandit problems similar?
I am reading the multi-armed bandit survey by Bubeck and Bianchi. This question is for the lower bound section (2.3) of the survey.
Let us define Kullback-Leibler divergence $kl(p, q) = p \log \frac{p}...
- 8,071