All Questions
Tagged with pr.probability st.statistics
1,135 questions
0
votes
2
answers
627
views
Generalized expression for balls and bins problem
$n$ number of balls are thrown randomly to $m$ number of bins, standing in a row. The balls are labeled as $1,2,3,....n$ and bins are also labeled as $1,2,3,...,m$. The probability of $i_{th}$ ball ...
- 25
4
votes
1
answer
229
views
How are the real-space RG transformations defined?
I'm reading Shang-keng Ma's book Modern theory of critical phenomena, and I'm a bit confused as to how the real-space RG transformations are defined. Ma basically says that these transformations are ...
- 161
6
votes
2
answers
2k
views
Absolute moments of symmetrical distributions
Suppose $F~$ is a probability distribution symmetrical about 0, for which all moments exist. Let $\mu_i~$be the $i$-th moment (of course $\mu_i=0$ if $i~$ is odd).
We know there are some conditions ...
- 37.7k
10
votes
2
answers
591
views
"Fractional sampling" from a probability distribution
My question concerns an operation on probability distributions which has arisen in some applied research. It is well-defined mathematically (at least in a limited context), but I don't know how to ...
- 8,501
1
vote
2
answers
388
views
Average Multivariate Gaussian
Suppose we have a (possibly infinite) collection k-variate gaussian distributions $\{(\mathcal{N}(\mu_{\lambda}, \Sigma_{\lambda}))\}$ ($\lambda$ is just a label), and for each distribution $\mu \in [-...
- 11
5
votes
1
answer
497
views
Size of KL-divergence neighbourhoods
I am new here. I was reading another
post
here and this got me wondering what can be said about the size of the following kl divergence neighborhoods.
Consider these two kl-divergence neighbourhood ...
- 59
4
votes
3
answers
2k
views
Correspondence between Viterbi algorithm and Smith-Waterman
Viterbi is an algorithm for finding the maximum likelihood assignment to the hidden variables of an HMM, given the observed variables (we know the transition and emission probabilities of the HMM). ...
- 233
3
votes
0
answers
158
views
How are these two multi-armed bandit problems similar?
I am reading the multi-armed bandit survey by Bubeck and Bianchi. This question is for the lower bound section (2.3) of the survey.
Let us define Kullback-Leibler divergence $kl(p, q) = p \log \frac{p}...
- 145
2
votes
0
answers
687
views
Placing Bounds on Correlation/Covariance Through Correlation with an Intermediate Variable
I am trying to make the most of computations that have already been performed in previous steps of an algorithm. Throughout this problem statement I am only mentioning correlation, but I think it is ...
- 21
0
votes
0
answers
322
views
Comparison of Parameter estimation using maximum likelihood and Maximum entropy
I am not sure if the question is appropriate but I want to try my luck. One can estimate a parameter using maximum likelihood and we know it is optimal. On the other hand there are methods which uses ...
- 495
3
votes
0
answers
98
views
Asymptotic results on statistical graph models
This post is partly inspired by this post.
Reference request: results on the asymptotic distribution of singular values related to a random orthogonal matrix
While it is well-known that two basic ...
- 8,071
2
votes
1
answer
150
views
Probability of collision of some family of hash functions
Given $x$ and $y$ in $\mathbb{R}$, and let $\mathcal{H} = \{ h \mid \mathbb{R} \to \mathbb{N} \}$ be a family of hash functions where $ h(x) = \left\lfloor x + \sum^C_{i=1} U_i \right\rfloor$ for some ...
- 321
1
vote
0
answers
533
views
Finding an error estimation for the De Moivre–Laplace theorem with Stirling's formula
Context for my question: For one part of my thesis I try to find an upper bound for the error in the normal approximation of the binomial distribution following the standard proof of the De Moivre–...
- 497
2
votes
1
answer
645
views
Estimating the variance of error in empirical approximation to a distribution
Let $X_1,X_2,\ldots,X_n$ be i.i.d. random variables in $\mathbb{R}$ with common cumulative distribution function (CDF) $F(x)$. The empirical approximation to $F(x)$ is defined as follows:
$$\hat{F}...
- 917
4
votes
1
answer
704
views
Central limit theorem for $P(x)\sim 1/x^3$ distribution
I have a random variable $x \in (0,\infty)$ with distribution $P(x)$ falling off slowly $P(x) \sim 1/x^3$ for large $x$. So the expectation value $\bar{x}$ is finite but the second moment $\bar{x^2}$ ...
- 1,150
1
vote
0
answers
55
views
Central Limit Like theorem for the distribution of F-statistics on all possible partitions?
I'd be happy for simply a reference or even search terms as I feel like this has to be known*.
Suppose we have a known probability distribution $X$ and a fixed integer $n$. I am interested in the ...
- 51
1
vote
1
answer
86
views
Clarification on margin bound uniform w.r.t. the margin parameter
Theorem 4.5. in the book "Foundations of Machine Learning" by Mohri et al:
http://prlab.tudelft.nl/sites/default/files/Foundations_of_Machine_Learning.pdf
derives a generalization bound to hold ...
- 517
2
votes
1
answer
3k
views
Empirical estimator fot the total variation distance on a finite space
I have two probability measures $p$ and $p'$ on a finite set $X$ which I do not know precisely, but which I can sample from. I would like to estimate their total variation (omitting multiplier $2$):
$$...
- 1,655
0
votes
1
answer
163
views
$\int_0^t f(s)\,dB_s$ normally distributed, mean and variance
Suppose that $f(t)$ is a (non-random) continuous function on $[0, \infty)$. Let$$Z_t = \int_0^t f(s)\,dB_s.$$
How do I see that $Z_t$ is normally distributed?
What is the mean and variance?
I need ...
- 43
2
votes
2
answers
268
views
Distribution of a random walk on a directed line
Is there a closed formula for the distribution of $x_t$ in the following random process, describing a random walk on a directed line?
$x_0 = n$
$x_t$ is a uniformly random integer between 1 and $x_{...
3
votes
1
answer
272
views
Learn a distribution from distributions on samples
There's many good ways to learn a distribution $p_X$ of an r.v. $X$ over $k$ symbols given many i.i.d. samples $X_1,\ldots, X_n$. The simplest is to use the sample relative frequencies $\hat{f}_X$ as ...
- 131
2
votes
1
answer
250
views
An Inequality Regarding the Squared Conditional Variance
Given absolutely continuous random variables $(X, Y)$ with joint distribution $P_{XY}$, we construct $Z:=\sqrt{\gamma} Y+N_\mathsf{G}$ where $N_\mathsf{G}\sim N(0, 1)$ and is independent of $(X,Y)$. ...
- 1,109
4
votes
1
answer
151
views
Negative population variable importance
I asked this question on stats.stackexchange and even elsewhere, but it never received an answer.
I just state the probabilistic problem here. It is about the optimality of the conditional ...
- 2,560
1
vote
1
answer
514
views
Showing non-attainment of supremum
This is just an extension of my previous question Tightness of probabilty distributions
Let $\mathcal{P}(\mathbb{N})$ be the set of all PMF's on $\mathbb{N}=\{1,2,\dots \}$. Let $E$ be a convex ...
- 779
1
vote
1
answer
1k
views
Null hypothesis test for independent but not identically distributed samples
I'm trying to figure out the best statistical test to use for an edge case I've run into: trying to figure out the likelihood of the null hypothesis for a set of samples that each (potentially) come ...
- 111
7
votes
3
answers
995
views
Kolmogorov probability axioms without non-negativity condition
What is a minimal consistent modification of probability axioms to include negative values?
Is it enough to use a minimal modification of axioms obtained by
formal exclusion of non-negativity ...
- 211
4
votes
2
answers
1k
views
Reducing system of equations involving Erf, Error Function
I have a system of equations:
$$1/2 + {\rm Erf}(x) - {\rm Erf}(\frac{x+y}{2})=0$$
$$-1/2 + {\rm Erf}(y) - {\rm Erf}(\frac{x+y}{2})=0,$$
Where $x \le y$ and ${\rm Erf}$ is the Error Function.
By ...
- 123
1
vote
1
answer
2k
views
Sum of covariance matrix of products of dependent variables
Consider the sequences of random variables $\{X_i\}_{i=1}^n$ and $\{Y_i\}_{i=1}^n$, as well as the corresponding sequence of products, $\{X_i Y_i\}_{i=1}^n$. All $X_i$ share the same mean value, $\...
- 965
1
vote
1
answer
147
views
Proving that an integral related to order statistics is increasing in a certain parameter
Let $f$ and $F$ denote, respectively, the pdf and cdf of a probability distribution on $\mathbb R$. Take any natural $n\ge3$ and any real $a$ and $c$ such that $a\le c$.
Does it always follow that
$$...
- 13
9
votes
2
answers
560
views
Integrating a simple exponential over the space of matrices that define a metric
I want to interpret an $n\times n$ matrix $D$ as a set of pairwise distances, and assume that $D$ obeys metric properties. Namely, $D_{ii} = 0$, $D_{ij} \geq 0$, $D_{ij} = D_{ji}$ and $D_{ij} \leq D_{...
- 147
1
vote
0
answers
701
views
limit distribution of multinomial distribution with increasing categories
If $\bf{X} \sim \text{multi}(n,p)$ with $k$ categories, we know
$$ \sqrt{n}\left( \frac{\bf{X}}{n} - \bf{p} \right) \rightarrow^D N(0,\Sigma),$$
where $\bf{X}=(X_1,\ldots,X_k)^T$ and $p=(p_1,\ldots,...
4
votes
0
answers
147
views
The asymptotic behavior of the ratio between the largest two of $n$ i.i.d. chi-square random variables
My question is about the asymptotic behavior of the ratio between the largest and second largest values of $n$ independent chi-square random variables.
Let $X_1, \ldots, X_n$ be $n$ independent and ...
- 1,127
5
votes
1
answer
472
views
Measures which exhibit the "uncorrelated implies independent" property
Let $X$ be a topological linear space, and let $X^*$ be its dual space. Suppose that $X$ is complete and Hausdorff, and $X^*$ separates points. Let $Y$ be another such space, and let $f : X \to Y$ be ...
- 8,512
2
votes
0
answers
366
views
Convergence rate of Pearson correlation matrix
I am interested in (rather sharp if not the finest) tail/concentration bounds for the Pearson correlation matrix: let $X_1,\ldots,X_N \sim \mathcal{N}(0,1)$ be correlated random variables; let $\rho(...
- 121
4
votes
2
answers
327
views
Estimate on gaussian distribution
Let X be an $\mathbb R^d$-valued random variable with distribution $N_d(0,\Sigma)$. I'm looking for a function $f$ such that
$$P(|X_1|\leq M, |X_2|\leq M,\dots, |X_d|\leq M)\geq f(M),$$
and such that $...
- 293
1
vote
1
answer
694
views
Rademacher complexity of a Lipschitz class: Are the boundedness constraints necessary?
Consider the following function class: $F={f:R^d\rightarrow [a,b], f(x)=\sigma(w^Tx)}$ where $\sigma(.)$ is Lipschitz, and $w\in R^d$ is a parameter vector. The problem I'm working on is a machine ...
- 517
1
vote
1
answer
712
views
Sequential sampling of Gaussian and von Mises-Fisher Random Variable
I don't find any article discussing this problem, so I dare to ask it.
Suppose we are dealing with a data $x_0 \in \mathbb{R}$ and a function $f:\mathbb{R} \to \mathbb{R}$. Say we repeatedly apply $f$...
21
votes
1
answer
3k
views
Intuitive Proof of Cramer's Decomposition Theorem
Cramer's decomposition theorem states that if $X$ and $Y$ are independent real random variables and $X+Y$ has normal distribution, then both $X$ and $Y$ are normally distributed. I've seen a few ...
- 4,952
4
votes
0
answers
867
views
For what sub-$\sigma$-algebra are these two measures equivalent?
In two statistics papers (linked inline below) I have come across two definitions of certain probability measures. I conjecture that for particular choices of the construction that they are ...
- 2,791
3
votes
1
answer
309
views
Bounds on the probability of k-of-n events in terms of bounds on single and pairwise probabilities
Let $A_1,\dotsc,A_n$ be events in a probability space, and let $N = \sum_{i=1}^n \mathbf{1}_{A_i}$ be the random number of events that occur. For a fixed value $k \in \{1,\dotsc,n\}$, what can be ...
- 325
0
votes
0
answers
444
views
How to decide a value of learning rate for Stochastic Gradient Descent?
I'd like to know how to decide a value of learning rate for Stochastic Gradient Descent (SGD), such as $\eta$ on the following parameter update iteration equation,
$w_{i+1} = w_i + -\eta \nabla E_n(...
0
votes
0
answers
124
views
Which sub-sequence selection rules preserve the iid property?
Let $\xi_1,\ldots,\xi_n$ be an iid sequence of random variables. If we take a sub-sequence $\xi_{i_1},\ldots,\xi_{i_k}$ with constant indices $1\leq i_1 <\ldots <i_k\leq n$, then the sub-...
- 251
1
vote
1
answer
202
views
An inequality for Maximal Correlation over a Markov Chain
Let a pair of random variables $(X,Y)$ be defined over finite alphabet $\mathcal{X}\times \mathcal{Y}$ with joint distribution $P_{XY}$. The maximal correlation $\rho(X;Y)$ between $X$ and $Y$ is ...
- 1,109
0
votes
1
answer
1k
views
Expected value with a kronecker product and Gaussian distributional assumption
What is the expected value, $ \mathbb{E}\left[ I \otimes \left( \operatorname{diag}(ZZ^T\mathbf{1}) - ZZ^T\right)\right]$ where $Z \sim N(0, \sigma^2I) $? The kronecker product is where the confusion ...
- 101
6
votes
1
answer
129
views
Choosing a sample based on where the density function is highest
Is there a name for the following process?
Say I have an absolutely continuous probability density function $f$ with compact support, and I take $k$ independent samples $x_1,\dots,x_k$ from $f$. ...
- 4,049
10
votes
1
answer
210
views
Distribution of the maximum of the norm of k-averages of n i.i.d. d-dimensional random vectors
Suppose $X_1, ... X_n$ are i.i.d. random vectors in $d$-dimensional space (i.e., $R^d$) with continuous centrally symmetric density function $f(\cdot)$ (i.e., symmetric with respect to the origin). ...
- 121
2
votes
0
answers
101
views
Best describing a stochastic process in terms of others
Intuitive Question
Suppose I'm given a set of $k$ time-series $\{X_t^1,\dots X_t^k\}$. Is there a way to determine how much of each series is dependent on the others.
Formal Question
More ...
- 5,405
1
vote
1
answer
22k
views
Covariance and standard deviation relationship
I would like to know if an increase in the covariance between two variables would imply that the standard deviation for one of the variables has increased?
This is assuming that the standard ...
- 21
4
votes
2
answers
463
views
Bounding the tail of an average using the the tail of individual members
Let $X_1,X_2,\ldots,X_n$ be an i.i.d. sequence of $n$ positive random variables with mean $E[X_1]=\mu_X<\infty$ and the second moment $E[X_1^2]=\infty$.
I am interested in upper-bounding $P\left(...
- 917
11
votes
2
answers
608
views
Covariance of INID order statistics [closed]
In the IID case, it is known that all order statistics are positively correlated.* Thus, we know that $$\text{Cov}(X_{(i)},X_{(j)}) \geq 0.$$ Is this known in the INID (independent, non-identically ...
- 111