All Questions
Tagged with pr.probability st.statistics
1,134 questions
5
votes
1
answer
373
views
assumptions on local rademacher complexities
A lot of the work on Local Rademacher complexities of Koltchinskii, and Bartlett for fast rates of convergence is based on Bousquet's version of Talagrand's inequality [1] (Theorem 2.11). However the ...
- 131
5
votes
1
answer
219
views
Do there exist (almost surely) $C^{\infty}$-smooth Gaussian random fields?
Let $d \ge 1$. Do there exist Gaussian random fields on $\mathbb R^d$ which are (almost surely) $C^{\infty}$-smooth, but which are not analytic?
If so, what are necessary and sufficient conditions ...
- 8,512
5
votes
2
answers
6k
views
Difference between Beta Process and Dirichlet process
I'm trying to understand the definition of a Beta process, as given in the paper:
www.ece.duke.edu/~lcarin/Paisley_BP-FA_ICML.pdf
The problem is that from the definition it follows that every ...
- 233
5
votes
1
answer
222
views
Switching oriented paths in a graph
Consider an oriented graph (e.g. a finite part of the standard grid with some random orientations).
Each minute the following operation takes place: we choose uniformly randomly an ordered pair $(A,B)...
- 5,063
5
votes
1
answer
164
views
Probability of ruin in the Sparre Andersen renewal risk model
In 1957, Erik Sparre Andersen proposed using a renewal risk model to describe the behavior of the insurers surplus
$$U(t)=u+ct-\sum\limits_{i=1}^{\Theta(t)}Z_i, \quad t \geqslant 0$$
where:
$u \...
5
votes
1
answer
567
views
Donsker's Theorem for triangular arrays
I should mention that I already posed this question on Math Stack Exchange, but didn't receive much feedback.
Assume we have a sequence of smooth i.i.d. random variables $(X_i)_{i=1}^{\infty}$. Given ...
- 233
5
votes
1
answer
2k
views
Earth mover/Wasserstein distance between a pdf and an empirical distribution
This question is inspired by this much older question:
Convergence of an empirical distribution w.r.t. the Hellinger distance
Let $P$ be a continuous probability distribution on a compact subset of $...
- 51
5
votes
1
answer
497
views
Size of KL-divergence neighbourhoods
I am new here. I was reading another
post
here and this got me wondering what can be said about the size of the following kl divergence neighborhoods.
Consider these two kl-divergence neighbourhood ...
- 59
5
votes
2
answers
2k
views
Process for a Gamma distribution with non integer shape parameter
I am sampling the distribution of lifetimes of computers participating in massive volunteer computing initiatives (BOINC projects). While a phenomenological Weibull distribution makes a good ...
- 437
5
votes
1
answer
350
views
Bounding the sensitivity of a posterior mean to changes in a single data point
There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...
- 1,068
5
votes
0
answers
711
views
Concentration inequality for max component of a multivariate Gaussian in the general case
I am looking to bound the variance of the maximum component of a vector distributed multivariate Gaussian in the general case where the Gaussian distribution has arbitrary mean and full covariance ...
- 283
5
votes
0
answers
485
views
Hierarchical Random Walk (also known as Hierarchical Hidden Markov Model)
Let us consider the following hierarchical (recursive) random walk model, which is also known as the hierarchical hidden Markov model in computer science (https://en.wikipedia.org/wiki/...
- 1,127
5
votes
0
answers
136
views
What's the variance in the Six Degrees model?
Recall the six degrees of Kevin Bacon game. You can even play the game at The Oracle of Bacon, and their search works via Breadth First Search.
I interpret the punchline as saying that if I start ...
- 30.3k
5
votes
0
answers
327
views
Inverse moment of the number of inversions of a permutation
Let $\pi$ be a permutation of $\{1,2,...,n\}$. A pair of elements ($\pi_i$,$\pi_j$) is called an inversion if $i$ $>$ $j$ and $\pi_i$ $<$ $\pi_j$. The total number of inversions in $\pi$ is ...
- 51
5
votes
0
answers
154
views
Positive estimator
Suppose that one knows how to generate (independent) random samples $X_1, X_2, \ldots$ distributed as the random varable $X$ with $\mathbb{E}[X]=\mu \in \mathbb{R}$. It is then easy to construct an ...
- 2,133
5
votes
0
answers
1k
views
Multidimensional Berry–Esseen for probability density functions
This is a follow up to this recent question: Berry Esseen type result for probability density functions
There exists a multidimensional version of the usual Berry–Esseen theorem (for cumulative ...
5
votes
0
answers
506
views
Missing mass estimate
Let $S$ be a finite set with probability distribution $P$. Define the random variable $m_i$ to be the "missing mass" after seeing $i$ iid samples from $S$ under $P$. That is, $m_i$ is the total mass ...
- 6,541
5
votes
1
answer
363
views
Inverse marginal property of a collection of $\sigma$-algebras
In my paper "On the inverse best approximation property of systems of subspaces of a Hilbert space"
I introduced the Inverse marginal property (IMP) for a collection of $\sigma$-algebras.
Let $(\...
- 935
4
votes
5
answers
2k
views
Martingales and Betting Strategies
Does anyone know of a good introduction to the theory of martingales and betting strategies from the point of view of statistics and/or probability theory? I'm looking for something basic, with lots ...
4
votes
3
answers
1k
views
Probability theory and measuring the true strength of chessplayers
If you wanted to measure the strength of, say, a chess player, the best way would involve knowing the true value of each position: then you could compute the frequency $W$ with which the player finds ...
- 17.6k
4
votes
2
answers
13k
views
how to find derivative of a stochastic process?
Consider the following equation for $X(t)$:
$$X(t)=e^{-bt}X(0)+\sigma\int_{0}^{b}e^{-b(t-s)}dW(t) \, ,$$
where $0 < b, \sigma\in\mathbb{R} $, $X(0)$ is the initial distribution of $X(t)$, ...
- 281
4
votes
2
answers
2k
views
Elo Rating System Help with the Maths around number of matches
I'm creating a system that will allow people to rate images.
My idea is to use an Elo Rating system (http://en.wikipedia.org/wiki/Elo_rating_system) for each image and then use crowdsourcing to have ...
- 151
4
votes
4
answers
4k
views
"Square root" of Beta(a,b) distribution
Under what conditions on a and b is there a distribution $f_{a,b}$ such that the product $XY$ of two independent realizations $X$ and $Y$ from $f_{a,b}$ has a Beta(a,b) distribution?
A standard ...
- 143
4
votes
5
answers
7k
views
Proof of Bellman optimality equation for finite Markov Decision Processes
This question has already been posed in Cross Validated without receiving a correct formal answer, so I reformulate it here to gain attention of mathematicians. I am referring to chapter 3 of Sutton ...
- 171
4
votes
2
answers
415
views
Effect of perturbing the atoms of a measure on the Wasserstein distance
Let $(X,d)$ be a metric space, $x_1,\ldots,x_N\in X$ and $x_1',\ldots,x_N'\in X$ be atoms, and $G=\sum_{i=1}^Np_i\delta_{x_i}$, $G'=\sum_{i=1}^Np_i'\delta_{x_i}$, and $G''=\sum_{i=1}^Np_i'\delta_{x_i'}...
- 710
4
votes
2
answers
1k
views
Reducing system of equations involving Erf, Error Function
I have a system of equations:
$$1/2 + {\rm Erf}(x) - {\rm Erf}(\frac{x+y}{2})=0$$
$$-1/2 + {\rm Erf}(y) - {\rm Erf}(\frac{x+y}{2})=0,$$
Where $x \le y$ and ${\rm Erf}$ is the Error Function.
By ...
- 123
4
votes
3
answers
467
views
Law of large numbers for stochastically chosen samples
Let $X_t$ be a sequence of i.i.d. random variables with mean $\mu$. Then the law of large numbers states that
$$\lim_{T \to \infty} \frac1T \sum_{t=1}^T X_t = \mu \quad a.s.$$
Now suppose that (in a ...
- 103
4
votes
2
answers
653
views
Reading Material on Couplings
Does anybody have suggestions on what to read to learn more about couplings pertaining to statistics?
I'm working on a research project on Poisson approximations and am looking to perform a coupling ...
4
votes
3
answers
439
views
Probability estimates for "beans & boxes"
From a discussion with some friends, this apparently easy problem has come out; I decided to post it here, because I believe that the answer is non-trivial and the maths beneath interesting. Partial ...
- 418
4
votes
2
answers
349
views
Does the average of correlated Gaussian random variables with mean zero and different variances converge in probability to their mean?
Let $X_i\sim N(0,\sigma_i^2)$ and $\operatorname{Corr}(X_i,X_j)>0$. Is it possible to show that $$\frac{1}{N} \sum_{i=1}^N X_i \overset{p}\rightarrow E[X_i]=0.$$ Do you have a reference to a law of ...
4
votes
3
answers
1k
views
Incremental entropy computation
After a quick internet search I found no method for incremental entropy computation.
Question 1
Let $\{x_i\}_{i=1}^n$ and $\{x_i\}_{i=1+n}^{n+m}$ be two samples and let $S_i^j:=\sum_{k=i}^j x_k$. ...
- 161
4
votes
2
answers
2k
views
Why MLEs are asymptotically efficient whereas method of moment estimators are not?
Under appropriate regularity conditions it is well-known that Maximum Likelihood Estimation (MLE) produces asymptotically efficient estimators in the sense that their asymptotic covariance is given by ...
4
votes
1
answer
489
views
CLT convergence rate for sum of uniforms (in TV distance)
Suppose $X_1, \cdots, X_n \sim_{\mathrm{iid}} U([-1,1])$, where $U([-1, 1])$ denotes the continuous uniform distribution over the interval $[-1, 1]$ (so $E[X_i] = 0$ and $\text{Var}[X_i]= 1/3$). Let $...
- 43
4
votes
1
answer
344
views
Degenerate Gaussian Integral
I have an integral over a subspace of $\mathbb{R}^n \times \mathbb{R}^n$ with an integrand of the form
$$\exp\left(-\frac{1}{2}\left[||u^2|| + \langle u, v \rangle + ||v||^2\right]\right)$$
The ...
- 435
4
votes
1
answer
203
views
Can samples be compressed?
The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is:
$\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...
- 22.1k
4
votes
1
answer
704
views
Central limit theorem for $P(x)\sim 1/x^3$ distribution
I have a random variable $x \in (0,\infty)$ with distribution $P(x)$ falling off slowly $P(x) \sim 1/x^3$ for large $x$. So the expectation value $\bar{x}$ is finite but the second moment $\bar{x^2}$ ...
- 1,150
4
votes
3
answers
3k
views
What is the name for a non-normalized distribution?
For some analysis work with probability distributions, I remember a common trick being to drop the "integrate to 1" requirement, so the set becomes closed under addition and is more convenient to work ...
4
votes
2
answers
1k
views
Conditional Probabilities - The Mad Kings' Draft
The Problem of the Mad King's Draft:
Suppose there is country which is ruled by a king who can be either 'mad' or 'normal.' The king rules a a large country with a continuum of citizens who have ...
4
votes
2
answers
1k
views
expected values over binomial distributions
In some works of economics/risk analysis etc., I have seen situations where people take the expected value of a function (such as a utility function/cost function) over a binomial distribution:
$$F(n)...
- 7,320
4
votes
1
answer
1k
views
Bound for a conditional expectation
Let $a_i, i=1, \ldots, n$ be real numbers. Let $\epsilon_i, \, i=1, \ldots, n$ be a random variables that take values $\pm 1$ with equal probability and $r_i, i=1, \ldots, n$ be random variables ...
- 343
4
votes
1
answer
423
views
Concentration inequalities on the supremum of average after time $n$
Let $R_1, R_2, \cdots$ be i.i.d. Rademacher random variables (taking values $-1,+1$ w.p. $0.5$). At time $k$, their average is $\frac{1}{k}\sum_{i=1}^k R_i$. One can imagine after $k\geq n$ for some $...
- 183
4
votes
1
answer
1k
views
Does variants of Bernstein and Freedman concentration inequalities exist with NO uniform bound on the range of RV or martingale differences
A classic formulation of the Bernstein inequality (from Wikipedia) is as follow:
Let $X_1, \ldots, X_n$ be independent zero-mean random variables. Suppose that $|X_i|\leq M$ almost surely, for all $i$...
- 53
4
votes
1
answer
161
views
Hellinger integral for the Student/Cauchy family
Let $p$ and $q$ be probability densities on $\mathbb R$, with respect to the Lebesgue measure $dx$. The corresponding Hellinger integral is
$H(p,q):=\int_{\mathbb R}\sqrt{pq}\,dx$.
Let now $p$ be ...
- 128k
4
votes
1
answer
821
views
Does bounding moments make distributions close in total variation distance?
Let $W\sim\mathcal{N}(0,\sigma^2)$ be a "reference" Gaussian random variable.
Suppose I have a set of distributions, $\mathcal{W}$, where $W_a\in\mathcal{W}$ if it satisfies the following criteria:
...
- 907
4
votes
2
answers
327
views
Estimate on gaussian distribution
Let X be an $\mathbb R^d$-valued random variable with distribution $N_d(0,\Sigma)$. I'm looking for a function $f$ such that
$$P(|X_1|\leq M, |X_2|\leq M,\dots, |X_d|\leq M)\geq f(M),$$
and such that $...
- 293
4
votes
1
answer
193
views
Variance of central limit distribution for $P(x) \sim 1/x^{1+\alpha}$ for finite but large $N$?
Is it known what the next-to-leading order term is in the variance of the central limit distribution for the average of $N$ variables each of which is distributed according to $P(x) \sim 1/x^{1+\alpha}...
- 1,150
4
votes
1
answer
5k
views
Asymptotic behavior of max of chi-squared distribution
Suppose $X_{\max}$ is the maximum in a sequence $X_1,X_2,\ldots,X_n$ where each $X_i\sim\chi^2_k$ is an i.i.d. chi-squared random variable with $k$ degrees of freedom.
Since chi squared distribution ...
- 907
4
votes
1
answer
577
views
Prove an inequality related to moments
I am reading a paper and stuck with an inequality used in that paper.
$\varepsilon^n=(\varepsilon_1^n, \varepsilon_2^n,\ldots,\varepsilon_n^n)^T$ is a vector of i.i.d. random variables with mean 0 ...
- 43
4
votes
3
answers
2k
views
Correspondence between Viterbi algorithm and Smith-Waterman
Viterbi is an algorithm for finding the maximum likelihood assignment to the hidden variables of an HMM, given the observed variables (we know the transition and emission probabilities of the HMM). ...
- 233
4
votes
1
answer
234
views
Maximum entropy probability distribution with fixed interval and variance?
What is the maximum entropy probability distribution if the support is a fixed interval (e.g. $[-1,1]$) with an already known variance?
If we know the support is a fixed interval, then the maximum ...
- 43