3
$\begingroup$

Suppose we have a probability density function $f(x)$ with a finite support $[a,b]$. If we take the probability convolution of $\lambda f $ with $(1-\lambda)f,0 <\lambda<1$ recursively for many times, does the resulting distribution converges to the mean of $f(x)$?

To be more specific: suppose $f_1(x)$ is probability distribution resulting form the convolution of $\lambda f $ with $(1-\lambda)f$, the second convolution would be $\lambda f_1 $ with $(1-\lambda)f_1$ and so forth...

I tried to run some simulation with a discrete probability distribution with three outcomes. It seems it would converge as I increase the converge times from 1 to 3 times . But trying $4$ times crashes my laptop...


Here is the definition of convolution operation: https://en.wikipedia.org/wiki/Convolution_of_probability_distributions

$\endgroup$
6
  • $\begingroup$ (i) What do you mean by "a probability distribution $f(x)$"? A probability distribution is a measure. (ii) What do you mean by $\lambda f$? If $f$ is indeed a probability distribution and $\lambda f$ is the usual product, then the convolutions will converge to the zero measure (even in total variation). (iii) How can "the resulting distribution converge[] to the mean"? Again, a probability distribution is a measure, whereas the mean is number. $\endgroup$ Feb 10, 2022 at 16:34
  • 1
    $\begingroup$ @IosifPinelis (iii) I think by "converging to the mean", she means that if we repeat the convolution operation, then the distribution will finally become a Dirac-delta distribution at the mean. (ii) Convolution means this" en.wikipedia.org/wiki/Convolution_of_probability_distributions. (i) $f$ usually means the probability density function of a distribution, I think. $\endgroup$
    – dodo
    Feb 10, 2022 at 17:08
  • 1
    $\begingroup$ If $f$ is a probability density function then the convolution $f*f$ is also a probability density function, so if $0<\lambda<1$ is a scalar, then then $\lambda^2 f*f$ and $(1-\lambda)^2 f*f$ and $\lambda(1-\lambda)f*f$ are functions whose integrals are less than $1.$ and $f*f$ is the density function of a distribution whose variance is twice that of the distribution whose density is $f.$ So this is moving away from the delta function, not toward it. $\endgroup$ Feb 10, 2022 at 17:24
  • $\begingroup$ I wonder whether what you want is a weighted average of i.i.d. random variables, rather than assigning weights to the density functions? $\endgroup$ Feb 10, 2022 at 17:54
  • 1
    $\begingroup$ @MichaelHardy it is the weighted average of i.i.d random variables. The answer provided below is a more accurate description of my problem and I agree with his answer. $\endgroup$
    – Emma
    Feb 14, 2022 at 9:20

1 Answer 1

13
$\begingroup$

$\newcommand{\si}{\sigma}\newcommand{\ep}{\varepsilon}\newcommand\E{\operatorname{\mathsf{E}}}\newcommand\Var{\operatorname{\mathsf{Var}}}\newcommand{\de}{\delta}$I think a meaning can be attached to your post as follows: You appear to confuse three related but quite different notions: (i) a random variable (r.v.), (ii) its distribution, and (iii) its pdf. (Unfortunately, many people do so.)

So, my guess at what you were trying to say is as follows: Let $X$ be a r.v. with values in $[a,b]$. Let $\mu:=\E X$ and $\si^2:=\Var X$. Let $X_{\cdot,\cdots}$ with various indices $_{\cdot,\cdots}$ denote independent copies of $X$. Let $t:=\lambda\in(0,1)$.

At the first step, we take any $X_1$ and $X_2$ (which are, according to the above convention, two independent copies of $X$). We multiply the r.v.'s $X_1$ and $X_2$ (not their distributions or pdf's) by $t$ and $1-t$, respectively, to get the independent r.v.'s $tX_1$ and $(1-t)X_2$. The latter r.v.'s are added, to get the r.v. \begin{equation} S_1:=tX_1+(1-t)X_2, \end{equation} whose distribution is the convolution of the distributions of the r.v.'s $tX_1$ and $(1-t)X_2$.

At the second step, take any two independent copies of $S_1$, multiply them by $t$ and $1-t$, respectively, and add the latter two r.v.'s, to get a r.v. equal in distribution to the r.v. \begin{equation} S_2:=t(tX_1+(1-t)X_2)+(1-t)(tX_3+(1-t)X_4) \\ =t^2 X_1+t(1-t)X_2+(1-t)tX_3+(1-t)^2X_4; \end{equation} (in accordance with the indexing convention, the r.v.'s $X_1,X_2,X_3,X_4$ are independent copies of $X$).

Continuing so, after the $n$th step we get a r.v. \begin{equation} S_n:=\sum_{k=0}^n t^k(1-t)^{n-k}\sum_{i=1}^{\binom nk} X_{n,k,i} \end{equation} (in accordance with the indexing convention, the $X_{n,k,i}$'s are independent copies of $X$). This expression for $S_n$ can be checked by induction on $n$. (One can also use/check the following re-indexed expression for $S_n$: \begin{equation} \sum_{(\de_1,\dots,\de_n)\in\{0,1\}^n}X_{\de_1,\dots,\de_n} \prod_{j=1}^n(t^{\de_j}(1-t)^{1-\de_j}); \end{equation} imagine the corresponding binary tree.)

Now we have \begin{equation} \begin{aligned} \E S_n&=\sum_{k=0}^n t^k(1-t)^{n-k}\sum_{i=1}^{\binom nk} \E X_{n,k,i} \\ &=\sum_{k=0}^n t^k(1-t)^{n-k}\binom nk \mu =\mu \end{aligned} \end{equation} and \begin{equation} \begin{aligned} \Var S_n&=\sum_{k=0}^n (t^k(1-t)^{n-k})^2 \sum_{i=1}^{\binom nk} \Var X_{n,k,i} \\ &=\sum_{k=0}^n (t^2)^k((1-t)^2)^{n-k}\binom nk \si^2 \\ &=q^n\si^2\to0 \end{aligned} \end{equation}
(as $n\to\infty$), where $q:=t^2+(1-t)^2\in(0,1)$.

So, by Chebyshev's inequality, for any real $\ep>0$, \begin{equation} P(|S_n-\mu|>\ep)\le\frac{\Var S_n}{\ep^2}=\frac{q^n\si^2}{\ep^2}\to0. \end{equation}

Thus, indeed $S_n\to\mu$ in probability.

Moreover, since \begin{equation} \sum_{n=1}^\infty P(|S_n-\mu|>\ep)\le\sum_{n=1}^\infty\frac{q^n\si^2}{\ep^2}<\infty \end{equation} for any real $\ep>0$, it follows by the Borel–Cantelli lemma that $S_n\to\mu$ almost surely.


Below is the image of a Mathematica notebook illustrating the geometrically fast convergence of $S_n$ to the mean $\mu$. It takes Mathematica about 0.01 sec to simulate values of $S_1,\dots,S_{10}$ and about 9 sec to simulate values of $S_1,\dots,S_{20}$ (in the second case, we deal with a binary tree with $2^{20}\approx10^6$ leaves).

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.