3
$\begingroup$

Suppose we have a probability density function $f(x)$ with a finite support $[a,b]$. If we take the probability convolution of $\lambda f $ with $(1-\lambda)f,0 <\lambda<1$ recursively for many times, does the resulting distribution converges to the mean of $f(x)$?

To be more specific: suppose $f_1(x)$ is probability distribution resulting form the convolution of $\lambda f $ with $(1-\lambda)f$, the second convolution would be $\lambda f_1 $ with $(1-\lambda)f_1$ and so forth...

I tried to run some simulation with a discrete probability distribution with three outcomes. It seems it would converge as I increase the converge times from 1 to 3 times . But trying $4$ times crashes my laptop...


Here is the definition of convolution operation: https://en.wikipedia.org/wiki/Convolution_of_probability_distributions

$\endgroup$
6
  • $\begingroup$ (i) What do you mean by "a probability distribution $f(x)$"? A probability distribution is a measure. (ii) What do you mean by $\lambda f$? If $f$ is indeed a probability distribution and $\lambda f$ is the usual product, then the convolutions will converge to the zero measure (even in total variation). (iii) How can "the resulting distribution converge[] to the mean"? Again, a probability distribution is a measure, whereas the mean is number. $\endgroup$ Commented Feb 10, 2022 at 16:34
  • 1
    $\begingroup$ @IosifPinelis (iii) I think by "converging to the mean", she means that if we repeat the convolution operation, then the distribution will finally become a Dirac-delta distribution at the mean. (ii) Convolution means this" en.wikipedia.org/wiki/Convolution_of_probability_distributions. (i) $f$ usually means the probability density function of a distribution, I think. $\endgroup$
    – dodo
    Commented Feb 10, 2022 at 17:08
  • 1
    $\begingroup$ If $f$ is a probability density function then the convolution $f*f$ is also a probability density function, so if $0<\lambda<1$ is a scalar, then then $\lambda^2 f*f$ and $(1-\lambda)^2 f*f$ and $\lambda(1-\lambda)f*f$ are functions whose integrals are less than $1.$ and $f*f$ is the density function of a distribution whose variance is twice that of the distribution whose density is $f.$ So this is moving away from the delta function, not toward it. $\endgroup$ Commented Feb 10, 2022 at 17:24
  • $\begingroup$ I wonder whether what you want is a weighted average of i.i.d. random variables, rather than assigning weights to the density functions? $\endgroup$ Commented Feb 10, 2022 at 17:54
  • 1
    $\begingroup$ @MichaelHardy it is the weighted average of i.i.d random variables. The answer provided below is a more accurate description of my problem and I agree with his answer. $\endgroup$
    – Emma
    Commented Feb 14, 2022 at 9:20

1 Answer 1

13
$\begingroup$

$\newcommand{\si}{\sigma}\newcommand{\ep}{\varepsilon}\newcommand\E{\operatorname{\mathsf{E}}}\newcommand\Var{\operatorname{\mathsf{Var}}}\newcommand{\de}{\delta}$I think a meaning can be attached to your post as follows: You appear to confuse three related but quite different notions: (i) a random variable (r.v.), (ii) its distribution, and (iii) its pdf. (Unfortunately, many people do so.)

So, my guess at what you were trying to say is as follows: Let $X$ be a r.v. with values in $[a,b]$. Let $\mu:=\E X$ and $\si^2:=\Var X$. Let $X_{\cdot,\cdots}$ with various indices $_{\cdot,\cdots}$ denote independent copies of $X$. Let $t:=\lambda\in(0,1)$.

At the first step, we take any $X_1$ and $X_2$ (which are, according to the above convention, two independent copies of $X$). We multiply the r.v.'s $X_1$ and $X_2$ (not their distributions or pdf's) by $t$ and $1-t$, respectively, to get the independent r.v.'s $tX_1$ and $(1-t)X_2$. The latter r.v.'s are added, to get the r.v. \begin{equation} S_1:=tX_1+(1-t)X_2, \end{equation} whose distribution is the convolution of the distributions of the r.v.'s $tX_1$ and $(1-t)X_2$.

At the second step, take any two independent copies of $S_1$, multiply them by $t$ and $1-t$, respectively, and add the latter two r.v.'s, to get a r.v. equal in distribution to the r.v. \begin{equation} S_2:=t(tX_1+(1-t)X_2)+(1-t)(tX_3+(1-t)X_4) \\ =t^2 X_1+t(1-t)X_2+(1-t)tX_3+(1-t)^2X_4; \end{equation} (in accordance with the indexing convention, the r.v.'s $X_1,X_2,X_3,X_4$ are independent copies of $X$).

Continuing so, after the $n$th step we get a r.v. \begin{equation} S_n:=\sum_{k=0}^n t^k(1-t)^{n-k}\sum_{i=1}^{\binom nk} X_{n,k,i} \end{equation} (in accordance with the indexing convention, the $X_{n,k,i}$'s are independent copies of $X$). This expression for $S_n$ can be checked by induction on $n$. (One can also use/check the following re-indexed expression for $S_n$: \begin{equation} \sum_{(\de_1,\dots,\de_n)\in\{0,1\}^n}X_{\de_1,\dots,\de_n} \prod_{j=1}^n(t^{\de_j}(1-t)^{1-\de_j}); \end{equation} imagine the corresponding binary tree.)

Now we have \begin{equation} \begin{aligned} \E S_n&=\sum_{k=0}^n t^k(1-t)^{n-k}\sum_{i=1}^{\binom nk} \E X_{n,k,i} \\ &=\sum_{k=0}^n t^k(1-t)^{n-k}\binom nk \mu =\mu \end{aligned} \end{equation} and \begin{equation} \begin{aligned} \Var S_n&=\sum_{k=0}^n (t^k(1-t)^{n-k})^2 \sum_{i=1}^{\binom nk} \Var X_{n,k,i} \\ &=\sum_{k=0}^n (t^2)^k((1-t)^2)^{n-k}\binom nk \si^2 \\ &=q^n\si^2\to0 \end{aligned} \end{equation}
(as $n\to\infty$), where $q:=t^2+(1-t)^2\in(0,1)$.

So, by Chebyshev's inequality, for any real $\ep>0$, \begin{equation} P(|S_n-\mu|>\ep)\le\frac{\Var S_n}{\ep^2}=\frac{q^n\si^2}{\ep^2}\to0. \end{equation}

Thus, indeed $S_n\to\mu$ in probability.

Moreover, since \begin{equation} \sum_{n=1}^\infty P(|S_n-\mu|>\ep)\le\sum_{n=1}^\infty\frac{q^n\si^2}{\ep^2}<\infty \end{equation} for any real $\ep>0$, it follows by the Borel–Cantelli lemma that $S_n\to\mu$ almost surely.


Below is the image of a Mathematica notebook illustrating the geometrically fast convergence of $S_n$ to the mean $\mu$. It takes Mathematica about 0.01 sec to simulate values of $S_1,\dots,S_{10}$ and about 9 sec to simulate values of $S_1,\dots,S_{20}$ (in the second case, we deal with a binary tree with $2^{20}\approx10^6$ leaves).

enter image description here

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .