11
$\begingroup$

Let $X=(X_1,\ldots,X_n)$ be an iid sequence of random variables, and let $\nu$ be a uniformly random integer in the range $1,\ldots,n$. Then $\xi_\nu$ is a random entry of $X$. Is it always true that after deleting such a random entry from $X$, the remaining sequence is still iid?

Due to the uniform randomness of the index $\nu$, it is tempting to answer that the leftover sequence always remains iid. This is correct, if we also assume that $\nu$ is chosen independently of $X$.

What happens, however, if no such independence is assumed? For example, consider a 0-1 valued iid sequence. Remove a random 1, chosen uniformly at random among all 1s. If there is no 1, then remove a random 0. In this situation, the position of the removed entry is still uniformly random, but not independent of the original sequence. Can such a removal destroy the iid property in the remaining sequence?

$\endgroup$
1
  • 5
    $\begingroup$ Your concluding example can be calculated by hand when $n=3$. The probability of getting $11$ is $1/8$ and the probability of getting $00$ is $1/2$. If the sequence were i.i.d. then we would have to have $\Pr(1) = 1/\sqrt{8}$ and $\Pr(0) = 1/\sqrt{2}$ , but $1/\sqrt{8} + 1/\sqrt{2} \ne 1$. $\endgroup$ Oct 4, 2022 at 12:26

1 Answer 1

19
$\begingroup$

The independence will be then in general lost. E.g., let $X_1,\dots,X_n$ be independent random variables each uniformly distributed on $[0,1]$. Let $M:=\max(X_1,\dots,X_n)=X_\nu$, so that $\nu$ is uniformly distributed on $[n]:=\{1,\dots,n\}$. Let $(Y_1,\dots,Y_{n-1})$ be the leftover sequence, after the removal of $X_\nu$. Then, conditionally on $M$, the $Y_i$'s are iid uniformly distributed on $[0,M]$.

So, for $n\ge2$ and $i\in[2]$ we have $E(Y_i|M)=M/2$ and $E(Y_1Y_2|M)=E(Y_1|M)E(Y_2|M)=(M/2)^2$. So, $$EY_1Y_2=E(M/2)^2>(EM/2)^2=EY_1\,EY_2,$$ with the inequality taking place because $Var\,M>0$. Thus, $Y_1$ and $Y_2$ are not independent.


Intuitively, it can be expected that the $Y_i$'s are positively dependent. Indeed, if $M$ is small, then all $Y_i$'s will be small. So, if $Y_1$ turns out to be small, a reason for that may be that $M$ is small, and then $Y_2$ will be small. Thus, the smallness of $Y_1$ seems to make $Y_2$ tend to be small.

$\endgroup$
7
  • $\begingroup$ Nice example! Could the argument be carried over to the example mentioned in the last paragraph of the original question? $\endgroup$ Oct 3, 2022 at 14:31
  • $\begingroup$ @AndrasFarago : Yes, of course. However, the expressions there are messy, because of ties, as the index $\nu$ of the maximum is not unique in that case. In that example, I suggest you consider $Y_1$ and $Y_2$ for $n=3$, which is quite elementary. $\endgroup$ Oct 3, 2022 at 14:47
  • 1
    $\begingroup$ I love this question and answer! Very interesting! $\endgroup$ Oct 3, 2022 at 18:44
  • $\begingroup$ @IosifPinelis The intuitive argument at the end of your answer does not seem to work for the 0-1 case, because the maximum is almost always the largest possible value (1). The only exception is when all terms are 0, but if the entries take value 1, say, with probability 1/2, and $n$ is large, then this happens with exponentially small probability. We may say that let's start with a conditionally iid sequence, where the condition is that not all terms are 0. Is it true that leaving out a random 1, the leftover will not be iid even in this case? $\endgroup$ Oct 3, 2022 at 19:12
  • $\begingroup$ @StanleyYaoXiao : Thank you for your comment. $\endgroup$ Oct 3, 2022 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.