11
$\begingroup$

Let $X=(X_1,\ldots,X_n)$ be an iid sequence of random variables, and let $\nu$ be a uniformly random integer in the range $1,\ldots,n$. Then $\xi_\nu$ is a random entry of $X$. Is it always true that after deleting such a random entry from $X$, the remaining sequence is still iid?

Due to the uniform randomness of the index $\nu$, it is tempting to answer that the leftover sequence always remains iid. This is correct, if we also assume that $\nu$ is chosen independently of $X$.

What happens, however, if no such independence is assumed? For example, consider a 0-1 valued iid sequence. Remove a random 1, chosen uniformly at random among all 1s. If there is no 1, then remove a random 0. In this situation, the position of the removed entry is still uniformly random, but not independent of the original sequence. Can such a removal destroy the iid property in the remaining sequence?

$\endgroup$
1
  • 5
    $\begingroup$ Your concluding example can be calculated by hand when $n=3$. The probability of getting $11$ is $1/8$ and the probability of getting $00$ is $1/2$. If the sequence were i.i.d. then we would have to have $\Pr(1) = 1/\sqrt{8}$ and $\Pr(0) = 1/\sqrt{2}$ , but $1/\sqrt{8} + 1/\sqrt{2} \ne 1$. $\endgroup$ Oct 4 at 12:26

1 Answer 1

19
$\begingroup$

The independence will be then in general lost. E.g., let $X_1,\dots,X_n$ be independent random variables each uniformly distributed on $[0,1]$. Let $M:=\max(X_1,\dots,X_n)=X_\nu$, so that $\nu$ is uniformly distributed on $[n]:=\{1,\dots,n\}$. Let $(Y_1,\dots,Y_{n-1})$ be the leftover sequence, after the removal of $X_\nu$. Then, conditionally on $M$, the $Y_i$'s are iid uniformly distributed on $[0,M]$.

So, for $n\ge2$ and $i\in[2]$ we have $E(Y_i|M)=M/2$ and $E(Y_1Y_2|M)=E(Y_1|M)E(Y_2|M)=(M/2)^2$. So, $$EY_1Y_2=E(M/2)^2>(EM/2)^2=EY_1\,EY_2,$$ with the inequality taking place because $Var\,M>0$. Thus, $Y_1$ and $Y_2$ are not independent.


Intuitively, it can be expected that the $Y_i$'s are positively dependent. Indeed, if $M$ is small, then all $Y_i$'s will be small. So, if $Y_1$ turns out to be small, a reason for that may be that $M$ is small, and then $Y_2$ will be small. Thus, the smallness of $Y_1$ seems to make $Y_2$ tend to be small.

$\endgroup$
7
  • $\begingroup$ Nice example! Could the argument be carried over to the example mentioned in the last paragraph of the original question? $\endgroup$ Oct 3 at 14:31
  • $\begingroup$ @AndrasFarago : Yes, of course. However, the expressions there are messy, because of ties, as the index $\nu$ of the maximum is not unique in that case. In that example, I suggest you consider $Y_1$ and $Y_2$ for $n=3$, which is quite elementary. $\endgroup$ Oct 3 at 14:47
  • 1
    $\begingroup$ I love this question and answer! Very interesting! $\endgroup$ Oct 3 at 18:44
  • $\begingroup$ @IosifPinelis The intuitive argument at the end of your answer does not seem to work for the 0-1 case, because the maximum is almost always the largest possible value (1). The only exception is when all terms are 0, but if the entries take value 1, say, with probability 1/2, and $n$ is large, then this happens with exponentially small probability. We may say that let's start with a conditionally iid sequence, where the condition is that not all terms are 0. Is it true that leaving out a random 1, the leftover will not be iid even in this case? $\endgroup$ Oct 3 at 19:12
  • $\begingroup$ @StanleyYaoXiao : Thank you for your comment. $\endgroup$ Oct 3 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.