Skip to main content
1 of 2

lower bound the probability of at least L collisions

Lets say we get a list $M$ containing $|M|=\sqrt{L\cdot N}$ randomly and independtly drawn elements from a set of size $N$. And lets denote the $i$-th element of the list $M$ by $M[i]$.

If we now ask for the amount of collisions $X$ in $M$, where a collision is defined as a pair of indices $(i,j)$, $i\neq j$ with $M[i]=M[j]$ we obviously have $\mathbb{E}[X]=\frac{\binom{|M|}{2}}{N}=\frac{\binom{\sqrt{LN}}{2}}{N}\approx L$.

But unfortunately this is just the expected value and gives no information about the probability distribution. Is there anything we can say about the probability of $X$ deviating from its expectation here?

We could define indicator variables $X_{i,j}$ with $X_{i,j}= 1 \Leftrightarrow M[i]=M[j]$. Because the elements of M are drawn independently at random it holds that $X_{i,j}\sim \textrm{Ber}_\frac{1}{N}$. Unfortunately these indicator variables are not independent, otherwise the number of collisions would be binomial distributed and we could use a Chernoff like argument. Another approach to bound the probability that $X\geq \alpha\mathbb{E}[X]$ for some $\alpha<1$ could therefore be to somehow bound the variance of the sum of indicator variables and use the Chebyshev inequality. So far I did not find a non-trivial way to bound this variance.

This problem looks so common, that I can not imagine, that it hasn't been studied exhaustively in literature. Unfortunately, I was not able to find a solution somewhere so far. Help in form of own calculations as well as literature suggestions would be appreciated