I would like to ask the following.

Let $(a_n)$ be a sequence of natural numbers such that $\sum_{k=1}^{\infty}\frac{1}{a_k}$ converges. Is it true that for infinitely many $m$, there is a $n<m$ such that $a_m-a_n$ has a prime divisor greater than $m$?

In other words, is it true that if for every $m, n$, the difference $a_m-a_n$ has all its prime factors less than or equal to $m$, then $\sum_{k=1}^{\infty}\frac{1}{a_k}=+\infty$?