Let $L = \text{lcm}(2,\ldots, n+1)$.  Then $p = Lm -1$ satisfies your conjecture for $n$ if  $Lm/(k+1)-1$ for $k = 0 \ldots n$ are all primes.  Dixon's conjecture implies that this is the case for infinitely many $m$.

See [OEIS sequence A078502](https://oeis.org/A078502).