Here's another approach... one can show for every $n$ there can be solutions for at most finitely many $p$; and for any given $n$ it's not hard to find these $p$ explicitly.

For fixed $n$ the question is when $x^n - (x-1)^n$ and $(x+1)^n - x^n$ can have a common factor (for some integer $x$). Applying the Euclidean algorithm to the two polynomials will yield an integer $N(n)$. For solutions to exist, $p$ must be a factor of $N(n)$, so only finitely many $p$ will do.

(To be perfectly rigorous about this I have to show that the two polynomials have no common factor in $\mathbb{Z}[x]$. But if they did then they would have a common root, say in $\mathbb{C}$. Considering absolute values, we see that the roots of $x^n = (x-1)^n$ all have real part $\frac{1}{2}$ while the roots of $x^n = (x+1)^n$ all have real part $-\frac{1}{2}$. So indeed the polynomials have no common factor, so the Euclidean algorithm will give a constant $N(n)$.

To see that $p$ must divide $N(n)$: the Euclidean algorithm guarantees that $N(n)$ is a linear combination of the two polynomials in $\mathbb{Z}[x]$. So for any value of $x$, $N(n)$ is a linear combination of $x^n - (x-1)^n$ and $(x+1)^n - x^n$. Hence if $x$ is a solution then $N(n)$ is a multiple of $p$.)

A quick calculation by hand gives

N(3) = 2

N(4) = 30

N(5) = 44.

Since $p$ cannot be $2$ or $3$, we see that...

For $n=3$ there are no solutions...

For $n=4$ there are solutions only when $p=5$...

For $n=5$ there are solutions only when $p=11$.