My answer is for $p \in (2,4)$, the other case should follow similar.
Let $t \in (0,1)$ be given, such that $$\frac1p = \frac{1-t}{2}+\frac{t}{4}.$$ I will go to use the following consequence of Hölder (this is similar to the Riesz Interpolation Theorem, but somewhat simpler to use in your case): $$\lVert f \rVert_p \le \lVert f\rVert_2^{1-t} \lVert f\rVert_4^t$$ Using $\delta(p) = (1-t)\;\delta(2) + t \; \delta(4)$ you find $$\lambda_k^{-\delta(p)} \lVert e_k \rVert_p \le \Big(\lambda_k^{-\delta(2)}\lVert e_k\rVert_2\Big)^{1-t}\Big(\lambda_k^{-\delta(4)}\lVert e_k\rVert_4\Big)^t = \Big(\lambda_k^{-\delta(4)}\lVert e_k\rVert_4\Big)^t.$$ This shows, that your lim-sup holds for all $p \in (2,4)$ if it holds for $p = 4$.
Edit: A intuition for the statement is the following: You have $$\lambda_k^{-\delta(p)} \lVert e_k \rVert_p$$ is bounded for $p \in [2,6]$ as $k \to \infty$. If for some $p \in (2,6)$, the $\limsup$ is zero, than you can interpolate and obtain a zero $\limsup$ for all $p \in (2,6)$. Note there is nothing special about $p = 4$.
A similar argument is: if a sequence of functions $f_k$ is bounded in $L^p$ and converges towards zero in $L^q$, $q < p$, then it converges to $0$ in $L^r$ for all $r \in [q,p)$.