Is there a pointed space $(X, p)$ such that for infinitely many integers $n\geq 1$ there is a map $(X, p)\to (X,p)$ non-trivial on $\pi_n(X, p)$?

In particular $\pi_n(X, p)$ must be non-trivial for infinitely many $n$.

What if require in addition $X$ to be a finite-dimensional CW complex?