$\newcommand{\Diff}{\mathrm{Diff}}$The previous examples have $\pi_2 \Diff(M)$ non-trivial, but finitely generated.
Here is an example of a manifold $M$ where $\pi_2 \Diff(M)$ is not finitely generated.
The manifold is $S^1 \times D^5$, and $\Diff(M)$ denotes the group of diffeomorphisms that acts trivially on the boundary. David Gabai and I proved this in the last year., in Knotted 3-balls in $S^4$, arXiv:https://arxiv.org/abs/1912.09029
In general we can show that $\pi_{n-3} \Diff(S^1 \times D^n)$ is not finitely generated, for all $n \geq 3$. We construct a non-trivial family $S^{n-3} \to \Diff(B)$ where $B$ is what we call a `barbell manifold'. This manifold is a boundary connect-sum of two copies of $S^{n-1} \times D^2$. We then embed the barbells in $S^1 \times D^n$, and extend the diffeomorphisms.
The map $\pi_{n-3} \Diff(S^1 \times D^n) \to \pi_{n-3} \mathrm{Emb}(D^n, S^1 \times D^n)$ detects our diffeomorphisms, and one can further descend via a Cerf `scanning' type construction to detect these elements in the homotopy-group $\pi_{2n-4} \mathrm{Emb}(I, S^1 \times D^n)$, which rationally looks somewhat like a two-variable Laurent polynomial ring.
If you want the manifold to be closed, I believe $\pi_{n-3} \Diff(S^1 \times S^n)$ will also not be finitely-generated. The $n=3$ case appears in the above paper, and the $n>3$ case will be in a follow-up.