In general, no, even if $M$ and $X$ are $C^\infty$.

A simple example on a flat torus $M$. The function $f(x,y):=\cos(x)\cos(y)$ has a non-degenerate maximum point at the origin. The unstable manifold of the origin wrto the gradient field of $f$ is the square $(-\pi,\pi)\times(-\pi,\pi)$, not smooth. 

More generally, by the Whitney extension theorem, one can construct a non-negative $C^\infty$ function $f$ on $\mathbb{R}^d$ with compact support a non-smooth closed nbd of the point $p$ (e.g., in dimension 2, a von Koch curve). Also, such that $p$ is a maximum for $f$, and the only critical point of $f$ in $U:=\{f >0\}$. In such a situation, $U$ is exactly the unstable manifold of $p$ for the gradient field $X$ of $f$. Note that since $f$ has compact support, you can transport it on any manifold $M$.