I believe that the idea described by Ian Agol works, and can be elaborated on as follows. The general fact we want to establish is that the convex hull of a finite collection $X$ of points in $M$ is not smooth. Actually, the so called horro-hull, that is the intersection of all horro-balls which contain $X$ is not smooth. It follows that the convex hull is not smooth, because it lies in the horro-hull.
First let us recall that balls in $M$ are convex, because the distance function from a point in $M$ is convex. Given a ball $B$ in $M$ and a point $x$ on the boundary $\partial B$ of $B$, we can always find a larger ball $B'$ which contains $B$ and intersects $\partial B$ only at $x$. Letting the radius of $B'$ go to infinity yields a horro-ball which contains $X$ and whose boundary passes through $x$.
Let $B$ be the ball of smallest radius containing $X$. Then some point $x$ of $X$ lies on $\partial B$. Let $B'$ be a ball which contains $B$ and intersects $\partial B$ only at $x$. Then all points of $X$ other than $x$ lie in the interior of $B'$. It follows that every point in an open neighborhood of the center of $B'$ is the center of a ball which contains $X$ and whose boundary passes through $x$. Hence the normal cone of the horro-hull of $X$ at $x$ must have interior points, which yields that the tangent cone of the boundary of the horro-hull at $x$ cannot be flat.
Addendum (12/8/2023): More generally, if we have a finite collection of points in $M$ which are convexly independent, i.e., no point is contained in the convex hull of others, then every point is a singularity of the convex hull. Establishing this fact requires a bit more work, because in general there may not exist a sphere which passes through each of the points and contains the other points in the interior of the ball that it bounds.
To prove this, given a point $p$ from the collection $X$ of points, consider the convex hull $C$ of $X-\{p\}$. By assumption $p$ has distance $d>0$ from $C$. Let $C_d$ be the outer parallel body of $C$ at distance $d$. Then the boundary of $C_d$ forms a smooth ($C^{1,1}$) convex surface passing through $p$, while $X-\{p\}$ lies in the interior of $C_d$ (convexity of $C_d$ is due to the fact that the distance function from a convex set in a Cartan-Hadamard manifold is convex, and smoothness follows from the fact that a ball of radius $d$ rolls freely inside $C_d$).
Now note that by perturbing the elements of $X-\{p\}$ we may construct many different smooth convex surfaces which pass through $p$ and contain $X-\{p\}$.
All these surfaces contains the convex hull of $X$.
Hence as discussed earlier, the tangent cone of the boundary of the convex hull of $X$ at $p$ cannot be flat. Indeed the normal cone of the convex hull at $p$ (which is dual to the tangent cone of the convex hull) has full dimension.
