Let us say that a (closed, connected) manifold has a symmetry if it admits a non-trivial action by a finite group. Note that I am not asking the action to be free. So for example rotating the 2-sphere by $\frac{2 \pi}{n}$ generates a non-trivial action by the cylic group of order $n$.

My question now is whether the following is true. (I remember discussing this some time ago in Bonn, but my memory is fuzzy. My internet searching has failed me, and so I ask it here).

Statement: Most manifolds do not admit any symmetries.

Question: Is the above statement true?

I think most naturally occurring manifolds, the ones we are familiar with and typically work with, do admit symmetries and so this would give another answer to this recent MO question: Situations where “naturally occurring” mathematical objects behave very differently from “typical” ones

Of course to answer this one needs to make precise what "most" means. I am quite flexible on how to interpret this. Also we better restrict to connected manifolds, and maybe also to simply connected manifolds. Is anything known about this question? I am likewise curious about anything known in this sort of direction.

This question can be asked in the smooth, PL, and Top categories.

In each case it can be rephrased in terms of torsion in the automorphism group of the manifold. For example in the smooth case the statement can be rephrased:

Alternative Statement: For most manifolds, their diffeomorphism group is torsion free.

In any manifold we can always find a disk. A boundary preserving automorphism of a disk always can be extended by the identity to the whole manifold. Thus if these admit torsion elements, then the automorphism group of any manifold of this dimension will admit torsion elements. I don't think this is a problem in the PL and Top cases. But in the Smooth case we have for $d \geq 5$:

$$ \pi_0(\textrm{Diff}_\partial(D^n)) \cong \Theta_{d+1}$$

where $\Theta_{d+1}$ is the group of exotic spheres.

Question 2: Can the finite group $\Theta_{d+1}$ be realized as a subgroup of $\textrm{Diff}_\partial(D^n)$? Is the latter group torsion free?

If $\textrm{Diff}_\partial(D^n)$ has torsion, then the main question is really most interesting in the PL and Top cases, I think.

I am aware that for Reimannian manifolds there are positive results in this direction. I am less surprised by these as I imagine a generic Reimanniant manifold to be quite "lumpy" and not even have local symmetries.

A closely related counter point which is also related to this question:

Counter-question: Are there any examples of manifolds without any symmetry? Do all manifolds admit a symmetry?