Is there a smooth manifold which admits only rigid metrics? Does there exist a (finite dimensional) smooth manifold $M$, such that every Riemannian metric on $M$ has no isometries except the identity?
Of course, such a manifold must not admit a diffeomorphism of finite order.
Since a surface $S$ admits a diffeomorphism of order $n$ iff its mapping  class group (MCP) has an element of order $n$ (see here), it follows that if $S$ has the above property, then its MCP has only elements of infinite order. 
 A: The survey paper "Do manifolds have little symmetry?" by Volker Puppe lists several known results about manifolds with no nontrivial finite group action. The ArXiv link is http://arxiv.org/pdf/math/0606714v1.pdf
In particular


*

*aspherical manifolds with $Z(\pi_1M)=0$ and $Out(\pi_1M)$ torsionfree 


have no finite group action (Borel) 
Examples are


*

*certain mapping tori of nilmanifolds (Conner-Raymond-Weinberger)

*certain hypertoral manifolds, i.e., n-manifolds with a degree 1 map to the n-torus (Schultz)

*certain 3-manifolds (Edwards)

*a certain Bieberbach manifold (Waldmüller)

A: The answer to the question in the first sentence is "yes".  Let $M$ be a hyperbolic 3-manifold whose isometry group is trivial.  Then by Theorem 1.1 of
Farb, Benson; Weinberger, Shmuel Hidden symmetries and arithmetic manifolds. Geometry, spectral theory, groups, and dynamics, 111–119, Contemp. Math., 387, Amer. Math. Soc., Providence, RI, 2005. (Reviewer: Bachir Bekka) 53C35 (53C23)
(which they attribute to Borel, though they do not give an original reference for it), the isometry group of every Riemannian metric on $M$ is isomorphic to a subgroup of the hyperbolic isometry group, and thus is trivial.
Along similar lines, you might be interested in Theorem H of 
Dinkelbach, Jonathan and Leeb, Bernhard,
Equivariant Ricci flow with surgery and applications to finite group actions on geometric 3-manifolds.
Geom. Topol. 13 (2009), no. 2, 1129–1173. 
It says that every finite-order diffeomorphism of a closed hyperbolic 3-manifold is smoothly conjugate to an isometry, so closed hyperbolic 3-manifolds with trivial isometry groups give examples of smooth manifolds with torsion-free diffeomorphism groups.
