For a higher genus Riemann surface $\Sigma$, is it true that every nontrivial (holomorphic) automorphism is of nontrivial mapping class, i.e., not isotopic to the identity?

A much more general fact is true: any isometry of any closed negatively curved Riemannian manifold is not homotopic to the identity. There are many proofs of this; one (perhaps not the most natural) is as follows. Hartman proved that if two harmonic maps $f_0,f_1\colon M\to N$ are homotopic, where $M$ is compact and $N$ is nonpositively curved, then $f_0$ and $f_1$ are the boundary of an isometrically embedded product $f\colon M\times [0,1]\to N$. If $M=N$ is negatively curved, this is impossible (such a product region would have sectional curvature 0 in certain directions), so we conclude that no nontrivial isometry is homotopic to the identity. To apply this to your example, note that by uniformization the universal cover of the Riemann surface $C$ is the unit disk $\Delta$. Thus your curve $C$ is the quotient $\Delta/\Gamma$ by some group $\Gamma$ of biholomorphic automorphisms of $\Delta$, namely Aut($\Delta$)=PSL$_2(\mathbb{R})$. Now note that the action of PSL$_2(\mathbb{R})$ on $\Delta$ preserves the Poincaré metric (of constant curvature $1$), which thus descends to a metric of constant negative curvature on $C$. The resulting metric is preserved by any automorphism of $C$ as a Riemann surface, so in particular your original map is an isometry in this metric. 


If a surfacediffeomorphism $h$ acts trivially on rational cohomology, the Lefschetz number of $h$ is equal to the Euler characteristic of the surface $\Sigma$. By the Lefschetz fixedpoint formula, this number equals the intersection number, in $\Sigma \times \Sigma$, of the graph of the automorphism with the diagonal. In the case that $h$ is a nontrivial holomorphic automorphism, the intersections are isolated and the intersection multiplicities positive. This can occur only when the Euler characteristic is nonnegative. 


Yes, this is an old result due to Hurwitz, and it is often used in Teichmuller theory. It is cited, for instance, at p. 152 of this paper by P. Lochak. However, I do not know the original reference. 

