I know that the Torelli morphism $t_g:\mathcal{M}_g\rightarrow \mathcal{A}_g$ between the stacks of smooth curves of genus $g$ and principally polarized abelian varieties of dimension $g$ is of order 2 ramified on the hyperelliptic locus. I also know that the natural map $\sigma:Aut(C)\rightarrow Aut (J(C),\Theta)$ which to an automorphism of the smooth curve $C$ associates the induced automorphism on $(J(C),\Theta)$ is an isomorphism for $C$ hyperelliptic and is such that $Aut(J(C),\Theta)=<Aut(C),-1>$ otherwise.

My question is: how are the two notions linked? I know this is a trivial question, but i can't see how, for a non-hyperelliptic curve $C$, $Aut(J(C),\Theta)=<Aut(C),-1>$ should imply that $t_g^{-1}(J(C),\Theta)$ consists of two points in the stack $\mathcal{M}_g$..