No. Let $\Gamma_i$ be the lower central series defined by $\Gamma_1=\pi_1(X,x)$, $\Gamma_{i+1}=[\Gamma_1,\Gamma_i]$. The *Johnson filtration* $\text{Mod}_g(k)$ is the descending filtration of the mapping class group relative to $x$ defined by:

$f\in \text{Mod}_g(k)\iff f$ acts trivially on $\Gamma_1/\Gamma_k$

The first term $\text{Mod}_g(2)$ is the *Torelli group*, consisting of diffeomorphisms acting trivially on homology. The next term $\text{Mod}_g(3)$ is the *Johnson kernel*. By a beautiful theorem of Johnson, this is the subgroup generated by Dehn twists around separating curves.

By residual nilpotence of surface groups, we have $\bigcap \text{Mod}_g(k)=\{1\}$, but every individual term in the filtration is nontrivial. It is not hard to see that every term of the Johnson filtration contains pseudo-Anosovs. (Indeed every normal subgroup of the mapping class group contains pseudo-Anosovs, from which Long concluded that any two normal subgroups intersect nontrivially!)

Thus since $\text{Mod}_g(k)$ is normal, if we take $f\in\text{Mod}_g(k)$ we have $\gamma^{-1}\cdot g^{-1}fg(\gamma)\in \Gamma_k$ for all $g$ and all $\gamma$.