When does the double coset representative for a congruence subgroup contain a $\text{SL}_2(\mathbb{Z})$-conjugacy class? In the paper p-adic L-functions and p-adic periods of modular forms, Greenberg/Stevens assert that if $\sigma_l:=\begin{pmatrix}l&0\\0&1\end{pmatrix}$ is the usual Hecke operator at $l$ double coset representative, and $\Gamma$ is the congruence group $\Gamma_1(N)$, then $g\sigma_lg^{-1}$ continues to lie in the same double coset class if $l\equiv 1 \pmod{N}$, for any $g\in \text{SL}_2(\mathbb{Z})$. I thought this would be straightforward to see via some sort of restricted row operations/Euclidean algorithm argument, but am struggling. What is the argument here, and what goes wrong when $N$ does not divide $l-1$?
 A: Why the condition $\ell = 1 \bmod N$ is necessary. Suppose $g \sigma_\ell g^{-1}$ lies in the double coset $\Gamma \sigma_\ell \Gamma$. Then $g \sigma_\ell g^{-1} = r \sigma_\ell s$ for some $r, s \in \Gamma_1(N)$; but $r \sigma_\ell s$ is the product of three matrices which are upper-triangular mod $N$, so it must be upper-triangular mod $N$.
If we take $g = \begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix}$, this amounts to requiring that $\begin{pmatrix} \ell & 0 \\ \ell-1 & 1 \end{pmatrix}$ be upper-triangular mod $N$, which is precisely the condition that $\ell = 1\bmod N$.

Why the condition is sufficient. One checks easily that if $\ell \nmid N$ then the double coset $\Gamma_1(N) \sigma_\ell \Gamma_1(N)$ contains every matrix of determinant $\ell$ which is congruent to $\begin{pmatrix} * & * \\ 0 & 1 \end{pmatrix}$ mod $N$. If $\ell = 1 \bmod N$, then $g \sigma_\ell g^{-1}$ satisfies these conditions, so it lies in the double coset $\Gamma_1(N) \sigma_\ell \Gamma_1(N)$.

Remark. For a general prime $\ell$ (even assuming $\ell \nmid N$), the double coset $\Gamma_1(N) \sigma_\ell \Gamma_1(N) = \Gamma_1(N) \begin{pmatrix} \ell & 0 \\ 0 & 1 \end{pmatrix}\Gamma_1(N)$ is not what is normally called $T_\ell$. It's sometimes known as $T_\ell^*$; and when $\ell \nmid N$ (as here), one has $T_\ell^* = \langle \ell \rangle^{-1} T_\ell$. 
So there are really two assertions being blended together here, both of which need $\ell = 1 \bmod N$ to work: firstly, that the double coset $T_\ell^*$ is stable under conjugation; secondly, that it coincides with the double coset $T_\ell$.
