How do modular functions of level $N>1$ transform under the full modular group?

Question

Let $f$ be a modular function of level $N>1$ and let $\gamma\in SL_2(\mathbf Z)$. Write $$f(\gamma\tau)=j(\gamma,\tau)f(\tau).$$ First question

What can we say in general about the factor $j(\gamma,\tau)$? In other words, how does the function $f$ transform under the full modular group $SL_2(\mathbf Z)$? Is there some reference on this? What about the case when $f$ is a root of some Hauptmodul?

Background

I will provide some examples to illustrate the problem. Set $\mu_n=e^{2\pi i /n}$ and $$\alpha=\begin{pmatrix}a&b\\c&d\end{pmatrix}\in SL_2(\mathbf Z).$$

Let $j$ be the modular invariant and $\gamma_2(\tau)=\sqrt[3]{j(\tau)}$. We have the formula $$\gamma_2(\alpha\tau)=\mu_3^{a^2cd+ac-cd-ab}=\gamma_2(\tau)$$ and $\gamma_2$ is a modular function of level $3$. Similarly, if $\gamma_3(\tau)=\sqrt[2]{j(\tau)-1728}$ then $$\gamma_2(\alpha\tau)=\mu_{2}^{ac+bd+bc}\gamma_2(\tau).$$ Thus $\gamma_3$ is a modular function of level $2$. (Note that we consider these roots of $j$ because $j(e^{2\pi i/3})=0$ with multiplicity $3$ and $j(i)-1728=0$ with multiplicity $2$.)

Next let $$\mathfrak f=\mu_{48}^{-1}\frac{\eta\left( \frac{\tau+1}{2}\right)}{\eta\left( \tau\right)}.$$ Here $\eta$ is the Dedekind eta function. We have $$\mathfrak f(\alpha\tau)^3=\mu_{16}^{2ad+2cd-ac-bd-2d^2}\mathfrak f(\tau)^3.$$

There are also formulas for $\mathfrak f$ and related functions. Observe that the function $\mathfrak f^{24}$ is a Hauptmodul for the subgroup of $SL_2(\mathbf Z)$ generated by the matrices $$\begin{pmatrix}1&2\\0&1\end{pmatrix},\begin{pmatrix}0&-1\\1&0\end{pmatrix}.$$

Second question

How can we investigate the corresponding maps $$\begin{pmatrix}a&b\\c&d\end{pmatrix}\longmapsto\mu_{16}^{2ad+2cd-ac-bd-2d^2},$$ $$\begin{pmatrix}a&b\\c&d\end{pmatrix}\longmapsto\mu_3^{a^2cd+ac-cd-ab},$$ in a systematic manner? Is there some reference?

More background

The functions $\gamma_2,\gamma_3$ and $\mathfrak f$ are considered by Weber in his Lehrbuch der Algebra. Weber uses them to generate certain class fields. This was subsequently used by Heegner to determine all imaginary quadratic fields of class number one.

Answer 1

In general, there is little that one can say about the function $j(\gamma,\tau)$ in the formula $f(\gamma \tau) = j(\gamma,\tau) f(\tau)$, and in general, it is not easy to take a modular function of level $N > 1$ and know for sure how it transforms under every matrix in ${\rm SL}_{2}(\mathbb{Z})$. What one normally does however (and you can see this in Shimura's book that Paul Garrett mentions in his comment) is you can take a finite-dimensional vector space $V$ of modular functions with the property that for any $f \in V$, $f(\gamma \tau)$ is once again in $V$.

One such example, discussed in Shimura's book, is to take all nonzero vectors $\vec{v} = \begin{bmatrix} a & b \end{bmatrix}$ with entries in $\mathbb{Z}/N\mathbb{Z}$ and look at the functions $f_{\vec{v}}(\tau) = \frac{E_{4}(\tau) E_{6}(\tau)}{\Delta(\tau)} \wp_{\langle 1,z \rangle}\left(\frac{az+b}{N}\right)$. This set of functions has the property that $f_{\vec{v}}(\gamma \tau) = f_{\vec{v} \gamma}(\tau)$.

This kind of object (essentially a vector space of functions together with an action of ${\rm SL}_{2}(\mathbb{Z})$) goes by the name of a vector-valued modular form, and have been studied quite a bit.

For your second question, the maps that you describe of the form $$ \left(\begin{bmatrix} a & b \\ c & d \end{bmatrix}\right) \mapsto \mu_{16}^{2ad + 2cd - ac - bd - 2d^{2}} $$ are $1$-cocycles of ${\rm SL}_{2}(\mathbb{Z})$ with values in $\mathbb{C}^{\times}$, and they are parametrized by group cohomology. Sometimes these also go by the name "multiplier systems". There's also a lot written about these. One place you might start lookin is the paper here.