Thank you. Yes, that are the invariants and relations I found, but I didn't see what variety they describe, or what singularities we get. So I asked if somebody may knew it :-)

Okay, thanks. That is a little bit illuminating. But why does $\sigma_2$ have the whole $\mathbb{P}^1$ in the fixed locus? It switches the coordinates, so shouldn't only the points $[1:1]$ and $[1:-1]$ be fixed? And what about the element $\sigma_3$ it maps $(x,y,[l:m])$ to $(-x,-y,[-m:l])$ so there we have fixed points $[1:i]$ and $[1:-i]$ lying over the point $x=y=0$. This singularity looks more complicated. How to describe it?

@t3suji : Ah, thanks. I think misunderstood that the whole time. So it makes sense to talk about the the invariant direct image of $End(E)$ on $X//G$ although this sheaf itself does not descend to $X//G$? The last point just means that there is no sheaf $F$ on the quotient such that $\pi^{*}F\cong End(E)$ as $G$-linearized sheaves?

@t3suji : No. I am saying $End(E)^G$, the sheaf of invariants, descends. G acts nontrivially via conjugation on End(E) with the finite stabilizers, so that this bundle will not descend.

@Jason: Thanks! This seems good. Since the birational map between the generic curves is induced by $\phi$, this birational map between the generic curves gives rise to the desired birational map between $C$ and $C'$.

@Francesco: Thanks. I shortened the title a bit, because I did not want it to be too long. Maybe it got shortened too much. I hope now it is not misleading.

Thank you. I was hoping that something was known because this is a line bundle in this case...but if this question is equivalent to finding the cohomology of a rank three vector bundle on $X$, I guess it is really hard in this generality.