There has been much research on computing $E$-polynomials of character varieties. You can find a lot of general theory by reading those papers (just do a search for key words).
In particular, the theorem you want is Proposition 2.1 here:
Hodge polynomials of the SL(2,C)-character variety of an elliptic curve with two marked points by Marina Logares, Vicente Muñoz.
I quote:
Suppose that $B$ is connected and $\pi : Z \longrightarrow B$ is an algebraic fibre bundle with fibre $F$ (not necessarily locally trivial in the Zariski topology) and that the action of $\pi_1(B)$ on $H^∗_c(F)$ is trivial. Suppose that $Z, F, B$ are smooth. Then $e(Z) = e(F)e(B).$
The hypotheses hold in particular in the following cases:
- $B$ is irreducible and $\pi$ is locally trivial in the Zariski topology.
- $\pi$ is a principal $G$-bundle with $G$ a connected algebraic group.