Let $X$ be a smooth projective variety over the complex numbers and assume that $X$ is equipped with the action of a finite group $G$.

Denote by $Z$ the closed subscheme of fixed points of $G$ and consider the blowup $Y=\mathrm{Bl}_Z X$ of $X$ along $Z$.

The action of $G$ extends to $Y$ and in particular to the exceptional divisor $E$. Simple examples show that $G$ doesn't need to act trivially on $E$. However, I suspect that it acts trivially on its cohomology.

So here is my question:

How does $G$ act on the complement of $H^\ast(X)$ in $H^\ast(Y)$?

I will actually be interested in the same question for the blow-up of any $G$-stable subscheme of $X$.

Thanks for your help