In some recent doodlings, I got myself to the point where what I was trying to understand would work out if the following claim were true:

Let $G$ be a group, $g\in G$, and $\rho:G \to \operatorname{End}_k(V)$ a $G$-module over some commutative ring $k$. Let $H < G$ denote the subgroup centralizing $g$ (i.e. $h\in H$ iff $hg=gh$), and consider restricting $V$ to $H$. Then $\operatorname{End}_H(V)$, the ring of $k$-linear maps commuting with all actions of $\rho(h)$ for all $h\in H$, obviously contains both $\rho(g)$ and $\operatorname{End}_G(V)$. The claim is that in fact $\rho(g)$ together with $\operatorname{End}_G(V)$ generate $\operatorname{End}_H(V)$ as a subring of $\operatorname{End}_k(V)$.

This is presumably a classical fact that I have since forgotten from a first course in representation theory. Does it have an easy proof?