Let $G$ be a complex adjoint group. Let $u\in G$ be unipotent. The group $A(u):=\pi_0(Z_G(u))$ acts on the set of components of the Springer fiber $\mathcal{B}_u$, the variety of Borel subgroups that contain $u$. We know $\mathcal{B}_u$ is equi-dimensional.
Could it be true that every element of $A(u)$ stabilizes at least one component of $\mathcal{B}_u$?
My motivation is the following: When $G$ is the corresponding split group over $\mathbb{F}_q$, I wish to say "$\#\mathcal{B}_u(\mathbb{F}_q)$ has the order of $q^{\dim\mathcal{B}_u}$." But then we need the above property to make sure the Frobenius stabilizes at least one component. Thank you!