# are irreducible representations with large fixed subspaces trivial?

Say that $G$ is a finite group, and $V$ is an irreducible representation of $G$, over an algebraically closed field $k$. Suppose that for every $g \in G$, there is some subspace $W_g \subset V$ which is (pointwise) fixed by $g$, such that the dimension of $W_g$ is at least half the dimension of $V$.

If $k$ has characteristic zero, then a simple argument with character tables shows that $V$ must be the trivial representation. (When $k = \mathbb{C}$, then $\sum_{g \in G} tr_V(g)$ has positive real part; so appealing to the Lefschetz principle, $\sum_{g \in G} tr_V(g)$ is nonzero whenever $k$ has characteristic zero.)

Is this still true when $k$ has positive characteristic?

-
Two questions: motivation for the question? need to invoke Lefschetz principle? (Characters of a fixed finite group in characteristic 0 only require a small extension of the rationals as splitting field.) – Jim Humphreys Jan 24 '11 at 15:43
I am behind on this, but is there a known counterexample to the famous exercise "if a character of an irreducible representation over $\mathbb C$ does not take the value $0$, then it is the trivial representation" over characteristic $p$? – darij grinberg Jan 24 '11 at 20:20
Uhm. Sorry. I meant "then it is a $1$-dimensional representation". – darij grinberg Jan 24 '11 at 22:53

Unless I have misread either your question or their result, Corollary 1.2 in "Average dimension of fixed point spaces with applications" by Bob Guralnick and Attila Mar\'oti includes a positive answer to the question. You can find this on the arxiv at

http://arxiv.org/PS_cache/arxiv/pdf/1001/1001.3836v1.pdf

-
+1! They claim that this was conjectured in the 60's and just resolved now! – Ben Webster Jan 26 '11 at 21:14
I think too that corollary 1.2 answers the question. Very nice reference ! – Alex Jan 27 '11 at 1:19
Very interesting. It answers the question ($N=G$) and more. So one needs CFSG to answer this apparently innocent question? – Frieder Ladisch Jan 27 '11 at 10:24

I don't know the answer, but here are a few remarks :

(0) As has been pointed out before, "irreducible" and "indecomposable" are not the same for representations in positive characteristic. "Irreducible" is a stronger property. (For example, irreducible representations of commutative groups are always $1$-dimensional, whereas indecomposable representations don't have to be. No commutative group is going to give a counterexample.)

(1) The answer is "yes" if we look at representations over a field of characteristic prime to the order of $G$ (because representations theory of $G$ will be the same as in characteristic zero).

(2) If we want to construct a counterexample, then we should not take $G$ to be a $p$-group (as in the two attempts above). Because then $k$ will have to be of characteristic $p$, but then every finite-dimensional representation of $G$ over $k$ has a nonzero fixed vector.

(3) More generally, it is not possible to construct a counterexample in characteristic $p$ if $G$ has a normal $p$-Sylow $H$ (let $W$ be the subspace of vectors fixed by $H$, it is nonzero because $H$ is a $p$-group, it is stable by $G$ because $H$ is normal, hence it has to be the whole space because the representation is irreducible, but then we are reduced to the case of $G/H$, which has order prime to $p$, and see (1)).

That means, I'm afraid, no easy counterexamples with very small groups.

-
Re (2): More radically, the only irreducible representation of a $p$-group in characteristic $p$ is the trivial one. In general, their number is equal to the number of $p$-regular conjugacy classes of elements of $G$ (of which there is only one, the identity in a $p$-group). – Alex B. Jan 24 '11 at 15:52
Moreover, if $G$ is generated by two elements $g$ and $h$, then $\mathrm{Fix}(G) = \mathrm{Fix}(g) \cap \mathrm{Fix}(h)$. If $G$ obeys the conditions of the problem (at least with strict inequality), then the left hand side is nontrivial, so $V$ has a trivial subrep. So you can't find examples which are generated by two elements, which rules out a lot of groups. – David Speyer Jan 24 '11 at 19:58

ADDED: The following is based on my misinterpretation of irreducible as "cannot be nontrivially written as a direct sum." Moreover as Torsten Ekedahl points out, it is easy to generalize this example to arbitrary characteristic.

Strictly speaking, this fails in characteristic 2. Take $G= \mathbb Z/2$ and $V$ to have the basis $u$ and $v$ with the non-trivial element of $G$ acting by $u \mapsto v$ and $v \mapsto u$. Then the $1$-dimensional vector subspace spanned by $u+v$ is invariant under $G$, so it satisfies your hypothesis, but $G$ is not trivial.

This is a rather marginal counterexample, though, and you could rule it out in a number of ways: assume characteristic not 2, assume that $V$ is simple, or weaken the conclusion to just ask that $V$ have a trivial submodule.

-
I don't understand, irreducibility is explicitly given as a condition in the question. (The $2$-dimensional example you give also exists in any characteristic $p>0$ with $G=\mathbb Z/p$, just let the generator be given by a $2\times2$ Jordan block.) – Torsten Ekedahl Jan 24 '11 at 8:12
@Torsten: are you sure that in Dustin's example the representation is reducible? It seems to me that the subspace $u+v=0$ has no $G$-invariant complement in char $2$. If char $\neq 2$, then the $G$-invariant complement is $u-v=0$, and one can find it by the usual average method; but if char $k$ divides $|G|$, as in this case, this is no longer possible. Or am I missing something? – Francesco Polizzi Jan 24 '11 at 14:11
@Francesco: Dustin's representation is indecomposable, but reducible. A similar attempt is in my answer – Andrei Moroianu Jan 24 '11 at 14:33
@Andrei Ok, I understand now. Thanks! – Francesco Polizzi Jan 24 '11 at 15:21

This is only a partial answer: There is no counterexample with $G$ a $p$-solvable group, in fact no counterexample such that $G$ contains a $p$-complement. This can be seen by mimicking the proof in characteristic 0, using the Brauer character. So let $\phi$ be the Brauer character of the module. The assumption assures that $\phi(g)$ has nonnegative real part.
Write $G_{p'}$ for the set of $p$-regular elements of $G$, and for a Brauer character $\phi$ write $\eta_{\phi}$ for the associated projective indecomposable character. For $\phi$ and $\theta$ any irreducible Brauer characters we have
$$\frac{1}{|G|} \sum_{g \in G_{p'}} \eta_{\phi}(g) \overline{\theta(g) } = \delta_{\phi\theta}.$$ Now if $G$ contains a $p$-complement $H$, then it is known that $\eta_1 = (1_H)^G$, the permutation character of $G$ on the cosets of $H$. In particular, the values of $\eta_1$ are non-negative integers. It follows that $\sum_{g\in G_{p'}} \eta_1(g) \phi(g) \neq 0$, and thus $\phi=1$ by the above orthogonality relation.

In general, $\eta_1$ is only a constituent of $(1_H)^G$ and may have negative values (of course, the values are integers), so this approach doesn't work.

-