show/hide this revision's text 5 deleted 920 characters in body

2nd UPDATE: Forget the old solutions. Let's assume that the representation does not contain the trivial representation, then $\sum_{g\in G}gx=0$ for every $x$. Therefore, for a norm one vector $x$, $$ 1=|(x,x)|=|\sum_{g\ne 1}(gx,x)|\le\sum_{g\ne 1}|(gx,x)| $$ So it will never occur, that $|(gx,x)|<\frac1{|G|-1}$ for every $g$.

UPDATE: There is always an $\varepsilon>0$ such that for every $x$ of norm one there is $g\ne 1$ in $G$ with $|(gx,x)|>\varepsilon$. Simply choose for $g$ any element whose order is not divisible by 4 and apply the old solution.

OLD SOLUTION: If the order of the group is not divisible by 4, even more is true. There is $\varepsilon>0$ depending on the group (or rather on the orders of its elements) such that if $|(gx,x)|<\varepsilon$ for some $x$ of norm one and all $g\ne 1$, then $(gx,x)=0$ for every $g\ne 1$. For a proof, consider ${\mathbb C}^n$ as ${\mathbb R}^{2n}$, then $U(n)\subset SO(n)$ and every $g\in G$ is a rotation. For given $g$, consider the map $x\mapsto |(gx,x)|^2$. It becomes smallest, when $x$ is perpendicular to the axis of $g$. Its value for such $x$ depends on the order of $g$ and so only finitely many minima $\ne 0$ are taken. Now choose $\varepsilon$ smaller than those minima.

show/hide this revision's text 4 added 339 characters in body

2nd UPDATE: Forget the old solutions. Let's assume that the representation does not contain the trivial representation, then $\sum_{g\in G}gx=0$ for every $x$. Therefore, for a norm one vector $x$, $$ 1=|(x,x)|=|\sum_{g\ne 1}(gx,x)|\le\sum_{g\ne 1}|(gx,x)| $$ So it will never occur, that $|(gx,x)|<\frac1{|G|-1}$ for every $g$.

UPDATE: There is always an $\varepsilon>0$ such that for every $x$ of norm one there is $g\ne 1$ in $G$ with $|(gx,x)|>\varepsilon$. Simply choose for $g$ any element whose order is not divisible by 4 and apply the old solution.

OLD SOLUTION: If the order of the group is not divisible by 4, even more is true. There is $\varepsilon>0$ depending on the group (or rather on the orders of its elements) such that if $|(gx,x)|<\varepsilon$ for some $x$ of norm one and all $g\ne 1$, then $(gx,x)=0$ for every $g\ne 1$. For a proof, consider ${\mathbb C}^n$ as ${\mathbb R}^{2n}$, then $U(n)\subset SO(n)$ and every $g\in G$ is a rotation. For given $g$, consider the map $x\mapsto |(gx,x)|^2$. It becomes smallest, when $x$ is perpendicular to the axis of $g$. Its value for such $x$ depends on the order of $g$ and so only finitely many minima $\ne 0$ are taken. Now choose $\varepsilon$ smaller than those minima.

show/hide this revision's text 3 added 248 characters in body

UPDATE: There is always an $\varepsilon>0$ such that for every $x$ of norm one there is $g\ne 1$ in $G$ with $|(gx,x)|>\varepsilon$. Simply choose for $g$ any element whose order is not divisible by 4 and apply the old solution.

OLD SOLUTION: If the order of the group is not divisible by 4, even more is true. There is $\varepsilon>0$ depending on the group (or rather on the orders of its elements) such that if $|(gx,x)|<\varepsilon$ for some $x$ of norm one and all $g\ne 1$, then $(gx,x)=0$ for every $g\ne 1$. For a proof, consider ${\mathbb C}^n$ as ${\mathbb R}^{2n}$, then $U(n)\subset SO(n)$ and every $g\in G$ is a rotation. For given $g$, consider the map $x\mapsto |(gx,x)|^2$. It becomes smallest, when $x$ is perpendicular to the axis of $g$. Its value for such $x$ depends on the order of $g$ and so only finitely many minima $\ne 0$ are taken. Now choose $\varepsilon$ smaller than those minima.

show/hide this revision's text 2 added 49 characters in body
show/hide this revision's text 1