A problem with Neumann's lemma.

In a paper I am reading at the moment (Hrushovski Martin, elimination of imaginaries in $Q_p$), in some proof they use the following fact (at least this would be enough to get their proof going, but maybe we have more hypothesis) :

If we have a group $G$ acting on a set $X$ containing elements $(e_i\mid i<\omega)$ such that all $g\in G$ fixes all the $e_i$ but a finite number, is it true that all but a finite number of $e_i$ are fixed by $G$?

The authors say it is a consquence of Neumann's lemma (my guess is the one saying that a group covered by a finite number of cosets of subgroups is covered by the ones of finite index) but I have been trying to figure it out for some time already and cannot. If anyone has any idea of how that might work, a little help would be very welcome.

Some days later : Thanks a lot for the counter-examples, they really helped me understand.

If anyone is interested, what I really needed was that there are at most a finite number of $e_i$ with an infinite orbit under $G$. This is false in general (see the counter-example given in the answer) but Newmann's lemma does imply that if there are an infinity of elements with an infinite orbit then for all $N$, we can find a $g$ and $N$ elements that are not fixed by $g$. And as it happens that my $G$ is in fact the automorphism group of a very saturated model (sorry, some model theory had to barge in) by compacity I can find an automorphism that moves an infinity of $e_i$.

I hope any of this made sense.

-
How about $\oplus_{n=1}^\infty(C_2)$, $C_2$ cyclic of order 2, acting on $X=\{1,2,3,4,...\}$ via the generator of the $n$th $C_2$ switching $2n-1$ and $2n$, and leaving the rest alone? This is perhaps a counterexample if it all adds up... –  Kevin Buzzard Jun 4 '11 at 14:04
It seems to work indeed... I'll go back to the paper to see if there are some hypothesis I forgot... –  Silvain Rideau Jun 4 '11 at 14:41
A "brute force" counterexample: Let $X$ be any infinite set and let $G$ consist of those permutations $g$ of $X$ such that $\{x\in X: g(x)\neq x\}$ is finite. (It's trivial to check that $G$ is a group.) Then take the $e_i$ to be any countable sequence of distinct elements of $X$. –  Andreas Blass Jun 5 '11 at 2:44
Let $S_\infty=\bigcup_{n=1}^\infty S_n$ be the infinite symmetric group consisting of permutations of $\mathbb{N}$ fixing all but finitely many elements. Then $\{e_i=i\}$ satisfy your conditions but no element $i\in \mathbb{N}$ is fixed by $S_\infty$.