To be more precise (but less snappy): is there an example of a group G with finitely many infinite-index subgroups H_1, ..., H_n and elements k_1, ..., k_n such that G is the union of the left cosets k_1 H_1 , ..., k_n H_n? And what if we relax the requirement that these all be left cosets, and ask: can G be the union of finitely many such cosets, some being left cosets, others being right cosets?
If G is amenable then this can't happen, since any coset of an infinite-index subgroup must have measure 0. So this immediately rules out any abelian group G.
I've tried playing around with the only non-amenable groups that I'm comfortable with, the free groups on two or more generators. A few months ago I thought I found a simple counterexample in the free group on $\aleph_0$ generators, but now I've lost my notes and am beginning to doubt I ever had such an example.
(This question was asked to me by a friend who's interested in some kind of application to model theory, but I think it's interesting as a stand-alone puzzle.)