To be more precise (but less snappy): is there an example of a group $G$ with finitely many infinite-index subgroups $H_1,\dots, H_n$ and elements $k_1,\dots, k_n$ such that $G$ is the union of the left cosets $k_1 H_1 , ..., k_n H_n\ ?$ And what if we relax the requirement that these all be *left* cosets, and ask: can $G$ be the union of finitely many such cosets, some being left cosets, others being right cosets?

If $G$ is amenable then this can't happen, since any coset of an infinite-index subgroup must have measure $0$. So this immediately rules out any abelian group $G$.

I've tried playing around with the only non-amenable groups that I'm comfortable with, the free groups on two or more generators. A few months ago I thought I found a simple counterexample in the free group on $\aleph_0$ generators, but now I've lost my notes and am beginning to doubt I ever had such an example.

(This question was asked to me by a friend who's interested in some kind of application to model theory, but I think it's interesting as a stand-alone puzzle.)