Question: Let $G$ be a finitely generated group with exponential growth. Is there a finite generating set $S \subset G$, such that the associated Cayley graph $Cay(G,S)$ contains a binary tree?
The existence of such a tree clearly implies exponential growth.
Kevin Whyte showed in Amenability, Bilipschitz Equivalence, and the Von Neumann Conjecture, Duke Journal of Mathematics 1999, p. 93-112, that such trees exist if $G$ is non-amenable. So the question is only open for amenable groups of exponential growth.
One good reason for such a binary tree to exist is the existence of a free semigroup inside $G$. In fact, if $G$ is solvable, then the existence of such a semigroup is known to be equivalent to exponential growth (and equivalent to being not virtually nilpotent). This is part of some version or extension of the Tits alternative. Grigorchuk constructed an amenable torsion group with exponential growth, which does not contain such a semigroup, but it contains a binary tree.
EDIT: Al Tal pointed out in an answer below that Benjamini and Schramm covered the non-amenable case (this is 2. from above) already in Benjamini and Schramm "Every Graph With A Positive Cheeger Constant Contains A Tree With A Positive Cheeger Constant", GAFA, 1997.