Is there a finite group such that, if you pick one element from each conjugacy class, these don't necessarily generate the entire group?

No, this is impossible. This is a standard lemma, but I'm finding it easier to give a proof than a reference: Let $G$ be your finite group. Suppose that $H$ were a proper subgroup, intersecting every conjugacy class of $G$. Then $G = \bigcup_{g \in G} g H g^{1}$. If $g_1$ and $g_2$ are in the same coset of $G/H$, then $g_1 H g_1^{1} = g_2 H g_2^{1}$, so we can rewrite this union as $\bigcup_{g \in G/H} g H g^{1}$. There are $G/H$ sets in this union, each of which has $H$ elements. So the only way they can cover $G$ is if they are disjoint. But they all contain the identity, a contradiction. UPDATE: I found a reference. According to Serre, this result goes back to Jordan, in the 1870's. 


It is impossible. As I mentioned in the comment to Richard Stanley's answer, you are looking for a finite group $G$ with a maximal subgroup $M$ such that $M$ intersects every conjugacy class. Then $G=\cup M^g$ is the union of $M$ and its conjugates, which is wellknown to never happen. Steve 


The impossibility also follows from Jordan's lemma: Let $G$ act transitively on a set $\Omega$ with $\Omega:=n\geq 2$ then there exists a $g\in G$ such that $\chi(g)=0$ (here $\chi(g)$ denotes the permutation character). Here $\chi(g)$ denotes the permutation character. In fact with some additional work one can show that the proportion of elements $g\in G$ such that $\chi(g)=0$ is larger than or equal to $\frac{1}{n}$. So now let us see how Jordan's lemma implies that the answer is negative. So let $H$ be the group generated by the representatives of each conjugacy class of $G$ and assume that $H$ is a proper subgroup of $G$. Then we may look at the left action of $G$ on $G/H$. Since $G/H\geq 2$ and the action is transitive there exists a $x\in G$ such that $x g_i H\neq g_i H$ for each left coset $g_i H$. In other words for each $g_i$ one has that $g_i^{1}x g_i\notin H$ which in turn implies that for all $g\in G$ one has that $g^{1}xg\notin H$. Therefore the conjugacy class of $x$ does not intersect $H$ which is absurd. Note also that one gets the following corollary from the previous argument: Let $H$ be a proper subgroup of $G$ then we may always find two distinct (linear) characters of $G$ that have the same restriction on $H$. Indeed, by the previous argument there exists a conjugacy class $C$ of $G$ that does not intersect $H$. Let $D=GC$ and define $f$ to be the class function which is equal to $0$ on $D$ and $1$ on $C$ and let $g$ be the class function which is equal to $1$ everywhere.
Since $f$ and $g$ are (in a unique way) linear combinations of irreducible characters of $G$ and $fH=gH$ there must exist distinct 


A superficially different counting argument, which boils down to the same proof as before: If $H$ is a proper subgroup whose conjugates completely cover $G$, then let $G$ act on the right cosets of $H$ by right multiplication. This action is transitive. Since $H$ is a point stabilizer, the conjugates of $H$ are just all the point stabilizers. Then saying that the conjugates of $H$ cover $G$ is saying that every element of this permutation group has a fixed point. In a transitive permutation group, the average number of fixed points is $1$. The number of fixed points of the identity is the number of points, $[G:H]$. The only way every permutation can have at least the average number of fixed points is for every permutation to have exactly the average number of fixed points, so $[G:H]=1$ contradicting the assumption that $H$ is proper. 


A subgroup intersecting all conjugacy classes is usually called a conjugately dense subgroup. Here is a few related papers: Levchuk, V. M. Sylow intersections and conjugately dense subgroups of Chevalley groups. (Russian) Algebra and linear optimization (Russian) (Ekaterinburg, 2002), 161–165, Ross. Akad. Nauk Ural. Otdel., Inst. Mat. Mekh., Ekaterinburg, 2002. 20G40 Zyubin, S. A.; Levchuk, V. M. Conjugately dense subgroups of locally finite Chevalley groups of Lie rank 1. (Russian) Sibirsk. Mat. Zh. 44 (2003), no. 4, 742748; translation in Siberian Math. J. 44 (2003), no. 4, 581–586. Zyubin, S. A. Conjugately dense subgroups of 3dimensional linear groups over locally finite field. Internat. J. Algebra Comput. 15 (2005), no. 56, 1273–1280. Zyubin, S. A. Conjugately dense subgroups of free products of groups with an amalgamated subgroup. (Russian) Algebra Logika 45 (2006), no. 5, 520537, 631; translation in Algebra Logic 45 (2006), no. 5, 296–305. Zyubin, S. A. On irreducible conjugately dense subgroups of linear groups. (Russian) Dokl. Akad. Nauk 413 (2007), no. 4, 450453; translation in Dokl. Math. 75 (2007), no. 2, 266–269 20E06 (20G15) Erfanian, Ahmad; Russo, Francesco Conjugately dense subgroups in generalized FCgroups. Acta Univ. Apulensis Math. Inform. No. 20 (2009), 79–91. 

