edit January 18th, 2021
Final nail in the f.g. splendid coffin. Apparently, the dihedral group and its variants are indeed the only non-splendid groups. So maybe I should've called non-splendid groups splendid and vice versa, because this would roll off the tongue better then. I of course originally thought non-splendid groups would usually be some crazy monsters, and that the dihedral group was kind of an outlier.
Let's say an automorphism $\phi : H \to H$ of a group $H$ flips it if $\phi(h) = h^{-1}$ for all $h \in H$. Obviously a flipping automorphism can only exist on an abelian group, since $a^{-1}b^{-1} = (ba)^{-1} = \phi(ba) = \phi(b)\phi(a) = b^{-1}a^{-1}$.
Theorem. A finitely-generated group $G$ is non-splendid if and only if it admits an internal automorphism that flips some non-trivial finite-index normal torsion-free abelian subgroup.
Proof. We know from the previous edit (from yesterday) that any f.g. non-splendid group $G$ is virtually abelian, and a non-trivial finite-index normal torsion-free abelian subgroup implies virtual abelianity, so we may restrict to virtually abelian $G$.
Let $H \leq G$ be any non-trivial finite-index normal torsion-free abelian subgroup (so $H \cong \mathbb{Z}^d$ for some $d \geq 1$), which exists by intersecting the conjugates of any non-trivial finite-index torsion-free abelian subgroup. We show that $H$ is flipped by an inner automorphism of $G$ if and only if $G$ is non-splendid. Because the choice of $H$ is arbitrary, this shows the a priori stronger fact $\exists H: P \implies \mbox{non-splendid} \implies \forall H: P$. Let $F \subset G$ be the finitely many coset representatives for $H$.
Suppose first that conjugation by $t \in G$ flips $H$. Write $h^t = t^{-1} h t$. Pick $C = \{ f^{-1} t, tf \;|\; f \in F \}$. Now, for any $g \in G$ we can write $g = h f$ for $h \in H, f \in F$, and then
$$ g C g \ni hf \cdot f^{-1}t \cdot hf = h t h f = t \cdot h^t h \cdot f = tf \in C. $$
So $G$ is not splendid.
Suppose then that for all $g \in G$, conjugation by $G$ does not flip $H$. Then there exists $h \in H$ which is not flipped by any $g \in G$, and in fact we can find $h$ such that conjugation by any $g \in G$ takes $h$ very far from $h^{-1}$. More precisely, for each $R$ there exists $h \in H$ such that $d(h^g, h) \geq R$ in the right-invariant word metric $d$ of $G$.
There are many ways to see the above, I guess you can geometrize and it's a basic lemma about finite reflection groups. A direct proof is also easy: Observe that it's enough to bound the word metric of $H$ since the distortion is bounded since $H$ is finite index. We only need to consider conjugation by each $f \in F$. Now take for each $f \in F$ an element $h_f$ not flipped by $f$, and consider products of the form $h_{f_1}^{n_1} \circ h_{f_2}^{n_2} \circ \cdots \circ h_{f_{|F|}}^{n_{|F|}}$ where the $n_i$ have different scales. Basically, take $h_{f_1}$ not flipped by $f_1$, so $h_{f_1}^{n_1}$ is conjugated very far from $(h_{f_1}^{n_1})^{-1}$, and take a massive $n_1$. If also $f_2$ does not flip $h_{f_1}$, we continue to $f_3$, if $f_2$ does flip $h_{f_1}$ then it does not flip $h_{f_1}^{n_1} \circ h_{f_2}^{n_2}$ where $n_2$ has a smaller scale, but $f_1$ does not flip this either since $n_1$ is much bigger than $n_2$. And so on. (In the geometric situation I suppose you can replace this by Baire or the fact $\mathbb{R^d}$ is not a union of finitely many $(d-1)$-dimensional subspaces.)
We can now use the elements $h$ given by the two paragraphs above to prove splendidness. Let $C$ be arbitrary, and suppose $C \subset B_R$, the ball of radius $R$ in the right-invariant word metric of $G$ around identity. Let $h$ be such that $d(h^a, h^{-1}) > 2R$ for all $a \in G$. If $hCh \cap C \neq \emptyset$ then $h^a h = a^{-1} h a h = a^{-1} b$ for some $a, b \in C$, so $d(h^a, h^{-1}) \leq 2R$, a contradiction. So $G$ is splendid. Square.
edit January 17th, 2021
Theorem. Suppose $G$ is a f.g. discrete group which is not splendid. Then $G$ is virtually abelian.
Proof. We know from the first version of my answer (see below after the text original) that $G$ is virtually nilpotent. Then it's well-known that $G$ has a finite-index torsion-free nilpotent subgroup $H$. Let $C$ be the finite set contradicting splendidness.
First, let's think about what splendidness says when $H$ is a torsion-free subgroup. For any $g \in H$ and $i \in \mathbb{N}$, we have $g^i a g^i = b$ for some $a, b \in C$. Now, pigeon says $g^i a g^i = b$, $g^{i+k} a g^{i+k} = b$ for some $k > 0$, and we get $g^i a g^i = g^{i+k} a g^{i+k}$, in other words $g^{-k} = a g^k a^{-1}$. Clearly this $k$ is bounded, and by taking the least common multiple of the $k$ that occur, we find $a \in C$ such that for all $g \in H$, $(g^k)^a = g^{-k}$ (true for either convention for the $g^a$ notation, because inversion is an involution).
Note that in particular $H$ is now a torsion-free nilpotent group which admits an automorphism such that for all $g \in H$, $(g^k)^a = g^{-k}$. I claim that if $H$ is not abelian, this is impossible, which will conclude the proof.
To see this, suppose $H$ is not abelian. I claim that in this case there is a three-dimensional Heisenberg group inside: We use the convention $[x,y] = xyx^{-1}y^{-1}$ if it matters. Let $(H_i)_i$ be the lower central series of $H$, $H_n \neq 1, [H_n, H] = 1$. Take $1 \neq c = [a,b] \in H_n$ a non-trivial commutator, where $a \in H_{n-1}, b \in H$. Now, $c$ is superlinearly distorted in $\langle a,b,c \rangle$, so certainly $a$ and $b$ cannot satisfy any identity. If $\langle a,b,c \rangle$ satisfies an identity that is not true in the Heisenberg group, then commuting elements to normal form $a^k b^\ell c^m$ we see that $c$ satisfies an identity on its own, contradicting the torsion-freeness assuption.
Now it suffices to show that the three-dimensional Heisenberg group $H = \langle a,b,c \;|\; [a,b] = c, [a,c], [b,c] \rangle$ does not admit an automorphism $\phi : H \to H$ satisfying $\phi(g^k) = g^{-k}$ for all $g \in H$. This is geometrically obvious, go around a big cycle on the $\langle a,b \rangle$-quotient, and you jump up by the area in the central direction. If you apply the inverse elementwise, you get a cycle of the same area in the same direction (just reordered). So you don't flip the direction.
In algebra, take $[a^{km}, b^{km}] = c^{k^2 m^2}$. Now apply $\phi$ to get
$$ [a^{-km}, b^{-km}] = [\phi(a^{km}), \phi(b^{km})] = \phi([a^{km}, b^{km}]) = \phi(c^{k^2 m^2}) = c^{-k^2 m^2} $$
But
$$ [a^{-km}, b^{-km}] = [a^{km}, b^{km}]^{b^{km} a^{km}} = [a^{km}, b^{km}] = c^{k^2 m^2} $$
since $[a^{km}, b^{km}]$ is central. So $c^{-k^2 m^2} = c^{k^2 m^2}$ which is the desired contradiction. Square.
Presumably you could spell out in more detail what the situations are where you get non-splendidness. I suppose it's just that some element flips the torsion-free part of the finite-index abelian subgroup. Also, the proof essentially uses the observation that non-splendidness implies essentially that some element conjugates things to their inverses, giving that inversion is almost an automorphism. This is strange since it is of course a flip automorphism, which sounds like it should ultimately be the reason that non-splendid groups must be virtualy abelian. Maybe you could cut out the middle man (ICC quotients) by working more on the algebra.
original
Joshua Frisch pointed out to me in private communication that ICC $\implies$ splendid, which seems to give a good partial answer to the question. Let me restrict to f.g. infinite discrete groups $G$.
Let $\mathcal{D}$ be the class of quotients of the fundamental group of the Klein bottle $\langle h, a \;|\; h^a = h^{-1} \rangle$, where $h$ is mapped to an element of infinite order and $a$ to a nontrivial element. For example the quotient where $a^2 = e$ is added gives the dihedral group.
Theorem. If $G$ f.g. infinite non-splendid, then $G$ is virtually nilpotent, and contains a group from $\mathcal{D}$.
This is presumably not a characterization, but at least it reduces the problem to virtually nilpotent groups, which are much better understood than general ones. First, there's the following result of [1] (which is indeed a consequence of Neumann's theorem, but there's a nice algebraic trick that I missed).
Theorem. If $G$ is an f.g. ICC group, then for every finite $Z \subset G$ there exists $g$ such that all solutions to $gxg^{\pm} = y$ with $x, y \in Z$ (if they exist) satisfy $x = y = e$.
This gives ICC $\implies$ splendid. There's a minor complication with involutions, so I give a proof:
Theorem. If $G$ is an f.g. ICC group, then it is splendid.
Proof. Suppose ICC, and fix some symmetric generating set. Let $n$ be arbitrary and take $Z = B_N$ for $N = 2n$. By the previous theorem, for some $g \in G$, among $x, y \in Z$ the equation $gxg^{\pm} = y$ can only have solutions with $x = y = e$. If $g$ is not an involution, there aren't any such solutions either, and we are done.
If $g$ is an involution, then take any $h$ of word norm exactly $n$, and consider $gh$ instead. Now if $gh \cdot k \cdot gh = k'$ for $k, k' \in B_n$, then $g \cdot hk \cdot g = k' h^{-1}$ and here $hk, k'h^{-1} \in Z$, and this cannot happen by the assumption on $g$. So we have $gB_ng \cap B_n = \emptyset$. Square.
Next, there's a theorem of [1,2] (see also [3]).
Theorem. Every f.g. infinite group is either virtually nilpotent, or has a non-trivial ICC quotient.
And also let's prove a lemma.
Lemma. If $G$ has a splendid quotient, it is splendid.
Proof. Let $\phi : G \to H$ be the quotient map, and $C \subset G$ be finite. Then $\phi(C) = C'$ is also finite, and thus $hC'h \cap C' = \emptyset$ for some $h \in H$. Letting $\phi(g) = h$, we have $\phi(gCg) \cap \phi(C) = hC'h \cap C' = \emptyset$, so $gCg \cap C = \emptyset$ as well, and this means $G$ is splendid. Square.
We obtain the following:
Theorem. Every non-splendid group is virtually nilpotent.
Proof. If $G$ is not virtually nilpotent, then it has a non-trivial ICC quotient. That quotient is then splendid, and thus $G$ is splendid. Square.
Now we can prove the theorem I stated at the beginning of the post.
Proof. An infinite f.g. nilpotent group contains $H \cong \mathbb{Z}$, which is abelian. By the lemma in my post, there exists $a \in G$ and $h \in H \setminus \{e\}$ such that $h^a = h^{-1}$ for all $h \in H$. Then $h$ and $a$ give the quotient, taking $n$ the order of $a$. Square.
[1] Erschler, A.; Kaimanovitch, V. A., Arboreal structures on groups and the associated boundaries, Arxiv https://arxiv.org/abs/1903.02095]
[2] Duguid, A. M.; McLain, D. H., (FC)-nilpotent and (FC)-soluble groups, Proc. Camb. Philos. Soc. 52, 391-398 (1956). ZBL0071.02204.
[3] McLain, D. H., Remarks on the upper central series of a group, Proc. Glasg. Math. Assoc. 3, 38-44 (1956). ZBL0072.25702.
[4] Frisch, J.; Ferdowsi, P. V., Non-virtually nilpotent groups have infinite conjugacy class quotients, Arxiv https://arxiv.org/abs/1803.05064]