Just saw this old question thanks to the link from
the new
MO question #212929. Another flavor of counterexample:
let $G$ be the $ax+b$ group over some field $k$, and let
$t(b) \in G$ be the transformation $x \mapsto x + b$.
Then the $t(b)$ with $b \neq 0$ are all conjugate, but
a pair $(t(b_1),t(b_2))$ is simultaneously conjugate with
$(t(b'_1),t(b'_2))$ **iff** $b_1/b_2 = b'_1/b'_2$.
Suppose, then, that $b_1/b_2 \neq b'_1/b'_2$,
and that both quotients are irrational
(i.e. if $mb_1=nb_2$ or $mb'_1=nb'_2$ for some integers $m,n$
then $m=n=0$ in $k$). Then the pairs
$(g,h) := (t(b_1),t(b_2))$ and $(g',h') := (t(b'_1),t(b'_2))$
are not related by simultaneous conjugacy, but cannot be
distinguished by the conjugacy class of any $w(g,h)$.

The same is true if we take $G = {\rm SL}_2(k)$
or $G = {\rm PSL}_2(k)$ and work in its $ax+b$ subgroup
$({* \; * \atop 0 \; *})$ [with $t(b) = ({1 \; b \atop 0 \; 1})$].
Indeed that was the example that first came to mind,
suggested by the restriction to diagonalizable elements in
MO 212929
(NB the $t(b) \in {\rm SL}_2(k)$ with $b\neq 0$ are not diagonalizable).

In the smallest finite examples of this kind, $k$ is the 4-element field, and
$b_1/b_2$, $b'_1/b'_2$ are the two elements of $k$ other than $0$ and $1$.
Then the $ax+b$ group is isomorphic with $A_4$, and
${\rm SL}_2(k) \cong {\rm PSL}_2(k) \cong A_5$,
and in each case we can take
$g=g'=(12)(34)$, $h=(13)(24)$, and $h'=gh=(14)(23)$.
[The pairs $(g,h)$ and $(g',h')$ become conjugate in $S_4$ and $S_5$,
which are the extensions of the $ax+b$ group and ${\rm SL}_2(k)$
by the Galois automorphism of $k$ that also switches the two
irrational elements of $k$.]