(This is a slightly reformatted and clarified version of my question from math.SE, since I believe
the answer there is wrong and its poster has not responded to my comment in over two weeks.)
Let $\: \langle \hspace{-0.02 in}G,\hspace{-0.04 in}\cdot,\hspace{-0.04 in}\mathcal{T}\hspace{.02 in}\rangle \:$ be a locally compact Hausdorff topological group, and $\mu$
be a left Haar measure on $\: \langle \hspace{-0.02 in}G,\hspace{-0.04 in}\cdot,\hspace{-0.04 in}\mathcal{T}\hspace{.02 in}\rangle$. Suppose that $\mu$ is not right-invariant.
Does it follow that there is a Borel subset $B$ of $G$ and an element
$g$ of $G$ such that $\:\operatorname{closure}\hspace{.015 in}(B)\:$ is compact and for $C$ given by $$C= \{ \; h\in G \:\: : \:\: g^{\hspace{-0.02 in}-1} h\hspace{.02 in}g \: \in \: B \; \}$$ which satisfies $\: C\subset B \;\;$ and $\;\; \operatorname{interior}\hspace{.02 in}(B\hspace{-0.04 in}-\hspace{-0.04 in}C\hspace{.02 in}) \neq \emptyset$ ?
(In other words, is there a topological group such that the Haar measures are
not bi-invariant but there is no "single-conjugacy" explanation for that fact?)