In my original question, I asked which compact Lie groups $G$ have a certain property. Jim and Dan showed that this property is equivalent to $G$ having exactly two irreducible 3-dimensional representations over the complex numbers which are dual to each other.
There are at least three compact Lie groups that have such a property, namely $SU(3)$, $SO(4)$ and $SU(2)\times U(1)$. Let us go through these examples. There are exactly two irreducible 3-dimensional represenations of $SU(3)$ which are the standard representation and its dual. The irreducible 3-dimensional representations of $SO(4)$ descend from its universal cover ${\mathrm{Spin}}(4)=SU(2)\times SU(2)$ which has exactly two dual irreducible 3-dimensional representations $S^2(V)\otimes {\mathbf{1}}$ and $ {\mathbf{1}}\otimes S^2(V)$ (here $V$ is the standard representation of $SU(2)$ and $ {\mathbf{1}}$ is the trivial representation). Finally, the irreducible 3-dimensional representations of $SU(2)\times U(1) $ are $S^2(V)\otimes W$ and $S^2(V)\otimes W^{\vee}$ where $V$ and $W$ are the standard representations of $SU(2)$ and $U(1)$ respectively.
Okay. That was quite a lot. There are any other examples of semisimple compact Lie groups having exactly two irreducible 3-dimensional representations which are dual to each other? From Jim's answer, it is likely that such Lie groups have to be of low rank.
The original question is given below.
Suppose that $G$ is a compact Lie group with at least two distinct irreducible 3-dimensional representations.
Can one classify those $G$ with the following two properties?
- For any irreducible 3-dimensional representations $\pi$, the multiplicity of $\pi\otimes\pi$ at the trivial representation is 0.
- For any two distinct irreducible 3-dimensional representations $\pi$ and $\pi'$, the multiplicity of $\pi\otimes \pi'$ at the trivial representation is 1.
EDIT: To make this question more concrete, let us assume that $G$ is semisimple of rank 2. From Jim's answer, this assumption of rank 2 is probably most relevant because we are looking at 3-dimensional representations.