Let $V$ be a finite-dimensional real inner product space. You want to know the dimension of $\text{End}_G(V \otimes V^{\ast})$ where $G = O(V)$. Using the inner product we have an isomorphism $V \cong V^{\ast}$, so
$$V \otimes V^{\ast} \cong V \otimes V \cong S^2(V) + \wedge^2(V)$$
(where $+$ denotes direct sum). The inner product is itself a $G$-invariant element of $S^2(V)$, so as a $G$-representation $S^2(V)$ further splits up as the direct sum of the trivial rep and another irreprep. If this other irreprep is always irreducible and nonisomorphic to $\wedge^2(V)$ then the dimension of $\text{End}_G$ is always $3$ as desired, and this is probably not hard to prove directly.
Heuristics based on dimension counts probably won't say much; already $SO(3)$ has irreducible representations of arbitrarily high dimension.