I assume that $\phi$ is an automorphism of $G.$ Note that if $\phi$ is inner then trivially $\rho$ and $\rho\circ\phi$ are equivalent, thus the answer depends only on the image of $\phi$ in the outer automorphism group $Out(G).$
If $G$ is a finite group (or, more generally, compact group) and representations are finite-dimensional, so that they are determined up to isomorphism by their characters, then this problem admits a complete theoretical solution using the character theory. The automorphism group $Aut(G)$ acts on the set $\{C_i\}$ of the conjugacy classes of $G$, this action factors through the action of $Out(G),$ and
$$\chi_{\rho\circ\phi}(C)=\chi_{\rho}(\phi(C)),\qquad (*)$$
where $\chi_\rho$ is the character of $\rho$ and $C$ is any conjugacy class. Since representations are determined by their characters,
$$\rho\circ\phi\simeq\sigma \iff \chi_{\rho\circ\phi}=\chi_\sigma,$$
which can be tested using $(*).$ Of course, applying this method in practice requires good knowledge of the character table of $G$ and the outer automorphism group $Out(G).$