The desired integral is given in equation (13) of arXiv:1812.06069:
$$\int_{{SU}(N)}s_\lambda(u)\overline{s_\mu(u)}du=\sum_{q=-\infty}^\infty\prod_{i=1}^N\delta_{\lambda_i,\mu_i+q},$$ where $\lambda=(\lambda_1,\lambda_2,\ldots\lambda_N)$ with $\lambda_1\geq\lambda_2\cdots\geq 0$, and similarly for $\mu$. The $U(N)$ integral corresponds to the $q=0$ term in the sum over $q$.