That's a good question. This is not quite an answer  but a bit long for a comment.

A quick remark is that for $N$ odd a Haar-random element of $O(N)$ can be obtained as $\epsilon U$ where $\epsilon=\pm1$ with equal probability, and $U$ is a Haar-random element of $SO(N)$. So if your monomial has an even number of factors the integrals over $O(N)$ and $SO(N)$ coincide and thus Weingarten calculus is applicable. This is of course because for $N$ odd $-I$ has determinant $-1$ and is in the center of $O(N)$. I don't know if there is a similar trick for $N$ even.

I said it is a good question because, when looking at the vast probability/representation theory literature, I didn't see much as far an analogue of Weingarten calculus for special groups. Even the work of Chatterjee (and Basu and Ganguly,...) on $SO(N)$ lattice gauge theories does not seem to use Weingarten calculus. So for $SO(N)$, my answer to the question is: I don't know. However, for $SU(N)$ there is a combinatorial calculus. It is explained in my two answers to

https://mathoverflow.net/questions/255492/how-to-constructively-combinatorially-prove-schur-weyl-duality?noredirect=1&lq=1

This technique was worked out explicitly by <a href="https://aip.scitation.org/doi/abs/10.1063/1.523581">Creutz</a> but it has its roots in the work of Clebsch and Hilbert in invariant theory. See for example, the averaging operator $[\cdot]$ used by Hilbert on p. 523 of <a href="https://eudml.org/doc/157506">"Ueber die Theorie der algebraischen Formen"</a> is basically the same as Creutz's formula for $SU(2)$. Also note that if a combinatorial Weingarten-like calculus for $SO(N)$ is perhaps missing, there is at least an Euler angle parametrization due to <a href="https://eudml.org/doc/58378">Hurwitz</a> (see <a href="https://arxiv.org/abs/1512.09229">this review</a> by Diaconis and Forrester).