$\DeclareMathOperator\Norm{Norm}$Suppose $E/\mathbb{Q}(j(E))$ is a CM elliptic curve and $d$ is a non-square. Let $E_d$ denote the twist of $E$ by $\mathbb{Q}(j(E))(\sqrt{d})$. I know if $d$ is relatively prime to the conductor of $E$, then we have $$N_{E_d} = d^2N_{E}$$ However, by computational investigations, there appear to be certain primes that always divide the norm of the conductor regardless of twist.
As a computational example: I will take a curve with CM by $K=\mathbb{Q}(\sqrt{-5})$. I take the curve with $j$-invariant $$j(E)= 5887918080 (31261995198\sqrt{5} - 69903946375)$$ (i just picked the one with smallest absolute norm). Taking the absolute norm of the conductor of $E$ gives the factorization $$\Norm(N_E)=5^2 * 7^4 * 11^4 * 19^2 * 41^2 * 47^4 * 199^2 $$ Taking $d=77$, I get the twisted curve $E_d$ which has conductor $$\Norm(N_{E_d}) =5^2 * 19^2 * 41^2 * 47^4 * 199^2 $$ No matter what integer I twist by, I cannot get rid of these factors. Do I need to twist by something in $\mathbb{Q}(j(E)) =\mathbb{Q}(\sqrt{5})$?
Other examples seem to indicate that there are primes of bad reduction common to all twists. For example for CM elliptic curves over $\mathbb{Q}$ with CM by $\mathbb{Q}(\sqrt{-q})$, where $q$ is an odd prime. $q$ is a prime of bad reduction among all curves with the same $j$-invariant.
My question: What is known about the primes of bad reduction common to all CM curves with the same $j$-invariant?