My question: Is there a known characterization for polynomials $P\in \mathbb{R}[x,y]$ with the property that $$P(x,y) = 0 \wedge P(y,x) =0 \Rightarrow x = \overline{y}$$ and the number of the solutions to the system is finite?

After a change of variables, one have the equivalent condition for $Q\in \mathbb{R}[x,y]$ that $$Q(x,iy) = 0 \wedge Q(x,-iy) =0 \Rightarrow x,y \in \mathbb{R}.$$

This is related to stable polynomials, but not quite. (There is a nice characterization of real stable polynomials, by Lax).

There exist plenty of such polynomials, for example, one could construct a series of such polynomials recursively $$P_n = x P_{n-1} - y P_{n-2}+P_{n-3}$$ with start values $P_0 = 0,P_1 = 0, P_2 = 1.$

This type of polynomials arise naturally from certain determinants of Toeplitz matrices.