Added: jc in his comment is right. $E(\mathbb R)$ also has subgroups of order $3$, by mistake I factored through such a group. Now I chose the correct subgroup, and the image $g(\mathbb R)$ isn't a circle anymore. In fact, $g(\mathbb R)$ is contained in a circle if and only if there is a linear fractional function $\lambda(X)$ over $\mathbb C$ such that $\lambda(g(X))\in\mathbb R(X)$. By the correct choice of the subgroup $C$ below this does not happen.
Data: In this file I provide Sage code which verifies the example given below. To use this program, one needs the data provided here.
Answer: The answer is no. In the following I'll describe how to find two rational functions $f,g\in\mathbb C(X)$ both of odd prime degree $\ell$ such that the following holds:
- $f(g(X))\in\mathbb R(X)$.
- $g(\mathbb R\cup\{\infty\})$ is a smooth closed Jordan curve in the complex plane.
Let $E$ be an elliptic curve, defined over the reals. For $p\in E(\mathbb C)$ we let $\bar p$ be the complex conjugate of $p$. Choose $E$ such that the following holds:
- There is a point $w\in E(\mathbb R)$ with no $y\in E(\mathbb R)$ with $w=2y$.
- There is a point $z\in E(\mathbb C)$ of order $\ell$ with $\bar z\notin\langle z\rangle$. (That is actually automatically satisfied at least for $\ell=3$.)
Then $C=\langle z\rangle$ is a subgroup of order $\ell$ of $E$, and $E'=E/C$ is an elliptic curve over $\mathbb C$. Let $\phi:E\to E'$ be the associated isogeny, and $\phi':E'\to E$ be the dual isogeny. Then $\phi'\circ\phi:E\to E$ is the multiplication by $\ell$ map.
Let $\beta$ be the automorphism of order $2$ sending $p\in E(\mathbb C)$ to $w-p$. Similarly, define the involutory automorphisms $\beta'$ of $E'$ and $\beta''$ of $E$ by $\beta'(p')=\phi(w)-p'$ and $\beta''(p)=\ell w-p=\phi'(\phi(w))-p$.
Let $\psi$ be the degree $2$ covering map $E\to E/\langle\beta\rangle=P^1(\mathbb C)$, and define likewise $\psi':E'\to E'/\langle\beta'\rangle=P^1(\mathbb C)$ and $\psi'':E\to P^1(\mathbb C)$.
Let $f(X)$ and $g(X)$ be the rational functions defined implicitly by $\psi'\circ\phi=g\circ\psi$ and $\psi''\circ\phi'=f\circ\psi'$. Note that $\psi$, $\psi'$ and $\psi''$ are defined over $\mathbb R$.
As the multiplication by $\ell$ map $\phi'\circ\phi$ is defined over the reals, we obtain $f(g(X))\in\mathbb R(X)$.
We next claim that $g(X)$ is injective on $\mathbb R$. Suppose there are distinct real $u,v$ with $g(u)=g(v)$. Pick $p,q\in E(\mathbb C)$ with $\psi(p)=u$, $\psi(q)=v$. Then $\psi'(\phi(p))=g(u)=g(v)=\psi'(\phi(q))$, so $\phi(p)=\phi(q)$ or $\phi(p)=\phi(w)-\phi(q)$. Upon possibly replacing $q$ with $w-q$ we may assume $\phi(p)=\phi(q)$, so $p-q\in C$.
Next we study the effect of complex conjugation. As $\psi$ is defined over the reals, and $\psi(p)=u$ is real, we have $\psi(\bar p)=\psi(p)$, so $\bar p=p$ or $\bar p=w-p$. Likewise, $\bar q=q$ or $\bar q=w-q$. Recall that $p-q\in C$, and $\bar C\cap C={0}$ by condition 2. So we can't have $(\bar p,\bar q)=(p,q)$, nor $(\bar p,\bar q)=(w-p,w-q)$.
Thus without loss of generality $\bar p=p$, $\bar q=w-q$. As $p-q$ and $\bar p-\bar q=p-w+q$ have order $\ell$, we see that $2p-w=r$ with $\ell r=0$ and $r\in E(\mathbb R)$. So $w=2(p+\frac{\ell-1}{2}r)$, contrary to condition 1. Furthermore, the function $g(X)$ behaves well at infinity by this geometric interpretation
I computed an explicit example in order to add evidence that these arguments are correct: I picked the elliptic curve $Y^2=X^3-91X+330$ and $w=(5,0)=2(5+4i,4-16i)=2y$ and $\ell=3$. The functions $g(X)$ and $f(X)$ are defined over a number field of degree $16$ (degree $8$ coming from $C$, and another degree $2$ extension from $i$), so I do not put them here. Instead they are given in the links at the beginning of the answer. The image $g([-100,100])$ is:
Remark: This example came out of a group and Galois theoretic examination of the question. If there is some interest, I might add some more details about this aspect.
