*Your (!)* [MO-question][1] (together with the answer/comments) shows that the group of order $3$ is also a Galois group of a Galois cover $\mathbb P^1_{\mathbb Q}\to\mathbb P^1_{\mathbb Q}$. In general, as $\text{Aut}_{\mathbb Q}(P^1_{\mathbb Q})=\text{PGL}_2(\mathbb Q)$, your question reduces to finding elements of finite order in $\text{PGL}_2(\mathbb Q)$. Note that if $A\in\text{GL}_2(\mathbb Q)$ represents an element of order $m$ in $\text{PGL}_2(\mathbb Q)$, then $A^m$ is a scalar matrix. Let $a$ be the element on the diagonal. The eigenvalues of $A$ fulfill a degree $2$ equation over the rationals. So $X^m-a$ has a factor of degree at most $2$. Working that out should give an answer.


  [1]: http://mathoverflow.net/questions/244541/action-of-mathbbz-3-mathbbz-on-p1