As pointed out by Stankewicz, the answer to tour first question is no.
Let me give an explicit example of a curve $F$ of genus $4$ which is an $\acute{\text{e}}$tale triple cover of a curve of genus $2$ but which is not hyperelliptic. The details of this construction can be found in my paper
Surfaces of general type with $p_g=q=1, K^2=8$ and bicanonical map of degree 2, Trans. Amer. Math. Soc. 358 (2006), no. 2, 759-798,
see in particular Section 4.
Let us consider the symmetric group $S_3$, presented as
$$S_3 = \langle r, \, s \;| \; r^3=s^2=1, \, sr = r^2s \rangle$$
and the fuchsian group $\Gamma$ of genus zero and of presentation
$$ \Gamma = \langle x_1, \ldots, x_6 \; | \; x_i^2=1, \; x_1x_2 \cdots x_6=1 \rangle.$$
Then the epimorphism $$x_1, \, x_2 \mapsto s, \quad x_3, \,x_4 \mapsto rs, \quad x_5, \,x_6 \mapsto r^2s$$
defines a Galois cover $F \longrightarrow \mathbb{P}^1$, with Galois group $S_3$, branched at $6$ points, such that the $3$-cycles $r$ and $r^2$ act on $F$ without fixed points. Then the Hurwitz formula gives $g(F)=4$.
Moreover:
$(i)$ we have $g(F/\langle r \rangle)=2$, i.e. $F$ is an $\acute{\text{e}}$tale triple cover of a curve of genus $2$;
$(ii)$ we have $g(F/\langle sr \rangle)=1$, i.e. $F$ is a double cover of an elliptic curve (branched at $6$ points).
Now, it is well-known that a bielliptic curve of genus at least $4$ cannot be hyperelliptic (the references are in my paper, see Remark 3.5). Hence $F$ is not hyperelliptic and we are done.
In this example $F$ is Galois cover of the projective line, but perhaps the construction can be modified in order to find also counterexamples to your second question.
Remark. Your guess about the genus $3$ case is correct. In fact, a curve of genus $3$ is an $\acute{\text{e}}$tale double cover of a curve of genus $2$ if and only if it is both hyperelliptic and bielliptic, see Theorem 3.4 in the paper I quoted.