As pointed out by Stankewicz, the answer  is **no**.

Let me give an explicit example of a curve $F$ of genus $4$ which is an etale triple cover of a curve of genus $2$ but which is *not* hyperelliptic. The details of this construction can be found in my paper 

[*Surfaces of general type with $p_g=q=1, K^2=8$ and bicanonical map of degree 2*][1],  Trans. Amer. Math. Soc. 358 (2006), no. 2, 759-798,

see in particular Section 4.  

Let us consider the symmetric group $S_3$, presented as
$$S_3 = \langle r, \, s \;| \; r^3=s^2=1, \, sr = r^2s \rangle$$
and the fuchsian group $\Gamma$ of genus zero and of presentation
$$ \Gamma = \langle x_1, \ldots, x_6 \; | \; x_i^2=1, \; x_1x_2 \cdots x_6=1 \rangle.$$ 
Then the epimorphism $$x_1, \, x_2 \mapsto s, \quad x_3, \,x_4 \mapsto rs, \quad x_5, \,x_6 \mapsto r^2s$$
defines a Galois cover $F \longrightarrow \mathbb{P}^1$, with Galois group $S_3$, branched at $6$ points, such that the $3$-cycles $r$ and $r^2$ act on $F$ without fixed points. Then the Hurwitz formula gives $g(F)=4$.

Moreover:

$(i)$  we have $g(F/\langle r \rangle)=2$, i.e. $F$ is an etale triple cover of a curve of genus $2$;

$(ii)$ we have $g(F/\langle sr \rangle)=1$, i.e.  $f$ is a double cover of an elliptic curve (branched at $6$ points).

Now, it is well-known that a bielliptic curve of genus at least $4$ cannot be hyperelliptic (the references are in my paper, see Remark 3.5). Hence $F$ is not hyperelliptic and we are done.

**Remark.** Your guess about the genus $3$ case is correct. In fact, a curve of genus $3$ is an etale double cover of a curve of genus $2$ if and only if it is both hyperelliptic and bielliptic, see Theorem 3.4 in the paper I quoted.


  [1]: http://www.ams.org/journals/tran/2006-358-02/S0002-9947-05-03673-1/