Let me consider the case of the Fermat quartic. Since you are talking about Castelnuovo's criterion, I guess you are interested in the projective situation.  

The $1$-form $$\psi \colon =\frac{dx}{y^3}=-\frac{dy}{x^3}$$ is holomorphic on the projective curve $F$, so the $2$-form $$\tau:=p^* \psi \wedge q^* \psi = p^* \frac{dx_1}{y_1^3} \wedge  q^* \frac{dx_2}{y_2^3}$$ (where $(x_1, \, y_1)$ are the coordinates on the first factor, $(x_2, \, y_2)$ those on the second factor and $p, \, q$ are the natural projections) is holomorphic on $F \times F$.   

One easily checks that $\tau$ is invariant by the two involutions $i_1, \, i_2$ of $F \times F$ defined by $$i_1:=i_x \times i_x \colon ((x_1,\, y_1), \, (x_2, \, y_2)) \mapsto ((-x_1,\, y_1), \, (-x_2, \, y_2)),$$ $$i_2:=i_y \times i_y \colon ((x_1,\, y_1), \, (x_2, \, y_2)) \mapsto ((x_1,\, -y_1), \, (x_2, \, -y_2)).$$ Moreover, denoting by $\iota$ the involution switching the two factors, namely $$\iota \colon ((x_1,\, y_1), \, (x_2, \, y_2)) \mapsto ((x_2,\, y_2), \, (x_1, \, y_1)),$$ we have  $\iota^* \tau = - \tau$. 

Thus, the tensor $\tau \otimes \tau$ is invariant by the action of the subgroup $G \subset \operatorname{Aut}(F \times F)$ given by $G = \langle i_1, \, i_2, \, \iota\rangle$, hence it descends to a non-zero, global holomorphic tensor on $S=(F \times F)/G$.

This shows that $h^0(S, \, K_S^{\otimes 2}) \geq 1$, in particular $S$ is *not* rational.