This is not true in general. Take your example and reembed everything using the $n$-th Veronese map. A subvariety $F$ of $X$ is reembedded as a variety of degree ${\rm deg}(F) n^{\dim(F)}$. In particular, bigger dimensional fibers will have larger degree than smaller dimensional fibers, provided $n$ is large enough.
EDIT: This is what I had in mind. Choose a positive integer $d$. Let $Z=(z_1,\ldots,z_N)$ be a list of all the monomials of degree at most $d$ in $x_1,\ldots,x_n$ such that $(z_1,\ldots,z_M)$ is a list of all the monomials of degree at most $d$ in $x_1,\ldots,x_m$. Using the monomials $Z$ we obtain an embedding $v_d \colon X \to \mathbb{A}^N$, called the $d$-th Veronese embedding. A property of $v_d$ is that for every subvariety $F$ of $\mathbb{A}^n$ the equality \[ deg(v_d(F)) = deg(F) \, d^{dim(F)} \] holds. Let $\pi_d \colon X \to \mathbb{A}^M$ be the projection on the first $M$ coordinates. The morphism $\pi_d$ is identical to the morphism $\pi$ (whatever that means), but the fibers of $\pi_d$ as subschemes of $\mathbb{A}^N$ are isomorphic to the image under the $d$-th Veronese embedding of the fibers of $\pi$. Thus, for $d$ large enough, the fibers of larger dimension will have degree larger than the fibers of smaller dimension.
I hope this clarifies your doubts!
