This answer explains some positive results.  Let $B$ be a smooth complex variety, and let $\pi:Y\to B$ be a smooth, projective morphism with connected fibers of dimension $n$.  Let $E_B$ and $F_B$ be locally free $\mathcal{O}_Y$-modules of ranks $e$ and $f$.  Let $\varphi_B:E_B\to F_B$ be a morphism of $\mathcal{O}_Y$-modules.  For every nonnegative integer $k\leq \text{min}(e,f)$, let $D_{\leq k}(\varphi_B)$ denote the maximal closed subscheme of $Y$ on which the rank of $\varphi_B$ is $\leq k$.  Denote by $B^o\subseteq B$ the maximal open subscheme such that for every nonnegative integer $\ell\leq k$, either $D_{\leq \ell}(\varphi_B)$ is empty if $(e-\ell)(f-\ell) > n$, or, if $(e-\ell)(f-\ell) \leq n$, then $D_{\leq \ell}(\varphi_B)\setminus D_{\leq (\ell-1)}(\varphi_B)$ is smooth over $B^o$ and every geometric fiber has pure dimension $n-(e-\ell)(f-\ell)$.  In other words, $B^o$ is the maximal open subscheme of $B$ over which each $D_{\leq \ell}(\varphi_B)$ is <i>transversal</i>.  Denote by $Z\subset B$ the closed complement of $B^o$.  Denote by $E\subset Z$ the union of all irreducible components of $Z$ that are divisors in $B$.

Now let $X$ be a smooth, projective complex variety.  Let $E$ and $F$ be locally free $\mathcal{O}_X$-modules of ranks $e$ and $f$ such that the locally free $\mathcal{O}_X$-module
$\textit{Hom}_{\mathcal{O}_X}(E,F)$ is globally generated.  Denote by $B$ the complex affine space of the (finite-dimensional) complex vector space $\text{Hom}_{\mathcal{O}_X}(E,F)$, i.e., there is a universal morphism of coherent sheaves $\varphi_B:\text{pr}_2^*E\to \text{pr}_2^*F$ on the product variety $B\times X$. 

<B>Proposition.</B>  (1) The open subscheme $B^o$ of $B$ is dense, i.e., it is not empty.<br>
(2) Assume that $n-(e-k)(f-k)$ is positive and that
$D_{\leq k}(\varphi_B)$ surjects to $B$. Then every (nonempty) connected component of the 
fiber of $D_{\leq k}(\varphi_B)$ over each generic point of $E$ has pure dimension $n-(e-k)(f-k)$, but it may be nonreduced.<br>
(3) Assuming that least common multiple of the multiplicities of the irreducible components of each connected component of the fiber of $D_{\leq k}(\varphi_B)$ equals $1$ for each generic point of $E$, then every fiber of $D_{\leq k}$ over every point of $B$ is connected.

<B>Proof.</B> The statements (1) and (2) follow from the method of incidence correspondences.  Form the geometric vector bundle $\rho:X\to X$ associated to the locally free sheaf $\textit{Hom}_{\mathcal{O}_X}(E,F)$ with its universal morphism $\phi:\rho^*E\to \rho^*F$.  By the usual theory of generic determinantal varieties, each locally closed subschemes $D_{\leq \ell}(\phi)\setminus D_{\leq (\ell-1)}(\phi)$ of $H$ is smooth over $X$ and every nonempty fiber has pure dimension $ef - (e-\ell)(f-\ell)$.  Moreover, this fiber is irreducible if the dimension $ef-(e-\ell)(f-\ell)$ is positive.  Since $\textit{Hom}_{\mathcal{O}_X}(E,F)$ is globally generated, the induced morphism from $B\times X$ to $H$ is smooth and geometrically surjective with irreducible geometric fibers.  Thus, the inverse image of each locally closed subscheme $D_{\leq \ell}(\varphi)\setminus D_{\leq (\ell-1)}(\varphi)$ is smooth and every irreducible component has pure codimension $(e-\ell)(f-\ell)$ in $B\times X$.  Combined with generic smoothness we deduce (1).  

If $B$-flatness of $D_{\leq k}(\varphi)$ fails over a divisor $E_i$ in $B$, then the dimension of the inverse image of $E_i$ equals the full dimension of $D_{\leq k}(\varphi)$.  Thus, the irreducible components of the inverse image of $E_i$ give irreducible components of $D_{\leq k}(\varphi)$.  Since $D_{\leq k}(\varphi)$ is irreducible of pure codimension $(e-k)(f-k)$, there is only one such irreducible component.  By hypothesis, this component surjects onto $B$, contradicting non-flatness.  This implies (2).

Finally, (3) follows from Minoccheri's connectedness theorem.  Since $B$ is simply connected, the finite part of the Stein factorization of $D_{\leq k}(\varphi_B)$ over $B$ either maps isomorphically to $B$, i.e., all fibers of $D_{\leq k}(\varphi_B)$ are connected, or it is branched over a nonempty, irreducible divisor $E_i$ in $B$.  The inverse image of $E_i$ in the Stein factorization has an irreducible component of multiplicity $m>1$.  This then gives connected components of the fibers of $D_{\leq k}(\varphi_B)$ over $E_i$ such that every irreducible component has multiplicity divisible by $m$, contrary to hypothesis. <B>QED</B>

Note, this does explain the examples in the comments.  In each example, it is straightforward to check that the degeneracy loci do become every nonreduced over a divisor in $B$.