The other answers and comments provided examples where all but one of the minors vanish, so you see that you need to check all of them. Let me put this in different context. Define the $m \times n$ generic matrix $X$ over a field $k$ (or a commutative ring) to have its entries $x_{i,j}$, where the $x_{i,j}$ are variables in a polynomial ring $k[x_{i,j}]$. The determinantal variety of this matrix is the ideal generated by the $r \times r$ minors ($r$ being the size you're interested in).
Your question amounts to asking if these $\binom{m}{r} \binom{n}{r}$ generators are linearly dependentindependent (over $k$) and if they generate a radical ideal. Let me explain why they cannot beat least frame the linear independence issue using some representation theory. It's definitely overkill, but I think it's an instructive way to think about trying to find equations when you have a large amount of symmetry. There is an action of $G = {\bf GL}_n(k) \times {\bf GL}_m(k)$ on the generic matrix via $(A,B) \cdot X = AXB^{-1}$ which preserves rank. Hence the set of conditions for this matrix to have rank $< r$ must be preserved by $G$. We can rewrite $k[x\_{i,j}]$ as $k[V^* \otimes W] \cong {\rm Sym}(V \otimes W^\*)$ where $V$ and $W$ are $k$-vector spaces of ranks $m$ and $n$, respectively. The $r \times r$ minors are polynomials of degree $r$, hence sit inside of ${\rm Sym}^r(V \otimes W^\*)$. In fact, they must span a $G$-submodule. Furthermore, this $G$-submodule is $\bigwedge^r V \otimes \bigwedge^r W^\*$ (which can be seen in various ways), which we know has dimension $\binom{m}{r} \binom{n}{r}$. Hence they are linearly independent (as polynomials over $k$). This explains why you will always find examples where all but 1 of the minors can vanish (when we specialize the $x_{i,j}$ to specific values in $k$).
Another question you can askAs to why the ideal is whetherradical, one can see this using the theory of Gröbner bases. (I'll freely make use of language from that theory.) Specifically, if one uses an antidiagonal term order (an ordering of the variables for which the antidiagonal term is the leading term for each $r \times r$ minor; take for example $x\_{1,1} > x\_{1,2} > \cdots > x\_{1,n} > x\_{2,1} > x\_{2,2} > \cdots > x\_{2,n}> \cdots > x\_{n,n}$), then the $r \times r$ minors form a Gröbner basis for the ideal they generate. See Theorem 16.28 of Miller and Sturmfels, Combinatorial Commutative Algebra, or also Corollary 4.10 of Sturmfels and Sullivant, "Combinatorial secant varieties" for a different proof. So the initial ideal is radical. The answer because it is yesgenerated by squarefree monomials, butand this requires some more workimplies that the original ideal is radical. So there are no shortcuts! Well
But practically speaking, for determining the rank of a matrix, you canshould do Gauss-Jordan elimination. That would be faster.