I think this is all about Smith Normal Form. Write $XAY = {\rm diag}(d_{1},d_{2},\ldots, d_{n})$ where $X$ and $Y$ are unimodular and where the $d_{i}$ are integers such that $d_{i} | d_{i+1}$ for each $i$. There are various terminologies for the $d_{i}$ sometimes conflicting. Let me call them the determinantal divisors (sometimes they are called elementary divisors sometimes invariant factors). In any case, they are unique up to sign and they determine and are determined by the structure of the Abelian group $\mathbb{Z}^{n}/{\rm Im}A$ when $A$ acts by multiplication on $\mathbb{Z}^{n}$ identified with $n$ long integer column vectors.

Your hypotheses imply that $p \not | d_{i}$ for $1 \leq i \leq n-(r-1).$ On the other hand, it is "well-known" that for $1 \leq i \leq m$,
the product $d_{1}d_{2} \ldots d_{m}$ is the gcd of all the $m \times m$ minors of $A$. Hence your assumptions imply that there is indeed some $(n-r +1) \times (n-r+1)$ minor of $A$ which is not divisible by $p$.