Yes, given any $N \in \mathbb{N}$ there do exist $r$ and $s$ so that $A_{r} \cap A_{s}$ contains at least $N$ elements. In fact, I will show that one can take $s = 6r$. (The approach I suggest below is definitely not the most efficient, nor is it along the lines you were pursuing.)
We want to find values of $r$ so that the equations
$$
C : a^2 = r^2 - b^2, \quad a^2 = 36r^2 - c^2
$$
have at least $N$ integer solutions for $a$. This system of two quadratic equations is isomorphic to the elliptic curve
$$
E : y^2 = x^3 - x^2 - 420x - 3168,
$$
which has infinitely many rational solutions. A generator for the group of rational points is $P = (69,540)$, and this generator maps to the solution $145^2 = 144^2 + 17^2$, $(6 \cdot 145)^2 = 144^2 + 858^2$, showing that $A_{145}$ and $A_{870}$ have (at least) one element in common. The point $2P$ is $(14113/576,352495/13824)$ and this
shows that $A_{430106689}$ and $A_{2580640134}$ also have an element in common (namely $101518560$). Taking the least common multiple of $145$ and $430106689$ shows that for $r = 145 \cdot 430106689$ there are at least two elements of $A_{r} \cap A_{6r}$.
Using the points $P$, $2P$, $\ldots$, $NP$, finding their images on $C$, and taking the least common multiple of the corresponding values of $n$ will give a value of $r$ so that $A_{r} \cap A_{6r}$ will have size at least $2N$.