I agree with your formula. You choose $j \in \{0,\ldots,m-\ell\}$ (it will be the cardinal of $A \cap Y$, and given such a $j$ you choose independently $j$ elements in $Y$ and $m-j$ elements in $X$.  
Here is an upper bound  
$$S_{\ell,m,n} \le \sum_{j=0}^{m} {n \choose m-j} {n \choose j} = {2n \choose m}.$$ 
One obtains the last equality by looking at the coefficient of $X^m$ in the product $(1+X)^n \times (1+X)^n$. My guess is that this bound is sharp when $m/2-\ell >> \sqrt{m}$.