The point is that the Frobenius at $\ell$ is nontrivial in this extension, so it generates an open subgroup, and the fixed field of this subgroup is precisely the field at which $\ell$ stops splitting.
To see that Frob$_\ell$ is nontrivial, recall that in the full $\mathbb{Z}_p^*$ extension $\mathbb{Q}(\mu_{p^\infty})$, Frob$_\ell$ is simply the element $\ell\in \mathbb{Z}_p^*$, which is clearly nontrivial. The cyclotomic $\mathbb{Z}_p$ extension is obtained by quotienting $\mathbb{Z}_p^*$ by the subgroup $\mu_{p-1}$. This subgroup does not contain $\ell$, so Frob$_\ell$ remains nontrivial in the $\mathbb{Z}_p$-extension.