This result (and much more) is in Exposé IX *Modèles de Néron et monodromie* by A.Grothendieck (in SGA7), and more precisely in section 11.1. In particular what you want is exactly Proposition 11.2 thereof. 

The proof is not hard but it also far from being obvious (to me, anyway). Let me recall it briefly. A separate, quasi-finite scheme $X$ over $\mathbb Z_{p}$ decomposes as a disjoint union of a finite part $X^{f}$ which has same special fiber as $X$ with a scheme $X'$ with zero special fiber. The $\ell^{n}$-torsion of the component group $\Phi_0$ is the quotient of the $\ell^{n}$-torsion of $E$ by the $\ell^{n}$-torsion of the special fiber of $E$ at $p$, which is thus the $\ell^{n}$-torsion of the special fiber of $E^{f}$. But the finite part of the special fiber of $E$ is given by the invariant under inertia of the $\ell$-adic Tate module.