[I've answered the wrong question -- I'm doing $\ell=p$ below and he's asking about $\ell\not=p$.]
(a) is not going to be true in general (and as Moret-Bailly says, (b) needs some more explanation).
Here's the "moral" reason (a) isn't true. Let $E$ be an elliptic curve over the rationals. If $E[p^n]$ is the generic fibre of a finite flat group scheme overunramified at $\mathbf{Z}_p$$\ell$ for all $n\geq1$, then $E$ will have good reduction. But if $E[p]$ just has good reductionis unramified then this is not enough -- this is the case $\ell=p$ of "level-lowering" a la Mazur/Ribet, which was crucial in the proof that STW -> FLT. So if you take, for example, a counterexample to FLT and let $E$ be the associated Frey curve, and if we are in "case 2" of FLT, then $E$$E[p]$ will have bad reductionbe unramified at all odd $p$$\ell\not=p$ but $E[p]$ will be flat anyway$E$ could still have bad reduction at many such -- this is why case 2 of FLT is harder than case 1 in the Wiles/TW approach$\ell$.
But one does not need a counterexample to FLT to build such a phenomenon. One just needs two elliptic curves $A$ and $B$ with $A$ having good reduction and $B$ having, say, multiplicative reduction at $p$$\ell$, such that $A$ and $B$ are congruent mod $p$. Then the $j$-invariant of $B$ will have valuation at $\ell$ a multiple of $p$, so the number of components will be a multiple of $p$, and this will give you your counterexample (if I've understood the question correctly).
For an explicit example, let $A$ be the first curve in the database, $X_0(11)$, and let because over $B$ be$\mathbf{Z}_\ell$ the curveidentity component of conductor 33. Then $B$ is congruent to $A$ mod 3 (if memory serves -- I mean the modular forms are congruent) so the 3-torsionspecial fibre of $A$ and $B$ will be isomorphic as representations of the absolute Galois group of $\mathbf{Q}$. The existence ofis missing some $A$ means that the 3$p$-torsion has good reduction. But $B$ will have $j$-invariant whose 3-adic valuation
My memory is a multiple of 3 (you can see this from a Tate curve argument and the fact that $B$ is congruent to $A$ -- this will be in Serre's paper on his 1987 conjecture or perhaps Ribet's or Ribet-Stein -- or you can just compute it) so the special fibre of the Neron model of $B$ at 3 will have $c$ components, with $c$ a multiple of 3. So if by $B^0$ you mean something which over $\mathbf{Z}_3$ looks like a group scheme whose special fibre at 3 is just the identity componentfull of the Neron model, then $B^0[3]$ is missing some 3-torsion in the special fibreexplicit examples of these phenomena.