There seem to be two issues you're addressing. One is "what's the right notion of quotient" and the comments show that it should really be the fiberwise quotient (i.e. pushout of $B\leftarrow E'\rightarrow E$), which I'll denote $\mathcal{E}$. Under some hypotheses, $\mathcal{E}$ has the property that the fiber space over $b\in B$ is $F_b'/F_b$ where $F_b'$ is the fiber over $b$ of the $E'$ fibration and $F_b$ for the $E$ fibration. See e.g. link
The next question is "when is $\mathcal{E}$ a Serre fibration?" A similar question was asked herehere for simplicial sets. Turns out the pushout is a Serre fibration when one of the two maps is a cofibration (by the way, a pullback of two fibrations is always a fibration, in any model category). I suspect the same will hold in your setup of CW complexes, i.e. $\mathcal{E}$ should be a fibration because $E'\rightarrow E$ is a cofibration. A fact which might help is that cofibrations of CW complexes are really just retracts. The last link probably answers your question if you dig around inside. I'll edit this if I find an exact reference or work out the details of the proof.
