If "strict fibre" just means the fibre over $*$, then youYou can make the construction as follows. There are twothree steps.
First make a Serre fibration over the unit interval, pi:T->I=[0,1] such that the fibre over 0 (resp. 1) is F (resp. F'). You will find how to make this in the general case; for example, for F=1 point and F'=the interval, here is a proof that the triangle does it.
Consider in the plane the triangle T: 0<=y<=x<=1 and the projection pi(x,y)=x. Given a CW-complex A, a continuous map from A to T:a->(x_0(a),y_0(a)) and a homotopy (x_t(a)) (0<=t<=1), consider K={(a,t)\in AxI/x_t(a)=0} and K_0=(Ax0)\cap K. On the complement (Ax0)\K_0 you have the slope function s_0(a):=y_0(a)/x_0(a); extend it to a continuous function s from (AxI)\K to I; the wanted homotopy is (a,t)->(x_t(a),s(a,t)x_t(a)) for (a,t)\notin K and (a,t)->(0,0) for (a,t)\in K. Hence, pi is a Serre fibration.
Second, let $f:X\rightarrow X$ be homotopic to the identity and contract some neighborhood $N$ of $*$ onto $*$. Pulling back your fibration $p$ through $f$, you are reduced to the case where $p$ is a projection $F\times N\rightarrow N$ over $N$. Then
Third, consider a function $g: N\rightarrow I$ whose value is $0$ on $\partial N$ and $1$ on $*$; consider the mapping cylinder $C $ of a homotopy equivalence $F\rightarrow F'$ and the projection $pr:C\rightarrow [0,1]$ with $pr^{-1}(1)=F'$; define define $E'$ over $N$ as the amalgamated product of $C$$T$ with $N$ over $G$$pi$ and $g$; define $E'$ as $E$ over $(X-N)$. If you have chosen $N$ to retract by deformation on $*$, then this $E'$ will have the property you want.
Maybe there is a more elegant way to present the construction (i'm not an expert in homotopy theory nor cw-complexes) Sorry, I come back to this answer which I realize is not quite correct, because $C$ is not a Serre fibration over $I=[0,1]$, in general. Instead of $C$, you should use a Serre fibration over $I$ whose fibres over $0$ and $1$ are $F$ and $F'$ respectively; there should be a canonical construction of this kind, but right now I cannot be more precise. Hope it helps.