The affirmative answer involves just going back to how the comparison isomorphism between cohomologies of different geometric fibers are defined by means of specialization maps for higher direct image sheaves on the base space, making sure we are using the "same" strict henselizations to make the specialization maps on the two sides (for $p$ and $p'$), and noting that such specialization maps for a fixed strict henselization are functorial in the sheaf on the base space.
To be precise, let $p:P \rightarrow S$ and $p':P' \rightarrow S$ be scheme morphisms and $f:P' \rightarrow P$ an $S$-morphism. Choose an etale abelian sheaf $F$ on $P$. For its pullback $F' = f^{\ast}F$ on $P'$ there are naturally induced "$f$-pullback" maps $\theta^i_F: R^i p_{\ast}(F) \rightarrow R^i p'_{\ast}(F')$ (in the style of "base change" morphisms, applied to the typically non-cartesian but commutative square with bottom side the identity map of $S$, top side $f$, and left and right sides $p'$ and $p$ respectively). Having set up this notation, put it to the side for a moment.
For any etale abelian sheaf $G$ on $S$ and points $s, \eta \in S$ with $s$ a specialization of $\eta$, upon choosing a geometric point $\overline{s}$ over $s$ and a geometric point $\overline{\eta}$ over $\eta$ on Spec($O_{S,\overline{s}}^{\rm{sh}}$)` we have a natural map ${\rm{sp}}_{\overline{s},\overline{\eta},G}:G_{\overline{s}} \rightarrow G_{\overline{\eta}}$. In the above setting, if $p$ is a smooth proper surjection and $F$ is lcc with torsion orders invertible on $P$ then $R^i p_{\ast}F$ is an lcc sheaf (smooth and proper base change theorems) and this specialization map for $G = R^i p_{\ast}(F)$ on $S$ is an isomorphism. That isomorphism is by definition the comparison isomorphism between cohomologies of $F$ on the geometric fibers at $\overline{s}$ and $\overline{\eta}$ (isomorphism depending on the "choice" of strict henselization).
So ultimately the affirmative answer to your question (implicitly assuming we are using a single such choice of strict henselization underlying the fibral comparison isomorphisms for both $p$ and $p'$!) amounts to the observations that
(i) the formation of the specialization map
${\rm{sp}}_{\overline{s},\overline{\eta},G}$ for general $G$ is functorial in $G$ with respect to any morphism between abelian etale sheaves on $S$ (such as the maps $\theta^i_F$), (ii) in the special case that $S$ is a geometric point, $\theta^i_F$ is the natural pullback map in degree-$i$ cohomologies associated to $F$ and $F'$.
As this argument makes plain, we have to use the same strict henselization on both sides or else it all breaks down. In terms of ACL's comment above in the topological analogue over the complex numbers (using constant coefficients), this corresponds to identifying the cohomologies on fibers over the various points of the contractible $U$ via their pullback identifications with the cohomologies on the total spaces $P_U$ and $P'_U$ over $U$ (i.e., the choice of strict henselization in the algebraic theory plays the role of $U$ in the topological case). That is, if we use a different contractible open $V$ in $S$ containing the same pair of points $s_1, s_2 \in S$ then we generally get a different isomorphism between cohomologies on the $s_i$-fibers (for both $P$ and $P'$), and we certainly wouldn't claim any compatibility for such comparison isomorphisms if we use $U$ for the definition of fibral comparison for $P$ and use $V$ for the definition of fibral comparison for $P'$.