Nearby cycles and extension by zero Let $f: X\to \text{Spec}(R)$ be a proper and smooth morphism, with $R$ a strictly henselian dvr. Call $s = \overline{s}$ the closed point and $\eta$ the geometric point of $\text{Spec}(R)$.
Call $i_s : X_{\overline{s}}\to X$ the closed immersion and $h : X_{\overline{\eta}}\to X$ the inclusion of the geometric generic fiber.
The proper base change theorem and universal local acyclicity of smooth morphisms give an isomorphism
$$(*)\ \ \ \ H^p(X_{\overline{s}},F)\xrightarrow{\simeq} H^p(X_{\overline{\eta}},F),$$
where $F$ is a locally constant constructible $\ell$-adic sheaf.
Suppose that $f$ is not proper anymore but affine of finite type and separated. We still assume that $f$ is smooth.

Do we still have an isomorphism between $\ell$-adic cohomologies with proper supports:
$$H^p_c(X_{\overline{s}},\mathbf{Q}_{\ell})\xrightarrow{\simeq} H^p_c(X_{\overline{\eta}},\mathbf{Q}_{\ell})\ ?$$

ATTEMPTED SOLUTION:
The main idea I have in mind is as follows.
Choose a proper $g : \overline{X}\to \text{Spec}(R)$ that compactifies $f$.
Although $g$ may not be smooth, the inclusion $j : X\to \overline{X}$ is quasi-compact by the assumptions, and $j_!\mathbf{Q}_{\ell}$ is constructible on $\overline{X}$. Proper base change still applies to $j_!\mathbf{Q}_{\ell}$.
Call $j_s$ the base change of $j$ along $s\to\text{Spec}(R)$, and same for $j_{\eta}$. 
Call $i_s$ and $h$ still the maps $\overline{X}_s\to \overline{X}$ and $\overline{X}_{\overline{\eta}}\to \overline{X}$ by abuse of notation.

I want to show that $i_s^*\Psi_g(j_!\mathbf{Q}_{\ell}) = 0$, for $\Psi_g$ the nearby cycles functor. If this is true, then the natural map
$$(j_s)_!\Psi_f(\mathbf{Q}_{\ell}) \to \Psi_g(j_!\mathbf{Q}_{\ell})$$
  is an isomorphism, since its cofiber is $(i_s)_*i_s^*\Psi_g(j_!\mathbf{Q}_{\ell})$ (shifted). This is because $j_s^*\Psi_g = \Psi_f$ (where I use that $j$, an open immersion, is smooth).
In other words the question has a positive answer if the nearby cycles functor commutes with extension by zero.

Suppose that this is true. By properness of $g$, we have
$$R\Gamma(\overline{X}_{\overline{s}}, \Psi_g(j_!\mathbf{Q}_{\ell})) = R\Gamma(\overline{X}_{\overline{\eta}}, j_!\mathbf{Q}_{\ell}).$$
By the claim we have $R\Gamma(\overline{X}_{\overline{s}}, \Psi_g(j_!\mathbf{Q}_{\ell})) = R\Gamma(\overline{X}_{\overline{s}}, (j_s)_!\Psi_f(\mathbf{Q}_{\ell}))$. By smoothness of $f$ we have $\Psi_f(\mathbf{Q}_{\ell}) = \mathbf{Q}_{\ell}[0]$.
Putting everything together we have an isomorphism
$$R\Gamma(\overline{X}_{\overline{s}},j_!\mathbf{Q}_{\ell}) = R\Gamma(\overline{X}_{\overline{\eta}},j_!\mathbf{Q}_{\ell})$$
which gives what we want.
So the question really is:

do we have $(j_s)_!\Psi_f(\mathbf{Q}_{\ell}) =\Psi_g(j_!\mathbf{Q}_{\ell})$?

For every closed geometric point $\overline{x}\to \overline{X}_{\overline{s}}$, we call $\overline{X}_{\overline{s}}(\overline{x})$ the strict henselianization of $\overline{X}_{\overline{s}}$ at $\overline{x}$ and $\overline{X}_{\overline{s}}(\overline{x})_{\overline{\eta}}$ its generic fiber. 
Call $t_x : \overline{X}_{\overline{s}}(\overline{x})\to \overline{X}_{\overline{s}}$ the obvious map.
If $\overline{x}\to \overline{X}_{\overline{s}}$ does not factor through $X_{\overline{s}}$, then $t_x^*j_!\mathbf{Q}_{\ell} = 0$. Right?
In particular the stalk $\Psi_g(j_!\mathbf{Q}_{\ell})_{\overline{x}}$ is zero and so $i_s^*\Psi_g(j_!\mathbf{Q}_{\ell}) = 0$ and we're done. 
Am I correct?
SUMMARY OF THE ANSWER BELOW: the answer is no, and below there is an example of a smooth $X$ of dimension $1$ for which $H^1_c(X_{\overline{s}},\mathbf{Q}_{\ell})\to H^1_c(X_{\overline{\eta}},\mathbf{Q}_{\ell})$ is not an isomorphism. 
This also gives an explicit example for which formation of the nearby cycles complex does not commute with extension by zero. My argument above breaks down when I say "$t_x^*j_!\mathbf{Q}_{\ell} = 0$".
However, if $X$ is of dimension $d$, smooth and with geometrically connected fibers, the specialization map is indeed an isomorphism in degree $0$ and $2d$ even though $X$ is not proper.
 A: The statement is false without proper assumption. 
Consider any "degeneration of a smooth elliptic curve to a nodal curve" and delete a singular point in a special fibre. This will give you a counterexample for the dimension reasons.
Details: Start with any proper morphism $f: \mathcal E' \to \operatorname{Spec}R$ such that its generic fibre is an elliptic curve (smooth geometrically connected curve of genus $1$ with a fixed section) and its special fibre is a nodal curve. For example, take $R=\overline{\mathbf F_p}[[t]]$ and consider a curve $\mathcal E'\subset \mathbf P^2_{R}$ given by the equation 
$$Y^2Z-X^3-X^2Z-t^3=0.
$$
This curve has exactly one singular point in a special fibre given by $p=[0:0:1] \subset \mathbf P^2_{R}(\overline{\mathbf F_p})$. Define 
$$
\mathcal E:=\mathcal E' \setminus {p}.
$$
By the construction $\mathcal E'$ is a smooth $R$-scheme of relative dimension one, but it is not proper. Let us compute cohomology with compact support of the geometric fibres of this curve.
Geometric generic fibre: We know $\mathcal E_{\overline{\eta}}$ is a connected proper smooth curve of genus 1, so 
$$
\mathrm{H}^1_{c}(\mathcal E_{\overline{\eta}}, \mathbf Q_l)=\mathrm{H}^1(\mathcal E_{\overline{\eta}}, \mathbf Q_l)=\mathbf Q_l^2.
$$
Geometric special fibre: Here we see that $\mathcal E_{\overline{s}}$ is a nodal cubic $Y^2Z-X^3-X^2Z=0$ minus one point $[0:0:1]$. It is a standard computation to show that this scheme is actually isomorphic to $\mathbf G_m$. We know that 
$$
\mathrm{H}^1(\mathbf G_m, \mathbf Q_l)=\mathbf Q_l.
$$
And finally using Poincare duality between usual cohomology and cohomology with compact support (using that $\mathbf G_m$ is smooth) we conclude that 
$$
\mathrm{H}^1_{c}(\mathcal E_{\overline{s}}, \mathbf Q_l) = \mathrm{H}^1_{c}(\mathbf G_m, \mathbf Q_l)=\mathrm{H}^1(\mathbf G_m, \mathbf Q_l)^{\vee}=\mathbf Q_l.
$$
So, just for dimension reasons we can't have an isomorphism 
$$
\mathrm{H}^1_{c}(\mathcal E_{\overline{s}}, \mathbf Q_l) \to \mathrm{H}^1_{c}(\mathcal E_{\overline{\eta}}, \mathbf Q_l).
$$
Remark 1: I assume everywhere that $\ell$ is coprime with $p$.
Remark 2: This example may be a little bit misleading. Actually the failure of a specialization map to be an isomorphism has nothing to do with singularities of the "compactified family". You can produce a lot of examples by deleting points from a special fibre of smooth proper families of curves as well. But I think that this example is easier to visualize.
