Etale cohomology of localizations of henselian rings Let $R$ be a ring (say noetherian of finite Krull dimension, possibly with additional hypotheses) henselian along the ideal $(p)$, and let $\hat{R}$  be the $p$-adic completion. Is it true that the étale cohomology of $R[1/p]$ and $\hat{R}[1/p]$ with mod $p$ coefficients coincide? I believe that this should be true (for $K$-theoretic reasons), but I was wondering if there is a direct argument. 
 A: TL;DR: Your expectation is right. In fact, there is a third object to compare with $R[1/p]$ and $\hat R[1/p]$, the affinoid rigid space ${\rm Spf}(\hat R)^{\rm rig}$. The cohomology comparison is given by the Gabber-Fujiwara theorem: see Corollary 6.6.4 in [1].
Let us consider the following general setup: let $(A, I)$ be a henselian couple with $A$ noetherian (i.e. $A$ is henselian along $I$), and let $\hat A$ be the $I$-adic completion. We set $$X={\rm Spec}\, A, \quad \hat X = {\rm Spec}\, \hat A,$$
$$U= X - V(I), \quad \hat U = \hat X - V(\hat I) \quad \text{ where }\hat I = I\cdot \hat A.$$
Let $\varepsilon\colon \hat U\rightarrow U$ be the canonical map.
The following is a classical theorem of Elkik.

Theorem 1 (Cor. p. 579 in [2]). The restriction functor
  $$ 
 \varepsilon^*\colon(\text{finite etale covers of }U)\to (\text{finite etale covers of }\hat U)
$$ 
  is an equivalence of categories.

Equivalently, for any finite group $G$, the restriction map $H^1(U, G)\to H^1(\hat U, G)$ is bijective, or $\pi_1(\hat U)\to \pi_1(U)$ is an isomorphism. 
The Gabber-Fujiwara theorem (see [1], Cor. 6.6.3) is a far-reaching generalization of this, treating the higher cohomology of arbitrary torsion etale sheaves. Actually, the theorem compares the cohomology of $U$ to the cohomology of the rigid space $U^{\rm rig} = {\rm Spf}(\hat A)^{\rm rig}$. Whatever this object is, it is clear from the notation that $U^{\rm rig}$ depends only on $(\hat A, \hat I)$, and in particular we get the following as a corollary (see [1], Cor.6.6.4):

Theorem 2 (The formal base change theorem). Let $\mathscr{F}$ be an etale sheaf of sets (resp. torsion groups, resp. torsion abelian groups) on $U$. Then the pull-back map
  $$ \varepsilon^*\colon H^q(U, \mathscr{F})\longrightarrow H^q(\hat U, \varepsilon^* \mathscr{F}) $$
  is an isomorphism for $q=0$ (resp. for $q\leq 1$, resp. for $q\geq 0$).

In particular, at least in good cases (finite cohomological dimension, geometrically unibranch) the Artin-Mazur etale homotopy types of $U$ and $\hat U$ are equivalent. (Disclaimer: as written, [1] treats only abelian cohomology, but I'm sure the nonabelian statements can be easily deduced from Elkik).
Addendum. A forthcoming book by Abbes (a sequel to [3]) is expected to contain a new proof of the Gabber-Fujiwara theorem, based on Gabber's affine analog of the proper base change theorem.
Curiosity. In fact in the $p$-adic situation you describe, both cohomology groups agree with the cohomology of the fundamental group: Theorem 6.7 in here states that for every noetherian $\mathbf{Z}_{(p)}$-algebra $A$ such that $(A, pA)$ is a henselian pair, the scheme $X={\rm Spec}\, A[1/p]$ is a $K(\pi, 1)$. For $\mathbf{F}_p$-coefficients, this can be deduced (via Gabber-Fujiwara) from earlier work of Scholze (Theorem 4.9 in [4]).
[1] Fujiwara, K.: Theory of tubular neighborhood in étale topology. Duke Math. J. 80 (1), 15–57 (1995).
[2] Renee Elkik, Solutions d’equations a coefficients dans un anneau henselien, Ann. Sci. Ecole Norm. Sup. (4) 6 (1973), 553–603 (1974). MR 0345966 (49 #10692)
[3] Abbes, A.: Éléments de géométrie rigide. Volume I. Progress in Mathematics, vol. 286, Birkhäuser/Springer Basel AG, Basel (2010).
[4] Scholze, P.: $p$-adic Hodge theory for rigid analytic varieties. Forum Math. Pi 1, e177 (2013)
