I believe that you can form a counterexample in the same manner "as usual". Begin with a projective morphism $\pi:\mathcal{C}_R\to \text{Spec}(R)$ whose closed fiber $\mathcal{C}_k$ over the residue field $k$ is a smooth, geometrically connected curve of genus $g>0$. For simplicity, assume that $\text{Pic}(\mathcal{C}_R)$ is $\mathbb{Z}$ and assume that $\mathcal{C}_k$ has a $k$-point that is linearly independent from the image of $\text{Pic}(\mathcal{C}_R)$ in $\text{Pic}(\mathcal{C}_k)$.
For instance, let $d\geq 3$ be an integer, let $\mathbb{P}^N = \mathbb{P}H^0(\mathbb{P}^2,\mathcal{O}(d))$ be the projective space of all degree $d$ plane curves, let $D_q\subset \mathbb{P}^N$ be the hyperplane parameterizing degree $d$ plane curves that contain a specified point $q\in \mathbb{P}^2$, let $\mathcal{C} \to \mathbb{P}^N$ be the universal curve, and let $R$ be the stalk $\mathcal{O}_{\mathbb{P}^N,\eta_H}$ at the generic point $\eta_H$ of $H_q$. Let $\mathcal{C}_R$ be the base chance of $\mathcal{C}$ to $\text{Spec}(R)$.
For the fraction field $K$ of $R$, $\mathcal{C}_K$ has Picard group generated by the restriction of $\mathcal{O}_{\mathbb{P}^2}(1)$, by a straightforward incidence correspondence argument. Yet, by construction, $\mathcal{C}_k$ contains the $k$-point $q$. Moreover, the divisor class of $\underline{q}$ is linearly independent from the restriction of $c_1(\mathcal{O}_{\mathbb{P}^2}(1))$ in $\text{Pic}(\mathcal{C}_k)$.
Anyway, now let
$$
\nu:\mathcal{C}'_R \to \mathcal{C}_R
$$
be the blowing up at the closed point $q$ (with its reduced structure).
The strict transform $\widetilde{\mathcal{C}}_k$ has normal sheaf $\mathcal{O}_{\mathcal{C}_k}(-\underline{q})$. By Grauert / Artin, there exists a contraction
$$
\mu:\mathcal{C}'_R \to \mathcal{C}''_R
$$
of $\widetilde{\mathcal{C}}_k$ in $\mathcal{C}'_R$. I claim that the projection morphism to $\text{Spec}(R)$ factors through $\mu$,
$$
\pi'':\mathcal{C}''_R \to \text{Spec}(R).
$$
If $\mathcal{C}''_R$ were projective, the pullback to $\mathcal{C}'_R$ of an ample invertible sheaf would restrict on $\mathcal{C}_K$ to an integer multiple of the generator of $\text{Pic}(\mathcal{C}_K)$. Yet on $\widetilde{\mathcal{C}}_k$ it restricts to the structure sheaf. Since the exceptional divisor on $\mathcal{C}'_R$ restricts on $\widetilde{\mathcal{C}}_k$ to $\mathcal{O}_{\mathcal{C}_k}(-\underline{q})$, ultimately this implies a linear relation between $\text{Pic}(\mathcal{C}_R)$ and the divisor class of $\underline{q}$. This contradiction implies that $\mathcal{C}''_R$ is not projective.
Edit. As Laurent Moret-Bailly points out, the surface $\mathcal{C}''$ produced above is usually an algebraic space, not a scheme.