I am just writing my comments as one answer. Without further hypotheses, there are counterexamples. Even without a specific example of $\mathcal{X}$, there are plenty of examples of a $K$-scheme $B_K$ and a family of smooth, projective, geometrically connected relative curves $\mathcal{C}_K\to B_K$ such that for every fppf $R$-scheme $B_R$ that has $B_K$ as its generic fiber, for every flat, proper relative curve $\mathcal{C}_R\to B_R$ that extends the $K$-family, the geometric fibers over $B_k$ are not integral.

Indeed, for every proper, flat $R$-scheme $B'_R$ whose geometric fibers are integral, for every proper, flat morphism $\mathcal{C}'_R\to B'_R$ whose fibers are at-worst-nodal, connected curves with ample dualizing sheaf, for every "modification" of the family over a proper, flat $R$-scheme $B_R$ whose geometric fibers are integral, $\mathcal{C}_R \to B_R$, the "stabilization" of the geometric generic fiber of $\mathcal{C}_k\to B_k$ equals the geometric generic fiber of $\mathcal{C}'_k\to B'_k$: this is part of the uniqueness in "stable reduction". So if the geometric generic fiber of $\mathcal{C}'_k\to B'_k$ is reducible, the same is true for the geometric generic fiber of $\mathcal{C}_k\to B_k$.

Thus, to get a positive answer, you need to add the hypothesis that for the family $\mathcal{C}_K\to B_K$ whose geometric fibers are assumed to be integral, there exists an extension $\mathcal{C}_R\to B_R$ over an fppf $R$-scheme $B_R$ whose geometric generic fiber over $B_k$ is integral (just as a proper, flat family of abstract curves, with no morphism to $\mathcal{X}_R$ specified).

Even with this hypothesis, there are still counterexamples, e.g., the example in my comment where $\mathcal{X}_K$ is a Hirzebruch surface $\mathbb{P}^1\times \mathbb{P}^1$, yet $\mathcal{X}_k$ is a different Hirzebruch surface, e.g., the minimal (crepant) resolution of a singular quadric cone in $\mathbb{P}^3$. The usual way to deal with this is to change the question slightly: does there exist an extension $\mathcal{C}_R \to B_R\times_{\text{Spec}(R)} \mathcal{X}_R$ and an irreducible component $\mathcal{C}_{k,i}$ of $\mathcal{C}_k$ satisfying all of your conditions. If you begin with a $K$-family of curves $\mathcal{C}_K\to B_K\times_{\text{Spec}(K)} \mathcal{X}_K$ that is constructed in a "sufficiently general" way, e.g., the family of all complete intersection curves of sufficiently very ample divisors in given linear equivalence classes on $\mathcal{X}_K$, then this new question has a positive answer if and only if there exists an irreducible component $\mathcal{X}_{k,i}$ of $\mathcal{X}_k$ that is geometrically irreducible.