Note that there is some small ambiguity here, as to talk about the reduction of $X/\mathbb{Q}$ modulo a prime $p$, one needs to choose a model $X$. i.e., a scheme $\mathcal{X} \to \mathbb{Z}$ whose generic fibre is isomorphic to $X$.

Anyway, if $X/\mathbb{Q}$ is geometrically integral, then the same holds over $\mathbb{F}_p$ for all but finitely many primes $p$. This follows from fact that being geometrically integral is a constructible property (see Section 9 of EGAIV). One applies this to the morphism $\mathcal{X} \to \mathbb{Z}$.

Precise references:

Definition of a constructible property: EGAIV Définition (9.2.1)

Proof that being geometrically integral is a constructible property: EGAIV Théorème (9.7.7).

You can also find a nice treatment of this in Chapter 10 of the book

Ulrich Görtz, Torsten Wedhorn - Algebraic Geometry: Part I: Schemes. With Examples and Exercises
 By