Yes, we can reconstruct $X$ (as a $k$-scheme).
It would be kind of trivial if you had asked for the small Zariski topos instead of the small crystalline topos since the small Zariski topos $\mathrm{Sh}(X)$ is just the space $X$ viewed as a topos (generalized space). So the question is how to reconstruct the small Zariski topos (including the structure sheaf) from the small crystalline topos.
Claim. Let $X$ be a scheme over an arbitrary base scheme $S$. Then $\mathrm{Sh}(X)$ (or rather $X$ itself, as a locale) is the localic reflection of $(X / S)_{\mathrm{cris}}$.
Proof. The opens of the localic reflection are given by the subterminal objects of the topos in question, i.e. by the subsheaves $\mathcal{F}$ of the terminal sheaf on the small crystalline site of $X$ over $S$. Whenever $\mathcal{F}(U \hookrightarrow T)$ is inhabited for an $S$-PD-thickening $T$ of an open subscheme $U$ of $X$, then also $\mathcal{F}(U \rightarrow U) =: \mathcal{F}(U)$ is inhabited, since there is a morphism from $(U \rightarrow U)$ to $(U \hookrightarrow T)$. Also, for a cover $U_i$ of $U$, the sheaf condition for $\mathcal{F}$ says that $\mathcal{F}(U)$ is inhabited if the $\mathcal{F}(U_i)$ are. In summary, a subsheaf of the terminal sheaf is precisely given by an open of $X$. (There is an isomorphism of frames.) $\blacksquare$
Since $\mathcal{O}_{X/S}(U \rightarrow U) = \mathcal{O}_X(U)$ we also have the structure sheaf.
Note that we don't even need the structure morphism you specified, only the one to $\mathrm{Sh}(\mathrm{Spec}\:k)$.