I will show that this is not the case for $\mathbb{F}_9$. The proof generalize to any $\mathbb{F}_{p^k}$, with $k >1$.

I'm starting from the observation that:
$\mathbb{F}_9 \simeq \mathbb{Z}[i]/(3) \simeq \mathbb{Z}[\sqrt{2}]/(3)$. where I'm using the isomorphism that identifies $i$ and $\sqrt{2}$ to identify $\mathbb{Z}[i]/(3)$ and $\mathbb{Z}[\sqrt{2}]/(3)$.

I'm now considering $A= \mathbb{Z}[i,\sqrt{2}]$. It has a surjective map $\phi:A \to \mathbb{F}_9$ induced by the two maps $\mathbb{Z}[i] \to \mathbb{F}_9$ and $\mathbb{Z}[\sqrt{2}] \to \mathbb{F}_9$ above.

I can localize $A$ at $\ker \phi$ to make it an object of your category.

So, in the category under consideration I have a diagram:

$$ \mathbb{Z}[i]_{(3)} \to A_{\ker \phi} \leftarrow \mathbb{Z}[\sqrt 2]_{(3)} $$

If there was an initial object $B$ in your category, it should factor though both $\mathbb{Z}[i]_{(3)} $ and $\mathbb{Z}[\sqrt 2]_{(3)} $ hence through their intersection, but the intersection is reduced to $\mathbb{Z}_{(3)}$ whose map to $\mathbb{F}_9$ is not surjective, so no object of your category can factor through it.