Using the strategies suggested by @Merosity on MSE and @Gro-Tsen and @RP_ on MO, I have found a proof that the density is indeed always rational.
Let $P$ be a polynomial with integer coefficients, and let $p$ be a prime number. If $P$ is a constant polynomial, then we obtain $\delta=0$ which is rational. So assume that $P$ is nonconstant. We shall prove that $\delta$ is rational by means of a chain of lemmas. Each lemma only uses the previous lemma.
Lemma
Let $g:\mathbb Z_p\rightarrow\mathbb Z_p$ be a power series
$g(x)=\sum_{i=0}^{\infty}g_ix^i$ with $g_i\in\mathbb Z_p$ for all $i$ and
$g_i\rightarrow0$ as $i\rightarrow\infty$. Suppose that $g'(0)=1$. Then the
restriction of $g$ to $p\mathbb Z_p$ has image $g(0)+p\mathbb Z_p$.
Proof
The proof is analoguous to that of Hensel's lemma.$\tag*{$\blacksquare$}$
Now let $v\in\mathbb Z_p$ for which $P'(v)=0$, and let
$n_v\geq2$ be the largest integer such that $(x-v)^{n_v}$
divides $P(x)-P(v)$. Let $Q_v(x)\in\mathbb Z_p[x]$ be the polynomial such that $P(x)=P(v)+(x-v)^{n_v}Q_v(x-v)$. Now $Q_v(0)\neq0$.
Lemma
For all $v\in\mathbb Z_p$ such that $P'(v)=0$, there is an integer $N_v\in\mathbb N$ such that $$P(v+p^{N_v}\mathbb Z_p)=P(v)+p^{n_vN_v}Q_v(0)f_v(\mathbb Z_p),$$ where $f_v:\mathbb Z_p\rightarrow\mathbb Z_p$ is defined by $f_v(z):=z^{n_v}$.
Proof
Define the function $R:\mathbb Z_p\rightarrow\mathbb Z_p$ by $$R(x)=\sum_{i=0}^{\infty}r_ix^i:=\sum_{i=0}^{\infty}p^{2i}n_v^{i}\binom{\frac{1}{n_v}}{i}x^i.$$ For all $i$, we have $v_p(r_i)=v_p(p^{2i}n_v^{i}\binom{\frac{1}{n_v}}{i})\geq v_p(p^{2i})-v_p(i!)>i$, so all $r_i$ are $p$-adic numbers and $\lim_{i\rightarrow\infty}r_i=0$. Therefore, $R(x)$ is well-defined.\
It follows from the definition of $R$ that $R(x)^{n_v}=1+p^2n_vx$ for all $x\in\mathbb Z_p$. Now define the function $K:\mathbb Z_p\rightarrow \mathbb Z_p$ by $$K(x):=R\left(\frac{Q_v(p^2n_vQ_v(0)x)-Q_v(0)}{p^2n_vQ_v(0)}\right).$$ Then $K(x)=\sum_{i=0}^{\infty}k_ix^i$ for coefficients $k_i\in\mathbb Z_p$ with $\lim_{i\rightarrow\infty}k_i=0$.
It follows for all $x\in\mathbb Z_p$ that $$Q_v(0)K(x)^{n_v}=Q_v(0)\left(1+p^2n_v\frac{Q_v(p^2n_vQ_v(0)x)-Q_v(0)}{p^2n_vQ_v(0)}\right)=Q_v(p^2n_vQ_v(0)x).$$
Therefore, we see that \begin{align*}P(v+p^2n_vQ_v(0)x)&=P(v)+(p^2n_vQ_v(0)x)^{n_v}Q_v(p^2n_vQ_v(0)x)\\
&=P(v)+(p^2n_vQ_v(0)x)^{n_v}Q_v(0)K(x)^{n_v}\\
&=P(v)+Q_v(0)(p^2n_vQ_v(0)xK(x))^{n_v}\end{align*}
for all $x\in\mathbb Z_p$. Since $\frac{d(xK(x))}{dx}\big|_{x=0}=K(0)=R(0)=r_0=1$, we can use the previous lemma to see that the image of $xK(x)$ restricted to $p\mathbb Z_p$ is $p\mathbb Z_p$. Therefore, the image of $P$ restricted to $v+p^3n_vQ_v(0)\mathbb Z_p$ is $P(v)+Q_v(0)(p^3n_vQ_v(0))^{n_v}f_v(\mathbb Z_p)$. So the lemma holds for $N_v:=v_p(p^3n_vQ_v(0))$.
$\tag*{$\blacksquare$}$
Lemma For all $v\in\mathbb Z_p$ such that $P'(v)=0$, there is a finite set $S_v\subset\mathbb Z$ such that $$P(v+p^{N_v}\mathbb Z_p)=P(v)+p^{n_vN_v}Q_v(0)\left(\{0\}\cup\left(\bigcup_{s\in S_v}\bigcup_{i=0}^{\infty}p^{in_v}(s+p^{2v_p(n_v)+1}\mathbb Z_p)\right)\right).$$
Proof
Define $S_v:=\{a^{n_v}\mid 0<a<p^{v_p(n_v)+1},p\nmid a\}$. Then it follows from arguments similar to the proof of Hensel's lemma that for all $i\geq0$, the image of $f_v$ restricted to $p^i\mathbb Z_p\backslash p^{i+1}\mathbb Z_p$ is equal to $p^{in_v}\bigcup_{s\in S_v}(s+p^{2v_p(n_v)+1}\mathbb Z_p)$. Taking the union over all $i$, we see that the image of $f_v$ equals $\{0\}\cup\left(\bigcup_{s\in S_v}\bigcup_{i=0}^{\infty}p^{in_v}(s+p^{2v_p(n_v)+1}\mathbb Z_p)\right)$. Therefore, this lemma follows from the previous lemma.
$\tag*{$\blacksquare$}$
Lemma
For all $\sigma\in P(\mathbb Z_p)$, there is an integer $M_{\sigma}\geq0$ such that $$P(\mathbb Z_p)\cap(\sigma+p^{M_{\sigma}}\mathbb Z_p)$$ has rational $p$-adic density.
Proof
First, suppose that there is an $x\in\mathbb Z_p$ such that $P(x)=\sigma$ and $P'(x)\neq0$. Then it follows from arguments similar to Hensel's lemma that $P(\mathbb Z_p)$ contains a neighbourhood of $\sigma$. This immediately proves the lemma.\
Now suppose that for all $x\in\mathbb Z_p$ such that $P(x)=\sigma$, we have $P'(x)=0$. Let $V_{\sigma}:=P^{-1}(\sigma)$, then $V_{\sigma}$ is a finite set since $P$ is nonconstant. Since $P$ is continuous, we can choose $M_{\sigma}\in\mathbb Z_{\geq0}$ such that $P^{-1}(\sigma+p^{M_{\sigma}}\mathbb Z_p)$ is contained in $\bigcup_{v\in V_{\sigma}}(v+p^{N_v}\mathbb Z_p)$. Now it follows that $$P(\mathbb Z_p)\cap(\sigma+p^{M_{\sigma}}\mathbb Z_p)=\bigcup_{v\in V_{\sigma}}P(v+p^{N_v}\mathbb Z_p)\cap(\sigma+p^{M_{\sigma}}\mathbb Z_p).$$
Using the previous lemma, we see that this set equals
$$\bigcup_{v\in V_{\sigma}}\left(\sigma+p^{n_vN_v}Q_v(0)\left(\{0\}\cup\left(\bigcup_{s\in S_v}\bigcup_{i=0}^{\infty}p^{in_v}(s+p^{2v_p(n_v)+1}\mathbb Z_p)\right)\right)\right)\cap(\sigma+p^{M_{\sigma}}\mathbb Z_p).$$
Let $n:=\mathrm{lcm}_{v\in V_{\sigma}}(n_v)$. Then there exists an integer $C>0$ and a finite collection of numbers $a_l\in \mathbb Z_p\backslash p^{C}\mathbb Z_p$, $1\leq l\leq L$, such that our set can be written as
$$P(\mathbb Z_p)\cap (\sigma+p^{M_{\sigma}}\mathbb Z_p)=
\{\sigma\}\cup\bigcup_{l=1}^{L}\left(\sigma+\bigcup_{i=0}^{\infty}p^{in}(a_l+p^{C}\mathbb Z_p)\right).$$
The $p$-adic density of this set is $$\lvert\{a_l\bmod p^{C}\mid 1\leq l\leq L\}\rvert\cdot \frac{p^n}{p^n-1}\cdot p^{-C}$$ which is a rational number.
$\tag*{$\blacksquare$}$
Theorem
The $p$-adic density of $P(\mathbb Z_p)$ is rational.
Proof
When we define $B_{\sigma}:=\sigma+p^{M_{\sigma}}\mathbb Z_p$ for all $\sigma\in P(\mathbb Z_p)$, then $\{B_{\sigma}\mid \sigma\in P(\mathbb Z_p)\}$ is an open cover of $P(\mathbb Z_p)$. Since $\mathbb Z_p$ is a compact set and $P$ is continuous, the image $P(\mathbb Z_p)$ is also a compact set. Therefore, the open cover has a finite subcover $\{B_{\sigma_1},\dots,B_{\sigma_q}\}$ which is minimal. The sets in this subcover must be pairwise disjoint, so it follows that $P(\mathbb Z_p)$ is the disjoint union of the sets $B_{\sigma_i}\cap P(\mathbb Z_p)$ for $1\leq i\leq q$. Therefore, the $p$-adic density of $P(\mathbb Z_p)$ is the sum of the densities of these sets. By the last lemma, all those densities are rational. Therefore their sum is also rational.$\tag*{$\blacksquare$}$
