Yes.

Suppose $p$ were not continuous. Then we could find a sequence of signed measures $\mu_n$ with norms $\|\mu_n\| \le 2^{-n}$ but $|p(\mu_n)| \ge 1$. Let $|\mu_n|$ denote the total variation measure of $\mu_n$, which has the same norm as $\mu_n$. Set $\mu = \sum_n |\mu_n|$; this sum converges in the Banach space $C(X)^*$. Now each $\mu_n$ is absolutely continuous with respect to $\mu$ so there exists $f_n \in L^1(\mu)$ with $d\mu_n = f_n \,d\mu$, or in your notation, $\mu_n = \Phi_\mu(f_n)$. Moreover, we have $\|f_n\|_{L^1(\mu)} = \|\mu_n\|$, so $f_n \to 0$ in $L^1(\mu)$. Yet $|p(\mu_n)| = |p(\Phi_\mu(f_n))| \ge 1$, contradicting the continuity of $p \circ \Phi_\mu$ on $L^1(\mu)$.

Yes.

Suppose $p$ were not continuous. Then we could find a sequence of signed Radon measures $\mu_n$ with norms $\|\mu_n\| \le 2^{-n}$ but $|p(\mu_n)| \ge 1$. Let $|\mu_n|$ denote the total variation measure of $\mu_n$, which is still Radon and has the same norm as $\mu_n$. Set $\mu = \sum_n |\mu_n|$; this sum converges in the Banach space $C(K)^*$, so $\mu$ is a positive finite Radon measure. Now each $\mu_n$ is absolutely continuous with respect to $\mu$, so let $f_n \in L^1(\mu)$ be its Radon–Nikodym derivative; then, in your notation, $\mu_n = \Phi_\mu(f_n)$. Moreover, we have $\|f_n\|_{L^1(\mu)} = \|\mu_n\|$, so $f_n \to 0$ in $L^1(\mu)$. Yet $|p(\mu_n)| = |p(\Phi_\mu(f_n))| \ge 1$, contradicting the continuity of $p \circ \Phi_\mu$ on $L^1(\mu)$.

Looking at it another way, this construction shows that any countable set $\{\mu_k\} \subset C(K)^*$ is contained in $L^1(\mu)$ for some $\mu$, namely $\mu = \sum_k a_k |\mu_k|$ for suitable positive coefficients $a_k$. Thus $p$ is continuous when restricted to any countable set, and by considering sequences (since $C(K)^*$ is a metric space), this is sufficient for continuity. (Indeed, since the spaces $L^1(\mu)$ are complete, this actually shows that any separable subset of $C(K)^*$ is contained in some $L^1(\mu)$.)