I sort of worked this out as I wrote it, so the answer is at the bottom. Consider it a whodunnit of sorts.
Let us do some explicit computations. Suppose $p(x) = \sum_{n \geq 0} p_n x^n$ and $r(x) = \sum_{n \geq 0} r_n x^n$, with $r_0 = 0$. Now, we just grind:
$p(x) A(r(x)) = (\sum_k p_k x^k) \sum _n a_n (\sum_l r_l x^l)^n = \sum_n a_n (\sum_k p_k x^k) (\sum_l r_l x^l)^n$
$= \sum_n a_n \sum_m (\sum_{k + l_1, \dots, l_n = m} p_k r_{l_1} \dots r_{l_n}) x^m = \sum_m (\sum_n f(m,n) a_n) x^m$,
where I have written $f(m,n) = \sum_{k + l_1, \dots, l_n = m} p_k r_{l_1} \dots r_{l_n}$. I require only one fact: $f(m,n) = 0$ if $n > m$, since $r_0 = 0$. Thus, if we write $F = [f(m,n)]$ as an infinite upper-triangular matrix and $\bar{a} = [a_n]$ as an infinite column vector, we have $p(x) A(r(x)) = \sum_m (F \bar{a})_m x^m$. You seek the condition:
If $F \bar{a} = \bar{b}$ (equivalently, $\bar{a} = F^{-1} \bar{b}$), then the entries of $\bar{a}$ are nonnegative if and only if those of $\bar{b}$ are.
Choosing $\bar{a}$ and $\bar{b}$ to be the infinite "basis" vectors, we see that this means that the entries of $F$ and $F^{-1}$ are necessarily all nonnegative. It is easy (for me :) ) to prove by induction that $F$ is therefore diagonal (with positive entries there).
How can this happen? We must have
$\sum_{k + l_1 + \dots + l_n = m} p_k r_{l_1} \dots r_{l_n} = 0$ if $m \neq n$, and $> 0$ if $n = m$.
For $m = n$, we get $p_0 r_1^n > 0$ for all $n$, so $p_0, r_1 > 0$.
For $m = n + 1$, we get $p_1 r_1^n + p_0 r_1^{n - 1} r_2 = 0$, and therefore $r_2 = -(p_1/p_0) r_1$.
For $m = n + 2$, we get $p_2 r_1^n + p_1 r_1^{n - 1} r_2 + p_0(r_1^{n - 1} r_3 + r_1^{n - 2} r_2^2) = 0$, and we observe the occurrence of $(p_1 r_1 + p_0 r_2)r_1^{n - 2} r_2 = f(n + 1, n)r_1^{-1} r_2 = 0$, which we eliminate to obtain $p_2 r_1^n + p_0 r_1^{n - 1} r_3 = 0$. Solving, we get $r_3 = -(p_2/p_0) r_1$.
More generally, we (or I, at any rate) have
$f(n + k, n) = p_k r_1^n + \sum_{i = 1}^{k - 1} f(n + i, n) r_1^{-1} r_{i + 1} + p_0 r_1^{n - 1} r_{k + 1}$,
so by induction we have $r_{k + 1} = -(p_k/p_0) r_1$. It follows that:
The pairs of functions $p(x), r(x)$ answering your question are exactly those of the form $p(x) = p_0(1 + xq(x))$, $r(x) = r_1(x - x^2q(x))$ with $p_0, r_1 > 0$, and $q(x) \in \mathbb{Q}[[x]]$.