Okay, so I'm pretty sure I have a sort-of answer for this for $f=x+y+\sum_{i,j>0}a_{ij}x^iy^j$, though it's not a closed form at all.
With a bit of fiddling one can see that $$ f\circ f = \sum_{i,j>0}a_{ij}\left(x+y+\sum_{l,k>o}a_{lk}x^ly^k\right)^iz^j-\sum_{i,j>0}a_{ij}x^i\left(y+z+\sum_{l,k>o}a_{lk}y^lz^k\right)^j,$$ and also that all of the monomials of less than 3 variables cancel out.
Now, if one attempts to work this out degree by degree, one quickly sees that the number of coefficients gets enormous quite quickly. As far as I can tell, this is because they are controlled by certain kinds of partitions of integers. Let me make a definition:
Suppose we have a collection of $n$ indistinguishable red stones and $m$ indistinguishable blue stones. Then I'm interested in partitions of this collection which have a certain form. Specifically, the only monochrome blocks have precisely one stone, e.g. $r\vert r\vert b$ and $r\vert rb$ are valid partitions of $\{r,r,b\}$ but $rr\vert b$ is not. I'll call these "restricted two-colored partitions" for the purposes of this answer. I think there are things called colored partitions in combinatorics, and I don't know if they're connected to such ideas. It should be clear that a partition is unchanged by switching two stones of identical color, but changed into a new partition by switching two stones of different color. Let the set of all restricted partitions of of $n$ red stones and $m$ blue stones be denoted by $Q(n,m)$ (since $P(n,m)$ means something else!).
Now, in the above expression for $f\circ f$, let's attempt to determine the coefficient of a given monomial $x^\alpha y^\beta z^\gamma$. On the left hand side of the expression, we can fix $j=\gamma$ and on the right hand side we can fix $i=\alpha$. Let's start with $j=\gamma$. So we're interested in determining all the ways we can get $x^\alpha y^\beta$ out of the product $\left(x+y+\sum_{l,k>o}a_{lk}x^ly^k\right)^i$, and what the associated coefficients will be. I claim that these are determined by the set $Q(\alpha,\beta)$. That is, given a partition $P\in Q(\alpha,\beta)$, there is a map which takes a spot in the partition say, $\vert r^pb^q\vert$ to the monomial $a_{pq}x^py^q$, and multiplies all such spots together, then multiplies that entire monomial by the coefficient $a_{b_P,\gamma}$, where $b_P$ is the number of spots in the partition $P$. In other words, we've defined a map (which I claim is a bijection) $\phi:Q(\alpha,\beta)\to\{\mathrm{monomials~in~}x,y\mathrm{~of~degree~(\alpha,\beta)}\},$ where perhaps the target should be better explained, but I think it's clear. Thus if we sum all the coefficients that exist in the image of $\phi$, we'll get the total coefficient of $x^\alpha y^\beta z^\gamma$ coming from fixing the power of $z$, which I'll denote $A^{\alpha,\beta}_\gamma$. We must do an analogous process with fixed power of $x$, and get another coefficient $A^{\beta,\gamma}_{\alpha}$. So the coefficient of $x^\alpha y^\beta z^\gamma$ is $A^{\alpha,\beta}_\gamma-A^{\beta,\gamma}_{\alpha}$, where I've packed quite a bit of meaning into those little symbols.
Anyway, the reason we're having to use so-called "restricted colored partitions" is because our polynomials are restricted, in the sense that there are no monomials of the form $a_{n0}x^n$ in them, except when $n=1$. However, I suspect that this is really unnecessary, and that if we ignored this restriction, we could actually just get everything in terms of all possible colored partitions of $n$ blue stones and $m$ red stones, or whatever. The cool thing about that is that if you set all your coefficients to be $1$, I suspect you get a sort of generating function for these types of partitions, which happen to subsume non-colored partitions. So I imagine this sort of power series actually has some kind of combinatorial meaning, and I'd be surprised if it wasn't described somewhere in that discipline's literature, which I know nothing about.
Anyway, I'll leave this answer up, though it's pretty unsatisfying. Basically what it tells me is that this power series is really hard to deal with. I tried computing all the coefficients of the monomial $x^3y^3$ and it gets really big really fast. Luckily a huge amount of stuff cancels out, so when you just have a computer calculate these terms, they get (relatively...) simpler.
