Functional equations based on composition

Question

I have asked this question here (*), but there are no answer.

Let $n \in \mathbb N^*$, $\{a_0,\ldots,a_n\} \subset \left] 0,+\infty\right]$. We suppose $Eq : \sum\limits_{k=0}^n a_k f^k(x)=0$ have no linear solution.

Determinate the solution of $Eq$ for $f \in C^\infty (\mathbb R)$, where $f^2(x)=f \circ f (x)$ and $f^0(x)=x$.

Reference: https://artofproblemsolving.com/community/c6h3164353_functional_equation

Answer 1

There are no solutions, even if we only assume that $f$ has a derivative at every point.

Note that if $f(x)>x$ for all $x$, then $f(f(x))>f(x)>x$ etc, thus $\sum a_k f^k(x)\geqslant (\sum a_k)x$ that is positive for $x>0$. Analogously, if $f(x)<x$ for all $x$, then $f(f(x))<f(x)<x$ etc, and $\sum a_k f^k(x)\leqslant (\sum a_k)x$ that is negative for $x<0$. Thus, by Intermediate Value Theorem, there exists $x_0$ such that $f(x_0)=x_0$. If $p=f'(x_0)$, taking the derivative of our equation at $x_0$ we get $\sum a_kp^k=0$, thus, $f(x)=px$ is a solution of Eq.

Answer 2

As it was pointed in the comments, the case of a linear function $px$ should be excluded. Let's investigate the given sum on a linear function $f(x)=px+q$ with $q\ne0$.

We have $$\sum_{k=0}^n a_k (px+q)^{\circ k} = g_n x + qh_n,$$ where $g_n$ and $h_n$ are polynomials in $p,a_0,\dots,a_n$. It can be shown by induction on $n$ that we also have the following identity: $$g_n + (1-p) h_n = \sum_{k=0}^n a_k.$$ It implies that both $h_n$ and $g_n$ cannot be zero since the right-hand side is strictly positive. In other words, $\sum_{k=0}^n a_k (px+q)^{\circ k} \ne 0$ for any $p$ and $q\ne0$.