Based on others and my comments, one can construct arbitrary solutions to the problem using a Taylor series ansatz. Use the functional equations from the comments
\begin{align}
\phi(x)&=f(x+\phi(x))\tag{1a}\\
\psi(x)&=F(x+\psi(x))\tag{1b}
\end{align}
fulfilling $\phi(x) = \psi'(x)$.
Insert the Taylor expansions
\begin{align}
\phi(x)&=\sum_{k=0}^n \frac{\phi^{(k)}(0)}{k!} \, x^k \tag{2a}\\
f(x)&=\sum_{k=0}^n \frac{f^{(k)}(\phi(0))}{k!} \, (x-\phi(0))^k\tag{2b}
\end{align}
into () to get 
\begin{align}
f^{(0)}(\phi(0)) &= \phi(0)\\
f^{(1)}(\phi(0)) &= \frac{\phi'(0)}{1+\phi'(0)}\\
f^{(2)}(\phi(0)) &= \frac{\phi''(0)}{(1+\phi'(0))^3} \\
f^{(3)}(\phi(0)) &= \frac{\phi'''(0) (1+\phi'(0))-3 \phi''(0)^2}{(1+\phi'(0))^5} \tag{3}\\
&\vdots
\end{align}
as well as the corresponding coefficients of $F$,
\begin{align}
F^{(0)}(0) &= 0 \\
F^{(1)}(0) &= \frac{\phi(0)}{1+\phi(0)} \\
F^{(2)}(0) &= \frac{\phi'(0)}{(1+\phi(0))^3} \\
F^{(3)}(0) &= \frac{\phi''(0) (1+\phi(0))-3 \phi'(0)^2}{(1+\phi(0))^5}. \tag{4}\\
&\vdots
\end{align}
For $\phi(0)=0$ we can now require that $F'(x)=a f(x)$ to get two solutions: $\phi(x)\equiv 0$ and
\begin{align}
\phi^{(1)}(0) &= a-1 \\
\phi^{(2)}(0) &= \frac{3a^2(a-1)}{a+1} \\
\phi^{(3)}(0) &= \frac{3 (a-1) a^3 (5 a^3+5 a^2+5 a-4)}{(a+1)^2 (a^2+a+1)} \\
\phi^{(4)}(0) &= \frac{15 (a-1) a^4 (7 a^6+7 a^5+14 a^4-5 a^3+11 a^2-20 a+4)}{(a+1)^3 (a^2+1) (a^2+a+1)} \\
\phi^{(5)}(0) &= \frac{45 (a-1) a^5 p_5(a)}{(a+1)^4 (a^2+1) (a^2+a+1)^2 (a^4+a^3+a^2+a+1)}\\
&\vdots
\end{align}
with $p_5(a)=(21 a^{13}+63 a^{12}+147 a^{11}+175 a^{10}$${}+231 a^9+104 a^8+75 a^7-109 a^6$${}-101 a^5-189 a^4-59 a^3-32 a^2+68 a-8)$, which reduces to $\phi(x)\equiv 0$ for $a\to 1$.

As the denominator looks familiar to me (from some $q$ series), this solution might be written down in closed form.

Some Mathematics code:
```
ordfs = 5;
\[Phi]s[x_] = Series[\[Phi][x], {x, 0, ordfs}](*/.\[Phi][0]->0*)
fs[x_] = Series[f[x], {x, \[Phi][0], ordfs}(*/.\[Phi][0]->0*)]
CoefficientList[Normal[\[Phi]s[x] - f[x + \[Phi]s[x]]], x]
sof = FullSimplify[Solve[% == 0, fs[x][[3]] /. Rational[_, _] :> 1]]
\[Psi]s[x_] = Integrate[\[Phi]s[x], x]
ordFs = Length[(\[Psi]s[x] - F[x + \[Psi]s[x]])[[3]]] - 1;
Fs[x_] = Series[F[x], {x, 0, ordFs}]
CoefficientList[Normal[\[Psi]s[x] - F[x+\[Psi]s[x]]], x]
soF = Simplify[Solve[% == 0, Table[Derivative[k][F][0], {k,0,ordFs}]]]

Simplify[Normal[\[Phi]s[x] - f[x + \[Phi]s[x]]] /. sof]
Simplify[Normal[\[Psi]s[x] - F[x + \[Psi]s[x]]] /. soF]
```