About literature related to the topic of this question:

In the answer to the question [f(f(x))=exp(x)-1 and other functions “just in the middle” between linear and exponential.][1] one can find an interesting link with many references related to the problem:

 - [Lars Kindermann: Iterative Roots and
   Fractional Iteration][2]

There is also Kindermanns PhD thesis about finding solutions to iterative functional equations using a neural network (in german only)

 - [Kindermann(2001): Neuronale Netze zur Berechnung
   Iterativer Wurzeln und Fraktionaler
   Iterationen][3]

which might be helpful.

  [1]: https://mathoverflow.net/questions/4347/ffxexpx-1-and-other-functions-just-in-the-middle-between-linear-and-expo/4423#4423
  [2]: http://reglos.de/lars/ffx.html
  [3]: http://archiv.tu-chemnitz.de/pub/2002/0154/index.htm