The usual proof of the inverse function theorem in the setting of Banach spaces uses the Banach fixed point theorem. We cannot make sense of the Banach fixed point theorem in a Fréchet space, since a Fréchet space is merely metrizable: there is no preferred choice of metric. I do not know exactly how crucial the Banach fixed point theorem actually is for proving the inverse function theorem in Banach spaces. Hopefully someone can come along and explain whether this is really the "hinge" that makes the inverse function theorem work.
Since you say IFT "apparently" fails for Fréchet spaces, I'm guessing you haven't seen a counterexample. Let $C^\infty[-1,1]$ denote the Fréchet space of smooth real-valued functions on $[-1,1]$. Consider the smooth map
$$P: C^\infty[-1,1] \longrightarrow C^\infty[-1,1],$$
$$f \mapsto f - xff'.$$
Now
$$DP(f)g = g - xgf' - xfg',$$
so $DP(0) = I$. Since $P(0) = 0$, if the inverse function theorem holds then $P$ should be invertible in some neighborhood of $f = 0$, with $P^{-1}(0) = 0$. To see that this is not true, consider the family of functions
$$g_n(x) = \frac{1}{n} + \frac{x^n}{n!}.$$
Now $g_n \longrightarrow 0$ in $C^\infty[-1,1]$, but we claim that $g_n$ is not in the image of $P$ for any $n$. To see this, write the power series expansion
$$f(x) = a_0 + a_1x + a_2x^2 + a_3 x^3 + \cdots.$$
Then we have
$$Pf(x) = a_0 + (1 - a_0)a_1 x + (a_2 - a_1^2 - 2a_0a_2)x^2 + (a_3 - 3a_1a_2 - 3a_0a_3)x^3 + \cdots.$$
Suppose $Pf(x) = g_n(x)$. Then $a_0 = \tfrac{1}{n}$, and an inductive argument shows that $a_1 = a_2 = \cdots = a_{n-1} = 0$, giving
$$Pf(x) = \frac{1}{n} + (1 - na_0)a_nx^n + \cdots.$$
The order $n$ term is then zero, contradicting the fact that $Pf(x) = g_n(x)$. Hence $g_n$ does not lie in the image of $P$ for any $n$. Since $P(0) = 0$, $DP(0) = I$ is invertible, and $g_n \longrightarrow 0$, this provides a counterexample to the inverse function theorem.
The generalization of the inverse function theorem to Fréchet spaces is the Nash-Moser theorem. You need some much better conditions on the map $f$ you want to invert in this case: First, $f$ must be invertible in a neighborhood, not just at a single point. Second, the Fréchet spaces involved and $f$ must satisfy the technical condition of "tameness." Finally, the inverses of $f$ in the neighborhood of interest must also be tame. There are counterexamples to show that each of these extra conditions is necessary.
For an overview of the Nash-Moser theorem, see Hamilton's The inverse function theorem of Nash and Moser.