# Higher order functional derivatives

Let $$E, F$$ be Banach spaces. A continuous bilinear functional $${\langle \cdot\,, \cdot \rangle }: E \times F \to \mathbb{R}$$ is called $$E$$-non-degenerate if $$\langle x,y\rangle = 0$$ for all $$y \in F$$ implies $$x=0$$ (Similarly for $$F$$-non-degenerate). Equivalently, the two maps of $$E$$ to $$F^{*}$$ and $$F$$ to $$E^{*}$$ defined by $$x \mapsto \langle x, \cdot \rangle$$ and $$y \mapsto \langle \cdot\,, y\rangle$$, respectivelly, are one-to-one. If they are isomorphisms (*), $$\langle \cdot\,, \cdot \rangle$$ is called $$E$$ or $$F$$-strongly non-degenerate. We say that $$E$$ and $$F$$ are in duality if there is a non-degenerate bilinear functional $$\langle \cdot\,, \cdot \rangle: E\times F \to \mathbb{R}$$, also called a pairing of $$E$$ with $$F$$. If the functional is strongly non-degenerate, we say the duality is strong.

Consider the following definition.

Definition: Let $$E$$ and $$F$$ be normed spaces and $$\langle \cdot, \cdot \rangle$$ a $$E$$-non-degenerate pairing. Let $$f: F \to \mathbb{R}$$ be Fréchet differentiable at the point $$\alpha \in F$$ (denote this derivative as $$Df(\alpha)$$). The functional derivative $$\delta f/\delta \alpha$$ of $$f$$ with respect to $$\alpha$$ is the unique element in $$E$$, if it exists, such that: $$\begin{eqnarray} Df(\alpha)(\gamma) = \left\langle \frac{\delta f}{\delta \alpha}, \gamma\right\rangle\quad\forall\gamma \in F. \tag{1}\label{1} \end{eqnarray}$$

Now, I'd like to know how to define higher order derivatives of functional derivatives. In other words, suppose the Fréchet derivative of $$f$$ at $$\alpha$$, $$Df(\alpha)$$ is Fréchet differentiable at $$\beta\in F$$, is it possible to define $$\dfrac{\delta^{2}f}{\delta \beta\delta\alpha}$$?

Now, I'd like to know how to define higher order derivatives of functional derivatives. In other words, suppose the Fréchet derivative of $$f$$ at $$\alpha$$, $$Df(\alpha)$$ is Fréchet differentiable at $$\beta\in F$$, is it possible to define $$\dfrac{\delta^{2}f}{\delta \beta\delta\alpha}?$$

Yes, it is possible to define higher order Fréchet derivatives directly as bilinear functionals in $$\mathscr{L}_2(F,\Bbb R)$$ (the space of bounded bilinear functionals from $$F$$ to $$\Bbb R$$). This is shown by Ambrosetti and Prodi (, pp. 23-29), for example, and while leaving to them the details, we can simply say that $$f:F\to\Bbb R$$ is twice Fréchet differentiable at $$\alpha\in F$$ iff

• $$F$$ is one time Fréchet differentiable and
• $$Df[\alpha+\beta](\gamma)-Df[\alpha](\gamma)= \mathfrak{B}[\alpha](\beta,\gamma) + o(\beta)$$ where $$o(\beta)/\Vert\beta\Vert_F\to 0$$ as $$\Vert\beta\Vert_F\to 0$$ and $$\mathfrak{B}\in\mathscr{L}\big(F,\mathscr{L}(F,\Bbb R)\big)\simeq\mathscr{L}_2(F,\Bbb R)$$ (again, the proof of this fact is found in  §1.3, p. 23, where it is precisely show that the isomorphism between these two spaces is actually an isometry).

This can bee seen also by using the definition of Fréchet derivative you give, but it seems to me that, when dealing with functional derivatives of order greater than one, it overshadows the intrinsic clarity and familiarity of the concept. Precisely, by using the standard definition of Fréchet differentiablity (as given for example in  §1.1, p. 9, definition 1.1) and \eqref{1} we get $$\begin{split} Df[\alpha+\varepsilon\beta](\gamma)-Df[\alpha](\gamma) &= \left\langle \frac{\delta f}{\delta \alpha}(\alpha+\beta),\gamma\right\rangle -\left\langle \frac{\delta f}{\delta \alpha} (\alpha),\gamma\right\rangle\\ & = \left\langle\frac{\delta f}{\delta \alpha}(\alpha+\beta)- \frac{\delta f}{\delta \alpha} (\alpha),\gamma\right\rangle\\ &=\left\langle\mathfrak{L}[\alpha](\beta)+ o(\beta),\gamma\right\rangle \end{split}$$ where $$\mathfrak{L}\in\mathscr{L}(F,E)$$ is a linear bounded operator.

Bibliography

 Ambrosetti, Antonio; Prodi, Giovanni, A primer of nonlinear analysis, Cambridge Studies in Advanced Mathematics, 34. Cambridge: Cambridge University Press, pp. viii+171 (1993), ISBN: 0-521-37390-5, MR1225101, ZBL0781.47046.

• I would even say: there is but one notion, the Fréchet differential; which allow a unified treatment of the theory. Then, in each individual space of the luxuriant family of Banach spaces we have special representation theorems for duals, k-linear forms, and linear operators, so that the only abstract notion takes plenty of different forms, and "functional derivative" is just one of these. Mar 21, 2020 at 22:32
• Thanks for the answer and comment guys! I didn't read it yet (gonna do it soon) but have you seen my latest question? It's an extension of this one and you might be able to contribute a lot! mathoverflow.net/questions/355279/… Mar 21, 2020 at 22:57
• @IamWill, you're welcome. I saw your latest question but I wanted to answer this one first because the definition you give here (which I learned from you that it is due to Abraham, Marsdeen and Ratiu) fooled me a little bit. Then I realized that you only asked how to define higher order Fréchet derivatives by using it: now I hope to have shown that this definition is not the most manageable one when having to deal with higher order differentiability of functionals, but nevertheless it gives the standard result. Mar 22, 2020 at 7:21
• @DanieleTampieri I just read you answer and it helps me a lot! Let me just clarify one thing: your $\mathcal{L}(\alpha)$ is a linear operator which is the first order derivative of $f$ at $\alpha$? Mar 22, 2020 at 14:11
• @IamWill, yes, it is easier than you think: It is simply a consequence of the definition you used and of the assumption of Fréchet differentiability of the Fréchet derivative of $F$. I'll add something on this to my answer above. Mar 22, 2020 at 15:02